The CMO’s agency evaluation checklist: 24 questions, and the answers that should worry you

How to use this checklist

This is the question set we would want asked of us — and the set we believe filters the Indian agency market fastest. Use it after the shortlist stage (build the shortlist with our selection guide), in a working session with the people who would actually run your account, not the pitch team. Ask in order, write the answers down, and score each theme green, amber or red rather than totalling points — a single red in measurement outweighs five greens in chemistry. Weight the attribution and incrementality sections double: in six years and 160+ brands we have found agencies honest about measurement are almost always honest about everything else, and the reverse.

Three mechanics make the session work. Send nothing in advance — rehearsed answers measure the agency’s preparation, unrehearsed ones measure its defaults. Put your own operator in the room, because evasions are easier to hear than to read in notes. And watch for the meta-signal throughout: the agencies worth hiring tend to enjoy this conversation, because specific questions are where good operators finally get to show their work.

Theme one: attribution and measurement (questions 1–5)

• 1. Which number do you optimise, and which should my budget answer to? Green: different numbers — platform metrics to steer, MER or contribution to budget. Red: platform ROAS is treated as both steering wheel and speedometer.
• 2. How will you handle COD and returns in reported revenue? Green: delivered-order reconciliation is standard and they name the lag it introduces. Red: dashboard revenue is the report; returns are your problem.
• 3. What attribution windows will you use, and what changes when you change them? Green: a specific default, a reason, and an offer to show numbers under two windows. Red: whatever the platform defaults to, plus visible discomfort.
• 4. How do you report branded search and retargeting? Green: broken out separately, because both over-claim credit. Red: blended into headline ROAS — the two cheapest ways to make a report beautiful.
• 5. What would make you tell us a channel is not working? Green: a threshold and a test, stated in advance. Red: more time is always the answer — no result would ever trigger the admission.

You are not hiring the agency that gave the best answers. You are hiring the agency that was least uncomfortable being asked — discomfort under specific questions is the cheapest preview of the relationship you will ever get.

Theme two: creative process (questions 6–10)

• 6. How many distinct concepts — not resizes — will ship monthly, and is that contracted? Green: a number, sized to your spend and fatigue rate, treated as a quota. Red: as needed, or a number that counts aspect ratios as creative.
• 7. Who writes the creative strategy, and how close are they to the media data? Green: strategist and buyer sit in one loop; losing hooks change next month’s briefs. Red: creative comes from a separate studio briefed quarterly.
• 8. Show me a creative learning log from a live account (redacted is fine). Green: it exists — hypotheses, results, banked learnings. Red: the learnings live in someone’s head, which means they leave in someone’s head.
• 9. What is your testing structure — and what kills an ad? Green: a written framework with kill thresholds, like the one in our testing guide. Red: winners are declared on feel, after three days, on spend too small to read.
• 10. What do you need from us to make creative that sells — and what happens if we are slow? Green: a concrete asset-and-access list plus a stated fallback. Red: no demands at all — an agency asking nothing of the brand is planning to recycle stock formats.

Theme three: incrementality and testing (questions 11–14)

• 11. When did you last run a test that made your own attributed numbers smaller? The single most diagnostic question on this list. Green: a specific story — a branded-search holdout, a retargeting holdout — plus what changed after. Red: silence, or the claim that such tests are academic.
• 12. What is your incrementality plan for our account, and when does it start? Green: a standing calendar — one meaningful test a quarter with pre-committed decisions. Red: we can do that if you want — testing exists only on request.
• 13. How will you size retargeting, and how will we know it is not just harvesting? Green: capped by tested lift, not by its flattering dashboard ROAS. Red: retargeting is the hero of the case studies they just showed you.
• 14. How do you measure what our D2C spend does to Amazon and quick commerce? Green: they raise the halo effect before you do and propose a read — the logic in our halo guide. Red: channels are reported as sealed boxes, so cross-channel lift silently credits nobody.

Theme four: reporting cadence and honesty (questions 15–19)

• 15. What is the reporting rhythm, and who reads it before we do? Green: weekly numbers with commentary, monthly business review, senior eyes on both. Red: a monthly deck assembled the morning of the call.
• 16. Show me a report where you told a client things were going badly. Green: produced without flinching — the diagnosis and the fix visible. Red: there has apparently never been a bad month in the agency’s history.
• 17. What does your first 30 days look like — before any optimisation? Green: audit, tracking verification, delivered-revenue reconciliation, baseline agreement — gates like the ones in our honest-timelines guide. Red: straight to launching campaigns on day two, on top of unverified tracking.
• 18. Which metrics will you never report, and why? Green: a real answer — vanity metrics named and excluded on principle. Red: everything is reportable, nothing is excluded, the dashboard has forty tiles.
• 19. If we call your two most recently departed clients, what will they say? Green: names offered, or an honest account of why an engagement ended. Red: references only from the three logos on the deck, all four years old.

Theme five: team structure and commercials (questions 20–24)

• 20. Who exactly works on our account weekly, and for how many hours? Green: named people, stated seniority, honest hours — juniors included and priced in. Red: the pitch team vanishes and a team will be assigned.
• 21. How many accounts does our lead buyer run in parallel? Green: a single-digit answer they will stand behind. Red: refusal to answer — which is an answer.
• 22. Who owns the ad accounts, pixels, audiences, data and creative files? Green: you do, unambiguously, from day one, in writing. Red: any variant of it is easier if it runs through our accounts. This one is disqualifying on its own.
• 23. How does the commercial model change your incentives? Green: they can argue honestly about retainer versus percentage-of-spend trade-offs — percentage models reward spend growth, flat fees reward inertia — and say how they mitigate their own. Red: our incentives are simply aligned, unexamined.
• 24. What would make you resign this account? Green: a real answer — unworkable unit economics, decisions overridden into failure, mutual mis-fit — showing they have thought about mutual accountability. Red: nothing; every account is a good account, forever.

Scoring it, and what to do next

After the session: score the five themes independently; treat question 22 (asset ownership) as pass/fail; weight themes one and three double. Any agency scoring green on measurement and incrementality but amber on polish is a better bet than the reverse — polish is buyable, honesty is structural. Expect no perfect scorecards: an honest market has trade-offs, and an agency that greens all twenty-four questions has either rehearsed this exact list or is telling you what you want to hear — both worth noticing. Cross-check the survivors’ claims against their case studies (timeframes, spend bands, and what role they actually played), take the reference calls including a departed client, and only then discuss commercials.

Where we stand in all this, plainly: The Shizz sits the exam too. Across 160+ D2C brands, ₹150 Cr+ of managed spend and ₹450 Cr+ of attributed revenue at a 3.8× portfolio-average ROAS, our answer to question 11 is on this site in public — the holdout logic is in our halo guide — and our answer to question 22 is that everything runs in accounts you own, always. If you want the checklist run against your current setup rather than a pitch, that is precisely what our performance marketing team does inside a Growth Audit.