Agentic evaluation means using AI to grade AI. Where a person might read a keyword list once and give one opinion, an eval can run 20 structured checks across brand tone, compliance and every step of a workflow. Here’s how Jeremy MacDonald showed it in his BrightonSEO talk, step by step, using keyword research for the running brand On.
(This is part three of my series on his talk. Start with why AI output needs real testing and cheaper, better, revolutionary.)
Steps 1 and 2: The RISE prompt and the eval prompt
His prompt framework is RISE: Role, Inputs, Context, Constraints, Examples. The generation prompt and the grading prompt use the same structure, aimed at different jobs:
| Element | Generation prompt | Eval prompt |
|---|---|---|
| Role | Act as an SEO expert | SEO expert with a deep understanding of the brand |
| Inputs | The brand PDF and the product details | The brand context again, plus the generated keywords |
| Context | Generating keywords for both AEO and SEO | Grade the keywords against a defined framework |
| Constraints | Keywords only, plus one line on why each is applicable | Score each against three criteria, output a list and sum the scores |
| Examples | A list of historic keywords you’ve already done | A sample scored keyword |
His criteria in the talk were relevancy, brand fit and opportunity. In the fuller demo, he scored keywords on four things: how relevant they are, their quality, product fit and brand fit.
Step 3: Chain it
The value comes from stringing evals together, with each step’s survivors feeding the next:
Jeremy called it a primitive example, but it shows the pattern: every stage is checked at a level of detail a person wouldn’t match by hand. Compare that with the one-pass, unchecked AI audits landing in inboxes everywhere.
Tips for your first version
- Use named criteria and a fixed scale so scores can be compared and summed.
- Keep the brand context in the eval prompt, not only the generation prompt.
- Start with one step, such as the keyword pool, before you chain.
- Read a sample of graded outputs yourself to see whether you agree. Calibrating the grader gets its own post later in this series.
Next up: how to feed the low scorers back into the brief so the workflow improves itself.
Want help building eval steps into your SEO or AEO workflows? Get in touch.
Leave a Reply