The result
Efforthigh, 7.5 minutes, 40 rows, trust grade high (“cells fill rate is 99% and trust per claim ratio is 99%”). Trimmed:
latest_round, and about a third of the source_url values are top-100 listicles rather than the announcement. The trust grade is high because every cell is filled and cited. Rows returned against criteria, not the grade, is the check. See Evaluating Results.
The fix is one step in the skill and schema, shown in Iterate below.
Build it
1
Pick the effort
Dataset building runs at
high; the API requires it. x-high returns roughly a quarter more rows at four times the price and is for exhaustive single-site catalogs. See Effort and Cost.2
Create the agent
Skill as the playbook, goals as hard constraints, a flat array schema with every field described.
3
Run it
The input states the filters, the channels to search and the row target. Filters in the input are hard constraints.Poll while
is_active is true, then fetch the result. Plan for 5 to 15 minutes at high. The loop is in the Quickstart.4
Check the rows, not the grade
Three checks, in order:
- Rows returned against rows asked. 40 of 40 here. Fewer means the ask was too wide or the criteria ran out.
- Each row against each filter. Read
primary_productagainst the vertical,hq_countryagainst the geography,latest_round_yearagainst the window. Here 13 rows fail the vertical and one fails the geography. - A sample of
source_url. Open five. An announcement or a news article about that round is evidence. A top-100 list is not.
5
Iterate
The rows that failed the vertical filter share a cause: “AI infrastructure” was defined by example in the input, and the agent stretched it to reach 40. The fix moves the definition into the skill and the schema so the agent has to classify each row before including it:
- Skill: define what qualifies and what does not (“companies whose primary product is a foundation model, an end-user application, a voice or video product, security analytics or quantum hardware do not qualify, however large their funding”).
- Schema: add
infrastructure_categoryas an enum of the qualifying categories, described as “if none fits, the company does not qualify and must not be included”. A row cannot be written without picking one. - Goals: make the exclusion a checkable goal.
- Sources:
prioritizethe announcement itself;avoid“top-100 lists, roundups and funding trackers as source_url”.
Before and after
Same effort, same price. Fewer rows, because the agent now had to qualify each one, and a longer run for the same reason. Not clean either: six rows are still arguable, one company appears twice, and five citations are still roundups. That is the normal shape of a second run. The remaining fixes are a dedupe on
domain in your own code, a canary check on the borderline rows, and a stricter avoid on the source. Two runs and $1 got a list a person can now review in ten minutes instead of building from scratch.
Best practices for dataset building
- Filters are hard constraints; define them so they can be checked. A category the agent has to pick from an enum is checkable. “AI infrastructure” by example is not. Nice-to-haves become columns, not filters. Keep filters to two or three.
- Set a row target and a stop rule. “Up to 40; do not stop before 40 unless the criteria are exhausted.” Without the stop rule, “at least 40” often returns exactly 40 padded rows, or 20 and a completed status.
- Scope each run to 50 to 100 rows. One run over a whole market or a whole catalog truncates. Split by country, category or site section and merge on
domain. See Running at Scale. - Name the channels. Funding news, directories, portfolio pages, the company’s own site. Naming them moved one user’s row-target adherence from half to full.
- Carry an exclusion list. On the second region or the second pass, put the domains already found in
input_dataand say “exclude these”. - Flat array, 5 to 8 described fields, enums for classification. Wide undescribed schemas are the top cause of partial output. See Output Schema.
- Discovery first, enrichment second. Keep this agent to identity and filter fields. A second enrichment agent fills the columns, ten rows at a time.
- Announcement URLs, not listicles. Say it in the schema description and in
sources.avoid. Listicles are how off-target rows get in. - Avoid “all” and “every” on the open web. On a single named site at
x-high, exhaustive is fine. Across the web, the agent stops when the budget ends.
From vague to production-ready
- Vague
- Better
- Production-ready
Troubleshooting
Rows that do not match the filters
Rows that do not match the filters
The filter was defined by example and stretched to reach the target. Define it in the skill with explicit exclusions, and add an enum field the agent must fill to include a row.
Far fewer rows than asked
Far fewer rows than asked
Either the criteria are genuinely exhausted or the schema was too wide to fill in budget. Narrow the schema first. Then split the ask by geography or category and merge.
Exactly the minimum, or the same number every time
Exactly the minimum, or the same number every time
“At least N” was read as a target. Say “do not stop before N unless the criteria are exhausted”. Check the last rows: padding shows up there.
Duplicate companies
Duplicate companies
Add “one row per company, deduplicated on domain” as a goal, and dedupe on
domain after merging runs anyway.Citations are lists and rankings
Citations are lists and rankings
sources.avoid: “top-100 lists, roundups and funding trackers as source_url”. Describe source_url as “the announcement or an article about this specific round”.A value failed schema validation
A value failed schema validation
A year landed in
latest_round, or a string in an integer field. The trust report flags the path with low and “Value failed schema validation”. Tighten the description (“Never a year or an amount”) and use an enum.422 on create or run
422 on create or run
Schema keyword,
$ref, or root shape. See hard limits.Output schema limits
Allowed roots
The root must be one of:- An object with non-empty
properties - An array of objects (
items.typemust be"object"withproperties)
output_schema (or pass {}) to let the generator infer the shape from the prompt.
Supported examples
Object root with nullable fields:Rejected patterns
The API returns 422 when a schema violates these hard limits:
Move validation constraints into field
description text instead of JSON Schema keywords.
Unsupported examples:
Still accepted
These are not hard-failed:requiredarrays on objectsadditionalProperties: false- Nested
anyOf/oneOffor nullability (not at the root) - Enums within the size limits above
- Nested arrays of objects or primitives under an object root
Field maps are invalid
Each field must be a JSON Schema subschema, not a bare type string:Agent generation behavior
When creating or updating an Agent, the design path can infer and sanitize a schema from your goals and template. The generator strips unsupported keywords and fixes common mistakes (such as standalonenull) before validation.
When you pass an explicit output_schema on create, update, or run, it is validated immediately. Fix the schema and retry on 422.
Common validation errors
Example 422 response: