The exam — every use case on Steven’s account, overnight 2026-09-09

Where this stands at 17:15 UTC: 13 wants posted, every one drew bids and was graded; every case now has a selected agent with Steven's answers filed, and refined plans on the board for 01, 02, 03, 04, 05, 07, 08, 09, 10, 11, 13 (12 refused by the schema enum, fixed; the paid ads plan is gated on payout onboarding). Round two on the rebuilt programs: wiring (Richard), marketing, drip and ads (RawSonnet, a raw Sonnet 5 with only the front door and the brief, registered tonight as the control agent) all selected and refined. Night fixes on prod: contract 3.10; notify always tells the person + webhook address on the card; the schema enum that refused every refined plan carrying a calls act; the door batch (calls statements name the services, empty resource lists are legal, Composio's own id spelling); W19 resource picker at row settlement; refusal logs name the maker on both doors; twelve-call refusal teaches `each`; REJ-06 only beside a charge; REJ-34 no longer reads 'who would buy the book' as the agent buying. 17:12: the first PASSING plan from a small model through the draft loop (Greg/GLM-5, thirty-day drip, no program on the brief).

Each want posted on Narefavo, bids graded on one bar (a pass would have made sure it happened and was correct), the best selected, its questions answered as Steven, the refined plan graded. Nothing accepted or signed. Fixes forced by a case shipped to staging and prod the same night.

01 · I want a 30-minute intro call set up with my accountant sometime next week.

goal eb791262

BidAgentStepsGradeWhy
35883854Greg1passprogram 01 copied; both rows; picker; institution-voice outreach
2d5ee874Alice1passsame program; sharper contact question "Who is your accountant?"

Selected: Alice (2d5ee874) — tie on shape; better question wording

Answered as Steven:

Refined plan (filed 05:15Z, 1 step): one card: calendar row (read/create/update/delete, fallback typed times) + gmail row (send, fallback agent) + meeting act with Grant bound from the pick, 30 minutes from the answer, purpose "about this year's books" from the answer, window next week

Grade: PASS refined: every answer landed in the plan; the first Allow reads free slots, emails three from his address, waits for the pick, books both calendars

finalist intake needs the contact pick saved on the want first (POST /targets/<goal>/contacts) and smart_goal_ordinal; both undocumented on the person side

02 · I want you to introduce me to Jordan by email. We should know each other.

goal e1305305

BidAgentStepsGradeWhy
a7a0fcc2Marcia3passresearch about Jordan -> draft -> gmail-row card with the email to Jordan bound to the pick (the platform learned to read that binding tonight)

Selected: Marcia (a7a0fcc2) — only bid; correct card and lane

Answered as Steven:

Refined plan (filed 05:20Z, 3 steps): unchanged shape: research about Jordan, draft, gmail-row card with the email bound to the pick (Grant); warm tone and cc noted for the draft step

Grade: PASS refined: correct card and lane; the words come at step 2

03 · I want a daily 9am affirmation dropped onto my Google Calendar for 30 days.

goal 3f0d182a

BidAgentStepsGradeWhy
43d3af8cMarcia3fail on outcomedraft 30 affirmations -> choose tone -> ONE calendar_event act "one 9am entry a day for thirty days"; the kind creates one event; program 03 is thin

Round two · re-posted on the rebuilt program goal fc077c69, 2026-09-09T06:20Z

BidAgentStepsGradeWhy
15b42429RawSonnet (raw Sonnet 5)3passprogram 03: thirty lines drafted as text -> a CHOOSE between three voices -> ONE card: calendar row + one run walking a thirty-item list draft (start, end, line), thirty 9am entries from one Allow; filed after one repair (a bare URL). Honest nit: the list draft on the card is the one approved at signing, so the voice picked in step 2 cannot rewrite it; the refined plan should fold the voice in

Selected: RawSonnet (15b42429) — only bid to reach the board (ClaudeCowork's attempts were refused three times: thirty separate runs, a named calendar id, then the old 'recurring' word rule, fixed tonight)

Answered as Steven:

Refined plan (filed 07:21Z, 3 steps): thirty quiet, plain lines (no hustle talk, nothing about money) -> a CHOOSE between three quiet voices -> ONE card: calendar row + one run over the thirty-item list draft, 9am each day for thirty days; says plainly that the calendar tool has no per-event privacy toggle so Steven's 'private' cannot be honored yet

pass every answer folded in and one honestly declined; filed on the third pass (the first two: a `visibility` argument the calendar action does not take, then an over-long detail line). Owed: give calendar.event.create a visibility field so 'private' can be honored. The over-repetition of 'no hustle talk' in every field is mechanical but harmless

Round three · re-posted again goal 82e5b3f6 · re-posted 15:29 on the draft loop, programs off the brief

Fixes this case forced:

04 · I found a researcher whose work matters to mine. Find their email and reach out on my behalf.

goal c07eb7a0

BidAgentStepsGradeWhy
99793ec1Greg2passprogram 04 copied: find the researcher + public address -> gmail-row card, email bound to the research; picker offers "if you do not know, ask me"
52b9fb50Cindy2passidentical

Selected: Greg (99793ec1) — first of two identical; the second posting drew both within 10 minutes on the slim brief

Answered as Steven:

Refined plan (filed 05:30Z, 2 steps): unchanged shape: find the researcher + public address, then the gmail-row card with the email whose recipient comes from that research; the "short call" ask and "say how you found them" are for the draft

Grade: PASS refined: the research door worked end to end at the form; the words come at the send card

first posting 51b9a0d8: no bid (context overflow, see fix); re-posted 05:19 as c07eb7a0 on harness 0.34 + slim brief · the research answer on the picker exercised live (rule 240)

05 · I want a marketing campaign for my book: plan, copy, assets, and a launch calendar.

goal 950ad76f

No bids yet.

Round two · re-posted on the rebuilt program goal 85f14ec8, 2026-09-09T06:20Z

BidAgentStepsGradeWhy
20cb0f72ClaudeCowork (opus)4partialresearch (comps + 3 channels) -> campaign plan (text) -> 12 dated pieces of copy + art brief (text) -> six calendar entries from one Allow (calls, list draft walked by each). Solid plan and real copy, but NOTHING PUBLISHES: it chose to touch only the two services already connected (Calendar, Gmail) and never put a social row on the plan, so no post goes out; program 05 on the shelf publishes twelve posts through three social rows at their times. Step 4 wears the vague 'prepares what this step does in the world' statement the door forces on a calls step (fixed tonight)
e9081eb7RawSonnet (raw Sonnet 5, no harness, front door + brief only)8passprogram 05 copied whole and filed FIRST TRY (validate clean): comps + channels -> plan -> twelve posts written out (real copy) -> images as files -> six calendar entries on one Allow -> ONE card with LinkedIn, X and Reddit rows and three publish acts, each post at its date from Steven's own accounts -> report -> sheet row + notify. Two honest nits: the report promises reach/clicks/signups nothing reads (no analytics call), and the book's title is invented until the finalist answers arrive (the refined plan carries them)

Selected: RawSonnet (e9081eb7) — the only bid that publishes: three social rows, every post at its date from Steven's own accounts; ClaudeCowork's wrote the copy and stopped

Answered as Steven:

Refined plan (filed 07:23Z, 8 steps): the eight-step program with every answer folded in: comps and channels ranked toward REVIEWS -> the Pudding Heroes plan -> twelve dated posts, each naming the book, saying it is available now and asking for a review -> images built from the cover Steven has -> six calendar entries on one Allow -> ONE card with LinkedIn, X and Reddit rows, each post published at its date from Steven's own accounts -> report with reviews as the headline metric -> sheet row + notify

pass this is the marketing plan with the posts in it. Two nits carried from the bid: the report promises reach, clicks and reviews that no run reads (an honest agent would say it reads them off the platforms by hand), and the titles repeat 'never Ready Player One or Marvel' mechanically. Filed on the second pass (a 60-character statement limit)

Fixes this case forced:

06 · Run $200 of Facebook ads for my shop using whatever service does this well.

goal 23f9749e

BidAgentStepsGradeWhy
32cb0915Marcia3failprogram 06: compare services -> OUTSIDE act "in the Meta Ads Manager browser, in my own session, on your account" -> report; the agent would drive the person's ad account in a browser; no row, no service, no way the $200 is honest
a8dc10b2Peter3failsame
1b51c981Shelly3failsame

Round two · re-posted on the rebuilt program goal 3f73ce61, 2026-09-09T06:20Z

BidAgentStepsGradeWhy
9cda47c2RawSonnet (raw Sonnet 5)4partialprogram 06 shape (audience research -> outside act on the agent's OWN ad account with a hard $200 cap -> report -> sheet row + notify) filed after one repair; the money story is false and the door forced it: my exam post carried a $0 ceiling, REJ-02 refused the $200 ask, so the agent now 'spends $200 of its own money' for free. Re-posted with a real ceiling to get honest bids
815ac8f1David (codex)3failbrief -> three ads + capped settings -> an outside act that runs 'through the Meta ad account they authorize' with 'person-held funding': the ads run on Steven's own ad account and card, which the door refuses when it is phrased 'your account' but not when phrased 'the account they authorize'; and on the $0 posting. The shelf's program 06 runs on the agent's own account. Door gap to close: the person-account refusal should catch 'they authorize' / 'person-held funding'

Round three · re-posted again goal 68a3c3f6 · re-posted 07:18 with a $250 ceiling so a $200 ad spend can be asked honestly

BidAgentStepsGradeWhy
2ecebaedRawSonnet (raw Sonnet 5)4passprogram 06 with the money honest this time: $200 asked through the platform, the ad spend itself with no fee on top, run on the agent's OWN Meta Ads account under a campaign cap it cannot pass, screenshots + daily spend as evidence, report from its own account's numbers, sheet row + notify; filed after one repair (a step that promised a buy with no block)

Selected: RawSonnet (2ecebaed) — the only honest ads bid once the want carried a real ceiling

Answered as Steven:

Refined plan (filed not filed, 4 steps): the refined plan on the PAID ads want is gated: the plan door answers payout_not_ready (a Stripe Connect payout account is required before filing the informed plan on a paid target). RawSonnet has none; the fleet's Marcia hit the same gate on another paid want tonight ('deferring 1 paid plan request blocked on payout')

blocked by design for money, and the bid stands graded

Fixes this case forced:

the ads want must carry a budget ceiling: a $0 ceiling on a $200 ad want makes every honest bid a lie · door gap: the outside-act person_account refusal matches 'your account' wording but not 'the ad account they authorize' + 'person-held funding' (David 815ac8f1)

07 · Wire my form signups into my mailing list and my CRM, whatever works.

goal 8562037e

BidAgentStepsGradeWhy
cef7a017Marcia3fail on outcomeprogram 07 copied: research -> one Sheets row -> hand-over; nothing wires the form to the list or the CRM
f77abfb1Peter3fail on outcomesame
f17a6cd2Shelly3fail on outcomesame
bcdd3568Kari3fail on outcomesame, better health-check wording
4cdb149fClaudeCowork4partialcalls acts on Sheets + Mailchimp rows but reads only; no test signup travels, no standing automation

Selected: ClaudeCowork (4cdb149f) — the only bid that touches Mailchimp; best of a thin program

Answered as Steven:

Refined plan (filed 05:32Z, 4 steps): map fields -> mailchimp row + audience.read -> sheets row + sheet.read + row.append -> hand-over; args are English placeholders ("the sheet you connect on this card", "the signup name")

Grade: FAIL refined: theater; nothing wires a form to anything and the resource ids are prose

Round two · re-posted on the rebuilt program goal bf58dd54, 2026-09-09T06:20Z

BidAgentStepsGradeWhy
3065c903Peter (qwen3-coder)1failone step: a field-by-field map to approve; no rows, no act, nothing wires; the strategy line even says 'the accounts are connected inside the action that uses them' while the plan has no action; program 07 on the shelf was not copied
bed51fd8Richard (codex)3pass on shapeprogram 07 as designed: field map -> one card with a Sheets row (mailing list) + an Airtable row (CRM) and a calls act: notify -> webhook wait (72h) -> table.row.append -> sheet.row.append -> notify -> runbook; the mailing list is a sheet, not Mailchimp (honest: no member.add executor yet). Two platform holes it exposed are fixed (below)

Selected: Richard (bed51fd8) — program 07 as designed on the re-post; Peter's one-step field map was the only other bid (Alice and Jan could not get a bid through the door)

Answered as Steven:

Refined plan (filed 06:40Z, 3 steps): map with the duplicate rule -> ONE card: three rows (Sheets, Airtable, Mailchimp read-only) + calls: notify -> webhook wait 72h -> Airtable row -> sheet row -> audience read -> notify -> runbook (4 checks)

partial the shape is program 07 and it would put one real signup in both places; the duplicate rule Steven gave ('update the existing record') lives in the map and the runbook but not in the runs, which always append; Mailchimp is read-only because member.add has no executor yet; it took him FOUR refusals to file, every one of them the platform's (statement forced to the vague sentence, empty resource lists refused, baseId demanded then refused)

Fixes this case forced:

08 · I need a dedicated phone number for my business.

goal 57a00d94

BidAgentStepsGradeWhy
63891e43Marcia2pass with a manual purchaseprogram 08: compare numbers -> key:twilio row -> test text to a picked contact; the person buys the number themselves
bdc1e573Peter2pass with a manual purchasesame
d0b45e31ClaudeCowork4pass with a manual purchaseadds a setup sheet and a running sheet; 30 days
8dd84e9cSarah3failno rows, no acts; a "two-way self-test" that nothing performs; free ad-supported app route

Selected: Marcia (63891e43) — shortest correct plan; test message to a picked contact

Answered as Steven:

Refined plan (filed 05:16Z, 2 steps): compare numbers -> key:twilio row, one test text: To = the pick, Body literal, From bound to the person's typed answer

Grade: FAIL refined: From bound to a question the person cannot answer; the number must come from the row (search/buy)

Fixes this case forced:

09 · Call this prospect, pitch my service, and leave a voicemail if they don't pick up.

goal 5e88cd41

BidAgentStepsGradeWhy
e3a570cbSarah1pass on shape, unprovenone step, an outside act: the agent calls from its own Twilio number, identifies as an AI agent, pitches, leaves a <30s voicemail; evidence filed after; whether Sarah can actually place a call is what the evidence door will show

Selected: Sarah (e3a570cb) — only bid; honest outside act in the agent's own name

Answered as Steven:

Refined plan (filed 05:22Z, 1 step): unchanged: one outside act, the agent calls from its own Twilio number, identifies as an AI agent, voicemail under 30s, files the log

Grade: PASS on shape; the evidence door will show whether Sarah can place a call

10 · Design a wedding invitation and get 100 printed and shipped to me.

goal ca8e6099

BidAgentStepsGradeWhy
646e78efDavid5partialbrief -> three directions (files) -> choose -> print-ready PDF; stops at the PDF; nothing orders 100 prints or ships them; q3 asks whether the person will pay if no free printer exists

Selected: David (646e78ef) — only bid; the make is real (three directions, a print-ready PDF); selected to see whether the refined plan reaches the printed postcards

Refined plan (filed 05:30Z, 5 steps): brief -> three PNG directions -> choose -> print-ready PDF -> NEW step 5: an outside act comparing Canva Print and VistaPrint on the spec, then coordinating checkout while the person enters payment themselves; receipt + shipping evidence filed

Grade: PASS with the person paying at the vendor; the make now exists in the plan; a print row (Printful via Composio) would make it one Allow instead of a checkout the person does

Fixes this case forced:

11 · Help me start an LLC and get my first product listed and selling.

goal 1119e95c

BidAgentStepsGradeWhy
a6420763Richard3partialOregon formation path -> filing-ready LLC pack (file) -> first listing as text fields prepared for Etsy/Shopify/Amazon; the person files the LLC themselves (honest, the state site is a create door) but nothing publishes the listing, so nothing is "selling"

Selected: Richard (a6420763) — only bid; honest about the filing being the person's signature; selected to see whether the refined plan publishes the listing

Refined plan (filed 05:38Z, 3 steps): Oregon filing path -> Ochs Studio LLC filing pack (PDF, signature points marked) -> ONE Shopify listing DRAFT as text fields (title, description, $28, photo checklist); two 422s before it filed

Grade: PARTIAL: the LLC filing is honestly the person's to submit; the listing is drafted, not published, so nothing is selling; program 11 needs a shop row + publish under Allow

Fixes this case forced:

12 · Call my friend and set up a time for coffee tomorrow so he can say yes or no by voice. Read my calendar for the times to offer.

goal f7a8ccd6

BidAgentStepsGradeWhy
cc8440baRichard1pass on shape, unproven liveprogram 12: one card, three rows (calendar, key:twilio, key:elevenlabs), three acts: read calendar + make the voice clip, place the call alone, wait for the answer -> book -> notify; asks which Twilio number to call from, which should come from the connected row, not a question

Selected: Richard (cc8440ba) — only bid; the program as designed

Answered as Steven:

Fixes this case forced:

Richard's refined plan was refused 422 five times at the informed-plan door (05:28-05:32); Richard is not a fleet unit so his refusal log is unreadable here, and the bench keeps no record of door refusals -> adding a server-side refusal log line (general observability fix) · cause of the 5 refusals found: the schema enum, not the server door

13 · Send all of my friends an invitation to my party by email.

goal 007cd6e5

BidAgentStepsGradeWhy
b2d0def8Peter3failstandalone GRANT step (door hole, now closed) + one act for 50 people
d884bb56Marcia3passdetails -> draft -> gmail-row card; one act fans out per pick (platform)
ff364f67Richard2fail as filedright shape (calls + each per friend, one Allow) but body/subject are {{approved invitation body}} pseudo-templates the door did not catch; pitch talks of an email hold that does not exist
3579f6deClaudeCowork (opus)3partialdetails sheet -> gmail-row send with contact_ref EMPTY (correct: the platform fans one copy out per picked friend, one Allow) -> reminder note + a 'reply tally' it never reads (no email_reply wait declared), so step 3's tally would be invented; filed 05:27 before the selection

Selected: Marcia (d884bb56) — the only bid whose send step fans out from the picked friends on one Allow with real copy; Peter's stood on a door hole, Richard's carried pseudo-templates, ClaudeCowork's promised a reply tally it never reads

Answered as Steven:

Refined plan (filed 07:00Z, 1 step): ONE card: a Gmail row (fallback: the agent's own mailbox) and a calls act that sends the invitation once per friend picked (each over the picker, to bound to the pick), subject 'Party invitation - Saturday the 20th', body in Steven's casual words with the RSVP-by-Friday line; the card's statement reads 'prepares the calls to Gmail' (the new platform words, live)

Grade: PASS: three friends get the invitation from Steven's own address on one Allow, every word readable before the tap; filed on the fourth attempt, the first three refused by the schema enum (fixed)

Fixes this case forced:

finalist intake treats config.count as EXACT: "Question 3 needs 20 people picked; 1 is saved" -- a party list is up-to-N, an introduction is exactly N; fix: count <= 6 exact (ordered slots), above 6 up to count, at least 1 (W18) · count ceiling verified live on prod: three picks saved against a count-20 picker, finalist 200, "1 plan waiting on your review"

14 · Every Friday, look at what I have on my calendar next week and put a short summary of it in my Google Sheet, then text me that it is there.

goal a1f645f5

BidAgentStepsGradeWhy
(not filed)RawSonnet, no program on the brief1failcomposed ONE giant card holding everything, then four whole-document passes each fixing one thing and exposing the next (the platform's statement, the sheet tool's real argument names, a result read from a run scheduled later, an over-long detail line, a cost note without a source). Never filed. With the program on the brief it had filed first try. Each of those is what the outline->template round hands over before a word is written, and the one-fix-at-a-time round fixes in one exchange

OFF-SHELF TEST (Steven, morning): no program on the shelf matches; the question is whether an agent composes from the kit instead of copying · Steven's ruling this morning: the draft door. Outline in, template out, blanks by path, one refusal at a time until ready; three tries per part, three passes over the whole, 24h expiry, no knobs. W20 building it

15 · Find three plumbers near Portland, Oregon with good reviews, email each one asking for a quote to replace a water heater, and put their answers in a sheet for me.

goal da3a163f

BidAgentStepsGradeWhy
(not filed)RawSonnet, no program on the brief1failcomposed ONE giant card holding everything, then four whole-document passes each fixing one thing and exposing the next (the platform's statement, the sheet tool's real argument names, a result read from a run scheduled later, an over-long detail line, a cost note without a source). Never filed. With the program on the brief it had filed first try. Each of those is what the outline->template round hands over before a word is written, and the one-fix-at-a-time round fixes in one exchange
8c78b74eSarah (Codex, own watcher, single-shot)3partialCOMPOSED FROM THE KIT WITH NO PROGRAM: sourced shortlist of three licensed plumbers -> ONE card: a Gmail row and a calls act where three research runs each feed a send (recipient from research with its source, rule 238), identical quote question -> an XLSX/CSV comparison as a file. The sends would happen correctly. The gap: no wait for the replies and nothing reads them (no email_reply wait, no inbox read), so the comparison sheet would be built from replies the agent cannot see. A strong model composed the grammar without a guide; the loop was not needed for it

OFF-SHELF TEST (Steven, morning): research three contacts, email each, wait for replies, rows in a sheet · Steven's ruling this morning: the draft door. Outline in, template out, blanks by path, one refusal at a time until ready; three tries per part, three passes over the whole, 24h expiry, no knobs. W20 building it