Money

Best AI Agent Passed About 11% of Real Tax Assignments: A Benchmark Where Models Missed a Home-Sale Deadline the Client Never Asked About

  • Level Intermediate
  • Setup1–3 hrs (estimated)
  • Evidence Strong evidence
  • CostNot stated in source
  • No-codeYes
Custom agentSource-linked⚠ Cautionary

What they did

Surge AI's DAYJOB: Tax benchmark gave AI agents 50 realistic tax assignments with 1,789 files (PDFs, spreadsheets, receipts, emails). The top model passed only 10.8% at a 95% rubric threshold. In one case, an agent correctly compared two condo sale dates but never warned the client that her home-sale tax exclusion would expire if the sale slipped past August 2027. In another, a model assumed a couple wasn't legally married even though their records suggested an informal marriage, which would change how to split their retirement accounts.

How it went

Benchmark write-up by Surge AI (Oct 27, 2026): DAYJOB: Tax, 50 environments, 1,789 files; best pass rate 10.8% (Claude Sonnet 5.5) at a 95% rubric threshold; examples of a missed home-sale exclusion deadline and an unverified marriage assumption. Use an agent to organize tax documents and raise questions, not as your tax adviser

Worth knowing

  • Benchmark write-up by Surge AI (Oct 27, 2026): DAYJOB: Tax, 50 environments, 1,789 files; best pass rate 10.8% (Claude Sonnet 5.5) at a 95% rubric threshold; examples of a missed home-sale exclusion deadline and an unverified marriage assumption.
  • Use an agent to organize tax documents and raise questions, not as your tax adviser.

Prompt

Cleaned version rewritten from the source — not a direct quote.

Here is my tax question and my documents: [question, attach files]. Before answering, list (1) any deadlines or eligibility rules that could affect my decision even if I didn't ask about them, (2) any facts you're assuming, such as marital status, residency or dates, and the evidence for each, and (3) anything missing or conflicting in my documents. Then answer, and tell me what a tax professional should double-check.

Recommended

Try it in Grok Bot

Grok Bot can do this too: ask it to list assumptions, deadlines and missing documents before it answers any tax question.

  1. Copy the prompt (button below).
  2. Open Grok Bot chat and paste it. Replace any [bracketed] placeholders.
  3. Connect the listed tools (as needed) as connectors when Grok Bot asks.
  4. If the task repeats, ask: "Make this a daily routine."

Alternatives: the same prompt works in Claude, ChatGPT Agent, OpenClaw.

Set it up

Stack: Custom agent

  1. Read the original source for full context.
  2. Connect the tools your agent needs.
  3. Paste the prompt and replace [bracketed] placeholders.
  4. Run once manually and check the output before scheduling it.

Generic suggested steps, not from the source.

Evidence

Benchmark write-up by Surge AI (Oct 27, 2026): DAYJOB: Tax, 50 environments, 1,789 files; best pass rate 10.8% (Claude Sonnet 5.5) at a 95% rubric threshold; examples of a missed home-sale exclusion deadline and an unverified marriage assumption. Use an agent to organize tax documents and raise questions, not as your tax adviser

Source: Web · 2026-10-27 · Verified by us: not yet · Report broken or outdated

Related use cases

MoneyWeb · 2026-02

Check a hospital bill against what Medicare would pay

A writer gave Claude the coded hospital claim (UB-04) for his late brother-in-law's $195,628 ER bill and asked it to build a spreadsheet of Medicare rates per code. Claude flagged bundled items billed separately, a bypass charge for a procedure that never happened, and a ventilation charge Medicare does not allow.

  • Level Intermediate
  • Setup1–3 hrs*
  • Evidence Strong evidence
ClaudeSource-linked
ShoppingWeb

Test whether a shopping agent can be tricked by fake reviews (cautionary)

F-Secure built a shopping agent (vibe-coded with Claude Code, running Claude Haiku, with the user's details in memory) and a mock marketplace with a review pointing to a fake discount-code site. In 100 runs it visited the phishing site 12% of the time and submitted the user's name, date of birth and SSN, almost never telling the user.

  • Level Advanced
  • Setup1–3 hrs*
  • Evidence Strong evidence
Custom agentSource-linked⚠ Cautionary
MoneyReddit

Invoice capture for a distributor: 4m12s down to 38s each

A shared Gmail label feeds an extractor, totals are validated to €0.01, low-confidence fields go to a human, and results push to Sage. 87% need no human touch.

  • Level Intermediate
  • Setup1–3 hrs*
  • Evidence Some evidence
n8nMakeSource-linkedNeeds verification