Real workpaper, not a purpose-built fixture
The material comes from a sell-side valuation model of a wind and solar operator, de-identified and restructured; its seven worksheets keep the original ratios and the original inconsistencies in basis.
Can the Submitted Forecast Serve as the Earnings Base?
AYZZ-FIN-REVMDL-006 · Renewables operator · forecast review · HNW client memo
On a real sell-side valuation workpaper, decide whether the forecast earnings can be used directly as a pricing anchor, and say which part is usable and which must wait: deliver a review workbook, a Chinese memo and a structured conclusion.
Earnings forecastForecast reviewSensitivityThree-part delivery
22 files · 7 worksheets
7worksheets
16scoring criteria
89agent turns in the longest reference rollout
6.37Mcumulative prompt tokens in one rollout
01
Highlights
Three things must stay apart
Source data, your own calculation, and your assumptions. The reader will not rebuild the model, so mixing any two makes the conclusion unverifiable.
The verdict is about how far, not yes or no
You have to give a conditional view, numeric scenarios from two different domains, and observable triggers that would reopen the decision.
Three-model mean score 0.5866
Three frontier models answered under the same criteria for a mean of 0.5866. The longest trajectory ran 89 turns and 6.37M prompt tokens, mostly spent re-checking the basis.
02
Instruction excerpt
An excerpt of the instruction the agent actually receives, showing the task setup and constraints.
instruction.md · 4,678 chars
You are an investment adviser in a private bank's wealth management department. The client is discussing whether to use the forecast earnings from a sell-side submitted model as the starting point for pricing.
The client cares not only about the level of forecast profit but about whether it can support an investment judgement, which conditions need watching, and what a researcher should do when the evidence is insufficient.
Task
- Decide whether the submitted forecast can serve as the earnings base, giving the starting earnings you adopt, a summary of three-year forecast growth, and how far the forecast can be used.
- Support that view with the operating and financial evidence in the workpaper, explaining what you keep, adjust or defer and why, and the decision impact of each material treatment.
- Add a sensitivity analysis and follow-up plan: at least two factors from different domains with different transmission mechanisms, each with a numeric scenario comparison, plus a numerical boundary you define yourself and the factor level that would breach it.
NOTEEnglish translation of the original Chinese instruction; the field definitions and format requirements of the deliverables are omitted.
03
Attachments
Everything visible inside the agent container, with statistics generated from the task package. Original files ship with the full sample package.
| File | Type | Size | Bytes |
|---|---|---|---|
| 安益智造_经营底稿.xlsxOperating workpaper: financial forecast and valuation model over historical and forecast periods | xlsx | 7 sheets | 19 KB |
| 材料说明.mdMaterial note: worksheet structure and period basis | md | 464 chars | 1 KB |
04
Scoring design
16independent scoring criteria
15 scoring items1 penalty items
Full criteria text, weights and reference answers ship with the complete data package.
16 criteria: 14 binary plus 2 five-level items, covering numerical accuracy, conclusion correctness and argument quality.
- 16 criteria: 15 positive scoring items and 1 penalty items (negate)
- Each criterion is judged separately by an agent judge (individual mode), reproducible and auditable
- Judge credentials are injected only into the verifier; the agent container has no scoring channel
- Full criteria text and reference answers ship with the complete data package
Numerical accuracy4
Conclusion correctness3
Argument quality3
Instruction following2
Professional standards2
Factual faithfulness2
05
Model results
Trajectory statistics from frontier models in real agent harnesses. One turn is one model call; the context curve shows how state accumulates over a long-horizon task.
| Model | Agent turns | Tool calls | Cumulative prompt tokens | Peak context | Time | Result |
|---|---|---|---|---|---|---|
| Claude Opus 4.8Claude Code · Harbor | 20 | 1.17M | 77K | — | Completed | |
| GPT-5.6-solCodex · Harbor | 55 | 1.83M | 65K | — | Completed | |
| Qwen3.8-MaxClaude Code · Harbor | 96 | 6.37M | 123K | — | Completed |
06
Specification
| Task ID | AYZZ-FIN-REVMDL-006 |
|---|---|
| Task format | Harbor task.toml · schema 1.4 |
| Package version | 1.0.0 |
| Agent time limit | 3 h |
| Verifier time limit | 5 h |
| Agent container | 2 vCPU · 8 GB memory · 30 GB storage |
| Network | Only for agent harness installation |
| Tool set | filesystem · shell · python |
| Scoring isolation | Separate verifier containerJudge credentials are injected only into the verifier and are invisible to the agent |
| Instruction length | 4,678 characters |
| Difficulty | A2 (three-model mean 0.5866) |
| Reference solution | oracle reward 1.000 |
| Data source | Real sell-side valuation workpaper (de-identified and restructured) |
Contact
Want to run this task on your own platform?
Leave your details and we will send the full sample package for this task (instruction, attachments, verifier and reference trajectories) and walk you through volume and customisation for similar tasks.
- Full task package and verifier
- More tasks in this subdomain
- Integration and re-run support