Not a modelling task — a model review
The key settings were made by someone else and the model is in your hands. The question is how many years the forecast can support and which input fails first.
Three-Year Earnings Forecast and Entry Decision
QYDZ-FIN-PRAS-142 · Assumption review · single-factor elasticity · usable horizon
The forecast is already finished; the job is to audit it. Report the three-year net profit you accept, trace every step to its input, measure which input matters most, and compute what level would overturn the entry decision.
Earnings forecastElasticityEntry decisionAssumption review
22 files · 5 worksheets
5worksheets
12scoring criteria
45agent turns in the longest reference rollout
2.60Mcumulative prompt tokens in one rollout
01
Highlights
Elasticity needs a common yardstick
Quantify how much three-year profit moves per one point or one unit of each input, put them on a comparable scale, and only then rank them.
The overturn threshold is solved backwards
The task asks for the parameter level at which the entry conclusion fails, which means inverting the calculation and substituting the critical value back.
Three-model mean score 0.4869
Three models answered under the same criteria for a mean of 0.4869. The longest run took 45 turns and 2.60M prompt tokens, almost all of it re-checking assumptions.
02
Instruction excerpt
An excerpt of the instruction the agent actually receives, showing the task setup and constraints.
instruction.md · 2,157 chars
You are a family financial planner at a family office. The stock is a candidate or existing position in the household's equity allocation, and your conclusion decides whether the family buys, adds, or holds, and roughly how much.
The family will set the position at its next meeting. What you are given is a finished earnings forecast, not a request to build one: the key assumptions were made by colleagues and you hold the model and its business material.
Task
- Report the three-year net profit attributable to shareholders that you accept, as a figure or a range, showing step by step how it was derived and which input each step used.
- Which of those inputs matters most? Quantify how much three-year profit changes per one-point or one-unit move in it.
- If the assumption fails, how does the entry conclusion change? Quantify the level that would overturn the first conclusion, and state the latest year the forecast can be used for, with reasons.
NOTEEnglish translation of the original Chinese instruction; the deliverable format requirements are omitted.
03
Attachments
Everything visible inside the agent container, with statistics generated from the task package. Original files ship with the full sample package.
| File | Type | Size | Bytes |
|---|---|---|---|
| 材料说明.mdMaterial note: worksheets and period basis | md | 163 chars | 359 B |
| 澄岳_经营底稿.xlsxOperating workpaper: product-level forecast inputs and the three-year forecast | xlsx | 5 sheets | 30 KB |
04
Scoring design
12independent scoring criteria
11 scoring items1 penalty items
Full criteria text, weights and reference answers ship with the complete data package.
12 criteria: 9 binary plus 3 five-level items, including one conclusion-correctness penalty item.
- 12 criteria: 11 positive scoring items and 1 penalty items (negate)
- Each criterion is judged separately by an agent judge (individual mode), reproducible and auditable
- Judge credentials are injected only into the verifier; the agent container has no scoring channel
- Full criteria text and reference answers ship with the complete data package
Numerical accuracy3
Conclusion correctness3
Professional standards2
Argument quality2
Instruction following1
Factual faithfulness1
05
Model results
Trajectory statistics from frontier models in real agent harnesses. One turn is one model call; the context curve shows how state accumulates over a long-horizon task.
| Model | Agent turns | Tool calls | Cumulative prompt tokens | Peak context | Time | Result |
|---|---|---|---|---|---|---|
| Claude Opus 4.8Claude Code · Harbor | 14 | 835K | 84K | — | Completed | |
| GPT-5.6-solCodex · Harbor | 25 | 648K | 51K | — | Completed | |
| Qwen3.8-MaxClaude Code · Harbor | 44 | 2.60M | 128K | — | Completed |
06
Specification
| Task ID | QYDZ-FIN-PRAS-142 |
|---|---|
| Task format | Harbor task.toml · schema 1.4 |
| Package version | 1.0.0 |
| Agent time limit | 3 h |
| Verifier time limit | 5 h |
| Agent container | 2 vCPU · 8 GB memory · 30 GB storage |
| Network | Only for agent harness installation |
| Tool set | filesystem · shell · python |
| Scoring isolation | Separate verifier containerJudge credentials are injected only into the verifier and are invisible to the agent |
| Instruction length | 2,157 characters |
| Difficulty | A3 (three-model mean 0.4869) |
| Reference solution | oracle reward 1.000 |
| Workpaper size | 5 worksheets · single file |
Contact
Want to run this task on your own platform?
Leave your details and we will send the full sample package for this task (instruction, attachments, verifier and reference trajectories) and walk you through volume and customisation for similar tasks.
- Full task package and verifier
- More tasks in this subdomain
- Integration and re-run support