491 pages of untrimmed filings
Four original PDFs (annual, interim and Q1 reports plus a generation bulletin) spanning three reporting periods, where the same metric is not always disclosed the same way.
Equity Valuation of XX Power
FIN-IB-DCF-001 · Bottom-up DCF valuation · A-share listed power company
Build a full valuation model from generation volume and tariffs through free cash flow on 491 pages of original Shanghai Stock Exchange filings, delivering a workbook, an investment-committee report and a recomputable record of key figures.
DCFFinancial modellingLong-document extractionExcel deliverable
55 PDFs · 491 pages
491pages of original PDFs
27scoring criteria
56agent turns in the longest reference rollout
3.58Mcumulative prompt tokens in one rollout
01
Highlights
No reference algorithm, only judgement
The model chooses the forecast structure, definitions and market parameters, but every historical figure must trace to a location in the source and every computed figure must be recomputable.
Three-model mean score 0.324
Claude Opus 4.8 scored 0.536 and GPT-5.6-sol 0.436; Qwen3.8-Max did not finish within 60 minutes. Frontier models are far from saturating this task, leaving ample headroom for reward.
The deadline is part of the task
The instruction states that late work counts as missing. The model must balance depth of argument against complete delivery within an hour, as a real analyst would.
02
Instruction excerpt
An excerpt of the instruction the agent actually receives, showing the task setup and constraints.
instruction.md · 5,350 chars
Background
You are an analyst on a securities firm's power and utilities research team. The investment committee will discuss at its next meeting whether to add XX Power Co., Ltd. to its core holdings, and has asked the team for a bottom-up equity valuation first.
The committee's requirement: no quick comparable-multiple shortcuts. It wants a cash-flow model built up layer by layer from generation volume, tariffs and costs, and every historical input in the model must trace back to the company's own public filings. Committee members will spot-check the figures you cite and recompute your conclusions from your inputs.
Task
Using 30 June 2026 as the valuation date, produce a bottom-up earnings forecast and discounted cash-flow valuation of XX Power and give an equity value per share. The explicit forecast must cover at least three full years, followed by a terminal value. How you build the forecast, define metrics and set parameters is up to you — the committee is judging your reasoning, not a fixed algorithm.
The four documents cover three different reporting periods, and the same metric may not be disclosed consistently across them. The valuation date overlaps the first forecast year — the part before the valuation date has already happened and already appears in the documents.
Hard constraints (excerpt)
- Source scope: use only the four public filings provided — no market data, third-party research, macro statistics, or anything you remember about this company from training.
- Information cutoff: 31 August 2026. Do not cite, assume or imply any fact that occurred or was disclosed after that date.
- Consistent definitions: the same economic benefit must not be counted twice, and the same burden must not be deducted twice.
- Form of conclusion: the valuation must be given as a range, not a single number, with the key assumptions that drive the range.
NOTEEnglish translation of the original Chinese instruction; the deliverable list and file-format requirements are omitted.
03
Attachments
Everything visible inside the agent container, with statistics generated from the task package. Original files ship with the full sample package.
| File | Type | Size | Bytes |
|---|---|---|---|
| company_2025_annual_report.pdf2025 annual report (audited) | 259 pages | 1.5 MB | |
| company_2026H1_generation_bulletin.pdf2026 H1 power generation bulletin | 1 pages | 96 KB | |
| company_2026Q1_report.pdf2026 Q1 report | 17 pages | 177 KB | |
| company_2026_interim_report.pdf2026 interim report | 214 pages | 1.2 MB | |
| MANIFEST.jsonDocument manifest (type, period, disclosure date, sha256) | json | 2,031 chars | 2 KB |
04
Scoring design
27independent scoring criteria
21 scoring items6 penalty items
Full criteria text, weights and reference answers ship with the complete data package.
27 criteria — 20 binary and 7 five-level Likert — covering numerical accuracy, reasoning and professional standards.
- 27 criteria: 21 positive scoring items and 6 penalty items (negate)
- Each criterion is judged separately by an agent judge (individual mode), reproducible and auditable
- Judge credentials are injected only into the verifier; the agent container has no scoring channel
- Full criteria text and reference answers ship with the complete data package
Reasoning depth9
Numerical accuracy8
Professional standards7
Factual faithfulness2
Instruction following1
05
Model results
Trajectory statistics from frontier models in real agent harnesses. One turn is one model call; the context curve shows how state accumulates over a long-horizon task.
| Model | Agent turns | Tool calls | Cumulative prompt tokens | Peak context | Time | Reward |
|---|---|---|---|---|---|---|
| Claude Opus 4.8Claude Code · Harbor | 31 | 2.11M | 112K | 14m 32s | 0.536 | |
| GPT-5.6-solClaude Code · Harbor | 69 | 3.58M | 90K | 34m 46s | 0.436 | |
| Qwen3.8-MaxClaude Code · Harbor | 44 | 2.07M | 91K | 1h 0m | Hit time limit |
06
Specification
| Task ID | FIN-IB-DCF-001 |
|---|---|
| Task format | Harbor task.toml · schema 1.4 |
| Package version | 1.3.3 |
| Agent time limit | 1 h |
| Verifier time limit | 5 h |
| Agent container | 2 vCPU · 8 GB memory · 30 GB storage |
| Tool set | filesystem · shell · python |
| Scoring isolation | Separate verifier containerJudge credentials are injected only into the verifier and are invisible to the agent |
| Instruction length | 5,350 characters |
| Difficulty tier | A3 (three-model mean < 0.5) |
| Reference solution | oracle reward 1.000 |
Contact
Want to run this task on your own platform?
Leave your details and we will send the full sample package for this task (instruction, attachments, verifier and reference trajectories) and walk you through volume and customisation for similar tasks.
- Full task package and verifier
- More tasks in this subdomain
- Integration and re-run support