The reader is not the workpaper's author
The reader is an investment decision-maker who needs to move quickly. The workbook has to let another analyst recompute, and the machine-readable summary has to feed the system directly.
Growth Quality and FX Exposure of an Overseas Software Company
CLMR-FIN-FXCALC-005 · Growth basis · FX transmission · reusable research file
Two growth datasets into one judgement: what actually drives growth, how much of it is FX in the reporting and in the client's return, and whether the growth data can still be ingested into the research system.
Cross-borderGrowth qualityFX transmissionData ingestion
33 files · 4 worksheets
4worksheets
16scoring criteria
23agent turns in the longest reference rollout
1.36Mcumulative prompt tokens in one rollout
01
Highlights
Two datasets, two bases
Business groups with a revenue ledger, and a financial series with a quarterly overview. Annual versus quarterly, and different currencies, must be aligned before anything is compared.
Do not compare different scales by size
A one-unit move in a multiple and a one-unit move in an exchange rate are not the same quantity; sensitivity is about elasticity, not the absolute change.
Three-model mean score 0.5702
Three models answered under the same criteria for a mean of 0.5702, while individual scores spread by nearly 0.35: a run that does not align basis and scale gets every later step wrong.
02
Instruction excerpt
An excerpt of the instruction the agent actually receives, showing the task setup and constraints.
instruction.md · 4,384 chars
You are an investment adviser in a securities firm's cross-border wealth business. The client's base currency is RMB and invests through Stock Connect, QDII, or an offshore brokerage account, so the exchange rate moves both the company's reporting and the client's actual return.
The client is assessing the growth quality of an overseas software company and plans to ingest this round of growth data into the research system. Numbers in the system will feed company tracking pages, peer comparisons and later research, so the analyst must leave behind a file that can be maintained.
The report must read on its own, the workbook must let another analyst recompute, and the machine-readable summary must support system ingestion. All three serve one judgement while carrying the explanation, the calculation and the structured record.
NOTEEnglish translation of the original Chinese instruction; the itemised business questions and the deliverable field definitions are omitted.
03
Attachments
Everything visible inside the agent container, with statistics generated from the task package. Original files ship with the full sample package.
| File | Type | Size | Bytes |
|---|---|---|---|
| 增长资料_F.xlsxBusiness groups and revenue ledger | xlsx | 2 sheets | 7 KB |
| 增长资料_K.xlsxFinancial series and quarterly overview | xlsx | 2 sheets | 8 KB |
| 材料说明.mdMaterial note: period, currency and unit basis | md | 236 chars | 570 B |
04
Scoring design
16independent scoring criteria
15 scoring items1 penalty items
Full criteria text, weights and reference answers ship with the complete data package.
16 binary criteria, with four each on factual faithfulness and argument quality.
- 16 criteria: 15 positive scoring items and 1 penalty items (negate)
- Each criterion is judged separately by an agent judge (individual mode), reproducible and auditable
- Judge credentials are injected only into the verifier; the agent container has no scoring channel
- Full criteria text and reference answers ship with the complete data package
Argument quality4
Factual faithfulness4
Professional standards3
Instruction following2
Conclusion correctness2
Numerical accuracy1
05
Model results
Trajectory statistics from frontier models in real agent harnesses. One turn is one model call; the context curve shows how state accumulates over a long-horizon task.
| Model | Agent turns | Tool calls | Cumulative prompt tokens | Peak context | Time | Result |
|---|---|---|---|---|---|---|
| Claude Opus 4.8Claude Code · Harbor | 16 | 1.24M | 99K | — | Completed | |
| GPT-5.6-solCodex · Harbor | 28 | 822K | 52K | — | Completed | |
| Qwen3.8-MaxClaude Code · Harbor | 22 | 1.36M | 82K | — | Completed |
06
Specification
| Task ID | CLMR-FIN-FXCALC-005 |
|---|---|
| Task format | Harbor task.toml · schema 1.4 |
| Package version | 1.1.0 |
| Agent time limit | 3 h |
| Verifier time limit | 5 h |
| Agent container | 2 vCPU · 8 GB memory · 30 GB storage |
| Network | Only for agent harness installation |
| Tool set | filesystem · shell · python |
| Scoring isolation | Separate verifier containerJudge credentials are injected only into the verifier and are invisible to the agent |
| Instruction length | 4,384 characters |
| Difficulty | A2 (three-model mean 0.5702) |
| Reference solution | oracle reward 1.000 |
| Data source | Real sell-side research model (amounts scaled, entities de-identified) |
Contact
Want to run this task on your own platform?
Leave your details and we will send the full sample package for this task (instruction, attachments, verifier and reference trajectories) and walk you through volume and customisation for similar tasks.
- Full task package and verifier
- More tasks in this subdomain
- Integration and re-run support