The pooled rate cannot stand in for any cohort
Each cohort has its own denominator and its own expand-or-stop rule. A single pooled 25% hides exactly the difference the decision turns on.
Basket Trial Intake Review Across Tumour Types
MED-HER2-BASKET-REVIEW-187 · HER2 basket · week-8 objective response · per-cohort verdict
The previous shift pooled four tumour types into one basket and reported a 25% overall response rate from 40 patients, recommending one expanded cohort. With 12 source tables, rebuild each cohort's denominator and decide whether the basket can advance as a whole.
Clinical researchBasket trialObjective responseCounting rules
1212 files
16Kcharacters of material
7scoring criteria
21agent turns in the longest reference rollout
948Kcumulative prompt tokens in one rollout
01
Highlights
Define the analysis set before counting
Central confirmation, tumour cell fraction, scan window, measurable target lesions at both timepoints, and at least one dose — five rules in the shift sheet, and missing one shifts the denominator.
In-vitro evidence is not a patient scan
The sensitivity plate measures inhibition in an overexpression well and the culture check validates the covalent-binding model. Neither substitutes for the patient's week-8 imaging.
Three-model mean score 0.4752
Three models answered under the same criteria for a mean of 0.4752. Seven criteria, five positive and two penalties, both aimed at signing off the pooled basket.
02
Instruction excerpt
An excerpt of the instruction the agent actually receives, showing the task setup and constraints.
instruction.md · 1,278 chars
You work in the core lab of a molecular oncology platform, handling intake review for a basket trial. The previous shift left a one-page basket note: of 40 patients with locally reported HER2 or HER3 alterations, 10 showed an objective response at week 8 — a 25% rate — so the next window should expand one basket of HER2/HER3-mutant solid tumours.
Your job is to review whether that page can go into the next window as written. The folder holds every source table for these patients: enrolment log, local variant reports, central confirmation, baseline and week-8 target lesions, metabolic uptake, dosing records, the sensitivity plate, the culture check, and the shift sheet stating this window's review rules.
Rules
- Judge the analysis set and objective response by the shift sheet; do not change the rules.
- Compute response per cohort; a pooled rate cannot stand in for any single cohort.
- The plate and the culture check are in-vitro evidence: keep them separate from the patient's week-8 scan.
This window covers intake and basket assignment only — not individual prescribing advice.
NOTEEnglish translation of the original Chinese instruction; the material table and deliverable fields are omitted.
03
Attachments
Everything visible inside the agent container, with statistics generated from the task package. Original files ship with the full sample package.
| File | Type | Size | Bytes |
|---|---|---|---|
| 中心复核.csvCentral protein alteration, tumour cell fraction, non-synonymous burden | csv | 48 rows | 2 KB |
| 代谢摄取.csvPET metabolic uptake | csv | 9 rows | 307 B |
| 值班夹页.txtThis window's rules: analysis set, objective response, expand/stop | txt | 468 chars | 1 KB |
| 入组流水.csvOne row per patient: ID, tumour type, enrolment date, in or out of the shift cutoff | csv | 48 rows | 2 KB |
| 培养核验.csvCulture and covalent-binding model check | csv | 105 rows | 2 KB |
| 基线靶灶.csvBaseline longest diameter of target lesions (CT / MR) | csv | 45 rows | 1 KB |
| 夜班并篮稿.mdThe previous shift's basket note | md | 319 chars | 675 B |
| 夹页目录.txtIndex of the 11 folder items | txt | 131 chars | 229 B |
| 服药记录.csvDosing record | csv | 48 rows | 1 KB |
| 本地变异.csvLocal lab gene and protein alterations | csv | 48 rows | 2 KB |
| 第八周靶灶.csvWeek-8 longest diameter and scan date | csv | 44 rows | 2 KB |
| 药敏板.csvIn-vitro sensitivity plate readings | csv | 105 rows | 6 KB |
04
Scoring design
7independent scoring criteria
5 scoring items2 penalty items
Full criteria text, weights and reference answers ship with the complete data package.
7 criteria: 4 five-level plus 3 binary, of which 2 are penalty items.
- 7 criteria: 5 positive scoring items and 2 penalty items (negate)
- Each criterion is judged separately by an agent judge (individual mode), reproducible and auditable
- Judge credentials are injected only into the verifier; the agent container has no scoring channel
- Full criteria text and reference answers ship with the complete data package
Professional standards2
Conclusion correctness1
Argument quality1
Factual faithfulness1
Instruction following1
Numerical accuracy1
05
Model results
Trajectory statistics from frontier models in real agent harnesses. One turn is one model call; the context curve shows how state accumulates over a long-horizon task.
| Model | Agent turns | Tool calls | Cumulative prompt tokens | Peak context | Time | Result |
|---|---|---|---|---|---|---|
| GPT-5.6-solCodex · Harbor | 16 | 948K | — | 4m 27s | Completed | |
| Claude Opus 4.8Claude Code · Harbor | 27 | — | — | 5m 45s | Completed | |
| Qwen3.8-MaxClaude Code · Harbor | 22 | — | — | 2m 33s | Completed |
06
Specification
| Task ID | MED-HER2-BASKET-REVIEW-187 |
|---|---|
| Task format | Harbor task.toml · schema 1.4 |
| Package version | 1.0.0 |
| Agent time limit | 20 h |
| Verifier time limit | 5 h |
| Agent container | 2 vCPU · 8 GB memory · 30 GB storage |
| Network | Only for agent harness installation |
| Tool set | filesystem · shell · python |
| Scoring isolation | Verifier runs as a separate stageJudge credentials are injected only into the verifier and are invisible to the agent |
| Instruction length | 1,278 characters |
| Difficulty | A3 (three-model mean 0.4752) |
| Reference solution | all gates 37/37, reward 1.000 |
| Material size | 12 source tables · 48 patients |
Contact
Want to run this task on your own platform?
Leave your details and we will send the full sample package for this task (instruction, attachments, verifier and reference trajectories) and walk you through volume and customisation for similar tasks.
- Full task package and verifier
- More tasks in this subdomain
- Integration and re-run support