Positivity rates are not mutually exclusive
One patient can score above several signatures at once, so adding the three rates counts the same person several times.
Endotype Prevalence Claim Review in Sepsis
MED-SEPSIS-ENDOT-REVIEW-212 · Overlapping signatures · mutually exclusive rates · window sizing
The draft adds three signature positivity rates into one 'mutually exclusive high-risk endotype' prevalence of 94.7% and sizes the sample from it. From eight source files, find the double counting, the cut-point basis and the pooled cohorts, rewrite the claim and size the window.
Clinical researchEndotypingPrevalenceCounting rules
88 files
52Kcharacters of material
5scoring criteria
16agent turns in the longest reference rollout
300Kcumulative prompt tokens in one rollout
01
Highlights
Samples are not people
The registry is one row per tube and includes baseline, day-3 repeat and technical re-runs. Counting tubes inflates the denominator with repeats.
Cohorts cannot be pooled
Emergency, ward and ICU are three sampling layers; a rate pooled across them is not the ICU window rate.
Three-model mean score 0.5707
Three models answered under the same criteria for a mean of 0.5707. Only five criteria, so each one carries outsized weight.
02
Instruction excerpt
An excerpt of the instruction the agent actually receives, showing the task setup and constraints.
instruction.md · 1,124 chars
You work on a critical-care transcriptomics platform. A multicentre sepsis cohort is endotyped with three deleterious gene signatures. This Friday the 'endotype prevalence' paragraph goes into the results section for the consortium.
The group already has a draft: it adds the three signature positivity rates together, reports 'three mutually exclusive high-risk endotypes, pooled prevalence 94.7%', and recommends sizing this centre's ICU window from that ratio.
Decide whether the paragraph can go in as written. If not, rewrite it into a publishable form using the material in the folder, and state how many samples this centre's ICU window should prepare. Give a disposition, not 'needs more review'.
Rules
- Counting rules follow the group note; the sample registry and the donor roster are at different grains, so decide which grain to count at.
- Cut points follow the platform's healthy controls; judge whether the generic absolute cut points apply and say why.
NOTEEnglish translation of the original Chinese instruction; the material table and deliverable fields are omitted.
03
Attachments
Everything visible inside the agent container, with statistics generated from the task package. Original files ship with the full sample package.
| File | Type | Size | Bytes |
|---|---|---|---|
| 三套高危初算.mdSplit table built on the generic absolute cut points | md | 368 chars | 688 B |
| 供体名册.csvOne row per donor: centre, sampling layer, infection category, admission severity | csv | 340 rows | 17 KB |
| 健康对照分位.csvPlatform healthy-control means and standard deviations | csv | 2 rows | 166 B |
| 内型患病率初稿.mdThe draft prevalence paragraph | md | 429 chars | 915 B |
| 夹页.txtIndex and provenance of the eight files | txt | 266 chars | 628 B |
| 样本登记.csvOne row per tube: sample ID, donor ID, centre, draw type, study day | csv | 394 rows | 21 KB |
| 签名连续分.csvContinuous signature scores, unclassified and unstandardised | csv | 394 rows | 26 KB |
| 组里便条.txtGroup rules: per person vs per tube, cut points, multiple signatures, which cohort | txt | 278 chars | 762 B |
04
Scoring design
5independent scoring criteria
4 scoring items1 penalty items
Full criteria text, weights and reference answers ship with the complete data package.
5 criteria: 3 binary plus 2 five-level, of which 1 is a penalty item.
- 5 criteria: 4 positive scoring items and 1 penalty items (negate)
- Each criterion is judged separately by an agent judge (individual mode), reproducible and auditable
- Judge credentials are injected only into the verifier; the agent container has no scoring channel
- Full criteria text and reference answers ship with the complete data package
Conclusion correctness2
Instruction following1
Argument quality1
Factual faithfulness1
05
Model results
Trajectory statistics from frontier models in real agent harnesses. One turn is one model call; the context curve shows how state accumulates over a long-horizon task.
| Model | Agent turns | Tool calls | Cumulative prompt tokens | Peak context | Time | Result |
|---|---|---|---|---|---|---|
| GPT-5.6-solCodex · Harbor | 7 | 300K | — | 2m 34s | Completed | |
| Claude Opus 4.8Claude Code · Harbor | 23 | — | — | 4m 15s | Completed | |
| Qwen3.8-MaxClaude Code · Harbor | 21 | — | — | 2m 44s | Completed |
06
Specification
| Task ID | MED-SEPSIS-ENDOT-REVIEW-212 |
|---|---|
| Task format | Harbor task.toml · schema 1.4 |
| Package version | 1.0.0 |
| Agent time limit | 20 h |
| Verifier time limit | 5 h |
| Agent container | 2 vCPU · 8 GB memory · 30 GB storage |
| Network | Only for agent harness installation |
| Tool set | filesystem · shell · python |
| Scoring isolation | Verifier runs as a separate stageJudge credentials are injected only into the verifier and are invisible to the agent |
| Instruction length | 1,124 characters |
| Difficulty | A2 (three-model mean 0.5707) |
| Reference solution | all gates 33/33, reward 1.000 |
| Material size | 8 files · 394 samples from 340 donors |
Contact
Want to run this task on your own platform?
Leave your details and we will send the full sample package for this task (instruction, attachments, verifier and reference trajectories) and walk you through volume and customisation for similar tasks.
- Full task package and verifier
- More tasks in this subdomain
- Integration and re-run support