Run once, reuse everywhere
medbench.kr evaluates medical AI models on open datasets from Seoul National University Hospital, once per model and task, and publishes the scores, predictions and exact versions so that nobody has to repeat the same experiment. AI agents can query everything through the AGENTS.md guide and the /v1 API.
- Tasks
- 2
- Models evaluated
- 10
- Results
- 12
- Reused (lookups)
- 0
Korean clinical knowledge (KMLE-level MCQ) preview
clinicalqa@0 · ClinicalQA: Korean Medical Knowledge QA Benchmark · text · primary metric: accuracy
| # | Model | Type | Tier | accuracy (95% CI) | n | Cost (USD) | Date |
|---|---|---|---|---|---|---|---|
| 1 | GLM-5.3 753B-A40B kgpu/glm-5.3-ax-753b-a40b |
llm | baseline | 0.909 (0.892–0.926) | 1045 | 0.0000 | 2026-10-29 |
| 2 | Gemini 3.8 Flash openrouter/google/gemini-3.8-flash |
llm | baseline | 0.908 (0.891–0.925) | 1045 | 2.3561 | 2026-10-29 |
| 3 | DeepSeek V4.1 Flash (2026-10-17) openrouter/deepseek/deepseek-v4.1-flash-1017 |
llm | baseline | 0.904 (0.887–0.922) | 1045 | 0.4731 | 2026-10-29 |
| 4 | DeepSeek V4.1 Flash 552B-A16B kgpu/deepseek-v4.1-flash-ax-552b-a16b |
llm | baseline | 0.900 (0.883–0.918) | 1045 | 0.0000 | 2026-10-29 |
| 5 | GPT-6 Luna openrouter/openai/gpt-6-luna |
llm | baseline | 0.900 (0.882–0.919) | 1045 | 0.1103 | 2026-10-29 |
Intraoperative hypotension 5 min ahead (numeric MAP) preview
vitaldb-ioh@0 · VitalDB · biosignal · primary metric: auroc
| # | Model | Type | Tier | auroc (95% CI) | n | Cost (USD) | Date |
|---|---|---|---|---|---|---|---|
| 1 | Gradient boosting on window summary features medbench/ioh-hgb-summary |
baseline | baseline | 0.910 (0.900–0.919) | 23106 | 0.0000 | 2026-10-29 |
| 2 | Logistic regression on last-minute MAP medbench/ioh-logreg-lastmap |
baseline | baseline | 0.852 (0.837–0.866) | 23106 | 0.0000 | 2026-10-29 |