Loading EMBERBench
EMBERBench
Can a decision model guide people to safety as a fire spreads?
Public development results. 19 worlds, 21 decision models and 8 reference agents; one trial per model. Models may train on this public set, so scores are not contamination-resistant.
Decision models
EMBER points out of 100, highest to lowest. EMBER combines safety and execution quality; it is not a survival percentage.
- typesafe/jev-1.14 51.70
- typesafe/jev-1.13 48.27
- nace-ai/drex-v2.0 32.66
- liquid/d1-ultra 32.39
- openai/gpt-6-luna-decisions 30.75
- nace-ai/drex-max 26.16
- perplexity/pplx-decider-v1.2 26.03
- nace-ai/drex-max-v1.1 25.26
- subq/subq-snap-preview 24.76
- respan/span-02 23.97
- cloudflare/clef 22.80
- microsoft/microsoft-decision-1 21.07
- sarvam/sarvam-prism 20.82
- inception/mercury-decide 20.68
- liquid/d1 19.94
- togethercomputer/tev1-4b-experimental 19.84
- perplexity/pplx-decider-flash-v1.2 18.87
- cloudflare/clef-flash 13.54
- cloudflare/clef-omni 11.77
- upstage/solar-decide-flash 0.21
- upstage/solar-decide 0.20
Reference agents
- cooperative-forecast-whca-r1 88.59
- forecast-shortest-path-r1 62.29
- congestion-aware-v1 53.68
- fire-aware-shortest-path-v1 40.81
- static-shortest-path-v1 36.10
- nearest-exit-greedy-v1 26.20
- random-legal-v1 0.00
- always-stay 0.00
Interactive charts and filters become available when JavaScript and the results file load. You can read these results without them. Download public results as JSON or read the methodology.