Public results and the official test

What this site shows

Most results on this site come from the public development suite. Models may train or tune on it, so the scores are reproducible but not contamination-resistant.

The official test

The official EMBERBench test is private and runs separately. This site publishes only its totals, on the results page. It does not publish its scenarios, seeds, maps, episode results or how it is made. No submission service is offered.

What is public

The observation and action interface, simulator rules, public training generator, public scenarios, reference agents, command line, metrics, EMBER Score, statistics, replay verification and fairness contract are public. The scoring specification gives the weights, gates and failure handling.

Official scoring is not the official test

A result that uses the official EMBER formula is not, for that reason, an official test result. Public development results can correctly use the official score.