Superhuman
DeepMind datasets and benchmarks for olympiad-level math reasoning in AI
What it is
A collection of datasets and research releases from Google DeepMind's Superhuman Reasoning team. It includes IMO Bench, with 400 short-answer olympiad problems, 60 expert-vetted proof problems, and 1000 human gradings for testing automatic proof evaluation, along with prompts and outputs from the Aletheia math research agent and pointers to the AlphaGeometry projects. AI researchers use it to measure how well language models solve and grade competition and research-level mathematics.