BioSimBench is a benchmark evaluating language models on real laboratory experiments by testing their ability to predict gene expression changes in cells responding to drugs, gene knockdowns, and immune signals. It features five tracks with a public leaderboard; frontier models recover only part of measured responses, while a 27B model trained in CellType's BioSimGym outperforms them on held-out data from multiple labs.
BioSimBench evaluates how well language models predict changes in gene expression after drugs, gene knockdowns, and immune signals, scoring predictions against measured laboratory responses. Its five tracks cover held-out drugs and cell contexts, gene knockdowns, drug-response conditions, immune signals, and tool-using research agents. CellType reports prediction-track results separately from the research-agent track and describes its scoring and evaluation setup on the benchmark page. The benchmark is paired with BioSimGym, CellType’s training environments for biological prediction tasks.
