Separating signal from noise in coding evaluations

OpenAI says a new analysis points to reliability and accuracy issues in SWE-Bench Pro, a widely used coding benchmark. The findings raise questions about how well current evaluations measure real model performance and whether benchmark results are as stable as they appear.

Read the original source ↗

← back to the front page