The article introduces Systema, a new evaluation framework designed to improve the prediction of transcriptional responses to genetic perturbations in functional genomics. The authors argue that existing computational methods for predicting these responses are often misleadingly successful because they predominantly capture systematic variation-consistent transcriptional differences between perturbed and control cells-rather than perturbation-specific effects. To address this overestimation of performance, Systema shifts the point of reference from the control cell centroid to the perturbed cell centroid, making evaluation more robust against these systemic biases. Using Systema, the research shows that predicting the effects of unseen genetic perturbations is more difficult than previously indicated by standard metrics, though some models, like fine-tuned scGPT, demonstrate a limited ability to infer coarse-grained, biologically meaningful perturbation effects. This work highlights the need for heterogeneous gene panels and reference-independent metrics to accurately assess the predictive power and biological utility of perturbation response models.

References:

• Viñas Torné R, Wiatrak M, Piran Z, et al. Systema: a framework for evaluating genetic perturbation response prediction beyond systematic variation[J]. Nature Biotechnology, 2025: 1-10.