From Residuals to Reasons: LLM-Guided Mechanism Inference from Tabular Data
The paper introduces MARICL, a multi-agent LLM framework that learns what a base model misses on tabular data and improves predictions across nine benchmarks.
Intelligence analysis by GPT-5.4 Mini
Rather than asking an LLM to solve a tabular prediction task from scratch, the paper anchors it to a base model and has agents explain the residuals. The resulting correction terms improve performance across nine benchmarks and appear to capture real biochemical structure in one test.
A computer model can make a good guess and still not explain why it was wrong. This paper teaches another system to look at those mistakes, like a detective checking clues after the first guess.
It is a bit like fixing a bike by first seeing where it slips, instead of rebuilding the whole bike from scratch. The new method writes down the missing rules it thinks the first model forgot.
The authors say this helped on many kinds of data, and in one biology test the fixes worked only when the real lab setup stayed the same. That suggests the system may have found a real pattern, not just a lucky trick.
Analysis
What the paper proposes
The paper starts from a familiar problem in scientific machine learning: strong prediction often comes without a satisfying explanation. Traditional interpretability tools can point to influential variables, but they usually stop short of describing how those variables combine into a mechanism.
The authors propose a different workflow. Instead of asking an LLM to predict the target directly, they first use a base model and then ask the LLM to focus on the errors the base model leaves behind. That residual-first setup narrows the search space and turns the model into a partner for refinement rather than a standalone predictor.
MARICL
The method is called Multi-Agent Residual In-Context Learning, or MARICL. In the paper’s description, multiple LLM agents inspect high-error examples, infer missing structure from those cases, and then write explicit correction terms. Those corrections are refined through multiple text-based optimization rounds.
Across nine benchmarks spanning scientific, biomedical, socioeconomic, and synthetic data, the paper says MARICL consistently improves on the underlying base model. The claim is not only that the system predicts better, but that it does so by generating reusable structure rather than ad hoc fixes.
Mechanistic check
The strongest evidence the abstract highlights is a transfer test on the Cell-Free Protein dataset. The authors freeze formulas learned on one experimental batch and apply them to held-out batches without retraining or any further LLM calls. Within the same reagent protocol, the frozen formulas improve predictions in more than 92% of cases. When moved to a different protocol, they fail systematically.
That boundary matters because it suggests the learned corrections track the underlying biochemistry rather than batch noise. The paper frames this as direct evidence of mechanistic generalization, not just a better fit to one slice of data.
Key points
- The paper focuses on turning prediction errors into explanations, not just feature rankings.
- MARICL uses LLM agents to study residuals from a base model and propose correction terms.
- The method improved the base model across nine benchmarks in scientific, biomedical, socioeconomic, and synthetic settings.
- A transfer test on the Cell-Free Protein dataset suggested the learned formulas captured real protocol-specific structure.
- The authors present the results as evidence of mechanistic generalization rather than batch-specific noise.
