What is explainable AI in drug discovery?

Explainable AI, or XAI, refers to methods that reveal why an AI model made a specific prediction, such as which molecular substructure drove a predicted activity or toxicity score, rather than returning only a number. In drug discovery, this matters because a score alone gives a chemist nothing actionable to test, modify or dispute using their own expertise.

Why is AI explainability important in pharma?

Explainability lets a black-box prediction function as a testable hypothesis rather than an unquestionable verdict, which is how drug discovery has always operated through structure-activity relationships. It also matters for regulatory purposes, since FDA and EMA both favor evidence that can be examined and questioned over an unexplained model output, especially as that output carries more weight in a decision.

What is SHAP in drug models?

SHAP, SHapley Additive exPlanations, is a game-theory-based method that assigns each input feature, such as a specific atom or molecular descriptor, a fair share of credit for a model's prediction. In drug discovery, it is commonly applied to identify which substructures of a molecule are driving a predicted activity, toxicity or ADMET property.

How do regulators view black-box AI?

Regulators do not categorically reject black-box models, but they scale explainability expectations to how much a model's output influences a regulatory decision. A model used only to help prioritize candidates for synthesis faces lighter expectations than one whose output directly supports a safety or efficacy claim, where traceable, examinable evidence is strongly preferred.

Explainable AI in Drug Discovery | Drug Discovery News

Highlights in this article

Why explainability matters
The black-box problem in practice
Main XAI approaches: SHAP, LIME, attention maps
What XAI means for SAR
Regulatory expectations
The frontier: mechanistically interpretable models
Key takeaways

A medicinal chemist cannot act on a number alone. Told that a candidate molecule scores 0.87 on some predicted-activity scale, the obvious next question—what about this molecule is driving that score, and what would happen if I changed this ring or that substituent—goes entirely unanswered by the score itself. Explainable AI drug discovery work exists precisely to close that gap. Artificial intelligence (AI) models built for molecular property prediction are frequently deep neural networks whose internal reasoning is not transparent by default, and for a field built around iterative, hypothesis-driven design, an opaque prediction is only marginally more useful than no prediction at all.

This connects to the regulatory guide to AI in drug discovery, which touches on explainability as a regulatory concern across FDA, EMA, and ICH; this piece stays closer to the bench, working through why the problem is real, what tools currently exist to address it, and how far those tools actually get a chemist. It also connects to the guide to AI across drug discovery for the wider technical picture.

Why explainability matters

Drug discovery has always run on structure-activity relationships, the working theory a chemist builds, molecule by molecule, about which parts of a compound drive its potency, selectivity, or toxicity. That theory is what actually gets acted on: it tells a chemist which position to substitute next, which scaffold to abandon, which liability to design around. A model that only outputs a probability or a score, with no indication of which structural features produced it, cannot feed that process directly. It can rank candidates, but it cannot teach a chemist anything transferable to the next molecule.

Explainability is what allows a prediction to function as a hypothesis rather than a verdict. A model that flags a specific ring system or functional group as the likely driver of a toxicity prediction gives a chemist something to test, modify or dispute, using their own expertise, in a way a bare score never does.

The black-box problem in practice

Deep neural networks, and particularly the graph neural networks (GNNs) now common for molecular property prediction, achieve strong accuracy in part because they learn their own internal representations of chemical structure rather than relying on hand-picked descriptors. That same flexibility is what makes them opaque: nothing guarantees the patterns a model has actually learned correspond to chemically meaningful concepts at all, rather than incidental correlations, an overrepresented scaffold in the training set, or a batch effect from a particular assay run that happens to predict the outcome without reflecting genuine structure-activity relationships.

The tools built to explain these models carry their own, less discussed limitation: a 2025 review in WIREs Computational Molecular Science notes that model-agnostic explanation methods trade faithfulness for generality, meaning an explanation can look chemically plausible while not actually reflecting what the underlying model is doing internally. A widely cited 2020 study went further, showing that post-hoc explanation methods can be deliberately manipulated to produce misleading explanations for a model's behavior. None of this makes explainability tools useless, but it does mean an explanation should be treated as a testable hypothesis about the model, not a verified fact about it.

Main XAI approaches: SHAP, LIME, attention maps

A handful of techniques account for most explainability work applied to molecular models today, each answering a slightly different version of the same question.

Method	How it works	Typical output
SHAP	Game-theory-based method that assigns each input feature a fair share of credit for a specific prediction	Per-atom or per-descriptor contribution scores
LIME	Builds a simple, locally faithful surrogate model around one specific prediction to approximate the complex model's behavior nearby	A short list of features driving that one prediction
Attention maps	Surfaces the internal attention weights a graph or transformer model already assigns to specific atoms or bonds while making a prediction	A heat-map-style highlight over the molecular structure
GNNExplainer and structure-aware explainers	Identify the smallest subgraph of a molecule sufficient to reproduce a GNN's prediction	A highlighted substructure specific to graph models
Counterfactual explanations	Generate a structurally similar molecule predicted to have the opposite outcome	A proposed alternative structure, not just a highlighted region

Counterfactual explanations are worth singling out because they have produced one of the field's clearer examples of the loop actually closing. Researchers using a graph attention model to flag molecules likely to aggregate in assays, a common and costly false-positive source in screening, generated a counterfactual for a known aggregator, clioquinol, predicting a close structural analog that would not aggregate. That proposed analog was synthesized and experimentally confirmed to behave as predicted, turning a model explanation into an actual, physically verified design decision rather than a plausible-looking diagram.

What XAI means for SAR

Classical structure-activity relationship work is inherently slow: a chemist proposes a modification, waits for synthesis and assay results, and updates their mental model accordingly, one compound at a time. Substructure-attribution methods attempt to compress a version of that same feedback loop into something that runs against an entire virtual library at once, surfacing which structural features a model associates with an outcome before a single additional compound is made.

The realistic framing is that this generates hypotheses faster, not answers. An attribution map or a counterfactual structure is a candidate explanation worth testing against a chemist's own domain knowledge and, where it matters, against actual synthesis and assay data, exactly as happened with the clioquinol analog. Treated that way, XAI functions as an accelerant for traditional SAR reasoning rather than a replacement for the experimental step that has always validated it.

Regulatory expectations

Both FDA and EMA treat explainability as a sliding-scale expectation rather than a fixed bar, tied to how much a given model's output actually influences a regulatory decision. A model used only to help a chemist prioritize which compounds to synthesize next faces a lighter explainability expectation than one whose output feeds directly into a safety or efficacy claim; that scaling logic is discussed in more depth in the companion piece on FDA's AI framework. What both agencies converge on, whatever the specific bar, is a preference for evidence that can be examined and questioned over an assertion that a model simply performed well on a validation set.

For a discovery-stage tool that never touches a regulatory submission directly, this may seem like someone else's problem. It rarely stays that way: a compound identified with the help of an unexplainable model can still end up, years later, supported by nonclinical or clinical evidence that traces back to that original, undocumented discovery-stage decision. Building explainability and documentation habits before a regulator asks for them costs less than reconstructing that history retroactively.

The frontier: mechanistically interpretable models

The methods discussed so far are mostly post-hoc: they explain a black-box model after the fact. A parallel, earlier-stage line of work tries instead to build models that are interpretable by construction, so there is less need to reverse-engineer an explanation later.

Physics-informed approaches are the most mature version of this idea. Schrödinger's free-energy perturbation platform, deployed with AstraZeneca since 2019, predicts binding affinity by simulating the actual thermodynamics of a ligand binding to its target, rather than learning a purely statistical mapping from structure to affinity. Because the calculation is grounded in physical law rather than pattern-matching alone, a chemist can trace a poorly predicted affinity back to a specific, physically meaningful interaction, not just a number the model associates with similar-looking molecules. AstraZeneca's own Biological Insights Knowledge Graph takes a related approach at the systems level: rather than an opaque learned embedding, target and disease relationships are represented as an explicit, traceable graph of more than 14 million entities and 130 million-plus edges, so a given prediction can be followed back through the specific relationships that produced it.

Newer academic work pushes further still: architectures built with named chemical concepts baked directly into their structure, so a prediction is expressed in terms of those concepts rather than requiring a separate explanation layer bolted on afterward, and large language models fine-tuned to produce a written chain of reasoning alongside a prediction rather than a bare score. Both directions remain earlier-stage than the post-hoc tools in wide use today, and both carry a familiar tradeoff: building interpretability into a model's architecture tends to constrain it, often trading away some raw predictive flexibility for the ability to explain itself honestly.


Key takeaways A prediction score alone gives a chemist nothing to act on; explainability is what turns a score into a testable design hypothesis. Common XAI tools—SHAP, LIME, attention maps, GNNExplainer, and counterfactuals—each answer a different version of "why," but post-hoc explanations can look plausible without faithfully reflecting a model's actual internal reasoning. The clearest validation of an XAI method so far came from actually synthesizing a model-proposed counterfactual molecule and confirming its predicted behavior in the lab. Regulators scale explainability expectations to how much a model's output influences a decision; the frontier is shifting toward models, physics-grounded or concept-based, that are interpretable by design rather than explained after the fact.

Key takeaways

A prediction score alone gives a chemist nothing to act on; explainability is what turns a score into a testable design hypothesis.
Common XAI tools—SHAP, LIME, attention maps, GNNExplainer, and counterfactuals—each answer a different version of "why," but post-hoc explanations can look plausible without faithfully reflecting a model's actual internal reasoning.
The clearest validation of an XAI method so far came from actually synthesizing a model-proposed counterfactual molecule and confirming its predicted behavior in the lab.
Regulators scale explainability expectations to how much a model's output influences a decision; the frontier is shifting toward models, physics-grounded or concept-based, that are interpretable by design rather than explained after the fact.

This article was produced in accordance with Drug Discovery News’ AI Editorial Policies.