Introduction and Motivations

Strong predictive performance is not enough when a model affects hiring, lending, medicine, or justice. In those settings we also want to know:

  • what factors drove the prediction,
  • whether the system behaves consistently,
  • whether it amplifies bias or hidden confounding,
  • and whether we can justify its use to stakeholders.

Explainability sits at that intersection between modeling and accountability.

At the most abstract level, a predictive model is a function

\[ f:\mathcal X \rightarrow \mathcal Y, \qquad x \mapsto \hat y. \]

Explainability asks for a second object in addition to the prediction itself: a human-readable account of why \(f(x)\) took the value it did, or how \(f\) behaves in general.

Your lecture frames explainability around three recurring motivations.

  • Trust and transparency: users are more likely to rely on a model when they understand its behavior.
  • Ethics and fairness: explanations help surface whether sensitive or proxy variables are shaping outcomes.
  • Regulatory pressure: in high-stakes domains, it is often not acceptable to answer, "the model just learned it."

This is why explainability is usually discussed alongside responsible AI rather than as a purely technical afterthought.

Setting What we usually need to justify
Lending Why an applicant was scored as risky
Hiring Whether the system is relying on biased proxies
Medicine Which signals pushed the prediction toward a diagnosis
Justice Whether the model behaves consistently and can be challenged

The lecture points to cases such as biased hiring tools and controversial criminal-risk systems. These examples matter because they show the practical cost of opaque models:

  • we may not notice unfair patterns early enough,
  • we may struggle to contest wrong predictions,
  • and we may deploy systems whose behavior is hard to audit after the fact.

Explainability does not automatically solve those problems, but it gives us tools to inspect them.

Many modern models are difficult to read directly because they combine:

  • many input variables,
  • non-linear transformations,
  • feature interactions,
  • and large numbers of learned parameters.

A model can therefore be very accurate while still being hard to debug or communicate. That tension is the starting point for the rest of this course.

A useful contrast is:

  • a transparent model lets us inspect rules, coefficients, or simple structure directly,
  • a black-box model may predict well, but its internal reasoning is harder to recover.

In this lesson we established:

  1. Why explainability matters beyond raw accuracy
  2. How trust, fairness, and compliance motivate interpretability work
  3. Why black-box models create practical risks in real applications
  4. Why explanation methods are now a core part of responsible ML

Next: We will organize the explainability landscape into a small number of useful categories.