Guide

How to audit a credit-scoring model

A sequence the lab uses. It is a method note, not a certificate and not a client case study.

Most "AI audits" start with a framework and never reach the book. A credit-model audit starts with a claim: who is scored, what decision is taken, and what "good" means in that market.

1. Write the claim down

Name the population (new-to-credit retail in Kenya; salaried UAE residents; SME overdraft). Name the decision (approve, price, limit). Name the threshold. If the owner cannot state this, the audit stops at a model card and says so.

2. Match the data to the claim

The test set has to look like the book the model will see. Western bureau samples do not stand in for thin-file African or GCC books. If the right data cannot be shared, say what cannot be tested.

3. Bias

Compare approval or score distributions across groups that exist in that market: nationality, gender, language, neighbourhood, employer type. Report the gap, the sample size, and the test. Do not collapse this into a single "fairness score".

4. Stability

Split by time window and by a second population slice. A model that only holds on last year's originations is not ready for a second country or a downturn.

5. Explainability

Pick a handful of real decisions and reconstruct them in terms a credit committee can read. If the story depends on a feature the bank is not allowed to use, that is the finding.

6. Write it so a second person can rerun it

Code, seeds, and the exact slices. SAFE Labs will not put its name on a result a second reader has not reproduced. See independence.

Credit-model toolkit · AI assurance · Request a review