Does a learned score
beat a constant baseline?
UCI Statlog German Credit: 1,000 historical records, 17 of 20 attributes used, 700 good and 300 bad credit-risk labels. Three fixed models share five stratified 800/200 train/test splits. All five splits and all 3,000 test scores are retained.
Cells show the mean and observed minimum–maximum across the five splits. Brier is lower-is-better; the other columns are higher-is-better. The ranges are descriptive, not confidence intervals.
| Model | ROC-AUC | Average precision | Brier ↓ | Top-10% precision | Bad-label capture |
|---|---|---|---|---|---|
| constant | 0.500 (0.500–0.500) | 0.300 (0.300–0.300) | 0.210 (0.210–0.210) | 0.230 (0.150–0.300) | 0.077 (0.050–0.100) |
| logistic | 0.767 (0.732–0.822) | 0.611 (0.553–0.719) | 0.171 (0.147–0.186) | 0.740 (0.650–0.850) | 0.247 (0.217–0.283) |
| hist gradient boosting | 0.771 (0.728–0.844) | 0.614 (0.545–0.729) | 0.169 (0.146–0.182) | 0.730 (0.650–0.850) | 0.243 (0.217–0.283) |
Each test set contains 200 rows and 60 bad labels. Every method receives exactly 20 inspection slots; score ties use the original source row number. The constant model's top-20 subset is an arbitrary tie break, not a learned ranking.
Run it and check it
From the source checkout after installing the project:
python examples/german_credit.py --out outputs/public-credit-comparison
python examples/german_credit.py --verify outputs/public-credit-comparisonAge, personal status/sex and foreign-worker status are excluded, matching the package calibration example. This does not establish fairness or remove proxies.
Preprocessing is fitted on training rows only. Verification recomputes metrics and the top-20 selections from saved scores, checks labels against the frozen source and checks every split. It does not retrain models or authenticate authorship.
This dataset has no decision or outcome-availability timestamps. This experiment cannot test point-in-time safety, 180-day 90+ DPD prediction, future-vintage performance or intervention benefits. It is separate from the synthetic temporal and policy experiments.
Source: Hans Hofmann (1994), Statlog (German Credit Data), UCI Machine Learning Repository, CC BY 4.0. Original data bytes and codebook are preserved; label 2 is mapped to positive class 1 for evaluation.