Constant baseline
7 / 89 recorded events
- Precision
- 0.70%
- Event capture
- 7.87%
82 events remain outside this review list.
2-minute case / observed SBA loans
Same review budget.
Different results.
Do better probabilities produce a better review list? Compare raw and independently calibrated scores on the same later loans in a 35,000-loan SBA study. Keep the weaker results visible.
10,000 held-out loans89 recorded charge-offs24-month outcome horizonNo sign-in or installation
01 / keep the workload equal
Give each method the same review slots
Current comparison: each method reviews 1,000 / 10,000 loans.
Raw Logistic
26 / 89 recorded events
- Precision
- 2.60%
- Event capture
- 29.21%
63 events remain outside this review list.
Independent isotonic
24 / 89 recorded events
- Precision
- 2.40%
- Event capture
- 26.97%
65 events remain outside this review list.
Raw Logistic captures 26 events: precision is 26 / 1,000 = 2.60%; event capture is 26 / 89 = 29.21%. The denominators answer different questions.
These are historical event concentrations, not prevented losses. All three capacities were fixed in the research protocol. Switching reads verified results; it does not refit a model or choose a deployment threshold. Constant-score ties use record ID, so that list is an arbitrary ordering.
02 / probability ≠ ranking
Calibration can lower error.
Ranking can still get worse.
| Fixed method | AUC ↑ | Brier ↓ |
|---|---|---|
| Constant baseline | 0.5000 | 0.009838 |
| Raw Logistic | 0.6723 | 0.009011 |
| Independent isotonic | 0.6584 | 0.008819 |
Brier measures squared probability error; AUC measures ordering between positive and negative records. Isotonic calibration can introduce tied scores, so probability error, ranking and fixed-budget capture need separate checks. There are only 89 test events: a single difference is no future guarantee. The full report retains paired funding-month bootstrap intervals.
03 / dates before models
Let outcomes mature
before the next stage.
- Train: 20,000 loans funded in 2006–2010. Fit preprocessing and Logistic only on training loans. The latest 24-month horizon ends in 2012.
- Calibrate separately: 5,000 loans funded in 2013. Only these labels fit isotonic calibration. Their latest horizon ends in 2015.
- Test: 10,000 loans funded in 2016–2017. Evaluate fixed models and capacities. Test outcomes do not tune features or hyperparameters.
The final public snapshot has actual funding dates but no historical first-publication times or feature revisions. This is retrospective funding-cohort validation. It cannot certify which features were available at past decisions or directly represent current Chinese retail credit.
See all 9 fixed-capacity comparisons
| Method | Capacity | Reviewed | Captured / all events | Precision | Event capture |
|---|---|---|---|---|---|
| Constant baseline | 5% | 500 | 4 / 89 | 0.80% | 4.49% |
| Constant baseline | 10% | 1,000 | 7 / 89 | 0.70% | 7.87% |
| Constant baseline | 20% | 2,000 | 16 / 89 | 0.80% | 17.98% |
| Raw Logistic | 5% | 500 | 14 / 89 | 2.80% | 15.73% |
| Raw Logistic | 10% | 1,000 | 26 / 89 | 2.60% | 29.21% |
| Raw Logistic | 20% | 2,000 | 38 / 89 | 1.90% | 42.70% |
| Independent isotonic | 5% | 500 | 17 / 89 | 3.40% | 19.10% |
| Independent isotonic | 10% | 1,000 | 24 / 89 | 2.40% | 26.97% |
| Independent isotonic | 20% | 2,000 | 37 / 89 | 1.85% | 41.57% |
Scroll the table horizontally on a small screen. Buttons and table share the same refitted, verified evidence. No new training or selection occurs here.
04 / follow a number back to its source
Follow one number through the project.
Fixed-capacity CSV → 10,000 test predictions → Training and verification code → Frozen protocol. This page contains aggregates only. Inspect the brief data and source digests.
Use Python 3.12+ in a virtual environment. Installation needs internet; the published v0.8.1 study then refits offline. The current website build generates this reader brief.
python -m pip install https://github.com/dev-belly/CreditVintage/releases/download/v0.8.1/creditvintage_research-0.8.1-py3-none-any.whl python -m creditvintage sba-benchmark --out outputs/sba-credit python -m creditvintage verify-sba outputs/sba-credit
Choose a new output directory. Successful verification reports status: verified and refitted: true: internal consistency under this implementation, not independent adoption or business benefit.