CreditVintage / observed SBA lending
Observed loans.
Separated by time.
A frozen funding-cohort study of 35,000 US small-business loans. Learn on earlier loans, calibrate later, then evaluate on 2016–2017 funding cohorts.
available_at timestamps.01 / dates before models
Three cohorts. Mature outcomes between stages.
The 24-month outcome window ends before the next stage starts. One earliest eligible loan per hashed recorded borrower group; no group appears twice. Fixed sizes and hash-based sampling do not balance outcomes.
| Stage | Loans | Events | Rate | Funded | Last horizon | Missing initial rate |
|---|---|---|---|---|---|---|
| train | 20,000 | 816 | 4.080% | 2006-01-05 → 2010-12-30 | 2012-12-30 | 15250 |
| calibration | 5,000 | 42 | 0.840% | 2013-01-01 → 2013-12-31 | 2015-12-31 | 15 |
| test | 10,000 | 89 | 0.890% | 2016-01-01 → 2017-12-31 | 2019-12-31 | 0 |
Wide tables scroll horizontally on small screens. Group keys use normalized recorded name/state/ZIP. Spelling and address changes can evade this heuristic. Large initial-rate missingness in earlier cohorts is retained and disclosed.
02 / fixed protocol, retained results
Calibration and ranking answer different questions.
Constant train rate, logistic regression and separate isotonic calibration use the same test loans. Lower Brier is better; higher AUC and average precision are better. Calibration can improve Brier while reducing ranking performance.
| Method | ROC AUC | Average precision | Brier score |
|---|---|---|---|
| constant | 0.50000095% 0.500000 – 0.500000 | 0.00890095% 0.007162 – 0.010643 | 0.00983895% 0.008242 – 0.011439 |
| logistic | 0.67234195% 0.624625 – 0.717464 | 0.01847095% 0.013555 – 0.026113 | 0.00901195% 0.007365 – 0.010662 |
| isotonic | 0.65839995% 0.604911 – 0.712604 | 0.01666195% 0.012190 – 0.022133 | 0.00881995% 0.007115 – 0.010509 |
95% percentile intervals: 300 paired resamples of the 24 test funding months. Models remain fixed. Intervals exclude source revision, training, parameter and sample-selection uncertainty; one-class replicates remain in the evidence with unavailable metrics and valid counts.
What the model actually sees
Log approval amount, SBA guaranteed share, initial interest rate and approval processing method. Median imputation with missing indicators, scaling and category encoding are learned from training loans only. No dates, cohort IDs, borrower identities, final status, current term or outcome fields enter the model.
03 / comparable workload
Hold the review budget fixed.
Each method ranks the same 10,000 test loans; ties use record ID. Captured events describe historical concentration. They do not measure prevented loss, approval policy, recoveries or causal benefit.
| Method | Budget | Reviewed | Events found | Precision | Event capture |
|---|---|---|---|---|---|
| constant | 5% | 500 | 4/89 | 0.80% | 4.49% |
| constant | 10% | 1000 | 7/89 | 0.70% | 7.87% |
| constant | 20% | 2000 | 16/89 | 0.80% | 17.98% |
| logistic | 5% | 500 | 14/89 | 2.80% | 15.73% |
| logistic | 10% | 1000 | 26/89 | 2.60% | 29.21% |
| logistic | 20% | 2000 | 38/89 | 1.90% | 42.70% |
| isotonic | 5% | 500 | 17/89 | 3.40% | 19.10% |
| isotonic | 10% | 1000 | 24/89 | 2.40% | 26.97% |
| isotonic | 20% | 2000 | 37/89 | 1.85% | 41.57% |
The downloadable 10% review queues contain only method, record ID, rank and score. Outcome labels remain in the separate evaluation table.
04 / keep the weaker year
Inspect each test funding year.
| Year | Method | Loans | Events | AUC | AP | Brier |
|---|---|---|---|---|---|---|
| 2016 | constant | 4977 | 46 | 0.500000 | 0.009243 | 0.010153 |
| 2016 | logistic | 4977 | 46 | 0.703491 | 0.027405 | 0.009193 |
| 2016 | isotonic | 4977 | 46 | 0.694365 | 0.021688 | 0.009117 |
| 2017 | constant | 5023 | 43 | 0.500000 | 0.008561 | 0.009527 |
| 2017 | logistic | 5023 | 43 | 0.641723 | 0.015299 | 0.008831 |
| 2017 | isotonic | 5023 | 43 | 0.620603 | 0.013094 | 0.008523 |
These are descriptive slices of the same fixed test cohort. They were not used to choose features, hyperparameters or sampling.
05 / bring the evidence with you
Run, refit, and check offline.
The installed wheel contains the frozen source, protocol and conversion audit. Running the study needs no source checkout, raw 574 MB download or optional PITBridge dependency.
python -m creditvintage sba-benchmark --out outputs/sba-credit python -m creditvintage verify-sba outputs/sba-credit
The verifier checks source and protocol, refits all three methods, rebuilds metrics, paired bootstrap, review order and HTML, then compares the portable ZIP. Rehashing fabricated metrics does not make them valid.
Canonical CSV SHA256: 08c60deb116345624e75cdbf22ea305be33ebeba17d2d2c11292a5e3ba1da8f1
Protocol SHA256: 99194b3f83ec07b94ef06b619c0e0230e7658b7afdd8d31c35da63275b39e829
Rebuild from both full official CSVs using scripts/prepare_sba.py --verify-frozen. The converter checks original file hashes, counts every exclusion, removes repeated recorded groups, and selects lowest row hashes before fitting.
Report an actual independent run with package version, OS, commands and the observed result. No independent adoption is claimed here.
Interpretation boundaries
- Retrospective final-snapshot funding-cohort validation; first-publication times and historical feature revisions are absent. Do not invent available_at.
- Recorded SBA charge-off is not 180-day delinquency, lifetime default, recoveries or prevented loss.
- US historical SBA lending is not current Chinese retail credit.
- Only about 90 held-out events; sampling, identity matching and missing-rate shifts restrict inference.
- A hash binds bytes, not truth or independent adoption. No deployment, causal, savings or external-use claim.
Source: U.S. SBA 7(a) & 504 FOIA, snapshot as of 2026-06-30. The catalogue designates U.S. Government Works. This study is an independent research example, with no SBA endorsement.