Independent model validation
Second Pass
Can you trust the model behind your next decision?
I review the code and the statistics behind your result. You get a written verdict on whether the evidence supports using it to commit capital, set a price, or roll out a change.
Request a model reviewHow I check the result
I start with the decision and reproduce the number your team relies on. The next checks depend on how you use it.
- Look-ahead and data leakage
- I trace inputs to the moment a decision would have been made. That can uncover a revised data point used too early or the same borrower appearing in both training and test data.
- Out-of-sample calibration
- I compare predictions with outcomes on held-out data. For a lender, that means checking whether borrowers assigned a 5% default risk actually default about that often.
- Performance in the tails
- I look at the errors that carry the largest cost. A demand forecast can be accurate most weeks and still leave you short every time demand peaks.
- Experiment and analysis code
- I trace the headline estimate back to the data. For an A/B test, I check assignment, who counted in each group, when the test stopped, and whether the uncertainty supports a rollout.
When to bring in a second reviewer
- Small funds, RIAs, and prop shops deciding whether to fund a backtested strategy, rely on a risk model, or show the results to an allocator.
- Pricing, forecasting, and underwriting teams deciding how much to charge, stock, lend, or insure based on a model.
- Product and growth teams deciding whether an A/B test in a notebook warrants shipping a change.
- Biotech and health startups preparing to share an analysis with investors or include its results in a submission.
A written verdict you can share
The report identifies the model version and decision, gives my verdict on its use, and shows the checks behind it. It explains which findings could change the decision, where the data are too thin to tell, and what to fix or retest. You can put it in front of an investor, allocator, or regulator.
I use AI agents to help trace code and rerun checks where appropriate. I verify the findings and take responsibility for the verdict.
To scope a review, tell me what decision depends on the result, who needs the report, and by when. We’ll agree on the evidence to review, the fee, and the timeline.
Request a model review