← Kalyan Venkatesh

PredictOps

A fraud-scoring API for card transactions, built so every feature is something a live authorization request could actually compute. Wrapped in a CI gate, drift detection, and registry promotion.

Try it

Flip the switch, shift a feature, watch PSI cross 0.15 and stop the deploy. Runs in your browser.

How it works

A live map. The moving dots are data in flight, coloured by kind.

01System design, stages in run order

Top to bottom: load, train and compare, export to ONNX, the CI gate, registry promotion, serving, and drift. Left is inputs, right is artifacts, the amber path is the drift monitor.

system.blueprint
hover a chip to spotlight its stage
Data & features, in flight
Drift alert (PSI > 0.15)
Cleared the gate, promoted & served
Stage · artifact · model

Every feature has to be servable

Amount, category, distance, and two velocity features computed live per card. Nothing here reads a label, a future row, or a global aggregate a live request could not assemble on its own.

The leaky version scored 0.0041

An earlier version trained on features only available in hindsight, and it showed: 0.0041 AUPRC, out-of-time. Rebuilt on serve-time-computable features, the winning model reaches 0.6472.

A tiny serving image, on purpose

A multi-stage build trains and exports to ONNX, then serves with onnxruntime alone, no scikit-learn, pandas or xgboost anywhere near the container that takes traffic. Final size: 347 MB.

Notes

The winner is picked honestly.

XGBoost beat Logistic Regression 0.6472 to 0.1228 on out-of-time AUPRC, the same metric that gates promotion. Accuracy is never reported: at a 0.386% fraud rate a model that predicts never-fraud scores 99.6%+ accuracy and catches nothing.

The gate blocks bad models.

CI asserts out-of-time AUPRC ≥ 0.55 or the build fails, and a second test asserts the winner was actually picked by AUPRC, not accuracy or ROC-AUC.

Multi-stage Docker, on purpose.

Trains, exports to ONNX, serves on onnxruntime alone, no scikit-learn, pandas or xgboost in the container that takes traffic. Final image: 347 MB.