Loan Sherlock
Screens 50,000 loan applications for fraud and default risk using windowed transaction behavior.
- LightGBM
- scikit-learn
- imbalanced-learn (SMOTE)
- Streamlit
- Status
- Proof of concept, inactive. Repository created 2025-06-15; all 9 commits and the final push land the same day, each labelled 'Add files via upload'. No activity since.
By the numbers · 11
50,000
Loan applications in the modelling corpus
50,000
Customer transactions joined for behavioural features
18,314
Distinct customers linked across both datasets
2.05%
Fraud base rate, 1,026 fraudulent of 50,000 applications
4
Distinct fraud typologies labelled in the data
0.8862
ROC AUC of the selected logistic-regression fraud model on a 10,000-row holdout
0.64
Fraud-class recall of the selected model, 131 of 205 held-out fraud cases caught, at 0.28 precision
0.9087 AUC / 0.00 recall
Random forest: highest ROC AUC of the bake-off, zero fraud cases predicted
18,368
Encoded feature dimensions produced from 36 raw columns
16
Windowed transaction-behaviour features (4 statistics x 30/90/180/365-day windows)
0.51 -> 0.68
Macro F1 gain from collapsing four loan-status classes into three
Summary
Loan Sherlock scores a loan application on two independent axes, fraud likelihood and approval outcome, by joining the application record to that customer's prior transaction history and summarising the history over four look-back windows before the application date. The architecture matters because fraud is a 2% minority class: the project's own evaluation shows a random forest reaching the highest ROC AUC (0.9087) while predicting zero fraud cases, and the pipeline deliberately selects a lower-AUC logistic regression that recovers 131 of 205 held-out fraud cases instead. A Streamlit front end wraps the pipeline in single-application scoring plus three analytics pages.
The problem
Lenders face two decisions on every application and very little time to make them: is this applicant likely to repay, and is the application itself fraudulent? The second question is the harder one, because outright fraud is rare, roughly one application in fifty, and rare events are easy to miss when a reviewer or a scoring rule optimises for the common case. The information that would expose fraud often sits outside the application form entirely, in the applicant's recent spending behaviour. This project builds a screening tool that reads both sources together.
Approach
Joined two 50,000-row datasets on customer_id and, for each application, aggregated only transactions dated strictly before the application date, transaction count, total spend, average spend and distinct merchant categories across 30-, 90-, 180- and 365-day look-back windows.
Capped every numeric column at its 1st and 99th percentiles before modelling, and backfilled the null fraud_type column as an explicit 'Not Fraudulent' category rather than dropping rows.
Derived income-relative ratios, existing EMI to monthly income, requested loan amount to monthly income, alongside application year, month and day-of-week.
Wrapped scaling and encoding in a single scikit-learn ColumnTransformer so the identical transform runs at training time and at inference time inside the Streamlit app.
Applied SMOTE to the training split only, never the holdout, lifting the binary fraud training set from 39,179 negative / 821 positive to a balanced 39,179 / 39,179.
Ran a three-model bake-off on the fraud task (logistic regression, random forest, LightGBM) and read per-class confusion matrices rather than headline accuracy, which is what surfaced that two of the three models never predict fraud at all.
Reformulated the loan-status target after the first attempt: the four-class version could not separate 'Fraudulent - Detected' from 'Fraudulent - Undetected', so the two labels were merged and the multi-class model retrained.
Shipped a four-page Streamlit front end covering single-application scoring with a Plotly risk gauge, a dataset analytics dashboard, portfolio insights, and a methodology walkthrough.
Architecture
loan_applications.csv + transactions.csvcleaning and 1st/99th-percentile cappingtemporal and income-ratio feature engineeringper-customer transaction aggregates over 30/90/180/365-day windows preceding the application dateColumnTransformer (StandardScaler + OneHotEncoder, 3618,368 dims)SMOTE on the training splittwo heads: binary fraud classifier and multi-class loan-status classifierpredict_loan_risk_and_fraud()four-page Streamlit UI
| Component | Role |
|---|---|
| loan_project.ipynb | 23-cell research notebook carrying the whole method: data inspection, outlier capping, feature engineering, the three-model fraud bake-off with printed ROC AUC and confusion matrices, and both loan-status target formulations. |
| loan_python_file.py | Extracted training and inference module. train_models() rebuilds the preprocessor and both LightGBM heads; predict_loan_risk_and_fraud() re-applies feature engineering to a single application dict and back-fills any missing transaction-window columns with zeros. |
| eda_analysis.py | Dataset aggregation helpers computing approval rate, fraud rate, CIBIL distributions, loan-type and employment splits, and state-level breakdowns parsed out of the address field, feeding the dashboard pages. |
| test_ui.py | 1,612-line Streamlit application with four pages, Risk Assessment (input form, Plotly risk gauge, factor breakdown), Analytics Dashboard, Market Insights, and an EDA/methodology walkthrough. |
| loan_applications.csv / transactions.csv | The two source datasets committed in-repo: 50,000 applications across 21 columns and 50,000 transactions across 16 columns, linked on customer_id. |
Trade-offs
Chose
Logistic regression as the fraud head, despite the lowest ROC AUC of the three candidates
Over
Random forest, which scored the highest ROC AUC of the bake-off at 0.9087
the random forest's holdout confusion matrix was [[9795, 0], [205, 0]], it never predicted fraud once, making the AUC advantage useless in operation. Logistic regression at 0.8862 AUC caught 131 of 205 fraud cases at 0.28 precision, trading reviewer workload for actually surfacing fraud.
Chose
SMOTE synthetic oversampling of the training split
Over
Training on the raw 2% class distribution
the fraud class held only 821 of 40,000 training rows. The notebook documents that both tree models still collapsed toward the majority class even after rebalancing, rather than hiding the failure.
Chose
Collapsing 'Fraudulent - Detected' and 'Fraudulent - Undetected' into a single Fraudulent class
Over
The original four-class loan-status target
the four-class LightGBM scored 0.01 and 0.03 recall on those two labels and mostly confused them with each other; merging them raised macro F1 from 0.51 to 0.68.
Chose
One blanket ColumnTransformer that one-hot encodes every object column
Over
Per-column encoding that excludes, hashes or target-encodes high-cardinality free text
it guarantees training and inference share one identical transform, at the cost of expanding 36 feature columns into 18,368 encoded dimensions, the free-text residential_address column alone carries 18,314 distinct values.
Chose
Training both model heads lazily in-process on the first prediction request
Over
Serialising fitted artifacts to disk and loading them at startup
no persistence layer was built, pickle is imported at the top of loan_python_file.py but never called, so the Streamlit app refits the entire pipeline the first time a user clicks Analyze.
At scale
Two committed datasets totalling roughly 22 MB of CSV, loan_applications.csv (11,967,173 bytes) and transactions.csv (10,795,997 bytes).
50,000 loan applications across 21 columns and 50,000 transactions across 16 columns, linked by 18,314 shared customer identifiers.
36 engineered feature columns expanding to 18,368 encoded dimensions after one-hot encoding.
SMOTE-augmented training sets of 78,358 rows (binary fraud), 130,824 rows (four-class loan status) and 98,118 rows (three-class loan status).
1,612-line Streamlit application spanning four pages, plus a 23-cell analysis notebook.
8 files and 9 commits in total, every commit dated 2025-06-15.
My role
Sole author. All 9 commits in the repository are by munib123, covering the notebook analysis and model selection, the extracted training/inference module, the EDA helpers, and the Streamlit front end.