City of Rochester
A model signal becomes a question for review.
Model
Explore fraud-risk signals using Python and XGBoost.
Explain
Use SHAP to inspect feature contributions.
Question
Consider false positives and missing information.
Investigate
Support a person’s review of the case.
A feature explanation is not proof of fraud. Its value is helping a reviewer understand and question the model’s output.
Illustrated from this case study. Examples are synthetic and contain no customer data.
The short version
The full story
A risk score is more useful when a reviewer can examine what influenced it. Through Simon Vision Consulting, I worked on an explainable fraud-risk modeling project for the City of Rochester.
The work
The project used Python and XGBoost for modeling, with SHAP to examine how features contributed to a prediction. My focus was on connecting the model output to an explanation a reviewer could inspect.
The product question
A model’s evaluation score is only one part of the decision. The team also needs to understand false positives, missing information and whether an explanation helps someone investigate a case.
An explanation is not proof of fraud. The output should support further review, with a person responsible for the decision.
What I took from it
Explainability belongs in the workflow from the start. It affects what the interface shows, how users question a result and how the team evaluates whether the model is helping.