“The new credit model didn’t perform as expected,” he said.
My colleagues from another team came out from the “war room” looking dejected.
This was my first time witnessing a data science “mistake” in a business context, so I didn’t quite understand the scale of it.
As I would learn over the next six months, this was a very costly error from both a financial and operational perspective, because of the downstream impact and reverberating effects it had across the company.
This issue impacted every team and we all worked tirelessly for months after:
- Analysts worked on metrics to track portfolio health,
- Data scientists worked on diagnosing the issue and patching it,
- Engineers worked with data scientists to deploy the new model,
- Product marketers worked on campaigns and messaging,
- Operational folks worked on customer outreach and education on repayment.
The first sign of trouble was much higher default rates relative to expectations, but this was just the…