How to Improve Customer Retention in FinTech

Introduction

, there is a pattern that any growing team quickly learns: acquiring a new user, or bringing back a user who has already churned, is much harder and more expensive than retaining someone who is still using the product. In digital banking, this is especially visible because customer acquisition costs are high.

User retention in fintech depends on many factors: service quality, product functionality, loyalty mechanics, communication, and other parts of the user experience. In this article, I will focus on one practical example: how ML-based personalization can help work with at-risk users and apply loyalty mechanics more precisely.

The task had three steps. First, we needed to learn how to identify users who were likely to churn. Then we had to define the retention mechanic, meaning what exactly could bring the user’s interest back to the product. After that, we needed to optimize the intervention itself: to understand which users in the risk segment would actually be affected by the retention offer.

In this article, I will share a practical guide to building an analytical system for this type of task. It is based on two ML models: a pre-churn model, which identifies users at risk, and an uplift model, which helps determine which of them are truly sensitive to a retention offer. I will explain the practical value of each stage: why it was needed from a product perspective, what problem it solved, and what conclusions we were able to draw from the research.

Preparation: Target and Features

Let’s start with the churn definition. I worked on the growth of a debit card product, where user behavior is best described through transactional activity. So we settled on a practical definition: churn means no card payments for 30 days. This definition is often used in financial products, and the logic behind it is fairly clear.

If a person has not made a single transaction in a month, the card has effectively dropped out of their wallet. Formally, the account may still be open, but from a product perspective, the user is no longer active. Accordingly, the model target is a binary flag: whether the user made at least one payment within the next 30 days.

Next, let’s move to feature collection. For this task, I collected several dozen features describing the client’s behavior in the service. I cannot disclose the exact production feature set, but I will share the main principles for building a feature representation of a client in a fintech product based on transactional activity and user profile data.

• User profile. Basic client characteristics: age, region, device type, connected products, tariff plan, subscriptions or loyalty program status, account status such as new user, active client, and so on.
• Activity aggregates over time windows. Behavioral metrics calculated over 7, 30, and 90 days: number of transactions, turnover, average transaction amount, number of unique spending categories, number of active days, and so on. Different windows help capture changes in user behavior.
• Time intervals. How much time has passed since the last or first payment, what the average gap between transactions was, and how the rhythm of activity changes over time. In practice, the signal that a user “has not appeared for a while” often turns out to be one of the strongest predictors of churn.
• Calendar features. Day of the week and month. Without these features, the model may confuse natural drops in activity with actual churn. Financial services are highly sensitive to seasonal patterns, so it is better to account for them upfront.
• Derived features. Ratios between time windows, shares of transactions of a certain type, normalized and relative metrics. These features help compare clients with different baseline levels of activity and avoid penalizing users who simply spend less.

Pre-churn Model

Let’s start with the pre-churn model. Our goal at this stage is to identify the segment of users for whom the existing loyalty mechanics are no longer working. In other words, users who are losing interest in the service under the current level of loyalty incentives.

At this stage, the task is to identify users with an elevated risk of leaving. For this, we need a model that predicts the probability of a future payment. If the predicted probability is low, the user falls into the pre-churn segment. In that case, the service should pay attention to the user in advance, while churn can still be prevented.

Training the baseline model

The model is built on the collected features and predicts the probability of a payment over a 30-day horizon. Essentially, this is a binary classification task with probability estimation. User characteristics are passed as input, and the output is the probability that the user will make at least one payment within the next 30 days.

$p\_base(x) = P(Y = 1 | X = x)$

where Y ∈ {0, 1} indicates whether the user will make a payment within the next 30 days, and X denotes the matrix of user features.

Many ML tools can be used for this type of task. In practice, gradient boosting models often deliver strong performance with a reasonable amount of analytical effort required for model training: XGBoost [1], LightGBM [2], and CatBoost [3].

Users can then be ranked by risk level. The lower the predicted probability of payment, the higher the risk of churn:

$risk\_score(x) = 1 – p\_base(x)$

A user enters the pre-churn segment if their predicted probability of payment is below a selected threshold t:

$S\_prechurn = \{x : p\_base(x) < t\}$

The threshold t is selected based on the segment size, the retention budget, and the acceptable level of risk. In practical terms, this means choosing a threshold that keeps loyalty costs within the promo campaign budget.

Validation

First, when validating this type of model, it is important to check whether it will work in the same mode in which it will be used in production. To do this, it is useful to hold out several more recent months for an additional model check. The model is trained on deeper historical data, and then we evaluate how well it predicts churn in a more recent time window.

Second, what matters here is not so much the quality of the final classification, but the quality of the predicted probability itself. We need to monitor how well the model ranks users by their risk of churn, for example using ROC-AUC and other ranking metrics. It is also important to look separately at probability calibration: whether any bias appears in the predicted probabilities. For this, we can use a calibration curve and a Hosmer–Lemeshow curve [5].

In our case, the model’s ranking quality barely degraded even on fresh data. However, calibration did deteriorate: over time, the predicted probabilities became shifted relative to the actual event rate under the influence of unobserved factors. This is a pattern I regularly see in similar models, so I want to emphasize it separately.

Calibration

There is a fairly effective way to address the problem of deteriorating model calibration. It involves building a separate calibration model on top of the main pre-churn model. The idea is simple: the calibration model takes the prediction of the main model as its only feature and transforms it into a more accurate probability on fresh data. Logistic regression is used for this step.

$p\_calibrated(x) = g(p\_base(x))$

where p_base(x) is the prediction of the baseline pre-churn model, and g is the calibration model.

The main advantage of this approach is its simplicity. The calibration model can be updated regularly, is quick to validate, and does not require a full redesign of the entire ML system. As a result, we get a process that helps keep the model stable over time without constant manual maintenance.

Offer for the Pre-Churn Segment

After building the pre-churn model, we launched a randomized experiment: part of the pre-churn segment received an increased cashback offer, while the rest remained in the regular flow.

The experiment showed that the direction was right: the increased cashback offer helped improve user retention, and we saw statistically significant gains in retention. However, we were not satisfied with the cost of retaining one user. It turned out to be even higher than the cost of acquiring new users. This was driven by a large share of organic users within the segment. Along with genuinely at-risk users, the pre-churn model also selected users who were less responsive to retention mechanics. In other words, some users received the cashback offer even though they were already going to make a payment, while others did not respond to it at all.

This led to the main conclusion of this stage: knowing that a client is in the risk zone is not enough. We also need to understand whose behavior can actually be changed by the communication, and whose behavior will remain the same. For that, we need the next stage: an uplift model, which we will move on to now.

Uplift Model

The goal of the uplift model is to estimate how much a specific offer can change user behavior. It identifies the clients whose probability of making a payment increases the most specifically because of the offer. This is where the cost saving comes from: the budget is not spent on the entire risk segment, but only on the users whom the offer can actually help retain.

Such a model cannot be trained only on user features and the fact of payment. It needs experimental data, where part of the audience received the offer at random and another part remained in the control group. Only then can the model observe the difference in response between the two scenarios and learn to separate users who are sensitive to the intervention from those who are indifferent. The A/B experiment we ran earlier provided exactly this data, so we could move straight to model building.

Model training

In essence, uplift is the difference between two probabilities of the target action: with and without the intervention.

$τ(x) = P(Y = 1 | X = x, T = 1) – P(Y = 1 | X = x, T = 0)$

where τ(x) is the expected uplift for user x, Y ∈ {0, 1} indicates whether the user made a payment within 30 days, X denotes the user features, and T ∈ {0, 1} indicates whether the intervention was applied. In our case, T = 1 means that the user received the cashback offer, while T = 0 means that the user did not receive the offer.

There are several ways to estimate this difference. I will describe two approaches [4]: the T-learner and the S-learner.

In a T-learner, two independent binary classification models are trained. The first predicts the probability of payment for users who received the offer. The second predicts the probability of payment for users in the control group.

$p₀(x) = P(Y = 1 | X = x, T = 0)$

$p₁(x) = P(Y = 1 | X = x, T = 1)$

After that, uplift is calculated as the difference between the two predictions for the same user:

$τ(x) = p₁(x) – p₀(x)$

In other words, the model estimates how much the offer increases the probability of payment for a specific user.

In an S-learner, a single model is used. It takes the user features and an additional feature indicating whether the user received the intervention or not.

$a(x, t) = P(Y = 1 | X = x, T = t)$

To estimate uplift, we make predictions for the same user under two scenarios: first as if the user received the offer, and then as if the user did not.

$τ(x) = a(x, 1) – a(x, 0)$

In practice, both approaches produced similar results, so the choice between them was not critical. The next step is intuitive: we take the users with the highest predicted uplift, meaning those for whom the offer increases the probability of payment the most.

Validation

After building the uplift model, we evaluated its effectiveness using an A/B experiment. The experiment design was as follows.

1. We selected the pre-churn segment. The size of the segment was determined by the available campaign budget.
2. We split the segment into two groups: one without the uplift model and one with the uplift model.
3. In the group without the uplift model, the offer was distributed in the standard way: some users received cashback, while others remained in the control group.
4. In the group with the uplift model, the model first selected users with the highest predicted uplift. Then, within this selected segment, some users received the offer, while others remained in the control group.
5. Finally, we compared the results. This allowed us to evaluate two strategies: a standard offer distribution across the pre-churn segment and a targeted distribution after filtering users through the uplift model.

As a result of the experiment, we saw that adding the uplift model to the retention system can significantly reduce the cost of retaining a user. The main effect comes from reducing the share of organic users: offers stop being distributed at scale to users who either would have made a payment without the offer anyway, or would not have responded to it in any case.

Customer Retention System

Let me describe how the final retention system worked. In essence, it was a sequential selection process for retention intervention.

At the first stage, the pre-churn model p_calibrated(x) estimated the probability of a payment within the next 30 days. If this probability was below a selected threshold, the user entered the pre-churn segment:

$S\_prechurn = \{x : p\_calibrated(x) < t\}$

At the second stage, the uplift model τ(x) was applied within the pre-churn segment. It estimated how much the cashback offer would change the probability of payment for a specific user and selected those who were most likely to respond to the offer:

$Target = \{x ∈ S\_prechurn : τ(x) > u\}$

where u is the minimum expected uplift at which it makes sense to show the offer to the user.

For ongoing operation of the system, we took into account that both models, the pre-churn model and the uplift model, would need to be recalibrated regularly. That means the system needed a constant flow of fresh experimental data. So we designed the offer distribution logic in a way that would generate this data continuously.

Some users were excluded from the retention system entirely, meaning no retention mechanic was applied to them. This group showed the real churn level without intervention and served as a reference point for calibrating the pre-churn model. Within the pre-churn segment itself, there was another split: some users did not receive the offer, while others received it randomly, independently of the uplift model. This random offer allocation provided clean data on response to the intervention, which was then used to recalibrate the uplift model.

Conclusion

In the end, we built a complete machine learning-based retention system. It relies on two models. The first one, the pre-churn model, identifies users with a low probability of payment, meaning those who are already in the risk zone. The second one, the uplift model, looks inside this segment and finds customers who are actually likely to respond to a personalized offer, such as increased cashback.

This combination solves two problems at once. On the one hand, we can see in advance who is losing interest in the service and react before the user actually leaves. On the other hand, the marketing budget is spent more precisely: offers are not sent to every user in the risk group, but only to those whose behavior is likely to be influenced by the intervention.

As a result, the system helps not just retain customers, but do it in an economically meaningful way. The company spends less on users who would have stayed anyway and concentrates the budget where the retention intervention produces a real incremental effect.

Sources

[1] Tianqi Chen and Carlos Guestrin. XGBoost: A Scalable Tree Boosting System. Proc. 22nd ACM SIGKDD, 2016. https://xgboost.readthedocs.io/

[2] Microsoft Corporation. LightGBM Documentation, 2026. https://lightgbm.readthedocs.io/

[3] Yandex. CatBoost Documentation, 2026. https://catboost.ai/docs/en/

[4] Sören R. Künzel, Jasjeet S. Sekhon, Peter J. Bickel, and Bin Yu. Metalearners for Estimating Heterogeneous Treatment Effects Using Machine Learning. Proceedings of the National Academy of Sciences, 116(10), 2019. https://www.pnas.org/doi/10.1073/pnas.1804597116

[5] Van Calster et al. Tutorial on Calibration Measurements and Calibration Models for Clinical Prediction Models. JAMIA, 27(4), 621–633, 2020. https://academic.oup.com/jamia/article/27/4/621/5762806