Real-Time Fraud Detection

When you tap a card, the whole approval, travelling from the shop to the bank and back, usually takes a few hundred milliseconds. A millisecond is a thousandth of a second, and a blink takes roughly 100 to 400 of them. The fraud check gets only a slice of that time, often a few tens of milliseconds.

That shapes everything. There is no time to dig through years of history or wait on slow services while the shopper stands at the till. All the heavy counting must already be done before the swipe arrives, so at decision time we only look up answers that are ready and waiting.

If the check runs out of time, the payment still needs an answer. So the system has a backup plan, such as using simple rules only, or a policy chosen in advance, like approving small amounts and declining risky ones. The trade-off is that the backup is less accurate, so it should be rare.

Every swipe is written as a small event into Kafka, a system that keeps streams of events in order and lets many programs read them. Think of a conveyor belt that never drops anything and that several workers can watch at once.

A stream in Kafka is called a topic, and it is split into partitions, which are like separate lanes on the belt. We pick the lane by card number, so every swipe from the same card always goes into the same lane. Inside one lane, order is kept, so one card's swipes are always read in the order they were written.

Lanes let many computers work side by side, one lane each, without ever needing to share a card's story. The trade-off is that order only holds inside a lane, and a very busy card or shop can make one lane much busier than the others.

Flink is a stream processor, a program that reads events from the belt and keeps counting as they go by. A sliding window is a moving time frame. Every minute we ask, how many times was this card used in the last 10 minutes, and how much was spent? Because the frame moves forward one minute at a time, the windows overlap, and each swipe is counted in ten of them.

Other useful counts are how many different shops, or how many different countries, a card was used in during the last hour. A card used in three countries within one hour is a strong clue.

Flink keeps these running counts in memory for each card, called keyed state. Every so often it saves a snapshot, called a checkpoint. If a computer crashes, Flink loads the last snapshot and replays the events since then from Kafka, so each swipe affects the counts exactly once, never lost and never doubled.

Swipes do not always arrive in order. A shop's machine might lose its connection and send a swipe late. Flink uses event time, the time written on the swipe itself, not the time it arrived. It also uses a watermark, a marker that says, I do not expect anything older than this anymore. A window is closed only after the watermark passes its end. The trade-off: waiting longer catches more late swipes, but the counts arrive a little later.

Features are the clues a model uses, such as swipes in the last 10 minutes or how much this card usually spends. A feature store is where we keep them, and it has two parts.

The online store is the sticky note by the register. It is a very fast key-value store, which works like a row of labeled cubbies: give it a card number and it hands back that card's latest clues, usually within a few milliseconds. The streaming jobs keep it up to date as swipes arrive.

The offline store is the big diary in the back room. It keeps the full history of every clue over time, which is what we need to train new models. It is large and cheap, but far too slow to use during a swipe.

Both stores must use the same clue recipes. If training counted swipes one way and the live system counted them another way, the model would practice one game and then be asked to play a different one. This mismatch is called training-serving skew. Training data must also be point-in-time correct: for each past swipe, we use the clues exactly as they were at that moment, never peeking at what happened later. Otherwise the model looks smarter in tests than it really is.

When a swipe arrives, the payment service asks the scoring service, is this OK? The scoring service fetches the card's clues from the online store, then adds a few clues it can only work out right now, such as how this amount compares with the card's usual amount.

Then it runs the model, a program trained on millions of past swipes, which returns a risk score, like a number between 0 and 1. Rules sit beside the model for things we always want, such as blocking cards that were reported stolen. Together they give one of three answers: approve, a step-up challenge that asks the person to prove it is them, for example with a code sent to their phone, or decline.

Every call has a strict timer. If the online store or the model is slow, the service does not wait. It falls back to rules only, or to the policy chosen in advance. Every decision is logged together with the clues that were used, so we can later check how good the decisions were.

The trade-off is between catching thieves and annoying honest customers. Set the bar too low and fraud slips through. Set it too high and real people get declined at the shop. The challenge answer is the middle path for unclear cases.

For many swipes, we only learn the truth later. A cardholder may spot a strange charge weeks afterwards and dispute it, which can lead to a chargeback, where the money is pulled back. Confirmed fraud reports like these are the labels that tell us which swipes were bad, and they arrive days or weeks after the swipe.

Those labels are joined with the logged decisions and the point-in-time clues from the offline store, and that becomes the training data for the next model. The new model is tested against the current one before it is trusted with real payments.

The trade-off is delay. Because labels arrive late, the newest fraud tricks show up in the training data only after a while. Fraudsters also change their tricks once they get blocked, so models are retrained regularly, and quick rules fill the gap in between.