For every dollar of fraud that financial services companies suffer, they incur $2.67 in costs to their business. With more entry points in the digital age and increasingly sophisticated attackers, tackling fraud manually is quickly fading to irrelevance: but Machine Learning offers a promising way to automate the process, as well as surface more nuanced fraud patterns.
This post will walk through the challenges of applying ML models to fraud detection, popular applications, and tradeoffs to think about in model selection.
Unique Use Cases
Machine Learning applied to financial fraud isn’t only about preventative measures: it’s also about the costs incurred by that prevention. Almost ¼ of fraud costs can be traced to fraud prevention and manual review: automating that process replaces teams of hundreds of analysts with fewer, smaller teams of Data Scientists.
Algorithms can tackle fraud prevention in a few different ways:
- Anomaly detection: alert fraud teams when there’s unusual activity, like a large charge on a credit card or use of a card in an unusual location
- Similarity: determine when accounts share information suspiciously, like when multiple bank accounts link to very similar addresses
- Clustering: given that fraud characteristics are hard to determine in advance, separate transactions into groups based on shared qualities
The method you use will largely depend on the kind of data you’re working with and the breadth of features that you have at your disposal.
Financial fraud was one of the earliest relevant applications of Machine Learning, and so there’s already a lot going on. One of the leaders in the space is Stripe, the ubiquitous payments company that powers the payment backend of hundreds of thousands of businesses. Radar is Stripe’s fraud prevention solution, and it’s built from the ground up with Machine Learning.
Radar uses a variety of different features – some unique and proprietary to Stripe’s network of businesses – to isolate and flag credit card fraud on their platform. Their Machine Learning teams found that geography – how many countries a card has recently been used in – is a major predictor of fraud.
But Radar wasn’t built overnight: it took years of work from Stripe’s Data Science teams to navigate some of the challenging problems that fraud presents.
What makes a something a unique vertical in Machine Learning is domain specific challenges in applications. Fraud has a number of those:
Long Tail Distribution
In a given population of transactions, bank accounts, trades, or any other target for fraud, the overwhelming majority of targets will be legitimate (not fraudulent). Because fraud is so rare, datasets will be seriously skewed towards non-fraud, which can severely hamper the models that you build from those datasets. Dealing with skewed classes typically involves data transformation and adjusted accuracy metrics.
Source: Becoming Human
At its core, all of Machine Learning is about sussing out the relationship between variables. But what if those variables don’t actually have a quantifiable relationship? Attackers are constantly changing their methods and trying out new approaches, which means that whatever fraud “looks like” is constantly evolving. You’ll need to update models frequently and be hyper-vigilant about changing ground truths.
Your Opponents Are Adversarial
Unlike in normal prediction environments, fraud is being perpetrated by an active, malicious external party: it’s not a natural phenomenon. These people are searching for your weakest points, and are always looking to improve and refine their methods. They can change on the fly, and your models need to be able to as well.
Any measures that you take to counteract fraud and make it into the user experience will likely be detrimental to it. Security measures like an authenticator, 2 factor authentication, and even passwords create user friction. Additionally, flagging too many transactions as fraudulent will also anger customers who get stuck in a false positive. You’ll need to think about these tradeoffs when deciding where and how to run your models.
Subjective Business Contexts
Fraud is not a binary, and it’s not objective: the definition varies across businesses, goals, and strategies. In credit card fraud for example, what businesses will want to flag as fraud will depend largely on their margins. Many credit card companies won’t tolerate a fraud rate of >1%, but lower margins increase the cost of fraud, leaving decision makers in a tough tradeoff. Good Machine Learning solutions need to be built to take this into account.
Model selection in any domain is largely dependent on that domain’s specific challenges, and fraud detection is no exception.
Merchants looking to prevent fraud often run into a natural issue: they don’t have enough data to create reliable models. More biased methods of Machine Learning work better with less data, but assume a certain structure for the underlying relationship. Deep Learning is more flexible, but requires tons of data to get working. In practice, organizations will often start with simpler models and build on them as more data becomes available.
When a model flags fraud, you’ll often need to explain to an end user why this is the case: you can’t shut down someone’s account without telling them why. But most of the cutting edge Deep Learning going on lacks explainability: accuracy is high, but it’s impossible to suss out why the model spit out a given classification. Algorithms like Linear Regression are easy to explain but lack sophistication in the function they use to map relationships. This tradeoff is a key part of model selection.
Speed and Scale
To keep fraud detection running without negatively impacting the user experience, it needs to be fast and reliable. Models can’t take minutes to analyze target events if you’re relying on real time action. Deploying even simple Machine Learning models at scale with little to no latency is a major challenge for organizations across the spectrum. Using a platform as a service like Algorithmia can automate the process, and make it much easier to get to meaningful ROI on your fraud detection efforts.
Introducing Algorithmia’s ML Vertical Framework
While helping companies across industry verticals deploy their Machine Learning, we’ve found that each industry has unique characteristics that make their applications challenging to develop. The framework used in this post can help stakeholders understand their ML potential, as well as navigate the difficulties they’ll face along the way.
Machine Learning and AI for fraud prevention: a primer (Feedzai) – “Fraud solutions needs to more sophisticated to keep in pace with the fraudsters and react within the short time fraud attacks happen to when they are discovered. Organizations that want to defend themselves against fraud need to have a superior, faster-learning solution that can constantly evolve yet is easy to use and maintain.”
Machine Learning and fraud prevention (Ravelin) – “Machine learning is simply a form of artificial intelligence that enables computers to learn without being explicitly programmed. It’s especially good at recognising patterns in data and therefore equally good at spotting anomalies in those patterns. This makes it a great approach for preventing fraud.”
Toward Scalable Learning with Non-Uniform Class and Cost Distributions: A Case Study in Credit Card Fraud Detection (Paper) – “Very large databases with skewed class distributions and non-uniform cost per error are not uncommon in real-world data mining tasks. We devised a multi-classifier meta-learning approach to address these three issues. Our empirical results from a credit card fraud detection task indicate that the approach can significantly reduce loss due to illegitimate transactions.”
Machine Learning techniques for fraud analytics, part 1 (ThreatMetrix) – “In the financial services world, fraudsters must be faster and smarter than the slowest bank to be “quids in”. And a bank must be better than the fraudster to avoid being a victim. Analytics and data science play a pivotal role in this. However, a troubling issue that banks often face is the bridge between the data scientist and the fraud analyst: one really understands statistics while the other understands fraud.”
Machine Learning Fraud Detection: A Simple Machine Learning Approach (Data Blogger) – “In this machine learning fraud detection tutorial, I will elaborate how got I started on the Credit Card Fraud Detection competition on Kaggle. The goal of the task is to automatically identify fraudulent credit card transactions using Machine Learning. My Pythonic approach is explained step-by-step.”