You have probably heard of machine learning, but are you sure you know what it is? If you are struggling to make sense of it, you are not alone. There is a lot of buzz that makes it hard to tell what’s science and what’s fiction, what’s rebranding and what’s new. This article is an attempt to provide a simple non-technical explanation of machine learning and why it matters.
But before we talk about machine learning, let’s take a step back and reflect on how we have been developing computer programs since the 1970’s. Traditionally, developing a computer program requires you to specify in advance exactly what you want the system to do and then formulate it as a precise program that a computer can execute. For example, if you are building a credit card fraud detection algorithm, you will have to define the precise rules for classifying a payment instruction as potentially fraudulent (e.g. if a payment originates from an online retailer that is based outside the cardholder’s home jurisdiction, mark it as potentially fraudulent).
Rules are extremely easy to understand and are typically developed by domain experts and consultants who translate their experience and best practices to code to make automated decisions. But when a rules-based software gets operationalised, one starts with say 100 scenarios and 100 rules to handle it. As time goes by and as the business requirements evolve, we encounter more and more scenarios and start adding more rules to keep the software relevant. There comes a point where nobody really knows or can measure how well the rules work – this is the situation today with a lot of legacy hand-crafted and rules-based computer programs.
Enter machine learning. In contrast to traditional software development, where the focus is on automating tasks by writing rules for a computer to follow, machine learning goes a step further – it automates the task of writing these rules. In this approach, an engineer does not give the computer rules for how to make decisions and take actions. Instead, the engineer collects domain-specific data, which are then fed into machine learning algorithms with the objective of finding a computer program that can make decisions and take actions. For example, in the case of credit card fraud detection, the domain-specific data could be a collection of historical transactions (where each transaction is labelled as fraudulent or not-fraudulent), and the output of the machine learning process would be a program that can detect fraudulent transactions (without a human explicitly encoding this knowledge).
The latest buzz around machine learning comes from the fact that there are many tasks and decisions that are far too complex to automate using hand-crafted rules (e.g. a self-driving car!). Machine learning could be the answer to some of these challenges.