When Dr Hendra Suryanto finished his PhD in Machine Learning in 2004, the term was only then being thrown around by academics and emerging tech giants like Google and Amazon. Fast forward to 2017, and Machine Learning is now one of the most talked about technologies of recent years, promising to revolutionise almost every area of modern business.

Banks use Machine Learning to assess credit risk, insurance companies use it to detect insurance claim fraud, and retailers uses machine learning to personalise loyalty campaigns. Companies from all industries claim they implement machine learning, but the maturity of these implementations varies – is it a pet project, a prototype, or a full end-to-end production system supported by an operations team? According to Dr Suryanto, we can only realise business outcomes in dollars by fully operationalising a production solution.

Dr Suryanto at Future Data 2017

Dr Hendra Suryanto is the Chief Data Scientist at Rich Data Corporation, with over 20 years’ experience in data science, big data, business intelligence and data warehousing. He holds a PhD in Artificial Intelligence, followed by further experience as a postdoctoral researcher in Machine Learning.

Addressing the crowd on the subject of Machine Learning at the recent Future Data 2017 Conference held in Sydney, Dr Suryanto posed a seemingly straight forward question: How can an organisation use raw data to achieve quantifiable enterprise level value?

The simple answer? In order to deliver business value, Machine Learning needs to be operationalised. And to do this, the machine still requires a bit of a human touch. Input of business logic is needed in order to bring predictive analytics into prescriptive analytics, and to transform scores into business decisions that can be quantified.

Understanding Machine Learning

In all use cases for Machine Learning, the machine needs feedback in order to learn. “A simple way to conceptualise how Machine Learning works is to liken it to a baby’s brain development,” explains Dr Hendra Suryanto, Chief Data Scientist at Rich Data Corporation.

“Babies start with no knowledge, learning from examples – both good and bad. When a baby demonstrates good behaviour, it gets rewarded, and conversely if a baby does something bad, they won’t get any reward. The same concepts can be applied to the machine; we need to provide feedback as to whether the machine prediction is correct depending on the use cases, the lag from prediction and various feedback.”

What’s needed to create Business Value

At its most basic level, a Machine Learning solution consists of data, feature engineering and analytics. When we implement Machine Learning solutions there are three main steps; prototyping, productionising, and translating predictions into decisions.

• Prototyping involves a one-off tactical solution, focusing on innovation
• Productionising focuses on automating the plumbing from data to prediction
• Operationalising means incorporating the client into the journey and ingraining the machine into the business – adding the human elements necessary to realise value.

 

 

Prototyping and productionising alone aren’t sufficient to create financial outcomes; model operationalization is needed to deliver quantifiable business value – making the model part of the fabric of business processes. As stated by Dr Suryanto, “When you actually operationalise and use the Machine Learning algorithms, you involve people and business processes to generate a specific organisational outcome and affect business activities as they are happening. More than this, you’re continuously learning and refining the solution.”

Learning in Real Life

The machine can build a scoring model automatically; however it still relies on knowledge from a domain expert in order to build features or to define business rules in the decision engine. This can be seen in the below case study from a Canadian financial lender that Rich Data Corporation helped transition from manual to automated credit decisioning.

 

 

The completion of the prototyping stage for this Machine Learning solution resulted in a Gini of 0.53 (Figure A). With the introduction of specific Canadian industry regulations, the model degrades to 0.52 (Figure B).

The biggest change occurred when the model was productionised (Figure C). We saw a production degrade due to data quality issues as the one-off data extraction used to build the initial model was not representative of the data in production. After fixing the data quality issue and including further features, we retrained the predictive model, and the Gini after retraining increased to 0.55.

The scenario above illustrates that the continuous learning that is only achieved through operationalisation of Machine Learning is one of the key factors to building and maintaining a successful predictive model.

But what did this mean in terms of actual business value?

• The average loan size increased by 10.7%
• The number of customers that were declined reduced by 55.7%
• The delinquency at loan due date +30 days was reduced by 22.9%

So in the most basic terms, our operationalised model enabled the client to provide more loans, to more people, increasing the loan book size for this particular lender with a reduced amount of risk as evidenced by decreased repayment delinquency. As evidenced in the example above, much more business value is created when data science and Machine Learning can be applied in real-time scenarios and in an ongoing manner.