It is known by the data science industry that creating ML models is relatively fast and easy, but putting them into production is difficult and expensive. You’d be surprised to find out that only a small percentage of ML projects go into production– only about 12% of them do. (source)  

This is why, after looking closely at what clients were building with Datagran’s pipelines, we determined that Rule Extraction was a step in the right direction to release some of the pain. 


In essence, Rule extractions help companies connect their data, run an algorithm and automatically detect events to then send that output to business applications. One of the best use cases for this model is in the e-commerce industry. Let’s go over an example:


Rappi, a billion-dollar delivery App with more than 7 million users, wanted to understand why their users were abandoning their shopping cart after almost completing their order.


First, they integrated Datagran’s SDK to extract customer behavior from their IOs and Android App. 


Then they chose an algorithm. Because rule extraction can be only applied to Classification or Regression models to extract a set of rules found within the model and the data belonging to each rule, they chose the Classification algorithm.


It is important to also note, that rule extraction works exclusively on Decision Tree Algorithms– they can be found in two types of models:



Once they run the algorithm, one of the findings was an event labeled as WARNING_MSG-> This event was making 70% of users drop off from their mobile App order. 


An ML model example


This finding led the team to pinpoint the exact reason behind their customer drop-offs just before they were in their shopping cart, and enter their delivery address. The message warned users by displaying a pop-up asking them if they were sure the address was correct since it seems they were too far away from the restaurant. 


Consequently, the team redesigned the warning pop-up so the message was not meant as a warning, but one more question throughout the checkout process.


It didn’t end there though, the team at Rappi used these predictions to allow the chatbot to offer alternatives and recommendations for users, in order to prevent churn. 


Rule extraction can be extremely powerful and useful considering companies can build such pipelines in minutes, test and iterate without high complexities due to low-code platforms like Datagran. Teams without ML OPs can pursue such projects or teams with robust teams can now test and iterate before having to put resources to build complex pipelines.


Churn Tutorial using Rule Extraction


In this tutorial, you will predict the events that cause churn by using data from a Telecommunications company, using Rule Extraction you will extract data from their customer service logs.


  1. First, you have to create a new data Integration. Learn more about Integrations here. Then, you will build the Classification model by heading to Pipelines. Name your model and then drag and drop your integration by selecting it from the right-hand side menu and dragging it into the canvas.


  1. Process your data by dragging and dropping an SQL operator and connecting it to the integration. Hover over the element and click the edit icon to open the query canvas. Use the sample query below and replace the values. Once done, run the query and save the table. Once in the canvas, hover over the operator and press play.

‍

SELECT CAST(Account_Length AS FLOAT64) Account_Length, CAST(Vmail_Message AS FLOAT64) Vmail_Message, CAST(Day_Mins AS FLOAT64) Day_Mins, CAST(Eve_Mins AS FLOAT64) Eve_Mins, CAST(Night_Mins AS FLOAT64) Night_Mins, CAST(Intl_Mins AS FLOAT64) Intl_Mins, CAST(CustServ_Calls AS FLOAT64) CustServ_Calls, CASE Intl_Plan   WHEN 'no' THEN 0   WHEN 'yes' THEN 1 ELSE 0END AS num_Intl_Plan, Intl_Plan, CASE Vmail_Plan   WHEN 'no' THEN 0   WHEN 'yes' THEN 1 ELSE 0END AS num_Vmail_Plan, Vmail_Plan, CAST(Day_Calls AS FLOAT64) Day_Calls, CAST(Day_Charge AS FLOAT64) Day_Charge, CAST(Eve_Calls AS FLOAT64) Eve_Calls, CAST(Eve_Charge AS FLOAT64) Eve_Charge, CAST(Night_Calls AS FLOAT64) Night_Calls, CAST(Night_Charge AS FLOAT64) Night_Charge, CAST(Intl_Calls AS FLOAT64) Intl_Calls, CAST(Intl_Charge AS FLOAT64) Intl_Charge, ABS(FARM_FINGERPRINT(State)) num_State, State, CAST(Area_Code AS FLOAT64) Area_Code, CAST(REPLACE(TRIM(Phone), "-", "") AS INT64) num_Phone, Phone, CASE Churn   WHEN 'no' THEN 0   WHEN 'yes' THEN 1 ELSE 0END AS num_Churn, ChurnFROM `Churn_Example__dataset`

‍

  1. Now, you will train your model. Drag and drop three SQL operators– one for training, one for testing, and one for predictions. Repeat step 2 and use the following sample queries for each operator. 

*Make sure you run your query, save the tables and then run the Operators after entering each query.


Train dataset: 

‍

SELECT dat.*FROM `operator_609ae007079b46d760f9bb1b__sql_output` datWHERE churn = 'yes' AND RAND() < 0.8UNION ALLSELECT dat.*FROM `operator_609ae007079b46d760f9bb1b__sql_output` datWHERE churn = 'no' AND RAND() < 0.68

 

Test dataset: 

‍

SELECT dat.*FROM `operator_609ae007079b46d760f9bb1b__sql_output` datWHERE churn = 'yes' AND RAND() < 0.2UNION ALLSELECT dat.*FROM `operator_609ae007079b46d760f9bb1b__sql_output` datWHERE churn = 'no' AND RAND() < 0.32


Prediction dataset:

‍

SELECT dat.*FROM `operator_609ae007079b46d760f9bb1b__sql_output` datWHERE churn = 'yes' AND RAND() < 0.2UNION ALLSELECT dat.*FROM `operator_609ae007079b46d760f9bb1b__sql_output` datWHERE churn = 'no' AND RAND() < 0.06



  1. Now, you will add the Classification operator to the pipeline. Drag and drop the Classification Operator from the right-hand side menu and connect it to the last three training operators you just set up. Then, press edit and set up the type of algorithm you will be using along with the Rule Extraction. Choose the Decision Tree algorithm and follow the instructions below. When you’re done, hit Save and press play on the element to run the operator.


A churn model example
A churn model example
A churn model example


  1. You will now visualize the model’s output with a chart. At the bottom of the Operator, select ‘Send to Charts’. This will make your data available in the Charts section.


Once all the operators are ready to go, press the plus sign located on the canvas and click Run to begin running your model.


TIP: Pipelines can be either run, saved, or scheduled. 


  1. Then, head over to Charts, create a new one, and select the operator’s data you want to visualize.



Visualization format:



An ml model visualization chart


Table format: 


an SQL editor



an SQL query results chart


The model found 29 rules of which 1 has a match of 259 people with the probability to churn due to the combination of variables below:


  • International charge value less than $43
  • Less than 3 nightly calls
  • Less than 172 calls to support
  • Less than 246 calls per day


With this data, the company can reassess support tickets and processes, product fallacies, discontent reasons and then strategize to avoid customers churning. 


Rule extraction can be used in multiple situations to find patterns causing a specific problem within a company. Oftentimes, data volumes are too large to understand the reasons behind a challenge, so having the right tools to understand the reasons is key in acting swiftly.