.jpg)
So far, most of us have been talking about batch operations in Machine Learning. Hey, it’s been complicated enough to create Machine Learning models in general, but as technology progresses and companies get more comfortable with the idea of using machine learning, real-time pops up. Why? Imagine you are a bank, and you want to take advantage of the fact that your users are navigating on your web site to offer them a credit card or a loan. What if you could right on the spot offer them a credit line just by asking them for their income, location, age and some other variables. This is only possible with real-time machine learning models. The thing is, this is HARD. Why? Let’s explore a bit.
Why is deploying real-time models hard?
The main challenge with building this kind of operation is maintaining standing infrastructuring. Simply said, in batch operations we are used to calling a database and waiting for a response. When we are working with real-time we have less than 100 milliseconds to retrieve data.
That said, I could expand about the challenges here but Instead of writing and duplicating very good blog posts on this topic I want to share this very good piece written by Demetrios https://mlops.community/why-real-time-data-pipelines-are-so-hard/
It is very important to note that although infrastructure is one of the key characteristics, there are many moving parts while maintaining a data pipeline like re-training and version control. All of these makes the task of building these types of projects monstrous and very different from batch operations.
That said, IT managers now have to consider 2 different parallel pipelines, one for batch and one for real time. On each they would need to consider:
- Data ingestion
- Updating data
- Model deployment
- Re-training
- Standing infrastructure
- Scheduling
- Versioning
- Automatic re deployment of APIs depending on versions
- Third Party APIs
- Scalability of APIs
This amount of work and resources can quickly get out of hand. That’s why in Datagran, we wanted to create a platform that offers Machine Learning systems with no engineering. We want companies to worry about understanding their business, their data and building models, not getting distracted by the Data OPs, MLOps and DeVOps. Businesses should focus on their business, not on building this type of complex infrastructure.
With that in mind let’s dive into what we built in Datagran for Real-Time data pipelines and bacth. This is a visual representation of the infrastructure we are offering to clients:
As represented in the above image, Datagran’s platform helps you to use your machine learning model both in batch processing in the Pipelines section AND real time processing by exposing it via API -- with the same model.
The process starts with storing the pickled model in Datagran's model store; in this section, you upload your pickled model.
The uploaded model appears on the Pipeline section -- which is used for batch processing. You can schedule the pipeline to run every week or day.
We also allow you to expose the model as API so you can consume it within your application when you need to use the model in real time environments (for example; recommendation engine). You just click on the "Deploy Model" button and Datagran deploys it so you can consume it with your client.
Here’s a full video demo on how to use the real time model.
In conclusion, creating your own infrastructure to have real-time and batch Machine Learning systems is hard. Data Scientists and IT teams should be focusing on their business instead of having to build complicated and heavy systems.
Datagran offers a fully managed end to end Machine Learning solution that is ready to use without the need for building it in-house.
