We look forward to presenting Transform 2022 in person again on July 19th and virtually from July 20th to 28th. Join us for insightful conversations and exciting networking opportunities. Register today!
Machine Learning (ML) is an invaluable asset for modern businesses across the board. However, when it comes to ML models, both B2C and B2B companies face the problem of delayed time to market. According to Algorithmia, the vast majority of companies need at least a month or more to develop and then deploy their ML model.
The reason for this is a complex and often very expensive two-stage process. Developing an ML model in and of itself can be a lengthy and potentially expensive process. However, what many companies often fail to recognize early on is that the initial phase must be followed by another, arguably more demanding phase – deployment. In this second phase, the finished model is transferred to production, tested and refined, and then scaled accordingly.
It is estimated that only about 10% of all companies have the experience, financial resources and technical know-how to put a new ML model into production within a week of its completion. Many struggle for up to a year, with at least 30% of all organizations taking at least three months after deployment. Exactly how long it takes depends largely on which of the three common model types the company chooses.
Stock, custom and custom adaptive models
Of the ML models currently available in the market, there are the following: generic models, custom models, and custom adaptive models.
Generic and custom models are fundamentally polar opposites. The difference is that generic models are both inexpensive and accurate, while custom models are both expensive and accurate. This is because generic models are designed to fit virtually any business in this industry. These are typically based on ResNet, BERT/GPT and similar off-the-shelf technologies. As a result, these models are affordable and reliable, but they’re also far from a perfect fit.
Custom-made products, on the other hand, are always tailored to the respective task and are therefore much more precise. However, due to their high development and maintenance costs, they also come with a much higher price tag. Those who start with a generic solution and then try to improve their ML model often venture beyond the model’s basic architecture. What they end up with is a custom model. A custom model that can be instantly scaled to broader business needs and eliminates most of the tedious post-deployment fine-tuning is a custom adaptive model.
An adaptive model is therefore a type of custom model with some advantages that generic models offer. Like all other custom models, adaptive models are developed with specific business needs in mind. Because of this, they are very accurate. At the same time, they don’t require the company to figure out MLops after the initial development stage. As a result, they function somewhat like generic models in the deployment and post-deployment phases with relatively low maintenance costs and a shorter time to market.
Choosing an ML model
Which model your company needs – i.e. whether the extra charge is worth it – depends on your specific situation. Your business may need something as simple as: B. Sending online orders to different warehouses depending on location. In this case, a generic ML model could be just the ticket, especially if you are a small business.
On the other hand, if it’s something specific like moderating content for an online community of doctors discussing medical devices, a custom model works better. What a generic ML model might see as inappropriate language—for example, mentions of genitals—is not only appropriate, but necessary in the context of medical discussions. In this case, the training model must be tailored to the individual needs of the company. And this tailor-made model can either be adaptive or not.
Let’s look at the pros and cons of each model:
Custom adaptive models
Custom ML models are expensive due to the often unforeseen costs before and after deployment. Because of these typically high start-up costs, some companies tend to avoid the customized option, opting instead for the less accurate but also less expensive generic route. How expensive a training model actually becomes depends on a number of factors, including the data labeling methodology chosen, which is reflected in the flexibility of the model, or lack thereof.
The following case illustrates a crowdsourced custom adaptive model in action, i.e. an adaptive model that relies on human-in-the-loop labeling:
A well-known company that offers an engineering editing environment wanted to improve the accuracy of their software and reduce the cost of training the model. The engineering team needed to find a more efficient solution for correcting English sentences. Each solution had to match an existing fully manual labeling pipeline.
The final solution was to use a pre-existing custom model for linguistic processing, customized to the client’s needs. Third-party AutoML was used for text classification within the target sentences. Subsequently, the accuracy of the phrase check increased by 6% – from 76% to 82%. This, in turn, reduced the model’s training costs by 3%. Furthermore, the customer did not have to make any additional investments – financial or otherwise – in the infrastructure of the model, as is usually the case with most custom models.
Important points to consider
Choosing the right ML model for your business can be a daunting task. Here’s a summary of what you should consider to make an informed decision:
- Consider how specific your needs are: As a rule of thumb, the more specific the needs are, the further you should stray from the generic model.
- Always consider scalability – if you know you’re going to need it, consider paying extra for something tailored just for you.
- If you don’t need high accuracy but need fast deployment, go for the generic route.
- If accuracy is important to you, consider how much time to market you can spare.
- If you are short on time and need high accuracy, consider Custom Adaptive Route. Otherwise, any custom solution might just as well serve your needs.
- In terms of overall cost, the generic route is the cheapest of all – followed by the custom adaptive route, which bypasses most of the MLops costs – and finally all other custom solutions, which can increase in cost significantly after deployment (the exact numbers differ affect strongly in individual cases).
- Consider whether you have in-house data scientists and MLEs available – if so, it may be feasible to go for the traditional, in-house developed custom option. if not – consider the other two (generic or custom adaptive).
- When choosing between custom versus custom adaptive options, consider how accurate and specific to your customers’ needs the ML model ultimately needs to be. The higher the accuracy and adaptability, the higher the cost and longer the waiting time to create and maintain the model.
Fedor Zhdanov is Head of ML Products at Toloka AI.
data decision maker
Welcome to the VentureBeat community!
DataDecisionMakers is the place where experts, including technical staff, working with data can share data-related insights and innovations.
If you want to read about innovative ideas and up-to-date information, best practices and the future of data and data technology, visit us at DataDecisionMakers.
You might even consider contributing an article of your own!
Read more from DataDecisionMakers