Introducing MLSecOps for secure machine learning at scale

Naturally, given the complexity, sensitivity, and size of a typical enterprise’s software stack, security has always been a key concern for most IT teams. But in addition to the well-known security challenges facing development teams, organizations must also consider a new source of security challenges: machine learning (ML).

ML adoption is skyrocketing across all sectors, with McKinsey finding that by the end of last year, 56% of companies had adopted ML in at least one business function. However, in the race to adoption, many encounter the unique security challenges that come with ML, as well as challenges in responsibly deploying and using ML. This is especially true in newer contexts where machine learning is deployed at scale for use cases involving critical data and infrastructure.

Given the scale of potential disruption created by security breaches, security concerns for ML become particularly acute when the technology is operating in a live enterprise environment. At the same time, ML also needs to be integrated into the existing practices of IT teams and avoids being a source of bottlenecks and downtime for the business. With the AI ​​Responsible Use Principles, this means teams are changing their practices to build robust security practices into their workloads.

The rise of MLSecOps

To address these concerns, there are moves among machine learning practitioners to adapt the practices they have developed for developers and IT security for the deployment of ML at scale. For this reason, professionals in the industry are building a specialization that integrates security, devops and ML – Machine Learning Security Operations or “MLSecOps” for short. In practice, MLSecOps works to bring ML infrastructure, automation between development and operations teams, and security policies together.

But what challenges does MLSecOps actually solve? And how?

The rise of MLSecOps has been fueled by the growing importance of a wide range of security challenges facing the industry. To give a sense of the scope and nature of the issues MLSecOps has encountered in response, let’s cover two in detail: access to model endpoints and supply chain vulnerabilities.

model access

Unrestricted access to machine learning models at various levels poses major security risks. The first and more intuitive level of access to a model can be defined as “black box” access, namely the ability to draw inferences about the ML models. While this is key to ensuring models are used by different applications and use cases to deliver business value, having unrestricted access to consumption forecasts for a model can introduce various security risks.

An exposed model may be subject to an “enemy” attack. In such an attack, a model is reverse engineered to generate “adversary samples,” which are inputs to the model with added statistical noise. This statistical noise serves to trick a model into misinterpreting an input and predicting a different class than what is intuitively expected.

A textbook example of an enemy attack is the image of a stop sign. Adding hostile sounds to the image can trick an AI-powered self-driving car into believing it’s a completely different sign — say, a “Give Way” sign — when to a human it still looks like a stop sign looks like.

Then there is “white box” model access, which consists of accessing the internals of a model at various stages of machine learning model development. At a recent software development conference, we demonstrated how it is possible to inject malware into a model that can trigger arbitrary and potentially malicious code when deployed to production.

There are other challenges that can arise related to data leakage. Researchers have managed to reverse engineer training data from a model’s internally learned weights, which can lead to sensitive and/or personally identifiable information being leaked and potentially cause significant damage.

Vulnerabilities in the supply chain

Another security issue that ML faces is one that much of the software industry also faces, and that is the problem of the software supply chain. Ultimately, this problem is related to the fact that an enterprise IT environment is incredibly complex and requires many software packages to function. And often, a breach of just one of these programs in a company’s supply chain can jeopardize an otherwise perfectly secure facility.

In a non-ML context, consider the SolarWinds breach in 2020, in which much of the US federal government and corporate world was breached over a supply chain vulnerability. This has led to an increased urgency to strengthen the software supply chain across all sectors, especially given the role of open source software in the modern world. Moreover, even the White House is now holding high-level summits on the subject.

Just as vulnerabilities in the supply chain can create a security hole in any software environment, they can also attack the ecosystem around an ML model. In this scenario, the impact can be even worse, especially considering how much ML relies on open-source advances and how complex models can be, including the downstream supply chain of libraries they need to operate effectively.

For example, this month it was discovered that the long-established Ctx Python package in the PyPI open source repository was compromised with information-stealing code, with more than 27,000 copies of the compromised packages being downloaded.

Because Python is one of the most popular languages ​​for ML, supply chain tradeoffs like the Ctx violation are particularly pressing for ML models and their users. Any maintainer, contributor, or user of software libraries would at some point have faced the challenges posed by second-, third-, or fourth-level or higher dependencies that libraries bring—for ML, these challenges can become significantly more complex.

Where does MLSecOps come in?

Something the above two examples have in common is that while they are technical issues, they do not require new technology to be addressed. Instead, these risks can be mitigated by existing processes and people by placing high demands on both. I see this as the motivating principle behind MLSecOps – the centrality of strong processes to harden ML for production environments.

For example, while we’ve only covered two general areas specific to ML models and code, there are also a variety of challenges surrounding ML systems infrastructure. Authentication and authorization best practices can be used to protect model access and endpoints, and ensure they are only deployed as needed. For example, access to models can leverage multi-level permission systems that can mitigate the risk of malicious parties having both black-box and white-box access. The role of MLSecOps in this case is to build strong practices that harden model access while minimally impeding the work of data scientists and development teams, allowing teams to work much more efficiently and effectively.

The same is true in the software supply chain, where good MLSecOps encourage teams to put in place a process to periodically review their dependencies, update them as necessary, and act quickly once a potential vulnerability is reported. MLSecOps’ challenge is to develop these processes and integrate them into the daily workflows of the rest of the IT team, with the idea of ​​automating them as much as possible to reduce the time spent manually reviewing a software supply chain.

There are also a variety of challenges surrounding the infrastructure behind ML systems. But what these examples have hopefully shown us is that while no ML model and its associated environment can be made unhackable, most security breaches only happen due to a lack of best practices at various stages of the development lifecycle.

The role of MLSecOps is to intentionally introduce security into the infrastructure that oversees the end-to-end machine learning lifecycle, including the ability to identify those vulnerabilities, how to fix them, and how those mitigations fit into everyday life . today’s life of the team members.

MLSecOps is an emerging field where people working in and around it continue to explore and define the security gaps and best practices at every stage of the machine learning lifecycle. If you are an ML practitioner, now is an excellent time to contribute to the ongoing discussion as the field of MLSecOps continues to evolve.

Alejandro Saucedo is Technical Lead for Machine Learning at Seldon.