MLOps in 2026 and the work of bringing AI to production

Imagine you train a model to anticipate how many products a store will sell. In tests, the predictions are good. You connect it to the ordering system and, for a few weeks, it helps adjust purchases. Then a promotion comes along, customer behavior changes, and estimates start to fail. The app is still responsive; The server does not show errors. However, the warehouse begins to fill up with products that no one buys.

That jump between a model that works in tests and a system that is still useful in the real world is the terrain of MLOps. To understand it, you have to look beyond the algorithm: what data it receives, how it is updated, who validates the changes and what happens when its predictions are no longer reliable.

The current ecosystem offers tools to solve each of these needs. Knowing its function allows you to choose a reasonable architecture, both for a first project and for a platform that operates many models. It also helps understand why language models and agents are extending this work.

What is MLOps and what does it add to DevOps

MLOps, short for Machine Learning Operations, brings together machine learning, software engineering, and operations practices to develop and maintain ML systems in production. Its objective is to be able to reconstruct how a model was obtained, introduce improvements with controls and detect when intervention is necessary.

This includes a way of working. The team preparing the data, training the model, and maintaining the service need to agree on what it means for the system to work well. In our store, reducing prediction error matters, but it also matters that the order arrives on time and that the model does not cause disproportionate purchases. There must be a person or team responsible for acting on an incident.

MLOps takes version control, automated testing, and collaboration between development and operations from DevOps. It adds a difficulty: some of the software's behavior is learned from the data. The same code, trained with other information, can generate a different model. Additionally, the environment can change without anyone modifying the application.

Therefore, software testing is still necessary, but it must be accompanied by controls on data and predictive quality. A service may respond quickly and return wrong results. The Google Cloud MLOps architecture guide precisely places these validations and monitoring within the development and operation process.

The cycle begins before training

Let's go back to the store. Before choosing an algorithm, you must decide what you want to predict, how far in advance and what information will be available when placing the order. If we train using data that we will only know after the sale, testing will give a misleading impression of performance. It is the so-called information leak or data leakage.

With the problem defined, the work connects data preparation with experimentation, training and evaluation. The candidate model is compared to a simple reference or to the version already in use. If it meets the requirements, it is registered and deployed. The predictions and actual results allow you to then review its behavior and decide whether to correct the data, change the model or retrain it.

Not all turns of that cycle end in a new deployment. A workout can fail tests, and an alert can uncover a problem in the data source. That ability to stop is part of the design.

Being able to reconstruct a result

Git saves the history of code and configuration. To relate to large data or model files, DVC maintains versioned references while files are kept in separate storage. This way we can identify which data set corresponds to a revision of the project without entering the entire data set into the Git repository.

The need to identify data exists from the first model. DVC is an option; They can also serve immutable snapshots or versions of tables, depending on the platform used. Saving only a query to a changing database does not necessarily allow you to reconstruct the original set.

During the experiment, MLflow Tracking records parameters, metrics, and artifacts, such as models or graphs. Weights & Biases allows you to track and compare runs with shared visualizations. Both tools help answer an everyday question: what configuration produced that result that looked so good.

Reproducing the training also requires preserving the relevant dependencies, transformations, and execution conditions. Setting a random seed helps, although it does not guarantee identical results with any hardware or library.

When we choose a candidate, a model registry organizes its versions and links them to their provenance. The MLflow Model Registry, for example, offers versions, tags, and aliases. This record must be connected to the deployment history: marking a version as approved does not in itself prove that it is the one that is serving requests.

Validate more than just a metric

In a demand prediction, it is advisable to evaluate with a temporal separation that respects the order of events. The model must face periods subsequent to those from which it learned. It is also interesting to review results by store or category: an average improvement can hide serious errors in the least sold products.

Validation includes expected inputs, missing values, and consistency of transformations between training and inference, that is, the timing of producing predictions. If during testing we calculate sales for the last seven days in one way and in production we do it another way, the model will receive different information than what it learned to interpret.

When many applications share variables, a feature store can help manage them. Feast, for example, makes it easy to access historical features for training and recent values for inference. Its usefulness increases when these variables have to be reused and their consistency controlled; For a single model, a shared and well-tested transformation process may be sufficient.

Automate the process with stopping criteria

A pipeline is a flow of tasks with dependencies: there is no point in evaluating a model before training it or training with data that has not passed quality checks. Orchestration adds scheduling, execution tracking, and failure management.

Apache Airflow fits into batch flows with defined tasks and dependencies. Prefect allows you to organize flows in Python with tracking and retries. Kubeflow Pipelines is aimed at ML flows composed of containerized steps, especially in Kubernetes environments. They are alternatives with different fits; It is not necessary to install all three.

Continuous integration, or CI, verifies project changes through testing. With a tool like GitHub Actions, a modification can trigger code testing, verify transformations, and build the package to be deployed. Continuous delivery prepares versions that can be published after approval; continuous deployment automates that last step as well. Both practices are usually grouped under the acronym CD.

Continuous training, or CT, also appears in ML: repeatably executing training when a condition is met, such as having new validated data. “Continuous” does not mean training without rest or automatically replacing the model that is in use.

For our store, the flow could run every week, checking data quality and comparing a candidate to the current version. If it gets too bad in an important category, it stops. If it passes the tests, it is available for approval. Thresholds and human intervention depend on the impact of an error and must be defined before automating the decision.

The infrastructure depends on how the model will be used

A daily forecast for picking orders can be run in batches: the process calculates the results, saves them, and terminates. A system that recommends products while someone browses needs to respond to each request with low latency. Other models run inside a device, close to where the data is captured.

This difference determines a good part of the infrastructure. Keeping an API on permanently doesn't do much if we only need a prediction file every morning.

For a simple service, FastAPI can expose an interface that receives input and returns predictions. Docker packages the application and its dependencies in a container, reducing differences between environments. The model can be included in the image or retrieved from versioned storage. In both cases you must identify exactly which version is loaded; system dependencies and GPU drivers still require attention.

Kubernetes manages distributed containerized applications: maintains replicas, replaces failing containers, and allows you to configure scaling. On that basis, KServe adds specialized capabilities to serve models. This infrastructure is useful when there are needs for scale and a team capable of operating it. A managed service or virtual machine can cover a smaller project.

Below that are networking, storage, permissions, and compute resources. Terraform It allows you to describe infrastructure using versioned files, review changes, and create environments consistently. Credentials require a secrets management system; They should not remain within the repository or the images.

You also have to design how to change versions. A canary deployment sends a portion of the traffic to the candidate before scaling it up. In shadow mode, the candidate receives a copy of the requests and their results are observed without using them to decide. Both strategies help verify real behavior. The rollback must recover a compatible combination of model, code and transformations, not just a weight file.

Monitor when the server says everything is fine

Infrastructure monitoring observes availability, errors, response times and resource consumption. Prometheus collects metrics and allows defining alerts; Visualization tools like Grafana help explore them. In an ML system it is also necessary to look at the quality of the inputs and predictions.

A change in the distribution of input data is known as data drift. For example, a promotion can increase the purchase rate of a certain product. Concept drift describes a change in the relationship between inputs and what we are trying to predict: signals that previously anticipated demand well may no longer do so. They are related phenomena, but not equivalent. Evidently explains this distinction and its implications for detection.

Evidently allows you to compare current data with a reference to detect distribution changes. A drift alert is a signal to investigate: it does not in itself prove that the model has worsened. Nor does the absence of drift guarantee correct predictions.

To measure the error we need to know the real result. In our example we will have to wait for sales and consider that running out of stock can hide demand. This delay forces us to combine early signals, such as missing data or strange values, with later quality evaluations.

Observability connects these signals with execution logs and versions in order to explain an incident. A useful alert should indicate what has changed, what part of the system it affects, and who should review it. If the problem is a broken sensor or a miscalculated column, automatically retraining can consolidate the error. First it is diagnosed; Then the answer is decided.

Cloud and costs of maintaining the system

Cloud platforms bring together many of these pieces. Amazon SageMaker AI offers managed services to build, train and deploy models; is the current name of the service formerly called Amazon SageMaker, today integrated into a broader platform. Azure Machine Learning covers the work cycle with training resources, pipelines and deployment.

At Google Cloud, the ecosystem known as Vertex AI has evolved into the Gemini Enterprise Agent Platform. The official documentation of name changes It includes this transition, which covers training services, pipelines, registration and inference. It is advisable to recognize both names when consulting tutorials and existing projects.

These platforms reduce the work of integrating and maintaining components, but the team is still responsible for data, evaluations, and publishing decisions. The choice also depends on the infrastructure already available, access requirements, and the ease of moving models and processes to another environment.

Comparing costs requires looking beyond the price of a machine. Computing hours, services that remain active without receiving requests, storage, transfers and observability matter. Operating proprietary tools also consumes engineering time. A managed service can be cheaper overall even if some of its rates are higher.

In the store, running predictions in batches and shutting down resources when finished can be a more valuable decision than optimizing a few milliseconds of inference. For other systems it will be essential to maintain available capacity. It is a good idea to measure the cost per useful unit, such as a thousand predictions or a completed task, along with quality and latency.

LLMOps and agents expand what we must control

Generative applications introduce another situation: the team can use an external model through an API without training it. Even so, you must evaluate changes and monitor their operation. LLMOps applies these principles to systems based on large language models.

Let's now think of an assistant who explains sales forecasts by consulting company documentation. If you use RAG, or retrieval augmented generation, you first find relevant information and then feed it into the model as context. The result will depend on the model, but also on the instructions, the documents recovered and the search settings.

Therefore, these components must be versioned, including the evaluation set and, where appropriate, the index and the embedding model that transforms documents into numerical representations for searching. If an answer contains a false fact, we need to distinguish between a mistaken recovery and a generation that deviated from the sources. Changing the model will not necessarily solve a problem of outdated documents.

The expansion of the tools is now visible: MLflow incorporates prompt, evaluation and trace management for generative applications and agents. A trace records the observable steps of an execution, such as model calls, queries, and tool usage, with their times and results. Allows you to locate where the flow fails; It is not equivalent to knowing the internal reasoning of the model.

Evaluate responses and actions

The evaluation must be based on representative tasks. The sales assistant is interested in checking if it retrieves the correct document, if it supports its statements and if it recognizes that it is missing information. Pre-deployment testing can be complemented by evaluations on a sample of real interactions. LangSmith documents both flows and how to turn production failures into new test cases.

Automatic rules are used to verify specific formats or conditions. Evaluators based on other models help evaluate open responses, but they can also make mistakes: their criteria need to be compared with human review. The LangSmith evaluation documentation addresses these different ways of evaluating.

If the assistant can create an order, we are also talking about an agent that acts on other systems. A well-written response does not guarantee that you have chosen the correct product or respected the permissions. You must observe tool calls, limit steps and expense, and require approval for impact actions when the case requires it. Retries should avoid duplicate orders.

In these applications, costs can accumulate over multiple calls per task. Context length, retries, and evaluations also count. Testing smaller models, reusing results when valid or reducing unnecessary calls allows you to find a measurable balance between quality, time and cost.

An architecture that the team can maintain

For a first demand model, a reasonable foundation might be Git, identifiable and recoverable data, experiment tracking, and a batch prediction process. From the beginning, validations are needed, a way to detect errors and a procedure to recover a previous version. The degree of sophistication can grow later.

As training or models grow, there are concrete reasons to add an orchestrator, shared registry, or deployment platform. If many teams repeat the same configuration, platform engineering can provide common paths to train and release with permissions, logging, and controls already in place.

Governance is incorporated into that daily work: documenting the intended use and limits of the model, preserving its provenance, controlling access and recording approvals. It also involves deciding what data is saved in the traces and for how long. Recording everything without criteria can expose sensitive information and make the operation more expensive.

Keep the model useful

The success of our store model will be verified in orders in the coming months, under promotions, seasonal changes and data errors. MLOps provides the means to observe that behavior and modify the system without losing the ability to explain what occurred.

The tools chosen may change. What needs to remain is the connection between the published version, the evidence that justified publishing it, and the signs that indicate whether it still works. For a project that comes out of the notebook, that is a good starting point: being able to reconstruct the result, detect a failure and respond to it. On that basis, an infrastructure can grow that accompanies the model during its real life.