---
title: MLOps (V8.0)
description: DataRobot Release 8.0 MLOps release notes
---

# MLOps (V8.0) {: #mlops-v80}

_March 14, 2022_


The DataRobot MLOps v8.0 release includes many new features and capabilities, described below. See also details of Release 8.0 in the [AutoML](v8.0.0-aml) and [time series (AutoTS)](v8.0.0-ats) release notes.

Release v8.0 provides updated UI string translations for the following languages:

* Japanese
* French
* Spanish
* Korean

##  New features and enhancements {: #new-features-and-enhancements }

See details of new features below:

**New deployment features**

* [Cancel retraining policies](#cancel-retraining-policies)
* [DataRobot MLOps library and third-party spooler types ](#datarobot-mlops-library-and-third-party-spooler-types)
* [Challenger accuracy](#challenger-accuracy)
* [Challenger insights](#challenger-insights)
* [Database integrations removed from integrations tab](#database-integrations-removed-from-integrations-tab)

**New prediction features**

* [Enhancements to Prediction Batch Job Definitions](#enhancements-to-prediction-batch-job-definitions)
* [Leaderboard Scoring Code enhancements](#leaderboard-scoring-code-enhancements)
* [Scoring Code in Snowflake](#scoring-code-in-snowflake)
* [Batch Prediction write support for Presto](#batch-prediction-write-support-for-presto)

<!--**New Model Registry features**-->

**New governance features**

* [Data drift separated into target monitoring and feature tracking]()

**Public preview features**

* [MLOps agent event log](#mlops-agent-event-log)
* [Multipart upload for batch prediction API](#multipart-upload-for-batch-prediction-api)


## New deployment features {: #new-deployment-features }

### Cancel retraining policies {: #cancel-retraining-policies }

To manage the automatic retraining of deployed models, you set up [retraining policies](set-up-auto-retraining#set-up-retraining-policies). The policies can be triggered manually or in response to a schedule, drift status, or accuracy status.  Now you can cancel policy runs that are in progress or scheduled.  You cannot cancel a run if it has finished successfully, has failed, has a status of "Creating challenger" or "Replacing model," or has already been cancelled.

![](images/rn-retrain-manage-policies.png)

### DataRobot MLOps library and third-party spooler types {: #datarobot-mlops-library-and-third-party-spooler-types }

The datarobot-mlops library no longer includes AWS (SQS) and RabbitMQ dependencies by default. If you are using these spooler types, you must install the spooler-specific dependencies See the documentation on [installing the DataRobot MLOps metrics reporting library](https://pypi.org/project/datarobot-mlops/) for details.

### Challenger accuracy {: #challenger-accuracy }

On a deployment's **Challengers** tab, the **Deployment Challengers** overview now includes an **Accuracy** column for the champion and every challenger. This column reports a model's accuracy score for the selected date range and, for challenger models, a comparison with the champion's accuracy score. You can use the **Accuracy metric** dropdown menu to compare different metrics.

![](images/rn-challenger-accuracy.png)

For more information on challenger accuracy comparison, see [Challenger models overview](challengers#challenger-models-overview).

### Challenger insights {: #challenger-insights }

Now generally available, the **Model Insights** on the **Model Comparison** tab allow you to compare the composition, reliability, and behavior of champion and challenger models using powerful visualizations. Choose two models to go head-to-head to determine if a challenger model outperforms the current champion and should replace the champion model in production.

After you select two models, DataRobot computes the following model comparisons for those models:

=== "Accuracy"

    The **Accuracy** list contains two columns to report accuracy metrics for each model. Highlighted numbers represent favorable values. In this example, the champion, **Model 1**, outperforms **Model 2** for most metrics shown:

    ![](images/challenger-compare-accuracy.png)

=== "Dual lift"

    A [dual lift chart](model-compare#dual-lift-chart) is a visualization comparing how two selected models underpredict or overpredict the actual values across the distribution of their predictions.

    ![](images/challenger-compare-dual-lift.png)

=== "Lift"

    A [lift chart](lift-chart) depicts how well a model segments the target population and how capable it is of predicting the target, allowing you to visualize the model's effectiveness.

    ![](images/challenger-compare-lift.png)

=== "ROC"

    !!! info "Availability information"
        The ROC tab is only available for binary classification projects.

    An [ROC curve](roc-curve) plots the true-positive rate against the false-positive rate for a given data source. Use the ROC curve to explore classification, performance, and statistics for the models you're comparing.

    ![](images/challenger-compare-roc.png)

=== "Predictions Difference"

    The **Predictions Difference** histogram shows the percentage of predictions that fall within the match threshold you specify in the **Prediction match threshold** field (along with the corresponding numbers of rows).

    ![](images/challenger-compare-predictions-diff-1.png)

    The list below the histogram shows the 1000 most divergent predictions (in terms of absolute value). The **Difference** column shows how far apart the predictions are.

    ![](images/challenger-compare-predictions-diff-2.png)

For more information on these challenger insights, see [Challenger model comparisons](challengers#challenger-model-comparisons).

### Database integrations removed from integrations tab {: #database-integrations-removed-from-integrations-tab }

To simplify the prediction database integration process, the **Database** section on the **Settings** > **Integrations** tab is now fully deprecated. This functionality is replaced by the **Prediction** > [**Job Definitions**](batch-pred-jobs) tab.

You can still find the [Qlik predictions integration](integration-code-snippets) code snippet, and with proper permissions the public preview [Tableau Analytics extension](tableau-extension), on the **Integrations** tab.

For more information on setting up prediction sources, see [Schedule recurring batch prediction jobs](batch-pred-jobs).

## New prediction features {: #new-prediction-features }

### Enhancements to Prediction Batch Job Definitions {: #enhancements-to-prediction-batch-job-definitions }

![](images/rn-batch-pred-job-def-list.png)

#### Disable a job definition  

In previous releases, you could disable a job description only by editing it and turning off the **Run this job automatically on a schedule** toggle.  Now, you can disable the description by selecting **Disable definition** in the action menu for a job definition. Jobs scheduled from that job definition will cease to run.  Select **Enable definition** to resume the jobs.

#### Clone a job definition

You can now create a copy of an existing job description and update it by selecting **Clone definition** in the action menu for a job definition. Update the fields as needed, and click **Save prediction job definition**. Note that the **Jobs schedule** settings are turned off by default.

#### Prediction source configurations

When you set a data source for a prediction job, DataRobot validates that the data is applicable for the deployed model. DataRobot also displays the user that configured the prediction source, the modification date, and a badge that represents the type of the source (in this case, STATIC).

![](images/rn-prediction-job-description-source-validation.png)

#### Select the default prediction instance

Now when you create a batch job definition, you can use the default prediction instance. The advanced options now include a **Use default prediction instance** toggle:

![](images/rn-prediction-job-default-pred-instance-on.png)

DataRobot checks that the default or previously selected prediction instance is accessible and valid, and if not, displays an error message.

If you turn off the toggle, you can select a different prediction instance:

![](images/rn-prediction-job-default-pred-instance-off.png)

#### Snowflake and Synapse connection improvements

The following improvements have been made to the prediction source and destination configurations for Snowflake and Synapse connections.

![](images/rn-prediction-source-snowflake.png)

The **Use external stage** options for Snowflake and Synapse are now optional. Toggling them off updates the connection to use a JDBC adapter directly.

You can now switch between a JDBC Snowflake connection setting and a Snowflake top-level connection without losing the connection details.

JDBC-Snowflake and  JDBC-Synapse connections now display as top-level connections, with the **Use external stage** option toggled off.

#### Batch job filtering

From the **Prediction Jobs** tab, you can now filter by prediction job ID, along with the existing filters: status, job type (based on the method used to generate the job), job start and end time, deployment, job definition ID, the prediction job ID, and the prediction environment.

![](images/rn-prediction-jobs-filter.png)

### Leaderboard Scoring Code enhancements {: #leaderboard-scoring-code-enhancements }

The Leaderboard Scoring Code functionality on the **Portable Predictions** page has been updated so that it is consistent with that of the **Portable Predictions** page for deployments. The page now includes the option of including [Prediction Explanations](pred-explain/index) in the Scoring Code, as well.

![](images/rn-scdl-1.png)

See [Download Scoring Code from the Leaderboard](sc-download-leaderboard) for details.

### Scoring Code in Snowflake {: #scoring-code-in-snowflake }

Now GA, you can use Scoring Code as a UDF in Snowflake. Bringing Scoring Code inside of the Snowflake database removes the need to extract and load data, resulting in a significant decrease in the time to score large data sets on comparable infrastructure.

See how to [generate UDF Scoring Code](snowflake-sc).

### Batch Prediction write support for Presto {: #batch-prediction-write-support-for-presto }

You can now write prediction data to Presto. To do so, set up Presto as a JDBC data connection. In your batch prediction job definition (**Predictions > Job Definitions**), select **JDBC** as the **Prediction destination**:

![](images/rn-batch-pred-destination-jdbc.png)

From the list of connectors, select the Presto connector:

![](images/rn-batch-pred-select-presto.png)

Enter credentials:

![](images/rn-batch-pred-presto-credentials.png)

Select the schema:

![](images/rn-batch-pred-presto-schema.png)

Select the output table or create a new table.

![](images/rn-batch-pred-presto-tables.png)

!!! note
    Presto requires the use of `auto commit: true` for many of the underlying connectors which can delay writes.

## New governance features

### Data drift separated into target monitoring and feature tracking {: #data-drift-separated-into-target-monitoring-and-feature-tracking }

To provide more granular control of data drift, accuracy, and fairness monitoring, the **Enable data drift tracking** setting on a deployment’s **Settings > Data** tab is now divided into two settings: **Enable target monitoring** and **Enable feature drift tracking**.

![](images/rn-split-data-drift.png)

You need to enable target monitoring to track accuracy ([Accuracy tab](deploy-accuracy)) and fairness ([Bias and Fairness tab](mlops-fairness)). Feature tracking must be enabled to monitor for data drift ([Data Drift tab](data-drift)). These settings are enabled by default. If you turn off either setting, you can still view historical data in the visualizations on the corresponding tabs.

## Public preview features

### MLOps agent event log {: #mlops-agent-event-log }

Now available for public preview, on a deployment's **Service Health** tab, you can view MLOps agent **Management** events (e.g., deployment actions) and **Monitoring** events (e.g., spooler channel events). Using **Monitoring Spooler Channel** error events, you can quickly diagnose and fix [spooler configuration](spooler) issues.

![](images/rn-mlops-agent-events.png)

To view **Monitoring** events, you must provide a `predictionEnvironmentID` in the agent configuration file (`conf\mlops.agent.conf.yaml`). If you haven't already installed and configured the MLOps agent, see the [Installation and configuration](agent) guide.

For more information on enabling and reading the MLOps agent event log, see the [documentation](agent-event-log).

### Multipart upload for batch prediction API {: #multipart-upload-for-batch-prediction-api }

Now available for public preview, multipart upload for the batch prediction API allows you to upload scoring data through multiple files to improve file intake for large datasets. The multipart upload process calls for multiple `PUT` requests followed by a `POST` request (`finalizeMultipart`) to finalize the upload manually.

This feature adds two new endpoints to the batch prediction API:

| Endpoint | Description |
|----------|-------------|
| `PUT /api/v2/batchPredictions/:id/csvUpload/part/0/` | Upload scoring data in multiple parts to the URL specified by `csvUpload`. Increment `0` by 1 in sequential order for each part of the upload. |
| `POST /api/v2/batchPredictions/:id/csvUpload/finalizeMultipart/` | Finalize the multipart upload process. Make sure each part of the upload has finished before finalizing. |

The feature adds two new intake settings for the local file adapter:

| Property | Type | Default | Description |
|-----------|------|---------|-------------|
| `intakeSettings.multipart` | boolean |`false` | <ul><li>`true`: Requires you to submit multiple files via `PUT` request and finalize the process manually via `POST` request (`finalizeMultipart`).</li><li>`false`: Finalizes intake after one file is submitted via `PUT` request.</li> |
| `intakeSettings.async` | boolean | `true` | <ul><li>`true`: Starts the scoring job when the initial `PUT` request for file intake is made.</li><li>`false`: Postpones the scoring job until the `PUT` request resolves or the `POST` request for `finalizeMultipart` resolves.</li></ul> |

!!! note
    For more information on the batch prediction API and local file intake, see [Batch Prediction API](batch-prediction-api/index) and [Prediction intake options](intake-options#local-file-streaming).

For more information on the multipart upload for batch predictions process, see the [public preview documentation](batch-pred-multipart-upload).

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
