---
title: MLOps and predictions (V9.1)
description: DataRobot Release 9.1 MLOps release announcements.
---

# MLOps and predictions (V9.1) {: #mlops-and-predictions-v91}

_July 31, 2023_

The DataRobot MLOps v9.1 release includes many new features and capabilities, described below. See additional details of Release 9.1 in the [data and modeling](v9.1.0-aml) and [code-first](v9.1.0-code) release announcements.

##  New features and enhancements {: #new-features-and-enhancements }

??? abstract "Features grouped by capability"

    Name       |  GA | Public Preview
    ---------- | ---- | ---
    **Predictions and MLOps** |  :~~:  | :~~:
    [Deployment creation workflow redesign](#deployment-creation-workflow-redesign) | ✔ |
    [Deployment settings redesign](#deployment-settings-redesign)| ✔ |
    [GitHub Actions for custom models](#github-actions-for-custom-models) | ✔ |
    [Assign training data to a custom model version](#assign-training-data-to-a-custom-model-version) | ✔ |
    [Prediction monitoring jobs](#prediction-monitoring-jobs) | ✔ |
    [Apache Spark API for Scoring Code](#apache-spark-api-for-scoring-code) | ✔ |
	[DataRobot provider for Apache Airflow](#datarobot-provider-for-apache-airflow) | ✔ |
    [Monitoring jobs for custom metrics](#monitoring-jobs-for-custom-metrics) |   | ✔ |
    [Timeliness indicators for predictions and actuals](#timeliness-indicators-for-predictions-and-actuals) |   | ✔ |
    [Versioning support in the Model Registry](#versioning-support-in-the-model-registry) |   | ✔ |
    [Extend compliance documentation with key values](#extend-compliance-documentation-with-key-values) |  | ✔ |
    [MLflow integration for the Model Registry](#mlflow-integration-for-the-datarobot-model-registry) |   | ✔ |
    [Automate deployment and replacement of Scoring Code in AzureML](#automate-deployment-and-replacement-of-scoring-code-in-azureml) | | ✔ |
    [MLOps reporting for unstructured models](#mlops-reporting-for-unstructured-models) | | ✔ |

## New GA features {: #new-ga-features }

### Deployment creation workflow redesign {: #deployment-creation-workflow-redesign }

Now generally available, the redesigned deployment creation workflow provides a better organized and more intuitive interface. Regardless of where you create a new deployment (the Leaderboard, the Model Registry, or the Deployments inventory), you are directed to this new workflow. The new design clearly outlines the capabilities of your current deployment based on the data provided, grouping the settings and capabilities logically and providing immediate confirmation when you enable a capability, or guidance when you’re missing required fields or settings. A new sidebar provides details about the model being used to make predictions for your deployment, in addition to information about the deployment review policy, deployment billing details (depending on your organization settings), and a link to the deployment information documentation.

![](images/deploy-create-settings-1.png)

For more information, see the [Configure a deployment](add-deploy-info) documentation.

### Deployment settings redesign {: #deployment-settings-redesign }

The new deployment settings workflow enhances the deployment configuration experience by providing the required options for each MLOps feature directly on the deployment tab for that feature. This new organization also provides improved tooltips and additional links to documentation to help you enable the functionality your deployment requires.

The new workflow separates the categories of deployment configuration tasks into dedicated settings on the following tabs:

* [Service Health](service-health-settings)
* [Predictions](predictions-settings)
* [Data Drift](data-drift-settings)
* [Accuracy](accuracy-settings)
* [Fairness](fairness-settings)
* [Humility](humility-settings)
* [Challengers](challengers-settings)
* [Retraining](retraining-settings)

The **Deployment > Settings** tab is now deprecated. During the deprecation period, a warning appears on the **Settings** tab to provide links to the new settings pages:

![](images/rn-deploy-settings.png)

In addition, on each deployment tab with a **Settings** page, you can click the setting icon to access the required configuration options:

![](images/rn-deploy-settings-1.png)

For more information, see the [Deployment settings](deployment-settings/index) documentation.

### GitHub Actions for custom models {: #github-actions-for-custom-models }

Now generally available, the custom models action manages custom inference models and their associated deployments in DataRobot via GitHub CI/CD workflows. These workflows allow you to create or delete models and deployments and modify settings. Metadata defined in YAML files enables the custom model action's control over models and deployments. Most YAML files for this action can reside in any folder within your custom model's repository. The YAML is searched, collected, and tested against a schema to determine if it contains the entities used in these workflows. For more information, see the [custom-models-action](https://github.com/datarobot-oss/custom-models-action){ target=_blank } repository.

The [quickstart](custom-model-github-action#github-actions-quickstart) example uses a [Python Scikit-Learn model template](https://github.com/datarobot/datarobot-user-models/tree/master/model_templates/python3_sklearn){ target=_blank } from the [datarobot-user-model](https://github.com/datarobot/datarobot-user-models/tree/master/model_templates){ target=_blank } repository. After you configure the workflow and create a model and a deployment in DataRobot, you can access the commit information from the model's version info and package info and the deployment's overview:

=== "Model version info"

    ![](images/pp-cus-model-github2.png)

=== "Model package info"

    ![](images/pp-cus-model-github4.png)

=== "Deployment overview"

    ![](images/pp-cus-model-github1.png)

For more information, see [GitHub Actions for custom models](custom-model-github-action).

### Assign training data to a custom model version {: #assign-training-data-to-a-custom-model-version }

To enable feature drift tracking for a custom model deployment, you must add training data. Currently, when you add training data, you assign it directly to the custom model. As a result, every version of that model uses the same data. In this release, the assignment of training data directly to a custom model is deprecated and scheduled for removal, replaced by the assignment of training data to each custom model version. To support backward compatibility, the deprecated method of training data assignment remains the default during the deprecation period, even for newly created models.

To assign training data to a custom model's versions, you must convert the model. On the **Assemble** tab, locate the **Training data for model versions** alert and click **Permanently convert**:

![](images/convert-custom-model.png)

!!! warning
    Converting a model's training data assignment method is a one-way action. It _cannot_ be reverted. After conversion, you can't assign training data at the model level. This change applies to the UI _and_ the API. If your organization has any automation depending on "per model" training data assignment, before you convert a model, you should update any related automation to support the new workflow. As an alternative, you can create a new custom model to convert to the "per version" training data assignment method and maintain the deprecated "per model" method on the model required for the automation; however, you should update your automation before the deprecation process is complete to avoid gaps in functionality.

After you convert the model, you can assign training data to a custom model version:

* If the model was already assigned training data, the **Datasets** section contains information about the existing training dataset. To replace existing training data, click the edit icon (![](images/icon-pencil.png)). In the **Change Training Data** dialog box, click the delete icon (![](images/icon-delete.png)) to remove the existing training data, then upload new training data.

* If the model version doesn't have training data assigned, click **Assign**, then, in the **Add Training Data** dialog box, upload training data.

When you create a new custom model version, you can **Keep training data from previous version**. This setting is enabled by default to bring the training data from the current version to the new custom model version:

![](images/cmodel-17.png)

For more information, see [Add training data to a custom model](custom-model-training-data) and [Add custom model versions](custom-model-versions).

### Prediction monitoring jobs {: #prediction-monitoring-jobs }

Now generally available, monitoring job definitions allow DataRobot to monitor deployments running and storing feature data, predictions, and actuals outside of DataRobot. For example, you can create a monitoring job to connect to Snowflake, fetch raw data from the relevant Snowflake tables, and send the data to DataRobot for monitoring purposes. The GA release of this feature provides a [dedicated API for prediction monitoring jobs](api-monitoring-jobs) and the ability to [use aggregation](ui-monitoring-jobs#set-aggregation-options) for external models with [large-scale monitoring enabled](agent-use#enable-large-scale-monitoring):

![](images/aggregation-options.png)

For more information, see [Prediction monitoring jobs](pred-monitoring-jobs/index).

### Apache Spark API for Scoring Code {: #apache-spark-api-for-scoring-code }

The Spark API for Scoring Code library integrates DataRobot Scoring Code JARs into Spark clusters. This update makes it easy to use Scoring Code in PySpark and Spark Scala without writing boilerplate code or including additional dependencies in the classpath, while also improving the performance of scoring and data transfer through the API.

This library is available as a [PySpark API](sc-apache-spark#pyspark-api) and a [Spark Scala API](sc-apache-spark#spark-scala-api). In previous versions, the Spark API for Scoring Code consisted of multiple libraries, each supporting a specific Spark version. Now, one library includes all supported Spark versions:

* The PySpark API for Scoring Code is included in the [`datarobot-predict`](https://pypi.org/project/datarobot-predict/){ target=_blank } Python package, released on PyPI. The PyPI project description contains documentation and usage examples.

* The Spark Scala API for Scoring Code is published on Maven as [`scoring-code-spark-api`](https://central.sonatype.com/artifact/com.datarobot/scoring-code-spark-api){ target=_blank } and documented in the [API reference](https://javadoc.io/doc/com.datarobot/scoring-code-spark-api_3.0.0/latest/com/datarobot/prediction/spark30/Predictors$.html){ target=_blank }.

For more information, see [Apache Spark API for Scoring Code](sc-apache-spark).

### DataRobot provider for Apache Airflow {: #datarobot-provider-for-apache-airflow  }

Now generally available, you can combine the capabilities of [DataRobot MLOps](mlops/index) and [Apache Airflow](https://airflow.apache.org/docs/){ target=_blank } to implement a reliable solution for retraining and redeploying your models; for example, you can retrain and redeploy your models on a schedule, on model performance degradation, or using a sensor that triggers the pipeline in the presence of new data. The DataRobot provider for Apache Airflow is a Python package built from [source code available in a public GitHub repository](https://github.com/datarobot/airflow-provider-datarobot){ target=_blank } and [published in PyPi (The Python Package Index)](https://pypi.org/project/airflow-provider-datarobot/){ target=_blank }. It is also [listed in the Astronomer Registry](https://registry.astronomer.io/providers/datarobot/versions/latest){ target=_blank }. The integration uses [the DataRobot Python API Client](https://pypi.org/project/datarobot/){ target=_blank }, which communicates with DataRobot instances via REST API.

![](images/airflow-datarobot-pipeline-dag.png)

For more information, see the [DataRobot provider for Apache Airflow](apache-airflow) quickstart guide.

## New public preview features {: #new-public-preview-features }

### Monitoring jobs for custom metrics {: #monitoring-jobs-for-custom-metrics }

Now available for public preview, monitoring job definitions allow DataRobot to pull calculated custom metric values from outside of DataRobot into the custom metric defined on the [Custom Metrics](custom-metrics) tab, supporting custom metrics with external data sources. For example, you can create a monitoring job to connect to Snowflake, fetch custom metric data from the relevant Snowflake table, and send the data to DataRobot:

![](images/monitoring-options-custom-metrics.png)

Public preview [documentation](custom-metric-monitoring-jobs).

**Feature flag:** Enable Custom Metrics Job Definitions

### Timeliness indicators for predictions and actuals {: #timeliness-indicators-for-predictions-and-actuals }

Deployments have several statuses to define the general health of a deployment, including [Service Health](service-health), [Data Drift](data-drift), and [Accuracy](deploy-accuracy). These statuses are calculated based on the most recent available data. For deployments relying on batch predictions made in intervals greater than 24 hours, this method can result in an unknown status value on the [Prediction Health indicators in the deployment inventory](deploy-inventory#prediction-health-lens). Now available for Public Preview, those deployment health indicators can retain the most recently calculated health status, presented along with _timeliness_ status indicators to reveal when they are based on old data. You can determine the appropriate timeliness intervals for your deployments on a case-by-case basis. Once you've enabled timeliness tracking on a deployment's **Usage > Settings** tab, you can view timeliness indicators on the [**Usage** tab](deploy-usage) and in the [**Deployments** inventory](deploy-inventory):

=== "Deployments inventory"

    View the **Predictions Timeliness** and **Actuals Timeliness** columns:

    ![](images/timeliness-columns.png)

=== "Usage tab"

    View the **Predictions Timeliness** and **Actuals Timeliness** tiles:

    ![](images/timeliness-tiles.png)

    Along with the status, you can view the **Updated** time for each timeliness tile.

!!! note
     In addition to the indicators on the **Usage** tab and the **Deployments** inventory, when a timeliness status changes to _Red / Failing_, a notification is sent through email or the [channel configured in your notification policies](web-notify).

Public preview [documentation](timeliness-status-indicators).

**Feature flag:** Enable Timeliness Stats Indicator for Deployments

### Versioning support in the Model Registry {: #versioning-support-in-the-model-registry }

The Model Registry is an organizational hub for various models used in DataRobot, where you can access models as deployment-ready model packages. Now available as a public preview feature, the **Model Registry > Registered Models** page provides an additional layer of organization to your models.

![](images/reg-models-page.png)

On this page, you can group model packages into _registered models_, allowing you to categorize them based on the business problem they solve. Registered models can contain:

* DataRobot, custom, and external models

* Challenger models (alongside the champion)

* Automatically retrained models.

Once you add registered models, you can search, filter, and sort them. You can also share your registered models (and the versions they contain) with other users.

Public preview [documentation](registry/index).

**Feature flag:** Enable Versioning Support in the Model Registry

### Extend compliance documentation with key values {: #extend-compliance-documentation-with-key-values }

Now available for public preview, you can create key values to reference in compliance documentation templates. Adding a key value reference includes the associated data in the generated template, limiting the manual editing needed to complete the compliance documentation. Key values associated with a model in the Model Registry are key-value pairs containing information about the registered model package:

![](images/key-values-tab.png)

When you [build custom compliance documentation templates](template-builder), you can include string, numeric, boolean, image, and dataset key values:

![](images/add-kv-to-template.png)

Then, when you [generate compliance documentation for a model package](reg-compliance) with a custom template referencing a supported key value, DataRobot inserts the matching values from the associated model package; for example, if the key value has an image attached, that image is inserted.

Public preview [documentation](reg-key-values).

**Feature flag:** Enable Extended Compliance Documentation

### MLflow integration for the DataRobot Model Registry {: #mlflow-integration-for-the-datarobot-model-registry }

The public preview release of the MLflow integration for DataRobot allows you to export a model from MLflow and import it into the DataRobot [Model Registry](registry/index), creating [key values](reg-key-values) from the training parameters, metrics, tags, and artifacts in the MLflow model. You can use the integration's command line interface to carry out the export and import processes:

```sh title="Import from MLflow"
DR_MODEL_ID="<MODEL_PACKAGE_ID>"

env PYTHONPATH=./ \
python datarobot_mlflow/drflow_cli.py \
  --mlflow-url http://localhost:8080 \
  --mlflow-model cost-model  \
  --mlflow-model-version 2 \
  --dr-model $DR_MODEL_ID \
  --dr-url https://app.datarobot.com \
  --with-artifacts \
  --verbose \
  --action sync
```

Public preview [documentation](mlflow-integration).

**Feature flag:** Enable Extended Compliance Documentation

### Automate deployment and replacement of Scoring Code in AzureML {: #automate-deployment-and-replacement-of-scoring-code-in-azureml }

Now available for public preview, you can create a DataRobot-managed AzureML prediction environment to deploy DataRobot Scoring Code in AzureML. With the [Managed by DataRobot option enabled](azureml-sc-deploy-replace#create-an-azure-prediction-environment), the model deployed externally to AzureML has access to MLOps management, including automatic Scoring Code replacement:

![](images/azure-pred-env-settings.png)

Once you've created an AzureML prediction environment, you can [deploy a Scoring Code-enabled model to that environment from the Model Registry](azureml-sc-deploy-replace#deploy-a-model-to-the-azure-prediction-environment):

![](images/azure-deploy-target.png)

Public preview [documentation](azureml-sc-deploy-replace).

**Feature flag:** Enable the Automated Deployment and Replacement of Scoring Code in AzureML

### MLOps reporting for unstructured models {: #mlops-reporting-for-unstructured-models }

Now available for public preview, you can report MLOps statistics for Python custom inference models [created in the Custom Model Workshop](custom-inf-model) with an **Unstructured (Regression)**, **Unstructured (Binary)**, or **Unstructured (Multiclass)** target type:

![](images/unstructured-model-reporting.png)

With this feature enabled, when you [assemble an unstructured custom inference model](unstructured-custom-models), you can [use new unstructured model reporting methods in your Python code](mlops-unstructured-models#unstructured-custom-model-reporting-methods) to report deployment statistics and predictions data to MLOps. For an example of an unstructured Python custom model with MLOps reporting, see the [DataRobot User Models repository](https://github.com/datarobot/datarobot-user-models/tree/master/model_templates/python3_unstructured_with_mlops_reporting){ target=_blank }.

Public preview [documentation](mlops-unstructured-models).

**Feature flag:** Enable MLOps Reporting from Unstructured Models

_All product and company names are trademarks&trade; or registered&reg; trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them_.
