---
title: Batch monitoring for deployment predictions
description: View and manage monitoring statistics organized by batch job, instead of by time, for DataRobot and external models.
section_name: MLOps
maturity: public-preview

---

# Batch monitoring for deployment predictions {: #batch-monitoring-for-deployment-predictions }

!!! info "Availability information"
    Batch monitoring for deployment predictions is off by default. Contact your DataRobot representative or administrator for information on enabling this feature.

    <b>Feature flags:</b> Enable Deployment Batch Monitoring, Enable Batch Custom Metrics for Deployments

Now available for public preview, you can view monitoring statistics organized by batch, instead of by time. With batch-enabled deployments, you can create batches, add predictions to those batches, and view service health, data drift, and accuracy statistics for the batches in your deployment.

## Enable batch monitoring for a new deployment

When you initiate model deployment, the **Deployments** tab opens to the [deployment configuration](add-deploy-info) page:

![](images/deploy-create-settings-1.png)

To enable batch monitoring for this deployment, go to the **Advanced Service Health Configuration** section and click **Enable batch monitoring**:

![](images/deploy-create-batch-monitoring.png)

## Enable batch monitoring for an existing deployment

If you already created the deployment you want to configure for batch monitoring, navigate to that deployment's **Predictions** > **Settings** tab. Then, under **Predictions Settings** and **Batch Monitoring**, click **Enable batch monitoring** and **Save** the new settings:

![](images/deploy-settings-batch-monitoring.png)

## View a batch monitoring deployment in the inventory

When a deployment has batch monitoring enabled, on the **Deployments** tab, in the **Deployment Name** column, you can see a **BATCH** badge. For these deployments, in the [**Prediction Health** lens](deploy-inventory#prediction-health-lens), the **Service**, **Drift**, and **Accuracy** indicators are based on the last few batches of predictions, instead of the last few days of predictions. The **Activity** chart shows any batch predictions made with the deployment over the last seven days:

![](images/batch-monitoring-inventory-indicators.png)

To configure the number of batches represented by the **Prediction Health** indicators, you select a batch range on the deployment's settings tabs for [**Service Health**](service-health-settings) and [**Data Drift**](data-drift-settings):

=== "Service Health"

    On the **Service Health > Settings** tab, in the **Definition** section, select the **Range** of prediction batches to include in the deployment inventory's **Service** indicator and click **Save**:

    ![](images/batch-monitoring-service-health-range.png)

=== "Data Drift"

    On the **Data Drift > Settings** tab, in the **Definition** section, select the **Range** of prediction batches to include in the deployment inventory's **Drift** indicator and click **Save**:

    ![](images/batch-monitoring-data-drift-range.png)


## Make or schedule UI batch predictions

To make a batch and add predictions to it, you can [make batch predictions](batch-pred) or [schedule a batch prediction job](batch-pred-jobs) for your deployment. Each time a batch prediction or scheduled batch prediction job runs, a batch is created automatically, and every prediction from the batch prediction job is added to that batch.

=== "Make one-time batch predictions"

    On the **Predictions > Make Predictions** tab, you can [make one-time batch predictions](batch-pred) as you would for a standard deployment; however, for batch deployments, you can also configure the **Batch monitoring prefix** to append to the prediction date and time in the **Batch Name** on the **Batch Management** tab:

    ![](images/batch-monitoring-one-time-batch-pred.png)

=== "Schedule a recurring batch prediction job"

    On the **Predictions > Prediction Jobs** tab, you can click **+ Add job definition** to [create a recurring batch prediction job](batch-pred-jobs) as you would for a standard deployment; however, for batch deployments, you can also configure the **Batch monitoring prefix** to append to the prediction date and time in the **Batch Name** on the **Batch Management** tab:

    ![](images/batch-monitoring-batch-pred-job.png)

## Create batches

In a deployment with the **BATCH** badge, you can access the **Predictions > Batch Management** tab, where you can add batches. Once you have the **Batch ID**, you can assign predictions to that batch. You can create batches using the UI or API:

=== "Create a batch on the Batch Management tab"

    On the **Predictions > Batch Management** tab, click **+ Create batch**, enter a **Batch name**, and then click **Submit**:

    ![](images/batch-monitoring-create-batch.png)

=== "Create a batch via the API"

    The following Python script makes a DataRobot API request to create a batch on the **Batch Management** tab:

    ``` py title="create_batch.py" linenums="1" hl_lines="3 4 6 14" 
    import requests

    API_KEY = <api_key>
    DATAROBOT_KEY = <datarobot_key>

    BATCH_CREATE_URL = "https://app.datarobot.com/api/v2/deployments/<deployment_id>/monitoringBatches/"

    headers = {
        'Content-Type': 'application/json; charset=UTF-8',
        'Authorization': 'Bearer {}'.format(API_KEY),
        'DataRobot-Key': DATAROBOT_KEY,
    }

    requests_body = {'batchName': '<batch_name>'}

    response = requests.post(url=BATCH_CREATE_URL, headers=headers, json=request_body)

    print(response.json())

    ```

    Before running this script, define the `API_KEY`, `DATAROBOT_KEY`, `BATCH_CREATE_URL`, and `batchName` on the highlighted lines above. DataRobot recommends storing your secrets externally and importing them into the script.

You can now add predictions to this batch. To add predictions to a batch, on the **Predictions > Batch Management** tab, copy the **Batch ID**, and add it to a realtime prediction header or the monitoring agent.

## Manage batches

On the **Predictions > Batch Management** tab, in addition to creating batches, you can perform a number of management actions, and view your progress towards the daily batch creation limit for the current deployment. 

At the top of the **Batch Management** page, you can view the **Batches created** progress bar. The default batch creation limit is 100 batches _per deployment per day_. Below the progress bar, you can view the when your daily limit resets:

!!! note
    While the default is 100 batches _per deployment per day_, your limit may vary depending on your organization's settings. For Self-managed AI Platform installations, this limit does not apply unless specifically set by your organization's administrator. 

![](images/batch-monitoring-daily-limit.png)

!!! tip
    You can also view your progress towards the **Batches created** limit on the [**Usage** tab](deploy-usage). 

In the list below, you can view batch information in the table, including the **Batch Name**, **Batch ID**, **Creation Date**, **Earliest Prediction**, **Latest Prediction**, **Created By**, and number of **Predictions**.

![](images/batch-monitoring-management-info.png)

Click a batch to open the  **Info** tab and view and edit (:material-lead-pencil:) the **Batch Name**, **Description**, and **External URL**:

![](images/batch-monitoring-batch-info.png)

In addition, you can click the copy icon (:fontawesome-solid-copy:) next to the **Batch ID** to copy the ID for use in prediction code snippets:

![](images/batch-monitoring-batch-id-copy.png)

Click the **Model History** tab to view the **Model Id** and the configurable **Start Time** and **End Time** fields. You can edit (:material-lead-pencil:) the dates to identify predictions made on historical data, allowing the deployments charts to display prediction information for when the historical data is from, instead of displaying when DataRobot received the predictions:

![](images/batch-monitoring-batch-model-history.png)

In the **Actions** column, you can lock or delete batches, and view the prediction job associated with a batch:

![](images/batch-monitoring-batch-actions.png)

Action                        | Description
------------------------------|-------------
:material-lock-open:          | Lock the batch to prevent the addition of new predictions.
:fontawesome-solid-trash-can: | Delete the batch to remove faulty predictions.
:octicons-link-external-16:   | _For batches associated with a job_, opens the the **Deployments > Batch Jobs** tab to view the prediction job associated with that batch.

## Create batches with the batch CLI tools

To make batch predictions using the DataRobot command line interface (CLI) for batch predictions, download the CLI tools from the [**Predictions > Prediction API** tab](code-py) of your deployment. To locate it make sure the **Prediction Type** is set to **Batch** and the **Interface** is set to **CLI**, then click **download CLI tools** (or **Copy script to clipboard**):

![](images/batch-monitoring-cli-tools.png)

Open the downloaded `predict.py` file (or paste the code you copied into a python file) and add the highlighted code below to the appropriate location in the file, defining `monitoringBatchPrefix` to assign a name to your batches:

``` py title="predict.py" linenums="408" hl_lines="8"
    elif args.predictions_start_date:
        logger.info("Cannot use --predictions_start_date without --predictions_end_date")
        return 1
    elif args.predictions_end_date:
        logger.info("Cannot use --predictions_end_date without --predictions_start_date")
        return 1

    payload['monitoringBatchPrefix'] = 'Batch From predict.py'

    try:
        make_datarobot_batch_predictions(input_file, output_file, payload)
    except DataRobotPredictionError as err:
        logger.exception("Error: {}".format(err))
        return 1
```

You can now use the CLI as normal, and the batch predictions are added to a batch with the batch prefix defined in this file.

## Add real-time predictions to an existing batch

To make real-time predictions and add them to an existing batch, from the **Predictions > Batch Management** tab, copy the **Batch ID** and save it for use in the next step:

![](images/batch-monitoring-batch-id-copy.png)

Next, navigate to the [**Predictions > Prediction API** tab](code-py), set the **Prediction Type** to **Real-time** and the **Interface** to **Python**, and then copy the real-time Python code snippet into a Python file:

![](images/batch-monitoring-real-time-py.png)

On the highlighted line below, define `X-DataRobot-Model-Monitoring-Batch-Id` (with the **Batch ID** you saved above) in `headers` dict:

``` py linenums="57" hl_lines="10" 
headers = {
        # As default, we expect CSV as input data.
        # Should you wish to supply JSON instead,
        # comment out the line below and use the line after that instead:
        'Content-Type': 'text/plain; charset=UTF-8',
        # 'Content-Type': 'application/json; charset=UTF-8',
        
        'Authorization': 'Bearer {}'.format(API_KEY),
        'DataRobot-Key': DATAROBOT_KEY,
        'X-DataRobot-Model-Monitoring-Batch-Id' = '<batch_id>'
    }
```

Predictions made using this code snippet are added to the batch you defined in the header.

## Add agent-monitored predictions to an existing batch

To add predictions from an agent-monitored external model to a batch, you can follow the [batch monitoring example provided in the MLOps agent tarball](agent-ex). Before you run the example, on the **Predictions > Batch Management** tab, copy the **Batch ID** of the batch you want to add external predictions to, saving it for use as an input argument in the agent example.

![](images/batch-monitoring-batch-id-copy.png)

In the `BatchMonitoringExample/binary_classification.py` example script included in the MLOps agent tarball, the batch ID you provide as an input argument defines the `batch_id` in the `report_deployment_stats` and `report_predictions_data` calls on the highlighted lines below:

``` py title="binary_classification.py" linenums="118" hl_lines="5 35" 
 # MLOPS: report deployment metrics: number of predictions and execution time
    m.report_deployment_stats(
        predictions_array.shape[0],
        (end_time - start_time) * 1000,
        batch_id=args.batch_id,
    )

    target_column_name = columns[len(columns) - 1]
    predictions = []

    orig_labels = test_df[target_column_name].tolist()
    # Based on prediction value and the threshold assign correct label to each prediction
    target_labels = []
    for value in predictions_array.tolist():
        if len(value) == 1:
            # Random forest classifier from scikit-learn can return a single probability value
            # instead of 2 values.  You need to infer the other one before reporting predictions,
            # because, 'report_predictions_data' expects probability for each class.
            value.append(1 - value[0])
        predictions.append(value)
        if value[0] < prediction_threshold:
            target_labels.append("0.0")
        else:
            target_labels.append("1.0")

    # Write csv file with labels and association Id, when output file is provided
    if args.actuals_filename is not None:
        write_actuals_file(args.actuals_filename, orig_labels, association_ids)

    # MLOPS: report test features and predictions and association_ids
    m.report_predictions_data(
        features_df=test_df,
        predictions=predictions,
        association_ids=association_ids,
        batch_id=args.batch_id,
    )

    m.shutdown()


if __name__ == "__main__":
    main()
```

Predictions reported by the agent are added to the batch you defined in the code.

## Create batches and add real-time predictions

!!! info "Availability information"
    This method requires the **Enable Beta API Routes** permission.

To create a new batch and add real-time predictions to it, navigate to the [**Predictions > Prediction API** tab](code-py), set the **Prediction Type** to **Real-time** and the **Interface** to **Python**, and then copy the real-time Python code snippet into a Python file:

![](images/batch-monitoring-real-time-py.png)

Add the highlighted code into the file as shown below, defining `DATAROBOT_URL` with the URL of the application you use and `BATCH_NAME_PREFIX` with the intended batch name:

``` py linenums="27" hl_lines="11-46 86 87" 
DEPLOYMENT_ID = '<deployment_id>'

# Don't change this. It is enforced server-side too.
MAX_PREDICTION_FILE_SIZE_BYTES = 52428800  # 50 MB


class DataRobotPredictionError(Exception):
    """Raised if there are issues getting predictions from DataRobot"""


import datetime
DATAROBOT_URL = '<datarobot_url>'
BATCH_NAME_PREFIX = '<batch_name>'


def create_batch(deployment_id):
    batch_creation_endpoint = DATAROBOT_URL + 'api/v2/deployments/' + deployment_id + '/monitoringBatches/'


    batch_creation_headers = {
        'Content-Type': 'application/json; charset=UTF-8',
        'Authorization': 'Bearer {}'.format(API_KEY),
        'DataRobot-Key': DATAROBOT_KEY,
    }


    batch_name = BATCH_NAME_PREFIX + ' - ' + datetime.datetime.now().strftime("%m/%d/%Y, %H:%M:%S")
    batch_creation_payload = {
        'batchName': batch_name
    }


    batch_create_response = requests.post(url=batch_creation_endpoint,
        headers=batch_creation_headers,
        json=batch_creation_payload)


    if batch_create_response.status_code != 201:
        print("Error during batch creation")
        err_msg = '{code} Error: {msg}'.format(
        code=batch_create_response.status_code, msg=batch_create_response.text)
        raise DataRobotPredictionError(err_msg)


    new_batch_id = batch_create_response.json()['id']
    return new_batch_id


def make_datarobot_deployment_predictions(data, deployment_id):
"""
Make predictions on data provided using DataRobot deployment_id provided.
See docs for details:
http://127.0.0.1/docs/predictions/api/dr-predapi.html


Parameters
----------
data : str
    If using CSV as input:
    Feature1,Feature2
    numeric_value,string

    Or if using JSON as input:
    [{"Feature1":numeric_value,"Feature2":"string"}]

deployment_id : str
    The ID of the deployment to make predictions with.

Returns
-------
Response schema:
    https://staging.datarobot.com/docs/predictions/api/dr-predapi.html#response-schema

Raises
------
DataRobotPredictionError if there are issues getting predictions from DataRobot
"""
# Set HTTP headers. The charset should match the contents of the file.
headers = {
    # As default, we expect CSV as input data.
    # Should you wish to supply JSON instead,
    # comment out the line below and use the line after that instead:
    'Content-Type': 'text/plain; charset=UTF-8',
    # 'Content-Type': 'application/json; charset=UTF-8',

    'Authorization': 'Bearer {}'.format(API_KEY),
    'DataRobot-Key': DATAROBOT_KEY,
}

batch_id = create_batch(deployment_id)
headers['X-DataRobot-Model-Monitoring-Batch-Id'] = batch_id

url = API_URL.format(deployment_id=deployment_id)
```

Real-time predictions made using the modified code snippet are added to a newly created batch each time you run the code.

## Monitoring for batch deployments

Batch deployments support batch-specific service health, data drift, and accuracy monitoring, in addition to [data export](data-export) and [custom metrics](custom-metrics). On each of these tabs, you can use the **Batch Name** selector to determine which batches are included:

=== "Service Health"

    ![](images/batch-monitoring-service-health-tab.png)

=== "Data Drift"

    ![](images/batch-monitoring-data-drift-tab.png)

=== "Accuracy"

    ![](images/batch-monitoring-accuracy-tab.png)

=== "Data Export"

    ![](images/batch-monitoring-data-export-tab.png)

=== "Custom Metrics"

    ![](images/batch-monitoring-custom-metrics-tab.png)

In the **Select batches** modal, click :material-chevron-right: to select a batch, moving it to the right (selected) column. You can also click :material-window-close: to remove a selected batch, returning it to the left (unselected) column.

![](images/batch-monitoring-batch-selector.png)
