# ERCOT Forecast Error Analysis Agent

This directory contains a CrewAI-based agent packaged for deployment to DataRobot. The agent analyzes forecast errors in the ERCOT (Electric Reliability Council of Texas) electricity market by examining weather data, renewable generation, and energy market news.

## Directory Structure

```
custom_model/
├── __init__.py                  # Package initialization
├── agent.py                     # Main agent implementation with tools
├── custom.py                    # DataRobot entry points (load_model, chat)
├── helpers.py                   # Helper functions for responses and ToolClient
├── crewai_event_listener.py    # Event listener for CrewAI telemetry
├── model-metadata.yaml          # Model metadata and runtime parameters
└── README.md                    # This file
```

## Agent Components

### 1. Agent Definition (`agent.py`)

**MyAgent Class**: Main agent class that orchestrates the forecast error analysis

**Agent Properties**:
- `agent_forecast_analyst`: Energy Market Analyst that interprets ERCOT forecast errors
- `task_error_analysis`: Task that analyzes forecast errors using weather and news context

**Tools**:
- `fetch_texas_weather`: Fetches historical weather data from Open-Meteo API
  - Uses ERCOT hub locations to determine relevant Texas cities
  - Returns temperature, wind speed, and cloud cover data
  
- `fetch_energy_news`: Fetches relevant energy news from Tavily API
  - Searches for ERCOT, natural gas, and international energy news
  - Filters by date range around the forecast timestamp

### 2. Entry Points (`custom.py`)

**Functions**:
- `load_model(code_dir)`: Called when the model is loaded (required by DataRobot)
- `chat(completion_create_params, model)`: Main entry point for agent invocation
  - Handles both streaming and non-streaming responses
  - Initializes authorization context
  - Passes inputs to the agent

**Instrumentation**:
- OpenTelemetry instrumentation for monitoring and tracing
- CrewAI event tracking for agent interactions

### 3. Helper Functions (`helpers.py`)

**Classes**:
- `CustomModelChatResponse`: Extended ChatCompletion with pipeline interactions
- `CustomModelStreamingResponse`: Extended ChatCompletionChunk for streaming
- `ToolClient`: Client for interacting with other DataRobot Agent Tools

**Functions**:
- `to_custom_model_chat_response`: Converts agent response to chat format
- `to_custom_model_streaming_response`: Handles streaming responses
- `initialize_authorization_context`: Sets up auth for downstream services

## Configuration

### Runtime Parameters (`model-metadata.yaml`)

The following runtime parameters can be configured in DataRobot:

1. **LLM_DATAROBOT_DEPLOYMENT_ID** (optional)
   - DataRobot deployment ID for the LLM to use
   - If not set, uses the DataRobot LLM Gateway

2. **TAVILY_API_KEY** (required)
   - API key for Tavily news search service
   - Set this in DataRobot or via environment variable

### Environment Variables

- `DATAROBOT_API_TOKEN`: DataRobot API token (automatically set in DataRobot)
- `DATAROBOT_ENDPOINT`: DataRobot endpoint URL (automatically set in DataRobot)
- `TAVILY_API_KEY`: Tavily API key for news search

## Input Format

The agent expects inputs in the following format:

```json
{
  "hub_name": "HB_HOUSTON",
  "timestamp": "2025-10-22T15:00:00",
  "error": "25.50"
}
```

Additional context can be provided in the user message content.

## Output Format

The agent returns a narrative analysis with two sections:

1. **Narrative Analysis**: 5-8 sentences explaining the forecast error by connecting weather patterns, renewable generation, market conditions, and geopolitical factors

2. **APPENDIX Section**:
   - Weather Appendix: Weather data for relevant Texas locations
   - News Appendix: Relevant energy news articles with sources and dates

## Deployment to DataRobot

### Prerequisites

1. DataRobot account with access to Agentic Workflows
2. Tavily API key for news search
3. Python 3.10-3.12

### Deployment Steps

1. **Package the agent**:
   ```bash
   cd /path/to/newsanalystagent/agent_crewai
   ```

2. **Test locally** (optional):
   ```bash
   # Set environment variables
   export DATAROBOT_ENDPOINT="https://your-datarobot-instance.com"
   export DATAROBOT_API_TOKEN="your-token"
   export TAVILY_API_KEY="your-tavily-key"
   
   # Run the agent
   python run_agent.py
   ```

3. **Deploy to DataRobot**:
   - Use the DataRobot UI or CLI to create a new Custom Model
   - Upload the entire `agent_crewai` directory
   - Set the model type to "Agentic Workflow"
   - Configure runtime parameters (especially TAVILY_API_KEY)

4. **Test the deployment**:
   - Use the DataRobot Playground to test the agent
   - Send a sample forecast error for analysis

## Agent Workflow

1. **Input Processing**: Agent receives forecast error data with hub, timestamp, and error amount
2. **Weather Data Fetch**: Calls `fetch_texas_weather` tool to get historical weather for relevant locations
3. **News Data Fetch**: Calls `fetch_energy_news` multiple times with different queries:
   - ERCOT Texas electricity grid
   - Natural gas prices and energy markets
   - International energy politics and LNG
4. **Analysis Synthesis**: Agent synthesizes all data into a coherent narrative explanation
5. **Appendix Creation**: Agent formats weather and news data into appendices
6. **Response Return**: Complete analysis returned to caller

## Tools and APIs Used

- **Open-Meteo**: Free historical weather data API (no key required)
- **Tavily**: News search API (requires API key)
- **DataRobot LLM Gateway**: For language model interactions
- **CrewAI**: Agent framework for orchestration

## Differences from Template

The `bloggeragent` template was used as the foundation. Key modifications:

1. **Agent Purpose**: Changed from blog writing to ERCOT forecast analysis
2. **Tools**: Added custom weather and news fetching tools
3. **Input Format**: Changed to support forecast error data structure
4. **Output Format**: Modified to include narrative analysis with weather and news appendices
5. **Runtime Parameters**: Added TAVILY_API_KEY parameter

## Troubleshooting

### Weather API Errors
- Check that timestamp is in valid ISO format
- Ensure hub_name is one of: HB_HOUSTON, HB_NORTH, HB_SOUTH, HB_WEST, HB_BUSAVG

### News API Errors
- Verify TAVILY_API_KEY is set correctly
- Check that API key has sufficient credits

### LLM Connection Errors
- Verify DATAROBOT_ENDPOINT and DATAROBOT_API_TOKEN are set
- Check network connectivity to DataRobot instance

## License

Copyright 2025 DataRobot, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

