This skill provides comprehensive guidance for monitoring deployed models, tracking performance metrics, detecting data drift, and managing model health.
\nMost common use case: Check deployment health and data drift
\ndeployment.get_service_stats(...) to review prediction volume/latencydeployment.get_feature_drift(...) / deployment.get_target_drift(...)get_service_stats_over_time(...) and drift periods to assess trendsExample: \"Check the health of deployment abc123 and report any data drift issues\"
\nUse this skill when you need to:
\nUser request: \"Check the health of deployment abc123 and report any data drift issues.\"
\nAgent workflow:
\nUser request: \"Set up alerts for deployment xyz789 to notify when feature drift exceeds 0.2.\"
\nAgent workflow:
\nThis skill guides you to use the DataRobot Python SDK directly. Install the SDK if needed:
\npip install datarobot\nUse these DataRobot SDK and MLOps API methods for monitoring:
\nDeployment Monitoring:
\ndeployment.get_service_stats(...) - Get service statistics (latency, volume, etc.)deployment.get_feature_drift(...) - Get feature drift metrics (returns FeatureDrift objects)deployment.get_target_drift(...) - Get target drift metrics (returns TargetDrift)deployment.get_prediction_results(...) - Retrieve recorded prediction results (if enabled)Model Performance:
\nmodel.metrics - Model performance metrics (dict of {metric: {partition: score}})model.get_roc_curve() - Get ROC curve for comparisonNote: Some monitoring features may require DataRobot MLOps API. See the Common Patterns section below for examples.
\nimport datarobot as dr\n\n# Initialize client\ndr.Client()\n\n# Get deployment\ndeployment = dr.Deployment.get(\"abc123\")\n\n# Get service stats (requires MLOps monitoring to be enabled)\n# ServiceStats exposes values via the .metrics dict, not as attributes.\nstats = deployment.get_service_stats()\nprint(f\"Prediction count: {stats.metrics['totalPredictions']}\")\nprint(f\"Mean response time (ms): {stats.metrics['responseTime']}\")\n\n# Get recorded prediction results (if available / enabled)\ntry:\n recent = deployment.get_prediction_results(limit=10)\n print(f\"Recent prediction results: {len(recent)}\")\nexcept Exception as e:\n print(f\"Prediction results not available: {e}\")\nimport datarobot as dr\n\n# Get deployment\ndeployment = dr.Deployment.get(\"abc123\")\n\n# Get feature drift (requires MLOps monitoring)\ntry:\n drifts = deployment.get_feature_drift()\n high = [d for d in drifts if (d.drift_score or 0) > 0.2]\n print(f\"Features with drift_score > 0.2: {len(high)}\")\n for d in high[:10]:\n print(f\"{d.name}: {d.drift_score}\")\nexcept Exception as e:\n print(f\"Feature drift requires MLOps monitoring: {e}\")\nRecommended thresholds:
\nAdjust thresholds based on your domain and use case sensitivity.
\nCommon errors and solutions:
\npip install datarobot\nimport datarobot as dr\n\ndr.Client()\nNote: Some monitoring features require DataRobot MLOps API access. Check your DataRobot plan for MLOps availability.
\n