Add sagemaker ops review

cloudformation/devops-agent-skill-policies.yaml

@@ -29,6 +29,7 @@ Metadata:
           - EnableDmsOperationReview
           - EnableAgentCoreObservabilitySetup
           - EnableAgentCoreOpsReview
+          - EnableSageMakerOpsReview
       - Label:
           default: Optional Resource Scoping
         Parameters:
@@ -124,6 +125,11 @@ Parameters:
     Description: AgentCore Operational Review skill (adds read-only bedrock-agentcore control-plane List/Get and ec2:DescribeSubnets for the multi-AZ check; observability-only mode needs none of these).
     AllowedValues: ['true', 'false']
     Default: 'true'
+  EnableSageMakerOpsReview:
+    Type: String
+    Description: SageMaker AI Operational Review skill (adds savingsplans:DescribeSavingsPlans for the Savings Plan check).
+    Default: 'false'
+    AllowedValues: ['true', 'false']
 
 Conditions:
   CreateNewRole: !Equals [!Ref ExistingRoleName, '']
@@ -136,6 +142,7 @@ Conditions:
   SkillDmsOperationReview: !Equals [!Ref EnableDmsOperationReview, 'true']
   SkillAgentCoreObservabilitySetup: !Equals [!Ref EnableAgentCoreObservabilitySetup, 'true']
   SkillAgentCoreOpsReview: !Equals [!Ref EnableAgentCoreOpsReview, 'true']
+  SkillSageMakerOpsReview: !Equals [!Ref EnableSageMakerOpsReview, 'true']
   HasRegionRestriction: !Not [!Equals [!Join ['', !Ref AllowedRegions], '']]
 
 Resources:
@@ -428,6 +435,29 @@ Resources:
                   - support.amazonaws.com
                   - ce.amazonaws.com
 
+
+  # sagemaker-ops-review: adds savingsplans:DescribeSavingsPlans for the Savings Plan check.
+  # Every other API the skill calls -- sagemaker List/Describe/ListTags, CloudWatch metric reads,
+  # application-autoscaling:Describe*, servicequotas:GetServiceQuota, Cost Explorer region
+  # discovery, and health:DescribeEvents/DescribeAffectedEntities -- is already covered by
+  # AIDevOpsAgentAccessPolicy. Without this policy the Savings Plan check reports
+  # "not evaluated -- permission not granted" and the other 19 checks run normally.
+  PolicySageMakerOpsReview:
+    Type: AWS::IAM::Policy
+    Condition: SkillSageMakerOpsReview
+    Properties:
+      PolicyName: DevOpsAgentSkill-SageMakerOpsReview
+      Roles:
+        - !If [CreateNewRole, !Ref DevOpsAgentRole, !Ref ExistingRoleName]
+      PolicyDocument:
+        Version: '2012-10-17'
+        Statement:
+          - Sid: SageMakerOpsReviewSavingsPlansRead
+            Effect: Allow
+            Action:
+              - savingsplans:DescribeSavingsPlans
+            Resource: '*'
+
 Outputs:
   DevOpsAgentRoleArn:
     Description: Role ARN to use with aws devops-agent associate-service.
@@ -453,6 +483,7 @@ Outputs:
         - database-migration-service-expertise: ${EnableDmsOperationReview} (dms:TestConnection)
         - agentcore-observability-setup: ${EnableAgentCoreObservabilitySetup} (bedrock-agentcore:Get/ListAgentRuntime, xray:GetTraceSegmentDestination, logs:DescribeDeliveries/DeliverySources/DeliveryDestinations/ResourcePolicies, lambda:GetFunctionConfiguration, ecs:DescribeTaskDefinition/DescribeServices/ListTasks, eks:DescribeCluster)
         - agentcore-ops-review: ${EnableAgentCoreOpsReview} (bedrock-agentcore read-only List/Get for runtimes/memories/gateways/browsers/code-interpreters/workload-identities, ec2:DescribeSubnets)
+        - sagemaker-ops-review: ${EnableSageMakerOpsReview} (savingsplans:DescribeSavingsPlans)
       Skills covered by AIDevOpsAgentAccessPolicy (no extra policy needed):
         - eks-operation-review, enrich-with-aws-security-agent, crm-production-investigation-guidelines
       No IAM required:

custom-agents/aws-operation-review/CHANGELOG.md

@@ -1,5 +1,11 @@
 # Changelog
 
+## 1.1.0
+
+- Added Amazon SageMaker AI support via the `sagemaker-ops-review` skill — endpoints, training jobs, pipelines, notebooks, feature store, model registry, and Studio domains
+- Documented that SageMaker AI reviews need `sagemaker-ops-review` uploaded with "All agents" selected, and that `AIDevOpsAgentAccessPolicy` covers every API it calls except the optional `savingsplans:DescribeSavingsPlans`
+- Noted the `sagemaker-ops-review` report schema (eight pillars, verbatim AI Disclaimer, severity-ranked Executive Summary) in the report-schema deference guidance, and added a SageMaker artifact naming example
+
 ## 1.0.0
 
 - Initial version

custom-agents/aws-operation-review/README.md

@@ -2,30 +2,32 @@
 
 ## Purpose
 
-This custom agent performs comprehensive operational reviews of AWS services (EKS clusters, RDS instances, Aurora clusters) against best practices and the Well-Architected Framework. It identifies gaps in security, reliability, performance, cost optimization, and operational excellence, producing actionable recommendations and a structured report artifact.
+This custom agent performs comprehensive operational reviews of AWS services (EKS clusters, RDS instances, Aurora clusters, Amazon SageMaker AI workloads) against best practices and the Well-Architected Framework. It identifies gaps in security, reliability, performance, cost optimization, and operational excellence, producing actionable recommendations and a structured report artifact.
 
 ## Key Capabilities
 
 - Assesses EKS clusters for version currency, security posture, networking, logging, and node group configuration
 - Evaluates RDS/Aurora instances for engine versions, backup configuration, encryption, Multi-AZ, parameter compliance, and cost optimization
+- Reviews Amazon SageMaker AI workloads across eight pillars and twenty checks — endpoint encryption and VPC isolation, Studio domain network posture, endpoint autoscaling and staleness, service quota headroom, AWS Health lifecycle events, data capture, and Well-Architected guidance
 - Assigns severity levels (critical, high, medium, low) based on security exposure, blast radius, and operational risk
 - Generates a prioritized report with remediation steps and effort/impact estimates
 - Produces a persisted Markdown artifact for sharing with stakeholders
 
 ## Prerequisites
 
 - An AWS DevOps Agent space
-- IAM permissions for EKS read APIs (`eks:DescribeCluster`, `eks:ListClusters`, `eks:ListNodegroups`, `eks:DescribeNodegroup`, `eks:ListAddons`, `eks:DescribeAddon`) and/or RDS read APIs (`rds:DescribeDBInstances`, `rds:DescribeDBClusters`, `rds:DescribeDBParameterGroups`, `rds:ListTagsForResource`)
+- IAM permissions for EKS read APIs (`eks:DescribeCluster`, `eks:ListClusters`, `eks:ListNodegroups`, `eks:DescribeNodegroup`, `eks:ListAddons`, `eks:DescribeAddon`) and/or RDS read APIs (`rds:DescribeDBInstances`, `rds:DescribeDBClusters`, `rds:DescribeDBParameterGroups`, `rds:ListTagsForResource`). For SageMaker AI reviews, the managed `AIDevOpsAgentAccessPolicy` already covers every API the skill calls except `savingsplans:DescribeSavingsPlans`, which is an optional add-on — see the [sagemaker-ops-review prerequisites](../../skills/sagemaker-ops-review/README.md#2-iam-permissions)
 - The [eks-operation-review skill](../../skills/eks-operation-review/) uploaded to your Agent Space. Important note: for the skill to be used by the custom agent, choose "All agents" in the "Agent Type" field when importing the skill, even that the skill's README file instructs to choose specific agent types
 - The [rds-operation-review skill](../../skills/rds-operation-review/) uploaded to your Agent Space. Important note: for the skill to be used by the custom agent, choose "All agents" in the "Agent Type" field when importing the skill, even that the skill's README file instructs to choose specific agent types
+- For SageMaker AI reviews, the [sagemaker-ops-review skill](../../skills/sagemaker-ops-review/) uploaded to your Agent Space with "All agents" selected in the "Agent Type" field
 
 ## Creating the Agent
 
 1. In the DevOps Agent web app, go to the "Agents" menu (on the bottom left pane)
 2. Click "Create agent" (on the right side), then on the new menu that popped up, click "Form" (the left-most option)
 3. In the "Name" field, use "aws-operation-review"
 4. Copy the content of the "SYSTEM_PROMPT.md" file from this directory, and paste it into the "System prompt" field in the custom agent creation form
-5. In the "Skills" drop-down list, select both the "eks-operation-review" and "rds-operation-review" skills, and click "Create agent"
+5. In the "Skills" drop-down list, select the skills for the services you want to review — "eks-operation-review", "rds-operation-review", and/or "sagemaker-ops-review" — and click "Create agent"
 6. Now we need to add the `use_aws` and `use_kubectl` tools - in the new custom agent's window, click "Edit"
 7. In the new popped up window, select "Chat". A new chat will start on the left side. Wait for DevOps Agent to finish thinking, and it'll ask you what would you like to change
 8. Type "Add the use_aws and use_kubectl tools to this custom agent". Once the chat is finished, verify in the custom agent's page that both `use_aws` and `use_kubectl` are shown under "Tools" for this custom agent
@@ -39,4 +41,5 @@ Once finished, the artifact is persisted on the **Artifacts** page in the DevOps
 
 - [eks-operation-review skill](../../skills/eks-operation-review/) — domain knowledge for EKS cluster assessments
 - [rds-operation-review skill](../../skills/rds-operation-review/) — domain knowledge for RDS/Aurora database assessments
+- [sagemaker-ops-review skill](../../skills/sagemaker-ops-review/) — domain knowledge for Amazon SageMaker AI operational reviews
 - [AWS DevOps Agent custom agents documentation](https://docs.aws.amazon.com/devopsagent/latest/userguide/working-with-devops-agent-custom-agents-index.html)

custom-agents/aws-operation-review/SYSTEM_PROMPT.md

@@ -2,15 +2,16 @@ You are an AWS Operations Review Specialist focused on assessing AWS services ag
 
 ## Goal
 
-Perform comprehensive operational reviews of AWS services (EKS clusters, RDS instances, Aurora clusters, Bedrock workloads) to identify gaps in security, reliability, performance, cost optimization, and operational excellence — aligned with AWS best practices and the Well-Architected Framework.
+Perform comprehensive operational reviews of AWS services (EKS clusters, RDS instances, Aurora clusters, Bedrock workloads, Amazon SageMaker AI workloads) to identify gaps in security, reliability, performance, cost optimization, and operational excellence — aligned with AWS best practices and the Well-Architected Framework.
 
 ## Approach
 
-1. Identify which AWS service the user wants reviewed (EKS, RDS, Aurora, or Bedrock).
+1. Identify which AWS service the user wants reviewed (EKS, RDS, Aurora, Bedrock, or SageMaker AI).
 2. Load the appropriate skill for the service:
    - For EKS clusters: use the `eks-operation-review` skill methodology
    - For RDS/Aurora databases: use the `rds-operation-review` skill methodology
    - For Bedrock workloads: use the `bedrock-operation-review` skill methodology
+   - For Amazon SageMaker AI workloads (endpoints, training jobs, pipelines, notebooks, feature store, model registry, Studio domains): use the `sagemaker-ops-review` skill methodology
 3. Follow the skill's structured assessment framework to evaluate the resource.
 4. For each finding, assess severity (critical, high, medium, low) based on security exposure, blast radius, and operational risk.
 5. Generate actionable recommendations with clear remediation steps.
@@ -44,12 +45,14 @@ Generate a shareable report artifact as a Markdown document.
 its own artifact naming and report structure (including its own pillars/categories) in
 its Step "Generate Report" section — follow that schema exactly when a skill is loaded.
 For example, the `bedrock-operation-review` skill organizes findings by its five
-pillars (Security, Performance, Service Quotas, Cost Optimization, Resilience), not the
-generic categories below. Do not force a skill's findings into the generic category set.
+pillars (Security, Performance, Service Quotas, Cost Optimization, Resilience), and the
+`sagemaker-ops-review` skill organizes them by its eight pillars with a verbatim AI
+Disclaimer and a severity-ranked Executive Summary — not the generic categories below. Do
+not force a skill's findings into the generic category set.
 
 **Artifact naming:** use the naming defined by the selected skill. If the skill does not
 specify one, fall back to `<service>-review-<resource-name>-<YYYY-MM-DD>.md`.
-Examples: `eks-review-prod-cluster-2026-06-21.md`, `rds-review-orders-db-2026-06-21.md`, `bedrock-review-1234567890-us-east-1-2026-08-21.md`
+Examples: `eks-review-prod-cluster-2026-06-21.md`, `rds-review-orders-db-2026-06-21.md`, `bedrock-review-1234567890-us-east-1-2026-08-21.md`, `sagemaker-review-1234567890-us-east-1-2026-09-18.md`
 
 **Report structure (fallback):** use the following only when the selected skill does not
 define its own report structure. When it does, the skill's structure takes precedence.

llms.txt

@@ -40,12 +40,13 @@ Tools can be used with these AWS DevOps Agent types:
 - [AgentCore Operational Review Skill](skills/agentcore-ops-review/SKILL.md): Read-only operational review of Amazon Bedrock AgentCore resources aligned with the AWS Well-Architected Framework, discovering runtimes, memories, gateways, browsers, code interpreters, and workload identities and assessing runtime resilience, gateway health, memory and knowledge effectiveness, and resource utilization from control-plane and CloudWatch signals, degrading missing signals to documented visibility limits rather than false findings
 - [RDS/Aurora Database Diagnostics Skill](skills/database-rds-devops/SKILL.md): Runs database-level data-plane diagnostics for Aurora MySQL and Aurora PostgreSQL via predefined read-only health check queries over the RDS Data API, covering buffer pool, connections, locks, replication, storage, performance, and index efficiency, using the rds-aidba MCP server
 - [Investigation Cost Guardrail Skill](skills/investigation-cost-guardrail/SKILL.md): Estimates and caps the cost of paid API calls during investigations across all AWS services and native agent tools, enforcing per-investigation budgets, flagging expensive operations, requiring time windows, and cancelling when thresholds are exceeded
+- [SageMaker AI Operational Review Skill](skills/sagemaker-ops-review/SKILL.md): Performs read-only Amazon SageMaker AI operational reviews across eight pillars and twenty checks — security, performance, cost optimization, service quotas, resiliency, operational excellence, sustainability, and Well-Architected best practices — producing severity-ranked findings with one recommendation per High or Medium finding
 
 ## Available Custom Agents
 
 - [AWS Health Report](custom-agents/aws-health-report/README.md): Generates a report of AWS Health events (service issues, scheduled changes, account notifications) over a configurable period, grouped by service and category
 - [Support Cases Report](custom-agents/support-cases-report/README.md): Generates a consolidated report of AWS Support cases over a configurable period, highlighting recurring patterns and items requiring follow-up
-- [AWS Operation Review](custom-agents/aws-operation-review/README.md): Performs comprehensive operational reviews of AWS services (EKS, RDS, Aurora) against best practices and the Well-Architected Framework, producing a structured report artifact
+- [AWS Operation Review](custom-agents/aws-operation-review/README.md): Performs comprehensive operational reviews of AWS services (EKS, RDS, Aurora, SageMaker AI) against best practices and the Well-Architected Framework, producing a structured report artifact
 - [Service Quotas Monitor](custom-agents/service-quotas-monitor/README.md): Proactively monitors AWS service quotas across active regions, flags quotas at 85%+ utilization, and requests increases or escalates via support cases
 - [Redshift Support Specialist](custom-agents/redshift-support-specialist/README.md): Amazon Redshift support agent for query optimization, operational reviews, and cost optimization, paired with the redshift-support-specialist skill
 

skills/sagemaker-ops-review/.skilleval.yaml

@@ -0,0 +1,9 @@
+# skill-eval audit configuration
+# See: https://github.com/aws-samples/sample-agent-skill-eval
+audit:
+  ignore:
+    # README.md alongside SKILL.md is intentional and required by this repo's
+    # contribution guide (README carries the non-production disclaimer,
+    # prerequisites, and upload steps). Matches the convention used by the
+    # other skills in this repository.
+    - STR-016