Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions components/mcp/stack/kafka-access/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

namespace: kafka

resources:
- https://raw.githubusercontent.com/streamshub/streamshub-mcp/v0.1.0/install/strimzi-mcp/optional/role-sensitive.yaml
- https://raw.githubusercontent.com/streamshub/streamshub-mcp/v0.1.0/install/strimzi-mcp/optional/rolebinding-sensitive.yaml

labels:
- pairs:
app.kubernetes.io/part-of: streamshub-developer-quickstart
includeSelectors: false
6 changes: 6 additions & 0 deletions components/mcp/stack/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
apiVersion: kustomize.config.k8s.io/v1alpha1
kind: Component

resources:
- strimzi-mcp
- kafka-access
14 changes: 14 additions & 0 deletions components/mcp/stack/strimzi-mcp/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- https://github.com/streamshub/streamshub-mcp//install/strimzi-mcp/base?ref=v0.1.0

labels:
- pairs:
app.kubernetes.io/part-of: streamshub-developer-quickstart
includeSelectors: false

images:
- name: quay.io/streamshub/strimzi-mcp
newTag: "0.1.0"
106 changes: 106 additions & 0 deletions docs/overlays/mcp-metrics.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
+++
title = 'MCP + Metrics'
weight = 3
cpu_total = '3 CPU cores'
memory_total = '5.5 GiB'
+++

The MCP + Metrics overlay combines the core stack with both Prometheus monitoring and the StreamsHub MCP server.
The MCP server is configured to use Prometheus as its centralized metrics provider.

## Quick-Start Install

```shell
curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | OVERLAY=mcp-metrics bash
```

## Manual Install

If you prefer step-by-step control, the MCP + Metrics overlay uses `overlays/mcp-metrics`:

```shell
# Phase 1 — Operators and CRDs (includes Prometheus Operator)
kubectl apply --server-side --force-conflicts -k 'https://github.com/streamshub/developer-quickstart//overlays/mcp-metrics/base?ref=main'

# Optionally, wait for the operators to be ready
kubectl wait --for=condition=Available deployment/prometheus-operator -n monitoring --timeout=120s
kubectl wait --for=condition=Available deployment/strimzi-cluster-operator -n strimzi --timeout=120s
kubectl wait --for=condition=Available deployment/apicurio-registry-operator -n apicurio-registry --timeout=120s
kubectl wait --for=condition=Available deployment/streamshub-console-operator -n streamshub-console --timeout=120s

# Phase 2 — Operands, MCP server, Prometheus instance and monitors
kubectl apply -k 'https://github.com/streamshub/developer-quickstart//overlays/mcp-metrics/stack?ref=main'
```

## Uninstall

```shell
curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/uninstall.sh | OVERLAY=mcp-metrics bash
```

## What Gets Added

On top of the components provided by the core overlay, the MCP + Metrics overlay adds:

| Component | Namespace | Description |
|------------------------|------------------|----------------------------------------------------------|
| Prometheus Operator | `monitoring` | Manages Prometheus instances and monitors |
| Prometheus instance | `monitoring` | Collects and stores metrics (400Mi memory, 1 replica) |
| PodMonitors | `monitoring` | Scrape targets for Kafka brokers and Strimzi operators |
| ServiceMonitor | `monitoring` | Scrape target for StreamsHub Console operator |
| StreamsHub MCP server | `streamshub-mcp` | MCP server configured with Prometheus metrics provider |
| MCP PodMonitor | `monitoring` | Scrape target for MCP server metrics |
| Sensitive Role | `kafka` | Grants access to TLS certificates and pod metrics |

The overlay patches existing resources:

- Kafka — enables the [Strimzi Metrics Reporter](https://strimzi.io/docs/operators/latest/deploying#proc-metrics-kafka-str) on the `dev-cluster`
- Console — adds Prometheus as a metrics data source so the Console UI displays Kafka metrics
- MCP server — configured to use Prometheus as centralized metrics provider instead of direct pod scraping

## Resource Requirements

The MCP + Metrics overlay requires at least {{< param cpu_total >}} and {{< param memory_total >}} of allocatable cluster resources.
This includes the resources for the [core](core.md) stack plus the additional components listed above.

## Accessing the MCP Server

Port-forward to the MCP server:

```shell
kubectl port-forward -n streamshub-mcp svc/streamshub-strimzi-mcp 8085:8080
```

The MCP endpoint is available at `http://localhost:8085/mcp`.

## Accessing Prometheus

Port-forward to the Prometheus UI:

```shell
kubectl port-forward -n monitoring svc/prometheus-operated 9090:9090
```

Open [http://localhost:9090](http://localhost:9090) and navigate to **Status > Targets** to verify all scrape targets are up, including the MCP server.

## Verify the Installation

Confirm all components are running:

```shell
kubectl get deployment -n streamshub-mcp streamshub-strimzi-mcp
kubectl get prometheus -n monitoring
kubectl get podmonitor -n monitoring
```

The MCP server deployment should show `1/1` ready replicas. Prometheus should reach `Available` status.

## Troubleshooting

### MCP Server Not Starting

See the [MCP overlay troubleshooting](mcp.md#troubleshooting) section.

### Metrics Not Appearing

See the [Metrics overlay troubleshooting](metrics.md#metrics-not-appearing) section.
101 changes: 101 additions & 0 deletions docs/overlays/mcp.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
+++
title = 'MCP'
weight = 2
cpu_total = '3 CPU cores'
memory_total = '5 GiB'
+++

The MCP overlay extends the core stack with the StreamsHub MCP server, giving AI assistants (Claude Code, VS Code Copilot, etc.) read-only access to the Strimzi-managed Kafka cluster.

## Quick-Start Install

```shell
curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/install.sh | OVERLAY=mcp bash
```

## Manual Install

If you prefer step-by-step control, the MCP overlay uses `overlays/mcp` instead of `overlays/core`:

```shell
# Phase 1 — Operators and CRDs
kubectl apply --server-side --force-conflicts -k 'https://github.com/streamshub/developer-quickstart//overlays/mcp/base?ref=main'

# Optionally, wait for the operators to be ready
kubectl wait --for=condition=Available deployment/strimzi-cluster-operator -n strimzi --timeout=120s
kubectl wait --for=condition=Available deployment/apicurio-registry-operator -n apicurio-registry --timeout=120s
kubectl wait --for=condition=Available deployment/streamshub-console-operator -n streamshub-console --timeout=120s

# Phase 2 — Operands and MCP server
kubectl apply -k 'https://github.com/streamshub/developer-quickstart//overlays/mcp/stack?ref=main'
```

## Uninstall

```shell
curl -sL https://raw.githubusercontent.com/streamshub/developer-quickstart/main/uninstall.sh | OVERLAY=mcp bash
```

## What Gets Added

On top of the components provided by the core overlay, the MCP overlay adds:

| Component | Namespace | Description |
|------------------------|------------------|----------------------------------------------------------|
| StreamsHub MCP server | `streamshub-mcp` | MCP server for AI-assisted Kafka cluster management |
| Sensitive Role | `kafka` | Grants access to TLS certificates and pod metrics |

The MCP server provides read-only tools for inspecting and troubleshooting Strimzi-managed Kafka clusters via the [Model Context Protocol](https://modelcontextprotocol.io/).

## Resource Requirements

The MCP overlay requires at least {{< param cpu_total >}} and {{< param memory_total >}} of allocatable cluster resources.
This includes the resources for the [core](core.md) stack plus the MCP server listed above.

## Accessing the MCP Server

Port-forward to the MCP server:

```shell
kubectl port-forward -n streamshub-mcp svc/streamshub-strimzi-mcp 8085:8080
```

The MCP endpoint is available at `http://localhost:8085/mcp`.

## Verify the Installation

Confirm the MCP server is running:

```shell
kubectl get deployment -n streamshub-mcp streamshub-strimzi-mcp
```

The deployment should show `1/1` ready replicas. Check the health endpoint:

```shell
kubectl port-forward -n streamshub-mcp svc/streamshub-strimzi-mcp 8085:8080 &
curl -s http://localhost:8085/q/health/ready
```

## Troubleshooting

### MCP Server Not Starting

If the MCP server deployment is not becoming ready:

```shell
# Check pod status
kubectl get pods -n streamshub-mcp

# Check logs
kubectl logs -n streamshub-mcp deployment/streamshub-strimzi-mcp

# Verify Strimzi CRDs are installed
kubectl get crd kafkas.kafka.strimzi.io
```

**Common causes:**

- Strimzi operator not installed — the MCP server requires Strimzi CRDs to exist
- Insufficient cluster resources — check node resource availability
- Image pull errors — verify the cluster can pull from `quay.io`
4 changes: 2 additions & 2 deletions docs/prerequisites.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@ nodes:
node-labels: "ingress-ready=true"
extraPortMappings:
- containerPort: 80
hostPort: 80
hostPort: 8080
protocol: TCP
- containerPort: 443
hostPort: 443
hostPort: 8443
protocol: TCP
EOF
```
Expand Down
51 changes: 43 additions & 8 deletions install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -88,10 +88,19 @@ main() {
stack_path="overlays/${OVERLAY}/stack"
fi

# Determine which optional features are included in this overlay
local has_metrics=false
local has_mcp=false
case "${OVERLAY}" in
metrics) has_metrics=true ;;
mcp) has_mcp=true ;;
mcp-metrics) has_metrics=true; has_mcp=true ;;
esac

# Compute total steps based on overlay and verification
local total_steps=5
if [ "${OVERLAY}" = "metrics" ]; then
total_steps=6
if [ "$has_metrics" = true ]; then
total_steps=$((total_steps + 1))
fi
if [ -z "${SKIP_VERIFY}" ]; then
total_steps=$((total_steps + 1))
Expand Down Expand Up @@ -127,8 +136,8 @@ main() {
kubectl apply --server-side --force-conflicts -k "${base_url}"
echo ""

# --- Step: Wait for prometheus-operator (metrics overlay only) ---
if [ "${OVERLAY}" = "metrics" ]; then
# --- Step: Wait for prometheus-operator (metrics overlays only) ---
if [ "$has_metrics" = true ]; then
step=$((step + 1))
info "Step ${step}/${total_steps}: Waiting for prometheus-operator to be ready (timeout: ${TIMEOUT})..."
kubectl wait --for=condition=Available deployment/prometheus-operator \
Expand Down Expand Up @@ -203,7 +212,7 @@ main() {
verify_failed=true
fi

if [ "${OVERLAY}" = "metrics" ]; then
if [ "$has_metrics" = true ]; then
info " Waiting for Prometheus..."
if kubectl wait --for=condition=Available prometheus.monitoring.coreos.com/prometheus \
-n monitoring --timeout="${TIMEOUT}" 2>/dev/null; then
Expand All @@ -214,6 +223,17 @@ main() {
fi
fi

if [ "$has_mcp" = true ]; then
info " Waiting for StreamsHub MCP server..."
if kubectl wait --for=condition=Available deployment/streamshub-strimzi-mcp \
-n streamshub-mcp --timeout="${TIMEOUT}" 2>/dev/null; then
info " StreamsHub MCP server is ready"
else
warn " StreamsHub MCP server did not become ready within ${TIMEOUT}"
verify_failed=true
fi
fi

echo ""
fi

Expand All @@ -234,26 +254,38 @@ main() {
echo " - Apicurio Registry instance (namespace: apicurio-registry, storage: in-memory)"
echo " - StreamsHub Console operator (namespace: streamshub-console)"
echo " - StreamsHub Console instance (namespace: streamshub-console)"
if [ "${OVERLAY}" = "metrics" ]; then
if [ "$has_metrics" = true ]; then
echo " - Prometheus operator (namespace: monitoring)"
echo " - Prometheus instance (namespace: monitoring)"
echo " - Kafka metrics (PodMonitors) (namespace: monitoring)"
fi
if [ "$has_mcp" = true ]; then
echo " - StreamsHub MCP server (namespace: streamshub-mcp)"
fi
echo ""

if [ -z "${SKIP_VERIFY}" ] && [ "${verify_failed}" != "true" ]; then
echo "Access the Console:"
echo " kubectl port-forward -n streamshub-console svc/streamshub-console-console-service 8090:80"
echo " Then open http://localhost:8090"
if [ "$has_mcp" = true ]; then
echo ""
echo "Access the MCP server:"
echo " kubectl port-forward -n streamshub-mcp svc/streamshub-strimzi-mcp 8085:8080"
echo " MCP endpoint: http://localhost:8085/mcp"
fi
echo ""
elif [ -z "${SKIP_VERIFY}" ] && [ "${verify_failed}" = "true" ]; then
echo "Check operand status:"
echo " kubectl get kafka -n kafka"
echo " kubectl get apicurioregistry3 -n apicurio-registry"
echo " kubectl get console -n streamshub-console"
if [ "${OVERLAY}" = "metrics" ]; then
if [ "$has_metrics" = true ]; then
echo " kubectl get prometheus -n monitoring"
fi
if [ "$has_mcp" = true ]; then
echo " kubectl get deployment -n streamshub-mcp streamshub-strimzi-mcp"
fi
echo ""
echo "Some resources may still be starting up. Re-check after a few minutes."
echo ""
Expand All @@ -265,10 +297,13 @@ main() {
echo " kubectl get apicurioregistry3 -n apicurio-registry"
echo " kubectl get deployment -n streamshub-console console-operator"
echo " kubectl get console -n streamshub-console"
if [ "${OVERLAY}" = "metrics" ]; then
if [ "$has_metrics" = true ]; then
echo " kubectl get prometheus -n monitoring"
echo " kubectl get podmonitor -n monitoring"
fi
if [ "$has_mcp" = true ]; then
echo " kubectl get deployment -n streamshub-mcp streamshub-strimzi-mcp"
fi
echo ""
echo "Note: It may take several minutes for all resources to become ready."
echo ""
Expand Down
5 changes: 5 additions & 0 deletions overlays/mcp-metrics/base/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- ../../metrics/base
18 changes: 18 additions & 0 deletions overlays/mcp-metrics/stack/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

components:
- ../../../components/core/stack
- ../../../components/metrics/stack
- ../../../components/mcp/stack

resources:
- mcp-podmonitor.yaml

patches:
- target:
group: apps
version: v1
kind: Deployment
name: streamshub-strimzi-mcp
path: mcp-prometheus-patch.yaml
Loading
Loading