Integrate azure load testing into CI

eng/load-tests/README.md

@@ -0,0 +1,136 @@
+# MCP Server Load Testing
+
+This directory contains the infrastructure for **Locust-based load testing** of the remote Azure MCP Server, integrated with **Azure Load Testing**. The load test pipeline is **on-demand only** — it never runs automatically.
+
+## How it works
+
+When you manually queue the pipeline (`eng/pipelines/load-test.yml`):
+
+1. **Builds** the MCP Server as a Docker image and pushes it to Azure Container Registry (ACR)
+2. **Deploys** an ephemeral Azure Container App running the server with `--dangerously-disable-http-incoming-auth` (no OAuth token required)
+3. **Runs** a Locust-based load test via Azure Load Testing against the live Container App
+4. **Tears down** the Container App and removes the image tag from ACR
+
+## What is tested?
+
+| Tool | Description |
+|------|-------------|
+| `azmcp_storage_account_get` | Lists / retrieves Azure Storage account details via the MCP Streamable HTTP protocol |
+
+Each simulated Locust user follows the full MCP session lifecycle:
+
+1. **`initialize`** – establishes a session with the server
+2. **`notifications/initialized`** – signals the client is ready
+3. **`tools/call`** – repeatedly invokes `azmcp_storage_account_get`
+
+## Directory layout
+
+```
+eng/load-tests/
+├── README.md                   # This file
+├── locustfile.py               # Locust test script (MCP protocol)
+├── config.yaml                 # Azure Load Testing configuration
+└── load-test-resources.bicep   # Bicep template (ALT + Container App Environment + Container App)
+
+eng/pipelines/
+└── load-test.yml               # Azure DevOps pipeline definition (manual trigger only)
+```
+
+## Prerequisites
+
+1. **Azure Container Registry (ACR)** – to host the per-run Docker images.
+
+2. **Azure Load Testing resource + Container App Environment** – provisioned by the Bicep template on first run (the pipeline calls `az deployment group create`).
+
+3. **Azure DevOps variable group** – create a variable group named `mcp-load-test-vars` with:
+
+   | Variable | Secret? | Description |
+   |----------|---------|-------------|
+   | `AZURE_SERVICE_CONNECTION` | No | Azure service connection name |
+   | `AZURE_LOAD_TEST_RESOURCE_GROUP` | No | Resource group for all load-test resources |
+   | `AZURE_LOAD_TEST_NAME` | No | Name of the Azure Load Testing resource |
+   | `AZURE_ACR_NAME` | No | Azure Container Registry name (e.g. `mcploadtestacr`) |
+   | `MCP_SUBSCRIPTION` | No | Azure subscription for the storage tool call |
+
+   > **Note:** No `MCP_ACCESS_TOKEN` is needed — the server is deployed with auth disabled.
+
+## Provisioning infrastructure (first time)
+
+The Bicep template provisions (idempotently):
+- Azure Load Testing resource
+- Log Analytics workspace
+- Container App Environment
+- Container App (image reference passed as parameter)
+
+```bash
+# One-time: create the resource group and ACR
+az group create -n mcp-load-test-rg -l eastus2
+az acr create -n mcploadtestacr -g mcp-load-test-rg --sku Basic --admin-enabled true
+
+# The Bicep deployment happens automatically in the pipeline,
+# but you can also run it manually for testing:
+az deployment group create \
+  --resource-group mcp-load-test-rg \
+  --template-file eng/load-tests/load-test-resources.bicep \
+  --parameters \
+      mcpServerImage=mcploadtestacr.azurecr.io/azure-mcp-server:latest \
+      containerAppName=mcp-loadtest-manual \
+      acrName=mcploadtestacr
+```
+
+## Running locally (for development)
+
+You can run the Locust test locally against a development server:
+
+```bash
+# Start the MCP server in unauthenticated HTTP mode
+dotnet run --project servers/Azure.Mcp.Server/src -- server start \
+  --transport http \
+  --dangerously-disable-http-incoming-auth
+
+# In another terminal, run Locust
+pip install locust
+cd eng/load-tests
+MCP_SERVER_URL=http://localhost:5000 \
+MCP_SUBSCRIPTION=<your-subscription> \
+locust --headless -u 5 -r 1 -t 60s --host http://localhost:5000
+```
+
+Open http://localhost:8089 (without `--headless`) for the Locust web UI.
+
+## Triggering a load test
+
+The pipeline has **no automatic triggers** (no CI, no schedule, no PR trigger). To run it:
+
+1. Go to **Azure DevOps → Pipelines → load-test**
+2. Click **Run pipeline**
+3. Optionally override parameters (`engineInstances`, `users`, `spawnRate`, `duration`)
+4. Click **Run**
+
+## Failure criteria
+
+| Metric | Threshold |
+|--------|-----------|
+| Average response time | < 5 000 ms |
+| Error rate | < 10 % |
+
+If either threshold is exceeded, the pipeline fails. Adjust thresholds in [config.yaml](config.yaml).
+
+## Customising test parameters
+
+The pipeline exposes parameters that can be overridden at queue time:
+
+| Parameter | Default | Description |
+|-----------|---------|-------------|
+| `engineInstances` | 1 | Number of Azure Load Testing engine instances |
+| `users` | 10 | Concurrent simulated users |
+| `spawnRate` | 2 | Users spawned per second |
+| `duration` | 120 | Test duration in seconds |
+
+## Cleanup
+
+The pipeline's **Cleanup** stage runs unconditionally (`condition: always()`) and:
+- Deletes the Container App
+- Removes the image tag from ACR
+
+The Container App Environment and Azure Load Testing resource are long-lived and reused across runs.

eng/load-tests/config.yaml

@@ -0,0 +1,39 @@
+# Azure Load Testing configuration for MCP Server load tests.
+# Test type: Locust-based load testing
+# Target tool: azmcp_storage_account_get (list storage accounts)
+#
+# The pipeline deploys a standalone MCP Server Container App with auth disabled
+# (--dangerously-disable-http-incoming-auth) for each test run, so no OAuth
+# token is needed.
+#
+# Reference: https://learn.microsoft.com/azure/load-testing/how-to-create-manage-test#locust
+
+version: v0.1
+testId: mcp-server-storage-account-load-test
+displayName: MCP Server - Storage Account Get Load Test
+description: >
+  Locust-based load test that exercises the azmcp_storage_account_get tool
+  via the MCP Streamable HTTP protocol on an ephemeral Azure MCP Server
+  deployed as an Azure Container App (auth disabled).
+
+testType: Locust
+testPlan: locustfile.py
+
+engineInstances: 1
+
+failureCriteria:
+  - avg(response_time_ms) > 5000
+  - percentage(error) > 10
+
+env:
+  - name: MCP_SERVER_URL
+    value: ${MCP_SERVER_ENDPOINT}
+  - name: MCP_SUBSCRIPTION
+    value: ${MCP_SUBSCRIPTION}
+
+locustConfig:
+  users: 10
+  spawnRate: 2
+  runTime: 120s
+  host: ${MCP_SERVER_ENDPOINT}
+  locustfile: locustfile.py

eng/load-tests/load-test-resources.bicep

@@ -0,0 +1,139 @@
+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+// Bicep template for Azure Load Testing infrastructure.
+// Provisions:
+//   - Azure Load Testing resource (long-lived, reused across runs)
+//   - Log Analytics workspace (long-lived)
+//   - Container App Environment (long-lived)
+//   - Container App running the MCP Server with auth disabled (ephemeral, per-run)
+
+@description('Azure region for all resources.')
+param location string = resourceGroup().location
+
+@description('Name of the Azure Load Testing resource.')
+param loadTestName string = 'mcp-load-test'
+
+@description('Fully qualified Docker image reference for the MCP Server.')
+param mcpServerImage string
+
+@description('Name of the Container App (unique per pipeline run).')
+param containerAppName string
+
+@description('Name of the Azure Container Registry (without .azurecr.io).')
+param acrName string
+
+// ---------- Log Analytics ----------
+resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2023-09-01' = {
+  name: '${loadTestName}-logs'
+  location: location
+  properties: {
+    sku: {
+      name: 'PerGB2018'
+    }
+    retentionInDays: 30
+  }
+}
+
+// ---------- Container App Environment ----------
+resource containerAppEnv 'Microsoft.App/managedEnvironments@2024-03-01' = {
+  name: '${loadTestName}-env'
+  location: location
+  properties: {
+    appLogsConfiguration: {
+      destination: 'log-analytics'
+      logAnalyticsConfiguration: {
+        customerId: logAnalytics.properties.customerId
+        sharedKey: logAnalytics.listKeys().primarySharedKey
+      }
+    }
+  }
+}
+
+// ---------- Container App (MCP Server – auth disabled) ----------
+resource containerApp 'Microsoft.App/containerApps@2024-03-01' = {
+  name: containerAppName
+  location: location
+  identity: {
+    type: 'SystemAssigned'
+  }
+  properties: {
+    managedEnvironmentId: containerAppEnv.id
+    configuration: {
+      ingress: {
+        external: true
+        targetPort: 8080
+        transport: 'http'
+        allowInsecure: false
+      }
+      registries: [
+        {
+          server: '${acrName}.azurecr.io'
+          identity: 'system'
+        }
+      ]
+    }
+    template: {
+      containers: [
+        {
+          name: 'azmcp'
+          image: mcpServerImage
+          command: [
+            './azmcp'
+            'server'
+            'start'
+            '--transport'
+            'http'
+            '--port'
+            '8080'
+            '--dangerously-disable-http-incoming-auth'
+          ]
+          resources: {
+            cpu: json('1.0')
+            memory: '2Gi'
+          }
+          probes: [
+            {
+              type: 'Readiness'
+              httpGet: {
+                path: '/health'
+                port: 8080
+              }
+              initialDelaySeconds: 5
+              periodSeconds: 10
+            }
+            {
+              type: 'Liveness'
+              httpGet: {
+                path: '/health'
+                port: 8080
+              }
+              initialDelaySeconds: 10
+              periodSeconds: 30
+            }
+          ]
+        }
+      ]
+      scale: {
+        minReplicas: 1
+        maxReplicas: 1
+      }
+    }
+  }
+}
+
+// ---------- Azure Load Testing ----------
+resource loadTest 'Microsoft.LoadTestService/loadTests@2022-12-01' = {
+  name: loadTestName
+  location: location
+  identity: {
+    type: 'SystemAssigned'
+  }
+}
+
+// ---------- Outputs ----------
+@description('FQDN of the deployed MCP Server Container App.')
+output mcpServerFqdn string = containerApp.properties.configuration.ingress.fqdn
+
+@description('Resource ID of the Azure Load Testing resource.')
+output loadTestResourceId string = loadTest.id