Scribe Automation Toolkit

A Python-based toolkit for automating TIBCO Scribe integration maps. This project provides a framework to programmatically generate, validate, and deploy complex Salesforce-to-Salesforce integration logic, replacing manual UI work with reproducible code.

🚀 Quick Start

1. Prerequisites

• Python 3.12+ with pip
• Salesforce CLI (sf) installed and authenticated (for metadata retrieval)
• TIBCO Scribe API Access with Organization ID and Bearer Token
• IP Whitelisting - Add your IP address in Scribe Org Settings → Security

2. Installation

# Clone the repository
git clone <your-repo-url>
cd ScribeAutomation

# Create and activate virtual environment
python3 -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install dependencies
pip install -r scripts/requirements.txt

# Verify installation
python -c "import mcp; print(f'MCP installed: {mcp.__version__}')"

```bash
# Developer note (recommended): For active development, install the project in editable mode
pip install -e .

# After editable install you can run modules with package-style imports (recommended):
python -m scripts.preflight_map_checks --file outputs/my_map.json --server-validate

# Or import modules from Python:
python -c "import scripts.preflight_map_checks as pre; print(pre.__file__)"

3. Configuration

⚠️ Security Note: Credentials are strictly managed via environment variables. Never commit .env files.

# Copy the template
cp secrets/.env.template secrets/.env

# Edit secrets/.env with your credentials

Required environment variables:

• SCRIBE_ORG_ID - Your TIBCO Scribe Organization ID
• SCRIBE_TOKEN - Your API Bearer Token (found in Scribe UI > User Options > API Access)
• SCRIBE_SOLUTION_ID - The UUID of the target solution

4. Authenticate Salesforce CLI

# Authenticate to your Salesforce orgs (for metadata retrieval)
sf org login web -a [Alias] -r https://test.salesforce.com
sf org login web -a [Alias] -r https://test.salesforce.com

5. Running the MCP Server

This project exposes a Model Context Protocol (MCP) server, allowing AI assistants (like Claude Desktop or Cursor) to interact with the Scribe API directly.

Two server versions are available:

# FastMCP v2 Server 
python scripts/scribe_mcp_server_v2.py

# Legacy MCP v1 Server - maintained for compatibility
python scripts/scribe_mcp_server.py

The MCP server is configured in .vscode/mcp.json with both versions available:

• scribe-v2: FastMCP v2 server (recommended)
• scribe: Legacy v1 server

📂 Project Structure

ScribeAutomation/
├── README.md                    # This file 
├── docs/
│   ├── LLM_CONTEXT.md          # Complete technical reference (for AI agents)
│   └── CONTRIBUTING.md         # Workflow guide for AI agents
├── .copilot-instructions       # VS Code Copilot rules
├── .vscode/
│   └── mcp.json                # MCP server configuration
├── scripts/                    # Core toolkit
│   ├── scribe_client.py        # Scribe REST API wrapper
│   ├── base_map_builder.py    # Base class for map generation
│   ├── scribe_mcp_server.py   # MCP server implementation
│   ├── map_validator.py       # Map JSON validation
│   ├── connection_registry.py # Connection lookup utilities
│   └── requirements.txt       # Python dependencies
├── templates/                  # Reusable block templates
│   └── blocks/
│       ├── query.json
│       ├── foreach.json
│       ├── insert.json
│       └── ...
├── examples/                   # Reference data
│   ├── connections.json        # All Scribe connections with IDs/aliases
│   ├── minimal_map_template.json
│   └── scribe_api_swagger.json
├── outputs/                    # Generated map JSON files
└── secrets/                    # Credentials (gitignored)
    └── .env.template

🛠️ Usage

Generating a Map

Maps are generated using Python Builder scripts that extend BaseMapBuilder. The base class handles UUID generation, block linking, and formula construction automatically.

1. Create a builder script (e.g., scripts/build_my_integration.py)
2. Run the script to generate JSON:

   python scripts/build_my_integration.py

3. Output is saved to the outputs/ directory

Validating & Importing

Always validate generated JSON before importing it into Scribe:

from scripts.scribe_client import ScribeClient

client = ScribeClient()

# Validate logic against live Scribe environment
validation_result = client.validate_map_server(map_json)

# Import the map
result = client.import_map(map_json)

Preflight (recommended)

Use scribe_preflight_map to run the standardized preflight (local + server validation) and emit a machine-readable JSON report. This is useful for attaching evidence to approval messages.

Example (command line):

# Run preflight (server validation ON by default)
python scripts/mcp_client.py scribe_preflight_map '{"map_json": <your_map_json>, "server_validate": true}'

# Or run the CLI wrapper directly (no server validation)
python -m scripts.preflight_map_checks --file outputs/my_map.json --no-server-validate --pretty

Quick note: the preflight JSON includes summary.error_count and summary.valid — ensure error_count == 0 before asking for approval and importing the map.

Using the MCP Tools

The MCP server exposes these tools for AI assistants:

Scribe MCP Server:

Connection Management:

• scribe_list_connections - List all connections (with type filtering)
• scribe_get_connection - Get connection by name/alias/ID

Metadata Discovery:

• scribe_list_entities - List Salesforce objects
• scribe_get_entity_fields - Get field metadata with types
• scribe_get_entity_relationships - Get lookups and parent-child relationships
• scribe_get_connection_actions - List supported operations
• scribe_check_action_support - Check if operation is available
• scribe_analyze_migration_gap - Compare source/target entities

Map Management:

• scribe_list_maps - List all maps (with filters)
• scribe_get_map - Get full map JSON
• scribe_import_map - Import map from JSON
• scribe_import_map_from_file - Import from file path
• scribe_delete_map - Delete map (imported only)
• scribe_clone_map - Clone existing map
• scribe_export_maps - Export maps as templates
• scribe_get_exported_maps - Retrieve exported package
• scribe_get_map_relationships - Get map relationships

Validation:

• scribe_validate_map - Client-side validation
• scribe_validate_map_server - Server-side validation
• scribe_validate_formula - Test formula syntax

Helpers:

• scribe_get_best_practices - Get critical lessons learned
• scribe_get_pattern - Get pre-validated patterns

Vision-Based Map Generation (v2 only):

• scribe_extract_flowchart - Extract structure from flowchart images (JPG/PNG/PDF)
• scribe_create_map_from_spec - Generate map JSON from MapSpec

Command-Line MCP Client:

Use the flexible MCP client to call any tool from the command line:

# List all available tools
python scripts/mcp_client.py --list-tools

# Get a map by ID
python scripts/mcp_client.py scribe_get_map '{"map_id": 16031332}' --output map.json

# List all maps
python scripts/mcp_client.py scribe_list_maps '{}'

# Get connection details
python scripts/mcp_client.py scribe_get_connection '{"identifier": "MyConnection"}'

# Validate a map
python scripts/mcp_client.py scribe_validate_map '{"map_json": {...}}'

Salesforce MCP Server:

• run_soql_query - Execute SOQL queries
• list_all_orgs - List authenticated orgs
• Additional metadata, testing, and user management tools

🧪 Testing

The test suite uses pytest with markers for different test types.

Running Tests

# Run all tests
.venv/bin/pytest tests/

# Run unit tests only (no API required)
.venv/bin/pytest tests/ -m unit

# Run integration tests (requires API credentials)
.venv/bin/pytest tests/ -m integration

# Exclude slow tests
.venv/bin/pytest tests/ -m "not slow"

# Run with coverage
.venv/bin/pytest tests/ --cov=scripts --cov-report=html

Test Categories

Marker	Description
unit	Unit tests with no external dependencies
integration	Tests requiring Scribe API access
slow	Long-running tests (map operations)
mcp	MCP server functionality tests
vision	Vision-based extraction tests

Test Files

• test_integration_workflows.py - End-to-end workflow tests
• test_metadata_discovery.py - Entity relationship tests
• test_formula_validation.py - Formula validation tests
• test_map_operations.py - Map cloning and export tests
• test_validator.py - Map validator rule tests
• test_v2_*.py - FastMCP v2 server tests
• test_vision_extraction.py - Vision-based flowchart extraction tests

🛡️ Safety Mechanisms

To prevent accidental data loss, the toolkit includes strict safeguards:

"Imported" Prefix Rule

The API client refuses to delete or modify any map unless its name contains "Imported". Manually created production maps are read-only to this toolkit.

This is enforced automatically by the ScribeClient class.

📋 API Capabilities

Operation	Supported	Notes
List/Get Maps	✅	GET endpoints work normally
Create Maps	✅	POST to /maps/import (accepts array)
Delete Maps	✅	DELETE by map ID (imported maps only)
Clone Maps	✅	POST to /maps/{id}/clone
Run Maps	✅	POST to /maps/{id}/run
Validate Maps	✅	POST to /maps/{id}/validate
Update Maps	⚠️	PUT /maps/advanced/{id} for advanced maps only For imported maps: delete + re-import recommended
List Connections	✅	GET with metadata on demand
Get Entity Fields	✅	Includes field types, nullability, primary keys

📝 Important Notes

1. Imported maps are prefixed - Scribe automatically adds "Imported - " to imported map names. Don't include this prefix in your map name.

2. IP Whitelisting required - Add your IP address in Scribe Org Settings → Security before using the API.

3. UUIDs must be valid - All block and mapping IDs must be valid GUIDs/UUIDs. Use Python's uuid.uuid4().

4. Block linking matters - Ensure nextBlockId/prevBlockId/parentBlockId form valid chains.

5. Maps import as invalid initially - After import, maps often show valid: false. This is normal - validate them in Scribe UI or via the API before running.

6. Boolean values in formulas - Use lowercase string "true" or "false", not Python booleans.

🔗 Resources

• TIBCO Scribe Documentation
• Scribe API Reference
• Model Context Protocol (MCP) Specification

📚 For AI Assistants

If you're an AI agent working on this project:

1. Read docs/LLM_CONTEXT.md - Complete technical reference (organized in 5 parts)
2. Read .copilot-instructions - Mandatory pre-work checklist
3. Read docs/CONTRIBUTING.md - Workflow guide for contributors
4. Review scripts/template_map_builder.py - Reference implementation

Additional Documentation

• docs/VISION_WORKFLOW.md - Vision-based map generation guide
• docs/SCRIBE_MAP_PATTERNS.md - Common design patterns (10 patterns documented)
• docs/FLOWCHART_EXTRACTION_ANALYSIS.md - Extraction checklist and lessons learned
• templates/ - Reusable block and pattern templates

These documents contain critical patterns, helper methods, common mistakes to avoid, and complete API documentation.