Go Convert

🔄 Convert third-party CI/CD pipelines and Harness v0 YAML to Harness v1 format

Overview

Go Convert provides comprehensive tooling to convert CI/CD pipeline configurations to the Harness pipeline format. It supports multiple platforms and offers flexible deployment options for different use cases.

🎯 Key Features

✅ Multi-Platform Support: Bitbucket, Drone, GitLab, Jenkins → Harness
✅ v0 to v1 Migration: Convert Harness legacy format to modern v1
✅ Batch Processing: Convert up to 100 items in a single API call
✅ Template Reference Mapping: Update nested template references during conversion
✅ Three Deployment Modes: Library, CLI, or HTTP Microservice
✅ Production Ready: Docker support, graceful shutdown, health checks

🚀 Three Ways to Use

Mode	Best For	Quick Start
📚 Go Library	Embedded in Go applications	`import "github.com/drone/go-convert"`
🖥️ Command Line	Local development & debugging	`./go-convert bitbucket pipeline.yml`
🌐 HTTP Microservice ⭐	Language-agnostic API access	`./scripts/start-service.sh`

HTTP Microservice

New in this release! Convert Harness v0 YAML (pipelines, templates, input sets) to v1 format via REST API with batch processing and template reference mapping support.

Architecture

┌─────────────────┐
│   Client App    │
│  (Any Language) │
└────────┬────────┘
         │ HTTP/JSON
         ▼
┌─────────────────────────────────┐
│   Go Convert Microservice       │
│  ┌────────────────────────────┐ │
│  │  POST /api/v1/convert/batch│ │
│  │  - Pipelines               │ │
│  │  - Templates               │ │
│  │  - Input Sets              │ │
│  │  - Template Ref Mapping    │ │
│  └────────────────────────────┘ │
└────────┬────────────────────────┘
         │
         ▼
┌─────────────────────────────────┐
│   Conversion Engine             │
│  - v0 → v1 Pipeline Converter   │
│  - Template Converter           │
│  - Input Set Converter          │
│  - Reference Replacer           │
└─────────────────────────────────┘

Quick Start

# Start the service
./scripts/start-service.sh

# Or using Make
make run

# Or using Docker
./scripts/start-docker.sh

Service starts on http://localhost:8090

API Endpoints

GET /healthz - Health check
POST /api/v1/convert/batch - Batch convert pipelines, templates, and input sets

Example Request

curl -X POST http://localhost:8090/api/v1/convert/batch \
  -H "Content-Type: application/json" \
  -d '{
    "items": [
      {
        "id": "pipeline-1",
        "entity_type": "pipeline",
        "yaml": "<v0 pipeline YAML>",
        "entity_ref_mapping": {
          "oldTemplateRef": "newTemplateRef_v1"
        }
      },
      {
        "id": "template-1",
        "entity_type": "template",
        "yaml": "<v0 template YAML>"
      }
    ]
  }'

Features

✅ Batch Processing: Convert up to 100 items per request
✅ Multi-Entity Support: Pipelines, templates (Pipeline/Stage/Step), and input sets
✅ Template Reference Mapping: Update nested template references during conversion
✅ Integrity Verification: SHA-256 checksums for all conversions
✅ Production Ready: Structured logging, graceful shutdown, health checks
✅ Container Ready: Docker and Kubernetes support
✅ Language Agnostic: REST API accessible from any language

When to Use Which Mode?

Use Case	Recommended Mode	Why?
Integrate into Go application	📚 Library	Direct function calls, type safety
Quick local conversion	🖥️ CLI	Simple, no setup required
Production API service	🌐 Microservice	Scalable, language-agnostic
Batch migrations	🌐 Microservice	Process 100s of files efficiently
CI/CD pipeline integration	🌐 Microservice	REST API, easy integration
Development & debugging	🖥️ CLI	Fast iteration, syntax highlighting

Documentation

QUICKSTART.md - Get the service running in 30 seconds
SERVICE.md - Complete API documentation and deployment guide
TECH_SPEC.md - Architecture and design details

Configuration

Environment Variable	Default	Description
`PORT`	8090	HTTP listen port
`LOG_LEVEL`	debug	Logging level (debug/info/warn/error)
`MAX_BATCH_SIZE`	100	Maximum items per batch request
`MAX_YAML_BYTES`	1048576	Maximum request body size (1MB)

Performance & Scalability

🚀 Lightweight: ~10MB binary, minimal memory footprint
⚡ Fast: Converts typical pipeline in <100ms
📦 Batch Optimized: Process up to 100 items per request
🔄 Stateless: Easily scale horizontally
🐳 Container Ready: Small distroless Docker image (~20MB)

Supported Conversions

Platform Migrations

Bitbucket Pipelines  ──┐
Drone CI             ──┤
GitLab CI            ──┼──➤  Harness v1 Pipeline
Jenkins              ──┤
GitHub Actions       ──┘

Harness v0 → v1 Migration

Harness v0 Pipeline  ──➤  Harness v1 Pipeline
Harness v0 Template  ──➤  Harness v1 Template (Pipeline/Stage/Step)
Harness v0 InputSet  ──➤  Harness v1 InputSet

Special Features:

🔗 Template reference mapping during conversion
✅ Validates converted YAML structure
📝 Generates SHA-256 checksums

Go Library Usage

Sample Usage

Sample code to convert a Bitbucket pipeline to a Harness pipeline:

import "github.com/drone/go-convert/convert/bitbucket"

converter := bitbucket.New(
	bitbucket.WithDockerhub(c.dockerConn),
	bitbucket.WithKubernetes(c.kubeConn, c.kubeName),
)
converted, err := converter.ConvertFile("bitbucket-pipelines.yml")
if err != nil {
	log.Fatalln(err)
}

Command Line Tools

This package provides command line tools for local development and debugging purposes. These command line tools are intentionally simple. For more robust command line tooling please use the harness-convert project.

Installation

git clone https://github.com/drone/go-convert.git
cd go-convert
go build

Bitbucket

Convert a Bitbucket pipeline:

./go-convert bitbucket samples/bitbucket.yaml

Convert a Gitlab pipeline and print the before after:

./go-convert bitbucket --before-after samples/bitbucket.yaml

Convert a Bitbucket pipeline and downgrade to the Harness v0 format:

./go-convert bitbucket --downgrade samples/bitbucket.yaml

Drone

Convert a Drone pipeline:

./go-convert drone samples/drone.yaml

Convert a Drone pipeline and print the before after:

./go-convert drone --before-after samples/drone.yaml

Convert a Drone pipeline and downgrade to the Harness v0 format:

./go-convert drone --downgrade samples/drone.yaml

Gitlab

Convert a Gitlab pipeline:

./go-convert gitlab samples/gitlab.yaml

Convert a Gitlab pipeline and print the before after:

./go-convert gitlab --before-after samples/gitlab.yaml

Convert a Gitlab pipeline and downgrade to the Harness v0 format:

./go-convert gitlab --downgrade samples/gitlab.yaml

Jenkins

Convert a Jenkinsfile:

./go-convert jenkins --token=<chat-gpt-token> samples/Jenkinsfile

Convert a Jenkinsfile and downgrade to the Harness v0 format:

./go-convert jenkins --token=<chat-gpt-token> --downgrade samples/Jenkinsfile

Syntax Highlighting

The command line tools are compatible with bat for syntax highlight.

./go-convert bitbucket --before-after samples/bitbucket.yaml | bat -l yaml

Microservice Deployment

Local Development

# Using scripts
./scripts/start-service.sh

# Using Makefile
make run-debug

# With custom configuration
PORT=9000 LOG_LEVEL=info make run

Docker

# Build image
make docker-build

# Run container
docker run -p 8090:8090 go-convert-service:latest

# Or use the script
./scripts/start-docker.sh

Kubernetes

apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-convert-service
spec:
  replicas: 3
  selector:
    matchLabels:
      app: go-convert-service
  template:
    metadata:
      labels:
        app: go-convert-service
    spec:
      containers:
      - name: go-convert-service
        image: go-convert-service:latest
        ports:
        - containerPort: 8090
        env:
        - name: LOG_LEVEL
          value: "info"
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8090
          initialDelaySeconds: 5
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /healthz
            port: 8090
          initialDelaySeconds: 3
          periodSeconds: 5

See SERVICE.md for complete deployment documentation.

Project Structure

go-convert/
├── cmd/
│   └── server/          # HTTP microservice entrypoint
├── convert/             # Conversion logic for various CI/CD platforms
│   ├── bitbucket/       # Bitbucket pipelines converter
│   ├── drone/           # Drone pipelines converter
│   ├── gitlab/          # Gitlab CI converter
│   ├── jenkins/         # Jenkins converter
│   └── v0tov1/          # Harness v0 to v1 converter
├── service/             # HTTP service implementation
│   ├── converter/       # v0→v1 conversion handlers
│   │   ├── pipeline.go  # Pipeline conversion
│   │   ├── template.go  # Template conversion (Pipeline/Stage/Step)
│   │   ├── inputset.go  # Input set conversion
│   │   └── template_refs.go  # Template reference replacement
│   ├── handler.go       # HTTP request handlers
│   ├── server.go        # HTTP server and middleware
│   └── request.go       # Request/response types
├── scripts/             # Launch scripts
│   ├── start-service.sh
│   ├── start-docker.sh
│   └── stop-service.sh
├── .vscode/             # VS Code launch configurations
├── .idea/               # IntelliJ run configurations
├── Dockerfile.service   # Docker build configuration
├── Makefile            # Build and run automation
├── QUICKSTART.md       # Quick start guide
├── SERVICE.md          # Service documentation
└── TECH_SPEC.md        # Technical specification

Development

Build

# Build CLI tool
go build -o go-convert

# Build microservice
go build -o go-convert-service ./cmd/server

# Or use Makefile
make build

Test

# Run tests
go test ./...

# With coverage
make test-coverage

Format

make fmt

Run with Live Reload

# Install air: make install-tools
make dev

IDE Integration

VS Code

Open project in VS Code
Press F5 to start debugging
Select "Launch go-convert Service"

Available configurations:

Launch go-convert Service (port 8090)
Launch go-convert Service (Custom Port 9000)
Attach to running service

IntelliJ IDEA / GoLand

Open run configurations dropdown
Select "Go Convert Service"
Click Run (▶️) or Debug (🐛)

Available configurations:

Go Convert Service
Go Convert Service (Port 9000)
Docker: Go Convert Service

API Examples

Health Check

curl http://localhost:8090/healthz
# Response: {"status":"ok"}

Convert Pipeline

curl -X POST http://localhost:8090/api/v1/convert/batch \
  -H "Content-Type: application/json" \
  -d '{
    "items": [{
      "id": "pipeline-1",
      "entity_type": "pipeline",
      "yaml": "pipeline:\n  identifier: test\n  name: Test Pipeline\n  stages: []"
    }]
  }'

Convert Template with Reference Mapping

curl -X POST http://localhost:8090/api/v1/convert/batch \
  -H "Content-Type: application/json" \
  -d '{
    "items": [{
      "id": "template-1",
      "entity_type": "template",
      "yaml": "template:\n  type: Stage\n  spec: {...}",
      "entity_ref_mapping": {
        "oldRef": "newRef_v1"
      }
    }]
  }'

Batch Conversion

curl -X POST http://localhost:8090/api/v1/convert/batch \
  -H "Content-Type: application/json" \
  -d @test_batch_with_mapping.json

Makefile Commands

Run make help to see all available commands:

Command	Description
`make help`	Show all available commands
`make build`	Build the service binary
`make run`	Build and run locally
`make run-debug`	Run with debug logging
`make test`	Run tests
`make test-coverage`	Run tests with coverage
`make docker-build`	Build Docker image
`make docker-run`	Run in Docker (background)
`make docker-stop`	Stop Docker container
`make health-check`	Check service health
`make example-request`	Send example request
`make clean`	Remove build artifacts

Troubleshooting

Common Issues

Port Already in Use

# Find what's using the port
lsof -i :8090

# Kill the process
kill -9 <PID>

# Or use a different port
PORT=9000 ./scripts/start-service.sh

Service Not Responding

# Check if service is running
curl http://localhost:8090/healthz

# Check logs (if running in Docker)
docker logs go-convert-service

# Restart the service
./scripts/stop-service.sh
./scripts/start-service.sh

Docker Build Fails

# Clean Docker cache
docker system prune -a

# Rebuild
make docker-build

Module Dependencies Issues

# Download dependencies
make mod-download

# Tidy dependencies
make mod-tidy

# Rebuild
make clean build

Quick Reference Card

🚀 Start Service

./scripts/start-service.sh          # Local
./scripts/start-docker.sh           # Docker
make run                            # Using Make

🧪 Test Service

curl localhost:8090/healthz         # Health check
make example-request                # Send test request

🛠️ Build & Test

make build                          # Build binary
make test                           # Run tests
make docker-build                   # Build Docker image

📚 Documentation

QUICKSTART.md - Get started in 30 seconds
SERVICE.md - Complete API documentation
TECH_SPEC.md - Architecture details

🔗 Useful Links

API Endpoint: http://localhost:8090/api/v1/convert/batch
Health Check: http://localhost:8090/healthz
Test Files: test_batch_request.json, test_batch_with_mapping.json

Contributing

We welcome contributions! Here's how:

Fork the repository
Create your feature branch
```
git checkout -b feature/amazing-feature
```
Make your changes
Test your changes
```
make test
make fmt
```
Commit your changes
```
git commit -m 'Add amazing feature'
```
Push to the branch
```
git push origin feature/amazing-feature
```
Open a Pull Request

Development Guidelines

Write tests for new features
Follow existing code style (run make fmt)
Update documentation as needed
Ensure all tests pass (make test)

Related Projects

harness-convert - Production-grade CLI tool
Harness Platform - https://harness.io

License

Apache 2.0 - See LICENSE file for details.

Support & Resources

📖 Documentation

Quick Start: QUICKSTART.md
Service Guide: SERVICE.md
Technical Spec: TECH_SPEC.md

🐛 Issues & Questions

Bug Reports: GitHub Issues
Feature Requests: GitHub Discussions

🎯 Production Use

For production-grade command line tooling, use harness-convert.

Made with ❤️ by the Harness Team

Documentation • Quick Start • Technical Spec

Name		Name	Last commit message	Last commit date
Latest commit History 642 Commits
.harness		.harness
cmd/server		cmd/server
command		command
convert		convert
currentStateSamples		currentStateSamples
internal		internal
samples		samples
scripts		scripts
service		service
.drone.yml		.drone.yml
.gitignore		.gitignore
Dockerfile.service		Dockerfile.service
LICENSE		LICENSE
Makefile		Makefile
QUICKSTART.md		QUICKSTART.md
README.md		README.md
SERVICE.md		SERVICE.md
TECH_SPEC.md		TECH_SPEC.md
go.mod		go.mod
go.sum		go.sum
main.go		main.go
server		server

Folders and files

Latest commit

History

Repository files navigation

Go Convert

Overview

🎯 Key Features

🚀 Three Ways to Use

📑 Table of Contents

HTTP Microservice

Architecture

Quick Start

API Endpoints

Example Request

Features

When to Use Which Mode?

Documentation

Configuration

Performance & Scalability

Supported Conversions

Platform Migrations

Harness v0 → v1 Migration

Go Library Usage

Command Line Tools

Installation

Microservice Deployment

Local Development

Docker

Kubernetes

Project Structure

Development

Build

Test

Format

Run with Live Reload

IDE Integration

VS Code

IntelliJ IDEA / GoLand

API Examples

Health Check

Convert Pipeline

Convert Template with Reference Mapping

Batch Conversion

Makefile Commands

Troubleshooting

Common Issues

Quick Reference Card

🚀 Start Service

🧪 Test Service

🛠️ Build & Test

📚 Documentation

🔗 Useful Links

Contributing

Development Guidelines

Related Projects

License

Support & Resources

📖 Documentation

🐛 Issues & Questions

🎯 Production Use

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages