zoo-template-common v0.1.8

📦 zoo-template-common v0.1.8

Avoid inserting the user name twice in the tmpUrl variable of the main section.

🔗 Links

• Changelog: v0.1.7...v0.1.8

zoo-template-common v0.1.7

📦 zoo-template-common v0.1.7

Fix typo in v0.1.6.

🔗 Links

• Changelog: v0.1.6...v0.1.7

License: Apache-2.0

zoo-template-common v0.1.6

📦 zoo-template-common v0.1.6

Change the output handling to create a minimal catalog for the File type.

🔗 Links

• Changelog: https://github.com/ZOO-Project/zoo-template-common/compare/v0.1.5... v0.1.6

License: Apache-2.0

zoo-template-common v0.1.5

📦 zoo-template-common v0.1.5

Maintenance release: PEP 621 compliance and package structure improvements

🔄 What's Changed

• PEP 621 Pattern: Adopted PEP 621 standard for project metadata in pyproject.toml
• Package Cleanup: Removed unnecessary __init__.py file for better package structure
• Version Bump: Updated to version 0.1.5 in project configuration

📝 Commits since v0.1.4

• Use PEP 621 pattern (3af1096)
• Version bump, remove __init__ (d49da54)

📦 Installation from PyPI

Install directly from PyPI:

pip install zoo-template-common==0.1.5

Or update in your pyproject.toml:

[tool.poetry. dependencies]
zoo-template-common = ">=0.1.5"

🔗 Links

• PyPI Package: https://pypi.org/project/zoo-template-common/
• Documentation: https://zoo-project.github.io/zoo-template-common/
• Repository: https://github.com/ZOO-Project/zoo-template-common
• Changelog: https://github.com/ZOO-Project/zoo-template-common/compare/v0.1.4... v0.1.5

License: Apache-2.0

zoo-template-common v0.1.4

📦 zoo-template-common v0.1.4

Maintenance release: Small fix to import botocore.session properly

🔄 What's Changed

• Version updated to 0.1.4 in pyproject.toml
• CommonExecutionHandler inherits from zoo_runner_common.handlers.ExecutionHandler

📦 Installation from PyPI

Install directly from PyPI:

pip install zoo-template-common==0.1.4

Or update in your pyproject.toml:

[tool.poetry.dependencies]
zoo-template-common = ">=0.1.4"

🔗 Links

• PyPI Package: https://pypi.org/project/zoo-template-common/
• Documentation: https://zoo-project.github.io/zoo-template-common/
• Repository: https://github.com/ZOO-Project/zoo-template-common
• Changelog: https://github.com/ZOO-Project/zoo-template-common/compare/v0.1.3... v0.1.4

License: Apache-2.0

zoo-template-common-0.1.3

📦 zoo-template-common v0.1.3

Maintenance release: Small fix to import botocore.session properly

🔄 What's Changed

• Version updated to 0.1.3 in pyproject.toml
• Fix import to enable use of botocore.session

📦 Installation from PyPI

Install directly from PyPI:

pip install zoo-template-common==0.1.3

Or update in your pyproject.toml:

[tool.poetry.dependencies]
zoo-template-common = ">=0.1.3"

🔗 Links

• PyPI Package: https://pypi.org/project/zoo-template-common/
• Documentation: https://zoo-project.github.io/zoo-template-common/
• Repository: https://github.com/ZOO-Project/zoo-template-common
• Changelog: https://github.com/ZOO-Project/zoo-template-common/compare/v0.1.2... v0.1.3

License: Apache-2.0

zoo-template-common-0.1.2

📦 zoo-template-common v0.1.2

Maintenance release: Dependency update to support the latest zoo-runner-common package structure

This maintenance release updates the dependency to zoo-runner-common v0.1.2 to ensure compatibility with its new package structure and improved Python module organization.

🔄 What's Changed

1. Dependency Update

• Updated zoo-runner-common dependency from >=0.1.0 to >=0.1.2
• Ensures compatibility with the new zoo_runner_common/ package structure
• Aligns with improved import paths: from zoo_runner_common import ...

2. Version Bump

• Version updated to 0.1.2 in pyproject.toml
• Maintains consistency with the zoo-runner-common release cycle

📦 Installation from PyPI

Install directly from PyPI:

pip install zoo-template-common==0.1.2

Or update in your pyproject.toml:

[tool.poetry. dependencies]
zoo-template-common = ">=0.1.2"

🔄 Migration Guide

If you're upgrading from v0.1.1, no code changes are required in your services. The dependency update is fully backward compatible.

However, if you were using zoo-runner-common directly, update your imports as per the zoo-runner-common v0.1.2 migration guide:

Before (zoo-runner-common v0.1.0 / v0.1.1):

from base_runner import BaseRunner
from zoo_conf import ZooConf, ZooInputs, ZooOutputs

After (zoo-runner-common v0.1.2):

from zoo_runner_common import BaseRunner, ZooConf, ZooInputs, ZooOutputs

🔗 Links

• PyPI Package: https://pypi.org/project/zoo-template-common/
• Documentation: https://zoo-project.github.io/zoo-template-common/
• Repository: https://github.com/ZOO-Project/zoo-template-common
• Changelog: https://github.com/ZOO-Project/zoo-template-common/compare/v0.1.1... v0.1.2

📝 Notes

This release ensures compatibility with the latest zoo-runner-common package structure. Core functionality remains identical to v0.1.1.

License: Apache-2.0

zoo-template-common-0.1.1

zoo-template-common 0.1.1

zoo-template-common is a Python package providing reusable execution handlers and utilities for CWL (Common Workflow Language) workflow templates in ZOO-Project. It simplifies the development of OGC Application Packages by offering a standardized, extensible foundation for geospatial processing services.

🎯 Why use zoo-template-common?

✅ Reduce boilerplate code – No need to rewrite execution logic for each workflow
✅ Standardized STAC output handling – Automatic processing and cataloging of workflow results
✅ Cloud-native ready – Built-in S3 integration and Kubernetes pod configuration
✅ Extensible architecture – Override hooks to add custom behavior without modifying core logic
✅ Production-tested – Used in EOEPCA and EOAP platforms for Earth Observation processing
✅ Simplifies Calrissian integration – Seamless workflow execution on Kubernetes

---

🚀 What's New in v0.1.1

Major Improvements

📦 PyPI Publication & Distribution

• Official PyPI Package: The package is now officially available on PyPI, making installation as simple as pip install zoo-template-common
• Automated Publishing: Set up CI/CD pipeline for automatic publication to PyPI on new releases
• Complete Package Metadata: Added comprehensive metadata including classifiers, keywords, and project URLs for better discoverability

📄 License Update

• Switched to Apache-2.0 License: Moved from previous licensing to Apache-2.0, providing clearer terms for commercial and open-source use
• Better Ecosystem Alignment: Apache-2.0 aligns with many geospatial and cloud-native projects

🐍 Python Version Requirements

• Minimum Python 3.10: Updated to require Python 3.10 or newer
• Support for Python 3.10, 3.11, and 3.12: Tested and compatible with the latest Python versions
• Improved Type Hinting: Takes advantage of modern Python features for better code quality

📚 Enhanced Documentation

• Updated Service Templates Documentation: Comprehensive docs reflecting the latest patterns and best practices
• Real-world Examples: Added examples from production deployments (EOEPCA, EOAP)
• API Reference: Complete documentation of all classes and methods
• Integration Guides: Step-by-step guides for integrating with ZOO-Project and Calrissian

🔗 Dependency Management

• Official zoo-runner-common Integration: Now uses the official zoo-runner-common package (>=0.1.0) as a dependency
• Simplified Dependency Tree: Cleaner, more maintainable dependency structure
• Better Version Pinning: Minimum version requirements ensure compatibility

---

🌟 Key Features

CommonExecutionHandler

The core of this package is the CommonExecutionHandler class, an extensible base class for CWL workflow execution:

• Pre/Post-Execution Hooks: Inject custom logic before and after workflow execution
• Automatic STAC Catalog Processing: Seamlessly handle SpatioTemporal Asset Catalog (STAC) outputs
• Kubernetes Pod Configuration:
  • Environment variable management
  • Node selector configuration
  • Resource allocation
• Secrets Handling: Secure management of sensitive configuration
• Tool Log Integration: Automatic collection and registration of workflow execution logs
• Output Management: Intelligent processing and validation of workflow outputs

CustomStacIO

A specialized STAC I/O class with cloud-native capabilities:

• S3 Integration: Read and write STAC catalogs directly from/to S3-compatible storage using boto3
• Multi-Backend Support: Works with both S3 and local file systems transparently
• Efficient I/O Operations: Optimized for large-scale geospatial data processing

---

💡 Use Cases

This package is designed to streamline the creation of geospatial processing services in ZOO-Project by providing a reusable foundation for:

1. CWL Workflow Execution: Execute Complex Workflow Language workflows with Calrissian on Kubernetes
2. STAC Catalog Management: Process and publish geospatial results via STAC catalogs
3. Cloud Storage Integration: Seamless integration with S3-compatible storage platforms (AWS S3, MinIO, Ceph, etc.)
4. OGC Application Packages: Standardized input/output management for OGC API - Processes
5. Earth Observation Processing: Production-ready foundation for EO data processing pipelines

---

📖 Quick Start

Installation

pip install zoo-template-common

Basic Example

from zoo_template_common import CommonExecutionHandler
from zoo_calrissian_runner import ZooCalrissianRunner

def my_workflow(conf, inputs, outputs):
    # Initialize the execution handler
    execution_handler = CommonExecutionHandler(conf=conf, outputs=outputs)
    
    # Create and configure the Calrissian runner
    runner = ZooCalrissianRunner(
        cwl=cwl,
        conf=conf,
        inputs=inputs,
        outputs=outputs,
        execution_handler=execution_handler,
    )
    
    # Execute the workflow
    exit_status = runner. execute()
    return exit_status

Extending for Custom Needs

One of the most powerful features is the ability to extend CommonExecutionHandler for specific platforms:

from zoo_template_common import CommonExecutionHandler
import jwt
import requests

class MyCustomHandler(CommonExecutionHandler):
    def __init__(self, conf, outputs):
        super().__init__(conf, outputs)
        # Add your custom initialization
        self.api_token = conf.get("auth_env", {}).get("token", "")
        self.workspace_url = conf.get("platform", {}).get("workspace_url", "")
    
    def pre_execution_hook(self):
        """Called before workflow execution"""
        super().pre_execution_hook()
        
        # Add your custom pre-execution logic
        self.setup_workspace()
        self.validate_permissions()
        self.prepare_inputs()
    
    def post_execution_hook(self, log, output, usage_report, tool_logs):
        """Called after workflow execution"""
        super().post_execution_hook(log, output, usage_report, tool_logs)
        
        # Add your custom post-execution logic
        self.register_results()
        self.update_catalog()
        self.send_notifications()
    
    def setup_workspace(self):
        """Example:  Set up a workspace for the execution"""
        response = requests.post(
            f"{self.workspace_url}/workspaces",
            headers={"Authorization": f"Bearer {self.api_token}"},
            json={"name": "my-workspace"}
        )
        self.workspace_id = response.json()["id"]

---

🏗️ Architecture

Design Principles

• Separation of Concerns: Clear separation between workflow execution, I/O handling, and platform-specific logic
• Open/Closed Principle: Open for extension (via hooks) but closed for modification
• Dependency Inversion: Depends on abstractions (base classes) rather than concrete implementations
• Single Responsibility: Each class has a well-defined, single purpose

Integration Points

┌─────────────────────────────────────────┐
│         ZOO-Project Service             │
└────────────────┬────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────┐
│     CommonExecutionHandler              │
│  ┌───────────────────────────────────┐  │
│  │  Pre-Execution Hook               │  │
│  └───────────────────────────────────┘  │
│                 │                        │
│                 ▼                        │
│  ┌───────────────────────────────────┐  │
│  │  ZooCalrissianRunner              │  │
│  │  (executes CWL on Kubernetes)     │  │
│  └───────────────────────────────────┘  │
│                 │                        │
│                 ▼                        │
│  ┌───────────────────────────────────┐  │
│  │  Post-Execution Hook              │  │
│  │  - STAC Processing                │  │
│  │  - Output Registration            │  │
│  └───────────────────────────────────┘  │
└─────────────────────────────────────────┘
                 │
                 ▼
┌─────────────────────────────────────────┐
│      CustomStacIO (S3/Local)            │
└─────────────────────────────────────────┘

---

🌍 Who is Using It?

Production Deployments

• eoepca-proc-service-template: Extends CommonExecutionHandler with EOEPCA Workspace API integration for the EOEPCA+ platform
• zoo-service-template: Uses CommonExecutionHandler directly or extends it for specific ZOO-Project deployments
• EOAP (Earth Observation Application Platform): Production Earth Observation processing platform

Example Projects

Browse the examples directory for:

• Basic workflow integration
• S3 storage configuration
• STAC catalog processing
• Custom handler implementations

---

📋 Requirements

• Python: >= 3.10
• Core Dependencies:
  • zoo-runner-common >= 0.1.0
  • loguru >= 0.7.0 (structured logging)
  • pystac >= 1.8.0 (STAC catalog handling)
  • pyyaml >= 6.0 (configuration parsing)
  • boto3 >= 1.28.0 (AWS S3 integration)
  • botocore >= 1.31.0 (AWS core functionality)

---

🔧 API Reference

CommonExecutionHandler

Constructor

CommonExecutionHandler(conf:  dict, outputs: dict = None)

Key Methods

• pre_execution_hook(): Override to add logic before workflow execution
• post_execution_hook(log, output, usage_report, tool_logs): Override to add logic a...

zoo-template-common-0.1.0

zoo-template-common is a Python package providing reusable execution handlers and utilities for CWL workflow templates in ZOO-Project. It simplifies the development of OGC Application Packages by offering a standardized, extensible foundation for geospatial processing services.

Why use zoo-template-common?

✅ Reduce boilerplate code – No need to rewrite execution logic for each workflow
✅ Standardized STAC output handling – Automatic processing and cataloging of workflow results
✅ Cloud-native ready – Built-in S3 integration and Kubernetes pod configuration
✅ Extensible architecture – Override hooks to add custom behavior without modifying core logic
✅ Production-tested – Used in EOEPCA and EOAP platforms for Earth Observation processing
✅ Simplifies Calrissian integration – Seamless workflow execution on Kubernetes

Key Features

• CommonExecutionHandler: Extensible base class for CWL workflow execution

  • Pre/post-execution hooks for customization
  • Automatic STAC catalog output processing
  • Pod environment variables and node selector management for Kubernetes
  • Secrets handling
  • Tool log integration

• CustomStacIO: STAC I/O class with S3 support via boto3

  • Read/write STAC catalogs from/to S3
  • Support for both S3 and local file systems

Use Cases

This package is designed to streamline the creation of geospatial processing services in ZOO-Project by providing a reusable foundation for:

• Executing CWL workflows with Calrissian on Kubernetes
• Processing and publishing results via STAC catalogs
• Integration with S3-compatible storage platforms
• Standardized input/output management for OGC Application Packages

Who is using it?

• eoepca-proc-service-template: Extends CommonExecutionHandler with EOEPCA Workspace API integration
• zoo-service-template: Uses CommonExecutionHandler directly or extends it for specific needs

Quick Start

pip install zoo-template-common

Basic Example

from zoo_template_common import CommonExecutionHandler
from zoo_calrissian_runner import ZooCalrissianRunner

def my_workflow(conf, inputs, outputs):
    execution_handler = CommonExecutionHandler(conf=conf, outputs=outputs)
    
    runner = ZooCalrissianRunner(
        cwl=cwl,
        conf=conf,
        inputs=inputs,
        outputs=outputs,
        execution_handler=execution_handler,
    )
    
    exit_status = runner. execute()
    return exit_status

Extending for Custom Needs

class MyCustomHandler(CommonExecutionHandler):
    def pre_execution_hook(self):
        # Add your custom logic before execution
        super().pre_execution_hook()
        self.setup_workspace()
    
    def post_execution_hook(self, log, output, usage_report, tool_logs):
        # Add your custom logic after execution
        super().post_execution_hook(log, output, usage_report, tool_logs)
        self.register_results()

Requirements

• Python >= 3.8
• loguru >= 0.7.0
• pystac >= 1.8.0
• pyyaml >= 6.0
• boto3 >= 1.28.0
• botocore >= 1.31.0

Documentation & Support

• 📚 Full Documentation
• 🐛 Issue Tracker
• 💬 Discussions

License

Apache License 2.0