v1.4.15

Sling v1.4.15 (2025-07-26T00:25:00Z)

• Changed FileSelect from pointer to slice for consistency across codebase to fix critical bug.
• Refactored hook parsing and execution system for improved clarity
• Added context management to replication tasks and hooks
• Fixed status check conditions to reference correct run status

v1.4.14

Sling v1.4.14 (2025-07-23T21:43:15Z)

Major Feature Categories

Data Chunking & Replication

Enhanced Chunking Functionality:

• New expression support for chunking configurations
• SQL stream chunking with conditional checks
• Improved chunking logic with new options and configurations
• File handling support in stream task configuration

Replication Improvements:

• Enhanced task preparation logic in replication configuration
• Thread-safe operations with proper locking mechanisms
• Clean state management for chunk loading operations

Database & Storage Enhancements

Database Improvements:

• MongoDB column casing handling fixes
• Redshift S3 unload operations with proper file format extensions
• MySQL to Snowflake decimal to string cast support
• DuckDB ARM64 architecture support

File System Updates:

• Google Drive integration improvements:
  • Enhanced folder info and support for all drives
  • Updated queries to properly handle trashed files
  • Improved client functionality
• File deletion before writing dataflow to ensure clean state

Testing & Quality Assurance

New Test Suites:

• API response extraction tests using YAML configuration
• HTTP call and response extraction comprehensive tests
• Setup and teardown lifecycle tests with mock responses
• Parquet file deletion pipeline tests
• CLI execution timeout handling

Developer Experience

Logging & Environment:

• Color output control with SLING_LOGGING=NO_COLOR support
• Thread child detection with env.IsThreadChild
• Conditional message colorization based on NoColor settings

Build & Development:

• Executable build script permissions updates
• Test timeout handling for CLI execution
• Enhanced error handling across multiple components

v1.4.13

Sling v1.4.13 (2025-07-09T15:41:01Z)

Key Features & Enhancements

Database & Connection Improvements

• Iceberg Support: Enhanced Iceberg connection with AWS config and improved database handling
• Glue Integration: Added support for iceberg_glue target and enhanced Glue connection capabilities
• DuckDB Updates: Updated DuckDB version and added adjustCol flag for DuckDB and related types
• Redshift Enhancement: Added Redshift parquet export functionality

API & Spec Management

• API Spec Refactoring: Refactored API specs to use curly braces for variable interpolation
• MCP Tools: Enhanced connection and API spec tools with detailed action support
• Pagination Fixes: Updated stop_condition in pagination to use strict less than comparison

File System & Storage

• S3 Enhancements:
  • Enhanced S3 client with custom HTTP settings and retry logic
  • Streamlined AWS credential handling by removing unnecessary components
• File Management:
  • Fixed local temporary file deletion after writing to prevent duplication
  • Fixed single file copy operation to prevent filename duplication
  • Added support for multiple Azure Storage files replication

Testing & Quality

• Test Suite Expansion:
  • Added new connection types for DuckLake variants
  • Enhanced test cases with output verification
  • Added encoding tests for latin1, windows1252, and utf8_bom
  • Added transform functions test suite
  • Added inspect hook tests for database tables and files
  • Added MSSQL to parquet overwrite test
  • Added timestamp string casting tests

Performance & Reliability

• Streaming Improvements:
  • Replaced error logging with LogError for better error handling
  • Improved reader robustness and concurrency
• Workflow Optimization:
  • Increased timeout for build and release jobs (25 minutes)
  • Enhanced streaming capabilities

Dependencies & Maintenance

• Dependency Updates:
  • Updated flarco/g dependency to v0.1.154
  • Updated mark3labs/mcp-go to v0.33.0
• Snowflake Fix: Fixed temporary file handling with SLING_KEEP_TEMP environment variable

v1.4.12

Sling v1.4.12 (2025-06-29T12:16:17Z)

• Critical fix for a bug when creating DuckLake secrets

v1.4.11

Sling v1.4.11 (2025-06-27T16:45:21Z)

🚀 Major Features & Enhancements

📊 Databricks Support

• NEW: Complete Databricks database integration
• Added connection, query, and data import/export functionality
• Implemented Databricks-specific SQL templates
• Added support for Databricks COPY INTO commands
• Enhanced AWS S3 integration for Databricks workflows
• Added comprehensive JSON manipulation functions support
• Implemented Databricks volume operations with multiple file format support

🧊 Iceberg & Data Lake Improvements

• Enhanced DuckDB connection handling with Iceberg catalog support
• Improved REST catalog interface for table operations
• Added CatalogSchema support to DuckLake connections
• Enhanced error handling for manifest retrieval
• Updated catalog type references and authentication

🦆 DuckDB Enhancements

• Bumped DuckDB version from 1.3.0 to 1.3.1
• Added R2 file type support in dbio
• Improved URI handling and secret management
• Enhanced schema handling in buildAttachSQL

🔧 Bug Fixes & Improvements

💾 Data Type & Precision Handling

• FIXED: Oracle NUMBER scale preservation issues
• Enhanced decimal column precision and scale handling
• Improved SQLColumns and Apply methods for data types
• Removed unnecessary DbScale checks

🔐 Authentication & Credentials

• Standardized AWS credential handling across Athena, Iceberg, and S3
• Improved error reporting for authentication issues
• Enhanced secret management for various connectors

📁 File System & Formats

• Improved CSV delimiter handling (changed from rune to string)
• Enhanced file format assignment logic
• Streamlined S3 GetReader functionality
• Added String method to FileType for better representation

🔍 SQL & Query Improvements

• Fully qualified table names in DELETE statements for upsert operations
• Enhanced error messaging in GetTableColumns
• Improved SQL template compatibility

🧪 Testing & Quality

✅ Test Enhancements

• Updated CLI suite configuration from TSV to YAML format
• Added comprehensive test cases for:
  • Starrocks with empty_as_null option
  • Pipeline copy operations
  • Delta R2 secret creation
  • Oracle NUMBER scale fixes
  • MySQL BIT type replication
• Improved test logging and error handling
• Enhanced environment variable initialization

v1.4.10

Sling v1.4.10 (2025-06-12T10:42:03Z)

• Upgrade to AWS SDK for Go v2: The AWS SDK has been upgraded to v2, which brings in a more modern, modular architecture. This required substantial code updates, including adding context to all AWS API calls for better cancellation and timeout control, and improving error handling.

• New Connector and Format Support:

  • Apache Iceberg: Initial support for the Apache Iceberg table format has been added, using the iceberg-go library. This allows Sling to interact with Iceberg tables for data replication tasks.
  • DuckLake: Support for DuckLake has been introduced as a new database type, extending the capabilities of DuckDB.
  • Google Drive: The tool now supports Google Drive as a file system source.
  • Apache Arrow: Support for the Apache Arrow file format has been implemented, including reader/writer channels and integration into the HTTP import functionality.

• Enhancements to Existing Connectors:

  • Amazon Athena: Support has been significantly improved with better handling of data and staging locations, standardized SQL keywords, and a PURGE option when dropping tables.
  • PostgreSQL: It's now possible to specify a role in the PostgreSQL connection URL for more granular access control.
  • MySQL/MariaDB: Support for BIT column transformations has been added.
  • DuckDB: The embedded DuckDB version has been updated from 1.1.3 to 1.3.0.

• General Improvements and Fixes:

  • Timeout handling via SLING_TIMEOUT has been added to replication and pipeline runs.
  • A new TableExists method has been added to the connection interface to check for table existence before performing operations.
  • Several bug fixes were made, including improvements to error messages, handling of dbt parameters for Athena, and fixing unnecessary semicolons in generated DuckDB SQL.

v1.4.9

Sling v1.4.9 (2025-05-25T01:01:37Z)

Bug Fixes & Improvements

• CSV Schema Handling: Fixed potential out-of-bounds errors and addressed schema inconsistencies
• Data Type Corrections:
  • Changed 'real' data type from decimal to float in multiple databases
  • Changed 'real' data type from float to decimal in SQLite and DuckDB
• Node Management: Corrected node addition in List function
• Metadata Processing: Skip metadata type coercion to improve performance
• Logging:
  • Added version information to logs
  • Prevented duplicate log messages in child threads

New Features

• Task Loading: Added support for loading compiled tasks from file
• Connection Properties: Added connect_time property to connection methods
• Scoop Integration: Improved scoop upgrade handling

Dependency Updates

• Bumped github.com/flarco/g from v0.1.143 → v0.1.146
• Updated github.com/spf13/cast from v1.6.0 → v1.7.1

v1.4.8

Sling v1.4.8 (2025-05-13T18:38:17Z)

• Fix Redshift Unload
• Fix critical bug with sling agent

v1.4.7

Sling v1.4.7 (2025-05-13T10:06:04Z)

• Error Handling & Robustness:
  • Improved error handling for incremental mode (Commit c9dadd0).
  • Handled missing primary keys in API sources (Commit cd31e26).
  • Enhanced error handling in GetSchemata (Commit 4d5c9b5).
  • Handled empty result sets in GetCount (Commit 4666114).
  • Improved handling of field selection in database reads (Commit 1d1d1d9).
• Configuration & Schema:
  • Added a JSON schema for Sling pipeline configuration (Commit 4244c34).
  • Enhanced the JSON schema for pipeline steps by adding an id property to each (Commit 87024b5).
  • Updated pipeline.json schema (Commit 87274d9, 4bcfb66).
• Database & Data Handling:
  • Added support for Clickhouse JSON data type (Commit 2b986d0), which was later reverted (Commit db7b294).
  • Added basic support for AWS Athena (Commit 7f70b19).
  • Improved CSV reader to handle string values (Commit 82fa1a5) and column number mismatches (Commit b201e68).
  • Fixed an issue where Sling was not respecting the private_key property (Commits baff658, 2f4b7a3 - merged from PR #553 by GeoffTearle).
  • Ensured database connections are closed after wildcard processing (Commit 0b004c3).
  • Introduced a TrimSQLComments function (Commit bf0c02b).
  • Added tests for parsing multi-statements and trimming SQL comments (Commit d40d044).
  • Increased nvarchar length factor for temporary tables on SQL Server to provide a buffer, addressing issue #554 (Commit 7f8f3ed).
  • Applied quotes to select fields in incremental select statements (Commit 6d43d8a).
• Pipeline & Execution:
  • Enhanced pipeline execution and logging (Commit 731ff55).
• Development & CI/CD:
  • Updated build workflow runner matrix (Commit 75e8000).
  • Handled missing bcp (Bulk Copy Program) executable in tests (Commit 4369307).

v1.4.6

Sling v1.4.6 (2025-04-27T01:17:57Z)

API & Database Improvements

• Improved API specs and documentation
• Added stream read mode to API
• Enhanced API request handling with bloom filter and concurrency
• Added incremental_select query for SQL Server
• Set incremental state check for API

Performance & Functionality

• Improved DuckDB performance
• Enhanced range function to support different data types
• Added database connection check
• Added unit tests for Excel date conversion

Bug Fixes

• Improved database connection handling
• Fixed start/end hooks when a thread child
• Handled nil args in ExecContext and empty target type
• Improved Snowflake COPY INTO Table command
• Ensured file closure in SFTP write operations

Dependency Updates

• Bumped github.com/flarco/g from 0.1.138 to 0.1.140
• Bumped timeplus-io/proton-go-driver/v2
• Bumped sftp from 1.13.7 to 1.13.9
• Updated bloom filter library
• Multiple other dependency updates

Other Changes

• Changed log level from debug to trace in BulkImportStream
• Increased max_column_length to 1024 in StarRocks
• Added comment to stripe replication test
• Improved CI workflow for version bumping

Files Modified

• Database connectors (Snowflake, ClickHouse, D1)
• API specifications and documentation
• File system handlers (SFTP)
• Data processing components
• CI/CD workflows