v1.5.20

Sling v1.5.20 (2026-06-07T10:37:59Z)

See https://github.com/slingdata-io/sling-cli/ for more details.

v1.5.19

Sling v1.5.19 (2026-05-25T11:50:17Z)

New Features

• Column reordering and renames in select: The select clause now supports reordering and renaming columns inline. Combined with stricter handling of select exclusions and excluded PKs, this gives finer control over output schemas without needing downstream transforms.

• Oracle LOB / binary fidelity to Snowflake: End-to-end row-level fidelity for Oracle LONG, LONG RAW, and large BLOB columns when replicating to Snowflake. Snowflake COPY VIA STAGE now casts hex-encoded binary to BINARY with explicit sizing up to 64MB, Snowflake parquet file formats set binary_as_text = false, DuckDB staging decodes hex-encoded binary to BLOB for parquet, DuckDB read_csv max_line_size is raised for large hex blobs, and BinaryToHex conversion is optimized from O(n²) to O(n). Type inference is refined to preserve native binary types and apply explicit lengths.

• null_if function: New function for normalizing values to null when they match a target — useful for cleaning sentinel values out of API responses and CSV inputs.

• JSON-RPC error detection in endpoints: API endpoints now recognize JSON-RPC error envelopes and surface them through the standard rule/retry pipeline instead of treating them as successful payloads.

• Idle DB-connection reaping in sling lsp: The language server now reaps idle database connections instead of holding them open for the lifetime of the editor session.

• FileType.Exts method: New helper returns all valid extensions for a FileType, enabling correct detection of multi-extension formats (e.g. .xlsm now recognized as Excel file).

Bug Fixes

• Inline iterate.over preserved on dynamic API endpoints: Inner iterate.over is now preserved across dynamic endpoint expansion, and the over field's JSON tag is corrected so YAML/JSON specs round-trip correctly.

• BigQuery errors masked by context.Canceled: BigQuery job-wait logic is consolidated into waitForJob, which re-fetches job status with a detached context if job.Wait returns context.Canceled. Real BigQuery API errors now surface from LoadFromReader, CopyFromGCS, and CopyToGCS instead of being hidden behind a generic cancellation error.

• GCS path format in importViaGoogleStorage: Corrected the GCS URI format used when importing via Google Storage, fixing failures on BigQuery loads that route through GCS.

• Auth-sequence responses skipped default rules: Default retry/fail rules are now applied to authentication-sequence responses in dbio/api. Previously these responses bypassed the rule pipeline, producing unclear errors on auth failures. Rule application is also guarded for idempotency to prevent stacking on re-compilation.

• Endpoint rule stacking on re-compilation: Added an idempotency guard to checkResponse so endpoint rules aren't duplicated when specs are re-compiled, and ensured mutations to setup/teardown call requests/responses persist by indexing directly into slices.

• CDC metadata column validation: Added support for retrieving and validating CDC metadata columns after the initial snapshot, with additional debug logging on CDC operations to make troubleshooting easier.

• Temp-table double-drop: Skip dropping a temp table that has already been marked as dropped, and make the temp-table cleanup key unique to avoid collisions between concurrent tasks.

v1.5.18

Sling v1.5.18 (2026-05-12T20:15:08Z)

New Features

• Queue-only API endpoints: New queue_only flag on API spec endpoints. Such endpoints drain synchronously to populate downstream queues and emit no records of their own, and are hidden from broad enumeration (sling conns discover, wildcard replication expansion) while remaining addressable by name. queue_only is also honored as a defaults value in the spec.

• Inline iterate lists/objects for dynamic API endpoints: Dynamic endpoints now accept native YAML arrays and objects in iterate (in addition to JSON literals and JMESPath strings). Nested YAML maps are normalized to map[string]any for consistent template/JMESPath use.

• sling conns exec output formats and --limit: sling conns exec now supports --output csv|json|arrow with streaming (Arrow IPC / CSV go straight to stdout, logs to stderr) and a --limit flag (default 100, 0 = unlimited). The limit is pushed down via the dialect's limit_sql template so the database truncates server-side. sling conns discover also gains CSV output.

Bug Fixes

• CDC nil target options on shared-reader path: CDC's shared-reader write path bypassed TaskExecution.Execute() and left target options (ColumnCasing, AddNewColumns, AdjustColumnType) nil. Now calls Config.SetDefault() on the cloned config so system-wide defaults apply.

• Dataflow.Pause timeout on slow stream starts: Pause now waits up to a configurable timeout (SLING_PAUSE_TIMEOUT, default 30s) and returns false if any stream can't pause in time or the dataflow context is canceled, instead of using a short random timer.

• MySQL LoadDataLocal missing batches: Reader handler is now registered per batch in MySQL's LoadDataLocal, ensuring every batch is loaded instead of only the first.

• DuckDB Describe errors no longer masked: SELECT/WITH queries fail fast on Describe errors in DuckDB instead of swallowing the error and returning misleading results.

• Nil reader panic in ConsumeCsvReaderChl: Guards against a nil reader to prevent panics.

• Spurious HTTP-URL check in GenerateInsertStatement: Removed an unnecessary check that could misclassify insert targets.

• chunk() function return type: The pipeline chunk() function now returns *ChunkResult instead of a bare channel for consistent downstream consumption.

• Cleaner error/messages: Clearer row-count-mismatch error in WriteToDb/writeToDbDirectly, simplified pause error handling on those same paths, corrected DuckDB pipeline path and sling command output message, clarified the sling run CLI description.

v1.5.17

Sling v1.5.17 (2026-05-05T02:25:34Z)

New Features

• CDC shared-reader replication: New CDCGroupRunner orchestrates multi-stream CDC replications through a shared reader, with phased prepare/read-into-cache/write-and-commit execution and bounded-parallel writes.
• Connection retry in HookQuery: Automatically retries query hooks against dead database connections instead of failing the run.
• NewArrowWriter options: Arrow writer accepts additional options for finer control over output, with quieter logging in InferColumnTypes.
• sling agent service commands: Manage the Sling agent as a system service on Windows, macOS, and Linux (install/uninstall/start/stop).
• sling project CLI subcommands: New commands for managing project executions, files, and connections — including filters, output formatting, create/update/delete on jobs and files, and confirmation prompts on destructive actions.
• MCP file:// SQL support: MCP server can now execute queries from saved SQL files.
• LSP working-directory awareness: Language server resolves env and project files relative to the current working directory.

Bug Fixes

• Concurrent map iteration in API requests: Locked context in NewSingleRequest to prevent concurrent map iteration and write panics during API calls.
• Source options on zipped files: Source options (delimiter, header, etc.) are now propagated when reading from zip archives. (thanks @hermogenes)
• Standardized casing for file paths across scripts and documentation to avoid case-sensitivity issues on Linux.

v1.5.15

Sling v1.5.15 (2026-04-15T22:44:09Z)

New Features

• CDC support for PostgreSQL, SQL Server, Oracle, and MongoDB: Change Data Capture now works with PostgreSQL (logical replication), SQL Server, Oracle, and MongoDB sources — previously only MySQL was supported.
• CDC initial snapshot cursor-based resume: Initial CDC snapshots now use keyset-based chunking for string/UUID primary keys, enabling reliable cursor-based resume after interruption.
• CDC catch-up mode with end position: CDC ReadChanges now captures the current position before streaming so catch-up runs terminate without waiting for new events.
• Default adjust_column_type for CDC: CDC replications now default adjust_column_type to true, automatically widening target columns (varchar length, decimal precision) when the source schema changes.
• adjust_column_type precision expansion: The adjust_column_type option now detects and expands varchar length and decimal precision/scale when the source columns grow wider, avoiding truncation or precision loss.
• description parameter for SQL query actions: MCP database.query and worksheet logging now accept an optional description field for improved context and observability.
• MCP worksheet query logging: MCP SQL queries are now appended to a worksheet log file with metadata (connection, description, timestamp, duration, columns, row count), with automatic size trimming.

Bug Fixes

• Oracle SID/service_name conflict: Fixed Oracle connection strings where the SID option could conflict with service_name, causing connection failures.
• Progress update crash when replication is nil: Fixed a panic when updating task progress on a task with no associated replication.
• API spec record limit not stopping iteration: Fixed an issue where the API spec record limit was not properly stopping inner iteration loops during rule evaluation.

v1.5.14

Sling v1.5.14 (2026-04-06T15:41:03Z)

New Features

• SQL Server BCP with Entra ID authentication: BCP bulk loading now supports Entra ID token-based auth and Azure CLI integration for passwordless authentication.
• DuckDB Arrow IPC streaming: Implement Arrow IPC streaming for DuckDB connections, improving read/write performance.
• CDC full-refresh mode: Added support for full-refresh mode with CDC options, clearing state before running an initial snapshot.
• CDC UUID primary key support: CDC initial snapshots now handle string/UUID primary keys with offset-based chunking, in addition to integer PKs.
• length() function supports scalar types: The length() function now safely handles integers, floats, booleans, and dates by converting to string representation, allowing use in wildcard transforms with ternary guards.
• sling project jobs run --wait flag: New --wait / -w flag polls execution status every 5 seconds until the job reaches a terminal state, then reports rows, bytes, and duration.
• OAuth2 custom credentials and spec URL: Enhanced OAuth2 initialization to support custom client credentials and spec URL for flexible token management.

Bug Fixes

• Oracle sqlldr column sizing: Fixed sqlldr control file generation to use target column precision instead of hardcoding char(400000), preventing excessive memory allocation during Oracle bulk imports.
• TINYINT type handling: Corrected data type mapping for tinyint in metadata queries across MariaDB, MySQL, and StarRocks templates.
• Cross-platform path handling: Replaced path package with filepath across the codebase for correct behavior on Windows.
• Token save in agent mode: Allow local SaveToken to proceed even when running in agent mode.

v1.5.13

Sling v1.5.13 (2026-03-24T13:18:25Z)

New Features

• OAuth2 authorization code flow with configurable redirect URL and port: The authorization code flow now supports a configured redirect_uri, using its host, port, and callback path instead of always binding to a random local port.
• Render OAuth2 scopes before configuration: OAuth2 scopes are now rendered (variable substitution) before being passed to the OAuth2 config, enabling dynamic scope values.
• CLIENT_SECRET support for Azure and OneLake connections: CLIENT_SECRET and client certificate properties are now properly passed through for Microsoft Fabric ABFS and Azure file system authentication.
• Unicode column name support: CSV header cleaning now preserves Unicode letters and numbers (e.g. Chinese, Japanese, accented characters) instead of stripping them.
• String length inference for file sources with column_typing: column_typing string max_length rules now apply to non-database sources (CSV/files), enabling auto-inference of varchar() precision when loading into databases like Oracle.

Bug Fixes

• BigQuery GetSchemata table retrieval: Fixed schema value not being cast to string, which caused table listing to fail when discovering BigQuery schemas.
• Oracle SQL*Loader out-of-memory handling: SQL*Loader now runs with context cancellation support (CommandContext) and surfaces actionable SQL*Loader-700 / out-of-memory errors instead of generic failures.
• Oracle SQL*Loader pipe error handling: Closed-pipe errors during CSV writing to SQL*Loader stdin are now caught and propagated correctly instead of being silently ignored.
• DuckDB connection leak during chunk loading: Added missing defer dbConn.Close() calls in ClearTableForChunkLoadWithRange and ProcessChunks to prevent connection leaks during chunked replication.
• Chunking error message without update_key: Fixed a spurious nil error being passed to g.Error when update_key is missing for stream chunking.
• Compact CDC State: Fixed CDC state so that GTID messages are stored compact

v1.5.12

Sling v1.5.12 (2026-03-09T18:18:58Z)

New Features

• --home-dir flag: New flag across sling run and all sling conns subcommands (list, test, set, unset, exec, discover) to specify a custom sling home directory. Sets SLING_HOME_DIR and reloads connections accordingly.

• Column merge with defaults (+ prefix): Stream-level columns: entries prefixed with + now merge with defaults.columns instead of replacing them. Use +column_name: ~ to unset a default. Without +, legacy replace behavior is preserved.

• Pipeline store variables in replications: {store.xxx} variables set in pipeline steps are now accessible in inline replication where: and sql: fields.

• API spec range_step support: Added range_step to the endpoint context map, enabling three-part range expressions for backfill.

• API spec state variable rendering in jmespath / jq: State variables (e.g. {state.endpoint_name}) are now resolved inside jmespath and jq response record expressions.

Bug Fixes

• SLING_LOGGING env from replication: Moved SLING_LOGGING env handling from per-task to replication-level so it takes effect before stream execution and properly reinitializes the logger.

• Direct insert incremental state: writeToDbDirectly now calls SyncColumns() and SyncStats() so incremental state (e.g. datetime update keys) is correctly captured after direct insert writes.

• Auth token refresh on retry: API retry logic now refreshes HTTP request headers from conn.State.Auth.Headers before resending, fixing stale token issues after server-side token rotation (both single-request and mid-pagination scenarios).

• Windows path handling: CleanWindowsPath applied to local file locations in ParseLocation, API spec file:// paths, env file paths, and SLING_HOME_DIR.

• Oracle chunking with custom SQL: Fixed AS table alias syntax for Oracle (which doesn't support AS for subquery aliases).

v1.5.11

Sling v1.5.11 (2026-03-02T19:52:22Z)

• Fix critical issue with API Spec state parsing when using setup

v1.5.10

Sling v1.5.10 (2026-03-02T14:00:35Z)

Connectors

• ADBC DuckDB — new ADBC connector with Docker testing
• MongoDB — support ISODate() in filters
• ClickHouse — fix table rename SQL syntax
• SFTP — fix incremental file timestamp filtering
• Google Drive — add context timeout for API calls
• Local files — UNC path support (Windows network paths)

Connections & Environment

• Support setting username in connection URL
• New .env.sling file support for per-project connection definitions

MCP Server

• working_dir parameter on connection, pipeline, and replication actions
• Auto-loads .env.sling from working directory

CDC (Change Data Capture)

• Soft delete now uses _sling_synced_op = 'D' instead of _sling_deleted_at; snapshot rows marked with 'S'
• Updated merge_change_capture_soft templates across all 14 databases

Other

• Updated godbc v0.0.5, purego v0.10.0
• Disable AWS EC2 metadata service to prevent warning messages
• JMESPath/JQ safe rendering with state variables
• Simplified update check logic