All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
Types of changes
Addedfor new features.Changedfor changes in existing functionality.Deprecatedfor soon-to-be removed features.Removedfor now removed features.Fixedfor any bug fixes.Securityin case of vulnerabilities.
Addedconfigurable CORS support to http command
FixedFilter Parameter Not Working in "lino http" Command #369
Addedflaglog-sql(shortl) to the push command
Addedpossibility to filter columns viaselectproperty in ingress descriptorsAddedcommandsset-parent-selectandset-child-selecttolino idAddedimport: nooption for columns intables.yamlconfigurationAddedexport: presenceoption for columns intables.yamlconfigurationFixedissue withimport: filewhen data is null, will not fail anymore
Fixedinvalid length for DATE or NUMBER bind variable, bumping oracle driver to 2.8.19
Fixedhidden global internal timeout of 240s with Oracle driver is now disabled by default
Changedtable or column names were not correctly matched in terms of case sensitivity (lowercase or uppercase) in table.yamlAddednew commandqueryto execute any SQL query on any SQL datasource
Addedflag--watch(short-w) tolino pushcommand
Fixedpanic during push on Oracle database with anullcolumn valueFixedissue with SQL queries involving limit orders in SQL Server databaseFixedpanic when using autotruncate on a null value
Addedcolumns information and export type using thelino table extractcommand, columns and keys organized according to the database orderAddedflag--only-tablestolino table extractcommand. This flag allows for the extraction of table information exclusively, excluding columns. It has been included to maintain the previous behaviorAddedflag--with-db-infostolino table extractcommand. This flag enables the extraction of information regarding column types, length, size, and precision if the column has been configured with these specificationsAddedflag--autotruncatetolino pushcommand. This flag will enable a truncate on each value based eachdbinfo.lengthparameters set in the table.yaml file for each columnsAddedpropertydbinfo.bytesto column definition in table.yaml file. Set it to true to truncate the value based on a maximum number of bytes and not characters (assuming utf-8 encoding for now)Addedflags--max-lengthand--bytestolino table add-columncommand. Use it to edit the propertiesdbinfo.lengthanddbinfo.bytesof the table.yaml file
Fixedperformance issue withimport: file
AddedsqlserversupportAddedflags tolino analyse:sample-size,distinct,limit,whereandtable(by default distinct values are not counted)Addedflags tolino analyse:exludeandexlude-pkto exclude columns from analyseAddedflags tolino pull:-Xor--exclude-from-fileto filter out lines from a JSON lines file referenceAddedflags tolino push:--savepointto write primary keys of all commited line to a fileAddednew import types totable.yaml:blobwill optimize push of blob column into database,filewill read content of a file at the specified path and import as a blob
Addedcommandanalyseto extract metrics from the database in YAML format
Addedgo-ora driver for oracle in replacement of old driver (remove technical prerequisite to install Oracle Instant Client)
Addedflag--pk-translationallow update of primary keys, by giving a cache.jsonl file containing old and new values for a specific keyAddedif a JSON object contains a__usingpk__with a dictionary of key/values, it will be used by the push command to select the target record to update (push update only)Addedwhere clauses (child and parent) in ingress descriptor to enable non-start table filteringFixedoracle connector disable/enable contraints cascade
Addedimport property intables.yamlallow to specify format of data to read from JSON stream (string(default),numeric,binaryorbase64(same),datetime,timestamp) thereforeimportandexportnow mirror each other (importis used at push andexportat pull) butimportstill allow to specify the data type to pass to database driver (backward compatibility)Addedimport property intables.yamlallow to specify a format AND a type at the same time with theformat(type)syntax (e.g.import: binary(int64))Addedwebsocket connector with basic auth, supported schemes : ws, wss (BETA)Addedflag--statsto generate a stat file or HTTP POSTAddedflag--statsTemplateto control the format of generated statsFixedlino push truncatewith--tableoption doesn't truncate table #123
Addedexport mode all intables.yamlto export all columns even if some columns are defined in the columns property
FixedBad SQL update statement for oracleFixedReset statement after error during push #54FixedContinue to close others row writers after error
Changedorder of keys in output JSON lines will be alphabetical when pulling (without configuration in tables.yaml)Addedconfiguration of export format / import type for columns in tables.yaml, see issue #33 for more informationAddedMariaDB/MySQL support (thanks to @joaking85)Addedauto-select columns required by a relation but not exported in tables.yamlAddednew commands to configure tables : add-column and remove-columnAddedNew command to count lines in tablesFixedlimit keyword on DB2 dialectFixedPush truncate respect child/parent constraint orderFixedPush truncate will trigger only for attainable tables in the ingress descriptor - tables that are cut out will not be truncatedFixedpush insert with mysql/mariadb connector now works properly with MySQL databaseFixedpush insert with mysql/mariadb connector will not update record if it exists
AddedHTTP connector will now close/reopen request when commit size is reached
Fixedcharset on Content-Type header when pushing to HTTP connector backend
Fixedsome HTTP library doesn't support a body payload on a HTTP GET request, therefore the HTTP connector will pass the "filter" parameter through the headers in addition to the request body
Addednew verb to extract, to get status and to update sequences
Addednew parameter to pull only distinct values from the start table
Addednew datasource type with string connectionhttp://...LINO can pull/push data to an HTTP Endpoint API
Addedoption to change ingress-descriptor filenameChangedupdate debian image to last stable (debian:stable-20210816-slim)
Addedupdate Pimo to v1.8.0
Addedstatistics report for push and pull executions (thanks to @CapKicklee)Changedsome info level logs to debug level in pull module
FixedRevert convert JSON date to Oracle date format as a workaround for godror
Addedflag to enable or disable coloring in output logs (--color [yes|no|auto])Addedupdate Pimo to v1.6.1
FixedRemove ENTRYPOINT and change CMD in oracle docker image
Addedstructured logs (debug & json format) (thanks to @CapKicklee)
Fixedextract composite primary keys for oracleFixedprotect columns names in insert statement
FixedMissing where keyword for Oracle SQL Query
Added--where flag to use a raw sql where clause to filter rows of start tableAddedOracle database supportSecurityremove connection string from log
AddedFirst public version released