AwsWranglerDeltaLakeIOManager: write mode and schema mode error.

[CHRISCARLON]

Issue

You can't have the "mode" set as "append" if the "schema_mode" is set as "overwrite".

Will look into why.

Will add the error message later.

Have made all assets overwrite for now - but will look into adding schema mode as a decorator arg.

As some assets will need to be append rather than overwrite.

Code as is:

try:
            wr.s3.to_deltalake(
                df=obj,
                path=table_path,
                mode=write_option,
                schema_mode="overwrite",  # INCLUDE AS A DECORATOR ARG AT SOME POINT LIKE FOR MODE ABOVE
                s3_allow_unsafe_rename=True,
            )
            context.log.info(f"Data written to Delta Lake table at {table_path}")
        except (DeltaLakeWriteError, Exception) as e:
            raise e

Fix

Schema mode arg would go:

@asset(
    group_name="energy_assets",
    io_manager_key="DeltaLake",
    metadata={"mode": "overwrite"},
    **ARG HERE**
    ins={"entsog_gas_uk_data_bronze": AssetIn("entsog_gas_uk_data_bronze")},
    required_resource_keys={"slack"}
)
@with_slack_notification("ENTSOG Gas UK data")
def entsog_gas_uk_data_silver(
    context: AssetExecutionContext, entsog_gas_uk_data_bronze
):