Describe the bug
As a payments analyst, I expect all Enghouse transactions entries to be associated with a non-null timestamp. However, some rows in staging.stg_enghouse__transactions contain null timestamp values. This leads to null transaction entries in downstream tables. On a brief investigation, it seems like this occurs when external_enghouse.transactions does not contain a seconds field
To Reproduce
with staging_transactions as (
select * from cal-itp-data-infra.staging.stg_enghouse__transactions where timestamp is null
),
external_transactions as (
select * from cal-itp-data-infra.external_enghouse.transactions
)
select
staging_transactions.id,
staging_transactions.operation,
staging_transactions.timestamp as staging_timestamp,
staging_transactions.dt,
staging_transactions.token as staging_token,
external_transactions.timestamp as external_timestamp,
external_transactions.token as external_token
from staging_transactions
left join external_transactions
on staging_transactions.operator_id = external_transactions.operatorid
and staging_transactions.id = external_transactions.id
Expected behavior
All timestamp values in staging.stg_enghouse__transactions and downstream tables are not null. If Enghouse sends timestamps without a seconds field, the seconds field should probably be entered as :00, but we should raise this as an issue with them.
Additional context
Have not investigated whether this is occuring in other Enghouse staging tables
Describe the bug
As a payments analyst, I expect all Enghouse transactions entries to be associated with a non-null
timestamp. However, some rows instaging.stg_enghouse__transactionscontain nulltimestampvalues. This leads to null transaction entries in downstream tables. On a brief investigation, it seems like this occurs whenexternal_enghouse.transactionsdoes not contain a seconds fieldTo Reproduce
Expected behavior
All
timestampvalues instaging.stg_enghouse__transactionsand downstream tables are not null. If Enghouse sends timestamps without a seconds field, the seconds field should probably be entered as :00, but we should raise this as an issue with them.Additional context
Have not investigated whether this is occuring in other Enghouse staging tables