Skip to content

Bug: staging.stg_enghouse__transactions Has Null timestamp Entries #5694

Description

@mrtopsyt

Describe the bug
As a payments analyst, I expect all Enghouse transactions entries to be associated with a non-null timestamp. However, some rows in staging.stg_enghouse__transactions contain null timestamp values. This leads to null transaction entries in downstream tables. On a brief investigation, it seems like this occurs when external_enghouse.transactions does not contain a seconds field

To Reproduce

with staging_transactions as (
  select * from cal-itp-data-infra.staging.stg_enghouse__transactions where timestamp is null
),
external_transactions as (
  select * from cal-itp-data-infra.external_enghouse.transactions 
)

select 
  staging_transactions.id, 
  staging_transactions.operation, 
  staging_transactions.timestamp as staging_timestamp,
  staging_transactions.dt,
  staging_transactions.token as staging_token,
  external_transactions.timestamp as external_timestamp,
  external_transactions.token as external_token
from staging_transactions 
left join external_transactions 
  on staging_transactions.operator_id = external_transactions.operatorid
  and staging_transactions.id = external_transactions.id

Expected behavior
All timestamp values in staging.stg_enghouse__transactions and downstream tables are not null. If Enghouse sends timestamps without a seconds field, the seconds field should probably be entered as :00, but we should raise this as an issue with them.

Additional context
Have not investigated whether this is occuring in other Enghouse staging tables

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions