Skip to content

Csv parser does not create check_s3_folder_exists task correctly #272

Description

@helen-m-lin

Describe the bug
The csv parser does not create the check_s3_folder_exists task correctly. There is a typo with the task name, which will result in the task not actually being skipped in Airflow:

"check_s3_folder_exists_task": check_s3_folder_exists_task,

To Reproduce
Steps to reproduce the behavior:

  1. Submit a csv or excel job with "force_cloud_sync" set to True
  2. Observe that the "check_s3_folder_exists" task is not skipped in Airflow.

Expected behavior
A csv with force_cloud_sync set to True should create an upload job with a check_s3_folder_exists Skip Task.

Additional context
I believe the final_check_s3_folder_exist task can also be removed. This task has been removed from the airflow DAG.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinggood first issueGood for newcomers

Type

No type

Fields

No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions