Remove and clean up unneeded data in Airtable

[evansiroky]

User story / feature request

As a user of Airtable,
I want any data that is not formally maintained or stale to be removed from Airtable and the data warehouse,
So that I am not misled into using data that is not maintained,
And so that I am forced to use better data sources.

Acceptance Criteria

• This work should focus on all tables within the California Transit Base and the Tech Stacks Base
• There should be some research done into which columns aren't used in the warehouse and are not used during any internal processes.
• There should focus on doing some outreach within our team to double check whether any member of the team is still using some columns.
• Some data like the NTD data in Airtable should be outright removed since there are better data sources in the warehouse and it has become stale.
• If some columns are not being used and can be removed, any necessary warehouse changes should be made to ensure the deprecated columns get removed from the warehouse and that the warehouse will be able to work in the future after deleting the columns/tables from Airtable.
• The columns and tables should be deleted from Airtable.

Notes

I don't believe we have cleaned up Airtable thoroughly since its inception about 4 years ago!

[vevetron]

I think this ticket should focus on just figuring out the unneeded columns and making a report about that. Focus should be on which columns are used in the reports product. A later ticket should systematically remove them. I believe @billysghub might work on this, but can pass to someone on Data Infra or DS.

[vevetron]

• We've never done a cleanup of this.
• Make an epic where another issue removes things!

[billysghub]

I think this ticket should focus on just figuring out the unneeded columns and making a report about that. Focus should be on which columns are used in the reports product. A later ticket should systematically remove them. I believe @billysghub might work on this, but can pass to someone on Data Infra or DS.

I am happy to help with this but if there is someone on the Data Infra or DS team who has direct experience with which fields are truly not needed maybe they should do a first pass at adding fields to the report, that would save me a lot of time because I would basically be starting from scratch. I could then focus on the less obvious fields and spend time researching and asking around about them.