Dates covered vs dates collected

[ruebot]

Should we make a distinction between the two in the metadata?

[edsu]

In b73897d I added an added dataset field to indicate when the dataset was added to the catalog, which is different from the date the dataset was published.

But it sounds like you are talking about a different kind of date distinction. Are you thinking of a situation where the time period in which data collection ran does not match the time period of the tweets collected? I think this can only happen when running searches right?

[ruebot]

Are you thinking of a situation where the time period in which data collection ran does not match the time period of the tweets collected? I think this can only happen when running searches right?

Exactly!

In the Scholars Portal Dataverse, where I put our datasets, we have the option of adding date ranges for "date covered" and "dates collected". This works well for all the major collections I've done with twarc, since I use a strategy of filter and search, and then deduplication.