As I've been going through with making the CONSORT diagram, I came upon some issues...
I'm not sure how, though I think it might have something to do with updates to packages, but there are some titles we reviewed that aren't in our "original" (the current file we have at data/review/titles.csv) titles we extracted from OpenAlex. I've been reviewing the commit history and I can't seem to find how it happened. One commit at f288d3f shows extracting around 735 articles (but originally there were ~4400)...
But then oddly in 69055f8 when I made duplicates of the files for us each to review, we ended up reviewing 770... Not sure where that extra 35 titles came from.
Then the later commit at d81da12 show 3102 (-1 for header line) records being extracted after I renamed the file in the targets pipeline at 60e0ef8 and than re-ran the build.
And even more odd! Only 2 of the 13 articles we reviewed until the end are actually in the current initial extraction (in data/review/titles.csv vs data/review/abstracts/)...
Sooooo my guess is either that the OpenAlex package was updated which may have changed how the extraction algorithm worked, or they added another data source or changed their existing data sources. I think we should add this to the paper to explain some things, to highlight the difficulties with being reproducible, with the way computation and software in general works, and to highlight even more of a need for things to be recorded down as code and built in a reproducible pipeline format (which we only started doing in a more structured way later, after many of these issues already happened).
So instead of making a CONSORT diagram, I'll just make a table and describe our issues there.
As I've been going through with making the CONSORT diagram, I came upon some issues...
I'm not sure how, though I think it might have something to do with updates to packages, but there are some titles we reviewed that aren't in our "original" (the current file we have at
data/review/titles.csv) titles we extracted from OpenAlex. I've been reviewing the commit history and I can't seem to find how it happened. One commit at f288d3f shows extracting around 735 articles (but originally there were ~4400)...But then oddly in 69055f8 when I made duplicates of the files for us each to review, we ended up reviewing 770... Not sure where that extra 35 titles came from.
Then the later commit at d81da12 show 3102 (-1 for header line) records being extracted after I renamed the file in the targets pipeline at 60e0ef8 and than re-ran the build.
And even more odd! Only 2 of the 13 articles we reviewed until the end are actually in the current initial extraction (in
data/review/titles.csvvsdata/review/abstracts/)...Sooooo my guess is either that the OpenAlex package was updated which may have changed how the extraction algorithm worked, or they added another data source or changed their existing data sources. I think we should add this to the paper to explain some things, to highlight the difficulties with being reproducible, with the way computation and software in general works, and to highlight even more of a need for things to be recorded down as code and built in a reproducible pipeline format (which we only started doing in a more structured way later, after many of these issues already happened).
So instead of making a CONSORT diagram, I'll just make a table and describe our issues there.