Clarify search scope and reconstruction methodology for Radjenovic_2013.csv

[JelsonMatheus]

Hi,

First, thank you for maintaining this dataset — it's been very useful for testing screening pipelines.

While validating Radjenovic_2013.csv against the original paper (Radjenović et al., 2013, Software fault prediction metrics: A systematic literature review, Information and Software Technology), I noticed some differences in scope that I'd like to clarify:

1. Search scope: The original paper searched 7 digital libraries (ACM, Compendex, IEEE Xplore, Inspec, ISI Web of Science, ScienceDirect, SpringerLink), retrieving 13,126 candidates in total. The CSV in this repository contains ~6,000 records. Could you confirm whether this dataset was built using only IEEE Xplore results, as referenced in the FASTREAD paper (Yu et al.)?

2. Search string: Was the original search string used as-is, or adapted/broadened for IEEE Xplore's syntax? If adapted, is the modified string documented anywhere?

3. Included studies: The paper's final list has 106 included primary studies (90 from database search + 13 from snowballing + 3 from author contact). The CSV has 48 labeled as included. Can you confirm these 48 are the subset of the 106 that were retrievable specifically from the IEEE Xplore search, as opposed to a different selection criterion?

Thanks again for the work on this — just trying to make sure I'm interpreting the dataset correctly before using it as a benchmark.

Best regards,
Jelson Araújo

[EmilyWes]

Hi Jelson,

Good questions, Radjenovic is indeed one of the more interesting datasets in this sense.

We took the dataset as it was created by Yu et al. and published here (https://doi.org/10.5281/zenodo.1162952).
This means that most of your questions can be answered by their paper which I believe you already found (https://arxiv.org/pdf/1612.03224)

Especially section 5.2 states the process:
"For each of the datasets, the search string S and the final inclusion list F from the original
publication are used for the data collection. We retrieve the initial candidate collection E
from IEEE Xplore with the search string (slightly modified to meet the requirement of IEEE
Xplore). Then make a final list of inclusion R as R = F ∩ E."

So to answer your questions directly:

1. Correct, we took the dataset from Yu et al. containing the records from only IEEE xplore.
2. Yes, they slightly modified it to work on IEEE Xplore. I am not aware of a location they published their adjusted search string.
3. Correct, the 48 is the subset of 106 that is findable in IEEE Xplore.

Kind regards,
Emily