Wenbin Hu, Huihao Jing, Haochen Shi, Changxuan Fan, Haoran Li, Yangqiu Song
Hong Kong University of Science and Technology
./data/rules contains our collected rules from regulations and policies.
./data/cases contains case data collected by our searching agent.
The dataset can also be found at: https://huggingface.co/datasets/hubin/OmniCompliance100K.
./search_agent: The code for searching agent.
In ./search_agent/config.py, please config your grok's xai api:
xai_key = 'your key'
Then, run:
cd search_agent
run_search.sh
In our paper, under the Section 4.3 Rule-Case Alignment Test, we evaluate the quality of our dataset, with a designed alignment test. The code is under ./alignment_test.
We benchmark common LLMs across all sub-domains in our proposed dataset.
-
Code:
./benchmarking. Please config your api in./benchmarking/config.py -
Results:
{
"example_name": "EDPS Decision on EUROPOL Biometric Data Processing",
"example_background": "EUROPOL, as an agency of the European Union, processes large volumes of personal data including criminal records, biometric identifiers, and travel information in its databases for law enforcement cooperation. In 2021, the EDPS examined EUROPOL's practices under Regulation 45/2001, particularly the expansion of facial recognition data storage beyond initial legal limits. The inquiry revealed that EUROPOL had continued processing deleted data and expanded retention without adequate legal basis or proportionality assessments. Data was sourced from member states and third countries, involving millions of records. Concerns included lack of transparency in data minimization and purpose limitation, with processing extending to non-criminal datasets. This occurred during the phase where Regulation 45/2001 was being adapted to GDPR standards.",
"process_and_outcome": "In February 2022, the EDPS issued a decision ordering EUROPOL to delete unlawfully stored data within two months and revise its data processing policies. The agency complied, implementing new governance structures and reporting mechanisms.",
"involved_parties": [
"European Data Protection Supervisor (EDPS)",
"EUROPOL"
],
"applicable_regulations_or_policies": [
"Regulation (EC) No 45/2001",
"GDPR principles via Article 98"
],
"relation_to_rule": "VIOLATES",
"dates": "2021"
}