Skip to content

Commit ef0ccf8

Browse files
committed
update
1 parent 2a88d69 commit ef0ccf8

4 files changed

Lines changed: 36 additions & 0 deletions

File tree

assets/file-list.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -451,3 +451,5 @@
451451
2026-04-30_AI_enhanced_Chinese.jsonl
452452
2026-05-01.jsonl
453453
2026-05-01_AI_enhanced_Chinese.jsonl
454+
2026-05-02.jsonl
455+
2026-05-02_AI_enhanced_Chinese.jsonl

data/2026-05-02.jsonl

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"id": "2604.27977", "categories": ["cs.AI", "cs.LG"], "pdf": "https://arxiv.org/pdf/2604.27977", "abs": "https://arxiv.org/abs/2604.27977", "authors": ["Hanane Nour Moussa", "Yifei Li", "Zhuoyang Li", "Yankai Yang", "Cheng Tang", "Tianshu Zhang", "Nesreen K. Ahmed", "Ali Payani", "Ziru Chen", "Huan Sun"], "title": "D3-Gym: Constructing Real-World Verifiable Environments for Data-Driven Discovery", "comment": null, "summary": "Despite recent progress in language models and agents for scientific data-driven discovery, further advancing their capabilities is held back by the absence of verifiable environments representing real-world scientific tasks.To fill this gap, we introduce D3-Gym, the first automatically constructed dataset with verifiable environments for scientific Data-Driven Discovery. D3-Gym comprises (1) 565 tasks sourced from 239 real scientific repositories across four disciplines where (2) each task is equipped with a natural language instruction, an executable environment with pre-installed dependencies, input dataset and artifact previews, a reference code solution, and an automatically synthesized evaluation script. Rigorous evaluation of the quality of the verification signal in D3-Gym confirms that our evaluation scripts achieve 87.5% agreement with human-annotated gold standards and strong alignment in domain-specific evaluation logic, showing their scientific soundness. Further, training on trajectories sampled from D3-Gym yields consistent and substantial gains across Qwen3 models of varying sizes on ScienceAgentBench, boosting Qwen3-32B by 7.8 absolute points and substantially shrinking the gap with strong proprietary models. All D3-Gym artifacts (environments, creation workflow, trajectories, and models) can be found at https://github.com/OSU-NLP-Group/D3-Gym."}

data/2026-05-02.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
<div id=toc></div>
2+
3+
# Table of Contents
4+
5+
- [cs.AI](#cs.AI) [Total: 1]
6+
7+
8+
<div id='cs.AI'></div>
9+
10+
# cs.AI [[Back]](#toc)
11+
12+
### [1] [D3-Gym: Constructing Real-World Verifiable Environments for Data-Driven Discovery](https://arxiv.org/abs/2604.27977)
13+
*Hanane Nour Moussa,Yifei Li,Zhuoyang Li,Yankai Yang,Cheng Tang,Tianshu Zhang,Nesreen K. Ahmed,Ali Payani,Ziru Chen,Huan Sun*
14+
15+
Main category: cs.AI
16+
17+
TL;DR: D3-Gym是一个自动构建的、具有可验证环境的科学数据驱动发现数据集,包含565个任务和评估脚本,用于训练和评估AI代理。
18+
19+
20+
<details>
21+
<summary>Details</summary>
22+
Motivation: 现有语言模型和代理在科学数据驱动发现中的能力提升受到缺乏可验证真实科学任务环境的阻碍。
23+
24+
Method: 从239个真实科学仓库中提取565个任务,每个任务包含自然语言指令、可执行环境、输入数据、参考代码和自动合成的评估脚本,并通过自动化流程构建。
25+
26+
Result: 评估脚本与人工标注标准的一致性达87.5%,在Qwen3模型上训练后,ScienceAgentBench性能提升(如Qwen3-32B提升7.8个百分点),缩小了与专有模型的差距。
27+
28+
Conclusion: D3-Gym提供了可靠的验证环境,能有效提升AI代理在科学发现任务中的性能,且资源已开源。
29+
30+
Abstract: Despite recent progress in language models and agents for scientific data-driven discovery, further advancing their capabilities is held back by the absence of verifiable environments representing real-world scientific tasks.To fill this gap, we introduce D3-Gym, the first automatically constructed dataset with verifiable environments for scientific Data-Driven Discovery. D3-Gym comprises (1) 565 tasks sourced from 239 real scientific repositories across four disciplines where (2) each task is equipped with a natural language instruction, an executable environment with pre-installed dependencies, input dataset and artifact previews, a reference code solution, and an automatically synthesized evaluation script. Rigorous evaluation of the quality of the verification signal in D3-Gym confirms that our evaluation scripts achieve 87.5% agreement with human-annotated gold standards and strong alignment in domain-specific evaluation logic, showing their scientific soundness. Further, training on trajectories sampled from D3-Gym yields consistent and substantial gains across Qwen3 models of varying sizes on ScienceAgentBench, boosting Qwen3-32B by 7.8 absolute points and substantially shrinking the gap with strong proprietary models. All D3-Gym artifacts (environments, creation workflow, trajectories, and models) can be found at https://github.com/OSU-NLP-Group/D3-Gym.
31+
32+
</details>
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"id": "2604.27977", "categories": ["cs.AI", "cs.LG"], "pdf": "https://arxiv.org/pdf/2604.27977", "abs": "https://arxiv.org/abs/2604.27977", "authors": ["Hanane Nour Moussa", "Yifei Li", "Zhuoyang Li", "Yankai Yang", "Cheng Tang", "Tianshu Zhang", "Nesreen K. Ahmed", "Ali Payani", "Ziru Chen", "Huan Sun"], "title": "D3-Gym: Constructing Real-World Verifiable Environments for Data-Driven Discovery", "comment": null, "summary": "Despite recent progress in language models and agents for scientific data-driven discovery, further advancing their capabilities is held back by the absence of verifiable environments representing real-world scientific tasks.To fill this gap, we introduce D3-Gym, the first automatically constructed dataset with verifiable environments for scientific Data-Driven Discovery. D3-Gym comprises (1) 565 tasks sourced from 239 real scientific repositories across four disciplines where (2) each task is equipped with a natural language instruction, an executable environment with pre-installed dependencies, input dataset and artifact previews, a reference code solution, and an automatically synthesized evaluation script. Rigorous evaluation of the quality of the verification signal in D3-Gym confirms that our evaluation scripts achieve 87.5% agreement with human-annotated gold standards and strong alignment in domain-specific evaluation logic, showing their scientific soundness. Further, training on trajectories sampled from D3-Gym yields consistent and substantial gains across Qwen3 models of varying sizes on ScienceAgentBench, boosting Qwen3-32B by 7.8 absolute points and substantially shrinking the gap with strong proprietary models. All D3-Gym artifacts (environments, creation workflow, trajectories, and models) can be found at https://github.com/OSU-NLP-Group/D3-Gym.", "AI": {"tldr": "D3-Gym是一个自动构建的、具有可验证环境的科学数据驱动发现数据集,包含565个任务和评估脚本,用于训练和评估AI代理。", "motivation": "现有语言模型和代理在科学数据驱动发现中的能力提升受到缺乏可验证真实科学任务环境的阻碍。", "method": "从239个真实科学仓库中提取565个任务,每个任务包含自然语言指令、可执行环境、输入数据、参考代码和自动合成的评估脚本,并通过自动化流程构建。", "result": "评估脚本与人工标注标准的一致性达87.5%,在Qwen3模型上训练后,ScienceAgentBench性能提升(如Qwen3-32B提升7.8个百分点),缩小了与专有模型的差距。", "conclusion": "D3-Gym提供了可靠的验证环境,能有效提升AI代理在科学发现任务中的性能,且资源已开源。"}}

0 commit comments

Comments
 (0)