Skip to content

Commit 2cff474

Browse files
emfelthamemfeltham
authored
Merge pull request #16 from human-nature-lab/readiness
Readiness
2 parents 8576979 + c498e4d commit 2cff474

114 files changed

Lines changed: 20959 additions & 19344 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/nightly-ci.yml‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
name: Nightly CI
2+
3+
on:
4+
schedule:
5+
# Run at 2 AM UTC every day
6+
- cron: '0 2 * * *'
7+
workflow_dispatch: # Allow manual triggering
8+
9+
jobs:
10+
comprehensive-validation:
11+
runs-on: ubuntu-latest
12+
13+
steps:
14+
- uses: actions/checkout@v4
15+
16+
- name: Setup Julia
17+
uses: julia-actions/setup-julia@v1
18+
with:
19+
version: '1.11'
20+
21+
- name: Cache Julia packages
22+
uses: julia-actions/cache@v1
23+
24+
- name: Install dependencies
25+
run: |
26+
julia --project=. -e "using Pkg; Pkg.instantiate()"
27+
28+
- name: Run package tests
29+
run: |
30+
julia --project=. -e "using Pkg; Pkg.test()"
31+
32+
- name: Run comprehensive seed sweep
33+
run: |
34+
julia --project=. scripts/seed_sweep.jl --seeds 30 --N 600 --T 200 --treated-share 0.10 --F 1:8 --L -30:-1 --phi 0.0 --rho 0.5 --sigma-y 0.2 --beta1 0.3 --beta2 0.5 --out test/validation/nightly_seed_sweep.json
35+
36+
- name: Run bias and coverage analysis
37+
run: |
38+
julia --project=. scripts/sim_bias_coverage.jl --seeds 20 --iterations 400 --out test/validation/nightly_bias_coverage.json
39+
40+
- name: Run placebo permutation tests
41+
run: |
42+
julia --project=. scripts/placebo_permutation.jl --permutations 300 --iterations 100 --out test/validation/nightly_placebo.json
43+
44+
- name: Upload comprehensive validation artifacts
45+
uses: actions/upload-artifact@v4
46+
if: always()
47+
with:
48+
name: nightly-validation-results
49+
path: |
50+
test/validation/nightly_*.json
51+
retention-days: 90
52+
53+
- name: Create validation summary
54+
if: always()
55+
run: |
56+
echo "# Nightly Validation Summary" > validation_summary.md
57+
echo "Generated: $(date)" >> validation_summary.md
58+
echo "" >> validation_summary.md
59+
60+
if [ -f test/validation/nightly_seed_sweep.json ]; then
61+
echo "## Seed Sweep Results" >> validation_summary.md
62+
julia --project=. -e "
63+
using JSON
64+
data = JSON.parsefile(\"test/validation/nightly_seed_sweep.json\")
65+
summary = data[\"summary\"]
66+
println(\"- MAE Mean: \", summary[\"mae_mean\"])
67+
println(\"- MAE SD: \", summary[\"mae_sd\"])
68+
println(\"- MxE Max: \", summary[\"mxe_max\"])
69+
" >> validation_summary.md
70+
fi
71+
72+
if [ -f test/validation/nightly_bias_coverage.json ]; then
73+
echo "" >> validation_summary.md
74+
echo "## Bias & Coverage Results" >> validation_summary.md
75+
julia --project=. -e "
76+
using JSON
77+
data = JSON.parsefile(\"test/validation/nightly_bias_coverage.json\")
78+
bias_summary = data[\"bias\"][\"summary\"]
79+
coverage_summary = data[\"coverage\"][\"summary\"]
80+
println(\"- Overall MAE: \", bias_summary[\"overall_mae\"])
81+
println(\"- Overall RMSE: \", bias_summary[\"overall_rmse\"])
82+
println(\"- Coverage Rate: \", coverage_summary[\"overall_coverage\"])
83+
" >> validation_summary.md
84+
fi
85+
86+
if [ -f test/validation/nightly_placebo.json ]; then
87+
echo "" >> validation_summary.md
88+
echo "## Placebo Test Results" >> validation_summary.md
89+
julia --project=. -e "
90+
using JSON
91+
data = JSON.parsefile(\"test/validation/nightly_placebo.json\")
92+
summary = data[\"placebo\"][\"summary\"]
93+
println(\"- Type I Error Rate: \", summary[\"overall_type1\"])
94+
println(\"- Successful Permutations: \", summary[\"n_successful\"], \"/\", summary[\"n_attempted\"])
95+
" >> validation_summary.md
96+
fi
97+
98+
- name: Upload validation summary
99+
uses: actions/upload-artifact@v4
100+
if: always()
101+
with:
102+
name: validation-summary
103+
path: validation_summary.md
104+
retention-days: 90

‎.github/workflows/pr-ci.yml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
name: PR CI
2+
3+
on:
4+
pull_request:
5+
branches: [ main ]
6+
7+
jobs:
8+
test:
9+
runs-on: ubuntu-latest
10+
11+
steps:
12+
- uses: actions/checkout@v4
13+
14+
- name: Setup Julia
15+
uses: julia-actions/setup-julia@v1
16+
with:
17+
version: '1.11'
18+
19+
- name: Cache Julia packages
20+
uses: julia-actions/cache@v1
21+
22+
- name: Install dependencies
23+
run: |
24+
julia --project=. -e "using Pkg; Pkg.instantiate()"
25+
26+
- name: Run package tests
27+
run: |
28+
julia --project=. -e "using Pkg; Pkg.test()"
29+
30+
- name: Run validation seed sweep
31+
run: |
32+
julia --project=. scripts/seed_sweep.jl --seeds 10 --N 800 --T 250 --treated-share 0.08 --F 1:8 --L -30:-1 --phi 0.0 --rho 0.5 --sigma-y 0.18 --beta1 0.3 --beta2 0.5 --out test/validation/pr_seed_sweep.json
33+
34+
- name: Upload validation artifacts
35+
uses: actions/upload-artifact@v4
36+
if: always()
37+
with:
38+
name: pr-validation-results
39+
path: |
40+
test/validation/pr_seed_sweep.json
41+
retention-days: 30

‎.gitignore‎

Lines changed: 60 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,22 @@
1-
# Julia
1+
# Julia package files
22
*.jl.cov
33
*.jl.*.cov
44
*.jl.mem
55
deps/deps.jl
66
deps/build.log
77
deps/downloads/
8+
LocalPreferences.toml
9+
10+
# Documentation builds
811
docs/build/
912
docs/site/
13+
14+
# Package management
1015
Manifest.toml
1116

17+
# Julia REPL history
18+
.julia_history
19+
1220
# OS generated files
1321
.DS_Store
1422
.DS_Store?
@@ -17,36 +25,80 @@ Manifest.toml
1725
.Trashes
1826
ehthumbs.db
1927
Thumbs.db
28+
*~
2029

21-
# Editor files
30+
# Editor and IDE files
2231
.vscode/
2332
.idea/
2433
*.swp
2534
*.swo
26-
*~
35+
.vim/
36+
.netrwhist
37+
.emacs.d/
38+
.atom/
2739

2840
# Jupyter Notebook
2941
.ipynb_checkpoints
42+
*.ipynb
3043

31-
# Environment files
44+
# Python (if mixed development)
45+
__pycache__/
46+
*.py[cod]
47+
*$py.class
48+
*.egg-info/
49+
.pytest_cache/
50+
51+
# R (if mixed development)
52+
.Rhistory
53+
.RData
54+
.Ruserdata
55+
56+
# Environment and secrets
3257
.env
3358
.env.local
59+
.env.*.local
3460

35-
# Temporary files
61+
# Temporary and log files
3662
*.tmp
3763
*.temp
3864
*.log
65+
*.out
66+
*.aux
67+
*.fls
68+
*.fdb_latexmk
69+
*.synctex.gz
3970

40-
# Package build artifacts
71+
# Build artifacts
4172
*.so
4273
*.dll
4374
*.dylib
75+
*.a
76+
*.lib
4477

45-
# Test coverage
78+
# Test coverage and profiling
4679
coverage/
4780
*.lcov
81+
*.info
82+
profile.json
4883

49-
# Local development
84+
# Development directories
5085
scratch/
5186
experiments/
5287
sandbox/
88+
tmp/
89+
temp/
90+
debug/
91+
92+
# Data files (customize based on project needs)
93+
# *.csv
94+
*.xlsx
95+
*.json
96+
*.jld2
97+
*.h5
98+
*.hdf5
99+
100+
# Output files
101+
plots/
102+
figures/
103+
results/
104+
output/

‎Manifest.toml‎

Lines changed: 41 additions & 3 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Project.toml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@ version = "0.1.0"
1111

1212
[deps]
1313
Accessors = "7d9f7c33-5ae7-4f3b-8dc6-eff91059b697"
14+
BenchmarkTools = "6e4b80f9-dd63-53aa-95a3-0cdb28fa8baf"
15+
CSV = "336ed68f-0bac-5ca0-87d4-7b16caf5d00b"
1416
Colors = "5ae59095-9a9b-59fe-a467-6f913c188581"
1517
DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
1618
DataFramesMeta = "1313f7d8-7da2-5740-9ea0-a2ca25f37964"
@@ -28,8 +30,10 @@ Statistics = "10745b16-79ce-11e8-11f9-7d13ad32a3b2"
2830
StatsBase = "2913bbd2-ae8a-5f71-8c99-4fb6c76f3a91"
2931

3032
[compat]
33+
BenchmarkTools = "1.6.0"
3134
Documenter = "1.14.1"
32-
julia = "1.6"
35+
JLD2 = "0.5.15"
36+
julia = "1.10"
3337

3438
[extras]
3539
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

0 commit comments

Comments
 (0)