Implement altrep mmap approach

[Bisaloo]

No description provided.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 81.45161% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.14%. Comparing base (1cde19b) to head (1b2b1bd).

Files with missing lines	Patch %	Lines
src/altrep_mmap.c	73.33%	16 Missing ⚠️
R/read_npy.R	68.42%	6 Missing ⚠️
src/type_conversion.c	97.72%	1 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #22      +/-   ##
==========================================
- Coverage   90.98%   87.14%   -3.85%     
==========================================
  Files           6        7       +1     
  Lines         355      420      +65     
==========================================
+ Hits          323      366      +43     
- Misses         32       54      +22     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 671889f is merged into main:

• ✔️pkg_load: 5.19ms -> 5.25ms [-0.91%, +3.16%]
• ❗🐌read_bigendian: 819µs -> 854µs [+3.36%, +5.21%]
• ❗🐌read_bool: 775µs -> 830µs [+4.98%, +9.19%]
• ❗🐌read_empty: 783µs -> 825µs [+4.01%, +6.92%]
• ❗🐌read_float16: 764µs -> 810µs [+4.42%, +7.52%]
• ❗🐌read_float32: 779µs -> 818µs [+3.87%, +6.16%]
• ❗🐌read_float64: 794µs -> 834µs [+3.51%, +6.56%]
• ❗🐌read_int16: 812µs -> 854µs [+3.85%, +6.38%]
• ❗🐌read_int32: 801µs -> 844µs [+4.6%, +6.3%]
• ❗🐌read_int64: 1.21ms -> 1.25ms [+2.66%, +4.03%]
• ❗🐌read_int8: 805µs -> 847µs [+4.32%, +6.09%]
• ❗🐌read_npz: 2.39ms -> 2.44ms [+0.6%, +3.89%]
• ❗🐌read_string: 725µs -> 765µs [+4.46%, +6.51%]
• ✔️read_structured: 1.54ms -> 1.57ms [-0.72%, +4.51%]
• ❗🐌read_uint16: 744µs -> 783µs [+4.13%, +6.32%]
• ❗🐌read_uint32: 993µs -> 1.04ms [+2.04%, +6.85%]
• ❗🐌read_uint64: 1.14ms -> 1.18ms [+1.77%, +4.06%]
• ❗🐌read_uint8: 804µs -> 850µs [+5%, +6.55%]
• ❗🐌read_unicode: 874µs -> 915µs [+3.04%, +6.3%]
  Further explanation regarding interpretation and methodology can be found in the documentation.

[Bisaloo]

On a int64 array with 1M elements:

cross::bench_branches(
  {
    library(grumpy)
    bench::mark(
      read_npy("inst/extdata/test_large.npy", lazy = TRUE),
      iterations = 50
    )
  }
)

✔ Installing branch 'lazy-mmap'
✔ Installing branch 'main'
✔ Running `expr` across variants
# A tibble: 2 × 14
  branch    expression     min median `itr/sec` mem_alloc `gc/sec` n_itr  n_gc total_time result   memory    
  <chr>     <bch:expr>  <bch:> <bch:>     <dbl> <bch:byt>    <dbl> <int> <dbl>   <bch:tm> <list>   <list>    
1 lazy-mmap "read_npy(… 1.82ms 1.97ms      496.    4.07MB     31.7    47     3     94.8ms <int[…]> <Rprofmem>
2 main      "read_npy(…  5.5ms  5.7ms      173.   11.68MB     54.6    38    12    219.6ms <int[…]> <Rprofmem>
# ℹ 2 more variables: time <list>, gc <list>

[github-actions]

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 1b2b1bd is merged into main:

• ✔️pkg_load: 4.88ms -> 4.87ms [-1.84%, +1.46%]
• ❗🐌read_bigendian: 841µs -> 859µs [+0.46%, +3.88%]
• ❗🐌read_bool: 837µs -> 866µs [+2.15%, +4.68%]
• ❗🐌read_empty: 816µs -> 836µs [+1.53%, +3.29%]
• ❗🐌read_float16: 862µs -> 887µs [+1.62%, +4.14%]
• ❗🐌read_float32: 877µs -> 901µs [+1.19%, +4.31%]
• ❗🐌read_float64: 860µs -> 888µs [+2.21%, +4.44%]
• ❗🐌read_int16: 811µs -> 854µs [+2.19%, +8.58%]
• ✔️read_int32: 833µs -> 848µs [-0.87%, +4.59%]
• ✔️read_int64: 1.3ms -> 1.31ms [-0.78%, +2.6%]
• ❗🐌read_int8: 808µs -> 838µs [+2.91%, +4.56%]
• ✔️read_npz: 2.45ms -> 2.47ms [-1.37%, +2.95%]
• ❗🐌read_string: 814µs -> 854µs [+1.81%, +8.01%]
• ✔️read_structured: 1.48ms -> 1.49ms [-0.69%, +2.18%]
• ❗🐌read_uint16: 866µs -> 899µs [+1.16%, +6.6%]
• ❗🐌read_uint32: 1.09ms -> 1.11ms [+1.23%, +2.38%]
• ❗🐌read_uint64: 1.18ms -> 1.21ms [+1.74%, +3.15%]
• ❗🐌read_uint8: 896µs -> 934µs [+2.27%, +6.19%]
• ✔️read_unicode: 991µs -> 1.01ms [-0.42%, +3.97%]
  Further explanation regarding interpretation and methodology can be found in the documentation.