Skip to content

add pyarrow dtype to oed json#262

Merged
benhayes21 merged 10 commits into
release/v5.0.0from
enhancement/add_pyarrow_dtype
Dec 18, 2025
Merged

add pyarrow dtype to oed json#262
benhayes21 merged 10 commits into
release/v5.0.0from
enhancement/add_pyarrow_dtype

Conversation

@sstruzik

@sstruzik sstruzik commented Nov 25, 2025

Copy link
Copy Markdown
Contributor

Add column specific pyarrow dtype information to the jsonoed schema

pyarrow dtype will store the information on what data type should be use when selecting pyarrow and the loading backend.

@sstruzik sstruzik self-assigned this Nov 25, 2025
@sstruzik sstruzik added the feature New feature or request label Nov 25, 2025
@github-actions

github-actions Bot commented Nov 25, 2025

Copy link
Copy Markdown

Build Preview

You can find files attached to the below linked Workflow Run URL (Logs).
Please note that files only stay for around 14 days!

Name Link
Commit 026b4ba
Build https://github.com/OasisLMF/ODS_OpenExposureData/actions/runs/19667093122
Excel File excel_spec.zip
JSON File extracted_spec.zip

@github-actions

github-actions Bot commented Nov 26, 2025

Copy link
Copy Markdown

Build Preview

You can find files attached to the below linked Workflow Run URL (Logs).
Please note that files only stay for around 14 days!

Name Link
Commit 4f4ffd8
Build https://github.com/OasisLMF/ODS_OpenExposureData/actions/runs/19701667835
Excel File excel_spec.zip
JSON File extracted_spec.zip

@github-actions

github-actions Bot commented Nov 27, 2025

Copy link
Copy Markdown

Build Preview

You can find files attached to the below linked Workflow Run URL (Logs).
Please note that files only stay for around 14 days!

Name Link
Commit 4ba098b
Build https://github.com/OasisLMF/ODS_OpenExposureData/actions/runs/19734072302
Excel File excel_spec.zip
JSON File extracted_spec.zip

@github-actions

github-actions Bot commented Nov 27, 2025

Copy link
Copy Markdown

Build Preview

You can find files attached to the below linked Workflow Run URL (Logs).
Please note that files only stay for around 14 days!

Name Link
Commit 6e158c0
Build https://github.com/OasisLMF/ODS_OpenExposureData/actions/runs/19735460221
Excel File excel_spec.zip
JSON File extracted_spec.zip

@github-actions

github-actions Bot commented Nov 27, 2025

Copy link
Copy Markdown

Build Preview

You can find files attached to the below linked Workflow Run URL (Logs).
Please note that files only stay for around 14 days!

Name Link
Commit 5f4d037
Build https://github.com/OasisLMF/ODS_OpenExposureData/actions/runs/19742499526
Excel File excel_spec.zip
JSON File extracted_spec.zip

@github-actions

github-actions Bot commented Nov 27, 2025

Copy link
Copy Markdown

Build Preview

You can find files attached to the below linked Workflow Run URL (Logs).
Please note that files only stay for around 14 days!

Name Link
Commit 62ebce9
Build https://github.com/OasisLMF/ODS_OpenExposureData/actions/runs/19742732131
Excel File excel_spec.zip
JSON File extracted_spec.zip

@github-actions

github-actions Bot commented Nov 27, 2025

Copy link
Copy Markdown

Build Preview

You can find files attached to the below linked Workflow Run URL (Logs).
Please note that files only stay for around 14 days!

Name Link
Commit a86040c
Build https://github.com/OasisLMF/ODS_OpenExposureData/actions/runs/19742904267
Excel File excel_spec.zip
JSON File extracted_spec.zip

@github-actions

github-actions Bot commented Nov 27, 2025

Copy link
Copy Markdown

Build Preview

You can find files attached to the below linked Workflow Run URL (Logs).
Please note that files only stay for around 14 days!

Name Link
Commit e29b7e7
Build https://github.com/OasisLMF/ODS_OpenExposureData/actions/runs/19743058357
Excel File excel_spec.zip
JSON File extracted_spec.zip

@github-actions

github-actions Bot commented Nov 27, 2025

Copy link
Copy Markdown

Build Preview

You can find files attached to the below linked Workflow Run URL (Logs).
Please note that files only stay for around 14 days!

Name Link
Commit a1715ec
Build https://github.com/OasisLMF/ODS_OpenExposureData/actions/runs/19743568781
Excel File excel_spec.zip
JSON File extracted_spec.zip

@benhayes21
benhayes21 changed the base branch from main to release/v5.0.0 December 17, 2025 11:03
@benhayes21
benhayes21 merged commit 32bdfc0 into release/v5.0.0 Dec 18, 2025
8 checks passed
@benhayes21 benhayes21 mentioned this pull request Dec 18, 2025
Merged
sambles added a commit that referenced this pull request Feb 3, 2026
@sstruzik @sambles
add pyarrow dtype to oed json (#262)
b277aff 
* add pyarrow dtype to oed json

* add support backend dtype

* drop 3.8 support

* set custom ENV_ODS_SCHEMA_PATH

* fixup

* test ci

* test ci

* retest-ci

* test ci

* fixup

---------

Co-authored-by: Sam Gamble <hexadessa@gmail.com>
@awsbuild awsbuild added this to the 5.0.0 milestone Feb 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@sstruzik sstruzik

Labels

feature New feature or request

Projects

Oasis Dev Team Tasks
Status: Done

Milestone

5.0.0

Development

Successfully merging this pull request may close these issues.

4 participants

@sstruzik @benhayes21 @awsbuild @sambles