fitsio 2 planning & ideas

[beckermr]

This issue is a collection of ideas for fitsio 2. The goal is to discuss, document, and plan.

It is unclear if these things ever need to be done and many of them are possible quality of life and/or API preference issues as opposed to things that are blocking or need fixing.

We also need to restrict ourselves to changes in fitsio as opposed to cfitsio and the FITS standard. For example, I'd love to support lossless snappy compression of tables and zstd compression, but that is a FITS standard issue, not something we can address here.

Exceptions

• Add a proper exception hierarchy so users can be sure they are catching exceptions from fitsio
• Further format the exceptions from the C code to provide useful information (e.g., line numbers in the C code)

Move HDU interactions to ~exclusively use python properties?

• Change the various "get_XYZ" methods on HDU to properties
• Add lazy loading of headers, setting keys in headers, etc. via a BaseHDU.header property and remove the various write_key, read_key, etc. methods
• Add methods to read table columns, write table columns, etc. via __getitem__ (This might encourage poor performing code if people add columns one-by-one to a table.)

NOTE: We'd retain some of the read_XYZ and write_XYZ methods for the image/table data.

Backend Changes & Performance

• Allowing building fitsio against the rust-based cfitsio rewrite
• Deal with thread safety so that we can release the GIL in the C code
• Enable fitsio to work with freethreading python
• Lazily load the HDU list for the main FITS object
• Remove all fits_flush_file and fits_flush_buffer calls and instead mark the C fitsfile pointer as stale and do the call only as needed on the next C interaction
• Finally move to pyproject.toml?
• Update build system to meson or something to make things easier to maintain and more portable to windows?

Data Handling Changes

• Array columns with length 1 are now possible in numpy 2 in some cases. Do we want to finally fix/remove the casting of array columns of length 1 to scalars?
• Support lossy table compression?

[beckermr]

cc @esheldon @sbailey @dstndstn comments, ideas, and wishlist items are welcome!

[beckermr]

Also CC @petesmc wrt using the rust-based port of cfitsio.