[REFACTOR] Add logging, os-->Path, streamlined loading

* [REFACTOR] Better readers to check existence of local file

* [REFACTOR] Streamlined readers with utilities.resolve_file_path()

* [REFACTOR] Add logging and fix tests, change local to data/

Author: eleanorfrajka

.gitignore

@@ -1,4 +1,6 @@
 amocarray/_version.py
+setup.sh
+ts_gridded.nc
 
 # VSCode settings
 .vscode/*

.pre-commit-config.yaml

@@ -36,7 +36,6 @@ repos:
         entry: pytest -q
         language: system
         types: [python]
-        files: ^(tests/.*\\.py$|amocarray/.*\\.py$|notebooks/.*\\.ipynb$)
         exclude: ^data/
 
   - repo: https://github.com/charliermarsh/ruff-pre-commit

README.md

@@ -1,8 +1,14 @@
 # MOC transports
 
-Amocarray is a python package for reading AMOC transport data from transport mooring arrays.  It does not modify, fix or grid data.  Functionality currently includes:
+**Clean, modular loading of AMOC observing array datasets, with optional structured logging and metadata enrichment.**
 
-- loading data from the RAPID 26°N array, OSNAP array, MOVE 16°N array and SAMBA 34.5°S array.
+> AMOCarray provides a unified system to access and process data from major Atlantic Meridional Overturning Circulation (AMOC) observing arrays:
+> - MOVE (16°N)
+> - RAPID (26°N)
+> - OSNAP (Subpolar North Atlantic)
+> - SAMBA (34.5°S)
+
+The project emphasizes clarity, reproducibility, and modular design, with per-dataset logging, metadata handling, and testable utilities.
 
 This is a work in progress, all contributions welcome!
 
@@ -21,6 +27,56 @@ Check out the demo notebook `notebooks/demo.ipynb` for example functionality.
 
 As input, amocarray downloads data from the observing arrays.
 
+### Quickstart
+
+#### Load a sample dataset
+```python
+from amocarray import readers
+
+# Load RAPID sample dataset
+ds = readers.load_sample_dataset("rapid")
+print(ds)
+```
+
+#### Load a full dataset
+
+```python
+from amocarray import readers
+
+datasets = readers.load_dataset("osnap")
+for ds in datasets:
+    print(ds)
+```
+A `*.log` file will be written to `logs/` by default.
+
+Data will be cached in `~/.amocarray_data/` unless you specify a custom location.
+
+### Project structure
+
+```
+amocarray/
+│
+├── readers.py               # Orchestrator for loading datasets
+├── read_move.py             # MOVE reader
+├── read_rapid.py            # RAPID reader
+├── read_osnap.py            # OSNAP reader
+├── read_samba.py            # SAMBA reader
+│
+├── utilities.py             # Shared utilities (downloads, parsing, etc.)
+├── logger.py                # Structured logging setup
+│
+└── tests/                   # Unit tests
+```
+
+### Roadmap
+
+- [] Add test coverage for utilities and readers
+- [] Add dataset summary output at end of load_dataset()
+- [] Optional global logging helpers (disable_logging(), enable_logging())
+- [] Extend load_sample_dataset() to support all arrays
+- [] Metadata enrichment (source paths, processing dates)
+
+
 ### Contributing
 
 All contributions are welcome!  See [contributing](CONTRIBUTING.md) for more details.
@@ -44,9 +100,21 @@ pytest --cov=amocarray --cov-report term-missing tests/
 
 Try to ensure that all the lines of your contribution are covered in the tests.
 
+
 ### Initial plans
 
 
 The **initial plan** for this repository is to simply load the volume transports as published by different AMOC observing arrays and replicate (update) the figure from Frajka-Williams et al. (2019) [10.3389/fmars.2019.00260](https://doi.org/10.3389/fmars.2019.00260).
 
 <img width="358" alt="image" src="https://github.com/user-attachments/assets/fb35a276-a41e-4cef-b78f-9c3c46710466" />
+
+
+
+## Acknowledgements
+
+- MOVE data: NOAA Climate Program Office and German Bundesministerium für Bildung und Forschung.
+- OSNAP data: www.o-snap.org
+- SAMBA data: NOAA AOML, Met Office, and associated research projects.
+- RAPID data: RAPID-MOCHA-WBTS collaboration.
+
+Dataset access and processing via [AMOCarray](https://github.com/AMOCcommunity/amocarray).

amocarray/logger.py

@@ -0,0 +1,96 @@
+# amocarray/logger.py
+import logging
+import datetime
+from pathlib import Path
+
+
+# Global logger instance (will be configured by setup_logger)
+log = logging.getLogger("amocarray")
+log.setLevel(logging.DEBUG)  # capture everything; handlers filter later
+
+# Global logging flag
+# Set to True to enable logging, False to disable
+LOGGING_ENABLED = True
+
+
+def enable_logging():
+    """Enable logging globally."""
+    global LOGGING_ENABLED
+    LOGGING_ENABLED = True
+
+
+def disable_logging():
+    """Disable logging globally."""
+    global LOGGING_ENABLED
+    LOGGING_ENABLED = False
+
+
+def log_info(message, *args):
+    """Log an info message, if logging is enabled."""
+    if LOGGING_ENABLED:
+        log.info(message, *args)
+
+
+def log_warning(message, *args):
+    """Log a warning message, if logging is enabled."""
+    if LOGGING_ENABLED:
+        log.warning(message, *args)
+
+
+def log_error(message, *args):
+    """Log an error message, if logging is enabled."""
+    if LOGGING_ENABLED:
+        log.error(message, *args)
+
+
+def log_debug(message, *args):
+    """Log a debug message, if logging is enabled."""
+    if LOGGING_ENABLED:
+        log.debug(message, *args)
+
+
+def setup_logger(array_name: str, output_dir: str = "logs") -> None:
+    """
+    Configure the global logger to output to a file for the given array.
+
+    Parameters
+    ----------
+    array_name : str
+        Name of the observing array (e.g., 'move', 'rapid', etc.).
+    output_dir : str
+        Directory to save log files.
+    """
+    if not LOGGING_ENABLED:
+        return
+    # Resolve output directory to project root
+    project_root = Path(__file__).resolve().parent.parent
+    output_path = project_root / output_dir
+    output_path.mkdir(parents=True, exist_ok=True)
+
+    timestamp = datetime.datetime.now().strftime("%Y%m%dT%H")
+    log_filename = f"{array_name.upper()}_{timestamp}_read.log"
+    log_path = output_path / log_filename
+
+    # Prevent duplicate handlers in case of multiple calls
+    if not any(
+        isinstance(h, logging.FileHandler) and h.baseFilename == log_path
+        for h in log.handlers
+    ):
+        file_handler = logging.FileHandler(log_path, encoding="utf-8", mode="w")
+        formatter = logging.Formatter(
+            fmt="%(asctime)s %(levelname)-8s %(funcName)s %(message)s",
+            datefmt="%Y%m%dT%H%M%S",
+        )
+        file_handler.setFormatter(formatter)
+
+        # Optional: console handler
+        console_handler = logging.StreamHandler()
+        console_handler.setFormatter(formatter)
+
+        # Clear existing handlers first
+        log.handlers.clear()
+
+        log.addHandler(file_handler)
+        log.addHandler(console_handler)
+
+        log.info(f"Logger initialized for array: {array_name}, writing to {log_path}")

amocarray/read_move.py

@@ -1,10 +1,13 @@
-import os
+from pathlib import Path
+from typing import Union
 
 import xarray as xr
 
-from amocarray import utilities
+from amocarray import utilities, logger
 from amocarray.utilities import apply_defaults
 
+log = logger.log  # ✅ use the global logger
+
 # Default source and file list
 MOVE_DEFAULT_SOURCE = "https://mooring.ucsd.edu/move/nc/"
 MOVE_DEFAULT_FILES = ["OS_MOVE_TRANSPORTS.nc"]
@@ -16,7 +19,7 @@
     "project": "Meridional Overturning Variability Experiment (MOVE)",
     "weblink": "https://mooring.ucsd.edu/move/",
     "comment": "Dataset accessed and processed via http://github.com/AMOCcommunity/amocarray",
-    # Acknowledgement and DOI can be added here when available
+    # DOI can be added here when available
     "acknowledgement": "The MOVE project is made possible with funding from the NOAA Climate Program Office. Initial funding came from the German Bundesministerium fuer Bildung und Forschung.",
 }
 
@@ -34,8 +37,8 @@ def read_move(
     source: str,
     file_list: str | list[str],
     transport_only: bool = True,
-    data_dir=None,
-    redownload=False,
+    data_dir: Union[str, Path, None] = None,
+    redownload: bool = False,
 ) -> list[xr.Dataset]:
     """
     Load the MOVE transport dataset from a URL or local file path into xarray Datasets.
@@ -48,6 +51,12 @@ def read_move(
     file_list : str or list of str, optional
         Filename or list of filenames to process.
         Defaults to MOVE_DEFAULT_FILES.
+    transport_only : bool, optional
+        If True, restrict to transport files only.
+    data_dir : str, Path or None, optional
+        Optional local data directory.
+    redownload : bool, optional
+        If True, force redownload of the data.
 
     Returns
     -------
@@ -61,45 +70,51 @@ def read_move(
     FileNotFoundError
         If the file cannot be downloaded or does not exist locally.
     """
-    source = utilities.get_local_file(source, data_dir, redownload)
+    log.info("Starting to read MOVE dataset")
+
     if transport_only:
         file_list = MOVE_TRANSPORT_FILES
     if isinstance(file_list, str):
         file_list = [file_list]
+
+    local_data_dir = Path(data_dir) if data_dir else utilities.get_default_data_dir()
+    local_data_dir.mkdir(parents=True, exist_ok=True)
+
     datasets = []
 
     for file in file_list:
         if not file.lower().endswith(".nc"):
+            log.warning("Skipping non-NetCDF file: %s", file)
             continue
 
-        # Validate source
-        if utilities._is_valid_url(source):
-            file_url = f"{source.rstrip('/')}/{file}"
-            dest_folder = os.path.join(os.path.expanduser("~"), ".amocarray_data")
-            try:
-                file_path = utilities.download_file(file_url, dest_folder)
-            except Exception as e:
-                raise FileNotFoundError(f"Failed to download {file_url}: {e}")
-        elif os.path.isdir(source):
-            file_path = os.path.join(source, file)
-            if not os.path.exists(file_path):
-                raise FileNotFoundError(f"Local file not found: {file_path}")
-        else:
-            raise ValueError("Source must be a valid URL or directory path.")
+        download_url = (
+            f"{source.rstrip('/')}/{file}" if utilities._is_valid_url(source) else None
+        )
+
+        file_path = utilities.resolve_file_path(
+            file_name=file,
+            source=source,
+            download_url=download_url,
+            local_data_dir=local_data_dir,
+            redownload=redownload,
+        )
 
         # Open dataset
         try:
+            log.info("Opening MOVE dataset: %s", file_path)
             ds = xr.open_dataset(file_path)
         except Exception as e:
+            log.error("Failed to open NetCDF file: %s: %s", file_path, e)
             raise FileNotFoundError(f"Failed to open NetCDF file: {file_path}: {e}")
 
         # Attach metadata
         file_metadata = MOVE_FILE_METADATA.get(file, {})
+        log.info("Attaching metadata to dataset from file: %s", file)
         utilities.safe_update_attrs(
             ds,
             {
                 "source_file": file,
-                "source_path": source,
+                "source_path": str(file_path),
                 **MOVE_METADATA,
                 **file_metadata,
             },
@@ -108,6 +123,8 @@ def read_move(
         datasets.append(ds)
 
     if not datasets:
+        log.error("No valid NetCDF files found in %s", file_list)
         raise FileNotFoundError(f"No valid NetCDF files found in {file_list}")
 
+    log.info("Successfully loaded %d MOVE dataset(s)", len(datasets))
     return datasets