PZ Server Combine Redshift Catalogs

Combines multiple reference redshift catalogs into a single sample with homogenized data formats and a unique system of quality flags translated from the survey's original files.

Acknowledgements

Software developed and delivered as part of the in-kind contribution program BRA-LIN, from LIneA to the Rubin Observatory's LSST. An overview of this and other contributions is available here. The pipelines take advantage of the software support layer developed by LINCC, available as Python libraries: hats, hats-import and lsdb.

Tests

Test data

This repository currently contains a basic dataset, for testing purposes only. The ideal is to connect the pipelines to systems with access to a larger datasets.

Install

The only requirement is to have micromamba available in PATH:

git clone https://github.com/linea-it/pzserver_combine_redshift_dedup && cd pzserver_combine_redshift_dedup
./setup.sh
source env.sh

To install all pipelines at once:

./install.sh

The setup.sh will suggest a directory where the pipelines and datasets are installed, type 'yes' to confirm or 'no' to configure the desired path in each case with the respective environment variables and then run again setup.sh.

The installation script creates the pipe_crd environment with micromamba.

By default the scripts use MAMBA_ROOT_PREFIX="$HOME/.micromamba". On a Slurm cluster, point this variable to a persistent location visible to the jobs if needed:

export MAMBA_ROOT_PREFIX=/path/to/shared/or/persistent/micromamba

Run a pipeline

To execute, simply:

# execute combine redshift catalogs
./run.sh config.yaml process001

Validation notebook

To validate your test results, use the notebook validation.ipynb:

conda install -c conda-forge jupyterlab ipykernel
python -m ipykernel install --user --name=pipe_crd
jupyter lab