Skip to content

Commit 9cc92a6

Browse files
Yifei-yang7Yifei-yang7
committed
Add docker image and update Readme.
1 parent 1a701f7 commit 9cc92a6

2 files changed

Lines changed: 55 additions & 30 deletions

File tree

‎docker/Dockerfile‎

Lines changed: 27 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -1,29 +1,32 @@
1-
FROM nvidia/cuda:12.5.1-cudnn-devel-ubuntu22.04
1+
FROM nvidia/cuda:12.6.1-devel-ubuntu22.04
2+
ENV DEBIAN_FRONTEND=noninteractive
3+
WORKDIR /root
24

3-
ENV TZ=Asia/Dubai
4-
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
5+
# Common dependencies
6+
RUN apt-get update && apt-get install -y git g++ cmake ninja-build libssl-dev wget vim unzip
57

6-
RUN apt update -y
7-
RUN apt-get update -y
8-
RUN apt install -y g++ make wget unzip vim htop git dstat
9-
RUN apt install -y software-properties-common
10-
RUN add-apt-repository -y ppa:ubuntu-toolchain-r/test
11-
RUN apt update
12-
RUN apt-get install -y build-essential
13-
RUN apt-get install -y libc-dev
14-
RUN update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-11 11
15-
RUN update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-11 11
8+
# CMake 3.30.4
9+
RUN wget https://github.com/Kitware/CMake/releases/download/v3.30.4/cmake-3.30.4-linux-x86_64.sh && \
10+
chmod +x cmake-3.30.4-linux-x86_64.sh && \
11+
./cmake-3.30.4-linux-x86_64.sh --skip-license --prefix=/usr/local && \
12+
rm cmake-3.30.4-linux-x86_64.sh
1613

17-
# install cmake
18-
RUN wget https://github.com/Kitware/CMake/releases/download/v3.28.0/cmake-3.28.0-linux-x86_64.sh
19-
RUN mkdir /opt/cmake
20-
RUN sh cmake-3.28.0-linux-x86_64.sh --skip-license --prefix=/opt/cmake/
21-
RUN ln -s /opt/cmake/bin/cmake /usr/local/bin/cmake
22-
RUN apt install -y ninja-build
14+
# Install Miniconda
15+
ENV CONDA_DIR=/opt/conda
16+
RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
17+
bash Miniconda3-latest-Linux-x86_64.sh -b -p $CONDA_DIR && \
18+
rm Miniconda3-latest-Linux-x86_64.sh
19+
ENV PATH=$CONDA_DIR/bin:$PATH
2320

24-
# Install nsight compute
25-
RUN apt-get install -y nsight-compute-2024.1.1
21+
# Create conda env with libcudf
22+
RUN conda update -n base -c defaults conda && \
23+
conda create -n libcudf-env -y && \
24+
conda run -n libcudf-env conda install -y -c rapidsai -c conda-forge -c nvidia rapidsai::libcudf
25+
ENV USE_CUDF=1
26+
ENV LIBCUDF_ENV_PREFIX=$CONDA_DIR/envs/libcudf-env
2627

27-
RUN mkdir -p /working_dir
28-
WORKDIR /working_dir
29-
RUN git config --global --add safe.directory "*"
28+
# Activates libcudf env at runtime beginning
29+
RUN echo 'echo "[INFO] Initializing conda environment... please wait (conda activation can take a few seconds)"' >> ~/.bashrc && \
30+
echo "source /opt/conda/etc/profile.d/conda.sh" >> ~/.bashrc && \
31+
echo "conda activate libcudf-env" >> ~/.bashrc && \
32+
echo 'echo "[INFO] Finished initializing conda environment"' >> ~/.bashrc

‎docs/README.md‎

Lines changed: 28 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,22 @@ Sirius is a GPU acceleration layer for SQL analytics. It plugs into existing eng
1717
- CUDA >= 11.2
1818
- CMake >= 3.30.4 (follow this [instruction](https://medium.com/@yulin_li/how-to-update-cmake-on-ubuntu-9602521deecb) to upgrade CMake)
1919

20-
## Installing dependencies
20+
## Dependencies (Option 1): Use Docker Image
21+
To use the docker image with dependencies fully installed:
22+
```
23+
sudo docker run --gpus all -it yifeiyang7/sirius_dependencies:latest bash
24+
```
25+
26+
Make sure both `nvidia-driver` and `nvidia-container-toolkit` are installed, `nvidia-driver` can be installed by
27+
```
28+
sudo apt install nvidia-driver-535
29+
```
30+
and `nvidia-container-toolkit` can be installed following the [instructions](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html). Then restart docker by
31+
```
32+
sudo systemctl restart docker
33+
```
34+
35+
## Dependencies (Option 2): Install Manually
2136

2237
### Install duckdb dependencies
2338
```
@@ -46,7 +61,8 @@ export USE_CUDF=1
4661
export LIBCUDF_ENV_PREFIX={PATH to libcudf-env}
4762
```
4863

49-
### Clone the Sirius repository
64+
## Building Sirius
65+
To clone the Sirius repository:
5066
```
5167
git clone --recurse-submodules https://github.com/sirius-db/sirius.git
5268
cd sirius
@@ -60,14 +76,20 @@ cd $SIRIUS_HOME_PATH
6076
```
6177
The `--recurse-submodules` will ensure DuckDB is pulled which is required to build the extension.
6278

63-
## Building Sirius
6479
To build Sirius:
6580
```
6681
make -j {nproc}
6782
```
6883

69-
## Generating TPC-H dataset
70-
Unzip `dbgen.zip` and run `./dbgen -s {SF}`.
84+
## Generating and Loading TPC-H dataset
85+
To generate the TPC-H dataset
86+
```
87+
unzip dbgen.zip
88+
cd dbgen
89+
./dbgen -s 1 && mkdir s1 && mv *.tbl s1 # this generates dataset of SF1
90+
cd ..
91+
```
92+
7193
To load the TPC-H dataset to duckdb:
7294
```
7395
./build/release/duckdb {DATABASE_NAME}.duckdb
@@ -79,7 +101,7 @@ To run Sirius, simply start the shell with `./build/release/duckdb {DATABASE_NAM
79101
From the duckdb shell, initialize the Sirius buffer manager with `call gpu_buffer_init`. This API accepts 2 parameters, the GPU caching region size and the GPU processing region size. The GPU caching region is a memory region where the raw data is stored in GPUs, whereas the GPU processing region is where intermediate results are stored in GPUs (hash tables, join results .etc).
80102
For example, to set the caching region as 1 GB and the processing region as 2 GB, we can run the following command:
81103
```
82-
call gpu_buffer_init("1 GB", "2 GB")
104+
call gpu_buffer_init("1 GB", "2 GB");
83105
```
84106

85107
After setting up Sirius, we can execute SQL queries using the `call gpu_processing`:

0 commit comments

Comments
 (0)