Heliopropa Web

A website writen in flask for the visualisation of solarwind propagation simulations.

Builds and serves heliopropa's visualizations.

It's a flask webserver, serving d3.js plots.

It also gathers NetCDF data from AMDA, and serves it as CSV to the plotter.

Quick Start

You can very quickly have a running app out of the box.

Just:

docker compose build
docker compose up -d

and point your browser to http://127.0.0.1:8080/

See later about configuring port number

Docker

Preferred way of running heliopropa app.

Configuration

docker stack mainly consists in two files:

• Dockerfile : for the container building
• compose.yaml : for the volumes configuration and links with nginx service and gunicorn

And your can tweak port configuration thanks to $PORT environment variable. (@see ports: section in compose.yaml )

Either by setting it in your shell

PORT=8081 docker compose up -d

Or in a config file:

cp compose.env-dist compose.env
${EDITOR} compose.env # set to your needs
docker compose --env-file=compose.env build

Or

cp compose.env-dist .env
${EDITOR} .env # set to your needs
docker compose build

Volumes and compose.override.yaml

You may want to understand where data is stored:

The web application saves *.csv files in the ./cache/ directory, which happens to be a docker volume named helio_cache.

The speasy library holds a cache dir located at container:/root/.cache/speasy/Cache/ which is also mounted as a volume: speasy_cache.

Two configuration files are also used:

• ./config.yml : for the web app.
• ${HOME}/.config/speasy/config.ini : for the speasy library.

You can use the compose.override.yaml file to make dynamic links between your local files or directories and the container.

That way you can live edit the config files, and also use already loaded cache directories.

To do so:

cp compose.override.yaml-dist compose.override.yaml
${EDITOR} compose.override.yaml # make sure its ok, comment what you dont need
docker compose build

Developer

Files Overview

• config.yml : the main configuration file.
• web/run.py : the front controller, holding most of the code.
• web/run.log : you'll find more information about failures in the log.
• web/view/home.html.jinja2 : the HTML template.
• web/static/js/main.js : most of the javascript client-side.

Configuration

cp config.yml-dist config.yml
$(EDITOR) config.yml # but defaults should be ok

Also take a look at the ./ressources/ directory for more available configuration files.

Python venv

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

Run locally

source venv/bin/activate
DEBUG=true python web/run.py

Then, browse localhost:5000.

Admin task

A tool called hp_cmd.py is available to make so administrative task.

It allows you to nightly populate the csv cache so later visitors won't have to wait for data download.

Basic usage:

# look at what it would do
python web/hp_cmd.py --dry-run -clog csv_gen -a

# do it
python web/hp_cmd.py -clog csv_gen -a

# see all usages
python web/hp_cmd.py --help

Docker usage:

docker compose exec web python web/hp_cmd.py --all

Http tasks:

http://${HOSTNAME}/cache/clear
http://${HOSTNAME}/cache/cleanup
http://${HOSTNAME}/cache/warmup
http://${HOSTNAME}/log
http://${HOSTNAME}/log/clear

Notebooks

If your want to dig into code usage take a look at our notebooks:

pip install -r requirements-nb.txt
jupyter notebook ./notebooks/

then browse to given url.

Each file name is self-explanatory, and each notebook contains explanations.

Gunicorn and Nginx deployment

see [ Digital Ocean documentation ] (https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-20-04-fr)

wep app service and Gunicorn

see ./resources/heliopropa.service

/etc/systemd/system/heliopropa.service

[Unit]
Description=Gunicorn instance to serve Heliopropa
After=network.target

[Service]
User=debian
Group=www-data
WorkingDirectory= /home/debian/heliopropa/
Environment="PATH=/home/debian/heliopropa/venv/bin/"
ExecStart=/home/debian/heliopropa/venv/bin/gunicorn --workers 3 --bind unix:heliopropa.sock -m 007 web.run:app

[Install]
WantedBy=multi-user.target

Nginx

see ./resources/heliopropa.nginx

/etc/nginx/sites-available/heliopropa

server {
	listen 80;
	server_name heliopropa heliopropa.co-libri.org;

	location / {
		include proxy_params;
		proxy_pass http://unix:/home/debian/heliopropa/heliopropa.sock;
	}
}

Start and Allow at system reboot

sudo systemctl start nginx
sudo systemctl start heliopropa 
sudo systemctl enable heliopropa 

Git hook auto deploy

see ./resources/post-receive.git-hook

Set bare repo with hook on vps

ssh user@vps
git clone --bare path-to-heliopropa-repo heliopropa.git
cd heliopropa.git/hooks
vim post-receive
chmod +x post-receive

Push from dev computer

git remote add helio-dev user@vps:heliopropa.git
git push heliopropa HEAD:DEV