Yoda manuals

_quarto.yml

@@ -11,6 +11,17 @@ website:
   repo-actions: [edit, issue]
   page-navigation: true
   bread-crumbs: true
+  sidebar:
+    - id: sb-yoda # will show up automatically on pages referenced in contents. 
+      style: floating
+      search: true
+      contents: 
+        - manuals/yoda/index.qmd
+        - section: Yoda Portal
+          contents: manuals/yoda/yoda_portal/*.qmd # Note the sorting is by filename
+        - section: Data Transfer
+          contents: manuals/yoda/yoda_data_transfer/*.qmd
+
   navbar:
     background: "#0080c9"
     logo: public/VU-logo-Dia-positief.png
@@ -37,6 +48,8 @@ website:
             href: /topics.html#category=Trainings%20and%20Knowledge%20Items
       - text: Guides
         file: guides.qmd
+      - text: Manuals
+        file: manuals.qmd
       - text: Blog
         file: blog.qmd
       - contributing.qmd
@@ -77,4 +90,4 @@ format:
 filters:
   - utils/include-files.lua
 execute:
-  freeze: auto
+  freeze: auto

manuals.qmd

@@ -0,0 +1,21 @@
+---
+title: "Manuals"
+listing:
+  contents: 
+    - "manuals/yoda/index.qmd"
+    - "manuals/ada/index.qmd"
+    - "manuals/scicloud/index.qmd"
+  filter-ui: [title]
+  type: grid
+  categories: false
+---
+
+::: {.callout-tip}
+
+## What is a manual?
+
+A manual shows you how to accomplish a specific task in a tool. They are organized per tool. These pages are a quick way to get started with a tool.
+
+Missing a manual? You can [submit suggestion using the Contribution portal](https://ez-github-contributor.netlify.app/).
+:::
+

manuals/yoda/index.qmd

@@ -0,0 +1,20 @@
+---
+title: Yoda Manuals
+categories: []
+listing:
+  contents: 
+    - ./*.qmd
+    - yoda_portal/*.qmd
+    - yoda_data_transfer/*.qmd
+  page-size: 40
+  filter-ui: true
+  type: grid
+  fields: [title,description]
+  sort: false
+image: '/public/yoda-logo.png'
+description: 'This section explains how to get started with Yoda.'
+---
+
+::: {.callout-tip}
+The [Yoda Topic page](/topics/yoda.qmd#how-to-request-access) explains how to request storage space in Yoda for your project.
+:::

manuals/yoda/yoda_data_transfer/index.qmd

@@ -0,0 +1,29 @@
+---
+title: Data Transfer overview
+categories: []
+---
+
+This page provides an overview of the 3 main ways to transfder data to and from Yoda:
+
+1. Via the Yoda Portal, upload via Drag & Drop, download inviduals files.
+
+This method works fine if you need to upload small amounts of data. 
+
+2. For transferring larger amounts of data (tens of GB) it is better to use the WebDAV interface.
+
+There are several methods to access Yoda data via WebDAV, they depend on the Operating System you are using:
+
+* [Accessing Yoda data on Windows](./yoda_data_windows.qmd)
+* [MacOS](./yoda_data_macos.qmd)
+* [Linux](./yoda_data_linux.qmd)
+
+Note that performance of WebDAV is limited, especially when transferring a large amount of files. 
+
+3. For transferring large amounts of date (>100GB) consider using direct access to iRODS. 
+
+A direct iRODS connection offers better performance by enabling multi-threaded transfers and the option to compare checksums.
+
+Power users who are comfortable with command line tools can use the [iRODS icommands](yoda_using_icommands.qmd) to
+interact with the iRODS backend of Yoda directly.
+
+_Last updated: {{< meta date >}}_

manuals/yoda/yoda_data_transfer/yoda_dap.qmd

@@ -0,0 +1,21 @@
+---
+title: Creating a Data Access Password
+categories: []
+---
+
+This page explains how to set a Data Access Password to access data via the Yoda WebDAV and iRODS interfaces. 
+
+- Click on your username in the top right and click Data Access Password or go to [https://portal.yoda.vu.nl/user/data_access](https://portal.yoda.vu.nl/user/data_access):
+
+![Data Access Password menu](/public/manuals/yoda/dap-menu.png)
+
+![Generate Data Access Password](/public/manuals/yoda/dap-generate.png)
+- Enter a suitable label and click on "Generate new data access password"
+
+![](/public/manuals/yoda/dap-generate2.png)
+- The password is shown. Click Copy to copy it to the clipboard. After this you can use CTRL-v to paste it.
+
+- Data access passwords expire in 90 days, you can view the expiration time in the list.
+![](/public/manuals/yoda/dap-list.png)
+
+- If a password has expired, simply create a new one. Don't forget to update the password in the WebDAV client.

manuals/yoda/yoda_data_transfer/yoda_data_linux.qmd

@@ -0,0 +1,98 @@
+---
+title: Data access on Linux
+categories: []
+---
+
+This page explains how to access your Yoda data via the WebDAV interface on Linux.
+
+# Connecting to the Yoda WebDAV interface using GNOME files
+
+GNOME users can connect to the Yoda WebDAV interface with GNOME Files (also known as Nautilus in older versions of GNOME).
+
+![GNOME Files screenshot](/public/manuals/yoda/linux-connect-to-server.png)
+
+Click on "+Other Locations" and enter the server address in the "Connect to Server" bar. The server address is
+`davs://data.yoda.vu.nl/`
+
+Then click on the Connect button.
+
+![GNOME Files password dialog screenshot](/public/manuals/yoda/linux-enter-password.png)
+
+You will be prompted for a name and password.
+Username is your email. Set a [Data Access Password](./yoda_dap.qmd) and copy it to the Password field.
+
+# Connecting to the Yoda WebDAV interface using davfs2
+
+This page describes how to connect to the Yoda WebDAV interface on a Linux system using davfs2. It assumes familiarity
+with using commandline tools. The instructions are suitable for RedHat-based distributions (e.g. Fedora or CentOS) and for
+Debian-based distributions (e.g. Ubuntu).
+
+## Installing the package
+
+Install the davfs2 package.
+
+For Debian/Ubuntu:
+```bash
+sudo apt -y install davfs2
+```
+- Confirm that unprivileged users should be able to mount davfs2 volumes (“Yes”).
+![GNOME Files password dialog screenshot](/public/manuals/yoda/linux-davfs2-dialog-unprivileged.png)
+
+For RedHat/CentOS/Fedora:
+```bash
+sudo yum -y install epel-release
+sudo yum -y install davfs2
+```
+
+## Configuring group membership
+
+Look up your user name, uid and gid using the `id` command.
+
+Add your user account to the davfs2 group: `sudo usermod -aG davfs2 user` (replace “user” with your user name).
+
+Close the terminal window and open a new one to activate the group change.
+
+## Configuring Davfs2
+
+Open the davfs2 configfile in a text editor (e.g. `sudo vi /etc/davfs2/davfs2.conf`). Ensure
+parameter *delay_upload* is set to 0 (zero). This limits the risk of data loss from a failure to flush data after
+large file transfers.
+
+Open the /etc/fstab file in a text editor (e.g. `sudo vi /etc/fstab`) and add a configuration
+line for Yoda:
+
+`https://data.yoda.vu.nl  /mnt davfs user,auto,uid=1000,gid=1000 0 0`
+
+And adjust the parameters as needed:
+- If you'd like to mount the Yoda in a different location, replace `/mnt` with a different local directory.
+- Replace the `uid` and `gid` parameters with your uid and gid, as shown by the `id` command.
+- If you don't want Yoda to be mounted automatically after your system starts, remove "auto," from the options.
+
+First set a [Data Access Password](./yoda_dap.qmd).
+
+Now use a text editor to create a secrets file, which contains your Yoda WebDAV URL, Yoda user name and password, separated by spaces.
+If you are an employee or student at Vrije Universiteit, your user name is your VU email address (in lowercase) and your password
+is your [Data Access Password](./yoda_dap.qmd). External users have usually received their user name via email, along with a link to set their password. 
+Example of a secrets file: 
+```
+https://data.yoda.vu.nl xxx@vu.nl  myDataAccessPassword
+```
+You need to escape any backslashes and double quotes in your password with a backslash (e.g. use `\\` instead of `\`).
+
+Install this secrets file as the global davfs2 secrets file:
+```bash
+sudo install -m 0600 -o root -g root secrets /etc/davfs2/secrets
+rm secrets
+```
+
+## Mounting the Yoda WebDAV interface
+
+The disk should be mounted automatically after a reboot if you have configured the `auto` option in `/etc/fstab`.
+
+To manually mount the Yoda WebDAV interface:
+- On Debian/Ubuntu: `mount /mnt`
+- On RedHat/CentOS/Fedora: `sudo mount /mnt` (you can ignore any warnings about writing to the mtab file)
+
+## Acknowledgements
+
+Thanks to [Joost de Graaf](https://www.uu.nl/medewerkers/JdeGraaf) for his contributions to this guide.

manuals/yoda/yoda_data_transfer/yoda_data_macos.qmd

@@ -0,0 +1,42 @@
+---
+title: Data access on MacOS
+categories: []
+---
+This page explains how to access your Yoda data via the WebDAV interface on MacOS.
+
+There are 2 basic ways to access Yoda data from a Mac: [Mounting the Yoda WebDAV interface in finder](#mounting-the-yoda-network-disk-in-finder) or via file transfer tools, in which case we recommend [Cyberduck](./yoda_using_cyberduck.qmd). Which one works best depends on your workflow.
+
+## Using Cyberduck
+First fownload Cyberduck from the VU software center or from the [Cyberduck website](https://cyberduck.io/download/). 
+
+Then configure Cyberduck using the [Information on configuring Cyberduck](./yoda_using_cyberduck.qmd).
+
+## Mounting the Yoda WebDAV in Finder
+
+You can open the Yoda WebDAV interface using the Finder app. By default, the Finder icon is shown in the bottom left corner of your screen, in the Dock. Click on this icon. 
+
+![Finder icon](/public/manuals/yoda/macos-finder.jpg)
+
+If you don't see the Finder icon, you can alternatively start Finder by pressing the command key and the space bar, then typing "finder" and pressing enter.
+
+Now press the command key and the "K" key to connect to the Yoda server. You should now see this dialog:
+
+![Connect to server dialog](/public/manuals/yoda/macos-connect-server.png)
+
+Enter https://data.yoda.vu.nl/ as server address.
+
+The Finder app will show a confirmation dialog, similar to the one below. The address shown in the dialog should be the address you entered in the previous dialog.
+
+![Connect to server confirmation dialog](/public/manuals/yoda/macos-connect-continue.jpg)
+
+Press the Connect button. 
+
+![Connect to server confirmation dialog](/public/manuals/yoda/macos-connect-credentials.jpg)
+
+You should now see a credentials dialog. The "Connect as" setting should be set to "Registered user".
+"Name" should be your email address. Create a [Data Access Password](./yoda_dap.qmd) and copy it to the Password field. Tick the checkbox “Remember this password
+in my keychain”.  Click on the connect button.
+
+You should now have a new Yoda location in Finder. Its name is the network address you entered before. You may have to scroll down in finder in order to see it.
+
+![Connect to server credentials dialog](/public/manuals/yoda/macos-finder-screen.png)

manuals/yoda/yoda_data_transfer/yoda_data_windows.qmd

@@ -0,0 +1,34 @@
+---
+title: Data access on Windows
+categories: []
+---
+
+This page explains how to access your Yoda data via the WebDAV interface on Windows machines.
+
+There are 2 basic ways to access the Yoda WebDAV interfacek from Windows. 
+Using a tool to mount the Yoda Disk as a drive letter or file transfer tools. Which one works best
+ depends on your workflow.
+
+### File transfer
+There are numerous free file transfer tools. Using these tools you must download the files you want to work on to 
+your computer and upload the changes. This way of working is more stable and robust. 
+
+- [Cyberduck](./yoda_using_cyberduck.qmd) is a free file transfer tool for Mac and Windows. Cyberduck is the preferred
+way to access the Yoda Disk. There is also a paid addon "Mountainduck" which adds functionality to access via a drive 
+letter.
+
+- [WinSCP](./yoda_using_winscp.qmd) is an alternative free file transfer tool.
+
+### Drive letter
+Using these tools you can access the Yoda Disk via a drive letter.
+
+- [WebDrive](./yoda_using_webdrive) is a VU-supported method to remotely access files and can also be used on "green"
+pc's.
+
+- Directly in [Windows Explorer](./yoda_using_windowsexplorer.qmd). However, this has restrictions: a maximum
+file size of 4GB on Windows 11 and 50MB(!) on Windows 10 and a maximum of 1000 files per folder (Windows 10&11). Take care if you expect your dataset to exceed these limitations.
+
+
+
+### Commandline
+If you are familiar with commandline tools [rclone](./yoda_using_rclone.qmd) is also a good option to access the Yoda via Webdav. It also provides the option to use a drive letter.