Neurodesktop

• 1: Portable Unprivileged NeuroDesktop
• 2: Linux
• 3: MacOS
• 4: Windows
• 5: Cloud
• 6: Data Storage

1 - Portable Unprivileged NeuroDesktop

Minimum System Requirements

1. At least 5GB free space for neurodesktop base image
2. At least 8GB of RAM

Downloading TinyRange

TinyRange (https://github.com/tinyrange/tinyrange) is a lightweight runtime for running Virtual Machines and Containers. It runs without admin privilages and doesn’t need Docker or Podman installed to work.

• Windows: https://github.com/tinyrange/tinyrange/releases/download/v0.1.4/tinyrange-windows-amd64.zip
• MacOS: https://github.com/tinyrange/tinyrange/releases/download/v0.1.4/tinyrange-darwin-arm64.zip
• Linux (x86_64): https://github.com/tinyrange/tinyrange/releases/download/v0.1.4/tinyrange-linux-amd64.zip
• Linux (arm64): https://github.com/tinyrange/tinyrange/releases/download/v0.1.4/tinyrange-linux-arm64.zip

Downloading QEMU

• Windows: TinyRange already includes QEMU so you can skip this step.
• Ubuntu: sudo apt install qemu-kvm
• MacOS: brew install qemu

Installing TinyRange

• Unzip the archive to some local path on your computer. It’s recommended not to extract it to a network drive.
• Open a terminal in the extracted archive and run ./tinyrange login.
• Once you see the tinyrange:~# type exit.

Running NeuroDesktop

• Open a terminal in the TinyRange folder and run ./tinyrange login -c https://github.com/NeuroDesk/neurodesktop/raw/refs/heads/main/neurodesk.yml
• Neurodesktop will start up. Copy and paste the Jupyterhub link at the end of the output to a browser.
• Use Control+C in the terminal to exit.

Folder Sharing

Use --mount ~/neurodesktop-storage to share neurodesktop-storage.

For example:

./tinyrange login -c https://github.com/NeuroDesk/neurodesktop/raw/refs/heads/main/neurodesk.yml --mount ~/neurodesktop-storage

Changing CPU Cores, RAM, and/or Storage

• CPU Cores: Add --cpu 8 to set the VM to 8 CPU cores.
• RAM: Add --ram 8192 to set 8GB of RAM for the Virtual Machine.
• Storage: Add --storage 16384 to allocate 16GB of disk for the Virtual MAchine.

Enabling Hardware Acceleration

• Windows: This requires admin privileges. Search in your start menu for “Turn Windows features on or off.”. Find “Hyper-V Hypervisor” or “Windows Hypervisor Platform” and make sure it’s enabled. Then restart your computer.
• Ubuntu/Other Linux: It should already work out of the box. If not make sure your user account has permission to read/write /dev/kvm.
• MacOS: No extra steps required. It already works.

2 - Linux

Minimum System Requirements

1. At least 3GB free space for neurodesktop base image
2. Docker requirements. Details found under https://docs.docker.com/get-docker/

Quickstart

0. Install Docker or Podman

Install Docker from here: https://docs.docker.com/get-docker/. Additional information is available below. Alternatively, Neurodesk also works with Podman (https://podman.io).

To set up Neurodesk on Ubuntu, ensure both Podman client and server are installed. Follow the Podman installation instructions provided at https://podman.io/docs/installation for server setup.

sudo apt update && sudo apt upgrade -y
sudo apt install curl
echo "deb https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04/ /" | sudo tee /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list
curl -L "https://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_20.04/Release.key" | sudo apt-key add -
sudo apt update
sudo apt-get -y install podman

1. Optional: only for ARM64 hardware

Neurodesk supports ARM64 hardware through binfmt

To enable Neurodesk on ARM64 hardware run this setup step:

sudo docker run --privileged --rm tonistiigi/binfmt --install all

2. Run Neurodesktop

Before the first run, create a local folder where the downloaded applications will be stored, e.g. mkdir ~/neurodesktop-storage

Then use one of the following options to run Neurodesktop:

Option 1 (Recommended for local installations): Neurodesk-App

Instructions on installing and using the app: https://www.neurodesk.org/docs/getting-started/neurodesktop/neurodeskapp/

Option 2 (Advanced and for remote installations): Using Terminal

0. If the Linux machine is remote (e.g. in the cloud), connect to the machine with a port forwarding first:

ssh -L 8888:127.0.0.1:8888 USER@IP

1. then start neurodesktop:

docker volume create neurodesk-home &&
sudo docker run \
  --shm-size=1gb -it --privileged --user=root --name neurodesktop \
  -v ~/neurodesktop-storage:/neurodesktop-storage \
  --mount source=neurodesk-home,target=/home/jovyan \
  -e NB_UID="$(id -u)" -e NB_GID="$(id -g)" \
  -p 8888:8888 \
  -e NEURODESKTOP_VERSION=2024-10-22 vnmd/neurodesktop:2024-10-22

or for podman:

podman volume create neurodesk-home &&
sudo podman run \
  --shm-size=1gb -it --privileged --user=root --name neurodesktop \
  -v ~/neurodesktop-storage:/neurodesktop-storage \
  --mount type=volume,source=neurodesk-home,target=/home/jovyan \
  -e NB_UID="$(id -u)" -e NB_GID="$(id -g)" \
  -p 8888:8888 \
  -e NEURODESKTOP_VERSION=2024-10-22 docker://vnmd/neurodesktop:2024-10-22

chmod a+rwx ~/neurodesktop-storage

docker volume create neurodesk-home &&
sudo docker run \
  --shm-size=1gb -it --privileged --user=root --name neurodesktop \
  --dns 8.8.8.8 \
  -v ~/neurodesktop-storage:/neurodesktop-storage \
  --mount source=neurodesk-home,target=/home/jovyan \
  -e NB_UID="$(id -u)" -e NB_GID="$(id -g)" \
  -p 8888:8888 \
  -e NEURODESKTOP_VERSION=2024-10-22 vnmd/neurodesktop:2024-10-22

2. Once neurodesktop is downloaded, leave the terminal open and check which server neurodesktop is running on (Avoid pressing CTRL+C).

3. To access neurodesktop, open your web browser and type in one of the provided URLs in your terminal (e.g. http://127.0.0.1:8888/lab?token=your_unique_token).

"http://127.0.0.1:8888 wants to
See text and images copied to the clipboard".

4. Press on “Desktop Auto-Resolution” under “ALL CONNECTIONS”

5. If it is the first time you have used Neurodesktop, wait until the desktop appears (it may take a few seconds). Otherwise, it should appear instantaneously.

6. Neurodesk is now ready to use! See the tutorials page for advice on how to use Neurodesk.

7. For an optimal experience, switch your browser to full-screen mode by following the instructions for your browser here: https://www.thewindowsclub.com/open-chrome-edge-or-firefox-browser-in-full-screen-mode

Deleting neurodesktop:

When done processing your data it is important to stop and remove the container - otherwise the next start or container update will give an error ("… The container name “/neurodesktop” is already in use…")

1. Click on the terminal from which you ran neurodesktop

2. Press Ctrl-C

3. Run:

docker stop neurodesktop

4. Run:

docker rm neurodesktop

Installing Docker

For general installation instructions, refer to https://docs.docker.com/get-docker/

RHEL/CentOS (yum-based)

Refer to https://docs.docker.com/engine/install/centos/

One example to install docker in a yum-based distribution could look like this:

sudo dnf install -y yum-utils 
sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf install docker-ce docker-ce-cli containerd.io
# or if dnf not found: sudo yum install docker-ce docker-ce-cli containerd.io
sudo systemctl enable docker
sudo systemctl start docker
sudo docker version
sudo docker info
sudo groupadd docker
sudo usermod -aG docker $USER
sudo chown root:docker /var/run/docker.sock
newgrp docker

Ubuntu/Debian (apt-based)

Refer to https://docs.docker.com/engine/install/ubuntu/

One example to install docker in a apt-based distribution could look like this:

sudo apt-get update
sudo apt-get install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update

GPU support

RHEL/CentOS (yum-based)

sudo yum install nvidia-container-toolkit -y

Ubuntu/Debian (apt-based)

sudo apt install nvidia-container-toolkit -y

Running neurodesktop container with GPU

sudo docker run \
  --shm-size=1gb -it --privileged --user=root --name neurodesktop \
  -v ~/neurodesktop-storage:/neurodesktop-storage \
  -e NB_UID="$(id -u)" -e NB_GID="$(id -g)" \
  --gpus all \
  -p 8888:8888 -e NEURODESKTOP_VERSION=2024-10-22 \
  vnmd/neurodesktop:2024-10-22

Running tensorflow (w/ GPU)

Using tensorflow (python)

mamba install tensorflow-gpu
python

import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))

Using tensorflow (singularity container in neurodesktop)

singularity pull docker://tensorflow/tensorflow:latest-gpu
singularity run --nv tensorflow_latest-gpu.sif
python

import tensorflow as tf
print("Num GPUs Available: ", len(tf.config.list_physical_devices('GPU')))

Connecting to a running Neurodesktop session via a plain shell

You can start a neurodesktop container using docker or the neurodeskapp. If you want to connect to this running session using a plain shell you can do this as well:

docker ps
# note the name of the running container, e.g. neurodeskapp-49977

# now connect to this container
docker exec -ti neurodeskapp-49977 bash

# then switch to the jovyan user
su jovyan

3 - MacOS

Quickstart

1. Install Docker or Podman

Install Docker from here: https://docs.docker.com/get-docker/ Alternatively, Neurodesk also works with Podman, follow the Podman installation instructions provided at https://podman.io/docs/installation.

Docker currently has a bug on ARM Apple Silicon Hardware (M1/M2/M3/M4) - the workaround is to install Podman: https://podman.io/docs/installation

and then select Podman in the Options of the Neurodesk App:

2. Run Neurodesktop

Use one of the following options to run Neurodesktop:

Option 1 (Recommended): Neurodesk-App

Instructions on installing and using the app: https://www.neurodesk.org/docs/getting-started/neurodesktop/neurodeskapp/

Option 2 (Advanced): Using Terminal

Create a local folder where the downloaded applications will be stored, e.g. ~/neurodesktop-storage

1. Open a terminal, and type the following command to automatically download the neurodesktop container and run it

docker volume create neurodesk-home &&
docker run \
--shm-size=1gb -it --privileged --user=root --name neurodesktop \
-v ~/neurodesktop-storage:/neurodesktop-storage \
--mount source=neurodesk-home,target=/home/jovyan \
-e NB_UID="$(id -u)" -e NB_GID="$(id -g)" \
-p 8888:8888 -e NEURODESKTOP_VERSION=2024-10-22 vnmd/neurodesktop:2024-10-22

If you get errors in neurodesktop then check if the ~/neurodesktop-storage directory is writable for all users. If it is not, run chmod a+rwx ~/neurodesktop-storage

2. Once neurodesktop is downloaded, leave the terminal open and check which server neurodesktop running on (Avoid pressing CTRL+C).

3. To access neurodesktop, open your web browser and navigate to one of the URLs shown in your terminal (e.g. http://127.0.0.1:8888/lab?token=your_unique_token).

3. If prompted, press on “Desktop Auto-Resolution” under “ALL CONNECTIONS”

4. If it is the first time you use Neurodesktop, wait until the desktop appears (it may take a few seconds). Otherwise, it should appear instantaneously.

5. Neurodesk is ready to use! See the tutorials page for advice on how to use Neurodesk.

Deleting neurodesktop:

When done processing your data it is important to stop and remove the container - otherwise the next start or container update will give an error ("… The container name “/neurodesktop” is already in use…")

1. Click on the terminal from which you ran neurodesktop

2. Press control-C

3. Type:

docker stop neurodesktop

4. Type:

docker rm neurodesktop

4 - Windows

Minimum System Requirements

1. At least 3GB free space for neurodesktop base image
2. Docker requirements. Details found under https://docs.docker.com/get-docker/
3. If installing docker using WSL, minimum 20GB space recommended for WSL with Ubuntu

Quickstart

1. Install Docker or Podman

Install Docker from here: https://docs.docker.com/get-docker/

Alternatively, Neurodesk also works with Podman, follow the Podman installation instructions provided at https://podman.io/docs/installation.

2. Run Neurodesktop

Use one of the following options to run Neurodesktop:

Option 1 (Recommended): Neurodesk-App

Instructions for installing and using the app: https://www.neurodesk.org/docs/getting-started/neurodesktop/neurodeskapp/

Option 2 (Advanced): Using Terminal

1. Open a terminal (e.g. Powershell), and type the following command to automatically download the neurodesktop container and run it

docker volume create neurodesk-home
# This creates a docker volume to store your /home/jovyan data inside a docker volume

docker run --shm-size=1gb -it --privileged --user=root --name neurodesktop -v C:/neurodesktop-storage:/neurodesktop-storage --mount source=neurodesk-home,target=/home/jovyan -p 8888:8888 -e NEURODESKTOP_VERSION=2024-10-22 vnmd/neurodesktop:2024-10-22

2. Once neurodesktop is downloaded, leave the terminal open and check which server neurodesktop running on (Avoid pressing CTRL+C). ]

3. To access neurodesktop, open your web browser and type in one of the URLs provided in your terminal (e.g. http://127.0.0.1:8888/lab?token=your_unique_token).

4. Press on “Desktop Auto-Resolution” under “ALL CONNECTIONS”

5. If it is the first time you use Neurodesktop, wait until the desktop appears (it may take a few seconds). Otherwise, it should appear instantaneously.

6. Neurodesk is ready to use! See the tutorials page for advice on how to use Neurodesk.

7. For an optimal experience, switch your browser to full-screen mode by following the instructions for your browser here: https://www.thewindowsclub.com/open-chrome-edge-or-firefox-browser-in-full-screen-mode

Deleting neurodesktop:

When done processing your data it is important to stop and remove the container - otherwise the next start or container update will give an error ("… The container name “/neurodesktop” is already in use…")

1. Click on the terminal from which you ran neurodesktop

2. Press control-C

3. Type:

docker stop neurodesktop

4. Type:

docker rm neurodesktop

5 - Cloud

Options for Running Neurodesk on cloud computing resources

There are a couple of ways how Neurodesktop can be run on cloud computing resources:

1. The first and easiest option is to provision a virtual Linux machine and run docker on this machine similar to a local Linux setup: https://www.neurodesk.org/docs/getting-started/neurodesktop/linux/
2. Another option is to use Windows VMs and some groups have had success with that. Here is a detailed instructions on this setup: https://github.com/NeuroDesk/neurodesk.github.io/blob/main/static/docs/getting-started/neurodesktop/Neurodesk_Windows_Technion.pdf
3. The third and most scalable solution is to run Neurodesk via Kubernetes. This setup is a bit more complex, but can handle many simultaneous users and is ideal for research groups and workshops. The easiest way to deploy Neurodesk on Kubernetes is to use Zero to Jupyterhub (https://z2jh.jupyter.org/en/stable/) - then you can use the Neurodesk image like any other jupyterhub image. If you do not want to run a privileged container you need to deploy the cvmfs setup on Kubernetes as well: https://github.com/cvmfs-contrib/cvmfs-csi/

6 - Data Storage

Drag and Drop

Uploading files

You can drag-and-drop files into the browser window to get files into Neurodesktop. This will then start a file upload:

Downloading files

To download files from the desktop using the same mechanism you will need to open the guacamole settings by pressing CTRL-ALT-SHIFT (Control-Command-Shift on Mac). This will open a menu on the side:

where you can click on “Shared Drive”:

a click (or double click on Mac) on the file will start the download.

You can browse into folders in the shared drive by clicking (double clicking on Mac) on them. To get back to the base of the shared drive, press on the drive icon in the top left of the side menu (just below the “Shared Drive” title).

To close the side menu, click on CTRL-ALT-SHIFT once more (Control-Command-Shift on Mac).

Note that it is only possible to upload or download one file at a time through this interface. If you have multiple files in a directory we recommend zipping the directory and then transferring one zip archive:

zip files.zip files/

Local storage

If you are running Neurodesktop on your own hardware there will be a direct connection between the “Storage” folder on the Desktop, which is a link between “/neurodesktop-storage” in neurodesktop and the “neurodesktop-storage” folder on your C-drive (Windows) or home directory (Mac/Linux). This connection can be used for data processing and data transfer.

Mounting external storage on your host-computer

The -v C:/neurodesktop-storage:/neurodesktop-storage part of the docker command links the directory “neurodesktop-storage” on the “C drive” of your Windows computer to the directory /neurodesktop-storage inside the Desktop environment. Everything you store in there will be available inside the desktop and on the host computer. You can also mount additional directories by adding another -v parameter set (e.g. -v D:/moredata:/data) - this will mount the directory moredata from your D drive to /data inside neurodesktop. Important: the mountpoint inside neurodesktop needs to be named /data, otherwise the applications will not see the files without modifying the SINGULARITY_BINDPATH variable in your .bashrc.

If you are using the NeurodeskApp, you can set an additional storage location through the settings

If you are starting Neurodesk through the command line, here is an example for Windows adding another storage directory:

docker run --shm-size=1gb -it --privileged --user=root --name neurodesktop -v C:/neurodesktop-storage:/neurodesktop-storage -v D:/moredata:/data -p 8888:8888 -e NEURODESKTOP_VERSION=2024-10-22 vnmd/neurodesktop:2024-10-22

Note for Windows users: Connecting network shares from Windows to Neurodesk can cause problems, so be careful when attempting this. Also, be aware that processing large amounts of files stored on a Windows filesystem inside Neurodesk will come with a performance penality due to the file system translation in the background. One option to get around these problems is to directly accessing your storage infrastructure inside Neurodesk.

Cloud storage

Another way to get your data into Neurodesktop is to use a cloud storage provider like Dropbox, OneDrive, OwnCloud, Nextcloud or more general tools like rclone or davfs2. Another good option could be to utilize Globus for large amounts of data.

Nextcloud and Owncloud desktop clients

Under the menu item “Accessories” you can find “Nextcloud” and “ownCloud” desktop sync clients that you can configure with your cloud service accounts.

Mounting webdav storage using davfs2

Another option is to directly mount webdav storage. Here is an example how to mount OwnCloud Storage into Neurodesktop:

sudo mount -t davfs https://yourOwnCloudInstance.com/plus/remote.php/webdav/ /data/

It then asks you for a username and password, which you can generate in the settings: yourOwnCloudInstance/plus/settings/personal?sectionid=security

Rclone

Rclone is a command line tool that enables interaction with various cloud services. Here is an example how to setup rclone with an OwnCloud account:

• start the configuration in a terminal window rclone config
• Create a new remote: n
• Provide a name for the remote: OwnCloud
• For the “Storage” option choose: webdav
• As “url” set: https://yourOwnCloudInstance.com/plus/remote.php/webdav/
• As “vendor” set OwnCloud: 2
• Set your OwnCloud username after generating an access token yourOwnCloudInstance/plus/settings/personal?sectionid=security
• Choose to type in your own password: y
• Enter the Password / Token from the OwnCloud App passwords page and confirm it again:
• Leave blank the bearer_token: <hit Enter>
• No advanced config necessary: <hit Enter>
• accept the configuration: <hit Enter>
• Quit the config: q
• Now we can download data to the HPC easily: rclone copy --progress --transfers 8 OwnCloud:/raw-data-for-science-paper .
• or upload data to OwnCloud: rclone copy --progress --transfers 8 . OwnCloud:/data-processed

Globus

We also provide the globus client, so you can transfer large amounts of data between globus endpoints and Neurodesktop. You can configure it by running the following commands in the Neurodesktop environment:

ml globus
# First run the setup:
globusconnectpersonal -setup

#Follow the instructions in the terminal: 
#1) copy the URL into a browser and generate the Native App Authorization Code
#2) then copy this code and paste it in the terminal
#3) then name the endpoint, e.g. Neurodesktop

# Then start the GUI:
globusconnectpersonal -gui

# If the connection fails, reset the permissions on the key file:
chmod 600 /home/jovyan/.globusonline/lta/relay-anonymous-key.pem

# If the connection still fails, start the client like this to get more information
globusconnectpersonal -debug

Then add the directories you want to share with globus, by opening File -> Preferences:

and then add the paths required and hit Save:

Then you can go to the globus file-manager https://app.globus.org/file-manager and your neurodesktop instance will be an endpoint for globus. You can change the path to any location you specified in the Preferences:

Mount volume using SSHFS

It is theoretically possible to mount an SSH target inside Neurodesktop, but it’s not a very reliable way of mounting storage:

sshfs -o allow_root USER@TARGET_HOST:TARGET_PATH SOURCE_PATH

A better option is to use scp and copy data from an SSH endpoint:

scp /neurodesk/myfile user@remoteserver:/data/

An alternative is to mount the SSHFS target into a parent directory on your local machine or VM and then use the -v option in the docker run command to bind the parent directory of the SSHFS mount. NOTE: the SSHFS has to be mounted to a subdirectory inside a parent directory which is then bound to the docker container. If you directly bind to the mounted directory itself, your Neurodesktop container will stop being able to access it if the SSHFS mount disconnects and will not be able to access it again without restarting the Neurodesktop container.

For example, on a local Linux machine or VM:

sshfs -o allow_root USER@TARGET_HOST:TARGET_PATH/MyData SOURCE_PATH/SSHFS_Mounts/MyData

Then add the following line to the docker run command when starting Neurodesktop (note the rshared flag):

-v /SSHFS_Mounts:/data:rshared \

TIP: If you use key pair authentication instead of password for your SSHFS mount, you can use the reconnect flag to reconnect automatically if the connection drops:

sshfs -o IdentityFile=~/.ssh/id_rsa,allow_root,ServerAliveInterval=5,ServerAliveCountMax=3 USER@TARGET_HOST:TARGET_PATH/MyData SOURCE_PATH/SSHFS_Mounts/MyData