This is a summary of the setup to run Jupyter notebooks or JupyterLab on a remote server. Steps 1-4 describe the configuration of the setup and only have to be done once. Step 5 describes the usage of the setup.

Generally, if you want to log into a remote server (e.g., tikoraluk) using the ssh command, you would use the following command in your terminal:

ssh <user>@<server>

where <user> and <server> are your user name on and domain (e.g., tikoraluk.mathstat.dal.ca) of the remote machine.

Typing in your user name and the full domain of your server can be cumbersome, but you can set up a configuration file that allows you to log in using an alias.

On your local computer, open a terminal, and update the file config in the folder .ssh in your home directory:

touch ~/.ssh/config

If the file did not yet exist, this will create an empty file for you.

Open this file with a text editor of your choice (note that the folder ~/.ssh might be hidden) and add the following lines at the end of that file:

Host tiko
     HostName tikoraluk.mathstat.dal.ca
     User     <user>

Here, tiko is the alias that you can choose freely, tikoraluk.mathstat.dal.ca is the domain of the remote server, and <user> is your user name on that machine.

Save the file with the name config in the .ssh folder in your local home directory. Now you should be able to log in to the remote server using the following command in your terminal:

ssh tiko

You will be prompted to enter your password on the the remote machine.

To log out of the remote server use the command ctrl+d or type exit.

If you do not want to enter your password every time, you can set up an ssh-key that gives you passwordless access to the remote server.

Open an terminal on your *local* machine and check if you already have an existing key pair by listing the contents of the .ssh folder in you home directory:

ls -l ~/.ssh

You are looking for the two files id_rsa and id_rsa.pub.

If these files do not exist, you can generate them with the following command in your terminal:

ssh-keygen -b 4096

The -b 4096 option specifies the byte size of the key.

You will be prompted to enter a passphrase, but you can leave it empty to skip this extra level of protection. If the key generation was successful, you will be shown a randomart image in your terminal.

Copy the public key to the remote server:

ssh-copy-id tiko

Note that this assumes that you have configured your ssh connection following the instructions above. If you did not set up an alias you will have to replace tiko by <user>@tikoraluk.mathstat.dal.ca, where <user> is your user name on the remote server.

Test the passwordless connection by logging in to the remote machine:

ssh tiko

If you chose a passphrase for your key, you will be prompted to enter it the first time. Afterwards, you should be able to connect to the remote server without it.

To log out of the remote server use the command ctrl+d or type exit.

To keep your code portable and reproducible, it is recommended to create a separate conda environment for each of your projects. This will also protect you from breaking your code if you have to update your Python packages in the future.

The best way to create and curate your conda environment is using an environment.yml file that lists all the Python packages that you need for your project.

In the following, we assume that you are logged in and working on the remote machine, but the same principles apply to the work on your local computer. Let us also assume that you have a folder my_project that contains everything related to your project. We will refer to this as the project folder. Typically, this folder would contain sub-directories that separate your data, code, literature, and write-ups with a structure similar to this:

my_project/
	├── data/
	│   ├── external/
	│   ├── processed/
	│   └── raw/
	├── notebooks/
	├── README.md
	├── references/
	├── reports/
	│   └── figures/
	└── src/

(You can download a simple shell program, that creates a similar folder structure for you.)

Open a new file with a text editor of your choice and add the following

name: my_project    # name of your environment
channels:           # channels to download packages from 
  - conda-forge
  - defaults
dependencies:       # list of packages you want to install 
  - python=3.9      # here we choose Python version 3.9
  - cartopy
  - cmocean
  - jupyter         # make sure to add the jupyter and 
  - jupyterlab      # jupyterlab packages
  - matplotlib
  - numpy
  - pandas
  - xarray

(Add or remove packages depending on your needs.)

Save the file as environment.yml in your project folder.

To create or update your conda environment, (e.g., if you had to add another package), open a terminal and log in to the remote server. Change into the project folder and install the packages for the environment.

cd ~/projects/my_project/
conda env update --file environment.yml --prune

You only have to do this once or every time you want to install a new package.

Once your conda environment is created, you need to activate it:

conda activate my_project

You will have to run this command every time you open a new terminal or log in to the remote server to run jupyter.

By default, the Jupyter notebook server is protected by a token that you have to enter every time you run it on a remote machine or if you run it with the –no-browser option. (When you run Jupyter on your local machine this is directly parsed into the URL if the notebook server opens your browser automatically).

Alternatively, you can set up a configuration file to enable password authentication which is generally more convenient to work with.

Check if you already have a configuration file. Assuming that you are still logged in to the remote server and have a conda environment activated that includes the Jupyter packages, list the contents of the .jupyter folder in your home directory:

ls -l ~/.jupyter

You are looking for the files jupyter_notebook_config.py and `jupyter_lab_config.py which are the configuration files for instances of Jupyter notebook and Jupyter lab, respectively.

If these files do not exist, you can create them using the commands

jupyter notebook --generate-config
jupyter lab --generate-config

To set the password for Jupyter notebooks, run the command

jupyter notebook password

This will ask you to enter and verify a password of your choice.

Similarly, to set the password for Jupyter labs, run the command

jupyter lab password

This will also ask you to enter and verify a password of your choice and can be the same as for Jupyter notebooks.

You can also use these commands to reset or change your password.

There are a variety of command line arguments that can be used to to run a notebook server. Please refer to the documentation for the available options.

On your local computer, open a terminal and log in to the remote server:

ssh tiko

After successful login, change into the project directory and activate the conda environment:

cd ~/projects/my_project/
conda activate my_project

If you have issues with conda activate, try source activate my_project.

Open Jupyter lab with the following options

jupyter lab --no-browser --port=XXXX

Change XXXX to the port number of your choice, typically 8889. It is possible that this port is occupied. In that case, try 8887, 8890, 8787, or similar.

This will start a Jupyter lab on the remote server and since you specified the –no-browser option, you will only see some status messages in the terminal.

Open a second terminal on your local computer and log in to the remote server, but this time using the port forwarding option -L and binding the remote port XXXX specified above to a local port:

ssh -N -f -L localhost:8888:localhost:XXXX tiko

Finally, open the URL http://localhost:8888/ in a browser of your choice. You will be prompted to enter your password specified in Step 4 above.

Once you are finished with your work, close the browser and stop the remote Jupyter notebook by using the ctrl+c command twice in the first terminal. To log out of the remote server use the command ctrl+d or type exit.

In the second terminal, stop the port forwarding with

lsof -ti:8888 | xargs kill -9

You can make your life easier by creating and alias in the SSH configuration file. Open the file config in the hidden folder .ssh in your home directory with a text editor of your choice and add the following lines at the end:

Host tiko-jupyter
     HostName tikoraluk.mathstat.dal.ca
     User     <user>
     LocalForward 8888 localhost:XXXX

Replace <user> by your user name and XXXX by the port number that you have found to be available above.

Instead of typing the lengthy command in the second shell, you can now use

ssh -N -f tiko-jupyter