UW-Madison Department of Chemistry HPC Center

Jupyter Notebook

You must setup your own Python environment and Jupyter installation to use Jupyter Notebook.

Python setup

    • Go to https://www.python.org/downloads/source/
    • Choose the “gzipped source tarball” for the Python release you want and copy the link location to be used with the wget command below
    • ssh to kestrel and perform the following commands to compile and install Python:

cd
mkdir python3
cd python3
wget https://www.python.org/ftp/python/3.10.5/Python-3.10.5.tgz (or your preferring version from the link you copied above)
tar xzvf Python-3.10.5.tgz
cd Python-3.10.5
./configure --prefix=/home/YOURUSERNAME/python3/
make
make install

Installing Jupyter

    • Use the following 2 commands to use your Python environment:

export PATH=/home/YOURUSERNAME/python3/bin:$PATH
export PYTHONPATH=/home/YOURUSERNAME/python3

    • Use the following 2 commands to install Jupyter:

pip3 install jupyter
jupyter notebook --generate-config

    • edit /home/YOURUSERNAME/.jupyter/jupyter_notebook_config.py and add the following line to end of the file:
c.NotebookApp.ip = '*'

Submitting a Jupyter Notebook job

    • Type qjupyter at the command line and answer the questions. An example is given below.
[user@kestrel ~]$ qjupyter
This script will submit a Jupyter Notebook job.
You will need to setup your own Python environment and Jupyter installation.
Please see https://hpc.chem.wisc.edu/software/kestrel-software/jupyter-notebook/ for details.

Press Ctrl-C at any time to quit without job submission.

Please provide a job name:
notebook1
How many cores do you wish to allocate to this job?
4
Which queue do you wish to use? (enter for default)

Please enter your Python Path:
/home/username/python3
Submitting your job named notebook1 to the default queue using 4 cores with a PYTHONPATH of
/home/username/python3
Proceed? (y/n)
y
105256.kestrel.chem.wisc.edu
Please check notebook1.log in your home directory for compute node name and URL info.

Setup SSH tunnel to connect to Jupyter

    • Look in the JOBNAME.log file in your home directory for the last few lines, similar to:
To access the notebook, open this file in a browser:
        file:///home/user/.local/share/jupyter/runtime/nbserver-131237-open.html
    Or copy and paste one of these URLs:
        http://compute-0-0.local:8888/?token=3868bc184d1b82bdcf5d71622a3ad3bc72cf6839e2c14b4a
     or http://127.0.0.1:8888/?token=3868bc184d1b82bdcf5d71622a3ad3bc72cf6839e2c14b4a
  • Take note of the name of the compute node starting with compute- and the line starting with http://127.0.0.1:8888.
  • Following instructions for your operating system below:
    • Enter the following at the command-line on your local machine, where compute-0-0 is the compute node name from the step above:

ssh -L 8888:compute-0-0:8888 kestrel.chem.wisc.edu

  • Enter your username and password.
  • Run PuTTY and enter kestrel.chem.wisc.edu the Hostname box of the main Session window.

putty_kestrel

    • Under the Connection section, navigate to SSH and then Tunnels.
    • Enter 8888 in the Source port box.
    • Enter compute-0-0:8888 in the Desination box, where compute-0-0 is the compute node name from the step above.
    • Click Add.

    ssh tunnel

    • Click Open and enter your username and password.
  • Open a browser on your local machine and enter the line from the JOBNAME.log file that started with http://127.0.0.1:8888, including the token hash.

jupyter notebook

  • You can close the browser window and your job will continue to run and use resources. You will need to click the Quit button in the web interface or delete the job using the qdel command to stop it from running.