Nextcloud is a tool, similar to Dropbox, that allows sharing files easily through the web. It is the most convenient way to give access to your file on the storage to colleagues who live abroad.
Nextcloud is accessed through the following URL: https://nextcloud.cism.ucl.ac.be. You will need to use your CISM login and password. The default storage space in Nextcloud is very small ; it is primarily dedicated to sharing files that are on the storage. To do so, you will need to configure your Nextcloud profile and enter your CISM login and password.
Once connected, click on the triangle near your login, and choose ‘Personal’. Then click on ‘External Storage’ and give the name you want it to have in your Nextcloud space, choose SFTP as protocol, and ‘Username and password’ as Authentication. Then enter storage.cism.ucl.ac.be, you home directory (which you can discover by typing pwd when you are connected to the storage server), your CISM login and password.
Once done, you will have a new folder in your Nextcloud space that lists the contents of your space on the storage servers.
From there you can use Nextcloud’s sharing facilities to exchange files with other colleagues.
We transitioned from ownCloud to NextCloud but we did not migrate the data. If you have configured ownCloud in the past, you can still access it through the usual URL. Please migrate your data and configurations to NextCloud as the latter infrastructure is more powerful.
The SSH gateways allow you to access resources that are available only inside the university network eventhough you are home or abroad. With the gateway, you can connect to the CECI clusters, to Manneback, to the interactive machines, or to the storage machines that are normally only accessible when you are connected to the university network (wired or wifi ).
There are two gateways that you can use, depending on the account you have.
- CISM account: use
- CÉCI account: use
Please note the following restrictions in the use of the gateways:
- the CÉCI gateway can only be used as a jump host, you will not be able to SSH to it directly.
- the CISM gateway can only reach the CISM clusters, mass storage and interactive machines.
To know how to configure the gateway, please refer to the CÉCI documentation:
The information you need is the following:
- Gateway address:
- Port: 22
- Username: <Your CISM username>
- Password: <The password of your CISM account>
- Gateway address:
- Gateway address:
- Port: 22
- Username: <Your CÉCI username>
- Private ssh key: <Your CÉCI key file (
- Passphrase: <The passphrase corresponding to your CÉCI SSH key>
- Gateway address:
You do not need to be abroad or home to use and test the connection to the gateway. Make sure it all works through the gateway when you are in your office before attempting to connect from home.
This directory was created for you when you registered. It is given as a simple means to organize a replication-based backup strategy. This strategy consists in duplicating all your data on distinct servers in distinct rooms so as to minimize losses due to hazard.
Every file that you copy in this directory is synchronized every hour on another server in another room. Note that therefore, the space you use in that directory is charged twice!
Note that because it is used twice on distinct servers ; data in ALSOreplicus are copied every hour onto another server to serve as a simple backup procedure.
We offer access to a Jupyterhub service running on our private cloud. It is meant for small tests rather heavy computations.
For heavy computations, you can run a Jupyterhub service in a Slurm job allocation on any cluster and then use the nice little tool named Sshuttle tool to access it.
Sshuttle is a program that, according to its documentation, acts as a “Transparent proxy server that works as a poor man’s VPN. Forwards over ssh. Doesn’t require admin. Works with Linux and MacOS.” Installation on a Linux or MacOS laptop can be done with
dnf depending on your Linux distribution, or with
brew on MacOS, but you can also install it with
git clone from GitHub directly. See the GitHub page for details.
After you have installed Sshuttle, make sure you can access the cluster using an SSH gateway or VPN if necessary. In the following, we will assume the connection to Manneback through
gwceci is properly configured in
Next, find out in which module Jupyterhub is installed on the cluster or install it by yourself with pip. In the example, it installed in the user’s directory for the default Python module. Also find out which private network range is used on the cluster for the compute nodes. Often, it will be
10.0.0.0/8, but if you are unsure, ask the cluster admin.
Those operations will only be needed once.
Then, create a tunnel with Sshuttle. Open a terminal window that will be dedicated to that and run the following in it.
$ sshuttle -r manneback%gwceci 10.0.0.0/8 [local sudo] Password: Warning: No xauth data; using fake authentication data for X11 forwarding. client: Connected.
It will ask for your password in order to elevate privileges (
sudo). That is needed because it will modify temporarily low-level network routing. As long as it runs, the private network of the cluster will be accessible from your laptop, whereever you are connected. Leave that terminal session open for as long as you need access to the Jupyterhub service in your job. About the job, connect to the frontend (in another terminal) and submit it with
[dfr@manneback ~]$ srun --pty bash -c 'ml Python ; jupyter notebook --ip $(hostname -i)'
srun options to your liking. Once the job start, you should see something like this:
[I 15:59:54.435 NotebookApp] Serving notebooks from local directory: /auto/home/users/d/f/dfr [I 15:59:54.435 NotebookApp] The Jupyter Notebook is running at: [I 15:59:54.435 NotebookApp] http://10.3.220.101:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223 [I 15:59:54.435 NotebookApp] or http://127.0.0.1:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223 [I 15:59:54.435 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [W 15:59:54.477 NotebookApp] No web browser found: could not locate runnable browser. [C 15:59:54.478 NotebookApp] To access the notebook, open this file in a browser: file:///auto/home/users/d/f/dfr/.local/share/jupyter/runtime/nbserver-14230-open.html Or copy and paste one of these URLs: http://10.3.220.101:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223 or http://127.0.0.1:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223
If your terminal supports it, you can then click on the URL that starts with
http://10.. Otherwise simply copy that URL in your browser and you should see the Jupyterhub interface.
Once you are finished hit
CTRL-C in both terminals to stop everything.
^C[I 16:02:40.877 NotebookApp] interrupted Serving notebooks from local directory: /auto/home/users/d/f/dfr 0 active kernels The Jupyter Notebook is running at: http://10.3.220.101:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223 or http://127.0.0.1:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223 Shutdown this notebook server (y/[n])? y [C 16:02:43.185 NotebookApp] Shutdown confirmed [I 16:02:43.186 NotebookApp] Shutting down 0 kernels [dfr@manneback ~]$
^Cclient: client: Keyboard interrupt: exiting.
See it in action below: