Nextcloud

Nextcloud is a tool, similar to Dropbox, that allows sharing files easily through the web. It is the most convenient way to give access to your file on the storage to colleagues who live abroad.

Nextcloud is accessed through the following URL: https://nextcloud.cism.ucl.ac.be. You will need to use your CISM login and password. The default storage space in Nextcloud is very small ; it is primarily dedicated to sharing files that are on the storage. To do so, you will need to configure your Nextcloud profile and enter your CISM login and password.

Once connected, click on the triangle near your login, and choose ‘Personal’. Then click on ‘External Storage’ and give the name you want it to have in your Nextcloud space, choose SFTP as protocol, and ‘Username and password’ as Authentication. Then enter storage.cism.ucl.ac.be, you home directory (which you can discover by typing pwd when you are connected to the storage server), your CISM login and password.

../../_images/owncloud-config.png

Once done, you will have a new folder in your Nextcloud space that lists the contents of your space on the storage servers.

../../_images/owncloud-result.png

From there you can use Nextcloud’s sharing facilities to exchange files with other colleagues.

Note

We transitioned from ownCloud to NextCloud but we did not migrate the data. If you have configured ownCloud in the past, you can still access it through the usual URL. Please migrate your data and configurations to NextCloud as the latter infrastructure is more powerful.

SSH gateways

The SSH gateways allow you to access resources that are available only inside the university network eventhough you are home or abroad. With the gateway, you can connect to the CECI clusters, to Manneback, to the interactive machines, or to the storage machines that are normally only accessible when you are connected to the university network (wired or wifi ).

There are two gateways that you can use, depending on the account you have.

  • CISM account: use gwcism
  • CÉCI account: use gwceci

To know how to configure the gateway, please refer to the CÉCI documentation:

The information you need is the following:

  • For gwcism :
    • Gateway address: gwcism.cism.ucl.ac.be
    • Port: 22
    • Username: <Your CISM username>
    • Password: <The password of your CISM account>
  • For gwceci :
    • Gateway address: gwceci.cism.ucl.ac.be
    • Port: 22
    • Username: <Your CÉCI username>
    • Private ssh key: <Your CÉCI key file (id_rsa.ceci)>
    • Passphrase: <The passphrase corresponding to your CÉCI SSH key>

To renew your account from outside UCLouvain, you can simply connect to gwcism.cism.ucl.ac.be with X forwarding and run firefox with the --no-remote option. Go to the website as usual, enter your email address and then copy/paste the URL you then receive by email in the remote firefox window and fill in the web form from there.

Note

You do not need to be abroad or home to use and test the connectio to the gateway. Make sure it all works through the gatewat when you are in your office before attempting to connect from home.

Note

Also make your connection to the gateway itself is functionnal before using it through a proxy command.

ALSOreplicus

This directory was created for you when you registered. It is given as a simple means to organize a replication-based backup strategy. This strategy consists in duplicating all your data on distinct servers in distinct rooms so as to minimize losses due to hazard.

Every file that you copy in this directory is synchronized every hour on another server in another room. Note that therefore, the space you use in that directory is charged twice!

Note that because it is used twice on distinct servers ; data in ALSOreplicus are copied every hour onto another server to serve as a simple backup procedure.

Jupyter

We offer access to a Jupyterhub service running on our private cloud. It is meant for small tests rather heavy computations.

For heavy computations, you can run a Jupyterhub service in a Slurm job allocation on any cluster and then use the nice little tool named Sshuttle tool to access it.

Sshuttle is a program that, according to its documentation, acts as a “Transparent proxy server that works as a poor man’s VPN. Forwards over ssh. Doesn’t require admin. Works with Linux and MacOS.” Installation on a Linux or MacOS laptop can be done with apt, pacman, dnf depending on your Linux distribution, or with brew on MacOS, but you can also install it with pip or git clone from GitHub directly. See the GitHub page for details.

After you have installed Sshuttle, make sure you can access the cluster using an SSH gateway or VPN if necessary. In the following, we will assume the connection to Manneback through gwceci is properly configured in .ssh/config as manneback%gwceci.

Next, find out in which module Jupyterhub is installed on the cluster or install it by yourself with pip. In the example, it installed in the user’s directory for the default Python module. Also find out which private network range is used on the cluster for the compute nodes. Often, it will be 10.0.0.0/8, but if you are unsure, ask the cluster admin.

Those operations will only be needed once.

Then, create a tunnel with Sshuttle. Open a terminal window that will be dedicated to that and run the following in it.

$ sshuttle -r manneback%gwceci 10.0.0.0/8
[local sudo] Password:
Warning: No xauth data; using fake authentication data for X11 forwarding.
client: Connected.

It will ask for your password in order to elevate privileges (sudo). That is needed because it will modify temporarily low-level network routing. As long as it runs, the private network of the cluster will be accessible from your laptop, whereever you are connected. Leave that terminal session open for as long as you need access to the Jupyterhub service in your job. About the job, connect to the frontend (in another terminal) and submit it with

[dfr@manneback ~]$ srun --pty bash -c 'ml Python ; jupyter notebook --ip $(hostname -i)'

Adapt the srun options to your liking. Once the job start, you should see something like this:

[I 15:59:54.435 NotebookApp] Serving notebooks from local directory: /auto/home/users/d/f/dfr
[I 15:59:54.435 NotebookApp] The Jupyter Notebook is running at:
[I 15:59:54.435 NotebookApp] http://10.3.220.101:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223
[I 15:59:54.435 NotebookApp]  or http://127.0.0.1:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223
[I 15:59:54.435 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 15:59:54.477 NotebookApp] No web browser found: could not locate runnable browser.
[C 15:59:54.478 NotebookApp]

    To access the notebook, open this file in a browser:
        file:///auto/home/users/d/f/dfr/.local/share/jupyter/runtime/nbserver-14230-open.html
    Or copy and paste one of these URLs:
        http://10.3.220.101:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223
     or http://127.0.0.1:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223

If your terminal supports it, you can then click on the URL that starts with http://10.. Otherwise simply copy that URL in your browser and you should see the Jupyterhub interface.

Once you are finished hit CTRL-C in both terminals to stop everything.

^C[I 16:02:40.877 NotebookApp] interrupted
Serving notebooks from local directory: /auto/home/users/d/f/dfr
0 active kernels
The Jupyter Notebook is running at:
http://10.3.220.101:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223
 or http://127.0.0.1:8888/?token=4bd6038c1f0df6e51d901891b95b98ebf8e899646cc68223
Shutdown this notebook server (y/[n])? y
[C 16:02:43.185 NotebookApp] Shutdown confirmed
[I 16:02:43.186 NotebookApp] Shutting down 0 kernels
[dfr@manneback ~]$
^Cclient:
client: Keyboard interrupt: exiting.

See it in action below: