Skip to content

Connecting your own SLURM cluster ​

Signed-in users can connect a personal SLURM account (for example Uni Jena Draco). The app never SSHs into the cluster. A small cron worker on the login nodes calls the app API with a per-cluster token, pulls jobs via short-lived S3 URLs, submits them to SLURM, and uploads results.

Before you start: access to Draco ​

Draco is the University of Jena HPC cluster for teaching and research, run by the University Computer Centre (URZ). The steps below summarise the official Accessing Draco and SSH-Login guides.

  1. Request access. Open a ticket at the URZ Service Desk, or email draco-admin@listserv.uni-jena.de with your URZ ID. Once access is granted you log in with your URZ ID and URZ password.

  2. Connect to the university network. The login nodes are only reachable from inside it, in one of three ways:

    • Ethernet on campus
    • the eduroam Wi-Fi on campus (guide)
    • the university VPN from anywhere (guide, German)

    Some institutes run public SSH gateways that can also reach Draco. That is enough for this app, because the worker only needs SSH access to install itself.

  3. Log in once by hand to accept the host key (type yes when asked):

    bash
    ssh <urzid>@login1.draco.uni-jena.de

    Draco has two login nodes, login1.draco.uni-jena.de and login2.draco.uni-jena.de.

  4. Set up key-based login (recommended). URZ requires a passphrase on keys:

    bash
    ssh-keygen -t ed25519 -a 100 -f ~/.ssh/id_ed25519_draco -C <urzid>@draco
    ssh-add ~/.ssh/id_ed25519_draco
    ssh-copy-id -i ~/.ssh/id_ed25519_draco.pub <urzid>@login1.draco.uni-jena.de
  5. Add SSH aliases to ~/.ssh/config on your laptop. The setup page uses draco1 (primary) and draco2 (standby). The URZ guide uses draco for the first node, so the config below accepts both names:

    Host draco draco1
        HostName login1.draco.uni-jena.de
        User <urzid>
        IdentityFile ~/.ssh/id_ed25519_draco
    
    Host draco2
        HostName login2.draco.uni-jena.de
        User <urzid>
        IdentityFile ~/.ssh/id_ed25519_draco

    Running ssh draco1 and ssh draco2 should now both log you in. If you would rather not use aliases, enter login1.draco.uni-jena.de,login2.draco.uni-jena.de in Login hosts when you create the cluster. Login hosts cannot contain user@, so set User <urzid> in ~/.ssh/config if your laptop username differs from your URZ ID.

  6. Optional: subscribe to the Draco-user mailing list for maintenance windows and system issues.

Only use the login nodes to manage files and submit SLURM jobs. The worker follows this rule and runs NWChem on the compute nodes. For problems with Draco itself, use the Draco Service Desk portal or email draco-admin@listserv.uni-jena.de.

Connect ​

  1. Sign in with ORCID.
  2. Open Clusters → Connect cluster.
  3. Choose the Draco preset (or Generic SLURM) and save.
  4. On the setup page, copy the one-time token and run the listed commands on the login node:
    • download and inspect setup.sh
    • run bash setup.sh (paste the token at the prompt, or set QMNMR_TOKEN)
    • from your laptop, arm the standby login node with the printed ssh … install-cron command
  5. Wait for the live checklist to turn green (worker ping, storage round-trip, SLURM, NWChem 7.2.2, compute test).

When setup.sh exits, the worker is installed but NWChem may still be compiling in a Slurm job (often for several hours). Until that build finishes, the cluster is not ready and cannot take jobs. The end of the setup.sh output says which case you are in, and the build log is at ~/qm-nmr-worker/logs/nwchem-build-<jobid>.out.

If the app has SMTP configured and your account has an email address (asked for when you connect the cluster, editable on the cluster page), you get:

  • an email after every setup.sh run, saying whether the cluster is ready or NWChem is still building;
  • an email when the build finishes and the cluster is ready to submit jobs;
  • an email if the build job ends without installing NWChem, with the log path and how to retry.

Submit jobs ​

On the submit form, choose your connected cluster under Execution Location. Offline or incomplete clusters are disabled.

Worker updates ​

From worker 0.2.0 on, the worker keeps its own code up to date. On each check-in the app reports its bundle version; when that differs from ~/qm-nmr-worker/code/bundle.json (and at least once a day), the active login node downloads the changed files from /clusters/bundle/, verifies each against the published sha256, and swaps them in. The new code runs from the next tick, about 2 minutes later. Jobs already in SLURM keep the job script and helpers they were submitted with. NWChem and the Spack environment are never touched.

Workers older than 0.2.0 cannot update themselves: re-run setup.sh once (see Connect). Running ~/qm-nmr-worker/bin/qmnmr-worker status shows the installed worker and bundle version. To pin the installed code, set "auto_update": false in ~/qm-nmr-worker/config/config.json.

Status badges ​

BadgeMeaning
onlinePing within the last 5 minutes
delayed5–15 minutes since last ping
offlineNo ping for 15+ minutes
degradedWorker alive but a check failed
building / setup incompleteInstall still in progress (NWChem build running); not ready for jobs
build_failedThe NWChem build job ended without installing NWChem; fix and re-run setup.sh

Use Test connection on the cluster detail page for an on-demand re-check (quick or full).

Troubleshooting ​

  • ssh draco1 hangs or times out: The login nodes are only reachable from the university network (campus Ethernet, eduroam, or VPN). Connect the VPN and check that the draco1/draco2 aliases are in ~/.ssh/config.
  • ssh says permission denied: Check that you are using your URZ ID. If you are, your account may not have Draco access yet, so request it through the Service Desk ticket linked above.
  • Worker never pings: Check the HTTP proxy was captured in ~/qm-nmr-worker/config/config.json, that BASE_URL is reachable from the login node, and that cron is installed (crontab -l should show # qm-nmr-worker).
  • Storage check fails: Confirm the app S3 endpoint is reachable (often direct, outside the proxy via no_proxy).
  • SLURM check fails: Partition/account must match what sbatch --test-only accepts.
  • NWChem version wrong: The worker uses an isolated Spack under ~/qm-nmr-worker/spack pinned to 7.2.2. Do not reuse a personal ~/spack develop tree.
  • Revoke: Revoking invalidates the token; the worker uninstalls itself after the next 401.

Security notes ​

  • Never put the token on a command line as an argument (ps on shared login nodes would show it).
  • The worker never holds S3 credentials — only presigned URLs.
  • Tokens and URLs must not appear in worker or app logs.