Skip to content

Connect a Slurm Site with InterLink

InterLink represents the site as one virtual Kubernetes node. Every Kubernetes pod assigned to that node becomes a separate Slurm job.

Kubernetes pod -> site virtual node -> InterLink plugin -> Slurm job

1. Send the site configuration to NRP

Send the following information to Nautilus Support:

  • site name;
  • Slurm service-account name and account;
  • CPU partition;
  • GPU partition and supported GPU constraints;
  • opportunistic GPU partition, if available;
  • maximum CPU, memory, GPU, and concurrent-job capacity to advertise; and
  • approved Slurm submit or service host where the plugin will run.

NRP will return the Kubernetes namespace, virtual-node name, tunnel hostname, and tunnel credential.

2. Verify the Slurm account

Log in to the approved submit host with the service account and verify the required commands:

Terminal window
command -v sbatch squeue scancel sinfo apptainer
sacctmgr show assoc user="$USER" format=User,Account,Partition,QOS

Submit one normal Slurm job successfully before continuing.

3. Run the installer

The NRP administrator securely transfers the tunnel credential to the site. Save it on the submit host, then download and run the installer:

Terminal window
curl --fail --location \
https://nrp.ai/downloads/interlink/install-interlink-slurm.sh \
--output /tmp/install-interlink-slurm.sh
bash /tmp/install-interlink-slurm.sh

The installer asks for the Kubernetes namespace, tunnel hostname, credential path, Slurm account, partitions, GPU constraint, and a shared storage path. It then:

  • verifies the required Slurm and Apptainer commands;
  • downloads checksum-verified InterLink and wstunnel releases;
  • creates SlurmConfig.yaml with the selected CPU and GPU flavors; and
  • installs start, stop, and status commands under $HOME/.local/bin.

No sudo access is required. Review $HOME/.config/interlink/SlurmConfig.yaml before starting the service.

For automated installation, list the supported flags with:

Terminal window
bash /tmp/install-interlink-slurm.sh --help

4. Start the plugin and tunnel

Ensure $HOME/.local/bin is on PATH, then start both processes:

Terminal window
export PATH="$HOME/.local/bin:$PATH"
interlink-slurm-start
interlink-slurm-status

View logs or stop both processes with:

Terminal window
tail -f "$HOME/.local/state/interlink/plugin.log" \
"$HOME/.local/state/interlink/tunnel.log"
interlink-slurm-stop

Do not expose plugin port 4001 directly to the Internet. After validation, configure the site’s service manager to keep both processes running.

5. Validate the site

NRP will install the InterLink virtual node and run these checks:

  1. Confirm the virtual node reports Ready.

  2. Submit the cpu flavor and confirm a Slurm job ID appears in the pod logs.

  3. Submit gpu and verify the requested GPU model with nvidia-smi.

  4. Submit several pods and confirm each creates a separate Slurm job.

  5. Delete a running pod and confirm the Slurm job is canceled.

  6. Test input and output with Ceph S3 for InterLink jobs.

Check jobs at the site with:

Terminal window
squeue -u "$USER"
scontrol show job <job-id>
sacct -j <job-id> --format=JobID,Account,Partition,State,ExitCode,NodeList,AllocTRES

Kubernetes block PVCs do not mount on remote Slurm nodes. Use Ceph S3 or storage already mounted on the Slurm compute nodes.

NSF Logo
This work was supported in part by National Science Foundation (NSF) awards CNS-1730158, ACI-1540112, ACI-1541349, OAC-1826967, OAC-2112167, CNS-2100237, CNS-2120019.