Why do sudo, chown, and chmod fail inside a Pyxis container?

By default, enroot remaps container root to your real user ID. That isolation is expected on SUNK, and it’s why sudo, tar --chown, chmod, and chown fail inside srun --container-image sessions. Pre-build images outside Slurm, or use fakeroot for in-container installs. --no-container-remap-root makes file ownership consistent but doesn’t grant real root.

Symptoms

These errors appear only inside a Pyxis or enroot container, not in a bare srun on the same node:

sudo: effective uid is not 0
tar: Cannot change ownership
chmod: changing permissions: Operation not permitted
Package installs (apt, dpkg, conda, and pip) fail while changing ownership or permissions.

Why remapping breaks those commands

By default, enroot starts the container in a user namespace. Inside the container you appear as root (UID 0). On the host, those operations run as your Slurm user ID. Setuid binaries such as sudo and ownership changes such as tar --chown therefore fail. This is a multi-tenant security control, not a cluster misconfiguration. The enroot runtime also ignores the Dockerfile USER instruction. With default remapping, the process appears as root inside the container. With --no-container-remap-root, the process runs as the Slurm user who submitted the job, not as the user named in the image.

Choose a workaround

Approach	When to use it	What still fails
Pre-build the image outside Slurm, then run it with Pyxis	Installs, compiles, and `chown` work that need real root	Nothing at runtime if the image is already built
`fakeroot` (or `proot`) inside the image	In-container builds that only need fake root	Kernel-enforced privileges that `fakeroot` can’t intercept
`--no-container-remap-root`	You need host file ownership to match your Slurm user	`sudo` still fails. You aren’t root inside the container

Pre-build the image

Build and push the image in Docker or your CI pipeline, then run the pre-built image on SUNK. Replace [REGISTRY], [IMAGE], and [TAG] with your image coordinates.

srun --container-image=[REGISTRY]/[IMAGE]:[TAG] \
  --container-mounts=/mnt/home:/mnt/home \
  --gres=gpu:1

Use fakeroot for in-container installs

Install fakeroot in the image, then wrap the install command:

RUN apt-get update && apt-get install -y fakeroot

srun --container-image=[REGISTRY]/[IMAGE]:[TAG] fakeroot make install

Disable UID translation

srun --container-image=[REGISTRY]/[IMAGE]:[TAG] \
  --no-container-remap-root \
  --container-mounts=/mnt/home:/mnt/home

--no-container-remap-root stops UID translation. File ownership on mounted filesystems matches your Slurm user, but sudo still fails because you aren’t root.

Build images outside the login node. To run Docker inside a Slurm allocation, see Use Docker in SUNK.

Workload Scheduling

​Symptoms

​Why remapping breaks those commands

​Choose a workaround

​Pre-build the image

​Use fakeroot for in-container installs

​Disable UID translation

​Related

Symptoms

Why remapping breaks those commands

Choose a workaround

Pre-build the image

Use fakeroot for in-container installs

Disable UID translation

Related