Enroot: Difference between revisions
Created page with "Enroot is a tool built by NVIDIA to run containers on Linux. Enroot is targetted for HPC environments with integration with the Slurm scheduler, but can also be used as a stan..." |
No edit summary |
||
| Line 1: | Line 1: | ||
Enroot is a tool built by NVIDIA to run containers on Linux. Enroot is | Enroot is a tool built by NVIDIA to run containers on Linux. Enroot is targeted for HPC environments with integration with the Slurm scheduler, but can also be used as a standalone tool to run containers as an unprivileged user. This is similar to Singularity, but with the added benefit of allowing users to read/write in the container and also to appear as a root user within the container environment. | ||
== Compiling and installing enroot == | ==Compiling and installing enroot== | ||
At the time of writing, there is no precompiled EL8-based version of Enroot from the releases page. Fortunately, the enroot RPM packages can be built relatively easily. On a RHEL8-based system, this should build the RPMs. | At the time of writing, there is no precompiled EL8-based version of Enroot from the releases page. Fortunately, the enroot RPM packages can be built relatively easily. On a RHEL8-based system, this should build the RPMs. | ||
{{Highlight | {{Highlight | ||
| Line 19: | Line 19: | ||
When installing Enroot, you can choose to use either the standard or the <code>+caps</code> version. The <code>+caps</code> version also contains additional capabilities for unprivileged users to import and convert container images which is handy for a HPC-based environment. | When installing Enroot, you can choose to use either the standard or the <code>+caps</code> version. The <code>+caps</code> version also contains additional capabilities for unprivileged users to import and convert container images which is handy for a HPC-based environment. | ||
=== Configuration === | ===Configuration=== | ||
Enroot has a configuration directory at <code>/etc/enroot</code>. The main configuration file is at <code>/etc/enroot/enroot.conf</code>. | Enroot has a configuration directory at <code>/etc/enroot</code>. The main configuration file is at <code>/etc/enroot/enroot.conf</code>. All settings defined in this configuration file can be overridden by defining an environment variable with the same name. | ||
A few parameters of note are listed below. | |||
* | *<code>ENROOT_MOUNT_HOME</code> - set this to yes will make Enroot always mount the user's home directory. | ||
*<code>ENROOT_SQUASH_OPTIONS='-comp lz4 -noD'</code> - Some systems don't support <code>lzo</code> (the default) which requires you to override it to <code>lz4</code>. | |||
== Quick usage == | |||
{| class="wikitable" | |||
!Task | |||
!Command | |||
|- | |||
|Import a new container from Docker Hub | |||
|<code>enroot import docker://centos:8</code> | |||
|- | |||
|Create an instance of a container | |||
|<code>enroot create --name centos centos+8.sqsh</code> | |||
|- | |||
|Destroy an instance | |||
|<code>enroot remove centos</code> | |||
|- | |||
|Convert an image from a local docker daemon | |||
You may override the squashfs options if desired. | |||
|<code>enroot import dockerd://centos:8</code> | |||
<code>ENROOT_SQUASH_OPTIONS='-comp lz4 -noD' enroot import dockerd://container:latest</code> | |||
|- | |||
|Run an enroot image | |||
|<code>enroot start --mount /tmp:tmp /images/fedora-mate.sqsh bash</code> | |||
|- | |||
|Run an enroot image as root | |||
|<code>enroot start --rw --root --mount /tmp:tmp /images/fedora-mate.sqsh bash</code> | |||
|- | |||
|Enter an existing Enroot container | |||
|<code>enroot exec $pid bash</code> | |||
Where <code>$pid</code> is the PID of <code>enroot start</code>. | |||
|} | |||
== Other notes == | |||
* There is Slurm integration using the Pyxis plugin. See more on my [[Slurm#Pyxis Plugin|Slurm]] page. | |||
* Becoming 'root' within the image (using the --root option) basically fakes your uid/gid to 0. There are consequences doing this that you should be aware of: | |||
** The user will be 'root' since their uid is 0. This might not be desirable. I have mangled <code>/etc/passwd</code> so that the user still appears as themself with this script: | |||
{{Highlight | |||
| code = if [ $(id -u) -eq 0 ] ; then | |||
ActualUsername=$(basename $HOME) | |||
echo "I see you want to be root. Updating user $ActualUsername to root" | |||
sed -i -e "s/^\($ActualUsername:[^:]\):[0-9]*:[0-9]*:/\1:0:0:/" /etc/passwd | |||
sed -i "/^root/d" /etc/passwd | |||
fi | |||
| lang = bash | |||
}} | |||
** The uid/gid cannot change once the container has started. <code>sudo</code>/<code>su</code> has no effect. | |||
** All file mounts not owned by the user will appear as nobody/nogroup. | |||
* | * | ||
Revision as of 21:22, 17 December 2021
Enroot is a tool built by NVIDIA to run containers on Linux. Enroot is targeted for HPC environments with integration with the Slurm scheduler, but can also be used as a standalone tool to run containers as an unprivileged user. This is similar to Singularity, but with the added benefit of allowing users to read/write in the container and also to appear as a root user within the container environment.
Compiling and installing enroot
At the time of writing, there is no precompiled EL8-based version of Enroot from the releases page. Fortunately, the enroot RPM packages can be built relatively easily. On a RHEL8-based system, this should build the RPMs.
# yum install -y rpm-build rpmlint libtool git libcap libtool automake jq squashfs-tools parallel fuse-overlayfs pigz squashfuse
# cd /
# git clone --recurse-submodules https://github.com/NVIDIA/enroot.git
# cd /enroot
## for hardened mode, use: make rpm PACKAGE=enroot-hardened
# CPPFLAGS="-DALLOW_SPECULATION -DINHERIT_FDS" make rpm
# ls /enroot/pkg/rpm/RPMS/x86_64/enroot*rpm
If desired, you can also build the enroot-hardened version of the package. The hardened version contains security mitigations against spectre-based exploits.
When installing Enroot, you can choose to use either the standard or the +caps version. The +caps version also contains additional capabilities for unprivileged users to import and convert container images which is handy for a HPC-based environment.
Configuration
Enroot has a configuration directory at /etc/enroot. The main configuration file is at /etc/enroot/enroot.conf. All settings defined in this configuration file can be overridden by defining an environment variable with the same name.
A few parameters of note are listed below.
ENROOT_MOUNT_HOME- set this to yes will make Enroot always mount the user's home directory.ENROOT_SQUASH_OPTIONS='-comp lz4 -noD'- Some systems don't supportlzo(the default) which requires you to override it tolz4.
Quick usage
| Task | Command |
|---|---|
| Import a new container from Docker Hub | enroot import docker://centos:8
|
| Create an instance of a container | enroot create --name centos centos+8.sqsh
|
| Destroy an instance | enroot remove centos
|
| Convert an image from a local docker daemon
You may override the squashfs options if desired. |
enroot import dockerd://centos:8
|
| Run an enroot image | enroot start --mount /tmp:tmp /images/fedora-mate.sqsh bash
|
| Run an enroot image as root | enroot start --rw --root --mount /tmp:tmp /images/fedora-mate.sqsh bash
|
| Enter an existing Enroot container | enroot exec $pid bash
Where |
Other notes
- There is Slurm integration using the Pyxis plugin. See more on my Slurm page.
- Becoming 'root' within the image (using the --root option) basically fakes your uid/gid to 0. There are consequences doing this that you should be aware of:
- The user will be 'root' since their uid is 0. This might not be desirable. I have mangled
/etc/passwdso that the user still appears as themself with this script:
- The user will be 'root' since their uid is 0. This might not be desirable. I have mangled
if [ $(id -u) -eq 0 ] ; then
ActualUsername=$(basename $HOME)
echo "I see you want to be root. Updating user $ActualUsername to root"
sed -i -e "s/^\($ActualUsername:[^:]\):[0-9]*:[0-9]*:/\1:0:0:/" /etc/passwd
sed -i "/^root/d" /etc/passwd
fi
- The uid/gid cannot change once the container has started.
sudo/suhas no effect. - All file mounts not owned by the user will appear as nobody/nogroup.
- The uid/gid cannot change once the container has started.