Metadata-Version: 2.1
Name: racker
Version: 0.1.0
Summary: An experimental harness tool based on systemd-nspawn containers
Home-page: https://github.com/cicerops/racker
Author: Andreas Motl
Author-email: andreas.motl@cicerops.de
License: AGPL-3.0, EUPL-1.2
Download-URL: https://pypi.org/project/racker/
Keywords: systemd,systemd-nspawn,systemd-container,systemd-run,machinectl,virtual,environment,build,test,testing,test-harness,harness-framework,test-environment,testing-tool,harness,oci,oci-image,oci-images,virtualbox,vagrant,docker,docker-imageskopeo,umoci
Platform: UNKNOWN
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: GNU Affero General Public License v3
Classifier: Development Status :: 3 - Alpha
Classifier: Operating System :: POSIX :: Linux
Classifier: Natural Language :: English
Classifier: Intended Audience :: Customer Service
Classifier: Intended Audience :: Developers
Classifier: Intended Audience :: Education
Classifier: Intended Audience :: Information Technology
Classifier: Intended Audience :: Manufacturing
Classifier: Intended Audience :: Science/Research
Classifier: Intended Audience :: System Administrators
Classifier: Intended Audience :: Telecommunications Industry
Classifier: Topic :: Communications
Classifier: Topic :: Education :: Testing
Classifier: Topic :: Software Development :: Build Tools
Classifier: Topic :: Software Development :: Libraries
Classifier: Topic :: Software Development :: Quality Assurance
Classifier: Topic :: Software Development :: Testing
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: System :: Emulators
Classifier: Topic :: System :: Networking
Classifier: Topic :: Utilities
Provides-Extra: test
License-File: LICENSE

######
Racker
######

.. container::

    *Operating system containers for humans and machines.*

    - **Documentation**: https://github.com/cicerops/racker
    - **Source code**: https://github.com/cicerops/racker
    - **PyPI**: https://pypi.org/project/racker/

----

.. image:: https://img.shields.io/badge/systemd-239%20and%20newer-blue.svg
    :target: https://github.com/systemd/systemd
    :alt: systemd System and Service Manager

.. image:: https://img.shields.io/pypi/pyversions/racker.svg
    :target: https://pypi.org/project/racker/

.. image:: https://img.shields.io/pypi/status/racker.svg
    :target: https://pypi.org/project/racker/

.. image:: https://img.shields.io/pypi/v/racker.svg
    :target: https://pypi.org/project/racker/

.. image:: https://img.shields.io/pypi/l/racker.svg
    :alt: License
    :target: https://pypi.org/project/racker/

.. image:: https://img.shields.io/pypi/dm/racker.svg?label=PyPI%20downloads
    :target: https://pypi.org/project/racker/

----


*****
About
*****

An experimental harness tool based on `systemd-nspawn`_  containers, in the
spirit to address some details of `Docker Considered Harmful`_.
At the same time, a tribute to the authors and contributors of GNU, Linux,
systemd, VirtualBox, Vagrant, Docker, Python and more.

Racker is ...

- A runtime harness for testing software packages and similar purposes, in
  different environments, mostly run headless and non-interactive.

- A lightweight wrapper around ``systemd-nspawn`` to provide container
  environments with ``systemd``.

- A lightweight wrapper around ``vagrant`` to provide convenient access to all
  things needing a full VM, like running Windows on Linux or macOS.


**********
Background
**********

Lennart Poettering identifies three pillars of containers [1]:

- Resource bundling
- Sandboxing
- Delivery

At [2] Lennart Poettering and Kai Sievers outline their vision of systemd as a
*platform for running systems* and their focus on containers in 2014. Fast
forward to 2022, and everything is pretty much there. ``systemd`` now provides
a plethora of features for containerization, specifically for *resource
bundling* and *sandboxing* [1].

[3] outlines how systemd-nspawn was originally conceived to aid in testing and
debugging systemd, [4] is the latest overview of systemd in 2018.
For approaching ``systemd-nspawn`` from a user's perspective, a concise
introductory walkthrough can be found at [5].

The most important bits being covered by the systemd software family already,
Racker tries to fill some gaps on the *delivery* aspects.


| [1] `Containers without a Container Manager, with systemd`_ (2018)
| [2] `Lennart Poettering und Kay Sievers über Systemd`_ (2014)
| [3] `Systemd-Nspawn is Chroot on Steroids`_ (2013)
| [4] `NYLUG Presents - Lennart Poettering on Systemd in 2018`_
| [5] `Running containers with systemd-nspawn`_ (2019)


*****
Setup
*****

Install prerequisites::

    apt-get update
    apt-get install --yes systemd-container skopeo umoci python3-pip python3-venv


Install Racker::

    python3 -m venv .venv
    source .venv/bin/activate
    pip install racker --upgrade

When needing to run the latest development version, use this command instead::

    pip install git+https://github.com/cicerops/racker --upgrade

.. note::

    If you are not running Linux on your workstation, the `Racker sandbox
    installation`_ documentation outlines how to run this program within
    a virtual machine using Vagrant.



*****
Usage
*****

Basic commands::

    # Invoke the vanilla Docker `hello-world` image.
    # FIXME: Does not work yet.
    # postroj run -it --rm hello-world

    # Acquire rootfs images.
    postroj pull debian-bullseye
    postroj pull fedora-37

    # Launch an interactive shell.
    postroj run -it --rm debian-bullseye bash
    postroj run -it --rm fedora-37 bash

    # Launch a single command.
    postroj run -it --rm debian-11 hostnamectl
    postroj run -it --rm opensuse-tumbleweed hostnamectl

    # Verbose mode.
    postroj --verbose run -it --rm fedora-37 hostnamectl

    # Use stdin and stdout, with timing.
    time echo "hello world" | postroj run -it --rm fedora-37 cat /dev/stdin > hello
    cat hello

More commands::

    # List available images.
    postroj list-images

    # Acquire rootfs images for all available distributions.
    postroj pull --all

    # Run a self test procedure, invoking `hostnamectl` on all containers.
    postroj selftest hostnamectl

Package testing::

    # Run a self test procedure, invoking example probes on all containers.
    postroj selftest pkgprobe

    # Run two basic probes on different operating systems.
    postroj pkgprobe --image=debian-bullseye --check-unit=systemd-journald
    postroj pkgprobe --image=fedora-37 --check-unit=systemd-journald
    postroj pkgprobe --image=archlinux-20220501 --check-unit=systemd-journald

    # Run two probes that need installing a 3rd party package beforehand.

    postroj pkgprobe \
        --image=debian-bullseye \
        --package=https://dl.grafana.com/oss/release/grafana_8.5.1_amd64.deb \
        --check-unit=grafana-server \
        --check-network=http://localhost:3000

    postroj pkgprobe \
        --image=centos-8 \
        --package=https://dl.grafana.com/oss/release/grafana-8.5.1-1.x86_64.rpm \
        --check-unit=grafana-server \
        --check-network=http://localhost:3000


***********
Performance
***********

A SuT which just uses a dummy probe ``/bin/systemctl is-active systemd-journald``
on Debian 10 "buster" cycles quite fast, essentially demonstrating that the
overhead of environment setup/teardown is insignificant.

::

    time postroj pkgprobe --image=debian-buster --check-unit=systemd-journald

    real    0m0.589s
    user    0m0.161s
    sys     0m0.065s

On a cold system, where the filesystem image would need to be acquired before
spawning the container, it's still fast enough::

    time postroj pkgprobe --image=debian-bookworm --check-unit=systemd-journald

    real    0m22.582s
    user    0m8.572s
    sys     0m3.136s


*********************
Questions and answers
*********************

- | Q: How does it work?
  | A: Directly quoting the `machinectl`_ documentation here:

    Note that `systemd-run`_ with its ``--machine=`` switch may be used in place of the
    ``machinectl shell`` command, and allows non-interactive operation, more detailed and
    low-level configuration of the invoked unit, as well as access to runtime and exit
    code/status information of the invoked shell process.

    In particular, use ``systemd-run``'s ``--wait`` switch to propagate exit status information
    of the invoked process. Use ``systemd-run``'s ``--pty`` switch for acquiring an interactive
    shell, similar to ``machinectl shell``. In general, ``systemd-run`` is preferable for
    scripting purposes.

- | Q: How does it work, really?
  | A: Roughly speaking...

  - `skopeo`_ and `umoci`_ are used to acquire root filesystem images from Docker image registries.
  - `systemd-nspawn`_ is used to run commands on root filesystems for provisioning them.
  - Containers are started with ``systemd-nspawn --boot``.
  - `systemd-run`_ is used to interact with running containers.
  - `machinectl`_ is used to terminate containers.

- | Q: How is this project related with Docker?
  | A: The runtime is completely independent of Docker, it is solely based on
       ``systemd-nspawn`` containers instead. However, root filesystem images can be
       pulled from Docker image registries in the spirit of `machinectl pull-dkr`_.
       Other than this, the ``racker`` command and library aim to be drop-in replacements
       for their corresponding Docker counterparts.

- | Q: Do I need to have Docker installed on my machine?
  | A: No, Racker works without Docker.

- | Q: How are machine names assigned?
  | A: Machine names for spawned containers are automatically assigned.
       The name will be assembled from the distribution's ``fullname`` attribute,
       prefixed with ``postroj-``.
       Examples: ``postroj-debian-buster``, ``postroj-centos-8``.

- | Q: Does the program need root privileges?
  | A: Yes, the program currently must be invoked with ``root`` or corresponding
       ``sudo`` privileges. However, it would be sweet to enable unprivileged
       operations soon. ``systemd-nspawn`` should be able to do it, using
       ``--private-users`` or ``--user``?

- | Q: Where does the program store its data?
  | A: Data is stored at ``/var/lib/postroj``.
       In this manner, it completely gets out of the way of any other images, for
       example located at ``/var/lib/machines``. Thus, any images created or managed
       by Racker will not be listed by ``machinectl list-images``.
  | A: The download cache is located at ``/var/cache/postroj/downloads``.

- | Q: Where are the filesystem images stored?
  | A: Activated filesystem images are located at ``/var/lib/postroj/images``.

- | Q: How large are curated filesystem images?
  | A: The preference for curated filesystem images is to use their corresponding
       "slim" variants where possible, aiming to only use artefacts with download
       sizes < 100 MB.

- | Q: Are container disks ephemeral?
  | A: Yes, by default, all container images will be ephemeral, i.e. all changes to
       them are volatile.


.. _machinectl: https://www.freedesktop.org/software/systemd/man/machinectl.html
.. _systemd-nspawn: https://www.freedesktop.org/software/systemd/man/systemd-nspawn.html
.. _systemd-run: https://www.freedesktop.org/software/systemd/man/systemd-run.html

.. _Docker Considered Harmful: https://catern.com/docker.html
.. _machinectl pull-dkr: https://github.com/cicerops/racker/blob/main/doc/machinectl-pull-dkr.rst
.. _Racker sandbox installation: https://github.com/cicerops/racker/blob/main/doc/sandbox.rst
.. _skopeo: https://github.com/containers/skopeo
.. _Running containers with systemd-nspawn: https://janma.tk/2019-10-13/systemd-nspawn/
.. _umoci: https://github.com/opencontainers/umoci

.. _Containers without a Container Manager, with systemd: https://invidious.fdn.fr/watch?v=sqhojVPr7xM
.. _Systemd-Nspawn is Chroot on Steroids: https://invidious.fdn.fr/watch?v=s7LlUs5D9p4
.. _Lennart Poettering und Kay Sievers über Systemd: https://invidious.fdn.fr/watch?v=6Q_iTG6_EF4
.. _NYLUG Presents - Lennart Poettering on Systemd in 2018: https://invidious.fdn.fr/watch?v=_obJr3a_2G8


