Merge pull request #3720 from giuseppe/installer-system-container

Installer as a system container

BUILD.md

@@ -42,3 +42,22 @@ The progress of the build can be monitored with:
 Once built, the image will be visible in the Image Stream created by the same command:
 
         oc describe imagestream openshift-ansible
+
+## Build the Atomic System Container
+
+A system container runs using runC instead of Docker and it is managed
+by the [atomic](https://github.com/projectatomic/atomic/) tool.  As it
+doesn't require Docker to run, the installer can run on a node of the
+cluster without interfering with the Docker daemon that is configured
+by the installer itself.
+
+The first step is to build the [container image](#build-an-openshift-ansible-container-image)
+as described before.  The container image already contains all the
+required files to run as a system container.
+
+Once the container image is built, we can import it into the OSTree
+storage:
+
+```
+atomic pull --storage ostree docker:openshift/openshift-ansible:latest
+```

Dockerfile

@@ -16,6 +16,12 @@ LABEL name="openshift-ansible" \
 
 USER root
 
+# Create a symlink to /opt/app-root/src so that files under /usr/share/ansible are accessible.
+# This is required since the system-container uses by default the playbook under
+# /usr/share/ansible/openshift-ansible.  With this change we won't need to keep two different
+# configurations for the two images.
+RUN mkdir -p /usr/share/ansible/ && ln -s /opt/app-root/src /usr/share/ansible/openshift-ansible
+
 RUN INSTALL_PKGS="skopeo" && \
     yum install -y --setopt=tsflags=nodocs $INSTALL_PKGS && \
     rpm -V $INSTALL_PKGS && \
@@ -39,4 +45,7 @@ ADD . /tmp/src
 # as per the INSTALL_OC environment setting above
 RUN /usr/libexec/s2i/assemble
 
+# Add files for running as a system container
+COPY system-container/root /
+
 CMD [ "/usr/libexec/s2i/run" ]

Dockerfile.rhel7

@@ -39,4 +39,7 @@ ENV PLAYBOOK_FILE=playbooks/byo/openshift_facts.yml \
     WORK_DIR=/usr/share/ansible/openshift-ansible \
     OPTS="-v"
 
+# Add files for running as a system container
+COPY system-container/root /
+
 CMD [ "/usr/libexec/s2i/run" ]

README_CONTAINER_IMAGE.md

@@ -47,3 +47,26 @@ Here is a detailed explanation of the options used in the command above:
 Further usage examples are available in the [examples directory](examples/) with samples of how to use the image from within OpenShift.
 
 Additional usage information for images built from `playbook2image` like this one can be found in the [playbook2image examples](https://github.com/aweiteka/playbook2image/tree/master/examples).
+
+## Running openshift-ansible as a System Container
+
+Building the System Container: See the [BUILD.md](BUILD.md).
+
+Copy ssh public key of the host machine to master and nodes machines in the cluster.
+
+If the inventory file needs additional files then it can use the path `/var/lib/openshift-installer` in the container as it is bind mounted from the host (controllable with `VAR_LIB_OPENSHIFT_INSTALLER`).
+
+Run the ansible system container:
+
+```sh
+atomic install --system --set INVENTORY_FILE=$(pwd)/inventory.origin openshift/openshift-ansible
+systemctl start openshift-ansible
+```
+
+The `INVENTORY_FILE` variable says to the installer what inventory file on the host will be bind mounted inside the container.  In the example above, a file called `inventory.origin` in the current directory is used as the inventory file for the installer.
+
+And to finally cleanup the container:
+
+```
+atomic uninstall openshift-ansible
+```

system-container/README.md

@@ -0,0 +1,13 @@
+# System container installer
+
+These files are needed to run the installer using an [Atomic System container](http://www.projectatomic.io/blog/2016/09/intro-to-system-containers/).
+
+* config.json.template - Template of the configuration file used for running containers.
+
+* manifest.json - Used to define various settings for the system container, such as the default values to use for the installation. 
+
+* run-system-container.sh - Entrypoint to the container.
+
+* service.template - Template file for the systemd service.
+
+* tmpfiles.template - Template file for systemd-tmpfiles.

system-container/root/exports/config.json.template

@@ -0,0 +1,223 @@
+{
+    "ociVersion": "1.0.0",
+    "platform": {
+        "os": "linux",
+        "arch": "amd64"
+    },
+    "process": {
+        "terminal": false,
+        "consoleSize": {
+            "height": 0,
+            "width": 0
+        },
+        "user": {
+            "uid": 0,
+            "gid": 0
+        },
+        "args": [
+            "/usr/local/bin/run-system-container.sh"
+        ],
+        "env": [
+            "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
+            "TERM=xterm",
+            "OPTS=$OPTS",
+            "PLAYBOOK_FILE=$PLAYBOOK_FILE"
+        ],
+        "cwd": "/opt/app-root/src/",
+        "rlimits": [
+            {
+                "type": "RLIMIT_NOFILE",
+                "hard": 1024,
+                "soft": 1024
+            }
+        ],
+        "noNewPrivileges": true
+    },
+    "root": {
+        "path": "rootfs",
+        "readonly": true
+    },
+    "mounts": [
+        {
+            "destination": "/proc",
+            "type": "proc",
+            "source": "proc"
+        },
+        {
+            "destination": "/dev",
+            "type": "tmpfs",
+            "source": "tmpfs",
+            "options": [
+                "nosuid",
+                "strictatime",
+                "mode=755",
+                "size=65536k"
+            ]
+        },
+        {
+            "destination": "/dev/pts",
+            "type": "devpts",
+            "source": "devpts",
+            "options": [
+                "nosuid",
+                "noexec",
+                "newinstance",
+                "ptmxmode=0666",
+                "mode=0620",
+                "gid=5"
+            ]
+        },
+        {
+            "destination": "/dev/shm",
+            "type": "tmpfs",
+            "source": "shm",
+            "options": [
+                "nosuid",
+                "noexec",
+                "nodev",
+                "mode=1777",
+                "size=65536k"
+            ]
+        },
+        {
+            "destination": "/dev/mqueue",
+            "type": "mqueue",
+            "source": "mqueue",
+            "options": [
+                "nosuid",
+                "noexec",
+                "nodev"
+            ]
+        },
+        {
+            "destination": "/sys",
+            "type": "sysfs",
+            "source": "sysfs",
+            "options": [
+                "nosuid",
+                "noexec",
+                "nodev",
+                "ro"
+            ]
+        },
+        {
+            "type": "bind",
+            "source": "$SSH_ROOT",
+            "destination": "/opt/app-root/src/.ssh",
+            "options": [
+                "bind",
+                "rw",
+                "mode=755"
+            ]
+        },
+        {
+            "type": "bind",
+            "source": "$SSH_ROOT",
+            "destination": "/root/.ssh",
+            "options": [
+                "bind",
+                "rw",
+                "mode=755"
+            ]
+        },
+        {
+            "type": "bind",
+            "source": "$VAR_LIB_OPENSHIFT_INSTALLER",
+            "destination": "/var/lib/openshift-installer",
+            "options": [
+                "bind",
+                "rw",
+                "mode=755"
+            ]
+        },
+        {
+            "type": "bind",
+            "source": "$VAR_LOG_OPENSHIFT_LOG",
+            "destination": "/var/log/ansible.log",
+            "options": [
+                "bind",
+                "rw",
+                "mode=755"
+            ]
+        },
+        {
+            "destination": "/root/.ansible",
+            "type": "tmpfs",
+            "source": "tmpfs",
+            "options": [
+                "nosuid",
+                "strictatime",
+                "mode=755"
+            ]
+        },
+        {
+            "destination": "/tmp",
+            "type": "tmpfs",
+            "source": "tmpfs",
+            "options": [
+                "nosuid",
+                "strictatime",
+                "mode=755"
+            ]
+        },
+        {
+            "type": "bind",
+            "source": "$INVENTORY_FILE",
+            "destination": "/etc/ansible/hosts",
+            "options": [
+                "bind",
+                "rw",
+                "mode=755"
+            ]
+        },
+        {
+            "destination": "/sys/fs/cgroup",
+            "type": "cgroup",
+            "source": "cgroup",
+            "options": [
+                "nosuid",
+                "noexec",
+                "nodev",
+                "relatime",
+                "ro"
+            ]
+        }
+    ],
+    "hooks": {
+
+    },
+    "linux": {
+        "resources": {
+            "devices": [
+                {
+                    "allow": false,
+                    "access": "rwm"
+                }
+            ]
+        },
+        "namespaces": [
+            {
+                "type": "pid"
+            },
+            {
+                "type": "mount"
+            }
+        ],
+        "maskedPaths": [
+            "/proc/kcore",
+            "/proc/latency_stats",
+            "/proc/timer_list",
+            "/proc/timer_stats",
+            "/proc/sched_debug",
+            "/sys/firmware"
+        ],
+        "readonlyPaths": [
+            "/proc/asound",
+            "/proc/bus",
+            "/proc/fs",
+            "/proc/irq",
+            "/proc/sys",
+            "/proc/sysrq-trigger"
+        ]
+    }
+}

system-container/root/exports/manifest.json

@@ -0,0 +1,11 @@
+{
+    "version": "1.0",
+    "defaultValues": {
+        "OPTS": "",
+        "VAR_LIB_OPENSHIFT_INSTALLER" : "/var/lib/openshift-installer",
+        "VAR_LOG_OPENSHIFT_LOG": "/var/log/ansible.log",
+        "PLAYBOOK_FILE": "/usr/share/ansible/openshift-ansible/playbooks/byo/config.yml",
+        "SSH_ROOT": "/root/.ssh",
+        "INVENTORY_FILE": "/dev/null"
+    }
+}

system-container/root/exports/service.template

@@ -0,0 +1,6 @@
+[Service]
+ExecStart=$EXEC_START
+ExecStop=-$EXEC_STOP
+Restart=no
+WorkingDirectory=$DESTDIR
+Type=oneshot

system-container/root/exports/tmpfiles.template

@@ -0,0 +1,2 @@
+d    $VAR_LIB_OPENSHIFT_INSTALLER - - - - -
+f    $VAR_LOG_OPENSHIFT_LOG - - - - -

system-container/root/usr/local/bin/run-system-container.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+
+export ANSIBLE_LOG_PATH=/var/log/ansible.log
+exec ansible-playbook -i /etc/ansible/hosts ${OPTS} ${PLAYBOOK_FILE}