First attempt at provisioning.

playbooks/aws/README.md

@@ -1,4 +1,269 @@
 # AWS playbooks
 
-This playbook directory is meant to be driven by [`bin/cluster`](../../bin),
-which is community supported and most use is considered deprecated.
+Parts of this playbook directory are meant to be driven by [`bin/cluster`](../../bin),
+which is community supported and use is considered **deprecated**.
+
+
+## Provisioning
+
+With recent desire for provisioning from customers and developers alike, the AWS
+ playbook directory now supports a limited set of ansible playbooks to achieve a
+ complete cluster setup. These playbooks bring into alignment our desire to
+ deploy highly scalable Openshift clusters utilizing AWS auto scale groups and
+ custom AMIs.
+
+### Where do I start?
+
+Before any provisioning may occur, AWS account credentials must be present in the environment.  This can be done in two ways:
+
+- Create the following file `~/.aws/credentials` with the contents (substitute your access key and secret key):
+   ```
+   [myaccount]
+   aws_access_key_id = <Your access_key here>
+   aws_secret_access_key = <Your secret acces key here>
+   ```
+   From the shell:
+   ```
+   $ export AWS_PROFILE=myaccount
+   ```
+ ---
+- Alternatively to using a profile you can export your AWS credentials as environment variables.
+   ```
+  $ export AWS_ACCESS_KEY_ID=AKIXXXXXX
+  $ export AWS_SECRET_ACCESS_KEY=XXXXXX
+   ```
+
+### Let's Provision!
+
+The newly added playbooks are the following:
+- build_ami.yml
+- provision.yml
+
+The current expected work flow should be to provide the `vars.yml` file with the
+desired settings for cluster instances.  These settings are AWS specific and should
+be tailored to the consumer's AWS custom account settings.
+
+```yaml
+clusterid: mycluster
+region: us-east-1
+
+provision:
+  clusterid: "{{ clusterid }}"
+  region: "{{ region }}"
+
+  build:
+    base_image: ami-bdd5d6ab # base image for AMI to build from
+    # when creating an encrypted AMI please specify use_encryption
+    use_encryption: False
+
+    yum_repositories: # this is an example repository but it requires sslclient info.  Use a valid yum repository for openshift rpms
+    - name: openshift-repo
+      file: openshift-repo
+      description: OpenShift Builds
+      baseurl: https://mirror.openshift.com/enterprise/online-int/latest/x86_64/os/
+      enabled: yes
+      gpgcheck: no
+      sslverify: no
+      # client cert and key required for this repository
+      sslclientcert: "/var/lib/yum/client-cert.pem"
+      sslclientkey: "/var/lib/yum/client-key.pem"
+      gpgkey: "https://mirror.ops.rhcloud.com/libra/keys/RPM-GPG-KEY-redhat-release https://mirror.ops.rhcloud.com/libra/keys/RPM-GPG-KEY-redhat-beta https://mirror.ops.rhcloud.com/libra/keys/RPM-GPG-KEY-redhat-openshifthosted"
+
+  # for s3 registry backend
+  openshift_registry_s3: True
+
+  # if using custom certificates these are required for the ELB
+  iam_cert_ca:
+    name: test_openshift
+    cert_path: '/path/to/wildcard.<clusterid>.example.com.crt'
+    key_path: '/path/to/wildcard.<clusterid>.example.com.key'
+    chain_path: '/path/to/cert.ca.crt'
+
+  instance_users:
+  - key_name: myuser_key
+    username: myuser
+    pub_key: |
+           ssh-rsa aaa<place public ssh key here>aaaaa user@<clusterid>
+
+  node_group_config:
+    tags:
+      clusterid: "{{ clusterid }}"
+      environment: stg
+    ssh_key_name: myuser_key # name of the ssh key from above
+
+    # configure master settings here
+    master:
+      instance_type: m4.xlarge
+      ami: ami-cdeec8b6 # if using an encrypted AMI this will be replaced
+      volumes:
+      - device_name: /dev/sdb
+        volume_size: 100
+        device_type: gp2
+        delete_on_termination: False
+      health_check:
+        period: 60
+        type: EC2
+      # Set the following number to be the same for masters.
+      min_size: 3
+      max_size: 3
+      desired_size: 3
+      tags:
+        host-type: master
+        sub-host-type: default
+      wait_for_instances: True
+...
+  vpc:
+    # name: mycluster  # If missing; will default to clusterid
+    cidr: 172.31.0.0/16
+    subnets:
+      us-east-1:  # These are us-east-1 region defaults. Ensure this matches your region
+      - cidr: 172.31.48.0/20
+        az: "us-east-1c"
+      - cidr: 172.31.32.0/20
+        az: "us-east-1e"
+      - cidr: 172.31.16.0/20
+        az: "us-east-1a"
+
+```
+
+Repeat the following setup for the infra and compute node groups.  This most likely
+ will not need editing but if further customization is required these parameters
+ can be updated.
+
+#### Step 1
+
+Once the vars.yml file has been updated with the correct settings for the desired AWS account then we are ready to build an AMI.
+
+```
+$ ansible-playbook build_ami.yml
+```
+
+1. This script will build a VPC. Default name will be clusterid if not specified.
+2. Create an ssh key required for the instance.
+3. Create an instance.
+4. Run some setup roles to ensure packages and services are correctly configured.
+5. Create the AMI.
+6. If encryption is desired
+  - A KMS key is created with the name of $clusterid
+  - An encrypted AMI will be produced with $clusterid KMS key
+7. Terminate the instance used to configure the AMI.
+
+#### Step 2
+
+Now that we have created an AMI for our Openshift installation, that AMI id needs to be placed in the `vars.yml` file.  To do so update the following fields (The AMI can be captured from the output of the previous step or found in the ec2 console under AMIs):
+
+```
+  # when creating an encrypted AMI please specify use_encryption
+  use_encryption: False # defaults to false
+```
+
+**Note**: If using encryption, specify with `use_encryption: True`.  This will ensure to take the recently created AMI and encrypt it to be used later.  If encryption is not desired then set the value to false. The AMI id will be fetched and used according to its most recent creation date. 
+
+#### Step 3
+
+Create an openshift-ansible inventory file to use for a byo installation.  The exception here is that there will be no hosts specified by the inventory file.  Here is an example:
+
+```ini
+[OSEv3:children]
+masters
+nodes
+etcd
+
+[OSEv3:children]
+masters
+nodes
+etcd
+
+[OSEv3:vars]
+# cluster specific settings maybe be placed here
+openshift_hosted_router_wait=False
+openshift_hosted_registry_wait=False
+
+[masters]
+
+[etcd]
+
+[nodes]
+```
+
+There are more examples of cluster inventory settings [`here`](../../inventory/byo/).
+
+#### Step 4
+
+We are ready to create the master instances and install Openshift.
+
+```
+$ ansible-playbook -i <inventory from step 3> provision.yml
+```
+
+This playbook runs through the following steps:
+1. Ensures a VPC is created
+2. Ensures a SSH key exists
+3. Creates an s3 bucket for the registry named $clusterid
+4. Create master security groups
+5. Create a master launch config
+6. Create the master auto scaling groups
+7. If certificates are desired for ELB, they will be uploaded
+8. Create internal and external master ELBs
+9. Add newly created masters to the correct groups
+10. Set a couple of important facts for the masters
+11. Run the [`byo`](../../common/openshift-cluster/config.yml)
+
+At this point we have created a successful cluster with only the master nodes.
+
+
+#### Step 5
+
+Now that we have a cluster deployed it might be more interesting to create some node types.  This can be done easily with the following playbook:
+
+```
+$ ansible-playbook provision_nodes.yml
+```
+
+Once this playbook completes, it should create the compute and infra node scale groups.  These nodes will attempt to register themselves to the cluster.  These requests must be approved by an administrator.
+
+#### Step 6
+
+The registration of our nodes can be automated by running the following script `accept.yml`.  This script can handle the registration in a few different ways.
+- approve_all - **Note**: this option is for development and test environments.  Security is bypassed
+- nodes - A list of node names that will be accepted into the cluster
+
+```yaml
+ oc_adm_csr:
+   #approve_all: True
+   nodes: < list of nodes here >
+   timeout: 0
+```
+Once the desired accept method is chosen, run the following playbook `accept.yml`:
+1. Run the following playbook.
+```
+$ ansible-playbook accept.yml
+```
+
+Login to a master and run the following command:
+```
+ssh root@<master ip address>
+$ oc --config=/etc/origin/master/admin.kubeconfig get csr
+node-bootstrapper-client-ip-172-31-49-148-ec2-internal   1h       system:serviceaccount:openshift-infra:node-bootstrapper   Approved,Issued
+node-bootstrapper-server-ip-172-31-49-148-ec2-internal   1h       system:node:ip-172-31-49-148.ec2.internal                 Approved,Issued
+```
+
+Verify the `CONDITION` is `Approved,Issued` on the `csr` objects.  There are two for each node required.
+1. `node-bootstrapper-client` is a request to access the api/controllers.
+2. `node-bootstrapper-server` is a request to join the cluster.
+
+Once this is complete, verify the nodes have joined the cluster and are `ready`.
+
+```
+$ oc --config=/etc/origin/master/admin.kubeconfig get nodes
+NAME                            STATUS                     AGE       VERSION
+ip-172-31-49-148.ec2.internal   Ready                      1h       v1.6.1+5115d708d7
+```
+
+### Ready To Work!
+
+At this point your cluster should be ready for workloads.  Proceed to deploy applications on your cluster.
+
+### Still to compute
+
+There are more enhancements that are arriving for provisioning.  These will include more playbooks that enhance the provisioning capabilities.

playbooks/aws/openshift-cluster/build_ami.yml

@@ -0,0 +1,134 @@
+---
+- hosts: localhost
+  connection: local
+  gather_facts: no
+  tasks:
+  - name: get the necessary vars for ami building
+    include_vars: vars.yml
+
+  - name: create a vpc with the name <clusterid>
+    include_role:
+      name: openshift_aws_vpc
+    vars:
+      r_openshift_aws_vpc_clusterid: "{{ provision.clusterid }}"
+      r_openshift_aws_vpc_cidr: "{{ provision.vpc.cidr }}"
+      r_openshift_aws_vpc_subnets: "{{ provision.vpc.subnets }}"
+      r_openshift_aws_vpc_region: "{{ provision.region }}"
+      r_openshift_aws_vpc_tags: "{{ provision.vpc.tags }}"
+      r_openshift_aws_vpc_name: "{{ provision.vpc.name | default(provision.clusterid) }}"
+
+  - name: create aws ssh keypair
+    include_role:
+      name: openshift_aws_ssh_keys
+    vars:
+      r_openshift_aws_ssh_keys_users: "{{ provision.instance_users }}"
+      r_openshift_aws_ssh_keys_region: "{{ provision.region }}"
+
+  - name: fetch the default subnet id
+    ec2_vpc_subnet_facts:
+      region: "{{ provision.region }}"
+      filters:
+        "tag:Name": "{{ provision.vpc.subnets[provision.region][0].az }}"
+    register: subnetout
+
+  - name: create instance for ami creation
+    ec2:
+      assign_public_ip: yes
+      region: "{{ provision.region }}"
+      key_name: "{{ provision.node_group_config.ssh_key_name }}"
+      group: "{{ provision.clusterid }}"
+      instance_type: m4.xlarge
+      vpc_subnet_id: "{{ subnetout.subnets[0].id }}"
+      image: "{{ provision.build.base_image }}"
+      volumes:
+      - device_name: /dev/sdb
+        volume_type: gp2
+        volume_size: 100
+        delete_on_termination: true
+      wait: yes
+      exact_count: 1
+      count_tag:
+        Name: ami_base
+      instance_tags:
+        Name: ami_base
+    register: amibase
+
+  - name: wait for ssh to become available
+    wait_for:
+      port: 22
+      host: "{{ amibase.tagged_instances.0.public_ip }}"
+      timeout: 300
+      search_regex: OpenSSH
+
+  - name: add host to group
+    add_host:
+      name: "{{ amibase.tagged_instances.0.public_dns_name }}"
+      groups: amibase
+
+- hosts: amibase
+  remote_user: root
+  tasks:
+  - name: included required variables
+    include_vars: vars.yml
+
+  - name: run openshift image preparation
+    include_role:
+      name: openshift_ami_prep
+    vars:
+      r_openshift_ami_prep_yum_repositories: "{{ provision.build.yum_repositories }}"
+      r_openshift_ami_prep_node: atomic-openshift-node
+      r_openshift_ami_prep_master: atomic-openshift-master
+
+- hosts: localhost
+  connection: local
+  become: no
+  tasks:
+  - name: bundle ami
+    ec2_ami:
+      instance_id: "{{ amibase.tagged_instances.0.id }}"
+      region: "{{ provision.region }}"
+      state: present
+      description: "This was provisioned {{ ansible_date_time.iso8601 }}"
+      name: "{{ provision.build.ami_name }}{{ lookup('pipe', 'date +%Y%m%d%H%M')}}"
+      wait: yes
+    register: amioutput
+
+  - debug: var=amioutput
+
+  - when: provision.build.use_encryption | default(False)
+    block:
+    - name: setup kms key for encryption
+      include_role:
+        name: openshift_aws_iam_kms
+      vars:
+        r_openshift_aws_iam_kms_region: "{{ provision.region }}"
+        r_openshift_aws_iam_kms_alias: "alias/{{ provision.clusterid }}_kms"
+
+    - name: augment the encrypted ami tags with source-ami
+      set_fact:
+        source_tag:
+          source-ami: "{{ amioutput.image_id }}"
+
+    - name: copy the ami for encrypted disks
+      include_role:
+        name: openshift_aws_ami_copy
+      vars:
+        r_openshift_aws_ami_copy_region: "{{ provision.region }}"
+        r_openshift_aws_ami_copy_name: "{{ provision.build.ami_name }}{{ lookup('pipe', 'date +%Y%m%d%H%M')}}-encrypted"
+        r_openshift_aws_ami_copy_src_ami: "{{ amioutput.image_id }}"
+        r_openshift_aws_ami_copy_kms_alias: "alias/{{ provision.clusterid }}_kms"
+        r_openshift_aws_ami_copy_tags: "{{ source_tag | combine(provision.build.openshift_ami_tags) }}"
+        r_openshift_aws_ami_copy_encrypt: "{{ provision.build.use_encryption }}"
+        # this option currently fails due to boto waiters
+        # when supported this need to be reapplied
+        #r_openshift_aws_ami_copy_wait: True
+
+    - name: Display newly created encrypted ami id
+      debug:
+        msg: "{{ r_openshift_aws_ami_copy_retval_custom_ami }}"
+
+  - name: terminate temporary instance
+    ec2:
+      state: absent
+      region: "{{ provision.region }}"
+      instance_ids: "{{ amibase.tagged_instances.0.id }}"

playbooks/aws/openshift-cluster/build_node_group.yml

@@ -0,0 +1,47 @@
+---
+- name: fetch recently created AMI
+  ec2_ami_find:
+    region: "{{ provision.region }}"
+    sort: creationDate
+    sort_order: descending
+    name: "{{ provision.build.ami_name }}*"
+    ami_tags: "{{ provision.build.openshift_ami_tags }}"
+    #no_result_action: fail
+  register: amiout
+
+- block:
+  - name: "Create {{ openshift_build_node_type }} sgs"
+    include_role:
+      name: openshift_aws_sg
+    vars:
+      r_openshift_aws_sg_clusterid: "{{ provision.clusterid }}"
+      r_openshift_aws_sg_region: "{{ provision.region }}"
+      r_openshift_aws_sg_type: "{{ openshift_build_node_type }}"
+
+  - name: "generate a launch config name for {{ openshift_build_node_type }}"
+    set_fact:
+      launch_config_name: "{{ provision.clusterid }}-{{ openshift_build_node_type }}-{{ ansible_date_time.epoch }}"
+
+  - name: create "{{ openshift_build_node_type }} launch config"
+    include_role:
+      name: openshift_aws_launch_config
+    vars:
+      r_openshift_aws_launch_config_name: "{{ launch_config_name }}"
+      r_openshift_aws_launch_config_clusterid: "{{ provision.clusterid }}"
+      r_openshift_aws_launch_config_region: "{{ provision.region }}"
+      r_openshift_aws_launch_config: "{{ provision.node_group_config }}"
+      r_openshift_aws_launch_config_type: "{{ openshift_build_node_type }}"
+      r_openshift_aws_launch_config_custom_image: "{{ '' if 'results' not in amiout else amiout.results[0].ami_id }}"
+      r_openshift_aws_launch_config_bootstrap_token: "{{ (local_bootstrap['content'] |b64decode) if local_bootstrap is defined else '' }}"
+
+  - name: "create {{ openshift_build_node_type }} node groups"
+    include_role:
+      name: openshift_aws_node_group
+    vars:
+      r_openshift_aws_node_group_name: "{{ provision.clusterid }} openshift {{ openshift_build_node_type }}"
+      r_openshift_aws_node_group_lc_name: "{{ launch_config_name }}"
+      r_openshift_aws_node_group_clusterid: "{{ provision.clusterid }}"
+      r_openshift_aws_node_group_region: "{{ provision.region }}"
+      r_openshift_aws_node_group_config: "{{ provision.node_group_config }}"
+      r_openshift_aws_node_group_type: "{{ openshift_build_node_type }}"
+      r_openshift_aws_node_group_subnet_name: "{{ provision.vpc.subnets[provision.region][0].az }}"

playbooks/aws/openshift-cluster/library/ec2_ami_find.py

@@ -1,303 +0,0 @@
-#!/usr/bin/python
-#pylint: skip-file
-# flake8: noqa
-#
-# This file is part of Ansible
-#
-# Ansible is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 3 of the License, or
-# (at your option) any later version.
-#
-# Ansible is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
-# GNU General Public License for more details.
-#
-# You should have received a copy of the GNU General Public License
-# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
-
-DOCUMENTATION = '''
----
-module: ec2_ami_find
-version_added: 2.0
-short_description: Searches for AMIs to obtain the AMI ID and other information
-description:
-  - Returns list of matching AMIs with AMI ID, along with other useful information
-  - Can search AMIs with different owners
-  - Can search by matching tag(s), by AMI name and/or other criteria
-  - Results can be sorted and sliced
-author: Tom Bamford
-notes:
-  - This module is not backwards compatible with the previous version of the ec2_search_ami module which worked only for Ubuntu AMIs listed on cloud-images.ubuntu.com.
-  - See the example below for a suggestion of how to search by distro/release.
-options:
-  region:
-    description:
-      - The AWS region to use.
-    required: true
-    aliases: [ 'aws_region', 'ec2_region' ]
-  owner:
-    description:
-      - Search AMIs owned by the specified owner
-      - Can specify an AWS account ID, or one of the special IDs 'self', 'amazon' or 'aws-marketplace'
-      - If not specified, all EC2 AMIs in the specified region will be searched.
-      - You can include wildcards in many of the search options. An asterisk (*) matches zero or more characters, and a question mark (?) matches exactly one character. You can escape special characters using a backslash (\) before the character. For example, a value of \*amazon\?\\ searches for the literal string *amazon?\.
-    required: false
-    default: null
-  ami_id:
-    description:
-      - An AMI ID to match.
-    default: null
-    required: false
-  ami_tags:
-    description:
-      - A hash/dictionary of tags to match for the AMI.
-    default: null
-    required: false
-  architecture:
-    description:
-      - An architecture type to match (e.g. x86_64).
-    default: null
-    required: false
-  hypervisor:
-    description:
-      - A hypervisor type type to match (e.g. xen).
-    default: null
-    required: false
-  is_public:
-    description:
-      - Whether or not the image(s) are public.
-    choices: ['yes', 'no']
-    default: null
-    required: false
-  name:
-    description:
-      - An AMI name to match.
-    default: null
-    required: false
-  platform:
-    description:
-      - Platform type to match.
-    default: null
-    required: false
-  sort:
-    description:
-      - Optional attribute which with to sort the results.
-      - If specifying 'tag', the 'tag_name' parameter is required.
-    choices: ['name', 'description', 'tag']
-    default: null
-    required: false
-  sort_tag:
-    description:
-      - Tag name with which to sort results.
-      - Required when specifying 'sort=tag'.
-    default: null
-    required: false
-  sort_order:
-    description:
-      - Order in which to sort results.
-      - Only used when the 'sort' parameter is specified.
-    choices: ['ascending', 'descending']
-    default: 'ascending'
-    required: false
-  sort_start:
-    description:
-      - Which result to start with (when sorting).
-      - Corresponds to Python slice notation.
-    default: null
-    required: false
-  sort_end:
-    description:
-      - Which result to end with (when sorting).
-      - Corresponds to Python slice notation.
-    default: null
-    required: false
-  state:
-    description:
-      - AMI state to match.
-    default: 'available'
-    required: false
-  virtualization_type:
-    description:
-      - Virtualization type to match (e.g. hvm).
-    default: null
-    required: false
-  no_result_action:
-    description:
-      - What to do when no results are found.
-      - "'success' reports success and returns an empty array"
-      - "'fail' causes the module to report failure"
-    choices: ['success', 'fail']
-    default: 'success'
-    required: false
-requirements:
-  - boto
-
-'''
-
-EXAMPLES = '''
-# Note: These examples do not set authentication details, see the AWS Guide for details.
-
-# Search for the AMI tagged "project:website"
-- ec2_ami_find:
-    owner: self
-    tags:
-      project: website
-    no_result_action: fail
-  register: ami_find
-
-# Search for the latest Ubuntu 14.04 AMI
-- ec2_ami_find:
-    name: "ubuntu/images/ebs/ubuntu-trusty-14.04-amd64-server-*"
-    owner: 099720109477
-    sort: name
-    sort_order: descending
-    sort_end: 1
-  register: ami_find
-
-# Launch an EC2 instance
-- ec2:
-    image: "{{ ami_search.results[0].ami_id }}"
-    instance_type: m4.medium
-    key_name: mykey
-    wait: yes
-'''
-
-try:
-    import boto.ec2
-    HAS_BOTO=True
-except ImportError:
-    HAS_BOTO=False
-
-import json
-
-def main():
-    argument_spec = ec2_argument_spec()
-    argument_spec.update(dict(
-            region = dict(required=True,
-                aliases = ['aws_region', 'ec2_region']),
-            owner = dict(required=False, default=None),
-            ami_id = dict(required=False),
-            ami_tags = dict(required=False, type='dict',
-                aliases = ['search_tags', 'image_tags']),
-            architecture = dict(required=False),
-            hypervisor = dict(required=False),
-            is_public = dict(required=False),
-            name = dict(required=False),
-            platform = dict(required=False),
-            sort = dict(required=False, default=None,
-                choices=['name', 'description', 'tag']),
-            sort_tag = dict(required=False),
-            sort_order = dict(required=False, default='ascending',
-                choices=['ascending', 'descending']),
-            sort_start = dict(required=False),
-            sort_end = dict(required=False),
-            state = dict(required=False, default='available'),
-            virtualization_type = dict(required=False),
-            no_result_action = dict(required=False, default='success',
-                choices = ['success', 'fail']),
-        )
-    )
-
-    module = AnsibleModule(
-        argument_spec=argument_spec,
-    )
-
-    if not HAS_BOTO:
-        module.fail_json(msg='boto required for this module, install via pip or your package manager')
-
-    ami_id = module.params.get('ami_id')
-    ami_tags = module.params.get('ami_tags')
-    architecture = module.params.get('architecture')
-    hypervisor = module.params.get('hypervisor')
-    is_public = module.params.get('is_public')
-    name = module.params.get('name')
-    owner = module.params.get('owner')
-    platform = module.params.get('platform')
-    sort = module.params.get('sort')
-    sort_tag = module.params.get('sort_tag')
-    sort_order = module.params.get('sort_order')
-    sort_start = module.params.get('sort_start')
-    sort_end = module.params.get('sort_end')
-    state = module.params.get('state')
-    virtualization_type = module.params.get('virtualization_type')
-    no_result_action = module.params.get('no_result_action')
-
-    filter = {'state': state}
-
-    if ami_id:
-        filter['image_id'] = ami_id
-    if ami_tags:
-        for tag in ami_tags:
-            filter['tag:'+tag] = ami_tags[tag]
-    if architecture:
-        filter['architecture'] = architecture
-    if hypervisor:
-        filter['hypervisor'] = hypervisor
-    if is_public:
-        filter['is_public'] = is_public
-    if name:
-        filter['name'] = name
-    if platform:
-        filter['platform'] = platform
-    if virtualization_type:
-        filter['virtualization_type'] = virtualization_type
-
-    ec2 = ec2_connect(module)
-
-    images_result = ec2.get_all_images(owners=owner, filters=filter)
-
-    if no_result_action == 'fail' and len(images_result) == 0:
-        module.fail_json(msg="No AMIs matched the attributes: %s" % json.dumps(filter))
-
-    results = []
-    for image in images_result:
-        data = {
-            'ami_id': image.id,
-            'architecture': image.architecture,
-            'description': image.description,
-            'is_public': image.is_public,
-            'name': image.name,
-            'owner_id': image.owner_id,
-            'platform': image.platform,
-            'root_device_name': image.root_device_name,
-            'root_device_type': image.root_device_type,
-            'state': image.state,
-            'tags': image.tags,
-            'virtualization_type': image.virtualization_type,
-        }
-
-        if image.kernel_id:
-            data['kernel_id'] = image.kernel_id
-        if image.ramdisk_id:
-            data['ramdisk_id'] = image.ramdisk_id
-
-        results.append(data)
-
-    if sort == 'tag':
-        if not sort_tag:
-            module.fail_json(msg="'sort_tag' option must be given with 'sort=tag'")
-        results.sort(key=lambda e: e['tags'][sort_tag], reverse=(sort_order=='descending'))
-    elif sort:
-        results.sort(key=lambda e: e[sort], reverse=(sort_order=='descending'))
-
-    try:
-        if sort and sort_start and sort_end:
-            results = results[int(sort_start):int(sort_end)]
-        elif sort and sort_start:
-            results = results[int(sort_start):]
-        elif sort and sort_end:
-            results = results[:int(sort_end)]
-    except TypeError:
-        module.fail_json(msg="Please supply numeric values for sort_start and/or sort_end")
-
-    module.exit_json(results=results)
-
-# import module snippets
-from ansible.module_utils.basic import *
-from ansible.module_utils.ec2 import *
-
-if __name__ == '__main__':
-    main()
-

playbooks/aws/openshift-cluster/provision.yml

@@ -0,0 +1,157 @@
+---
+- name: Setup the vpc and the master node group
+  hosts: localhost
+  tasks:
+  - name: get provisioning vars
+    include_vars: vars.yml
+
+  - name: create default vpc
+    include_role:
+      name: openshift_aws_vpc
+    vars:
+      r_openshift_aws_vpc_clusterid: "{{ provision.clusterid }}"
+      r_openshift_aws_vpc_cidr: "{{ provision.vpc.cidr }}"
+      r_openshift_aws_vpc_subnets: "{{ provision.vpc.subnets }}"
+      r_openshift_aws_vpc_region: "{{ provision.region }}"
+      r_openshift_aws_vpc_tags: "{{ provision.vpc.tags }}"
+      r_openshift_aws_vpc_name: "{{ provision.vpc.name | default(provision.clusterid) }}"
+
+  - name: create aws ssh keypair
+    include_role:
+      name: openshift_aws_ssh_keys
+    vars:
+      r_openshift_aws_ssh_keys_users: "{{ provision.instance_users }}"
+      r_openshift_aws_ssh_keys_region: "{{ provision.region }}"
+
+  - when: provision.openshift_registry_s3 | default(false)
+    name: create s3 bucket for registry
+    include_role:
+      name: openshift_aws_s3
+    vars:
+      r_openshift_aws_s3_clusterid: "{{ provision.clusterid }}-docker-registry"
+      r_openshift_aws_s3_region: "{{ provision.region }}"
+      r_openshift_aws_s3_mode: create
+
+  - name: include scale group creation for master
+    include: build_node_group.yml
+    vars:
+      openshift_build_node_type: master
+
+  - name: fetch new master instances
+    ec2_remote_facts:
+      region: "{{ provision.region }}"
+      filters:
+        "tag:clusterid": "{{ provision.clusterid }}"
+        "tag:host-type": master
+        instance-state-name: running
+    register: instancesout
+    retries: 20
+    delay: 3
+    until: instancesout.instances|length > 0
+
+  - name: bring iam_cert23 into scope
+    include_role:
+      name: lib_utils
+
+  - name: upload certificates to AWS IAM
+    iam_cert23:
+      state: present
+      name: "{{ provision.clusterid }}-master-external"
+      cert: "{{ provision.iam_cert_ca.cert_path }}"
+      key: "{{ provision.iam_cert_ca.key_path }}"
+      cert_chain: "{{ provision.iam_cert_ca.chain_path | default(omit) }}"
+    register: elb_cert_chain
+    failed_when:
+    - "'failed' in elb_cert_chain"
+    - elb_cert_chain.failed
+    - "'msg' in elb_cert_chain"
+    - "'already exists' not in elb_cert_chain.msg"
+    when: provision.iam_cert_ca is defined
+
+  - debug: var=elb_cert_chain
+
+  - name: create our master external and internal load balancers
+    include_role:
+      name: openshift_aws_elb
+    vars:
+      r_openshift_aws_elb_clusterid: "{{ provision.clusterid }}"
+      r_openshift_aws_elb_region: "{{ provision.region }}"
+      r_openshift_aws_elb_instance_filter:
+        "tag:clusterid": "{{ provision.clusterid }}"
+        "tag:host-type": master
+        instance-state-name: running
+      r_openshift_aws_elb_type: master
+      r_openshift_aws_elb_direction: "{{ elb_item }}"
+      r_openshift_aws_elb_idle_timout: 400
+      r_openshift_aws_elb_scheme: internet-facing
+      r_openshift_aws_elb_security_groups:
+      - "{{ provision.clusterid }}"
+      - "{{ provision.clusterid }}_master"
+      r_openshift_aws_elb_subnet_name: "{{ provision.vpc.subnets[provision.region][0].az }}"
+      r_openshift_aws_elb_name: "{{ provision.clusterid }}-master-{{ elb_item }}"
+      r_openshift_aws_elb_cert_arn: "{{ elb_cert_chain.arn }}"
+    with_items:
+    - internal
+    - external
+    loop_control:
+      loop_var: elb_item
+
+  - name: add new master to masters group
+    add_host:
+      groups: "masters,etcd,nodes"
+      name: "{{ item.public_ip_address }}"
+      hostname: "{{ provision.clusterid }}-master-{{ item.id[:-5] }}"
+    with_items: "{{ instancesout.instances }}"
+
+  - name: set facts for group normalization
+    set_fact:
+      cluster_id: "{{ provision.clusterid }}"
+      cluster_env: "{{ provision.node_group_config.tags.environment | default('dev') }}"
+
+  - name: wait for ssh to become available
+    wait_for:
+      port: 22
+      host: "{{ item.public_ip_address }}"
+      timeout: 300
+      search_regex: OpenSSH
+    with_items: "{{ instancesout.instances }}"
+
+
+- name: set the master facts for hostname to elb
+  hosts: masters
+  gather_facts: no
+  remote_user: root
+  tasks:
+  - name: include vars
+    include_vars: vars.yml
+
+  - name: fetch elbs
+    ec2_elb_facts:
+      region: "{{ provision.region }}"
+      names:
+      - "{{ item }}"
+    with_items:
+    - "{{ provision.clusterid }}-master-external"
+    - "{{ provision.clusterid }}-master-internal"
+    delegate_to: localhost
+    register: elbs
+
+  - debug: var=elbs
+
+  - name: set fact
+    set_fact:
+      openshift_master_cluster_hostname: "{{ elbs.results[1].elbs[0].dns_name }}"
+      osm_custom_cors_origins:
+      - "{{ elbs.results[1].elbs[0].dns_name }}"
+      - "console.{{ provision.clusterid }}.openshift.com"
+      - "api.{{ provision.clusterid }}.openshift.com"
+    with_items: "{{ groups['masters'] }}"
+
+- name: normalize groups
+  include: ../../byo/openshift-cluster/initialize_groups.yml
+
+- name: run the std_include
+  include: ../../common/openshift-cluster/std_include.yml
+
+- name: run the config
+  include: ../../common/openshift-cluster/config.yml

playbooks/aws/openshift-cluster/provision_nodes.yml

@@ -0,0 +1,47 @@
+---
+# Get bootstrap config token
+# bootstrap should be created on first master
+# need to fetch it and shove it into cloud data
+- name: create the node scale groups
+  hosts: localhost
+  connection: local
+  gather_facts: yes
+  tasks:
+  - name: get provisioning vars
+    include_vars: vars.yml
+
+  - name: fetch master instances
+    ec2_remote_facts:
+      region: "{{ provision.region }}"
+      filters:
+        "tag:clusterid": "{{ provision.clusterid }}"
+        "tag:host-type": master
+        instance-state-name: running
+    register: instancesout
+    retries: 20
+    delay: 3
+    until: instancesout.instances|length > 0
+
+  - name: slurp down the bootstrap.kubeconfig
+    slurp:
+      src: /etc/origin/master/bootstrap.kubeconfig
+    delegate_to: "{{ instancesout.instances[0].public_ip_address }}"
+    remote_user: root
+    register: bootstrap
+
+  - name: set_fact on localhost for kubeconfig
+    set_fact:
+      local_bootstrap: "{{ bootstrap }}"
+      launch_config_name:
+        infra: "infra-{{ ansible_date_time.epoch }}"
+        compute: "compute-{{ ansible_date_time.epoch }}"
+
+  - name: include build node group
+    include: build_node_group.yml
+    vars:
+      openshift_build_node_type: infra
+
+  - name: include build node group
+    include: build_node_group.yml
+    vars:
+      openshift_build_node_type: compute

playbooks/aws/openshift-cluster/vars.yml

@@ -31,3 +31,126 @@ deployment_vars:
   enterprise: "{{ deployment_rhel7_ent_base }}"
   openshift-enterprise: "{{ deployment_rhel7_ent_base }}"
   atomic-enterprise: "{{ deployment_rhel7_ent_base }}"
+
+clusterid: mycluster
+region: us-east-1
+
+provision:
+  clusterid: "{{ clusterid }}"
+  region: "{{ region }}"
+
+  build:  # build specific variables here
+    ami_name: "openshift-gi-"
+    base_image: ami-bdd5d6ab  # base image for AMI to build from
+    yum_repositories:  # this is an example repository but it requires sslclient info
+    - name: openshift-repo
+      file: openshift-repo
+      description: OpenShift Builds
+      baseurl: https://mirror.openshift.com/enterprise/online-int/latest/x86_64/os/
+      enabled: yes
+      gpgcheck: no
+      sslverify: no
+      sslclientcert: "/var/lib/yum/client-cert.pem"
+      sslclientkey: "/var/lib/yum/client-key.pem"
+      gpgkey: "https://mirror.ops.rhcloud.com/libra/keys/RPM-GPG-KEY-redhat-release https://mirror.ops.rhcloud.com/libra/keys/RPM-GPG-KEY-redhat-beta https://mirror.ops.rhcloud.com/libra/keys/RPM-GPG-KEY-redhat-openshifthosted"
+
+    # when creating an encrypted AMI please specify use_encryption
+    use_encryption: False
+
+    openshift_ami_tags:
+      bootstrap: "true"
+      openshift-created: "true"
+      clusterid: "{{ clusterid }}"
+
+  # Use s3 backed registry storage
+  openshift_registry_s3: True
+
+  # if using custom certificates these are required for the ELB
+  iam_cert_ca:
+    name: "{{ clusterid }}_openshift"
+    cert_path: '/path/to/wildcard.<clusterid>.example.com.crt'
+    key_path: '/path/to/wildcard.<clusterid>.example.com.key'
+    chain_path: '/path/to/cert.ca.crt'
+
+  instance_users:
+  - key_name: myuser_key
+    username: myuser
+    pub_key: |
+           ssh-rsa AAAA== myuser@system
+
+  node_group_config:
+    tags:
+      clusterid: "{{ clusterid }}"
+      environment: stg
+
+    ssh_key_name: myuser_key
+
+    # master specific cluster node settings
+    master:
+      instance_type: m4.xlarge
+      ami: ami-cdeec8b6  # if using an encrypted AMI this will be replaced
+      volumes:
+      - device_name: /dev/sdb
+        volume_size: 100
+        device_type: gp2
+        delete_on_termination: False
+      health_check:
+        period: 60
+        type: EC2
+      min_size: 3
+      max_size: 3
+      desired_size: 3
+      tags:
+        host-type: master
+        sub-host-type: default
+      wait_for_instances: True
+
+    # compute specific cluster node settings
+    compute:
+      instance_type: m4.xlarge
+      ami: ami-cdeec8b6
+      volumes:
+      - device_name: /dev/sdb
+        volume_size: 100
+        device_type: gp2
+        delete_on_termination: True
+      health_check:
+        period: 60
+        type: EC2
+      min_size: 3
+      max_size: 100
+      desired_size: 3
+      tags:
+        host-type: node
+        sub-host-type: compute
+
+    # infra specific cluster node settings
+    infra:
+      instance_type: m4.xlarge
+      ami: ami-cdeec8b6
+      volumes:
+      - device_name: /dev/sdb
+        volume_size: 100
+        device_type: gp2
+        delete_on_termination: True
+      health_check:
+        period: 60
+        type: EC2
+      min_size: 2
+      max_size: 20
+      desired_size: 2
+      tags:
+        host-type: node
+        sub-host-type: infra
+
+  # vpc settings
+  vpc:
+    cidr: 172.31.0.0/16
+    subnets:
+      us-east-1:  # These are us-east-1 region defaults. Ensure this matches your region
+      - cidr: 172.31.48.0/20
+        az: "us-east-1c"
+      - cidr: 172.31.32.0/20
+        az: "us-east-1e"
+      - cidr: 172.31.16.0/20
+        az: "us-east-1a"

roles/lib_utils/library/iam_cert23.py

@@ -0,0 +1,314 @@
+#!/usr/bin/python
+# pylint: skip-file
+# flake8: noqa
+# This file is part of Ansible
+#
+# Ansible is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# Ansible is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Ansible.  If not, see <http://www.gnu.org/licenses/>.
+ANSIBLE_METADATA = {'metadata_version': '1.1',
+                    'status': ['preview'],
+                    'supported_by': 'community'}
+
+
+DOCUMENTATION = '''
+---
+module: iam_cert
+short_description: Manage server certificates for use on ELBs and CloudFront
+description:
+     - Allows for the management of server certificates
+version_added: "2.0"
+options:
+  name:
+    description:
+      - Name of certificate to add, update or remove.
+    required: true
+  new_name:
+    description:
+      - When state is present, this will update the name of the cert.
+      - The cert, key and cert_chain parameters will be ignored if this is defined.
+  new_path:
+    description:
+      - When state is present, this will update the path of the cert.
+      - The cert, key and cert_chain parameters will be ignored if this is defined.
+  state:
+    description:
+      - Whether to create(or update) or delete certificate.
+      - If new_path or new_name is defined, specifying present will attempt to make an update these.
+    required: true
+    choices: [ "present", "absent" ]
+  path:
+    description:
+      - When creating or updating, specify the desired path of the certificate.
+    default: "/"
+  cert_chain:
+    description:
+      - The path to, or content of the CA certificate chain in PEM encoded format.
+        As of 2.4 content is accepted. If the parameter is not a file, it is assumed to be content.
+  cert:
+    description:
+      - The path to, or content of the certificate body in PEM encoded format.
+        As of 2.4 content is accepted. If the parameter is not a file, it is assumed to be content.
+  key:
+    description:
+      - The path to, or content of the private key in PEM encoded format.
+        As of 2.4 content is accepted. If the parameter is not a file, it is assumed to be content.
+  dup_ok:
+    description:
+      - By default the module will not upload a certificate that is already uploaded into AWS.
+        If set to True, it will upload the certificate as long as the name is unique.
+    default: False
+
+
+requirements: [ "boto" ]
+author: Jonathan I. Davila
+extends_documentation_fragment:
+    - aws
+    - ec2
+'''
+
+EXAMPLES = '''
+# Basic server certificate upload from local file
+- iam_cert:
+    name: very_ssl
+    state: present
+    cert: "{{ lookup('file', 'path/to/cert') }}"
+    key: "{{ lookup('file', 'path/to/key') }}"
+    cert_chain: "{{ lookup('file', 'path/to/certchain') }}"
+
+# Basic server certificate upload
+- iam_cert:
+    name: very_ssl
+    state: present
+    cert: path/to/cert
+    key: path/to/key
+    cert_chain: path/to/certchain
+
+# Server certificate upload using key string
+- iam_cert:
+    name: very_ssl
+    state: present
+    path: "/a/cert/path/"
+    cert: body_of_somecert
+    key: vault_body_of_privcertkey
+    cert_chain: body_of_myverytrustedchain
+
+# Basic rename of existing certificate
+- iam_cert:
+    name: very_ssl
+    new_name: new_very_ssl
+    state: present
+
+'''
+from ansible.module_utils.basic import AnsibleModule
+from ansible.module_utils.ec2 import ec2_argument_spec, get_aws_connection_info, connect_to_aws
+import os
+
+try:
+    import boto
+    import boto.iam
+    import boto.ec2
+    HAS_BOTO = True
+except ImportError:
+    HAS_BOTO = False
+
+
+def boto_exception(err):
+    '''generic error message handler'''
+    if hasattr(err, 'error_message'):
+        error = err.error_message
+    elif hasattr(err, 'message'):
+        error = err.message
+    else:
+        error = '%s: %s' % (Exception, err)
+
+    return error
+
+
+def cert_meta(iam, name):
+    certificate = iam.get_server_certificate(name).get_server_certificate_result.server_certificate
+    ocert = certificate.certificate_body
+    opath = certificate.server_certificate_metadata.path
+    ocert_id = certificate.server_certificate_metadata.server_certificate_id
+    upload_date = certificate.server_certificate_metadata.upload_date
+    exp = certificate.server_certificate_metadata.expiration
+    arn = certificate.server_certificate_metadata.arn
+    return opath, ocert, ocert_id, upload_date, exp, arn
+
+
+def dup_check(module, iam, name, new_name, cert, orig_cert_names, orig_cert_bodies, dup_ok):
+    update = False
+
+    # IAM cert names are case insensitive
+    names_lower = [n.lower() for n in [name, new_name] if n is not None]
+    orig_cert_names_lower = [ocn.lower() for ocn in orig_cert_names]
+
+    if any(ct in orig_cert_names_lower for ct in names_lower):
+        for i_name in names_lower:
+            if cert is not None:
+                try:
+                    c_index = orig_cert_names_lower.index(i_name)
+                except NameError:
+                    continue
+                else:
+                    # NOTE: remove the carriage return to strictly compare the cert bodies.
+                    slug_cert = cert.replace('\r', '')
+                    slug_orig_cert_bodies = orig_cert_bodies[c_index].replace('\r', '')
+                    if slug_orig_cert_bodies == slug_cert:
+                        update = True
+                        break
+                    elif slug_cert.startswith(slug_orig_cert_bodies):
+                        update = True
+                        break
+                    elif slug_orig_cert_bodies != slug_cert:
+                        module.fail_json(changed=False, msg='A cert with the name %s already exists and'
+                                         ' has a different certificate body associated'
+                                         ' with it. Certificates cannot have the same name' % orig_cert_names[c_index])
+            else:
+                update = True
+                break
+    elif cert in orig_cert_bodies and not dup_ok:
+        for crt_name, crt_body in zip(orig_cert_names, orig_cert_bodies):
+            if crt_body == cert:
+                module.fail_json(changed=False, msg='This certificate already'
+                                                    ' exists under the name %s' % crt_name)
+
+    return update
+
+
+def cert_action(module, iam, name, cpath, new_name, new_path, state,
+                cert, key, cert_chain, orig_cert_names, orig_cert_bodies, dup_ok):
+    if state == 'present':
+        update = dup_check(module, iam, name, new_name, cert, orig_cert_names,
+                           orig_cert_bodies, dup_ok)
+        if update:
+            opath, ocert, ocert_id, upload_date, exp, arn = cert_meta(iam, name)
+            changed = True
+            if new_name and new_path:
+                iam.update_server_cert(name, new_cert_name=new_name, new_path=new_path)
+                module.exit_json(changed=changed, original_name=name, new_name=new_name,
+                                 original_path=opath, new_path=new_path, cert_body=ocert,
+                                 upload_date=upload_date, expiration_date=exp, arn=arn)
+            elif new_name and not new_path:
+                iam.update_server_cert(name, new_cert_name=new_name)
+                module.exit_json(changed=changed, original_name=name, new_name=new_name,
+                                 cert_path=opath, cert_body=ocert,
+                                 upload_date=upload_date, expiration_date=exp, arn=arn)
+            elif not new_name and new_path:
+                iam.update_server_cert(name, new_path=new_path)
+                module.exit_json(changed=changed, name=new_name,
+                                 original_path=opath, new_path=new_path, cert_body=ocert,
+                                 upload_date=upload_date, expiration_date=exp, arn=arn)
+            else:
+                changed = False
+                module.exit_json(changed=changed, name=name, cert_path=opath, cert_body=ocert,
+                                 upload_date=upload_date, expiration_date=exp, arn=arn,
+                                 msg='No new path or name specified. No changes made')
+        else:
+            changed = True
+            iam.upload_server_cert(name, cert, key, cert_chain=cert_chain, path=cpath)
+            opath, ocert, ocert_id, upload_date, exp, arn = cert_meta(iam, name)
+            module.exit_json(changed=changed, name=name, cert_path=opath, cert_body=ocert,
+                             upload_date=upload_date, expiration_date=exp, arn=arn)
+    elif state == 'absent':
+        if name in orig_cert_names:
+            changed = True
+            iam.delete_server_cert(name)
+            module.exit_json(changed=changed, deleted_cert=name)
+        else:
+            changed = False
+            module.exit_json(changed=changed, msg='Certificate with the name %s already absent' % name)
+
+
+def load_data(cert, key, cert_chain):
+    # if paths are provided rather than lookups read the files and return the contents
+    if cert and os.path.isfile(cert):
+        cert = open(cert, 'r').read().rstrip()
+    if key and os.path.isfile(key):
+        key = open(key, 'r').read().rstrip()
+    if cert_chain and os.path.isfile(cert_chain):
+        cert_chain = open(cert_chain, 'r').read()
+    return cert, key, cert_chain
+
+
+def main():
+    argument_spec = ec2_argument_spec()
+    argument_spec.update(dict(
+        state=dict(required=True, choices=['present', 'absent']),
+        name=dict(),
+        cert=dict(),
+        key=dict(no_log=True),
+        cert_chain=dict(),
+        new_name=dict(),
+        path=dict(default='/'),
+        new_path=dict(),
+        dup_ok=dict(type='bool')
+    )
+    )
+
+    module = AnsibleModule(
+        argument_spec=argument_spec,
+        mutually_exclusive=[
+            ['new_path', 'key'],
+            ['new_path', 'cert'],
+            ['new_path', 'cert_chain'],
+            ['new_name', 'key'],
+            ['new_name', 'cert'],
+            ['new_name', 'cert_chain'],
+        ],
+    )
+
+    if not HAS_BOTO:
+        module.fail_json(msg="Boto is required for this module")
+
+    region, ec2_url, aws_connect_kwargs = get_aws_connection_info(module)
+
+    try:
+        if region:
+            iam = connect_to_aws(boto.iam, region, **aws_connect_kwargs)
+        else:
+            iam = boto.iam.connection.IAMConnection(**aws_connect_kwargs)
+    except boto.exception.NoAuthHandlerFound as e:
+        module.fail_json(msg=str(e))
+
+    state = module.params.get('state')
+    name = module.params.get('name')
+    path = module.params.get('path')
+    new_name = module.params.get('new_name')
+    new_path = module.params.get('new_path')
+    dup_ok = module.params.get('dup_ok')
+    if state == 'present' and not new_name and not new_path:
+        cert, key, cert_chain = load_data(cert=module.params.get('cert'),
+                                          key=module.params.get('key'),
+                                          cert_chain=module.params.get('cert_chain'))
+    else:
+        cert = key = cert_chain = None
+
+    orig_cert_names = [ctb['server_certificate_name'] for ctb in
+                       iam.get_all_server_certs().list_server_certificates_result.server_certificate_metadata_list]
+    orig_cert_bodies = [iam.get_server_certificate(thing).get_server_certificate_result.certificate_body
+                        for thing in orig_cert_names]
+    if new_name == name:
+        new_name = None
+    if new_path == path:
+        new_path = None
+
+    changed = False
+    try:
+        cert_action(module, iam, name, path, new_name, new_path, state,
+                    cert, key, cert_chain, orig_cert_names, orig_cert_bodies, dup_ok)
+    except boto.exception.BotoServerError as err:
+        module.fail_json(changed=changed, msg=str(err), debug=[cert, key])
+
+
+if __name__ == '__main__':
+    main()

roles/lib_utils/library/oo_iam_kms.py

@@ -0,0 +1,172 @@
+#!/usr/bin/env python
+'''
+ansible module for creating AWS IAM KMS keys
+'''
+# vim: expandtab:tabstop=4:shiftwidth=4
+#
+#   AWS IAM KMS ansible module
+#
+#
+#   Copyright 2016 Red Hat Inc.
+#
+#   Licensed under the Apache License, Version 2.0 (the "License");
+#   you may not use this file except in compliance with the License.
+#   You may obtain a copy of the License at
+#
+#       http://www.apache.org/licenses/LICENSE-2.0
+#
+#   Unless required by applicable law or agreed to in writing, software
+#   distributed under the License is distributed on an "AS IS" BASIS,
+#   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+#   See the License for the specific language governing permissions and
+#   limitations under the License.
+#
+# Jenkins environment doesn't have all the required libraries
+# pylint: disable=import-error
+import time
+import boto3
+# Ansible modules need this wildcard import
+# pylint: disable=unused-wildcard-import, wildcard-import, redefined-builtin
+from ansible.module_utils.basic import AnsibleModule
+
+AWS_ALIAS_URL = "http://docs.aws.amazon.com/kms/latest/developerguide/programming-aliases.html"
+
+
+class AwsIamKms(object):
+    '''
+    ansible module for AWS IAM KMS
+    '''
+
+    def __init__(self):
+        ''' constructor '''
+        self.module = None
+        self.kms_client = None
+        self.aliases = None
+
+    @staticmethod
+    def valid_alias_name(user_alias):
+        ''' AWS KMS aliases must start with 'alias/' '''
+        valid_start = 'alias/'
+        if user_alias.startswith(valid_start):
+            return True
+
+        return False
+
+    def get_all_kms_info(self):
+        '''fetch all kms info and return them
+
+        list_keys doesn't have information regarding aliases
+        list_aliases doesn't have the full kms arn
+
+        fetch both and join them on the targetKeyId
+        '''
+        aliases = self.kms_client.list_aliases()['Aliases']
+        keys = self.kms_client.list_keys()['Keys']
+
+        for alias in aliases:
+            for key in keys:
+                if 'TargetKeyId' in alias and 'KeyId' in key:
+                    if alias['TargetKeyId'] == key['KeyId']:
+                        alias.update(key)
+
+        return aliases
+
+    def get_kms_entry(self, user_alias, alias_list):
+        ''' return single alias details from list of aliases '''
+        for alias in alias_list:
+            if user_alias == alias.get('AliasName', False):
+                return alias
+
+        msg = "Did not find alias {}".format(user_alias)
+        self.module.exit_json(failed=True, results=msg)
+
+    @staticmethod
+    def exists(user_alias, alias_list):
+        ''' Check if KMS alias already exists '''
+        for alias in alias_list:
+            if user_alias == alias.get('AliasName'):
+                return True
+
+        return False
+
+    def main(self):
+        ''' entry point for module '''
+
+        self.module = AnsibleModule(
+            argument_spec=dict(
+                state=dict(default='list', choices=['list', 'present'], type='str'),
+                region=dict(default=None, required=True, type='str'),
+                alias=dict(default=None, type='str'),
+                # description default cannot be None
+                description=dict(default='', type='str'),
+                aws_access_key=dict(default=None, type='str'),
+                aws_secret_key=dict(default=None, type='str'),
+            ),
+        )
+
+        state = self.module.params['state']
+        aws_access_key = self.module.params['aws_access_key']
+        aws_secret_key = self.module.params['aws_secret_key']
+        if aws_access_key and aws_secret_key:
+            boto3.setup_default_session(aws_access_key_id=aws_access_key,
+                                        aws_secret_access_key=aws_secret_key,
+                                        region_name=self.module.params['region'])
+        else:
+            boto3.setup_default_session(region_name=self.module.params['region'])
+
+        self.kms_client = boto3.client('kms')
+
+        aliases = self.get_all_kms_info()
+
+        if state == 'list':
+            if self.module.params['alias'] is not None:
+                user_kms = self.get_kms_entry(self.module.params['alias'],
+                                              aliases)
+                self.module.exit_json(changed=False, results=user_kms,
+                                      state="list")
+            else:
+                self.module.exit_json(changed=False, results=aliases,
+                                      state="list")
+
+        if state == 'present':
+
+            # early sanity check to make sure the alias name conforms with
+            # AWS alias name requirements
+            if not self.valid_alias_name(self.module.params['alias']):
+                self.module.exit_json(failed=True, changed=False,
+                                      results="Alias must start with the prefix " +
+                                      "'alias/'. Please see " + AWS_ALIAS_URL,
+                                      state='present')
+
+            if not self.exists(self.module.params['alias'], aliases):
+                # if we didn't find it, create it
+                response = self.kms_client.create_key(KeyUsage='ENCRYPT_DECRYPT',
+                                                      Description=self.module.params['description'])
+                kid = response['KeyMetadata']['KeyId']
+                response = self.kms_client.create_alias(AliasName=self.module.params['alias'],
+                                                        TargetKeyId=kid)
+                # sleep for a bit so that the KMS data can be queried
+                time.sleep(10)
+                # get details for newly created KMS entry
+                new_alias_list = self.kms_client.list_aliases()['Aliases']
+                user_kms = self.get_kms_entry(self.module.params['alias'],
+                                              new_alias_list)
+
+                self.module.exit_json(changed=True, results=user_kms,
+                                      state='present')
+
+            # already exists, normally we would check whether we need to update it
+            # but this module isn't written to allow changing the alias name
+            # or changing whether the key is enabled/disabled
+            user_kms = self.get_kms_entry(self.module.params['alias'], aliases)
+            self.module.exit_json(changed=False, results=user_kms,
+                                  state="present")
+
+        self.module.exit_json(failed=True,
+                              changed=False,
+                              results='Unknown state passed. %s' % state,
+                              state="unknown")
+
+
+if __name__ == '__main__':
+    AwsIamKms().main()

roles/openshift_ami_prep/defaults/main.yml

@@ -0,0 +1,50 @@
+---
+
+
+r_openshift_ami_prep_packages:
+- atomic-openshift-master
+- atomic-openshift-node
+- atomic-openshift-docker-excluder
+- atomic-openshift-sdn-ovs
+- openvswitch
+- docker
+- etcd
+#- pcs
+- haproxy
+- dnsmasq
+- ntp
+- logrotate
+- httpd-tools
+- bind
+- firewalld
+- libselinux-python
+- conntrack-tools
+- openssl
+- cloud-init
+- iproute
+- python-dbus
+- PyYAML
+- yum-utils
+- python2-boto
+- python2-boto3
+- cloud-utils-growpart
+# gluster
+- glusterfs-fuse
+- heketi-client
+# nfs
+- nfs-utils
+- flannel
+- bash-completion
+# cockpit
+- cockpit-ws
+- cockpit-system
+- cockpit-bridge
+- cockpit-docker
+# iscsi
+- iscsi-initiator-utils
+# ceph
+- ceph-common
+# systemcontainer
+# - runc
+# - container-selinux
+# - atomic

roles/openshift_ami_prep/tasks/main.yml

@@ -0,0 +1,42 @@
+---
+- name: install repositories
+  include: yum_repos.yml
+  static: yes
+
+- name: install needed rpm(s)
+  package:
+    name: "{{ item }}"
+    state: present
+  with_items: "{{ r_openshift_ami_prep_packages }}"
+
+- name: create the directory for node
+  file:
+    state: directory
+    path: "/etc/systemd/system/{{ r_openshift_ami_prep_node }}.service.d"
+
+- name: laydown systemd override
+  copy:
+    dest: "/etc/systemd/system/{{ r_openshift_ami_prep_node }}.service.d/override.conf"
+    content: |
+      [Unit]
+      After=cloud-init.service
+
+- name: update the sysconfig to have KUBECONFIG
+  lineinfile:
+    dest: "/etc/sysconfig/{{ r_openshift_ami_prep_node }}"
+    line: "KUBECONFIG=/root/csr_kubeconfig"
+    regexp: "^KUBECONFIG=.*"
+
+- name: update the ExecStart to have bootstrap
+  lineinfile:
+    dest: "/usr/lib/systemd/system/{{ r_openshift_ami_prep_node }}.service"
+    line: "{% raw %}ExecStart=/usr/bin/openshift start node --bootstrap --kubeconfig=${KUBECONFIG} $OPTIONS{% endraw %}"
+    regexp: "^ExecStart=.*"
+
+- name: systemctl enable origin-node
+  systemd:
+    name: "{{ item }}"
+    enabled: no
+  with_items:
+  - "{{ r_openshift_ami_prep_node }}.service"
+  - "{{ r_openshift_ami_prep_master }}.service"

roles/openshift_ami_prep/tasks/yum_repos.yml

@@ -0,0 +1,14 @@
+---
+- name: Create our install repository
+  yum_repository:
+    description: "{{ item.description | default(omit) }}"
+    name: "{{ item.name }}"
+    baseurl: "{{ item.baseurl }}"
+    gpgkey: "{{ item.gpgkey | default(omit)}}"
+    gpgcheck: "{{ item.gpgcheck | default(1) }}"
+    sslverify: "{{ item.sslverify | default(1) }}"
+    sslclientkey: "{{ item.sslclientkey | default(omit) }}"
+    sslclientcert: "{{ item.sslclientcert | default(omit) }}"
+    file: "{{ item.file }}"
+    enabled: "{{ item.enabled }}"
+  with_items: "{{ r_openshift_ami_prep_yum_repositories }}"

roles/openshift_aws_ami_copy/README.md

@@ -0,0 +1,50 @@
+openshift_aws_ami_perms
+=========
+
+Ansible role for copying an AMI
+
+Requirements
+------------
+
+Ansible Modules:
+
+
+Role Variables
+--------------
+
+- openshift_aws_ami_copy_src_ami: source AMI id to copy from
+- openshift_aws_ami_copy_region: region where the AMI is found
+- openshift_aws_ami_copy_name: name to assign to new AMI
+- openshift_aws_ami_copy_kms_arn: AWS IAM KMS arn of the key to use for encryption
+- openshift_aws_ami_copy_tags: dict with desired tags
+- openshift_aws_ami_copy_wait: wait for the ami copy to achieve available status.  This fails due to boto waiters.
+
+Dependencies
+------------
+
+
+Example Playbook
+----------------
+```yaml
+    - name: copy the ami for encrypted disks
+      include_role:
+        name: openshift_aws_ami_copy
+      vars:
+        r_openshift_aws_ami_copy_region: us-east-1
+        r_openshift_aws_ami_copy_name: myami
+        r_openshift_aws_ami_copy_src_ami: ami-1234
+        r_openshift_aws_ami_copy_kms_arn: arn:xxxx
+        r_openshift_aws_ami_copy_tags: {}
+        r_openshift_aws_ami_copy_encrypt: False
+
+```
+
+License
+-------
+
+Apache 2.0
+
+Author Information
+------------------
+
+Openshift

roles/openshift_aws_ami_copy/tasks/main.yml

@@ -0,0 +1,26 @@
+---
+- fail:
+    msg: "{{ item }} needs to be defined"
+  when: "{{ item }} is not defined"
+  with_items:
+  - r_openshift_aws_ami_copy_src_ami
+  - r_openshift_aws_ami_copy_name
+  - r_openshift_aws_ami_copy_region
+
+- name: "Create copied AMI image and wait: {{ r_openshift_aws_ami_copy_wait | default(False) }}"
+  ec2_ami_copy:
+    region: "{{ r_openshift_aws_ami_copy_region }}"
+    source_region: "{{ r_openshift_aws_ami_copy_region }}"
+    name: "{{ r_openshift_aws_ami_copy_name }}"
+    source_image_id: "{{ r_openshift_aws_ami_copy_src_ami }}"
+    encrypted: "{{ r_openshift_aws_ami_copy_encrypt | default(False) }}"
+    kms_key_id: "{{ r_openshift_aws_ami_copy_kms_arn | default(omit) }}"
+    wait: "{{ r_openshift_aws_ami_copy_wait | default(omit) }}"
+    tags: "{{ r_openshift_aws_ami_copy_tags }}"
+  register: copy_result
+
+- debug: var=copy_result
+
+- name: return AMI ID with setfact - openshift_aws_ami_copy_retval_custom_ami
+  set_fact:
+    r_openshift_aws_ami_copy_retval_custom_ami: "{{ copy_result.image_id }}"

roles/openshift_aws_elb/README.md

@@ -0,0 +1,75 @@
+openshift_aws_elb
+=========
+
+Ansible role to provision and manage AWS ELB's for Openshift.
+
+Requirements
+------------
+
+Ansible Modules:
+
+- ec2_elb
+- ec2_elb_lb
+
+python package:
+
+python-boto
+
+Role Variables
+--------------
+
+- r_openshift_aws_elb_instances: instances to put in ELB
+- r_openshift_aws_elb_elb_name: name of elb
+- r_openshift_aws_elb_security_group_names: list of SGs (by name) that the ELB will belong to
+- r_openshift_aws_elb_region: AWS Region
+- r_openshift_aws_elb_health_check: definition of the ELB health check. See ansible docs for ec2_elb
+```yaml
+  ping_protocol: tcp
+  ping_port: 443
+  response_timeout: 5
+  interval: 30
+  unhealthy_threshold: 2
+  healthy_threshold: 2
+```
+- r_openshift_aws_elb_listeners: definition of the ELB listeners. See ansible docs for ec2_elb
+```yaml
+- protocol: tcp
+  load_balancer_port: 80
+  instance_protocol: ssl
+  instance_port: 443
+- protocol: ssl
+  load_balancer_port: 443
+  instance_protocol: ssl
+  instance_port: 443
+  # ssl certificate required for https or ssl
+  ssl_certificate_id: "{{ r_openshift_aws_elb_cert_arn }}"
+```
+
+Dependencies
+------------
+
+
+Example Playbook
+----------------
+```yaml
+- include_role:
+    name: openshift_aws_elb
+  vars:
+    r_openshift_aws_elb_instances: aws_instances_to_put_in_elb
+    r_openshift_aws_elb_elb_name: elb_name
+    r_openshift_aws_elb_security_groups: security_group_names
+    r_openshift_aws_elb_region: aws_region
+    r_openshift_aws_elb_health_check: "{{ elb_health_check_definition }}"
+    r_openshift_aws_elb_listeners: "{{ elb_listeners_definition }}"
+```
+
+
+License
+-------
+
+Apache 2.0
+
+Author Information
+------------------
+
+Openshift

roles/openshift_aws_elb/defaults/main.yml

@@ -0,0 +1,33 @@
+---
+r_openshift_aws_elb_health_check:
+  ping_protocol: tcp
+  ping_port: 443
+  response_timeout: 5
+  interval: 30
+  unhealthy_threshold: 2
+  healthy_threshold: 2
+
+r_openshift_aws_elb_cert_arn: ''
+
+r_openshift_aws_elb_listeners:
+  master:
+    external:
+    - protocol: tcp
+      load_balancer_port: 80
+      instance_protocol: ssl
+      instance_port: 443
+    - protocol: ssl
+      load_balancer_port: 443
+      instance_protocol: ssl
+      instance_port: 443
+      # ssl certificate required for https or ssl
+      ssl_certificate_id: "{{ r_openshift_aws_elb_cert_arn }}"
+    internal:
+    - protocol: tcp
+      load_balancer_port: 80
+      instance_protocol: tcp
+      instance_port: 80
+    - protocol: tcp
+      load_balancer_port: 443
+      instance_protocol: tcp
+      instance_port: 443

roles/openshift_aws_elb/meta/main.yml

@@ -0,0 +1,12 @@
+---
+galaxy_info:
+  author: OpenShift
+  description: Openshift ELB provisioning
+  company: Red Hat, Inc
+  license: ASL 2.0
+  min_ansible_version: 1.2
+  platforms:
+  - name: EL
+    versions:
+    - 7
+dependencies: []

roles/openshift_aws_elb/tasks/main.yml

@@ -0,0 +1,57 @@
+---
+- name: fetch the default subnet id
+  ec2_remote_facts:
+    region: "{{ r_openshift_aws_elb_region }}"
+    filters: "{{ r_openshift_aws_elb_instance_filter }}"
+  register: instancesout
+
+- name: fetch the default subnet id
+  ec2_vpc_subnet_facts:
+    region: "{{ r_openshift_aws_elb_region }}"
+    filters:
+      "tag:Name": "{{ r_openshift_aws_elb_subnet_name }}"
+  register: subnetout
+
+- name:
+  debug:
+    msg: "{{ r_openshift_aws_elb_listeners[r_openshift_aws_elb_type][r_openshift_aws_elb_direction]
+                   if 'master' in r_openshift_aws_elb_type  or 'infra' in r_openshift_aws_elb_type
+                   else r_openshift_aws_elb_listeners }}"
+
+- name: "Create ELB {{ r_openshift_aws_elb_name }}"
+  ec2_elb_lb:
+    name: "{{ r_openshift_aws_elb_name }}"
+    state: present
+    security_group_names: "{{ r_openshift_aws_elb_security_groups }}"
+    idle_timeout: "{{ r_openshift_aws_elb_idle_timout }}"
+    region: "{{ r_openshift_aws_elb_region }}"
+    subnets:
+    - "{{ subnetout.subnets[0].id }}"
+    health_check: "{{ r_openshift_aws_elb_health_check }}"
+    listeners: "{{ r_openshift_aws_elb_listeners[r_openshift_aws_elb_type][r_openshift_aws_elb_direction]
+                   if 'master' in r_openshift_aws_elb_type  or 'infra' in r_openshift_aws_elb_type
+                   else r_openshift_aws_elb_listeners }}"
+    scheme: "{{ r_openshift_aws_elb_scheme }}"
+    tags:
+      KubernetesCluster: "{{ r_openshift_aws_elb_clusterid }}"
+  register: new_elb
+
+# It is necessary to ignore_errors here because the instances are not in 'ready'
+#  state when first added to ELB
+- name: "Add instances to ELB {{ r_openshift_aws_elb_name }}"
+  ec2_elb:
+    instance_id: "{{ item.id }}"
+    ec2_elbs: "{{ r_openshift_aws_elb_name }}"
+    state: present
+    region: "{{ r_openshift_aws_elb_region }}"
+    wait: False
+  with_items: "{{ instancesout.instances }}"
+  ignore_errors: True
+  retries: 10
+  register: elb_call
+  until: elb_call|succeeded
+
+- debug:
+    msg: "{{ item }}"
+  with_items:
+  - "{{ new_elb }}"

roles/openshift_aws_iam_kms/README.md

@@ -0,0 +1,43 @@
+openshift_aws_iam_kms
+=========
+
+Ansible role to create AWS IAM KMS keys for encryption
+
+Requirements
+------------
+
+Ansible Modules:
+
+oo_iam_kms
+
+Role Variables
+--------------
+
+- r_openshift_aws_iam_kms_region: AWS region to create KMS key
+- r_openshift_aws_iam_kms_alias: Alias name to assign to created KMS key
+
+Dependencies
+------------
+
+lib_utils
+
+Example Playbook
+----------------
+```yaml
+- include_role:
+    name: openshift_aws_iam_kms
+  vars:
+    r_openshift_aws_iam_kms_region: 'us-east-1'
+    r_openshift_aws_iam_kms_alias: 'alias/clusterABC_kms'
+```
+
+
+License
+-------
+
+Apache 2.0
+
+Author Information
+------------------
+
+Openshift

roles/openshift_aws_iam_kms/meta/main.yml

@@ -0,0 +1,13 @@
+---
+galaxy_info:
+  author: OpenShift
+  description: AWS IAM KMS setup and management
+  company: Red Hat, Inc
+  license: ASL 2.0
+  min_ansible_version: 1.2
+  platforms:
+  - name: EL
+    versions:
+    - 7
+dependencies:
+- lib_utils

roles/openshift_aws_iam_kms/tasks/main.yml

@@ -0,0 +1,18 @@
+---
+- fail:
+    msg: "{{ item.name }} needs to be defined."
+  when: "{{ item.cond }}"
+  with_items:
+  - name: "{{ r_openshift_aws_iam_kms_alias }}"
+    cond: "{{ r_openshift_aws_iam_kms_alias is undefined }}"
+  - name: "{{ r_openshift_aws_iam_kms_region }}"
+    cond: "{{ r_openshift_aws_iam_kms_region is undefined }}"
+
+- name: Create IAM KMS key with alias
+  oo_iam_kms:
+    state: present
+    alias: "{{ r_openshift_aws_iam_kms_alias }}"
+    region: "{{ r_openshift_aws_iam_kms_region }}"
+  register: created_kms
+
+- debug: var=created_kms.results

roles/openshift_aws_launch_config/README.md

@@ -0,0 +1,72 @@
+openshift_aws_launch_config
+=========
+
+Ansible role to create an AWS launch config for a scale group.
+
+This includes the AMI, volumes, user_data, etc.
+
+Requirements
+------------
+
+Ansible Modules:
+
+
+Role Variables
+--------------
+- r_openshift_aws_launch_config_name: "{{ launch_config_name }}"
+- r_openshift_aws_launch_config_clusterid: "{{ clusterid }}"
+- r_openshift_aws_launch_config_region: "{{ region }}"
+- r_openshift_aws_launch_config: "{{ node_group_config }}"
+```yaml
+    master:
+      instance_type: m4.xlarge
+      ami: ami-cdeec8b6  # if using an encrypted AMI this will be replaced
+      volumes:
+      - device_name: /dev/sdb
+        volume_size: 100
+        device_type: gp2
+        delete_on_termination: False
+      health_check:
+        period: 60
+        type: EC2
+      min_size: 3
+      max_size: 3
+      desired_size: 3
+      tags:
+        host-type: master
+        sub-host-type: default
+      wait_for_instances: True
+```
+- r_openshift_aws_launch_config_type: compute
+- r_openshift_aws_launch_config_custom_image: ami-xxxxx
+- r_openshift_aws_launch_config_bootstrap_token: <string of kubeconfig>
+
+Dependencies
+------------
+
+
+Example Playbook
+----------------
+```yaml
+  - name: create compute nodes config
+    include_role:
+      name: openshift_aws_launch_config
+    vars:
+      r_openshift_aws_launch_config_name: "{{ launch_config_name }}"
+      r_openshift_aws_launch_config_clusterid: "{{ clusterid }}"
+      r_openshift_aws_launch_config_region: "{{ region }}"
+      r_openshift_aws_launch_config: "{{ node_group_config }}"
+      r_openshift_aws_launch_config_type: compute
+      r_openshift_aws_launch_config_custom_image: ami-1234
+      r_openshift_aws_launch_config_bootstrap_token: abcd
+```
+
+License
+-------
+
+Apache 2.0
+
+Author Information
+------------------
+
+Openshift

roles/openshift_aws_launch_config/defaults/main.yml

@@ -0,0 +1 @@
+---

roles/openshift_aws_launch_config/meta/main.yml

@@ -0,0 +1,12 @@
+---
+galaxy_info:
+  author: OpenShift
+  description: Openshift AWS VPC creation
+  company: Red Hat, Inc
+  license: ASL 2.0
+  min_ansible_version: 2.3
+  platforms:
+  - name: EL
+    versions:
+    - 7
+dependencies: []

roles/openshift_aws_launch_config/tasks/main.yml

@@ -0,0 +1,50 @@
+---
+- name: fail when params are not set
+  fail:
+    msg: Please specify the role parameters.
+  when:
+  - r_openshift_aws_launch_config_cluseterid is undefined
+  - r_openshift_aws_launch_config_type is undefined
+  - r_openshift_aws_launch_config_region is undefined
+  - r_openshift_aws_launch_config is undefined
+
+- name: fetch the security groups for launch config
+  ec2_group_facts:
+    filters:
+      group-name:
+      - "{{ r_openshift_aws_launch_config_clusterid }}"  # default sg
+      - "{{ r_openshift_aws_launch_config_clusterid }}_{{ r_openshift_aws_launch_config_type }}"  # node type sg
+      - "{{ r_openshift_aws_launch_config_clusterid }}_{{ r_openshift_aws_launch_config_type }}_k8s"  # node type sg k8s
+    region: "{{ r_openshift_aws_launch_config_region }}"
+  register: ec2sgs
+
+# Create the scale group config
+- name: Create the node scale group config
+  ec2_lc:
+    name: "{{ r_openshift_aws_launch_config_name }}"
+    region: "{{ r_openshift_aws_launch_config_region }}"
+    image_id: "{{ r_openshift_aws_launch_config_custom_image if 'ami-' in r_openshift_aws_launch_config_custom_image else r_openshift_aws_launch_config[r_openshift_aws_launch_config_type].ami }}"
+    instance_type: "{{ r_openshift_aws_launch_config[r_openshift_aws_launch_config_type].instance_type }}"
+    security_groups: "{{ ec2sgs.security_groups | map(attribute='group_id')| list }}"
+    user_data: |-
+      #cloud-config
+      {%  if r_openshift_aws_launch_config_type != 'master' %}
+      write_files:
+      - path: /root/csr_kubeconfig
+        owner: root:root
+        permissions: '0640'
+        content: {{ r_openshift_aws_launch_config_bootstrap_token | default('') | to_yaml }}
+      - path: /root/openshift_settings
+        owner: root:root
+        permissions: '0640'
+        content:
+          openshift_type: "{{ r_openshift_aws_launch_config_type }}"
+      runcmd:
+      - [ systemctl, enable, atomic-openshift-node]
+      - [ systemctl, start, atomic-openshift-node]
+      {% endif %}
+    key_name: "{{ r_openshift_aws_launch_config.ssh_key_name }}"
+    ebs_optimized: False
+    volumes: "{{ r_openshift_aws_launch_config[r_openshift_aws_launch_config_type].volumes }}"
+    assign_public_ip: True
+  register: test

roles/openshift_aws_launch_config/templates/cloud-init.j2

@@ -0,0 +1,9 @@
+{% if r_openshift_aws_launch_config_bootstrap_token is defined and r_openshift_aws_launch_config_bootstrap_token is not '' %}
+#cloud-config
+write_files:
+- path: /root/csr_kubeconfig
+  owner: root:root
+  permissions: '0640'
+  content: |-
+  {{ r_openshift_aws_launch_config_bootstrap_token }}
+{% endif %}

roles/openshift_aws_node_group/README.md

@@ -0,0 +1,77 @@
+openshift_aws_node_group
+=========
+
+Ansible role to create an aws node group.
+
+This includes the security group, launch config, and scale group.
+
+Requirements
+------------
+
+Ansible Modules:
+
+
+Role Variables
+--------------
+```yaml
+- r_openshift_aws_node_group_name: myscalegroup
+- r_openshift_aws_node_group_clusterid: myclusterid
+- r_openshift_aws_node_group_region: us-east-1
+- r_openshift_aws_node_group_lc_name: launch_config
+- r_openshift_aws_node_group_type: master|infra|compute
+- r_openshift_aws_node_group_config: "{{ node_group_config }}"
+```yaml
+master:
+  instance_type: m4.xlarge
+  ami: ami-cdeec8b6  # if using an encrypted AMI this will be replaced
+  volumes:
+  - device_name: /dev/sdb
+    volume_size: 100
+    device_type: gp2
+    delete_on_termination: False
+  health_check:
+    period: 60
+    type: EC2
+  min_size: 3
+  max_size: 3
+  desired_size: 3
+  tags:
+    host-type: master
+    sub-host-type: default
+  wait_for_instances: True
+```
+- r_openshift_aws_node_group_subnet_name: "{{ subnet_name }}"
+
+```yaml
+us-east-1a  # name of subnet
+```
+
+Dependencies
+------------
+
+
+Example Playbook
+----------------
+```yaml
+  - name: "create {{ openshift_build_node_type }} node groups"
+    include_role:
+      name: openshift_aws_node_group
+    vars:
+      r_openshift_aws_node_group_name: "{{ clusterid }} openshift compute"
+      r_openshift_aws_node_group_lc_name: "{{ launch_config_name }}"
+      r_openshift_aws_node_group_clusterid: "{{ clusterid }}"
+      r_openshift_aws_node_group_region: "{{ region }}"
+      r_openshift_aws_node_group_config: "{{ node_group_config }}"
+      r_openshift_aws_node_group_type: compute
+      r_openshift_aws_node_group_subnet_name: "{{ subnet_name }}"
+```
+
+License
+-------
+
+Apache 2.0
+
+Author Information
+------------------
+
+Openshift

roles/openshift_aws_node_group/defaults/main.yml

@@ -0,0 +1,58 @@
+---
+r_openshift_aws_node_group_type: master
+
+r_openshift_aws_node_group_config:
+  tags:
+    clusterid: "{{ r_openshift_aws_node_group_clusterid }}"
+  master:
+    instance_type: m4.xlarge
+    ami: "{{ r_openshift_aws_node_group_ami }}"
+    volumes:
+    - device_name: /dev/sdb
+      volume_size: 100
+      device_type: gp2
+      delete_on_termination: False
+    health_check:
+      period: 60
+      type: EC2
+    min_size: 3
+    max_size: 3
+    desired_size: 3
+    tags:
+      host-type: master
+      sub-host-type: default
+    wait_for_instances: True
+  compute:
+    instance_type: m4.xlarge
+    ami: "{{ r_openshift_aws_node_group_ami }}"
+    volumes:
+    - device_name: /dev/sdb
+      volume_size: 100
+      device_type: gp2
+      delete_on_termination: True
+    health_check:
+      period: 60
+      type: EC2
+    min_size: 3
+    max_size: 100
+    desired_size: 3
+    tags:
+      host-type: node
+      sub-host-type: compute
+  infra:
+    instance_type: m4.xlarge
+    ami: "{{ r_openshift_aws_node_group_ami }}"
+    volumes:
+    - device_name: /dev/sdb
+      volume_size: 100
+      device_type: gp2
+      delete_on_termination: True
+    health_check:
+      period: 60
+      type: EC2
+    min_size: 2
+    max_size: 20
+    desired_size: 2
+    tags:
+      host-type: node
+      sub-host-type: infra

roles/openshift_aws_node_group/tasks/main.yml

@@ -0,0 +1,32 @@
+---
+- name: validate role inputs
+  fail:
+    msg: Please pass in the required role variables
+  when:
+  - r_openshift_aws_node_group_clusterid is not defined
+  - r_openshift_aws_node_group_region is not defined
+  - r_openshift_aws_node_group_subnet_name is not defined
+
+- name: fetch the subnet to use in scale group
+  ec2_vpc_subnet_facts:
+    region: "{{ r_openshift_aws_node_group_region }}"
+    filters:
+      "tag:Name": "{{ r_openshift_aws_node_group_subnet_name }}"
+  register: subnetout
+
+- name: Create the scale group
+  ec2_asg:
+    name: "{{ r_openshift_aws_node_group_name }}"
+    launch_config_name: "{{ r_openshift_aws_node_group_lc_name }}"
+    health_check_period: "{{ r_openshift_aws_node_group_config[r_openshift_aws_node_group_type].health_check.period }}"
+    health_check_type: "{{ r_openshift_aws_node_group_config[r_openshift_aws_node_group_type].health_check.type }}"
+    min_size: "{{ r_openshift_aws_node_group_config[r_openshift_aws_node_group_type].min_size }}"
+    max_size: "{{ r_openshift_aws_node_group_config[r_openshift_aws_node_group_type].max_size }}"
+    desired_capacity: "{{ r_openshift_aws_node_group_config[r_openshift_aws_node_group_type].desired_size }}"
+    region: "{{ r_openshift_aws_node_group_region }}"
+    termination_policies: "{{ r_openshift_aws_node_group_config[r_openshift_aws_node_group_type].termination_policy if 'termination_policy' in  r_openshift_aws_node_group_config[r_openshift_aws_node_group_type] else omit }}"
+    load_balancers: "{{ r_openshift_aws_node_group_config[r_openshift_aws_node_group_type].elbs if 'elbs' in r_openshift_aws_node_group_config[r_openshift_aws_node_group_type] else omit }}"
+    wait_for_instances: "{{ r_openshift_aws_node_group_config[r_openshift_aws_node_group_type].wait_for_instances | default(False)}}"
+    vpc_zone_identifier: "{{ subnetout.subnets[0].id }}"
+    tags:
+    - "{{ r_openshift_aws_node_group_config.tags | combine(r_openshift_aws_node_group_config[r_openshift_aws_node_group_type].tags) }}"

roles/openshift_aws_s3/README.md

@@ -0,0 +1,43 @@
+openshift_aws_s3
+=========
+
+Ansible role to create an s3 bucket
+
+Requirements
+------------
+
+Ansible Modules:
+
+
+Role Variables
+--------------
+
+- r_openshift_aws_s3_clusterid: myclusterid
+- r_openshift_aws_s3_region: us-east-1
+- r_openshift_aws_s3_mode:  create|delete
+
+Dependencies
+------------
+
+
+Example Playbook
+----------------
+```yaml
+- name: create an s3 bucket
+  include_role:
+    name: openshift_aws_s3
+  vars:
+    r_openshift_aws_s3_clusterid: mycluster
+    r_openshift_aws_s3_region: us-east-1
+    r_openshift_aws_s3_mode: create
+```
+
+License
+-------
+
+Apache 2.0
+
+Author Information
+------------------
+
+Openshift

roles/openshift_aws_s3/tasks/main.yml

@@ -0,0 +1,6 @@
+---
+- name: Create an s3 bucket
+  s3:
+    bucket: "{{ r_openshift_aws_s3_clusterid }}"
+    mode: "{{ r_openshift_aws_s3_mode }}"
+    region: "{{ r_openshift_aws_s3_region }}"

roles/openshift_aws_sg/README.md

@@ -0,0 +1,59 @@
+openshift_aws_sg
+=========
+
+Ansible role to create an aws security groups
+
+Requirements
+------------
+
+Ansible Modules:
+
+
+Role Variables
+--------------
+
+- r_openshift_aws_sg_clusterid: myclusterid
+- r_openshift_aws_sg_region: us-east-1
+- r_openshift_aws_sg_type: master|infra|compute
+```yaml
+# defaults/main.yml
+  default:
+    name: "{{ r_openshift_aws_sg_clusterid }}"
+    desc: "{{ r_openshift_aws_sg_clusterid }} default"
+    rules:
+    - proto: tcp
+      from_port: 22
+      to_port: 22
+      cidr_ip: 0.0.0.0/0
+    - proto: all
+      from_port: all
+      to_port: all
+      group_name: "{{ r_openshift_aws_sg_clusterid }}"
+```
+
+
+Dependencies
+------------
+
+
+Example Playbook
+----------------
+```yaml
+- name: create security groups for master
+  include_role:
+    name: openshift_aws_sg
+  vars:
+    r_openshift_aws_sg_clusterid: mycluster
+    r_openshift_aws_sg_region: us-east-1
+    r_openshift_aws_sg_type: master
+```
+
+License
+-------
+
+Apache 2.0
+
+Author Information
+------------------
+
+Openshift

roles/openshift_aws_sg/defaults/main.yml

@@ -0,0 +1,48 @@
+---
+r_openshift_aws_sg_sg:
+  default:
+    name: "{{ r_openshift_aws_sg_clusterid }}"
+    desc: "{{ r_openshift_aws_sg_clusterid }} default"
+    rules:
+    - proto: tcp
+      from_port: 22
+      to_port: 22
+      cidr_ip: 0.0.0.0/0
+    - proto: all
+      from_port: all
+      to_port: all
+      group_name: "{{ r_openshift_aws_sg_clusterid }}"
+  master:
+    name: "{{ r_openshift_aws_sg_clusterid }}_master"
+    desc: "{{ r_openshift_aws_sg_clusterid }} master instances"
+    rules:
+    - proto: tcp
+      from_port: 80
+      to_port: 80
+      cidr_ip: 0.0.0.0/0
+    - proto: tcp
+      from_port: 443
+      to_port: 443
+      cidr_ip: 0.0.0.0/0
+  compute:
+    name: "{{ r_openshift_aws_sg_clusterid }}_compute"
+    desc: "{{ r_openshift_aws_sg_clusterid }} compute node instances"
+  infra:
+    name: "{{ r_openshift_aws_sg_clusterid }}_infra"
+    desc: "{{ r_openshift_aws_sg_clusterid }} infra node instances"
+    rules:
+    - proto: tcp
+      from_port: 80
+      to_port: 80
+      cidr_ip: 0.0.0.0/0
+    - proto: tcp
+      from_port: 443
+      to_port: 443
+      cidr_ip: 0.0.0.0/0
+    - proto: tcp
+      from_port: 30000
+      to_port: 32000
+      cidr_ip: 0.0.0.0/0
+  etcd:
+    name: "{{ r_openshift_aws_sg_clusterid }}_etcd"
+    desc: "{{ r_openshift_aws_sg_clusterid }} etcd instances"

roles/openshift_aws_sg/tasks/main.yml

@@ -0,0 +1,53 @@
+---
+- name: Validate role inputs
+  fail:
+    msg: Please ensure to pass the correct variables
+  when:
+  - r_openshift_aws_sg_region is undefined
+  - r_openshift_aws_sg_region is undefined
+
+
+- name: Fetch the VPC for vpc.id
+  ec2_vpc_net_facts:
+    region: "{{ r_openshift_aws_sg_region }}"
+    filters:
+      "tag:Name": "{{ r_openshift_aws_sg_clusterid }}"
+  register: vpcout
+
+- name: Create default security group for cluster
+  ec2_group:
+    name: "{{ r_openshift_aws_sg_sg.default.name }}"
+    description: "{{ r_openshift_aws_sg_sg.default.desc }}"
+    region: "{{ r_openshift_aws_sg_region }}"
+    vpc_id: "{{ vpcout.vpcs[0].id }}"
+    rules: "{{ r_openshift_aws_sg_sg.default.rules | default(omit, True)}}"
+  register: sg_default_created
+
+- name: create the node group sgs
+  ec2_group:
+    name: "{{ item.name}}"
+    description: "{{ item.desc }}"
+    rules: "{{ item.rules if 'rules' in item else [] }}"
+    region: "{{ r_openshift_aws_sg_region }}"
+    vpc_id: "{{ vpcout.vpcs[0].id }}"
+  register: sg_create
+  with_items:
+  - "{{ r_openshift_aws_sg_sg[r_openshift_aws_sg_type]}}"
+
+- name: create the k8s sgs for the node group
+  ec2_group:
+    name: "{{ item.name }}_k8s"
+    description: "{{ item.desc }} for k8s"
+    region: "{{ r_openshift_aws_sg_region }}"
+    vpc_id: "{{ vpcout.vpcs[0].id }}"
+  register: k8s_sg_create
+  with_items:
+  - "{{ r_openshift_aws_sg_sg[r_openshift_aws_sg_type] }}"
+
+- name: tag sg groups with proper tags
+  ec2_tag:
+    tags:
+      KubernetesCluster: "{{ r_openshift_aws_sg_clusterid }}"
+    resource: "{{ item.group_id }}"
+    region: "{{ r_openshift_aws_sg_region }}"
+  with_items: "{{ k8s_sg_create.results }}"

roles/openshift_aws_ssh_keys/README.md

@@ -0,0 +1,49 @@
+openshift_aws_ssh_keys
+=========
+
+Ansible role for sshind SSH keys
+
+Requirements
+------------
+
+Ansible Modules:
+
+
+Role Variables
+--------------
+
+- r_openshift_aws_ssh_keys_users: list of dicts of users
+- r_openshift_aws_ssh_keys_region: ec2_region to install the keys
+
+Dependencies
+------------
+
+
+Example Playbook
+----------------
+```yaml
+users:
+- username: user1
+  pub_key: <user1 ssh public key>
+- username: user2
+  pub_key: <user2 ssh public key>
+
+region: us-east-1
+
+- include_role:
+    name: openshift_aws_ssh_keys
+  vars:
+    r_openshift_aws_ssh_keys_users: "{{ users }}"
+    r_openshift_aws_ssh_keys_region: "{{ region }}"
+```
+
+
+License
+-------
+
+Apache 2.0
+
+Author Information
+------------------
+
+Openshift

roles/openshift_aws_ssh_keys/tasks/main.yml

@@ -0,0 +1,8 @@
+---
+- name: Add the public keys for the users
+  ec2_key:
+    name: "{{ item.key_name }}"
+    key_material: "{{ item.pub_key }}"
+    region: "{{ r_openshift_aws_ssh_keys_region }}"
+  with_items: "{{ r_openshift_aws_ssh_keys_users }}"
+  no_log: True

roles/openshift_aws_vpc/README.md

@@ -0,0 +1,62 @@
+openshift_aws_vpc
+=========
+
+Ansible role to create a default AWS VPC
+
+Requirements
+------------
+
+Ansible Modules:
+
+
+Role Variables
+--------------
+
+- r_openshift_aws_vpc_clusterid: "{{ clusterid }}"
+- r_openshift_aws_vpc_cidr: 172.31.48.0/20
+- r_openshift_aws_vpc_subnets: "{{ subnets }}"
+```yaml
+    subnets:
+      us-east-1:  # These are us-east-1 region defaults. Ensure this matches your region
+      - cidr: 172.31.48.0/20
+        az: "us-east-1c"
+      - cidr: 172.31.32.0/20
+        az: "us-east-1e"
+      - cidr: 172.31.16.0/20
+        az: "us-east-1a"
+```
+- r_openshift_aws_vpc_region: "{{ region }}"
+- r_openshift_aws_vpc_tags: dict of tags to apply to vpc
+- r_openshift_aws_vpc_name: "{{ vpc_name | default(clusterid) }}"
+
+Dependencies
+------------
+
+
+Example Playbook
+----------------
+
+```yaml
+  - name: create default vpc
+    include_role:
+      name: openshift_aws_vpc
+    vars:
+      r_openshift_aws_vpc_clusterid: mycluster
+      r_openshift_aws_vpc_cidr: 172.31.48.0/20
+      r_openshift_aws_vpc_subnets: "{{ subnets }}"
+      r_openshift_aws_vpc_region: us-east-1
+      r_openshift_aws_vpc_tags: {}
+      r_openshift_aws_vpc_name: mycluster
+
+```
+
+
+License
+-------
+
+Apache 2.0
+
+Author Information
+------------------
+
+Openshift

roles/openshift_aws_vpc/defaults/main.yml

@@ -0,0 +1 @@
+---

roles/openshift_aws_vpc/tasks/main.yml

@@ -0,0 +1,53 @@
+---
+- name: Create AWS VPC
+  ec2_vpc_net:
+    state: present
+    cidr_block: "{{ r_openshift_aws_vpc_cidr }}"
+    dns_support: True
+    dns_hostnames: True
+    region: "{{ r_openshift_aws_vpc_region }}"
+    name: "{{ r_openshift_aws_vpc_clusterid }}"
+    tags:
+      Name: "{{ r_openshift_aws_vpc_clusterid }}"
+  register: vpc
+
+- name: Sleep to avoid a race condition when creating the vpc
+  pause:
+    seconds: 5
+  when: vpc.changed
+
+- name: assign the vpc igw
+  ec2_vpc_igw:
+    region: "{{ r_openshift_aws_vpc_region }}"
+    vpc_id: "{{ vpc.vpc.id }}"
+  register: igw
+
+- name: assign the vpc subnets
+  ec2_vpc_subnet:
+    region: "{{ r_openshift_aws_vpc_region }}"
+    vpc_id: "{{ vpc.vpc.id }}"
+    cidr: "{{ item.cidr }}"
+    az: "{{ item.az }}"
+    resource_tags:
+      Name: "{{ item.az }}"
+  with_items: "{{ r_openshift_aws_vpc_subnets[r_openshift_aws_vpc_region] }}"
+
+- name: Grab the route tables from our VPC
+  ec2_vpc_route_table_facts:
+    region: "{{ r_openshift_aws_vpc_region }}"
+    filters:
+      vpc-id: "{{ vpc.vpc.id }}"
+  register: route_table
+
+- name: update the route table in the vpc
+  ec2_vpc_route_table:
+    lookup: id
+    route_table_id: "{{ route_table.route_tables[0].id }}"
+    vpc_id: "{{ vpc.vpc.id }}"
+    region: "{{ r_openshift_aws_vpc_region }}"
+    tags:
+      Name: "{{ r_openshift_aws_vpc_name }}"
+    routes:
+    - dest: 0.0.0.0/0
+      gateway_id: igw
+  register: route_table_out

roles/openshift_hosted/defaults/main.yml

@@ -5,6 +5,9 @@ r_openshift_hosted_router_use_firewalld: False
 r_openshift_hosted_registry_firewall_enabled: True
 r_openshift_hosted_registry_use_firewalld: False
 
+openshift_hosted_router_wait: True
+openshift_hosted_registry_wait: True
+
 registry_volume_claim: 'registry-claim'
 
 openshift_hosted_router_edits:

roles/openshift_hosted/tasks/registry/registry.yml

@@ -132,34 +132,36 @@
     edits: "{{ openshift_hosted_registry_edits }}"
     force: "{{ True|bool in openshift_hosted_registry_force }}"
 
-- name: Ensure OpenShift registry correctly rolls out (best-effort today)
-  command: |
-    oc rollout status deploymentconfig {{ openshift_hosted_registry_name }} \
-                      --namespace {{ openshift_hosted_registry_namespace }} \
-                      --config {{ openshift.common.config_base }}/master/admin.kubeconfig
-  async: 600
-  poll: 15
-  failed_when: false
-
-- name: Determine the latest version of the OpenShift registry deployment
-  command: |
-    {{ openshift.common.client_binary }} get deploymentconfig {{ openshift_hosted_registry_name }} \
-           --namespace {{ openshift_hosted_registry_namespace }} \
-           --config {{ openshift.common.config_base }}/master/admin.kubeconfig \
-           -o jsonpath='{ .status.latestVersion }'
-  register: openshift_hosted_registry_latest_version
-
-- name: Sanity-check that the OpenShift registry rolled out correctly
-  command: |
-    {{ openshift.common.client_binary }} get replicationcontroller {{ openshift_hosted_registry_name }}-{{ openshift_hosted_registry_latest_version.stdout }} \
-           --namespace {{ openshift_hosted_registry_namespace }} \
-           --config {{ openshift.common.config_base }}/master/admin.kubeconfig \
-           -o jsonpath='{ .metadata.annotations.openshift\.io/deployment\.phase }'
-  register: openshift_hosted_registry_rc_phase
-  until: "'Running' not in openshift_hosted_registry_rc_phase.stdout"
-  delay: 15
-  retries: 40
-  failed_when: "'Failed' in openshift_hosted_registry_rc_phase.stdout"
+- when: openshift_hosted_registry_wait
+  block:
+  - name: Ensure OpenShift registry correctly rolls out (best-effort today)
+    command: |
+      oc rollout status deploymentconfig {{ openshift_hosted_registry_name }} \
+                        --namespace {{ openshift_hosted_registry_namespace }} \
+                        --config {{ openshift.common.config_base }}/master/admin.kubeconfig
+    async: 600
+    poll: 15
+    failed_when: false
+
+  - name: Determine the latest version of the OpenShift registry deployment
+    command: |
+      {{ openshift.common.client_binary }} get deploymentconfig {{ openshift_hosted_registry_name }} \
+             --namespace {{ openshift_hosted_registry_namespace }} \
+             --config {{ openshift.common.config_base }}/master/admin.kubeconfig \
+             -o jsonpath='{ .status.latestVersion }'
+    register: openshift_hosted_registry_latest_version
+
+  - name: Sanity-check that the OpenShift registry rolled out correctly
+    command: |
+      {{ openshift.common.client_binary }} get replicationcontroller {{ openshift_hosted_registry_name }}-{{ openshift_hosted_registry_latest_version.stdout }} \
+             --namespace {{ openshift_hosted_registry_namespace }} \
+             --config {{ openshift.common.config_base }}/master/admin.kubeconfig \
+             -o jsonpath='{ .metadata.annotations.openshift\.io/deployment\.phase }'
+    register: openshift_hosted_registry_rc_phase
+    until: "'Running' not in openshift_hosted_registry_rc_phase.stdout"
+    delay: 15
+    retries: 40
+    failed_when: "'Failed' in openshift_hosted_registry_rc_phase.stdout"
 
 - include: storage/glusterfs.yml
   when:

roles/openshift_hosted/tasks/router/router.yml

@@ -94,36 +94,38 @@
     stats_port: "{{ item.stats_port }}"
   with_items: "{{ openshift_hosted_routers }}"
 
-- name: Ensure OpenShift router correctly rolls out (best-effort today)
-  command: |
-    {{ openshift.common.client_binary }} rollout status deploymentconfig {{ item.name }} \
-                      --namespace {{ item.namespace | default('default') }} \
-                      --config {{ openshift.common.config_base }}/master/admin.kubeconfig
-  async: 600
-  poll: 15
-  with_items: "{{ openshift_hosted_routers }}"
-  failed_when: false
+- when: openshift_hosted_router_wait
+  block:
+  - name: Ensure OpenShift router correctly rolls out (best-effort today)
+    command: |
+      {{ openshift.common.client_binary }} rollout status deploymentconfig {{ item.name }} \
+                        --namespace {{ item.namespace | default('default') }} \
+                        --config {{ openshift.common.config_base }}/master/admin.kubeconfig
+    async: 600
+    poll: 15
+    with_items: "{{ openshift_hosted_routers }}"
+    failed_when: false
 
-- name: Determine the latest version of the OpenShift router deployment
-  command: |
-    {{ openshift.common.client_binary }} get deploymentconfig {{ item.name }} \
-           --namespace {{ item.namespace }} \
-           --config {{ openshift.common.config_base }}/master/admin.kubeconfig \
-           -o jsonpath='{ .status.latestVersion }'
-  register: openshift_hosted_routers_latest_version
-  with_items: "{{ openshift_hosted_routers }}"
+  - name: Determine the latest version of the OpenShift router deployment
+    command: |
+      {{ openshift.common.client_binary }} get deploymentconfig {{ item.name }} \
+             --namespace {{ item.namespace }} \
+             --config {{ openshift.common.config_base }}/master/admin.kubeconfig \
+             -o jsonpath='{ .status.latestVersion }'
+    register: openshift_hosted_routers_latest_version
+    with_items: "{{ openshift_hosted_routers }}"
 
-- name: Poll for OpenShift router deployment success
-  command: |
-    {{ openshift.common.client_binary }} get replicationcontroller {{ item.0.name }}-{{ item.1.stdout }} \
-           --namespace {{ item.0.namespace }} \
-           --config {{ openshift.common.config_base }}/master/admin.kubeconfig \
-           -o jsonpath='{ .metadata.annotations.openshift\.io/deployment\.phase }'
-  register: openshift_hosted_router_rc_phase
-  until: "'Running' not in openshift_hosted_router_rc_phase.stdout"
-  delay: 15
-  retries: 40
-  failed_when: "'Failed' in openshift_hosted_router_rc_phase.stdout"
-  with_together:
-  - "{{ openshift_hosted_routers }}"
-  - "{{ openshift_hosted_routers_latest_version.results }}"
+  - name: Poll for OpenShift router deployment success
+    command: |
+      {{ openshift.common.client_binary }} get replicationcontroller {{ item.0.name }}-{{ item.1.stdout }} \
+             --namespace {{ item.0.namespace }} \
+             --config {{ openshift.common.config_base }}/master/admin.kubeconfig \
+             -o jsonpath='{ .metadata.annotations.openshift\.io/deployment\.phase }'
+    register: openshift_hosted_router_rc_phase
+    until: "'Running' not in openshift_hosted_router_rc_phase.stdout"
+    delay: 15
+    retries: 40
+    failed_when: "'Failed' in openshift_hosted_router_rc_phase.stdout"
+    with_together:
+    - "{{ openshift_hosted_routers }}"
+    - "{{ openshift_hosted_routers_latest_version.results }}"

roles/openshift_master/tasks/bootstrap.yml

@@ -0,0 +1,28 @@
+---
+
+- name: ensure the node-bootstrap service account exists
+  oc_serviceaccount:
+    name: node-bootstrapper
+    namespace: openshift-infra
+    state: present
+  run_once: true
+
+- name: grant node-bootstrapper the correct permissions to bootstrap
+  oc_adm_policy_user:
+    namespace: openshift-infra
+    user: system:serviceaccount:openshift-infra:node-bootstrapper
+    resource_kind: cluster-role
+    resource_name: system:node-bootstrapper
+    state: present
+  run_once: true
+
+# TODO: create a module for this command.
+# oc_serviceaccounts_kubeconfig
+- name: create service account kubeconfig with csr rights
+  command: "oc serviceaccounts create-kubeconfig node-bootstrapper -n openshift-infra"
+  register: kubeconfig_out
+
+- name: put service account kubeconfig into a file on disk for bootstrap
+  copy:
+    content: "{{ kubeconfig_out.stdout }}"
+    dest: "{{ openshift_master_config_dir }}/bootstrap.kubeconfig"

roles/openshift_master/tasks/main.yml

@@ -218,6 +218,20 @@
   - restart master api
   - restart master controllers
 
+- name: modify controller args
+  yedit:
+    src: /etc/origin/master/master-config.yaml
+    edits:
+    - key: kubernetesMasterConfig.controllerArguments.cluster-signing-cert-file
+      value:
+      - /etc/origin/master/ca.crt
+    - key: kubernetesMasterConfig.controllerArguments.cluster-signing-key-file
+      value:
+      - /etc/origin/master/ca.key
+  notify:
+  - restart master controllers
+  when: openshift_master_bootstrap_enabled | default(False)
+
 - include: set_loopback_context.yml
   when:
   - openshift.common.version_gte_3_2_or_1_2
@@ -366,3 +380,7 @@
   shell: echo {{ openshift_master_cluster_password | quote }} | passwd --stdin hacluster
   when:
   - l_install_result | changed
+
+- name: node bootstrap settings
+  include: bootstrap.yml
+  when: openshift_master_bootstrap_enabled | default(False)