Static inventory autogeneration (#550)

* At the provisioning stage, allow users to auto-generate a static
  inventory w/o manual steps needed. The alternative to
  go fully dynamic TBD.
* Move openshift pre-install playbook to the post provision playbook,
  where the second part of the pre install tasks is already placed.

Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>

playbooks/provisioning/openstack/README.md

@@ -93,8 +93,9 @@ steps, and the Neutron subnet for the Heat stack is updated to point to that
 server in the end. So the provisioned servers will start using it natively
 as a default nameserver that comes from the NetworkManager and cloud-init.
 
-`openstack_ssh_key` is a Nova keypair -- you can see your keypairs with
-`openstack keypair list`.
+`openstack_ssh_key` is a Nova keypair - you can see your keypairs with
+`openstack keypair list`. This guide assumes that its corresponding private
+key is `~/.ssh/openshift`, stored on the ansible admin (control) node.
 
 `openstack_default_image_name` is the name of the Glance image the
 servers will use. You can
@@ -127,6 +128,14 @@ The `required_packages` variable also provides a list of the additional
 prerequisite packages to be installed before to deploy an OpenShift cluster.
 Those are ignored though, if the `manage_packages: False`.
 
+The `openstack_inventory` controls either a static inventory will be created after the
+cluster nodes provisioned on OpenStack cloud. Note, the fully dynamic inventory
+is yet to be supported, so the static inventory will be created anyway.
+
+The `openstack_inventory_path` points the directory to host the generated static inventory.
+It should point to the copied example inventory directory, otherwise ti creates
+a new one for you.
+
 #### Security notes
 
 Configure required `*_ingress_cidr` variables to restrict public access
@@ -164,21 +173,48 @@ variables for the `inventory/group_vars/OSEv3.yml`, `all.yml`:
     origin_release: 1.5.1
     openshift_deployment_type: "{{ deployment_type }}"
 
+### Configure static inventory
+
+Example inventory variables:
+
+    openstack_private_ssh_key: ~/.ssh/openshift
+    openstack_inventory: static
+    openstack_inventory_path: ../../../../inventory
+
+
+In this guide, the latter points to the current directory, where you run ansible commands
+from.
+
+To verify nodes connectivity, use the command:
+
+    ansible -v -i inventory/hosts -m ping all
+
+If something is broken, double-check the inventory variables, paths and the
+generated `<openstack_inventory_path>/hosts` file.
+
+The `inventory: dynamic` can be used instead to access cluster nodes directly via
+floating IPs. In this mode you can not use a bastion node and should specify
+the dynamic inventory file in your ansible commands , like `-i openstack.py`.
+
 ## Deployment
 
 ### Run the playbook
 
 Assuming your OpenStack (Keystone) credentials are in the `keystonerc`
-file, this is how you stat the provisioning process:
+this is how you stat the provisioning process from your ansible control node:
 
     . keystonerc
-    ansible-playbook -i inventory --timeout 30  --private-key ~/.ssh/openshift openshift-ansible-contrib/playbooks/provisioning/openstack/provision.yaml
+    ansible-playbook openshift-ansible-contrib/playbooks/provisioning/openstack/provision.yaml
+
+Note, here you start with an empty inventory. The static inventory will be populated
+with data so you can omit providing additional arguments for future ansible commands.
+
 
 ### Install OpenShift
 
 Once it succeeds, you can install openshift by running:
 
-    ansible-playbook --user openshift --private-key ~/.ssh/openshift -i inventory/ openshift-ansible/playbooks/byo/config.yml
+    ansible-playbook openshift-ansible/playbooks/byo/config.yml
 
 
 ## License

playbooks/provisioning/openstack/post-provision-openstack.yml

@@ -1,72 +1,90 @@
 ---
+- hosts: cluster_hosts
+  name: Wait for the the nodes to come up
+  become: False
+  gather_facts: False
+  tasks:
+    - wait_for_connection:
+
+- hosts: cluster_hosts
+  gather_facts: True
+  tasks:
+    - name: Debug hostvar
+      debug:
+        msg: "{{ hostvars[inventory_hostname] }}"
+        verbosity: 2
+
+- name: OpenShift Pre-Requisites (part 1)
+  include: pre-install.yml
+
 - name: Assign hostnames
   hosts: cluster_hosts
   gather_facts: False
   become: true
   pre_tasks:
-  - include: pre_tasks.yml
+    - include: pre_tasks.yml
   roles:
-  - role: hostnames
+    - role: hostnames
 
 - name: Subscribe DNS Host to allow for configuration below
   hosts: dns
   gather_facts: False
   become: true
   roles:
-  - role: subscription-manager
-    when: hostvars.localhost.rhsm_register|default(False)
-    tags: 'subscription-manager'
+    - role: subscription-manager
+      when: hostvars.localhost.rhsm_register|default(False)
+      tags: 'subscription-manager'
 
 - name: Determine which DNS server(s) to use for our generated records
   hosts: localhost
   gather_facts: False
   become: False
   roles:
-  - dns-server-detect
+    - dns-server-detect
 
 - name: Build the DNS Server Views and Configure DNS Server(s)
   hosts: dns
   gather_facts: False
   become: true
   pre_tasks:
-  - include: pre_tasks.yml
-  - name: "Generate dns-server views"
-    include: openstack_dns_views.yml
+    - include: pre_tasks.yml
+    - name: "Generate dns-server views"
+      include: openstack_dns_views.yml
   roles:
-  - role: infra-ansible/roles/dns-server
+    - role: infra-ansible/roles/dns-server
 
 - name: Build and process DNS Records
   hosts: localhost
-  gather_facts: False
+  gather_facts: True
   become: False
   pre_tasks:
-  - include: pre_tasks.yml
-  - name: "Generate dns records"
-    include: openstack_dns_records.yml
+    - include: pre_tasks.yml
+    - name: "Generate dns records"
+      include: openstack_dns_records.yml
   roles:
-  - role: infra-ansible/roles/dns
+    - role: infra-ansible/roles/dns
 
 - name: Switch the stack subnet to the configured private DNS server
   hosts: localhost
   gather_facts: False
   become: False
   vars_files:
-  - stack_params.yaml
+    - stack_params.yaml
   tasks:
-  - include_role:
-      name: openstack-stack
-      tasks_from: subnet_update_dns_servers
+    - include_role:
+        name: openstack-stack
+        tasks_from: subnet_update_dns_servers
 
-- name: OpenShift Pre-Requisites
+- name: OpenShift Pre-Requisites (part 2)
   hosts: OSEv3
   gather_facts: true
   become: true
   pre_tasks:
-  - name: "Include DNS configuration to ensure proper name resolution"
-    lineinfile:
-      state: present
-      dest: /etc/sysconfig/network
-      regexp: "IP4_NAMESERVERS={{ hostvars['localhost'].private_dns_server }}"
-      line: "IP4_NAMESERVERS={{ hostvars['localhost'].private_dns_server }}"
+    - name: "Include DNS configuration to ensure proper name resolution"
+      lineinfile:
+        state: present
+        dest: /etc/sysconfig/network
+        regexp: "IP4_NAMESERVERS={{ hostvars['localhost'].private_dns_server }}"
+        line: "IP4_NAMESERVERS={{ hostvars['localhost'].private_dns_server }}"
   roles:
-  - node-network-manager
+    - node-network-manager

playbooks/provisioning/openstack/provision-openstack.yml

@@ -8,6 +8,10 @@
     - include: pre_tasks.yml
   roles:
     - role: openstack-stack
+    - role: static_inventory
+      when: openstack_inventory|default('static') == 'static'
+      inventory_path: "{{ openstack_inventory_path|default(inventory_dir) }}"
+      private_ssh_key: "{{ openstack_private_ssh_key|default('~/.ssh/id_rsa') }}"
 
 - name: Refresh Server inventory
   hosts: localhost
@@ -17,19 +21,4 @@
   tasks:
     - meta: refresh_inventory
 
-- hosts: cluster_hosts
-  name: Wait for the the nodes to come up
-  become: False
-  gather_facts: False
-  tasks:
-    - wait_for_connection:
-
-- hosts: cluster_hosts
-  gather_facts: True
-  tasks:
-    - name: Debug hostvar
-      debug:
-        msg: "{{ hostvars[inventory_hostname] }}"
-        verbosity: 2
-
 - include: post-provision-openstack.yml

playbooks/provisioning/openstack/provision.yaml

@@ -2,5 +2,3 @@
 - include: "prerequisites.yml"
 
 - include: "provision-openstack.yml"
-
-- include: "pre-install.yml"

playbooks/provisioning/openstack/sample-inventory/group_vars/all.yml

@@ -60,3 +60,14 @@ ansible_user: openshift
 
 # # Use a single security group for a cluster (default: false)
 #openstack_flat_secgrp: false
+
+# # Openstack inventory type and cluster nodes access pattern
+# # Defaults to 'static'.
+# # Use 'dynamic' to access cluster nodes directly, via floating IPs
+# # and given a dynamic inventory script, like openstack.py
+#openstack_inventory: static
+# # The path to checkpoint the static inventory from the in-memory one
+#openstack_inventory_path: ../../../../inventory
+
+# # The Nova key-pair's private SSH key to access inventory nodes
+#openstack_private_ssh_key: ~/.ssh/openshift