Merge pull request #10541 from oybed/osp-etcd

This PR will allow OSP provisioning of separate 'etcd' instances from the master instances

playbooks/openstack/configuration.md

@@ -18,6 +18,7 @@ Environment variables may also be used.
 * [DNS Configuration](#dns-configuration)
 * [Floating IP Address Configuration](#floating-ip-address-configuration)
 * [All-in-one Deployment Configuration](#all-in-one-deployment-configuration)
+* [Separate etcd Deployment Configuration](#separate-etcd-deployment-configuration)
 * [Multi-env Deployment Configuration](#multi-env-deployment-configuration)
 * [Building Node Images](#building-node-images)
 * [Kuryr Networking Configuration](#kuryr-networking-configuration)
@@ -39,10 +40,12 @@ In `inventory/group_vars/all.yml`:
 * `openshift_openstack_keypair_name` OpenStack keypair to use.
 * Role Node Counts
   * `openshift_openstack_num_masters` Number of master nodes to create.
+  * `openshift_openstack_num_etcd` Number of etcd nodes to create (0 if co-hosted on master hosts).
   * `openshift_openstack_num_infra` Number of infra nodes to create.
   * `openshift_openstack_num_nodes` Number of app nodes to create.
 * Role Node Floating IP Allocation
   * `openshift_openstack_master_floating_ip` Assign floating IP to master nodes. Defaults to `True`.
+  * `openshift_openstack_etcd_floating_ip` Assign floating IP to etcd nodes (if any). Defaults to `True`.
   * `openshift_openstack_infra_floating_ip` Assign floating IP to infra nodes. Defaults to `True`.
   * `openshift_openstack_compute_floating_ip` Assign floating IP to app nodes. Defaults to `True`.
 * Role Images
@@ -514,6 +517,7 @@ First, set the following in `inventory/group_vars/all.yml`:
 ```
 openshift_use_all_in_one_cluster_deployment: True
 openshift_openstack_num_masters: 1
+openshift_openstack_num_etcd: 0
 openshift_openstack_num_infra: 0
 openshift_openstack_num_nodes: 0
 
@@ -547,6 +551,29 @@ added, because there are no dedicated infra nodes, so you will have to add it
 manually. See
 [Custom DNS Records Configuration](#custom-dns-records-configuration).
 
+
+## Separate etcd Deployment Configuration
+
+If you want to deploy OpenShift Container Platform with the etcd running on separate hosts
+appart from the master hosts, the following changes need to be made to the inventory:
+
+Single master and single etcd host:
+```
+ :
+openshift_openstack_num_masters: 1
+openshift_openstack_num_etcd: 1
+ :
+```
+
+Multiple master and multiple etcd hosts:
+```
+ :
+openshift_openstack_num_masters: 3
+openshift_openstack_num_etcd: 3
+ :
+```
+
+
 ## Multi-env Deployment Configuration
 
 If you want to deploy multiple OpenShift environments in the same OpenStack

playbooks/openstack/resources.py

@@ -49,10 +49,11 @@ def base_openshift_inventory(cluster_hosts):
                       if server.metadata['host-type'] == 'lb']
 
     # NOTE: everything that should go to the `[nodes]` group:
-    nodes = list(set(masters + etcd + infra_hosts + app + cns))
+    nodes = list(set(masters + infra_hosts + app + cns))
 
-    # NOTE: all OpenShift nodes, including `[lb]`, `[nfs]`, etc.:
-    osev3 = list(set(nodes + load_balancers))
+    # NOTE: all OpenShift nodes + any "supporting" roles,
+    #       i.e.: `[etcd]`, `[lb]`, `[nfs]`, etc.:
+    osev3 = list(set(nodes + etcd + load_balancers))
 
     inventory['OSEv3'] = {'hosts': osev3, 'vars': {}}
     inventory['openstack_nodes'] = {'hosts': nodes}

playbooks/openstack/sample-inventory/group_vars/all.yml

@@ -125,6 +125,7 @@ openshift_openstack_num_masters: 1
 openshift_openstack_num_infra: 1
 openshift_openstack_num_cns: 0
 openshift_openstack_num_nodes: 2
+openshift_openstack_num_etcd: 0
 
 # # Public IP Allocation
 # # - manage which node roles are allocated public IP addresses

roles/openshift_openstack/defaults/main.yml

@@ -14,6 +14,7 @@ openshift_openstack_use_provider_network: False
 openshift_openstack_stack_state: 'present'
 
 openshift_openstack_ssh_ingress_cidr: 0.0.0.0/0
+openshift_openstack_master_ingress_cidr: 0.0.0.0/0
 openshift_openstack_node_ingress_cidr: 0.0.0.0/0
 openshift_openstack_lb_ingress_cidr: 0.0.0.0/0
 openshift_openstack_num_etcd: 0
@@ -165,6 +166,11 @@ openshift_openstack_etcd_secgroup_rules:
   - direction: ingress
     protocol: tcp
     port_range_min: 2379
+    port_range_max: 2379
+    remote_ip_prefix: "{{ openshift_openstack_master_ingress_cidr }}"
+  - direction: ingress
+    protocol: tcp
+    port_range_min: 2380
     port_range_max: 2380
     remote_mode: remote_group_id
 openshift_openstack_node_secgroup_rules: