Allow custom OpenStack network and subnet

This adds two new parameters to the `openshift_openstack` role:
`openshift_openstack_node_network_name` and
`openshift_openstack_node_subnet_name`.

When set, the OpenStack servers will be created in the given network
and subnet. This, in conjunction with the floating IP customisation
patch allows deploying OpenShift without requiring floating IPs.

playbooks/openstack/configuration.md

@@ -16,6 +16,7 @@ Environment variables may also be used.
 * [OpenStack With SSL Configuration](#openstack-with-ssl-configuration)
 * [Stack Name Configuration](#stack-name-configuration)
 * [DNS Configuration](#dns-configuration)
+* [Floating IP Address Configuration](#floating-ip-address-configuration)
 * [All-in-one Deployment Configuration](#all-in-one-deployment-configuration)
 * [Building Node Images](#building-node-images)
 * [Kuryr Networking Configuration](#kuryr-networking-configuration)
@@ -414,6 +415,86 @@ These must point to the publicly-accessible IP addresses of your
 master and infra nodes or preferably to the load balancers.
 
 
+## Floating IP Address Configuration
+
+Every OpenShift node as well as the API and Router load balancer will receive a
+floating IP address by default. This is to make the deployment and debugging
+experience easier.
+
+You may want to change that behaviour, for example to prevent any possibility
+of external access to the nodes (defense in depth) or if your floating IP pool
+is not large enough.
+
+### Overview
+
+It possible to configure the playbooks to not asssign floating IP addresses.
+However, the Ansible playbooks will then not be able to SSH and install
+OpenShift.
+
+The nodes will only be accessible from the subnet they are assigned to.
+
+To solve this, we need to create the network the nodes will be placed in
+beforehnd, then boot up a bastion host in the same network and run the
+playbooks from there.
+
+### Node Network
+
+We will have to create a Neutron Network, Subnet and a Router for external
+connectivity. Take note of any DNS servers you would normally put under
+`openshift_openstack_dns_nameservers` -- they must be added to the subnet.
+
+In this example, we will call the network and its subnet `openshift` and configure
+a DNS server with IP address `10.20.30.40`. The external network will be called `public`.
+
+```
+$ openstack network create openshift
+$ openstack subnet create --subnet-range 192.168.0.0/24 --dns-nameserver 10.20.30.40 --network openshift openshift
+$ openstack router create openshift-router
+$ openstack router set --external-gateway public openshift-router
+$ openstack router add subnet openshift-router openshift
+```
+
+### Bastion host
+
+To provide SSH connectivity (that Ansible requires) to the OpenShift nodes
+without using floating IP addresses, the playbooks must be running on a server
+inside the same subnet.
+
+This will create such server and place it into the subnet created above.
+
+We will use an image called `CentOS-7-x86_64-GenericCloud`, and assume that the
+created floating IP address will be `172.24.4.10`.
+
+```
+$ openstack server create --wait --image CentOS-7-x86_64-GenericCloud --flavor m1.medium --key-name openshift --network openshift bastion
+$ openstack floating ip create public
+$ openstack server add floating ip bastion 172.24.4.10
+$ ping 172.24.4.10
+$ ssh centos@172.24.4.10
+```
+
+### openshift-ansible Configuration
+
+In addition to the rest of openshift-ansible configuration, we will need to
+specify the node netwok, subnet and that we do not want any floating IP
+addresses.
+
+You must do this from inside the "bastion" host created in the previous step.
+
+Put the following to `inventory/group_vars/all.yml`:
+
+```yaml
+openshift_openstack_node_network_name: openshift
+openshift_openstack_node_subnet_name: openshift
+openshift_openstack_master_floating_ip: false
+openshift_openstack_infra_floating_ip: false
+openshift_openstack_compute_floating_ip: false
+openshift_openstack_load_balancer_floating_ip: false
+```
+
+And then run the `playbooks/openstack/openshift-cluster/*.yml` as usual.
+
+
 ## All-in-one Deployment Configuration
 
 If you want to deploy OpenShift on a single node (e.g. for quick evaluation),

roles/openshift_openstack/defaults/main.yml

@@ -77,6 +77,8 @@ openshift_openstack_lb_image: "{{ openshift_openstack_default_image_name }}"
 openshift_openstack_etcd_image: "{{ openshift_openstack_default_image_name }}"
 openshift_openstack_provider_network_name: null
 openshift_openstack_external_network_name: null
+openshift_openstack_node_network_name: null
+openshift_openstack_node_subnet_name: null
 openshift_openstack_private_network: >-
   {% if openshift_openstack_provider_network_name | default(None) -%}
   {{ openshift_openstack_provider_network_name }}

roles/openshift_openstack/templates/heat_stack.yaml.j2

@@ -167,6 +167,8 @@ resources:
 {% if openshift_use_kuryr|default(false)|bool %}
       vip_address: {{ openshift_openstack_kuryr_service_subnet_cidr | ipaddr('1') | ipaddr('address') }}
       vip_subnet: { get_resource: service_subnet }
+{% elif openshift_openstack_node_subnet_name %}
+      vip_subnet: {{ openshift_openstack_node_subnet_name }}
 {% else %}
       vip_subnet: { get_resource: subnet }
 {% endif %}
@@ -333,6 +335,7 @@ resources:
 
 {% endif %}
 
+{% if not openshift_openstack_node_network_name %}
   net:
     type: OS::Neutron::Net
     properties:
@@ -341,7 +344,9 @@ resources:
           template: openshift-ansible-cluster_id-net
           params:
             cluster_id: {{ openshift_openstack_full_dns_domain }}
+{% endif %}
 
+{% if not openshift_openstack_node_subnet_name %}
   subnet:
     type: OS::Neutron::Subnet
     properties:
@@ -359,6 +364,7 @@ resources:
 {% for nameserver in openshift_openstack_dns_nameservers %}
         - {{ nameserver }}
 {% endfor %}
+{% endif %}
 
 {% if openshift_use_flannel|default(False)|bool %}
   data_net:
@@ -387,11 +393,13 @@ resources:
       external_gateway_info:
         network: {{ openshift_openstack_external_network_name }}
 
+{% if not openshift_openstack_node_subnet_name %}
   interface:
     type: OS::Neutron::RouterInterface
     properties:
       router_id: { get_resource: router }
       subnet_id: { get_resource: subnet }
+{% endif %}
 
 {% if openshift_use_kuryr|default(false)|bool %}
   pod_subnet_interface:
@@ -618,8 +626,17 @@ resources:
           net:         {{ openshift_openstack_provider_network_name }}
           net_name:         {{ openshift_openstack_provider_network_name }}
 {% else %}
+{% if openshift_openstack_node_network_name %}
+          net:         {{ openshift_openstack_node_network_name }}
+          net_name:    {{ openshift_openstack_node_network_name }}
+{% else %}
           net:         { get_resource: net }
+{% endif %}
+{% if openshift_openstack_node_subnet_name %}
+          subnet:      {{ openshift_openstack_node_subnet_name }}
+{% else %}
           subnet:      { get_resource: subnet }
+{% endif %}
 {% if openshift_use_kuryr|default(false)|bool %}
           pod_net:     { get_resource: pod_net }
           pod_subnet:  { get_resource: pod_subnet }
@@ -646,7 +663,7 @@ resources:
           attach_float_net: false
 {% endif %}
           volume_size: {{ openshift_openstack_etcd_volume_size }}
-{% if not openshift_openstack_provider_network_name %}
+{% if not openshift_openstack_provider_network_name and not openshift_openstack_node_subnet_name  %}
     depends_on:
       - interface
 {% endif %}
@@ -694,9 +711,15 @@ resources:
 {% if openshift_openstack_provider_network_name %}
           net:         {{ openshift_openstack_provider_network_name }}
           net_name:         {{ openshift_openstack_provider_network_name }}
+{% elif openshift_openstack_node_network_name %}
+          net:         {{ openshift_openstack_node_network_name }}
 {% else %}
           net:         { get_resource: net }
+{% if openshift_openstack_node_subnet_name %}
+          subnet:      {{ openshift_openstack_node_subnet_name }}
+{% else %}
           subnet:      { get_resource: subnet }
+{% endif %}
 {% if openshift_use_kuryr|default(false)|bool %}
           pod_net:     { get_resource: pod_net }
           pod_subnet:  { get_resource: pod_subnet }
@@ -723,7 +746,7 @@ resources:
           attach_float_net: false
 {% endif %}
           volume_size: {{ openshift_openstack_lb_volume_size }}
-{% if not openshift_openstack_provider_network_name %}
+{% if not openshift_openstack_provider_network_name and not openshift_openstack_node_subnet_name  %}
     depends_on:
       - interface
 {% endif %}
@@ -762,8 +785,17 @@ resources:
           net:         {{ openshift_openstack_provider_network_name }}
           net_name:         {{ openshift_openstack_provider_network_name }}
 {% else %}
+{% if openshift_openstack_node_network_name %}
+          net:         {{ openshift_openstack_node_network_name }}
+          net_name:    {{ openshift_openstack_node_network_name }}
+{% else %}
           net:         { get_resource: net }
+{% endif %}
+{% if openshift_openstack_node_subnet_name %}
+          subnet:      {{ openshift_openstack_node_subnet_name }}
+{% else %}
           subnet:      { get_resource: subnet }
+{% endif %}
 {% if openshift_use_kuryr|default(false)|bool %}
           pod_net:     { get_resource: pod_net }
           pod_subnet:  { get_resource: pod_subnet }
@@ -810,7 +842,7 @@ resources:
           scheduler_hints:
             group: { get_resource: master_server_group }
 {% endif %}
-{% if not openshift_openstack_provider_network_name %}
+{% if not openshift_openstack_provider_network_name and not openshift_openstack_node_subnet_name  %}
     depends_on:
       - interface
 {% endif %}
@@ -848,8 +880,17 @@ resources:
           net:         {{ openshift_openstack_provider_network_name }}
           net_name:         {{ openshift_openstack_provider_network_name }}
 {% else %}
+{% if openshift_openstack_node_network_name %}
+          net:         {{ openshift_openstack_node_network_name }}
+          net_name:    {{ openshift_openstack_node_network_name }}
+{% else %}
           net:         { get_resource: net }
+{% endif %}
+{% if openshift_openstack_node_subnet_name %}
+          subnet:      {{ openshift_openstack_node_subnet_name }}
+{% else %}
           subnet:      { get_resource: subnet }
+{% endif %}
 {% if openshift_use_kuryr|default(false)|bool %}
           pod_net:     { get_resource: pod_net }
           pod_subnet:  { get_resource: pod_subnet }
@@ -881,7 +922,7 @@ resources:
           attach_float_net: false
 {% endif %}
           volume_size: {{ openshift_openstack_node_volume_size }}
-{% if not openshift_openstack_provider_network_name %}
+{% if not openshift_openstack_provider_network_name and not openshift_openstack_node_subnet_name  %}
     depends_on:
       - interface
 {% endif %}
@@ -921,8 +962,17 @@ resources:
           net:         {{ openshift_openstack_provider_network_name }}
           net_name:         {{ openshift_openstack_provider_network_name }}
 {% else %}
+{% if openshift_openstack_node_network_name %}
+          net:         {{ openshift_openstack_node_network_name }}
+          net_name:    {{ openshift_openstack_node_network_name }}
+{% else %}
           net:         { get_resource: net }
+{% endif %}
+{% if openshift_openstack_node_subnet_name %}
+          subnet:      {{ openshift_openstack_node_subnet_name }}
+{% else %}
           subnet:      { get_resource: subnet }
+{% endif %}
 {% if openshift_use_kuryr|default(false)|bool %}
           pod_net:     { get_resource: pod_net }
           pod_subnet:  { get_resource: pod_subnet }
@@ -964,7 +1014,7 @@ resources:
           scheduler_hints:
             group: { get_resource: infra_server_group }
 {% endif %}
-{% if not openshift_openstack_provider_network_name %}
+{% if not openshift_openstack_provider_network_name and not openshift_openstack_node_subnet_name  %}
     depends_on:
       - interface
 {% endif %}
@@ -999,8 +1049,17 @@ resources:
           net:         {{ openshift_openstack_provider_network_name }}
           net_name:    {{ openshift_openstack_provider_network_name }}
 {% else %}
+{% if openshift_openstack_node_network_name %}
+          net:         {{ openshift_openstack_node_network_name }}
+          net_name:    {{ openshift_openstack_node_network_name }}
+{% else %}
           net:         { get_resource: net }
+{% endif %}
+{% if openshift_openstack_node_subnet_name %}
+          subnet:      {{ openshift_openstack_node_subnet_name }}
+{% else %}
           subnet:      { get_resource: subnet }
+{% endif %}
 {% if openshift_use_kuryr|default(false)|bool %}
           pod_net:     { get_resource: pod_net }
           pod_subnet:  { get_resource: pod_subnet }
@@ -1032,7 +1091,7 @@ resources:
           floating_network: {{ openshift_openstack_external_network_name }}
 {% endif %}
           volume_size: {{ openshift_openstack_cns_volume_size }}
-{% if not openshift_openstack_provider_network_name %}
+{% if not openshift_openstack_provider_network_name and not openshift_openstack_node_subnet_name  %}
     depends_on:
       - interface
 {% endif %}
@@ -1055,7 +1114,11 @@ resources:
   router_lb:
     type: OS::{{ openshift_openstack_lbaasv2_provider }}::LoadBalancer
     properties:
+{% if openshift_openstack_node_subnet_name %}
+      vip_subnet: {{ openshift_openstack_node_subnet_name }}
+{% else %}
       vip_subnet: { get_resource: subnet }
+{% endif %}
 
 {% if openshift_openstack_load_balancer_floating_ip | default(True) | bool %}
   router_lb_floating_ip: