{"id":3291,"date":"2022-02-11T13:20:28","date_gmt":"2022-02-11T01:20:28","guid":{"rendered":"https:\/\/www.deltics.co.nz\/blog\/?p=3291"},"modified":"2022-02-11T13:20:32","modified_gmt":"2022-02-11T01:20:32","slug":"kubernetes-home-load-balancer","status":"publish","type":"post","link":"https:\/\/www.deltics.co.nz\/blog\/posts\/3291\/","title":{"rendered":"Kubernetes @ Home: Load Balancer"},"content":{"rendered":"<span class=\"rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">[Estimated Reading Time: <\/span> <span class=\"rt-time\">10<\/span> <span class=\"rt-label rt-postfix\">minutes]<\/span><\/span>\n<p>In the <a href=\"https:\/\/www.deltics.co.nz\/blog\/posts\/3233\/\" data-type=\"URL\" data-id=\"https:\/\/www.deltics.co.nz\/blog\/posts\/3233\/\">previous post<\/a> we installed the <strong>Kubernetes Dashboard<\/strong> and saw how we could access the dashboard service which itself runs in the <strong>Kubernetes<\/strong> cluster.  We also saw how clunky that was and I promised a better way, which we turn to now.<\/p>\n\n\n\n<!--more-->\n\n\n\n<h2 class=\"wp-block-heading\" id=\"kubernetes-services-and-ip-addresses\">Kubernetes Services and IP Addresses<\/h2>\n\n\n\n<p>To this point, we have installed the <strong>Kubernetes Dashboard<\/strong> with the default IP configuration.  Which is to say that the <mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-vivid-purple-color\">alternative.yaml<\/mark> file provided no configuration in this respect and so was endowed with the default.<\/p>\n\n\n\n<p>As a result, the dashboard service is assigned a <strong>ClusterIP<\/strong>.<\/p>\n\n\n\n<p>If we look again at the list of all services in my cluster:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" loading=\"lazy\" width=\"640\" height=\"68\" src=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=640%2C68&#038;ssl=1\" alt=\"\" class=\"wp-image-3237\" srcset=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=1024%2C109&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=512%2C55&amp;ssl=1 512w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=768%2C82&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=1536%2C164&amp;ssl=1 1536w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=380%2C40&amp;ssl=1 380w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?w=1802&amp;ssl=1 1802w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?w=1280&amp;ssl=1 1280w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/figure>\n\n\n\n<p>We see that <em>all<\/em> of these services have a <strong><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-luminous-vivid-orange-color\">CLUSTER-IP<\/mark><\/strong>.<\/p>\n\n\n\n<p>This is an IP address from the pool of IP addresses reserved for the internal network <em>within the cluster<\/em>.  Configured in this way, the service is accessible only from within the cluster (or via <strong>kubeproxy<\/strong> or port-forwarding, <a href=\"https:\/\/www.deltics.co.nz\/blog\/posts\/3233#accessing-the-dashboard\">as we saw previously<\/a>).<\/p>\n\n\n\n<p>The options in this area are as follows:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>ClusterIP<\/strong><\/td><td>Assigned an IP in the cluster network for access within the cluster.  To access the <strong>Service<\/strong> from outside the cluster, <strong>kubeproxy<\/strong> or port-forwarding must be used.<\/td><\/tr><tr><td><strong>NodePort<\/strong><\/td><td>Establishes a port-forwarding rule on <em>every<\/em> worker node.  To access the <strong>Service<\/strong> from outside the cluster, use the specified port on <em>any<\/em> of the worker nodes.<\/td><\/tr><tr><td><strong>LoadBalancer<\/strong><\/td><td>Provides an IP address that may be used to access the Service from outside the cluster.<\/td><\/tr><tr><td><strong>ExternalName<\/strong><\/td><td>Establishes a Service that resolves to a different DNS name.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>The descriptions of these above are highly simplified, but we won&#8217;t be looking at all of these in detail.<\/p>\n\n\n\n<p><strong>ClusterIP<\/strong> we have already covered, with one final note to mention that in a micro-services architecture where a consumable service may be supported by a number of &#8220;background&#8221; services that a client application does not interact with directly, <strong>ClusterIP<\/strong> is perfectly adequate for those background services.<\/p>\n\n\n\n<p><strong>LoadBalancer<\/strong> is what I will use to provide access to my dashboard (and other services) from outside the cluster and what we will look at next.  Although technically this mechanism also establishes <strong>NodePorts<\/strong>, this is an internal implementation detail that we will not need to be concerned with.<\/p>\n\n\n\n<p>For those that are interested <a href=\"https:\/\/itnext.io\/kubernetes-clusterip-vs-nodeport-vs-loadbalancer-services-and-ingress-an-overview-with-722a07f3cfe1\" data-type=\"URL\" data-id=\"https:\/\/itnext.io\/kubernetes-clusterip-vs-nodeport-vs-loadbalancer-services-and-ingress-an-overview-with-722a07f3cfe1\">Arseny Zinchenko has an excellent detailed discussion of them<\/a>.<\/p>\n\n\n\n<p>In the list of services in the cluster, only the <strong>kubernetes-dashboard<\/strong> service has an <strong><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-luminous-vivid-orange-color\">EXTERNAL-IP<\/mark><\/strong>.  This is because the <strong>kubernetes-dashboard<\/strong> is configured with the <strong><mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-vivid-cyan-blue-color\">LoadBalancer<\/mark><\/strong> type.<\/p>\n\n\n\n<p>So let&#8217;s see how this is achieved.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"configuring-a-load-balancer\">Configuring a Load Balancer<\/h2>\n\n\n\n<p>Actually configuring a service to use a <strong>LoadBalancer<\/strong> is trivial, involving only specifying the required type in the service descriptor.  If we look again at that descriptor:<\/p>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-yaml\">kind: Service\napiVersion: v1\nmetadata:\n  labels:\n    k8s-app: kubernetes-dashboard\n  name: kubernetes-dashboard\n  namespace: kubernetes-dashboard\nspec:\n  ports:\n    - port: 80\n      targetPort: 9090\n  selector:\n    k8s-app: kubernetes-dashboard<\/code><\/pre>\n\n\n\n<p>The part we need to change (or rather, add to) is the <strong>spec<\/strong>.  We simply add a <strong>type<\/strong> entry with the value <strong>LoadBalancer<\/strong>:<\/p>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-yaml\">kind: Service\napiVersion: v1\nmetadata:\n  labels:\n    k8s-app: kubernetes-dashboard\n  name: kubernetes-dashboard\n  namespace: kubernetes-dashboard\nspec:\n  type: LoadBalancer\n  ports:\n    - port: 80\n      targetPort: 9090\n  selector:\n    k8s-app: kubernetes-dashboard<\/code><\/pre>\n\n\n\n<p>And that&#8217;s it.<\/p>\n\n\n\n<p>But <strong>Kubernetes<\/strong> does not provide a <strong>LoadBalancer<\/strong> by default.  In a cloud context, this would usually be provided by the cloud provider.  But in a <strong>Bare Metal<\/strong> cluster we need to install one ourselves.<\/p>\n\n\n\n<p>This is where <strong><a rel=\"noreferrer noopener\" href=\"https:\/\/metallb.org\/\" data-type=\"URL\" data-id=\"https:\/\/metallb.org\/\" target=\"_blank\">MetalLB<\/a><\/strong> comes in.<\/p>\n\n\n\n<p>If we were to apply this service descriptor without a <strong>LoadBalancer<\/strong> installed, the service will be assigned a <strong>CLUSTER-IP<\/strong>, but the <strong><em>EXTERNAL-IP<\/em><\/strong> will be stuck in a <strong>Pending<\/strong> state.<\/p>\n\n\n\n<p>So let&#8217;s install <strong>MetalLB<\/strong>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"installing-the-metallb-loadbalancer\">Installing The MetalLB LoadBalancer<\/h2>\n\n\n\n<p>Installation of MetalLB consists of essentially two steps:<\/p>\n\n\n\n<ol><li>Install MetalLB itself<\/li><li>Install a ConfigMap<\/li><\/ol>\n\n\n\n<p>The first is again trivially simple.<\/p>\n\n\n\n<p>Two manifests are installed, the first establishes the <strong>metallb-system<\/strong> namespace while the second installs all the required <strong>MetalLB<\/strong> components into that namespace.  This provides everything needed for <strong>MetalLB<\/strong> to run.<\/p>\n\n\n\n<p>We can verify this by listing pods in the <strong>metallb-system<\/strong> namespace:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" loading=\"lazy\" width=\"640\" height=\"65\" src=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/metallb-pods.png?resize=640%2C65&#038;ssl=1\" alt=\"\" class=\"wp-image-3298\" srcset=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/metallb-pods.png?resize=1024%2C104&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/metallb-pods.png?resize=512%2C52&amp;ssl=1 512w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/metallb-pods.png?resize=768%2C78&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/metallb-pods.png?resize=1536%2C156&amp;ssl=1 1536w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/metallb-pods.png?resize=380%2C39&amp;ssl=1 380w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/metallb-pods.png?w=1685&amp;ssl=1 1685w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/metallb-pods.png?w=1280&amp;ssl=1 1280w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/figure>\n\n\n\n<p>We see a <strong>controller<\/strong> and 4 <strong>speaker<\/strong>s.  There is a <strong>speaker<\/strong> on each node that is involved in advertising mechanisms, with a single <strong>controller<\/strong> that is the piece responsible for actually allocating IP addresses.<\/p>\n\n\n\n<p>So <strong>MetalLB<\/strong> is running, but it isn&#8217;t doing anything useful at this point as we have not told it anything about the network environment it is running in. <\/p>\n\n\n\n<p>To have <strong>MetalLB<\/strong> do useful work, we need to provide some further information.<\/p>\n\n\n\n<p>Conveniently all of this is externalised in a separate <strong>ConfigMap<\/strong> object.  To change the configuration of <strong>MetalLB<\/strong>, all that is required is to update and reapply this <strong>ConfigMap<\/strong>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"configuring-metallb-layer2-or-bgp\">Configuring MetalLB: Layer2 or BGP<\/h2>\n\n\n\n<p>As described in the <a rel=\"noreferrer noopener\" href=\"https:\/\/metallb.org\/configuration\/\" data-type=\"URL\" data-id=\"https:\/\/metallb.org\/configuration\/\" target=\"_blank\">(excellent) documentation<\/a>, <strong>MetalLB<\/strong> can be configured to work in one of two ways (plus variations when taking into account &#8220;advanced&#8221; configuration):<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>Layer2<\/strong><\/td><td>The simplest to configure, you simply allocate a range of IP addresses for MetalLB to assign to services.  Should work with any router on any network.<br><br>In this configuration, your service IP addresses are in the same subnet as the rest of your network (assuming you have a single subnet).<\/td><\/tr><tr><td><strong>BGP<\/strong><\/td><td>BGP (or <a rel=\"noreferrer noopener\" href=\"https:\/\/www.fortinet.com\/resources\/cyberglossary\/bgp-border-gateway-protocol\" data-type=\"URL\" data-id=\"https:\/\/www.fortinet.com\/resources\/cyberglossary\/bgp-border-gateway-protocol\" target=\"_blank\">Border Gateway Protocol<\/a>) is more sophisticated with a number of advantages over <strong>Layer2<\/strong>.  But it is also more complicated to configure and requires a router that supports BGP (and not all do).<br><br>In this configuration, service IP addresses can be allocated on an entirely separate subnet to your main network.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>I opted for BGP and this is what my <strong>ConfigMap<\/strong> looks like:<\/p>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-yaml\">apiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: config\n  namespace: metallb-system\ndata:\n  config: |\n    peers:\n    - peer-address: 192.168.1.1\n      peer-asn: 64512\n      my-asn: 64512\n    address-pools:\n    - name: bgp\n      protocol: bgp\n      addresses:\n      - 10.10.10.10-10.10.10.250<\/code><\/pre>\n\n\n\n<p>The <strong>ConfigMap<\/strong> object itself is very straightforward.  It provides a single value (<strong>config<\/strong>) which is itself a YAML document that provides the configuration.<\/p>\n\n\n\n<p>To understand the configuration, this simplified schematic of the desired state of my (logical) network may help: <\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img decoding=\"async\" loading=\"lazy\" width=\"640\" height=\"298\" src=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-network.png?resize=640%2C298&#038;ssl=1\" alt=\"\" class=\"wp-image-3293\" srcset=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-network.png?w=692&amp;ssl=1 692w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-network.png?resize=512%2C238&amp;ssl=1 512w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-network.png?resize=380%2C177&amp;ssl=1 380w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/figure>\n\n\n\n<p>USG is my <strong>UniFi Security Gateway<\/strong> router.<\/p>\n\n\n\n<p>As mentioned before, I already have a number of VLANS&#8217;s configured in my router configuration, though not all of them are shown, for simplicity.<\/p>\n\n\n\n<p>The three of interest here are:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table><tbody><tr><td><strong>Subnet<\/strong><\/td><td><strong>Domain<\/strong><\/td><td><strong>Purpose\/Description<\/strong><\/td><\/tr><tr><td><strong>192.168.1.0\/24<\/strong><\/td><td>.local<\/td><td>Used for my primary networking needs. This is where my dev workstations hang out, for example.<\/td><\/tr><tr><td><strong>10.10.0.0\/24<\/strong><\/td><td>.k8s<\/td><td>The subnet for the VM&#8217;s in my <strong>Kubernetes Cluster<\/strong>.<br><br><strong>NOTE: <\/strong>This is <strong>NOT<\/strong> the subnet of the <strong>pod network<\/strong> that runs <em>inside<\/em> the cluster.  The pod network is entirely private &#8211; the wider network, including the USG, knows nothing of that network.<\/td><\/tr><tr><td><strong>10.10.10.0\/24<\/strong><\/td><td>.service<\/td><td>This is the new subnet for (external) IP&#8217;s allocated to services running in the <strong>Kubernetes Cluster<\/strong>, that will be managed by the <strong>MetalLB<\/strong> <strong>LoadBalancer<\/strong>.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background\">To be clear: The schematic above is a <strong>logical<\/strong> representation of these networks.  <strong>MetalLB<\/strong> itself runs <em>inside<\/em> the Kubernetes Cluster remember.<\/p>\n\n\n\n<p>Returning to the <strong>ConfigMap<\/strong>.<\/p>\n\n\n\n<p>The <strong>peer-address<\/strong> (from the MetalLB) perspective, is the gateway for the primary network on the USG, hence <strong>192.168.1.1<\/strong>.<\/p>\n\n\n\n<p>The two <strong>ASN<\/strong>s (Autonomous System Numbers) are ID&#8217;s for each peer in the relationship (&#8220;me&#8221; and &#8220;the other peer&#8221;).  You will notice that they are the same for reasons that I honestly do not understand.  I instinctively felt they should be different but this did not seem to work and all the guidance I found used the same ASN for each peer, and this works, so&#8230; \u00af\\_(\u30c4)_\/\u00af<\/p>\n\n\n\n<p>Finally, the pool of IP addresses is defined, along with the fact that the BGP protocol is in use.<\/p>\n\n\n\n<p>With this <strong>ConfigMap<\/strong> applied the <strong>MetalLB LoadBalancer<\/strong> will spring into life and start allocating External IP addresses to <strong>Services<\/strong> that require them (i.e. of type <strong>LoadBalancer<\/strong>).<\/p>\n\n\n\n<p>But this isn&#8217;t enough.<\/p>\n\n\n\n<p>When using <strong>BGP<\/strong>, each peer in the relationship needs to know about the other.  We&#8217;ve configured <strong>MetalLB<\/strong> with the required knowledge of the UniFi Security Gateway (USG).  But the USG also needs to be configured with knowledge of MetalLB.  Simply defining the network on the USG is not enough.<\/p>\n\n\n\n<p>The USG does not know which hosts are actually allocated IP&#8217;s on that network &#8211; only MetalLB knows that.  USG knows that the network exists and now needs to be told that traffic destined for that network needs to be sent to MetalLB.<\/p>\n\n\n\n<p>Unlike the creation of networks, the USG has no UI for BGP configuration.  Fortunately, configuring BGP on the USG is almost as straightforward as configuring MetalLB.<\/p>\n\n\n\n<p>But it will first help to understand a little about how such configuration is applied.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"unifi-device-provisioning\">UniFi Device Provisioning<\/h2>\n\n\n\n<p>In a UniFi network, there is a <strong><a href=\"https:\/\/blog.linitx.com\/ubiquiti-unifi-controller-how-what\/\" data-type=\"URL\" data-id=\"https:\/\/blog.linitx.com\/ubiquiti-unifi-controller-how-what\/\" target=\"_blank\" rel=\"noreferrer noopener\">Controller<\/a><\/strong> which &#8211; among other things &#8211; holds the network configuration and provides a pretty slick UI for managing (most aspects of) the network, to an advanced level.  The <strong>Controller<\/strong> software is free and can be run on any computer on the network.<\/p>\n\n\n\n<p>In my case, I run <a rel=\"noreferrer noopener\" href=\"https:\/\/store.ui.com\/collections\/unifi-accessories-cloud-key\/products\/unifi-cloudkey-plus\" data-type=\"URL\" data-id=\"https:\/\/store.ui.com\/collections\/unifi-accessories-cloud-key\/products\/unifi-cloudkey-plus\" target=\"_blank\">a UniFi CloudKey device<\/a> which is a Linux system in a convenient, efficient and small form-factor that draws all the power it needs over PoE.<\/p>\n\n\n\n<p>All UniFi managed devices (WiFi access points, switches, cameras, etc) go through a process called &#8220;provisioning&#8221; when booting up.  This provisioning process involves loading their configuration from the <strong>Controller<\/strong>.<\/p>\n\n\n\n<p>In addition to that configuration which is possible through the UI, the Controller also provides additional configuration to specific devices from pre-defined files.<\/p>\n\n\n\n<p>So, if you know what you are doing, you can upload configuration files to the prescribed location on the <strong>Controller<\/strong> to be provided to the requisite devices when they are provisioned.<\/p>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background\">It is important to do things this way.<br><br>Everything that can be achieved through these configuration files can also usually be performed by issuing specific commands on each device (over an SSH session), but almost all configuration changes made in this way will be lost when that device reboots and is re-provisioned.<br><br>&#8220;<em>Fine, so don&#8217;t reboot them<\/em>&#8220;.<br><br>Power outages happen and Ubiquiti are constantly pushing out firmware updates, which also require a reboot and\/or re-provisioning after they are applied.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"usg-configuration\">USG Configuration<\/h2>\n\n\n\n<p>Additional configuration for the <strong>UniFi Security Gateway<\/strong> is held in a file called <mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-vivid-purple-color\">config.gateway.json<\/mark>.  The location of this file on the <strong>Controller<\/strong> depends on the controller itself (it is different for a CloudKey vs a self-hosted installation on a PC, for example).  The name of the file is also prescribed: specific files are provided to specific devices during provisioning.<\/p>\n\n\n\n<p>As well as the BGP configuration, this file is also where static host-IP allocations can be made.<\/p>\n\n\n\n<p>In my case, this is where I establish the <strong>k8s-dashboard<\/strong> hostname for the IP allocated to the <strong>kubernetes-dashboard<\/strong> service in my cluster:<\/p>\n\n\n\n<pre class=\"wp-block-prismatic-blocks\"><code class=\"language-json\">{\n    &quot;protocols&quot;: {\n        &quot;bgp&quot;: {\n            &quot;64512&quot;: {\n                &quot;neighbor&quot;: {\n                    &quot;10.10.0.200&quot;: {\n                        &quot;remote-as&quot;: &quot;64512&quot;\n                    },\n                    &quot;10.10.0.101&quot;: {\n                        &quot;remote-as&quot;: &quot;64512&quot;\n                    },\n                    &quot;10.10.0.102&quot;: {\n                        &quot;remote-as&quot;: &quot;64512&quot;\n                    },\n                    &quot;10.10.0.103&quot;: {\n                        &quot;remote-as&quot;: &quot;64512&quot;\n                    }\n                },\n                &quot;parameters&quot;: {\n                    &quot;router-id&quot;: &quot;192.168.1.1&quot;\n                }\n            }\n        }\n    },\n    &quot;system&quot;: {\n        &quot;static-host-mapping&quot;: {\n            &quot;host-name&quot;: {\n                &quot;k8s-dashboard.service&quot;: {\n                    &quot;alias&quot;: [\n                        &quot;k8s-dashboard&quot;\n                    ],\n                    &quot;inet&quot;: [\n                        &quot;10.10.10.10&quot;\n                    ]\n                }\n            }\n        }\n    }\n}<\/code><\/pre>\n\n\n\n<p class=\"has-luminous-vivid-amber-background-color has-background\">The <strong>Controller<\/strong> UI allows for static IP&#8217;s to be assigned to MAC addresses but frustratingly does not provide a means of assigning an IP to a hostname (except where the corresponding host has a MAC address).<\/p>\n\n\n\n<p>Having already seen the <strong>MetalLB<\/strong> <strong>ConfigMap<\/strong>, this configuration should make some sense.<\/p>\n\n\n\n<p>There is a <strong>protocols<\/strong> object which in turn contains a <strong>bgp<\/strong> object.  This contains an object named <strong>64512<\/strong>, which you may recall is the <strong>ASN<\/strong> of both the USG and MetalLB.  From what follows we can determine that this object name is the <strong>ASN<\/strong> of the USG.<\/p>\n\n\n\n<p>Within this <strong>64512<\/strong> object are a <strong>neighbour<\/strong> object and a <strong>parameters<\/strong> object.<\/p>\n\n\n\n<p>The <strong>parameters<\/strong> object identifies the USG&#8217;s gateway IP address, while the <strong>neighbour<\/strong> object contains multiple objects, one for <em>each of the nodes<\/em> in the <strong>Kubernetes Cluster<\/strong> (each of which has a <strong>remote-as<\/strong> property of <strong>64512<\/strong> which we can deduce is the ASN of the remote peer &#8211; i.e. <strong>MetalLB<\/strong>).<\/p>\n\n\n\n<p>At first glance this may not make much sense&#8230; this configuration is supposed to be sending traffic to <strong><em>MetalLB<\/em><\/strong> to be routed to pods within the Kubernetes cluster.  But it appears to be sending traffic to all of the <em><strong>nodes<\/strong><\/em>.<\/p>\n\n\n\n<p>To be honest, I&#8217;m not 100% clear on this, but I <em>think<\/em> the answer is that it&#8217;s doing <strong><em>both<\/em><\/strong>.<\/p>\n\n\n\n<p>If I have understood things correctly, since we cannot know which node <strong>MetalLB<\/strong> is actually running on in the <strong>Cluster<\/strong>, we route traffic to all of the nodes.  In the case of three of those nodes, the traffic goes nowhere &#8211; a dead-end hop.  But on one of those nodes will be <strong>MetalLB<\/strong> which will then route the traffic accordingly.<\/p>\n\n\n\n<p>Again, this currently falls into the category of things I have learned just enough to get working and don&#8217;t pretend to understand deeply.<\/p>\n\n\n\n<p>In any event, after uploading this configuration file to the prescribed location using <strong>scp<\/strong>, the <strong>Controller<\/strong> UI can then be used to trigger the provisioning of the USG, so that this new configuration will be applied.<\/p>\n\n\n\n<p>And we&#8217;re done!<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"the-results\">The Results<\/h2>\n\n\n\n<p>We&#8217;ve already seen what success looks like in the <strong>Kubernetes Cluster<\/strong>.  We have a service of type <strong>LoadBalancer<\/strong> that has been assigned an external IP address:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" loading=\"lazy\" width=\"640\" height=\"68\" src=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=640%2C68&#038;ssl=1\" alt=\"\" class=\"wp-image-3237\" srcset=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=1024%2C109&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=512%2C55&amp;ssl=1 512w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=768%2C82&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=1536%2C164&amp;ssl=1 1536w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?resize=380%2C40&amp;ssl=1 380w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?w=1802&amp;ssl=1 1802w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-all-services-default.png?w=1280&amp;ssl=1 1280w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/figure>\n\n\n\n<p>On the USG side of things, we can check that the BGP configuration has been loaded successfully by logging in to the USG over SSH and issuing the command:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted has-cyan-bluish-gray-background-color has-background\" style=\"font-size:14px\">show ip bgp<\/pre>\n\n\n\n<p>If all has gone well, you will see something similar to:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" loading=\"lazy\" width=\"640\" height=\"108\" src=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/usg-show-ip.png?resize=640%2C108&#038;ssl=1\" alt=\"\" class=\"wp-image-3297\" srcset=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/usg-show-ip.png?resize=1024%2C173&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/usg-show-ip.png?resize=512%2C86&amp;ssl=1 512w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/usg-show-ip.png?resize=768%2C129&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/usg-show-ip.png?resize=1536%2C259&amp;ssl=1 1536w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/usg-show-ip.png?resize=2048%2C345&amp;ssl=1 2048w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/usg-show-ip.png?resize=380%2C64&amp;ssl=1 380w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/usg-show-ip.png?w=1280&amp;ssl=1 1280w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/usg-show-ip.png?w=1920&amp;ssl=1 1920w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/figure>\n\n\n\n<p>This identifies the four neighbours that we established, with one of them marked as &#8220;best&#8221;.<\/p>\n\n\n\n<p>Again, to be honest I&#8217;m not sure how this is determined to be the best.  I initially thought I would find that this was the node on which the <strong>MetalLB<\/strong> <strong>controller<\/strong> pod was running; that the <strong>controller<\/strong> was responding to whatever probes the BGP protocol uses to determine this &#8220;best&#8221; neighbour.  i.e. that <strong>worker1<\/strong> was running the controller and that the other nodes were the &#8220;dead-hops&#8221;.<\/p>\n\n\n\n<p>But it turns out that the <strong>controller<\/strong> is running on <strong>worker2<\/strong>.  So, another \u00af\\_(\u30c4)_\/\u00af<\/p>\n\n\n\n<p>The important thing is that when I <mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-luminous-vivid-orange-color\"><strong>nslookup<\/strong><\/mark> my <strong>k8s-dashboard.service<\/strong> host, I get the expected IP address:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" loading=\"lazy\" width=\"640\" height=\"74\" src=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/nslookup-k8sdashboard.png?resize=640%2C74&#038;ssl=1\" alt=\"\" class=\"wp-image-3300\" srcset=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/nslookup-k8sdashboard.png?resize=1024%2C119&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/nslookup-k8sdashboard.png?resize=512%2C59&amp;ssl=1 512w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/nslookup-k8sdashboard.png?resize=768%2C89&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/nslookup-k8sdashboard.png?resize=380%2C44&amp;ssl=1 380w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/nslookup-k8sdashboard.png?w=1430&amp;ssl=1 1430w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/nslookup-k8sdashboard.png?w=1280&amp;ssl=1 1280w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/figure>\n\n\n\n<p>And, unsurprisingly given that result, if I type <mark style=\"background-color:rgba(0, 0, 0, 0)\" class=\"has-inline-color has-luminous-vivid-orange-color\">http:\/\/k8s-dashboard.service<\/mark> in my browser, I am taken to the <strong>Kubernetes Dashboard<\/strong>, can skip the signing-in process and see my cluster in all its glory:<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" loading=\"lazy\" width=\"640\" height=\"531\" src=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-dashboard-all-workloads-1.png?resize=640%2C531&#038;ssl=1\" alt=\"\" class=\"wp-image-3302\" srcset=\"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-dashboard-all-workloads-1.png?resize=1024%2C849&amp;ssl=1 1024w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-dashboard-all-workloads-1.png?resize=512%2C424&amp;ssl=1 512w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-dashboard-all-workloads-1.png?resize=768%2C637&amp;ssl=1 768w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-dashboard-all-workloads-1.png?resize=314%2C260&amp;ssl=1 314w, https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/k8s-dashboard-all-workloads-1.png?w=1326&amp;ssl=1 1326w\" sizes=\"(max-width: 640px) 100vw, 640px\" data-recalc-dims=\"1\" \/><\/figure>\n\n\n\n<p>Success!<\/p>\n\n\n\n<p>As I deploy additional services into the <strong>Cluster<\/strong>, I shall need to do a little housekeeping to add additional static hostnames to my USG configuration for the services exposed via the MetalLB <strong>LoadBalancer<\/strong>.<\/p>\n\n\n\n<p>But that&#8217;s a small price to pay and something I could probably automate if it becomes too onerous. \ud83d\ude42<\/p>\n\n\n\n<h2 class=\"wp-block-heading\" id=\"in-summary\">In Summary<\/h2>\n\n\n\n<p>That brings me to the end of this series on establishing a <strong>Kubernetes Cluster<\/strong> lab.<\/p>\n\n\n\n<p>I appreciate that not all of it may be relevant to you in a different environment &#8211; particularly the UniFi networking aspects.  Hopefully, by sharing my goals and the steps to achieve them, you may have gained some insights &#8211; and perhaps some inspiration &#8211; for your own projects.<\/p>\n\n\n\n<p>Do you have a Kubernetes Lab?<\/p>\n\n\n\n<p>Do you run Kubernetes in the cloud?<\/p>\n\n\n\n<p>Do you do\/have you done <em>both<\/em>?<\/p>\n\n\n\n<p>I&#8217;d love to hear your experiences in the comments.<\/p>\n\n\n\n<p>Or perhaps you are wondering &#8220;<em>Why go to all this trouble?<\/em>&#8221; \ud83d\ude42<\/p>\n\n\n\n<p>On that last point, I have some projects lined up which will involve deployments into that cluster and may blog about those, which may help sell the idea.<\/p>\n","protected":false},"excerpt":{"rendered":"<p><span class=\"rt-reading-time\" style=\"display: block;\"><span class=\"rt-label rt-prefix\">[Estimated Reading Time: <\/span> <span class=\"rt-time\">10<\/span> <span class=\"rt-label rt-postfix\">minutes]<\/span><\/span> In the previous post we installed the Kubernetes Dashboard and saw how we could access the dashboard service which itself runs in the Kubernetes cluster. We also saw how clunky that was and I promised a better way, which we turn to now.<\/p>\n","protected":false},"author":2,"featured_media":3111,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"jetpack_post_was_ever_published":false,"jetpack_publicize_message":"The concluding post in my short series on establishing a bare metal Kubernetes cluster: exposing a service to my network with a friendly hostname.","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":[]},"categories":[4,321,1],"tags":[362,352,360,363,361],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/kubernetes-banner.png?fit=1171%2C416&ssl=1","jetpack_shortlink":"https:\/\/wp.me\/p1TKYv-R5","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":3148,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/3148\/","url_meta":{"origin":3291,"position":0},"title":"Kubernetes @ Home: PowerShell VM&#8217;s","date":"01 Feb 2022","format":false,"excerpt":"Following the Vagrant experiment (reminding me of a Bill Bailey metaphor... \"a long walk down a windy beach to a cafe that was closed\") I next set about automating my VM creation using PowerShell, with greater success. Though still not perfect, the final gap to full automation is something I\u2026","rel":"","context":"In &quot;automation&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/kubernetes-banner.png?fit=1171%2C416&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":3233,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/3233\/","url_meta":{"origin":3291,"position":1},"title":"Kubernetes @ Home: Dashboard","date":"07 Feb 2022","format":false,"excerpt":"At the end of the previous post in this series, we reached the point where I had my Kubernetes cluster up and running, including a Dashboard service. As I mentioned, this Dashboard is not part of a default installation. Taking a look at how I got this up and running\u2026","rel":"","context":"In &quot;Delphi&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/kubernetes-banner.png?fit=1171%2C416&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":3076,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/3076\/","url_meta":{"origin":3291,"position":2},"title":"A Kubernetes Cluster @ Home","date":"26 Jan 2022","format":false,"excerpt":"I set myself a new challenge for the New Year: Stand Up a Kubernetes cluster of my own. At home. I have reasons (long term costs being [potentially] lower than cloud-hosted alternatives being just one of them) but honestly since Kubernetes is a dominant presence in my day-to-day work these\u2026","rel":"","context":"In &quot;Delphi&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/kubernetes-banner.png?fit=1171%2C416&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":3099,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/3099\/","url_meta":{"origin":3291,"position":3},"title":"Kubernetes @ Home: Vagrant Nodes","date":"27 Jan 2022","format":false,"excerpt":"The first significant practical step in establishing my Bare Metal Kubernetes cluster was to provision 4 VM's. At this point it is worth mentioning that actually installing Kubernetes is almost trivial once you have the necessary hardware in place. Since my 'hardware' was going to be virtual, this meant that\u2026","rel":"","context":"In &quot;automation&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/kubernetes-banner.png?fit=1171%2C416&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":2983,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/2983\/","url_meta":{"origin":3291,"position":4},"title":"Duget &#8211; PUSH-ing Packages","date":"25 Sep 2019","format":false,"excerpt":"We now know a little more about this duget thing, and have seen how to create a package. But a package cannot be consumed 'in situ' - it must be made available via a feed. Which brings us to the PUSH command.NOTE: Don\u2019t worry, I have my priorities straight. This\u2026","rel":"","context":"In &quot;automation&quot;","img":{"alt_text":"","src":"https:\/\/i0.wp.com\/www.deltics.co.nz\/blog\/wp-content\/uploads\/duget-warriors.png?fit=929%2C256&ssl=1&resize=350%2C200","width":350,"height":200},"classes":[]},{"id":321,"url":"https:\/\/www.deltics.co.nz\/blog\/posts\/321\/","url_meta":{"origin":3291,"position":5},"title":"Delphi 2009 Trial &#8211; First Impressions","date":"11 Sep 2008","format":false,"excerpt":"I imagine the news has spread like wildfire - the Delphi 2009 Trial Edition is now available for download!\u00a0 I got me one, and these are my initial impressions. 1. The Installation The installer itself is a small downloader stub, so the initial download is very quick.\u00a0 During installation of\u2026","rel":"","context":"In &quot;Delphi&quot;","img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/3291"}],"collection":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/comments?post=3291"}],"version-history":[{"count":7,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/3291\/revisions"}],"predecessor-version":[{"id":3305,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/posts\/3291\/revisions\/3305"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/media\/3111"}],"wp:attachment":[{"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/media?parent=3291"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/categories?post=3291"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.deltics.co.nz\/blog\/wp-json\/wp\/v2\/tags?post=3291"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}