T1t0: /* HAProxy install */

2026-07-15T23:33:51Z

HAProxy install

← Older revision		Revision as of 23:33, 15 July 2026
Line 106:		Line 106:
	== HAProxy install ==		== HAProxy install ==

	HAProxy will be installed in any server to forward requests incoming from the external network load balancer to the ~~container's~~ services		HAProxy will be installed in any server to forward requests incoming from the external network load balancer to the internal LXC containers services

	In the follow example, HAProxy will forward external incoming requests on 6443 port to the Kubernetes api listening at 6443 port inside each of the ~~two~~ LXC containers of this example, roundrobin balancing the request between them:		In the follow example, HAProxy will forward external incoming requests on 6443 port to the Kubernetes api listening at 6443 port inside each of the LXC containers of this example, roundrobin balancing the request between them:

	<pre>		<pre>
Line 140:		Line 140:
	server c101 10.10.1.1:6443 check		server c101 10.10.1.1:6443 check
	server c102 10.10.1.2:6443 check		server c102 10.10.1.2:6443 check
			server c201 10.10.2.1:6443 check
			server c202 10.10.2.2:6443 check

	frontend ft_lxc_deploy		frontend ft_lxc_deploy
Line 148:		Line 150:
	server c101 10.10.1.1:30666 check		server c101 10.10.1.1:30666 check
	server c102 10.10.1.2:30666 check		server c102 10.10.1.2:30666 check
			server c201 10.10.2.1:30666 check
			server c202 10.10.2.2:30666 check

	frontend ft_lxc_gitolite		frontend ft_lxc_gitolite
Line 156:		Line 160:
	server c101 10.10.1.1:30022 check		server c101 10.10.1.1:30022 check
	server c102 10.10.1.2:30022 check		server c102 10.10.1.2:30022 check
			server c201 10.10.2.1:30022 check
			server c202 10.10.2.2:30022 check

	frontend ft_lxc_traefik_http		frontend ft_lxc_traefik_http
Line 164:		Line 170:
	server c101 10.10.1.1:30080 check		server c101 10.10.1.1:30080 check
	server c102 10.10.1.2:30080 check		server c102 10.10.1.2:30080 check
			server c201 10.10.2.1:30080 check
			server c202 10.10.2.2:30080 check

	frontend ft_lxc_traefik_https		frontend ft_lxc_traefik_https
Line 172:		Line 180:
	server c101 10.10.1.1:30443 check		server c101 10.10.1.1:30443 check
	server c102 10.10.1.2:30443 check		server c102 10.10.1.2:30443 check
			server c201 10.10.2.1:30443 check
			server c202 10.10.2.2:30443 check
	EOF		EOF
	systemctl restart haproxy		systemctl restart haproxy

T1t0: Created page with "= Architecture = Architecture used in this hands-on guide * name.com - for domain names * Oracle network load balancer - for load balancing request to servers * 2 Oracle ampere instances - as provisioned servers * HAProxy systemd daemon - in each server, to balancing requests from the load balancer to forward to the internal LXC containers * 2 LXC containers per server - to accommodate the Kubernetes cluster * LXC network - to provide connections between container..."

2026-07-15T22:14:46Z

Created page with "= Architecture = Architecture used in this hands-on guide * name.com - for domain names * Oracle network load balancer - for load balancing request to servers * 2 Oracle ampere instances - as provisioned servers * HAProxy systemd daemon - in each server, to balancing requests from the load balancer to forward to the internal LXC containers * 2 LXC containers per server - to accommodate the Kubernetes cluster * LXC network - to provide connections between container..."

New page

= Architecture =

Architecture used in this hands-on guide

* name.com - for domain names

* Oracle network load balancer - for load balancing request to servers

* 2 Oracle ampere instances - as provisioned servers

* HAProxy systemd daemon - in each server, to balancing requests from the load balancer to forward to the internal LXC containers

* 2 LXC containers per server - to accommodate the Kubernetes cluster

* LXC network - to provide connections between containers of all servers

* kubeadm - to setup the Kubernetes cluster

* Flannel - to setup the Kubernetes network

* Traefik - in global deploy mode to be in each LXC container, to receive requests from HAProxy to forward to the Kubernetes services

== static IP network definition ==

internal network CIDR

from https://www.davidc.net/sites/default/subnets/subnets.html?network=10.10.0.0&mask=16&division=1.0
<pre>
10.10.0.0/16 10.10.0.0 - 10.10.255.255 10.10.0.1 - 10.10.255.254 65534 Divide
</pre>

note about the nomenclature scheme for the 10.10.0.0 ip address
* the first two numbers (10.10.) are the prefix under that will be contained all the internal LXC resources
* the third number will be used to identify provisioned servers (0-255, 255 max)
* and forth number will be used to identify LXC containers accommodated by servers (1-254, 253 max theoretically, however we just want a handful of them per server, we can use 10.10.x.100 for the provisioned server, so the range will be 1-99, 98 max that is more than enough)

In this way, we can chose container names that remember both the parent server as well as the static IP used by the container

* ip table:

{| class="wikitable"
! resource name !! external network addr !! internal bridge addr
|-
! * server1 !! (unknown at this point) !! 10.10.1.100
|-
| c101 || 10.10.1.1 ||
|-
| c102 || 10.10.1.2 ||
|-
! * server2 !! (unknown at this point) !! 10.10.2.100
|-
| c201 || 10.10.2.1 ||
|-
| c202 || 10.10.2.2 ||
|}

== System cleanup ==

on the hosting machines, cleanup usefulness stuffs to avoid disk waste and provider's agents resource waste

on Debian, you can use '''apt-mark''' and '''apt-get autoremove --purge'''

Example useful for Jammy cleanup:

turn off phases updates, Ubuntu 24 specific stuff, see https://ubuntu.com/server/docs/explanation/software/about-apt-upgrade-and-phased-updates/#how-do-i-turn-off-phased-updates
<pre>
cat > /etc/apt/apt.conf.d/99-Phased-Updates << EOF
Update-Manager::Always-Include-Phased-Updates true;
APT::Get::Always-Include-Phased-Updates true;
EOF
</pre>

then
<pre>
apt-get update
apt-mark auto `apt-mark showmanual`
apt-get install apt apt-utils base-files base-passwd coreutils dash debconf dpkg grep gzip hostname libc6 libc-bin nano passwd procps psmisc readline-common sed sysvinit-utils tar
apt-get install bash binutils bsdutils bzip2 ca-certificates curl dialog dnsutils findutils iptables iputils-ping kmod less locales lsof man mount mutt ncal netcat-traditional nmap telnet tzdata util-linux
apt-get install busybox-static command-not-found conntrack cron dmidecode friendly-recovery hwdata hdparm info iputils-tracepath irqbalance jq linux-oracle linux-headers-oracle mtr-tiny net-tools netcat-openbsd netfilter-persistent rsync rsyslog socat sudo tcl tcpdump time ubuntu-minimal ubuntu-standard unzip
apt-get autoremove --purge
apt-get dist-upgrade
apt-get clean
shutdown -r now
</pre>

shutdown just to grant avoid surprises in the next future reboot

== LXC install ==

Note, we will use the vanilla LXC, not the Ubuntu LXD one

Follow [[Lxc]] instructions

== HAProxy install ==

HAProxy will be installed in any server to forward requests incoming from the external network load balancer to the container's services

In the follow example, HAProxy will forward external incoming requests on 6443 port to the Kubernetes api listening at 6443 port inside each of the two LXC containers of this example, roundrobin balancing the request between them:

<pre>
apt-get install haproxy
apt-get clean
cat > /etc/haproxy/haproxy.cfg << 'EOF'
global
log stdout format raw local0
chroot /var/lib/haproxy
user haproxy
group haproxy
daemon

defaults
log global
mode tcp
timeout connect 10s
timeout client 300s
timeout server 300s
timeout tunnel 300s
option tcp-check
tcp-check connect
balance roundrobin
default-server downinter 5s fastinter 1s inter 3s fall 2 rise 3

frontend ft_lxc_kubeapi
bind *:6443
default_backend bk_lxc_kubeapi

backend bk_lxc_kubeapi
server c101 10.10.1.1:6443 check
server c102 10.10.1.2:6443 check

frontend ft_lxc_deploy
bind *:30666
default_backend bk_lxc_deploy

backend bk_lxc_deploy
server c101 10.10.1.1:30666 check
server c102 10.10.1.2:30666 check

frontend ft_lxc_gitolite
bind *:30022
default_backend bk_lxc_gitolite

backend bk_lxc_gitolite
server c101 10.10.1.1:30022 check
server c102 10.10.1.2:30022 check

frontend ft_lxc_traefik_http
bind *:30080
default_backend bk_lxc_traefik_http

backend bk_lxc_traefik_http
server c101 10.10.1.1:30080 check
server c102 10.10.1.2:30080 check

frontend ft_lxc_traefik_https
bind *:30443
default_backend bk_lxc_traefik_https

backend bk_lxc_traefik_https
server c101 10.10.1.1:30443 check
server c102 10.10.1.2:30443 check
EOF
systemctl restart haproxy
systemctl status haproxy
</pre>

== Kubernetes install under LXC ==

Follow the Install section of [[Kubernetes]]

In one of the LXC machines with Kubernetes binaries installed, do
<pre>
LB_ENDPOINT=<INTERNAL_LOADBALANCER_IP>:6443
kubeadm init --v=5 --ignore-preflight-errors=NumCPU,Mem --cri-socket unix:/run/containerd/containerd.sock --pod-network-cidr=192.168.0.0/16 --control-plane-endpoint "${LB_ENDPOINT}" --upload-certs
mkdir -p $HOME/.kube; cp -a /etc/kubernetes/admin.conf $HOME/.kube/config
kubectl get nodes
</pre>

Fix kube proxy error inside LXD trying to change /proc/sys/stuffs
<pre>
kubectl get all -A
kubectl logs -n kube-system -l k8s-app=kube-proxy
kubectl -n kube-system get configmap kube-proxy -o yaml | sed 's/maxPerCore: null/maxPerCore: 0/' | sed 's/min: null/min: 0/' | kubectl apply -f -
kubectl -n kube-system delete pods -l k8s-app=kube-proxy
## wait pending state
kubectl get all -A
</pre>

Configure Kubernetes network
<pre>
mkdir flannel
VERSION="0.28.7"
curl -fsSL "https://github.com/flannel-io/flannel/releases/download/v${VERSION}/kube-flannel.yml" | sed 's:10.244.0.0/16:192.168.0.0/16:g' > flannel/flannel.yml
kubectl apply -f flannel/flannel.yml
## wait ready state
kubectl get nodes
</pre>

Obtain Kubernetes join command, all the nodes will be control-plane
<pre>
echo $(kubeadm token create --print-join-command) --v=5 --ignore-preflight-errors=NumCPU,Mem --control-plane --certificate-key $(kubeadm init phase upload-certs --upload-certs 2>/dev/null | sed -n '/Using certificate key/{n;p;}')
</pre>

On each other nodes to configure
<pre>
kubeadm join ...
mkdir -p $HOME/.kube; cp -a /etc/kubernetes/admin.conf $HOME/.kube/config
kubectl get nodes
</pre>

Untaint manager nodes to be able to accommodate services
<pre>
kubectl taint nodes --all node-role.kubernetes.io/control-plane-
</pre>

[[Category:linux]]
[[Category:debian]]
[[Category:container]]
[[Category:lxc]]
[[Category:kubernetes]]
[[Category:haproxy]]

Kubernetes HA Cluster using LXC - Revision history

T1t0: /* HAProxy install */