a6b2a95b7e
* Test cluster using molecule * Fix detection of first control node * Include --flannel-iface and --node-ip as k3s arguments * Store logs of k3s-init.service as GitHub job artifacts
119 lines
3.9 KiB
Markdown
119 lines
3.9 KiB
Markdown
# Automated build of HA k3s Cluster with `kube-vip` and MetalLB
|
|
|
|
|
|
|
|
This playbook will build an HA Kubernetes cluster with `k3s`, `kube-vip` and MetalLB via `ansible`.
|
|
|
|
This is based on the work from [this fork](https://github.com/212850a/k3s-ansible) which is based on the work from [k3s-io/k3s-ansible](https://github.com/k3s-io/k3s-ansible). It uses [kube-vip](https://kube-vip.chipzoller.dev/) to create a load balancer for control plane, and [metal-lb](https://metallb.universe.tf/installation/) for its service `LoadBalancer`.
|
|
|
|
If you want more context on how this works, see:
|
|
|
|
📄 [Documentation](https://docs.technotim.live/posts/k3s-etcd-ansible/) (including example commands)
|
|
|
|
📺 [Video](https://www.youtube.com/watch?v=CbkEWcUZ7zM)
|
|
|
|
## 📖 k3s Ansible Playbook
|
|
|
|
Build a Kubernetes cluster using Ansible with k3s. The goal is easily install a HA Kubernetes cluster on machines running:
|
|
|
|
- [X] Debian
|
|
- [X] Ubuntu
|
|
- [X] CentOS
|
|
|
|
on processor architecture:
|
|
|
|
- [X] x64
|
|
- [X] arm64
|
|
- [X] armhf
|
|
|
|
## ✅ System requirements
|
|
|
|
- Deployment environment must have Ansible 2.4.0+. If you need a quick primer on Ansible [you can check out my docs and setting up Ansible](https://docs.technotim.live/posts/ansible-automation/).
|
|
- `server` and `agent` nodes should have passwordless SSH access, if not you can supply arguments to provide credentials `--ask-pass --ask-become-pass` to each command.
|
|
|
|
## 🚀 Getting Started
|
|
|
|
### 🍴 Preparation
|
|
|
|
First create a new directory based on the `sample` directory within the `inventory` directory:
|
|
|
|
```bash
|
|
cp -R inventory/sample inventory/my-cluster
|
|
```
|
|
|
|
Second, edit `inventory/my-cluster/hosts.ini` to match the system information gathered above
|
|
|
|
For example:
|
|
|
|
```ini
|
|
[master]
|
|
192.168.30.38
|
|
192.168.30.39
|
|
192.168.30.40
|
|
|
|
[node]
|
|
192.168.30.41
|
|
192.168.30.42
|
|
|
|
[k3s_cluster:children]
|
|
master
|
|
node
|
|
```
|
|
|
|
If multiple hosts are in the master group, the playbook will automatically set up k3s in [HA mode with etcd](https://rancher.com/docs/k3s/latest/en/installation/ha-embedded/).
|
|
|
|
This requires at least k3s version `1.19.1` however the version is configurable by using the `k3s_version` variable.
|
|
|
|
If needed, you can also edit `inventory/my-cluster/group_vars/all.yml` to match your environment.
|
|
|
|
### ☸️ Create Cluster
|
|
|
|
Start provisioning of the cluster using the following command:
|
|
|
|
```bash
|
|
ansible-playbook site.yml -i inventory/my-cluster/hosts.ini
|
|
```
|
|
|
|
After deployment control plane will be accessible via virtual ip-address which is defined in inventory/group_vars/all.yml as `apiserver_endpoint`
|
|
|
|
### 🔥 Remove k3s cluster
|
|
|
|
```bash
|
|
ansible-playbook reset.yml -i inventory/my-cluster/hosts.ini
|
|
```
|
|
|
|
>You should also reboot these nodes due to the VIP not being destroyed
|
|
|
|
## ⚙️ Kube Config
|
|
|
|
To copy your `kube config` locally so that you can access your **Kubernetes** cluster run:
|
|
|
|
```bash
|
|
scp debian@master_ip:~/.kube/config ~/.kube/config
|
|
```
|
|
|
|
### 🔨 Testing your cluster
|
|
|
|
See the commands [here](https://docs.technotim.live/posts/k3s-etcd-ansible/#testing-your-cluster).
|
|
|
|
### Troubleshooting
|
|
|
|
Be sure to see [this post](https://github.com/techno-tim/k3s-ansible/discussions/20) on how to troubleshoot common problems
|
|
|
|
### Testing the playbook using molecule
|
|
|
|
This playbook includes a [molecule](https://molecule.rtfd.io/)-based test setup.
|
|
It is run automatically in CI, but you can also run the tests locally.
|
|
This might be helpful for quick feedback in a few cases.
|
|
You can find more information about it [here](molecule/README.md).
|
|
|
|
## Thanks 🤝
|
|
|
|
This repo is really standing on the shoulders of giants. To all those who have contributed.
|
|
|
|
Thanks to these repos for code and ideas:
|
|
|
|
- [k3s-io/k3s-ansible](https://github.com/k3s-io/k3s-ansible)
|
|
- [geerlingguy/turing-pi-cluster](https://github.com/geerlingguy/turing-pi-cluster)
|
|
- [212850a/k3s-ansible](https://github.com/212850a/k3s-ansible)
|