Skip to content

Security

Jump to bottom
Arun Sriraman edited this page Apr 16, 2019 · 3 revisions

List of Open Ports

Port / IP Protocol Notes
22 TCP ssh
2379 TCP etcd (master machine only)
2380 TCP etcd (master machine only)
6443 TCP kubernetes-api-server (master machine only)
8472 UDP flannel vxlan backend (kernel default for encap packets)
9202 TCP exporter (prometheus metrics exporter service)
10250 TCP kubelet (secure kubelet port)
10255 TCP kubelet (readonly port)
10256 TCP kube-proxy (default port)
224.0.0.18/32 Ingress & Egress Keepalived VRRP multicast address

Certificates & Keys

Klusterkit automatically creates self-signed CA certs for Kubernetes, etcd and front-proxy if user does not specify them explicitly. Certs can be explicitly specified by the user during cluster create using the following command line flags:

 --apiserver-ca-cert string     The API Server CA certificate. Used to sign kubelet certificate requests and verify client certificates.
 --apiserver-ca-key string      The API Server CA certificate key.
 --etcd-ca-cert string          The etcd CA certificate. Used to sign and verify client and peer certificates.
 --etcd-ca-key string           The etcd CA certificate key.
 --front-proxy-ca-cert string   The front proxy CA certificate. Used to verify client certificates on incoming requests.
 --front-proxy-ca-key string    The front proxy CA certificate key.

As of cctl v0.0.5, if none of these are specified, cctl will create a single CA cert and key pair that will be used for all three of the components (k8s, etcd and front-proxy). Klusterkit uses a public-private key pair that is used to sign service tokens. This is also auto-created by cctl if not specified during cluster creation. This key can be specified by using the following command line flags:

  --sa-private-key string          Location of file containing private key used for signing service account tokens
  --sa-public-key string           Location of file containing public key used for signing service account tokens

Users & Files

Klusterkit requires superuser/root privileges for use. This is to secure the cluster state file stored on disk along with other related files, such as admin kubeconfig. Compromise of a self service (standard) user account on a machine should not compromise the k8s cluster & its management plane. As a security measure, the toolkit writes files to directories owned by root and with file permissions set to only be accessible by root. Thus, a standard user or a privileged user with root permissions can access the toolkit as well as these files using sudo. The following list describes the files that the toolkit stores on disk, along with a description:

  • cctl-state.yaml (/etc/) : Stores all of the cluster & machine metadata including ca certs, tokens, configuration and current state. This is created when the first create command is executed (cctl create credential / cctl create cluster)
  • admin.conf (/etc/kubernetes/) : Created when a machine is added to the cluster (cctl create machine). This file contains the admin kubeconfig which is required to interact with the K8s API. An admin service token is used by this kubeconfig file.

Directories

  • /var/cache/cctl/* : Contains versioned cctl binaries
  • /var/cache/nodeadm/* : Contains versioned dependencies required by nodeadm (docker images & binaries - cni, kubelet, kubectl and others)
  • /var/cache/etcdadm/* : Contains versioned dependencies required by etcdeadm (etcd, etcdctl binaries)
  • /var/cache/ssh-provider/* : Contains etcdadm and nodadm versioned binaries These directories should contain versioned binaries of the specific Klusterkit tools as well as any docker image required by the toolkit for bootstrapping kubernetes on a machine.

Supporting files & directories

  • /etc/etcd : Stores etcd related files. Contains .env files with environment variables that are required by etcd or etcdctl. etcd related certs is placed within the pki directory.
  • /etc/kubernetes : Kubernetes specific files (certs), configuration et.al.
  • /etc/nodeadm.yaml : Configuration file written by the cctl tool for consumption by nodeadm binary when bringing up kubernetes control plane on a machine (result of cctl create machine) systemd unit and configuration files
  • keepalived.* (/etc/systemd/system/) : The keepalived.conf configuration file is written by nodeadm on all machines that are masters. This file provides configuration to the keepalived daemon that is run for Virtual IP service for the k8s api server. The directly also contains the systemd unit file that controls the lifecycle of keepalived (run as a docker container in privileged mode)
  • kubelet.* (/etc/systemd/system/) : kubelet systemd unit file
  • etcd.* (/etc/systemd/system/) : etcd systemd unit file
Clone this wiki locally