Skip to main content
Version: main 🚧

Self assessment guide - Control Plane Configuration

This section covers cluster-wide security areas, including:

  • Authentication and authorization mechanisms
  • Audit logging configuration

Assessment focus for vCluster: Key areas include verifying audit logging is enabled, and alternative mechanisms such as OIDC are used.

3.1 Authentication and Authorization​

3.1.1 Client certificate authentication should not be used for users (Manual)​

Result: WARN

Remediation: Alternative mechanisms provided by Kubernetes such as the use of OIDC should be implemented in place of client certificates.

3.1.2 Service account token authentication should not be used for users (Manual)​

Result: WARN

Remediation: Alternative mechanisms provided by Kubernetes such as the use of OIDC should be implemented in place of service account tokens.

3.1.3 Bootstrap token authentication should not be used for users (Manual)​

Result: WARN

Remediation: Alternative mechanisms provided by Kubernetes such as the use of OIDC should be implemented in place of bootstrap tokens.

3.2 Logging​

3.2.1 Ensure that a minimal audit policy is created (Automated)​

Result: PASS

Remediation: Follow the Kubernetes Documentation and create a config map with a minimal audit policy

audit-config-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: audit-config
  namespace: vcluster-my-vcluster
data:
  audit-policy.yaml: |
    apiVersion: audit.k8s.io/v1
    kind: Policy
    rules:
    - level: Metadata

Pass the below configuration as arguments to the API Server while creating the vCluster as:

vcluster.yaml
controlPlane:
  distro:
    k8s:
      enabled: true
      apiServer:
        extraArgs: 
        - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
  statefulSet:
    persistence:
      addVolumes:
      - name: audit-policy
        configMap:
          name: audit-config
      addVolumeMounts:
      - name: audit-policy
        mountPath: /etc/kubernetes

Audit: Create the vCluster using the above values file.

vcluster create my-vcluster -f vcluster.yaml --connect=false

Run the following command against the vCluster pod:

kubectl exec -n vcluster-my-vcluster my-vcluster-0 -c syncer -- ps -ef | grep kube-apiserver

Verify that the --audit-policy-file is set.

Expected Result:

'--audit-policy-file' is present

Returned Value:

26 root      0:08 /binaries/kube-apiserver --advertise-address=127.0.0.1 --service-cluster-ip-range=10.96.0.0/12 --bind-address=127.0.0.1 --allow-privileged=true --authorization-mode=RBAC --client-ca-file=/data/pki/client-ca.crt --enable-bootstrap-token-auth=true --etcd-servers=unix:///data/kine.sock --proxy-client-cert-file=/data/pki/front-proxy-client.crt --proxy-client-key-file=/data/pki/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/data/pki/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/data/pki/sa.pub --service-account-signing-key-file=/data/pki/sa.key --tls-cert-file=/data/pki/apiserver.crt --tls-private-key-file=/data/pki/apiserver.key --endpoint-reconciler-type=none --profiling=false --audit-policy-file=/etc/kubernetes/audit-policy.yaml

3.2.2 Ensure that the audit policy covers key security concerns (Manual)​

Result: WARN

Remediation: Review the audit policy provided for the cluster and ensure that it covers at least the following areas :-

  • Access to Secrets managed by the cluster. Care should be taken to only log Metadata for requests to Secrets, ConfigMaps, and TokenReviews, to avoid the risk of logging sensitive data.
  • Modification of pod and deployment objects.
  • Use of pods/exec, pods/portforward, pods/proxy and services/proxy.

For most requests, minimally logging at the Metadata level is recommended (the most basic level of logging). Consider modification of the audit policy in use on the cluster to include these items, at a minimum. A sample policy that satisfies this criteria is as below.

audit-config-configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: audit-config
  namespace: vcluster-my-vcluster
data:
  audit-policy.yaml: |
    apiVersion: audit.k8s.io/v1
    kind: Policy
    rules:
    - level: Metadata
      resources:
      - group: ""
        resources: ["secrets", "configmaps"]
      - group: "authentication.k8s.io"
        resources: ["tokenreviews"]
    - level: RequestResponse
      resources:
      - group: ""
        resources: ["pods"]
      - group: "apps"
        resources: ["deployments", "replicasets"]
      verbs: ["create", "update", "patch", "delete"]
    - level: RequestResponse
      resources:
      - group: ""
        resources: ["pods/exec", "pods/portforward", "pods/proxy", "services/proxy"]
    - level: Metadata
      omitStages:
      - RequestReceived