Enforcing Security and Compliance with Kubernetes Policy Engines

Introduction

Kubernetes Policy engines are specialized tools that help manage and enforce policies across your Kubernetes clusters. They act like guards, evaluating and validating resources before they're deployed, ensuring they comply with your defined rules and best practices. Their key functionalities include - Validation, Mutation and Compliance.

Validation: Policies define what's allowed and forbidden within the cluster. The engine checks newly created or updated Kubernetes resources against these policies, catching any potential issues before they cause harm.
Mutation: Some engines can go beyond validation and modify resources to comply with policies. This could involve adjusting security settings, resource limits, or even rejecting non-compliant resources altogether.
Compliance: By enforcing policies consistently across your cluster, engines help ensure compliance with regulations or internal standards. This can be crucial for organizations dealing with sensitive data or operating in highly regulated environments.

Policies can be applied across a namespace or different pods based on a specific label in the cluster. These can block objects that could harm the cluster. Policies also enable users to build complex configurations that other tools like Ansible, Terraform, etc can’t. Some of the popular Kubernetes policy engines are OPA (Open Policy Agent), Kyverno and jsPolicy. Here are the basics of the three policy engines.

OPA (Open Policy Agent)

OPA is a CNCF graduate project. It is a general purpose policy agent, ie, it can also be used for use cases other than Kubernetes like microservice authorization, data source filtering, CI/CD pipelines and more. In OPA, the policies are written in a language called Rego, which is a query language that extends Datalog to support structured data models such as JSON. OPA can also report performance metrics in runtime.

One of the most important facts about OPA is that it enables decoupling. Meaning, it can decouple policies from infrastructure, service or application. In other words, people responsible for policy management can control policies separately without interfering with the service or application. OPA works by making decisions and not necessarily enforcing them and also, the policy decisions need not just be a “yes/no” or “allow/deny” like query inputs. Policies can have structured data as an output.

Here is an example of an OPA policy in Rego that restricts users from creating Kubernetes objects in the default namespace.

package k8s

default_namespace = "default"

deny[msg] {
    input.request.kind.kind == "Namespace"
    input.request.operation == "CREATE"
    input.request.namespace != default_namespace
    msg = "Creating namespaces is only allowed in the default namespace."
}

deny[msg] {
    input.request.operation == "CREATE"
    input.request.namespace == default_namespace
    msg = "Creating objects in the default namespace is not allowed."
}

This Rego policy has two rules:

The first rule (deny[msg]) denies the creation of any namespace in a namespace other than the default namespace.
The second rule (deny[msg]) denies the creation of any object in the default namespace.

OPA Using Gatekeeper

Usually in Kubernetes, OPA is not employed directly, instead it uses another project, Gatekeeper. Gatekeeper is a project that extends Open Policy Agent (OPA) to Kubernetes. In other words, Gatekeeper is built on top of OPA and provides a framework for managing and enforcing policies in a Kubernetes environment. It simplifies the integration of OPA with Kubernetes and enhances its usability for policy enforcement. Now, to employ the same example from above, to create a policy that restricts users from creating Kubernetes objects in the default namespace. But this time, using Gatekeeper and OPA.

For that first, we’ll have to create a ConstraintTemplate, apply the yaml file and then create a Constraint using this constraint template and apply the constraint also.

ConstraintTemplate

Create a ConstraintTemplate that defines the template for your constraint. Save this YAML as default-namespace-constraint-template.yaml

# default-namespace-constraint-template.yaml
apiVersion: templates.gatekeeper.sh/v1beta1
kind: ConstraintTemplate
metadata:
  name: defaultnamespace
spec:
  crd:
    spec:
      names:
        kind: DefaultNamespace
  targets:
    - target: admission.k8s.gatekeeper.sh
      rego: |
        package k8sdefaultnamespace

        violation[{"msg": msg}] {
          input.request.kind.kind == "Namespace"
          input.request.operation == "CREATE"
          input.request.namespace != "default"
          msg := "Creating namespaces is only allowed in the default namespace."
        }

        violation[{"msg": msg}] {
          input.request.operation == "CREATE"
          input.request.namespace == "default"
          msg := "Creating objects in the default namespace is not allowed."
        }

Now, apply the ConstraintTemplate using the kubectl apply command.

kubectl apply -f default-namespace-constraint-template.yaml

Constraint

Next, create a Constraint using the ConstraintTemplate. Save this YAML as default-namespace-constraint.yaml


# default-namespace-constraint.yaml
apiVersion: constraints.gatekeeper.sh/v1beta1
kind: DefaultNamespace
metadata:
  name: default-namespace
spec:
  match:
    kinds:
      - apiGroups: [""]
        kinds: ["Namespace"]
      - apiGroups: [""]
        kinds: ["Pod", "Service", "Deployment", "ReplicaSet", "StatefulSet"]
  parameters:
    message: "Creating objects in the default namespace is not allowed."

Now, apply the Constraint using the kubectl apply command.

kubectl apply -f default-namespace-constraint.yaml

Now, any attempt to create Kubernetes objects in the default namespace will be denied by the admission controller.

Kyverno

Kyverno is a Kubernetes native policy solution that employs a declarative management paradigm to construct policies for changing, validating, or generating resources or configurations. It's different from OPA because you don't have to write complex code. Kyverno uses YAML, so these policies are taken as Kubernetes resources and can be written without learning a new language. This makes it easy to view and process policy results.

Kyverno is easy to use with familiar tools like Git and kubectl. It's great for checking if things follow the rules you set when you create them in Kubernetes. It also uses tools like Cosign to check if images are safe and uses Grafana to expose and collect metrics from the cluster. It simplifies and consolidates policy distribution using a container registry (OCI registry).

Kyverno Policy Example

Here is an example of a Kyverno policy that restricts users from creating Kubernetes objects in the default namespace. Create a Kyverno policy in YAML format. Save the following content as restrict-default-namespace.yaml

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-default-namespace
spec:
  rules:
  - name: check-default-namespace
    match:
      exclude:
        namespaces:
        - "default"
    validate:
      message: "Creating objects in the default namespace is not allowed."
      pattern:
        exclude:
          namespace: "default"

Now, apply the policy using the kubectl apply command.

kubectl apply -f restrict-default-namespace.yaml

This policy checks if someone is trying to create any Kubernetes object in the "default" namespace and denies it with an error message. Now, any attempt to create Kubernetes objects in the default namespace will be denied by Kyverno.

jsPolicy

jsPolicy is a policy engine for Kubernetes that allows users to create policies using JavaScript or TypeScript. Managing policies with JavaScript is easier and more straightforward because many people are already familiar with the language and its tools. jsPolicy is different from other policy engines like Kyverno and OPA because it uses JavaScript, making it more flexible and easier to modify. It comes with an SDK for creating and packaging policies using npm packages.

One unique feature of jsPolicy is its ability to respond to Kubernetes events, allowing you to perform actions and validate or modify events using JavaScript. Unlike other platforms, jsPolicy solves the complexity of the Rego language used by OPA. You can use familiar tools like kubectl and the Kubernetes API with jsPolicy. It efficiently executes policies using prebuilt JavaScript sandboxes in your cluster, speeding up the process. The policy compiler turns your jsPolicy code into a bundle that runs in the sandbox to identify policy violations.

jsPolicy is developer-friendly, allowing you to leverage the entire JavaScript ecosystem for writing, testing, and maintaining policies. It provides a practical solution for those who prefer JavaScript over other languages in the Kubernetes environment.

jsPolicy Example

Unlike Kyverno or OPA, jsPolicy requires a separate installation process and the application involves both Helm and kubectl commands. First you’ll have to install jsPolicy into the Kubernetes cluster using Helm (which is the recommended method of installation) by running the following command:

helm install jspolicy jspolicy -n jspolicy --create-namespace --repo https://charts.loft.sh

This creates a dedicated namespace for jsPolicy and installs the necessary resources. Now Create a jsPolicy in YAML format and save the following content as restrict-default-namespace.jspolicy.yaml

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: restrict-default-namespace-js
spec:
  validationFailureAction: enforce
  rules:
    - name: check-default-namespace
      match:
        any:
          - resources:
              kinds:
                - "*"
      validate:
        message: "Creating resources in the default namespace is not allowed."
        rule: |-
          if (request.object.metadata.namespace == "default") {
            return { "valid": false, "message": errorMessage };
          } else {
            return { "valid": true };
          }

Now, apply the policy using the kubectl apply command.

kubectl apply -f restrict-default-namespace.jspolicy.yaml

Check if the jsPolicy is up by running the following command

kubectl get pods -n jspolicy

You should see a pod named jspolicy-controller running in the jspolicy namespace.

Now, any attempt to create a resource in the default namespace using kubectl, the request should be blocked, and you should see an error message indicating that creating resources in the default namespace is not allowed.

Conclusion

This blog discussed the concepts surrounding Kubernetes policy engines and compared three different Kubernetes policy engines: OPA, Kyverno, and jsPolicy.

Choosing the right policy engine depends on your priorities and preferences. If simplicity and direct control are your top concerns, and you're comfortable with JavaScript or TypeScript, jsPolicy might be the ideal fit. However, if you prefer a YAML-based approach and want to leverage your existing Kubernetes expertise, Kyverno offers a familiar and powerful option. Ultimately, the best engine for you will depend on finding the right balance between ease of use and flexibility.

Ultimately, there's no one-size-fits-all answer. Try out the engines that resonate with your preferences and technical expertise, and choose the one that best empowers you to secure and manage your Kubernetes clusters with confidence.