Skip to main content

Enforcing Policies

Policies allow you to customize admission control and extend vCluster Platform. As vCluster Platform internally uses Kubernetes resources for anything (including UI requests), you can customize almost everything with policies.

The following use cases are possible with policies:

  • Custom validation of users, teams, spaces, clusters and other objects. For example, you can:
    • Deny creation, updating or deletion of resources only for specific users or teams
    • Deny security critical objects such as privileged pods in clusters
    • Ensure unique names, hosts or other properties within a cluster
  • Automatic mutation of created or updated resources. For example, you can:
    • Add a label or annotation which user or team created an object
    • Inject sidecar containers into a pod
    • Add groups, cluster account templates etc. automatically to a user
  • Automation of certain cluster tasks, such as:
    • Automatic creation, update or deletion of users, accounts, spaces etc. on a certain condition
    • Sync certain resources within a cluster
    • Garbage collection of not needed resources

Installing JsPolicy​

Policies are based on jsPolicy and you will need to install the jsPolicy app into the cluster if you want to use policies. JsPolicy is a recommended app in vCluster Platform and can be installed via the vCluster Platform UI.

JsPolicy
Installing jsPolicy via the vCluster Platform UI

Alternatively, you can also install jsPolicy with helm.

Creating policies​

Policies can be created either through the vCluster Platform UI or through directly creating JsPolicy objects in the cluster. For a more extensive documentation how to write policies, please check the jsPolicy docs.

Go to the Clusters > Policies view and click on 'Add Policy'.

Create Policy
Create a new JsPolicy

Metadata​

In the metadata section of the policy, you can set the name of the policy, which has to be in domain format (e.g. my-policy.jspolicy.com or any other domain you want), the annotations and labels of the policy object.

In the options section, you can choose from a variety of examples to start from or create your own from scratch. The following options exist:

Policy Type​

The type of the policy. For a more extensive explanation check the jsPolicy documentation. The type can be one of:

  • Validating: Validating policies are executed as part of Kubernetes requests after the execution of mutating policies. The objective of validating policies is to inspect the request and then to either deny or allow it.
  • Mutating: Mutating policies are executed as part of Kubernetes requests right after the API server performs authentication and authorization (RBAC). The objective of mutating policies is to change the payload (Kubernetes object) provided in a request, e.g. add/remove a label in the metadata.labels of a Kubernetes object that is being created.
  • Controller: Unlike mutating and validating policies, controller policies are not part of the lifecycle of a request to the Kubernetes API server. Controller policies are triggered by the Events that Kubernetes creates for each change of the cluster state in etcd, e.g. if you create a new Deployment or update an Ingress, Kubernetes creates an Event for this change. The objective of controller policies is to react to changes to your cluster's state.

Policy Scope​

The scope of the policy. This can be either *, Cluster or Namespaced and refers to the scope of Kubernetes resources. A Namespaced policy will only match resources that can be created within a namespace, while Cluster will only match cluster-wide resources such as ClusterRole etc.

Matching Resources​

The Kubernetes resources this policy should match. You can specify multiple resources here or a wildcard that matches all resources.

Matching Operations​

Operations are the operations the policy cares about and can be one or more of: CREATE, UPDATE, DELETE, CONNECT or *.

Policy Javascript​

This is the actual JavaScript payload that is executed if a policy matches a resource and operation. Please checkout the jsPolicy documentation and jsPolicy examples for more information about how the payload could look like.

After you have configured the policy, press 'Create' to add it to the cluster.

Deleting policies​

You can delete a policy either via the vCluster Platform UI or kubectl directly.

Go to the Clusters > Policies view and click on 'delete' in the actions column.

Delete Policy
Delete a JsPolicy

Policy Audit​

If a request was denied by a policy, it will write this violation into its policy log. You can view all policy violations in the Audit > Policy Violations view.

Policy Violations
View policy violations

This view shows information about the request that was denied, such as which user was responsible for the violation (if there was any), the action taken, the returned message etc.

Policy Examples​

This section holds several examples for useful policies within vCluster Platform. For more examples and explanations checkout the jsPolicy documentation.

You can apply these examples by creating a policy.yaml and running kubectl apply -f policy.yaml in the target cluster.

Deny Space Owned By Team​

This policy denies spaces that are owned by teams.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-teams-spaces.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["spaces"]
  javascript: |
    // Do not allow creation of spaces that have a team as owner
    if (request.object?.spec?.account) {
        const account = get("Account", "config.kiosk.sh/v1alpha1", request.object.spec.account);
        if (account?.metadata?.labels?.["loft.sh/team"]) {
            deny("Teams cannot own spaces");
        }
    }

Deny Pods of User​

This policy will deny pod creation only for a single vCluster Platform user called test.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-user-pods.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["pods"]
  javascript: |
    // Do not allow the user "test" to create pods anywhere
    if (request.userInfo?.groups?.includes("loft:user:test")) {
        deny("Sorry test, but you are not allowed to create pods in this cluster");
    }

Deny Resources in default Namespace​

This policy will deny all new resources that should be created in the default namespace.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-default-namespace.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["*"]
  scope: Namespaced
  javascript: |
    if (request.namespace === "default") {
        deny("Creation of resources within the default namespace is not allowed!");
    }

Deny Privileged Pods​

This policy will deny privileged pods.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-privileged-pods.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["pods"]
  javascript: |
    const containers = request.object.spec?.containers || [];
    const initContainers = request.object.spec?.initContainers || [];

    [...containers, ...initContainers].forEach(container => {
        if (container.securityContext?.privileged) {
            deny("spec.containers[*].securityContext.privileged is not allowed")
        }
    })

Deny Ingress Classes​

This policy will deny all ingress classes except nginx.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "deny-ingress-classes.resource.example"
spec:
  operations: ["CREATE", "UPDATE"]
  resources: ["ingresses"]
  javascript: |
    // ingress class can be set via annotation "kubernetes.io/ingress.class" or through spec.ingressClassName.
    const allowedIngressClasses = ["nginx"];
    const ingressClasses = [request.object.metadata?.annotations?.["kubernetes.io/ingress.class"], request.object.spec.ingressClassName];
    const notAllowed = ingressClasses.filter(ingressClass => ingressClass && !allowedIngressClasses.includes(ingressClass));
    if (notAllowed.length > 0) {
      deny(`ingress class ${notAllowed[0]} is not allowed`);
    }

Allow Space only if Part of Team​

This policy will only allow space creation if the user is part of team 'admins'.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "allow-space-creation.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["spaces"]
  javascript: |
    // Allow space creation only if part of team "admins"
    if (!request.userInfo?.groups?.includes("loft:team:admins")) {
        deny("Sorry you cannot create spaces since you are not part of the admins team");
    }

Set User as Annotation On New Object​

This policy will set the user that has created it on a new kubernetes object.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "set-user-as-annotation.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["*"]
  scope: Namespaced
  type: Mutating
  javascript: |
    // Set the current user as an annotation on the object
    const allAccounts = list("Account", "config.kiosk.sh/v1alpha1");

    // Find account of the user by subject
    const userAccount = allAccounts.find(account => {
        // flatten account subjects
        const subjects = account.spec?.subjects?.map(subject => subject.name) || [];
        return subjects.includes(request.userInfo.username);
    });
    if (userAccount) {
        if (!request.object.metadata) {
            request.object.metadata = {};
        }
        if (!request.object.metadata.annotations) {
            request.object.metadata.annotations = {};
        }
        request.object.metadata.annotations["loft.sh/created-by-user"] = userAccount.metadata?.labels?.["loft.sh/user"];
        mutate(request.object);
    }

Inject Sidecar Container​

This policy will inject a sidecar container into pods that have an annotation "inject-side-car".

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "inject-sidecar.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["pods"]
  type: Mutating
  javascript: |
    // if the annotation inject-side-car: 'true' is set on the pod we inject a sidecar
    if (request.object.metadata?.annotations?.["inject-side-car"] === "true") {
      if (!request.object.spec.containers) {
        request.object.spec.containers = [];
      }

      // add sidecar
      request.object.spec.containers.push({
        name: "injected-container",
        image: "busybox",
        command: ["sleep", "3600"]
      });

      mutate(request.object);
    }

Unique Ingress Hosts​

This policy will ensure that ingress hosts are unique in the cluster.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "unique-ingress-hosts.resource.example"
spec:
  operations: ["CREATE", "UPDATE"]
  resources: ["ingresses"]
  type: Mutating
  javascript: |
    // iterate over all ingresses in cluster and check for same hosts
    list("Ingress", "networking.k8s.io/v1").forEach(ingress => {
      // don't check self on UPDATE
      if (ingress.metadata.name === request.name && ingress.metadata.namespace === request.namespace) {
        return;
      }

      ingress.spec.rules?.forEach(rule => {
        request.object.spec.rules?.forEach(objRule => {
          if (rule.host === objRule.host) {
            deny(\`ingress ` + "${ingress.metadata.namespace}/${ingress.metadata.name} already uses host ${rule.host}" + `\`);
          }
        })
      })
    })

Sync Secrets​

This policy will sync secrets that have the labels owner-name and owner-namespace with a parent secret that has the label sync. On a change to a parent or child secret, the controller policy will evaluate if an update is necessary and update the child secret if it differs from the parent secret.

apiVersion: policy.jspolicy.com/v1beta1
kind: JsPolicy
metadata:
  name: "sync-secrets.resource.example"
spec:
  operations: ["CREATE"]
  resources: ["secrets"]
  type: Controller
  javascript: |
    // this policy will sync changes to a secret to all other secrets in the cluster
    // that have labels in the form of: {"owner-name": SECRET, "owner-namespace": NAMESPACE}

    // is this a parent or child secret?
    const labels = request.object.metadata?.labels;
    let syncSecret = undefined;
    if (labels?.["sync"] === "true") {
      syncSecret = request.object;
    } else if (labels?.["owner-name"] && labels?.["owner-namespace"]) {
      syncSecret = get("Secret", "v1", labels?.["owner-namespace"] + "/" + labels?.["owner-name"]);
    }

    // this is not a secret we should sync
    if (!syncSecret) {
      allow();
    }

    // list all owned secrets
    list("Secret", "v1", {
      labelSelector: \`owner-name=`+"${syncSecret.metadata.name},owner-namespace=${syncSecret.metadata.namespace}"+`\`
    }).forEach(secret => {
      // only update if data has actually changed
      if (JSON.stringify(secret.data) === JSON.stringify(syncSecret.data)) {
        return;
      }

      // sync the secret data
      secret.data = syncSecret.data;
      const updateResult = update(secret);
      if (!updateResult.ok) {
        // just requeue if we encounter an error
        requeue(updateResult.message);
      } else {
        print(\`Successfully synced secret `+"${secret.metadata.namespace}/${secret.metadata.name}"+`\`);
      }
    });

Why jsPolicy and not OPA / Kyverno?​

vCluster Platform does not require you to use jsPolicy for policies, and you can always use your own preferred solution. Every policy engine should work fine with vCluster Platform. However, if you want to use the policies view in vCluster Platform, this currently only works with jsPolicy.

We decided to integrate jsPolicy instead of another policy engine, because jsPolicy is the most flexible one that can cover the most use cases within vCluster Platform. Especially with controller policies and mutating webhooks that are not at all or only partly supported in other policy engines, jsPolicy has a huge advantage in terms of flexibility and expressiveness.