How-to GuidesSelf-hosting Guard

Deploying to Kubernetes

This reference guide will walk you through self-hosting Lakera Guard with Kubernetes (K8s).

A Lakera Guard pod requires access to:

  • two cache volume mounts
    1. /home/llm-guard/.local
    2. /home/llm-guard/.cache
  • a writable /tmp directory

Storage configuration

An ephemeral storage volume is recommended, but you should be able to use other storage layers, too:

1spec:
2  volumes:
3    - name: ephemeral
4      emptyDir:
5        sizeLimit: 500Mi
6
7  containers:
8    - name: guard
9      volumeMounts:
10        - name: ephemeral
11          mountPath: "/home/llm-guard/.local"
12        - name: ephemeral
13          mountPath: "/home/llm-guard/.cache"
14        - name: ephemeral
15          mountPath: "/tmp"

Security configuration

Disable root access, privilege escalation, and write access to the root filesystem and set the user and group IDs with the following securityContext configuration for the pod and the guard container.

Pod securityContext

1spec:
2  securityContext:
3    runAsNonRoot: true
4    runAsUser: 1000
5    runAsGroup: 1000
6    fsGroup: 1000

Container securityContext

1spec:
2  containers:
3    - name: guard
4      securityContext:
5        runAsNonRoot: true
6        runAsUser: 1000
7        runAsGroup: 1000
8        readOnlyRootFilesystem: true
9        allowPrivilegeEscalation: false
10        privileged: false

Liveness, readiness and startup probes

The readiness and liveness probes can be defined using the /readyz and /livez endpoints respectively. It’s recommended to have a higher failureThreshold value for the liveness probe than for the readiness one.

The readiness and liveness probes require the gunicorn workers to start. Because of that a startup probe is required. For that, you can use the /startupz endpoint.

1startupProbe:
2    httpGet:
3        path: /startupz
4        port: 8000
5    periodSeconds: 10
6    failureThreshold: 30
7readinessProbe:
8    httpGet:
9        path: /readyz
10        port: 8000
11    periodSeconds: 5
12    failureThreshold: 1
13livenessProbe:
14    httpGet:
15        path: /livez
16        port: 8000
17    periodSeconds: 5
18    failureThreshold: 3

Example deployment

Here is an example deployment configuration for a Lakera Guard pod:

1apiVersion: apps/v1
2kind: Deployment
3metadata:
4  name: guard
5  labels:
6    app: guard
7spec:
8  replicas: 1
9  selector:
10    matchLabels:
11      app: guard
12  template:
13    metadata:
14      labels:
15        app: guard
16    spec:
17      securityContext:
18         runAsNonRoot: true
19         runAsUser: 1000
20         runAsGroup: 1000
21         fsGroup: 1000
22      containers:
23      - name: guard
24        # `CONTAINER_PATH` is provided by Lakera to Enterprise self-hosting customers
25        image: $CONTAINER_PATH:stable
26        imagePullPolicy: IfNotPresent
27        volumeMounts:
28        - name: ephemeral
29          mountPath: "/home/llm-guard/.local"
30        - name: ephemeral
31          mountPath: "/home/llm-guard/.cache"
32        - name: ephemeral
33          mountPath: "/tmp"
34        securityContext:
35          runAsNonRoot: true
36          runAsUser: 1000
37          runAsGroup: 1000
38          readOnlyRootFilesystem: true
39          allowPrivilegeEscalation: false
40          privileged: false
41        env:
42          - name: LAKERA_GUARD_LICENSE
43            value: "<YOUR_LICENSE_KEY>" # Replace with your actual license key or use a secret
44        startupProbe:
45            httpGet:
46                path: /startupz
47                port: 8000
48            periodSeconds: 10
49            failureThreshold: 30
50        readinessProbe:
51            httpGet:
52                path: /readyz
53                port: 8000
54            periodSeconds: 5
55            failureThreshold: 1
56        livenessProbe:
57            httpGet:
58                path: /livez
59                port: 8000
60            periodSeconds: 5
61            failureThreshold: 3
62      volumes:
63      - name: ephemeral
64        emptyDir:
65          sizeLimit: 500Mi

If you need assistance deploying to K8s, or another platform, please reach out to support@lakera.ai for guidance.