Enabling Observability on KubeArchive

KubeArchive is instrumented to emit observability data, specifically metrics and traces. This can be enabled and configured with two environment variables present on all of the KubeArchive components:

  • KUBEARCHIVE_OTEL_MODE: one of "enabled", "delegated" or "disabled". Defaults to an "disabled". It controls when observability data is sent to OTEL_EXPORTER_OTLP_ENDPOINT:

    • "disabled": do not send traces or metrics.

    • "enabled": always send traces and metrics. Useful for observability in development.

    • "delegated": always send metrics but send traces only when the incoming request sent its traces (see W3 Trace Context sampled flag). Useful for observability in production.

  • OTEL_EXPORTER_OTLP_ENDPOINT: an OTLP compatible endpoint where traces are sent. By default it is set to an empty string.

To change these environment variables values, edit or patch the kubearchive-api-server, kubearchive-sink and kubearchive-operator Deployments on the kubearchive namespace:

  1. Patch for the kubearchive-api-server Deployment:

    ---
apiVersion: v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: kubearchive-api-server
          env:
            - name: KUBEARCHIVE_OTEL_MODE
              value: "delegated"
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://otel-collector.observability.svc.cluster.local:4318"

  2. Patch for the kubearchive-sink Deployment:

    ---
apiVersion: v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: kubearchive-sink
          env:
            - name: KUBEARCHIVE_OTEL_MODE
              value: "delegated"
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://otel-collector.observability.svc.cluster.local:4318"

  3. Patch for the kubearchive-operator Deployment:

    ---
apiVersion: v1
kind: Deployment
spec:
  template:
    spec:
      containers:
        - name: manager
          env:
            - name: KUBEARCHIVE_OTEL_MODE
              value: "delegated"
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://traces-collector.observability.svc.cluster.local:4318"

  4. Apply the patches:

    kubectl patch -n kubearchive deployment <deployment-name> --patch-file path/to/patch.yaml