Kibaba Authentication using OAuth2 Proxy in Kubernetes

NOTE: There appears to be a bug with Kibanas impersonation features, and SIEM detection rules (and possibly elswhere): https://github.com/elastic/kibana/issues/74828

Recently I had reason to want to integrate Kibana with Azure Active Directory for authentication. This might be easily possible if you have a commercial license with Elastic, but this wasn’t the case this time.

After a little bit of research I found this article, from February 2017:

User Impersonation with X-Pack: Integrating Third Party Auth with Kibana

Obviously it’s starting to get a little long in the tooth, but as long as user impersonation is still supported, the basic outline should work.

The main trouble with the article is the specific setup used, illustrated by the image:

Problems:

1. oauth2_proxy terminating the browser connection (and possibly TLS)
2. oauth2_proxy running in reverse proxy mode

This is more what I was looking for:

For this deployment, Kibana and OAuth2 Proxy would be deployed on Kubernetes, and would be made available behind the standard k8s ingress controller, Ingress Nginx.

Therefore the browser connection would be terminated on the Ingress Nginx controller. As a side note, this takes advantage of cert-manager to automatically provision and manage TLS certificates, very handy.

I also didn’t really want to run oauth2_proxy in full reverse proxy mode, as that’s the job of the ingress controller. Don’t need more proxies involved here.

Luckily, nginx has just the feature for this, the ngx_http_auth_request module. As per the documentation:

The ngx_http_auth_request_module module (1.5.4+) implements client authorization based on the result of a subrequest. If the subrequest returns a 2xx response code, the access is allowed. If it returns 401 or 403, the access is denied with the corresponding error code. Any other response code returned by the subrequest is considered an error.

So, in theory as long as a header called es-security-runas-user can be passed to Kibana, with a value of a valid username, and be authrorized to do so by using HTTP Basic auth as a user with rights to impersonate another use, we’re good to go.

The short form is:

• A “Service Account” is required in Kibana, with a Role that allows it to impersonate users.
• User accounts in Kibana for your users, with whatever roles they require.
• All requests from the ingress controller to Kibana must have 2 headers:
  • Basic Auth header for the “Service Account”
  • An es-security-runas-user with a valid username created in Kibana.

So let’s get down to it, then, here’s the specific configurations required to make this work!

These will be a mix between Helm chart values, and some “raw-ish” Kube configs (deployed using the raw Helm chart).

OAuth2 Proxy Helm values.yaml

image:
  repository: "quay.io/pusher/oauth2_proxy"
  tag: "v6.0.0"
  pullPolicy: "IfNotPresent"

extraArgs:
  provider: 'azure'
  email-domain: '<your email domain here>'
  azure-tenant: '<your tenant id here>'
  client-id: '<your client id here>'
  client-secret: '<your client secret here>'

  redirect-url: https://kibana.example.com/oauth2/callback

  cookie-secret: '<cookie secret>'
  cookie-domain: <cookie domain>
  cookie-samesite: none

  set-xauthrequest: true

  session-store-type: redis
  redis-connection-url: 'redis://redis-master:6379/0'

  request-logging: true
  auth-logging: true
  standard-logging: true
  silence-ping-logging: true

Notes:

• Ensure your callback URL does NOT have a trailing slash /. This caused me some problems.
• To create an app in Azure for OAuth2 Proxy to use, please follow their documentation here: Azure Auth provider
• You’ll notice the Redis connection config. Please read this section of the docs carefully, particularly if you’re using Azure authentication.

The key is really the set-xauthrequest config. As per the documentation:

set X-Auth-Request-User, X-Auth-Request-Email and X-Auth-Request-Preferred-Username response headers (useful in Nginx auth_request mode)

So when this is enabled, OAuth2 Proxy will return those 3 headers to the nginx subrequest, making their values available to nginx, rather than just returning an HTTP 2xx without them.

Is this specific case, we’re after the X-Auth-Request-Email which is returned from Azure AD with the users email address, assuming successful authentication.

We then need to get that header value (some.users@example.com) into the es-security-runas-user and pass that to the upstream Kibana instance.

Nginx Ingress Resources

- apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    name: kibana-ingress
    annotations:
      kubernetes.io/ingress.class: nginx
      nginx.ingress.kubernetes.io/auth-response-headers: X-Auth-Request-Email
      nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
      nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"

      nginx.ingress.kubernetes.io/configuration-snippet: |
        proxy_set_header 'es-security-runas-user' $authHeader0;
        proxy_set_header Authorization "Basic <your basic auth string>";
  spec:
    rules:
    - host: kibana.example.com
      http:
        paths:
        - path: /
          backend:
            serviceName: kibana
            servicePort: 5601

- apiVersion: extensions/v1beta1
  kind: Ingress
  metadata:
    name: kibana-oauth2-ingress
    annotations:
      kubernetes.io/ingress.class: nginx
  spec:
    rules:
    - host: kibana.example.com
      http:
        paths:
        - path: /oauth2
          backend:
            serviceName: oauth2-proxy
            servicePort: 80

You’ll notice there’s actually 2 ingresses defined here, for the same hostname, but different paths. I’m not sure this is required, but it appears to be based on my testing. If you attempt to use a single host, with separate paths for Kibana and OAuth2 Proxy it simply doesnt work.

In the Nginx Ingress configs, the important pieces are as follows.

nginx Part One

nginx.ingress.kubernetes.io/auth-url: “https://$host/oauth2/auth” nginx.ingress.kubernetes.io/auth-signin: “https://$host/oauth2/start?rd=$escaped_request_uri”

The auth-url and auth-signin annotations activate the ngx_http_auth_request_module, so that every request through this location ( / ) must be authenticated by the external source specified. This source is $host/oauth2, which is the same hostname as kibana, but on the oauth2 path, so its the second of the ingress resources specified above.

nginx Part Two

nginx.ingress.kubernetes.io/auth-response-headers: X-Auth-Request-Email

The auth-response-headers annotation simply passes the X-Auth-Request-Email header that we received from Oauth2 Proxy onto the upstream, Kibana, assuming successful authentication.

Relevant documentation for the annotation:

nginx.ingress.kubernetes.io/auth-response-headers: <Response_Header_1, …, Response_Header_n> to specify headers to pass to backend once authentication request completes.

This by itself doesn’t help much, as Kibana has no idea to do anything with that specific header, but the trick is that the ingress controller does this by setting an nginx var to the value of that header as returned by Oauth2 Proxy, and then setting the same header to be passed upstream using proxy_set_header.

Relevant nginx config snippet:

auth_request_set $authHeader0 $upstream_http_x_auth_request_email;
proxy_set_header 'X-Auth-Request-Email' $authHeader0;

nginx Part Three

proxy_set_header ‘es-security-runas-user’ $authHeader0; proxy_set_header Authorization “Basic ";

The important piece for Kibana authentication comes next, by re-using the nginx var $authHeader0, creating and setting the es-security-runas-user header using that var, and passing that upstream using proxy_set_header.

As long as Kibana receives the es-security-runas-user and the basic auth header Authorization: Basic <your basic auth string>, it will attempt to “login” using the value of the es-security-runas-user header.

In this case we’re using emails as the username, but if your auth source provides a different value that OAuth2 Proxy can return to nginx, the same approach could be used.

Since this isintegrated with the Azure AD authentication flow, it can also take advantage of other authentication requirements, such as requiring MFA, or even Azure AD’s Conditional Access system. These sorts of features are available on most other auth providers I’ve seen as well of course.

References