From b46a2f0267666e1861f42675b23bcd5d37efaea8 Mon Sep 17 00:00:00 2001
From: Ryan Richard <richardry@vmware.com>
Date: Thu, 22 Sep 2022 12:31:31 -0700
Subject: [PATCH] Add more details about OIDCClients to
 configure-auth-for-webapps.md

Co-authored-by: Ryan Richard <richardry@vmware.com>
Co-authored-by: Benjamin A. Petersen <ben@benjaminapetersen.me>
---
 .../docs/howto/configure-auth-for-webapps.md  | 71 ++++++++++++++++---
 1 file changed, 63 insertions(+), 8 deletions(-)

diff --git a/site/content/docs/howto/configure-auth-for-webapps.md b/site/content/docs/howto/configure-auth-for-webapps.md
index 1eaa13f0..1b535eb6 100644
--- a/site/content/docs/howto/configure-auth-for-webapps.md
+++ b/site/content/docs/howto/configure-auth-for-webapps.md
@@ -17,6 +17,13 @@ identity provider.
 
 This guide explains how to use the Supervisor to provide authentication services for a web application.
 
+Note that this feature is not part of how Pinniped provides authentication for `kubectl` users. By default,
+the Pinniped Supervisor will contain an OIDC client called `pinniped-cli` which requires no configuration and is
+used to provide authentication for `kubectl` (and other kubeconfig-based Kubernetes API clients).
+If you are only setting up authentication for `kubectl` users of your Kubernetes clusters, then you do not need to
+read this guide. If you want to use the Pinniped Supervisor to provide authentication services for a web application,
+then this guide is for you.
+
 ## Prerequisites
 
 This guide assumes that you have installed and configured the Pinniped Supervisor, and configured it with an
@@ -42,6 +49,19 @@ framework (e.g. Spring, Rails, Django, etc.) to implement authentication. The Su
 
 Most web application frameworks offer all these capabilities in their OAuth2/OIDC libraries.
 
+## Performance implications of using OIDCClients in the Supervisor
+
+The Pinniped Supervisor is an efficient application which typically does not use a lot of CPU and memory resources.
+Using the OIDCClient CR, as described below, will cause the Supervisor to perform
+bcrypt operations to validate the client's secret during authorization and refresh flows. While each of these bcrypt operations
+takes only about a quarter second of CPU time, in aggregate, when lots of users are perform authorization and refresh flows,
+these bcrypts will constitute the majority of the CPU usage of the Supervisor.
+
+The administrator of the Supervisor may need to adjust the Supervisor Deployment once they are familiar with usage patterns of
+their Supervisor. Very heavy usage by clients might result in the Supervisor pods reaching their cpu limit and being
+throttled, resulting in poor performance. This can be alleviated by adjusting the number of Pod replicas, and the CPU
+requests and limits on each Pod.
+
 ## Create an OIDCClient
 
 For each web application, the administrator of the Pinniped Supervisor will create an OIDCClient describing what
@@ -133,7 +153,7 @@ secret for the client. The client secrets are random strings auto-generated by t
 The plaintext secret will only be returned once upon creation.
 
 ```sh
-cat <<EOF | kubectl create -f -
+cat <<EOF | kubectl create -o yaml -f -
 apiVersion: clientsecret.supervisor.pinniped.dev/v1alpha1
 kind: OIDCClientSecretRequest
 metadata:
@@ -146,18 +166,35 @@ EOF
 
 The server will respond with the newly generated client secret, e.g.:
 
-```
-NAMESPACE               NAME                                        SECRET     TOTAL
-pinniped-supervisor     client.oauth.pinniped.dev-my-webapp-client  abc123     1
+```yaml
+apiVersion: clientsecret.supervisor.pinniped.dev/v1alpha1
+kind: OIDCClientSecretRequest
+metadata:
+  creationTimestamp: "2022-09-22T19:04:46Z"
+  name: client.oauth.pinniped.dev-my-webapp-client
+  namespace: supervisor
+spec:
+  generateNewSecret: true
+  revokeOldSecrets: false
+status:
+  generatedSecret: e593049b02d0b647af4ac99bd5963c3612f9ea9c414a9b8f6acd23bc43cbf084
+  totalClientSecrets: 1
 ```
 
-Take care to make a note of the secret. After it has been returned by the create API, there is no other way to
-retrieve it in the future. The secret is not stored in plaintext on the server, which only stores a
-bcrypt-hashed version of the secret.
+Take care to make a note of the `status.generatedSecret`. _It can never be retrieved again_. After it has been returned
+once in the response of the create API, there is no other way to retrieve it in the future. The secret is not stored
+in plaintext on the server, which only stores a bcrypt-hashed version of the secret.
+
+The `status.totalClientSecrets` reports the total number of client secrets associated with this OIDCClient at the
+end of the request. This can also be observed on the `status` of the OIDCClient CR itself.
 
 This is the client secret that should be used, along with the client ID, by the web application when interacting
 with the Supervisor's OIDC token endpoint.
 
+The OIDCClientSecretRequest is a special API which only supports the `create` verb. After creating a client secret,
+you cannot use `kubectl get`, `kubectl delete`, `kubectl apply`, or any other API verbs to access those client secret
+resources.
+
 ## Rotating the client secret for an OIDCClient
 
 To facilitate rotating client secrets, an OIDCClient may have several active secrets. This enables the following process
@@ -170,7 +207,7 @@ for the Supervisor administrator to change a client secret without causing web a
    remove the old client secret:
 
     ```sh
-    cat <<EOF | kubectl create -f -
+    cat <<EOF | kubectl create -o yaml -f -
     apiVersion: oauth.virtual.supervisor.pinniped.dev/v1alpha1
     kind: OIDCClientSecretRequest
     metadata:
@@ -182,6 +219,24 @@ for the Supervisor administrator to change a client secret without causing web a
     EOF
     ```
 
+Note that when there are multiple active client secrets for an OIDCClient, clients who use an older client secret will
+pay a small performance penalty during authorization and refresh flows. The client secret provided by client during
+authorization and refresh flows is compared against the stored bcrypt hashes of each active client secret, in order from
+the most recently generated secret to the least recently generated secret. Each comparison operation is a somewhat
+expensive bcrypt. As a best practice, an OIDCClient should usually have one active secret, except during a window of
+rotation, when it will have two active secrets.
+
+The server will only allow an OIDCClient to have five active secrets. Asking the server to generate a sixth secret will
+fail, unless you also ask the server to revoke all the old secrets in the same (or in a previous) request.
+
+## Deleting an OIDCClient
+
+An OIDCClient can be deleted in the usual way that Kubernetes CRs are deleted. User sessions using that client
+will fail at their next refresh request. The corresponding client secrets will also be deleted. Even if the client
+is created again with the same name, a new client secret will need to be generated for that client. Since the
+client secret is new, webapps that were using the old client secret will not be able to perform refresh requests
+(unless they are updated to use the new secret).
+
 ## What the web application will receive from the authorization code flow
 
 When the web application completes the authorization code flow with the Supervisor, it will receive three tokens: