djpbessems/ContainerImage.Pinniped
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

test/integration: increase confidence that a cert has rotated

Browse Source 
It looks like requests to our aggregated API service on GKE vacillate
between success and failure until they reach a converged successful
state. I think this has to do with our pods updating the API serving
cert at different times. If only one pod updates its serving cert to
the correct value, then it should respond with success. However, the
other pod would respond with failure. Depending on the load balancing
algorithm that GKE uses to send traffic to pods in a service, we could
end up with a success that we interpret as "all pods have rotated
their certs" when it really just means "at least one pod has rotated
its certs."

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
This commit is contained in:
Andrew Keesler 2020-08-28 10:16:57 -04:00
parent 9d7e073a9d
commit a4fe76f6a9
No known key found for this signature in database
GPG Key ID: 27CE0444346F9413
1 changed files with 15 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
test/integration/api_serving_certs_test.go
View File

@ -131,7 +131,10 @@ func TestAPIServingCertificateAutoCreationAndRotation(t *testing.T) {
// Check that we can still make requests to the aggregated API through the kube API server, // Check that we can still make requests to the aggregated API through the kube API server,
// because the kube API server uses these certs when proxying requests to the aggregated API server, // because the kube API server uses these certs when proxying requests to the aggregated API server,
// so this is effectively checking that the aggregated API server is using these new certs. // so this is effectively checking that the aggregated API server is using these new certs.
// We ensure that 10 straight requests succeed so that we filter out false positives where a single
// pod has rotated their cert, but not the other ones sitting behind the service.
aggregatedAPIWorking := func() bool { aggregatedAPIWorking := func() bool {
for i := 0; i < 10; i++ {
_, err = pinnipedClient.PinnipedV1alpha1().CredentialRequests().Create(ctx, &v1alpha1.CredentialRequest{ _, err = pinnipedClient.PinnipedV1alpha1().CredentialRequests().Create(ctx, &v1alpha1.CredentialRequest{
TypeMeta: metav1.TypeMeta{}, TypeMeta: metav1.TypeMeta{},
ObjectMeta: metav1.ObjectMeta{}, ObjectMeta: metav1.ObjectMeta{},
@ -140,6 +143,10 @@ func TestAPIServingCertificateAutoCreationAndRotation(t *testing.T) {
Token: &v1alpha1.CredentialRequestTokenCredential{Value: "not a good token"}, Token: &v1alpha1.CredentialRequestTokenCredential{Value: "not a good token"},
}, },
}, metav1.CreateOptions{}) }, metav1.CreateOptions{})
if err != nil {
break
}
}
// Should have got a success response with an error message inside it complaining about the token value. // Should have got a success response with an error message inside it complaining about the token value.
return err == nil return err == nil
} }