Commit Graph

143 Commits

Author SHA1 Message Date
Ryan Richard
8c660f09bc More debugging of tests which only fail in main CI 2021-05-17 13:53:17 -07:00
Ryan Richard
ac431ddc6d Add more to failure message in test which only fails in main CI 2021-05-17 12:57:34 -07:00
Ryan Richard
6a350aa4e1 Fix some LDAP CA bundle handling
- Make PINNIPED_TEST_LDAP_LDAPS_CA_BUNDLE optional for integration tests
- When there is no CA bundle provided, be careful to use nil instead of
  an empty bundle, because nil means to use the OS defaults
2021-04-22 16:58:48 -07:00
Ryan Richard
5c28d36c9b Redact some params of URLs in logs to avoid printing sensitive info 2021-04-15 07:59:38 -07:00
Ryan Richard
939b6b12cc ldap_client_test.go: refactor to use the LDAP server on the K8s cluster 2021-04-14 17:49:40 -07:00
Ryan Richard
8d75825635 Merge branch 'main' into initial_ldap 2021-04-14 17:47:26 -07:00
Ryan Richard
923938ab26 Avoid multi-line integration test env vars
Avoid them because they can't be used in GoLand for running integration
tests in the UI, like running in the debugger.

Also adds optional PINNIPED_TEST_TOOLS_NAMESPACE because we need it
on the LDAP feature branch where we are developing the upcoming LDAP
support for the Supervisor.
2021-04-14 17:26:12 -07:00
Ryan Richard
1c55c857f4 Start to fill out LDAPIdentityProvider's fields and TestSupervisorLogin
- Add some fields to LDAPIdentityProvider that we will need to be able
  to search for users during login
- Enhance TestSupervisorLogin to test logging in using an upstream LDAP
  identity provider. Part of this new test is skipped for now because
  we haven't written the corresponding production code to make it
  pass yet.
- Some refactoring and enhancement to env.go and the corresponding env
  vars to support the new upstream LDAP provider integration tests.
- Use docker.io/bitnami/openldap for our test LDAP server instead of our
  own fork now that they have fixed the bug that we reported.

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-04-07 12:56:09 -07:00
Ryan Richard
2b6859b161
Add stub LDAP API type and integration test
The goal here was to start on an integration test to get us closer to the red
test that we want so we can start working on LDAP.

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-04-06 13:10:01 -04:00
Andrew Keesler
c53507809d Rename dex namespace, add new ytt value to deploy/tools, and remove Tilt
- Rename the test/deploy/dex directory to test/deploy/tools
- Rename the dex namespace to tools
- Add a new ytt value called `pinny_ldap_password` for the tools
  ytt templates
- This new value is not used on main at this time. We intend to use
  it in the forthcoming ldap branch. We're defining it on main so
  that the CI scripts can use it across all branches and PRs.

Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-04-05 15:01:49 -07:00
Matt Moyer
defad3cdd7
Remove library.DumpLogs test helper.
We had this code that printed out pod logs when certain tests failed, but it is a bit cumbersome. We're removing it because we added a CI task that exports all pod logs after every CI run, which accomplishes the same thing and provides us a bunch more data.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-26 16:43:02 -05:00
Margo Crawford
6bf8bfe9a8 Merge remote-tracking branch 'origin/main' into impersonation-proxy 2021-03-24 17:22:40 -07:00
Matt Moyer
cda8bd6e26
Allow running CLI-related integration tests with pre-built binary.
This allows setting `$PINNIPED_TEST_CLI` to point at an existing `pinniped` CLI binary instead of having the test build one on-the-fly. This is more efficient when you're running the tests across many clusters as we do in CI.

Building the CLI from scratch in our CI environment takes 1.5-2 minutes, so this change should save nearly that much time on every test job.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-23 17:19:09 -05:00
Matt Moyer
c0d32f10b2
Add some test debug logging when running the CLI.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-23 12:07:34 -05:00
Matt Moyer
ce5b05f912
Add some debug logging to measure how long the CLI build takes.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-23 12:06:35 -05:00
Matt Moyer
176fb6a139
Authenticators are no longer namespaced, so clean up these test logs.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-23 10:33:05 -05:00
Andrew Keesler
f73c70d8f9
test/integration: use Ryan's 20x rule to harden simple access tests
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-03-19 13:18:10 -04:00
Andrew Keesler
05a188d4cd
Merge remote-tracking branch 'upstream/main' into impersonation-proxy
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-03-18 10:36:28 -04:00
Matt Moyer
74df6d138b
Memoize library.IntegrationEnv so it's only constructed once per test.
This is probably a good idea regardless, but it also avoids an infinite recursion from IntegrationEnv() -> assertNoRestartsDuringTest() -> NewKubeclient() -> IntegrationEnv() -> ...

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-17 13:37:48 -05:00
Matt Moyer
0dd2b358fb
Extend assertNoRestartsDuringTest to dump logs from containers that restarted.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-17 13:37:47 -05:00
Matt Moyer
6520c5a3a1
Extend library.DumpLogs() to dump logs from the previous container, if one exists.
This is important in case the container has crashed and has been restarted.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-17 11:46:40 -05:00
Matt Moyer
5a43a5d53a
Remove library.AssertNoRestartsDuringTest and make that assertion implicit in library.IntegrationEnv.
This means we (hopefully) can't forget to include these assertions in any integration test.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-17 11:18:10 -05:00
Matt Moyer
10a1e29e15
Merge branch 'main' of github.com:vmware-tanzu/pinniped into impersonation-proxy
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-16 14:35:07 -05:00
Matt Moyer
c5b784465b
Describe "anonymousAuthenticationSupported" test cluster capability and add more managed cluster types.
This new capability describes whether a cluster is expected to allow anonymous requests (most do since k8s 1.6.x, but AKS has it disabled).

This commit also contains new capability YAML files for AKS and EKS, mostly to document publicly how we expect our tests to function in those environments.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-16 13:54:29 -05:00
Margo Crawford
22ca2da1ff
test/integration: add "kubectl attach" test to TestImpersonationProxy
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-03-11 15:10:16 -05:00
Ryan Richard
d13bb07b3e Add integration test for using WhoAmIRequest through impersonator 2021-03-10 16:57:15 -08:00
Ryan Richard
0b300cbe42 Use TokenCredentialRequest instead of base64 token with impersonator
To make an impersonation request, first make a TokenCredentialRequest
to get a certificate. That cert will either be issued by the Kube
API server's CA or by a new CA specific to the impersonator. Either
way, you can then make a request to the impersonator and present
that client cert for auth and the impersonator will accept it and
make the impesonation call on your behalf.

The impersonator http handler now borrows some Kube library code
to handle request processing. This will allow us to more closely
mimic the behavior of a real API server, e.g. the client cert
auth will work exactly like the real API server.

Signed-off-by: Monis Khan <mok@vmware.com>
2021-03-10 10:30:06 -08:00
Ryan Richard
d8c6894cbc All controller unit tests should not cancel context until test is over
All controller unit tests were accidentally using a timeout context
for the informers, instead of a cancel context which stays alive until
each test is completely finished. There is no reason to risk
unpredictable behavior of a timeout being reached during an individual
test, even though with the previous 3 second timeout it could only be
reached on a machine which is running orders of magnitude slower than
usual, since each test usually runs in about 100-300 ms. Unfortunately,
sometimes our CI workers might get that slow.

This sparked a review of other usages of timeout contexts in other
tests, and all of them were increased to a minimum value of 1 minute,
under the rule of thumb that our tests will be more reliable on slow
machines if they "pass fast and fail slow".
2021-03-04 17:26:01 -08:00
Matt Moyer
fea626b654
Remove this proxy-related test code that we ended up not needing.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-04 17:19:59 -06:00
Matt Moyer
274e6281a8
Whoops, missed these fixes in test/library/env.go.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-04 15:21:49 -06:00
Matt Moyer
9dfbe60253
Do the kubeconfig proxy environment injection, but actually render back out the YAML.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-03-04 14:41:20 -06:00
Ryan Richard
41140766f0 Add integration test which demonstrates double impersonation
We don't support using the impersonate headers through the impersonation
proxy yet, so this integration test is a negative test which asserts
that we get an error.
2021-03-01 17:53:26 -08:00
Ryan Richard
3fcde8088c concierge_impersonation_proxy_test.go: Make it work on more clusters
Should work on cluster which have:
- load balancers not supported, has squid proxy (e.g. kind)
- load balancers supported, has squid proxy (e.g. EKS)
- load balancers supported, no squid proxy (e.g. GKE)

When testing with a load balancer, call the impersonation proxy through
the load balancer.

Also, added a new library.RequireNeverWithoutError() helper.

Signed-off-by: Margo Crawford <margaretc@vmware.com>
2021-02-25 14:40:18 -08:00
Margo Crawford
dac1c9939e concierge_impersonation_proxy_test.go: Test all the verbs
Also:
- Shut down the informer correctly in
  concierge_impersonation_proxy_test.go
- Remove the t.Failed() checks which avoid cleaning up after failed
  tests. This was inconsistent with how most of the tests work, and
  left cruft on clusters when a test failed.

Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-02-23 10:38:32 -08:00
Andrew Keesler
069b3fba37
Merge remote-tracking branch 'upstream/main' into impersonation-proxy
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-02-23 12:10:52 -05:00
Ryan Richard
80ff5c1f17 Fix bug which prevented watches from working through impersonator
Also:
- Changed base64 encoding of impersonator bearer tokens to use
  `base64.StdEncoding` to make it easier for users to manually
  create a token using the unix `base64` command
- Test the headers which are and are not passed through to the Kube API
  by the impersonator more carefully in the unit tests
- More WIP on concierge_impersonation_proxy_test.go

Signed-off-by: Margo Crawford <margaretc@vmware.com>
2021-02-22 17:23:11 -08:00
Monis Khan
abc941097c
Add WhoAmIRequest Aggregated Virtual REST API
This change adds a new virtual aggregated API that can be used by
any user to echo back who they are currently authenticated as.  This
has general utility to end users and can be used in tests to
validate if authentication was successful.

Signed-off-by: Monis Khan <mok@vmware.com>
2021-02-22 20:02:41 -05:00
Andrew Keesler
957cb2d56c
Merge remote-tracking branch 'upstream/main' into impersonation-proxy
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-02-18 13:37:28 -05:00
Matt Moyer
6565265bee
Use new 'go.pinniped.dev/generated/latest' package.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-02-16 13:00:08 -06:00
Andrew Keesler
9b87906a30
Merge remote-tracking branch 'upstream/main' into impersonation-proxy
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-02-11 11:03:33 -05:00
Monis Khan
89b00e3702
Declare war on namespaces
Signed-off-by: Monis Khan <mok@vmware.com>
2021-02-10 21:52:07 -05:00
Andrew Keesler
0fc1f17866
internal/groupsuffix: mutate TokenCredentialRequest's Authenticator
This is a partial revert of 288d9c999e. For some reason it didn't occur to me
that we could do it this way earlier. Whoops.

This also contains a middleware update: mutation funcs can return an error now
and short-circuit the rest of the request/response flow. The idea here is that
if someone is configuring their kubeclient to use middleware, they are agreeing
to a narrow-er client contract by doing so (e.g., their TokenCredentialRequest's
must have an Spec.Authenticator.APIGroup set).

I also updated some internal/groupsuffix tests to be more realistic.

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-02-10 15:53:44 -05:00
Ryan Richard
e4c49c37b9 Merge branch 'main' into impersonation-proxy 2021-02-09 13:45:37 -08:00
Margo Crawford
dfcc2a1eb8 Introduce clusterhost package to determine whether a cluster has control plane nodes
Also added hasExternalLoadBalancerProvider key to cluster capabilities
for integration testing.

Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-02-09 11:16:01 -08:00
Ryan Richard
2a921f7090
Merge branch 'main' into credential_request_spec_api_group 2021-02-04 13:44:53 -08:00
Andrew Keesler
ae498f14b4
test/integration: ensure no pods restart during integration tests
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-02-04 10:24:33 -05:00
Ryan Richard
288d9c999e Use custom suffix in Spec.Authenticator.APIGroup of TokenCredentialRequest
When the Pinniped server has been installed with the `api_group_suffix`
option, for example using `mysuffix.com`, then clients who would like to
submit a `TokenCredentialRequest` to the server should set the
`Spec.Authenticator.APIGroup` field as `authentication.concierge.mysuffix.com`.

This makes more sense from the client's point of view than using the
default `authentication.concierge.pinniped.dev` because
`authentication.concierge.mysuffix.com` is the name of the API group
that they can observe their cluster and `authentication.concierge.pinniped.dev`
does not exist as an API group on their cluster.

This commit includes both the client and server-side changes to make
this work, as well as integration test updates.

Co-authored-by: Andrew Keesler <akeesler@vmware.com>
Co-authored-by: Ryan Richard <richardry@vmware.com>
Co-authored-by: Margo Crawford <margaretc@vmware.com>
2021-02-03 15:49:15 -08:00
Monis Khan
efe1fa89fe Allow multiple Pinnipeds to work on same cluster
Yes, this is a huge commit.

The middleware allows you to customize the API groups of all of the
*.pinniped.dev API groups.

Some notes about other small things in this commit:
- We removed the internal/client package in favor of pkg/conciergeclient. The
  two packages do basically the same thing. I don't think we use the former
  anymore.
- We re-enabled cluster-scoped owner assertions in the integration tests.
  This code was added in internal/ownerref. See a0546942 for when this
  assertion was removed.
- Note: the middlware code is in charge of restoring the GV of a request object,
  so we should never need to write mutations that do that.
- We updated the supervisor secret generation to no longer manually set an owner
  reference to the deployment since the middleware code now does this. I think we
  still need some way to make an initial event for the secret generator
  controller, which involves knowing the namespace and the name of the generated
  secret, so I still wired the deployment through. We could use a namespace/name
  tuple here, but I was lazy.

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
Co-authored-by: Ryan Richard <richardry@vmware.com>
2021-02-02 15:18:41 -08:00
Andrew Keesler
906bfa023c
test: wire API group suffix through to tests
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-01-19 17:23:20 -05:00
Andrew Keesler
8a916ce8ae
test/integration: add test helper to avoid race conditions
We were seeing a race in this test code since the require.NoError() and
require.Eventually() would write to the same testing.T state on separate
goroutines. Hopefully this helper function should cover the cases when we want
to require.NoError() inside a require.Eventually() without causing a race.

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
Co-authored-by: Margo Crawford <margaretc@vmware.com>
Co-authored-by: Monis Khan <i@monis.app>
2021-01-14 10:19:35 -05:00