Commit Graph

662 Commits

Author SHA1 Message Date
Matt Moyer
b42b1c1110
Relax the timeout for TestLegacyPodCleaner a bit.
This test is asynchronously waiting for the controller to do something, and in some of our test environments it will take a bit longer than we'd previously allowed.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-28 13:08:57 -05:00
Matt Moyer
48c8fabb5c
Fix backwards condition in E2E test assertion.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-28 12:40:07 -05:00
Margo Crawford
474266f918 Merge branch 'main' of github.com:vmware-tanzu/pinniped into active-directory-identity-provider 2021-07-27 15:06:58 -07:00
Ryan Richard
71cae75758
Merge branch 'main' into merge_impersonator_service_annotations 2021-07-27 11:57:16 -07:00
Ryan Richard
58ab57201f Suppress lint errors 2021-07-26 17:20:49 -07:00
Margo Crawford
53b58f65b2 Add integration test for wrong password with ldap 2021-07-26 16:32:46 -07:00
Margo Crawford
cc3875f048 PR feedback 2021-07-26 16:03:12 -07:00
Ryan Richard
9e27c28b39 Fix TestImpersonationProxy integration test changes from previous commit
Forgot to account for our new booking annotation on the impersonator's
Service.
2021-07-23 14:23:24 -07:00
Margo Crawford
1050f39789 Integration test deactivated ad account 2021-07-23 13:01:41 -07:00
Margo Crawford
91085e68f9 Refactoring defaulting logic 2021-07-23 13:01:41 -07:00
Margo Crawford
cb0ee07b51 Fetch AD search base from defaultNamingContext when not specified 2021-07-23 13:01:41 -07:00
Margo Crawford
5d8d7246c2 Refactor active directory and ldap controllers to share almost everything
Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-07-23 13:01:41 -07:00
Ryan Richard
3b4f521596 Changed TestLDAPUpstream.TestUsernameAttributeName back to TestUserMailAttributeName
Also added TestUserSAMAccountNameValue

Signed-off-by: Margo Crawford <margaretc@vmware.com>
2021-07-23 13:01:40 -07:00
Ryan Richard
aaa4861373 Custom API Group overlay for AD
Signed-off-by: Margo Crawford <margaretc@vmware.com>
2021-07-23 13:01:40 -07:00
Margo Crawford
b3d0b28bd0 Integration test fixes, fixing objectGUID handling 2021-07-23 13:01:40 -07:00
Margo Crawford
5c283d941c Helper script for running active directory tests 2021-07-23 13:01:40 -07:00
Margo Crawford
94e90a5d26 groups related env variables for AD 2021-07-23 13:01:40 -07:00
Margo Crawford
be6f9f83ce RBAC rules for activedirectoryidentityprovider 2021-07-23 13:01:40 -07:00
Margo Crawford
3b8edb84a5 WIP on active directory integration test 2021-07-23 13:01:40 -07:00
Margo Crawford
8fb35c6569 Active Directory cli options 2021-07-23 13:01:40 -07:00
Ryan Richard
ac4bc02817 Enhance integration test for CredentialIssuer spec annotations 2021-07-23 09:46:40 -07:00
Ryan Richard
e30cf6e51a
Merge branch 'main' into cli_username_password_env_vars 2021-07-22 09:29:03 -07:00
Matt Moyer
ae72d30cec
Switch to GHCR tools images for local tests, with imagePullPolicy: IfNotPresent.
This is more consistent with our CI environment.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-21 09:21:05 -05:00
Ryan Richard
deb699a84a e2e test: PINNIPED_USERNAME/PINNIPED_PASSWORD env vars during LDAP login 2021-07-19 17:08:52 -07:00
Ryan Richard
b3208f0ca6 wait for lb dns to resolve in the impersonation proxy integration test
this will hopefully fix some flakes where aws provisioned a host for the
load balancer but the tests weren't able to resolve it.

Signed-off-by: Margo Crawford <margaretc@vmware.com>
2021-07-15 16:39:15 -07:00
Ryan Richard
48b58e2fad Clear the browser cookies between each TestE2EFullIntegration test
It seems like page.ClearCookies() only clears cookies for the current
domain, so there doesn't seem to be a function to clear all browser
cookies. Instead, we'll just start a whole new browser each test.
They start fast enough that it shouldn't be a problem.
2021-07-13 16:20:02 -07:00
Ryan Richard
33461ddc14
Merge branch 'main' into deflake-serving-certificate-rotation-test 2021-07-13 14:04:34 -07:00
Matt Moyer
5527566a36
Fix TestCLILoginOIDC when running directly against Okta.
Our actual CLI code behaved correctly, but this test made some invalid assumptions about the "upstream" IDP we're testing. It assumed that the upstream didn't support `response_mode=form_post`, but Okta does. This means that when we end up on the localhost callback page, there are no URL query parameters.

Adjusting this regex makes the test pass as expected.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 16:29:42 -05:00
Matt Moyer
43f66032a9
Extend TestE2EFullIntegration to test manual OIDC flow.
Using the same fake TTY trick we used to test LDAP login, this new subtest runs through the "manual"/"jump box" login flow. It runs the login with a `--skip-listen` flag set, causing the CLI to skip opening the localhost listener. We can then wait for the login URL to be printed, visit it with the browser and log in, and finally simulate "manually" copying the auth code from the browser and entering it into the waiting CLI prompt.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:45 -05:00
Matt Moyer
d0b37a7c90
Adjust TestFormPostHTML to work on Linux chromedriver.
For some reason our headless Chrome test setup behaves slightly differently on Linux and macOS hosts. On Linux, the emoji characters are not recognized as valid text, so they are URL encoded. This change updates the test to cope with both cases correctly.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer
5029495fdb
Add manual paste flow to pinniped login oidc command.
This adds a new login flow that allows manually pasting the authorization code instead of receiving a browser-based callback.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer
9fba8d2203
Adjust TestE2EFullIntegration for new form_post flow.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer
428f389c7d
Add missing t.Helper() on RequireEventuallyf().
This gives us nicer test assertion failure messages.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer
71d4e05fb6
Add custom response_mode=form_post HTML template.
This is a new pacakge internal/oidc/provider/formposthtml containing a number of static files embedded using the relatively recent Go "//go:embed" functionality introduced in Go 1.16 (https://blog.golang.org/go1.16).

The Javascript and CSS files are minifiied and injected to make a single self-contained HTML response. There is a special Content-Security-Policy helper to calculate hash-based script-src and style-src rules.

This new code is covered by a new integration test that exercises the JS/HTML functionality in a real browser outside of the rest of the Supervisor.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer
1904f8ddc3
In browsertest.Open(), capture console INFO logs.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer
6b801056b5
Add testlib.RandBytes() helper.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer
2823d4d1e3
Add "response_modes_supported" to Supervisor discovery response.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer
3a840cee76
Make TestAPIServingCertificateAutoCreationAndRotation less flaky.
This test would occasionally flake for me when running locally. This change moves more of the assertions into the "eventually" loop, so they can temporarily fail as long as they converge on the expected values.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 11:29:02 -05:00
Matt Moyer
04e9897d51
Make TestImpersonationProxy less flaky.
This test did not tolerate this connection failing, which can happen for any number of flaky networking-related reasons. This change moves the connection setup into an "eventually" retry loop so it's allowed to fail temporarily as long as it eventually connects.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 11:28:33 -05:00
Ryan Richard
74f3ce5dcd
Merge branch 'main' into ldap-client-int-tests-only-on-kind 2021-07-08 12:54:56 -07:00
Ryan Richard
2f7dbed321 Try increasing the "eventually" timeouts in one integration test
There were 10 second timeouts in
`TestAPIServingCertificateAutoCreationAndRotation` which fail often
on CI. Maybe increasing the timeouts will help?
2021-07-08 11:17:22 -07:00
Ryan Richard
709c10227f Run the LDAP client's integration tests only on Kind
TestSimultaneousLDAPRequestsOnSingleProvider proved to be unreliable
on AKS due to some kind of kubectl port-forward issue, so only
run the LDAP client's integration tests on Kind. They are testing
the integration between the client code and the OpenLDAP test server,
not testing anything about Kubernetes, so running only on Kind should
give us sufficient test coverage.
2021-07-08 11:10:53 -07:00
Monis Khan
d78b845575
Fix bad test package name
Signed-off-by: Monis Khan <mok@vmware.com>
2021-06-22 11:23:19 -04:00
Matt Moyer
3efa7bdcc2
Improve our integration test "Eventually" assertions.
This fixes some rare test flakes caused by a data race inherent in the way we use `assert.Eventually()` with extra variables for followup assertions. This function is tricky to use correctly because it runs the passed function in a separate goroutine, and you have no guarantee that any shared variables are in a coherent state when the `assert.Eventually()` call returns. Even if you add manual mutexes, it's tricky to get the semantics right. This has been a recurring pain point and the cause of several test flakes.

This change introduces a new `library.RequireEventually()` that works by internally constructing a per-loop `*require.Assertions` and running everything on a single goroutine (using `wait.PollImmediate()`). This makes it very easy to write eventual assertions.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-06-17 16:56:03 -05:00
Monis Khan
524ff21b7f
TestServiceAccountPermissions: handle extra permissions on EKS
Signed-off-by: Monis Khan <mok@vmware.com>
2021-06-15 11:17:59 -04:00
Monis Khan
269db6b7c2
impersonator: always authorize every request
This change updates the impersonator to always authorize every
request instead of relying on the Kuberentes API server to perform
the check on the impersonated request.  This protects us from
scenarios where we fail to correctly impersonate the user due to
some bug in our proxy logic.  We still rely completely on the API
server to perform admission checks on the impersonated requests.

Signed-off-by: Monis Khan <mok@vmware.com>
2021-06-14 12:53:09 -04:00
Monis Khan
898f2bf942
impersonator: run as a distinct SA with minimal permissions
This change updates the impersonation proxy code to run as a
distinct service account that only has permission to impersonate
identities.  Thus any future vulnerability that causes the
impersonation headers to be dropped will fail closed instead of
escalating to the concierge's default service account which has
significantly more permissions.

Signed-off-by: Monis Khan <mok@vmware.com>
2021-06-11 12:13:53 -04:00
Benjamin A. Petersen
492f6cfddf
impersonator: honor anonymous authentication being disabled
When anonymous authentication is disabled, the impersonation proxy
will no longer authenticate anonymous requests other than calls to
the token credential request API (this API is used to retrieve
credentials and thus must be accessed anonymously).

Signed-off-by: Benjamin A. Petersen <ben@benjaminapetersen.me>
Signed-off-by: Monis Khan <mok@vmware.com>
2021-06-04 09:00:56 -04:00
Matt Moyer
df78e00df3
Parameterize our test images in ytt.
These are images we use for local and some CI testing.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-06-03 15:25:09 -05:00
Matt Moyer
500b444bad
Merge pull request #657 from vmware-tanzu/fix-ldap-supervisor-login-test-flake
Avoid a rare flake in TestSupervisorLogin.
2021-06-03 13:31:15 -05:00
Matt Moyer
5686591420
Avoid a rare flake in TestSupervisorLogin.
There was nothing to guarantee that _all_ Supervisor pods would be ready to handle this request. We saw a rare test flake where the LDAPIdentityProvider was marked as ready but one of the Supervisor pods didn't have it loaded yet and returned an HTTP 422 error (`Unprocessable Entity: No upstream providers are configured`).

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-06-03 12:13:56 -05:00
Matt Moyer
6903196c18
Fix a data race in TestImpersonationProxy.
The `require.Eventually()` function runs the body of the check in a separate goroutine, so it's not safe to use other `require` assertions as we did here. Our `library.RequireEventuallyWithoutError()` function does not spawn a goroutine, so it's safer to use here.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-06-03 12:07:19 -05:00
Matt Moyer
0e66b0b165
Remove an invalid test assertion in TestCredentialIssuer.
The LastUpdateTime is no longer updated on every resync. It only changes if the underlying status has changed, so that it effectively shows when the transition happened.

This change happened in ab750f48aa, but we missed this test. It only fails when it has been more than ten minutes since the CredentialIssuer transitioned into a healthy state, but that can happen in our long-running CI environments.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-06-02 12:05:02 -05:00
Matt Moyer
2ee3cec5ed
Refactor TestImpersonationProxy "apply annotation" test for clarity.
This test felt overly complex and some of the cleanup logic wasn't 100% correct (it didn't clean up in all cases).

The new code is essentially the same flow but hopefully easier to read.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-06-01 15:50:51 -05:00
Matt Moyer
75d92079e4
Allow some flexibility in "kubectl logs --tail=10" test.
We see that occasionally kubectl returns 11 lines (probably related to https://github.com/kubernetes/kubernetes/issues/72628).

This test doesn't need to be so picky, so now it allows +/- one line from the expected count.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-06-01 15:50:51 -05:00
Matt Moyer
f62c6e806d
In TestImpersonationProxy tests, avoid mutating anything in parallel block of tests.
We had this one test that mutated the CredentialIssuer, which could cause the impersonation proxy to blip on one or both of the running concierge pods. This would sometimes break other concurrently running tests.

Instead, this bit of code is split into a separate non-concurrent test.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-06-01 14:02:27 -05:00
Ryan Richard
79e3980f1f Fix nil function deference in an integration test from previous commit 2021-05-28 17:06:01 -07:00
Ryan Richard
5263e0bae5
Merge branch 'main' into do-not-log-usernames-that-might-be-passwords 2021-05-28 16:16:01 -07:00
Ryan Richard
b8205006ca Enable skipping of LDAP int tests when a firewall will block them 2021-05-28 16:13:20 -07:00
Matt Moyer
7ee1f8c441
In LDAP, do not log username until we know the user exists.
This prevents accidentally logging a password if the user enters it into the username field by mistake.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-05-28 16:57:48 -05:00
Ryan Richard
cedbe82bbb Default groupSearch.attributes.groupName to "dn" instead of "cn"
- DNs are more unique than CNs, so it feels like a safer default
2021-05-28 13:27:11 -07:00
Matt Moyer
e25de9e559
Update ID token tests for latest Fosite.
The new version has different behavior for the `nonce` claim, which is now omitted if it would be empty (see https://github.com/ory/fosite/pull/570).

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-05-28 12:53:37 -05:00
Ryan Richard
4722422aae Fix OIDC assertion bug in TestSupervisorLogin introduced by LDAP branch 2021-05-28 10:37:46 -07:00
Matt Moyer
a69fe68362
Merge branch 'main' of github.com:vmware-tanzu/pinniped into credentialissuer-spec-api 2021-05-27 17:11:40 -05:00
Ryan Richard
d2251d2ea7 Use base64 binary-encoded value as UID for LDAP
This is to allow the use of binary LDAP entry attributes as the UID.
For example, a user might like to configure AD’s objectGUID or maybe
objectSid attributes as the UID attribute.

This negatively impacts the readability of the UID when it did not come
from a binary value, but we're considering this an okay trade-off to
keep things simple for now. In the future, we may offer more
customizable encoding options for binary attributes.

These UIDs are currently only used in the downstream OIDC `sub` claim.
They do not effect the user's identity on the Kubernetes cluster,
which is only based on their mapped username and group memberships from
the upstream identity provider. We are not currently supporting any
special encoding for those username and group name LDAP attributes, so
their values in the LDAP entry must be ASCII or UTF-8 in order for them
to be interpreted correctly.
2021-05-27 13:47:10 -07:00
Matt Moyer
af2af567be
Merge branch 'main' of github.com:vmware-tanzu/pinniped into credentialissuer-spec-api 2021-05-27 15:13:36 -05:00
Matt Moyer
67d5c91713
Wait for successful TCR in TestImpersonationProxy.
This test setup should tolerate when the TokenCredentialRequest API isn't quite ready to authenticate the user or issue a cert.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-05-27 13:03:07 -05:00
Matt Moyer
349d3dad83
Make temporary errors return Pending in impersonatorconfig.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-05-27 11:13:10 -05:00
Matt Moyer
049abfb94c
Remove a "fail fast" check from TestImpersonationProxy.
This check is no longer valid, because there can be ephemeral, recoverable errors that show as ErrorDuringSetup.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-05-27 09:22:47 -05:00
Ryan Richard
033e1f0399 Add user search base to downstream subject for upstream LDAP
- Also add some tests about UTF-8 characters in LDAP attributes
2021-05-26 17:04:20 -07:00
Margo Crawford
d2d0dae4ed Wait for credentialissuer to be updated and always use proxy on clusterip test 2021-05-26 15:58:46 -07:00
Matt Moyer
c970dd1fb0
Merge branch 'main' of github.com:vmware-tanzu/pinniped into credentialissuer-spec-api 2021-05-25 17:32:58 -05:00
Matt Moyer
89eff28549
Convert LDAP code to use endpointaddr package.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-05-25 16:17:27 -05:00
Margo Crawford
75dd98a965 Integration test for impersonation proxy cluster ip 2021-05-25 13:50:50 -07:00
Matt Moyer
fabc08b01b
Merge branch 'main' of github.com:vmware-tanzu/pinniped into credentialissuer-spec-api 2021-05-24 15:49:13 -05:00
Margo Crawford
722aa72206 Integration test tests update functionality 2021-05-21 10:19:33 -07:00
Ryan Richard
b01665386d Use latest container image of our fork of bitnami-docker-openldap 2021-05-20 15:49:34 -07:00
Ryan Richard
901ddd1870 Merge branch 'main' into ldap_starttls 2021-05-20 13:40:56 -07:00
Ryan Richard
8b549f66d4 Add integration test for LDAP StartTLS 2021-05-20 13:39:48 -07:00
Ryan Richard
fff90ed2ca
Merge branch 'main' into initial_ldap_group_support 2021-05-20 12:36:04 -07:00
Matt Moyer
e4dd83887a
Merge remote-tracking branch 'origin/main' into credentialissuer-spec-api 2021-05-20 10:53:53 -05:00
Ryan Richard
025b37f839 upstreamldap.New() now supports a StartTLS config option
- This enhances our LDAP client code to make it possible to optionally
  dial an LDAP server without TLS and then use StartTLS to upgrade
  the connection to TLS.
- The controller for LDAPIdentityProviders is not using this option
  yet. That will come in a future commit.
2021-05-19 17:17:44 -07:00
Matt Moyer
657488fe90
Create CredentialIssuer at install, not runtime.
Previously, our controllers would automatically create a CredentialIssuer with a singleton name. The helpers we had for this also used "raw" client access and did not take advantage of the informer cache pattern.

With this change, the CredentialIssuer is always created at install time in the ytt YAML. The controllers now only update the existing CredentialIssuer status, and they do so using the informer cache as much as possible.

This change is targeted at only the kubecertagent controller to start. The impersonatorconfig controller will be updated in a following PR along with other changes.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-05-19 17:15:25 -05:00
Margo Crawford
3bb95f1de2 Give kubeclient_test some default values for credentialissuer spec 2021-05-19 11:56:54 -07:00
Matt Moyer
297a484948
Add more validation and update tests for impersonationProxy as pointer.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-05-19 12:42:31 -05:00
Margo Crawford
94c370ac85 Annotations for impersonation load balancer 2021-05-18 16:54:59 -07:00
Ryan Richard
b5063e59ab Merge branch 'initial_ldap_group_support' into ldap_starttls 2021-05-18 16:39:59 -07:00
Ryan Richard
a6f95cfff1 Configure openldap to disallow non-TLS clients
- For testing purposes, we would like to ensure that when we connect
  to the LDAP server we cannot accidentally avoid using TLS or StartTLS.
- Also enabled the openldap `memberOf` overlay in case we want to
  support group search using `memberOf` in the future.
- This required changes to the docker.io/bitnami/openldap container
  image, so we're using our own fork for now. Will submit a PR to
  bitnami/openldap to see if they will accept it (or something similar)
  upstream.
2021-05-18 16:38:12 -07:00
Matt Moyer
9af3cb1115 Change impersonation integration test to use CredentialIssuer spec
rather than a configmap

Signed-off-by: Margo Crawford <margaretc@vmware.com>
2021-05-18 09:51:11 -07:00
Ryan Richard
742b70d6a4 Merge branch 'main' into initial_ldap_group_support 2021-05-17 14:24:56 -07:00
Ryan Richard
dab5ff3788 ldap_client_test.go: Forgot to change an assertion related to groups 2021-05-17 14:21:57 -07:00
Ryan Richard
99099fd32f Yet more debugging of tests which only fail in main CI 2021-05-17 14:20:41 -07:00
Ryan Richard
65cab53a11 Merge branch 'main' into initial_ldap_group_support 2021-05-17 14:12:20 -07:00
Ryan Richard
8c660f09bc More debugging of tests which only fail in main CI 2021-05-17 13:53:17 -07:00
Ryan Richard
ac431ddc6d Add more to failure message in test which only fails in main CI 2021-05-17 12:57:34 -07:00
Ryan Richard
3e1e8880f7 Initial support for upstream LDAP group membership
Reflect the upstream group membership into the Supervisor's
downstream tokens, so they can be added to the user's
identity on the workload clusters.

LDAP group search is configurable on the
LDAPIdentityProvider resource.
2021-05-17 11:10:26 -07:00
Ryan Richard
20b1c41bf5 Experiment to see if we can ignore read /dev/ptmx: input/output error
This error seems to always happen on linux, but never on MacOS.
2021-05-13 16:02:24 -07:00
Ryan Richard
514ee5b883 Merge branch 'main' into initial_ldap 2021-05-13 14:24:10 -07:00
Margo Crawford
39d7f8b6eb
Merge pull request #614 from vmware-tanzu/gc-bug-tests
Tests for garbage collection behavior for access and refresh tokens
2021-05-13 13:08:07 -07:00
Ryan Richard
609883c49e Update TestSupervisorOIDCDiscovery for versioned IDP discovery endpoint 2021-05-13 13:07:31 -07:00