Commit Graph

2161 Commits

Author SHA1 Message Date
Margo Crawford d204b46c18
Merge pull request #721 from vmware-tanzu/resolve-load-balancer-dns
wait for lb dns to resolve in the impersonation proxy integration test
2021-07-15 17:02:08 -07:00
Ryan Richard b3208f0ca6 wait for lb dns to resolve in the impersonation proxy integration test
this will hopefully fix some flakes where aws provisioned a host for the
load balancer but the tests weren't able to resolve it.

Signed-off-by: Margo Crawford <margaretc@vmware.com>
2021-07-15 16:39:15 -07:00
Ryan Richard be7bf9c193
Merge pull request #718 from vmware-tanzu/workaround_for_flaky_unit_test
TestAgentController unit test is flaky, try to add workaround
2021-07-15 14:17:11 -07:00
Ryan Richard 2bba39d723 TestAgentController unit test is flaky, try to add workaround
TestAgentController really runs the controller and evaluates multiple
calls to the controller's Sync with real informers caching updates.
There is a large amount of non-determinism in this unit test, and it
does not always behave the same way. Because it makes assertions about
the specific errors that should be returned by Sync, it was not
accounting for some errors that are only returned by Sync once in a
while depending on the exact (unpredictable) order of operations.

This commit doesn't fix the non-determinism in the test, but rather
tries to work around it by also allowing other (undesired but
inevitable) error messages to appear in the list of actual error
messages returned by the calls to the Sync function.

Signed-off-by: Margo Crawford <margaretc@vmware.com>
2021-07-15 13:41:31 -07:00
anjalitelang dc567d0d1f
Update ROADMAP.md
Added https://github.com/vmware-tanzu/pinniped/issues/577 to Roadmap
2021-07-15 12:29:51 -04:00
Ryan Richard 143837c136
Merge pull request #714 from vmware-tanzu/ytt_install_doc_fix
ytt install docs suggest that you checkout the release tag
2021-07-14 12:52:23 -07:00
Ryan Richard 11eb18d348 ytt install docs suggest that you checkout the release tag
Previously, the ytt install docs suggested that you use ytt templates
from the HEAD of main with the container image from the latest public
release, which could result in a mismatch.
2021-07-14 10:59:51 -07:00
Ryan Richard d5cf5b91d6
Merge pull request #711 from vmware-tanzu/e2e_test_clear_cookies
Clear the browser cookies between each TestE2EFullIntegration subtest
2021-07-13 16:43:57 -07:00
Ryan Richard 48b58e2fad Clear the browser cookies between each TestE2EFullIntegration test
It seems like page.ClearCookies() only clears cookies for the current
domain, so there doesn't seem to be a function to clear all browser
cookies. Instead, we'll just start a whole new browser each test.
They start fast enough that it shouldn't be a problem.
2021-07-13 16:20:02 -07:00
Ryan Richard 7ef3d42e01
Merge pull request #704 from mattmoyer/deflake-serving-certificate-rotation-test
Make TestAPIServingCertificateAutoCreationAndRotation less flaky.
2021-07-13 14:58:54 -07:00
Ryan Richard 33461ddc14
Merge branch 'main' into deflake-serving-certificate-rotation-test 2021-07-13 14:04:34 -07:00
Mo Khan 238c9e6743
Merge pull request #709 from vmware-tanzu/dependabot/go_modules/github.com/tdewolff/minify/v2-2.9.19
Bump github.com/tdewolff/minify/v2 from 2.9.18 to 2.9.19
2021-07-12 14:48:16 -04:00
dependabot[bot] 25cda4f3e6
Bump github.com/tdewolff/minify/v2 from 2.9.18 to 2.9.19
Bumps [github.com/tdewolff/minify/v2](https://github.com/tdewolff/minify) from 2.9.18 to 2.9.19.
- [Release notes](https://github.com/tdewolff/minify/releases)
- [Commits](https://github.com/tdewolff/minify/compare/v2.9.18...v2.9.19)

---
updated-dependencies:
- dependency-name: github.com/tdewolff/minify/v2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-07-12 01:20:59 +00:00
Matt Moyer c71703e4db
Merge pull request #707 from mattmoyer/fix-okta-cli-integration-test
Fix TestCLILoginOIDC when running against Okta, and lower CLI server shutdown timeout.
2021-07-09 14:30:19 -07:00
Matt Moyer 5527566a36
Fix TestCLILoginOIDC when running directly against Okta.
Our actual CLI code behaved correctly, but this test made some invalid assumptions about the "upstream" IDP we're testing. It assumed that the upstream didn't support `response_mode=form_post`, but Okta does. This means that when we end up on the localhost callback page, there are no URL query parameters.

Adjusting this regex makes the test pass as expected.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 16:29:42 -05:00
Matt Moyer b6580b303a
Reduce CLI callback shutdown timeout (5s -> 500ms).
I found that there are some situations with `response_mode=form_post` where Chrome will open additional speculative TCP connections. These connections will be idle so they block server shutdown until the (previously 5s) timeout. Lowering this to 500ms should be safe and makes any added latency at login much less noticeable.

More information about Chrome's TCP-level behavior here: https://bugs.chromium.org/p/chromium/issues/detail?id=116982#c5

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 16:29:29 -05:00
Matt Moyer 405a27ba90
Merge pull request #687 from mattmoyer/add-response-mode-form-post
Add support for "response_mode=form_post" in Supervisor and CLI.
2021-07-09 10:37:59 -07:00
Matt Moyer 43f66032a9
Extend TestE2EFullIntegration to test manual OIDC flow.
Using the same fake TTY trick we used to test LDAP login, this new subtest runs through the "manual"/"jump box" login flow. It runs the login with a `--skip-listen` flag set, causing the CLI to skip opening the localhost listener. We can then wait for the login URL to be printed, visit it with the browser and log in, and finally simulate "manually" copying the auth code from the browser and entering it into the waiting CLI prompt.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:45 -05:00
Matt Moyer 91a1fec5cf
Add hidden `--skip-listen` flag for `pinniped login oidc`.
This flag is (for now) meant only to facilitate end-to-end testing, allowing us to force the "manual" login flow. If it ends up being useful we can un-hide it, but this seemed like the safest option to start with.

There is also a corresponding `--oidc-skip-listen` on the `pinniped get kubeconfig` command.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer d0b37a7c90
Adjust TestFormPostHTML to work on Linux chromedriver.
For some reason our headless Chrome test setup behaves slightly differently on Linux and macOS hosts. On Linux, the emoji characters are not recognized as valid text, so they are URL encoded. This change updates the test to cope with both cases correctly.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer 5029495fdb
Add manual paste flow to `pinniped login oidc` command.
This adds a new login flow that allows manually pasting the authorization code instead of receiving a browser-based callback.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer ac6ff1a03c
Deprecate oidcclient.WithBrowserOpen() option, add simpler oidcclient.WithSkipBrowserOpen().
This is a more restrictive library interface that more closely matches the use cases of our new form_post login flow.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer 95ee9f0b00
Add ctx params to promptForValue() and promptForSecret().
This allows the prompts to be cancelled, which we need to be able to do in the case where we prompt for a manually-pasted auth code but the automatic callback succeeds.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer 9fba8d2203
Adjust TestE2EFullIntegration for new form_post flow.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer 428f389c7d
Add missing t.Helper() on RequireEventuallyf().
This gives us nicer test assertion failure messages.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:44 -05:00
Matt Moyer 71d4e05fb6
Add custom response_mode=form_post HTML template.
This is a new pacakge internal/oidc/provider/formposthtml containing a number of static files embedded using the relatively recent Go "//go:embed" functionality introduced in Go 1.16 (https://blog.golang.org/go1.16).

The Javascript and CSS files are minifiied and injected to make a single self-contained HTML response. There is a special Content-Security-Policy helper to calculate hash-based script-src and style-src rules.

This new code is covered by a new integration test that exercises the JS/HTML functionality in a real browser outside of the rest of the Supervisor.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer 1904f8ddc3
In browsertest.Open(), capture console INFO logs.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer 6b801056b5
Add testlib.RandBytes() helper.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer 674cd4a88c
Adjust our securityheader pkg to support form_post.
Our Supervisor callback handler now needs to load JS and CSS from the provider endpoint, and this JS needs to make a `fetch()` call across origins (to post the form to the CLI callback). This requires a custom Content-Security-Policy compared to other pages we render.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer 7217cf4892
In form_post mode, expect params via POST'ed form.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer 40c931bdc5
When supported, use "response_mode=form_post" in client.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer 2823d4d1e3
Add "response_modes_supported" to Supervisor discovery response.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer 6d83ecb420
Unit test response_mode=form_post in internal/oidc/callback.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:43 -05:00
Matt Moyer c27eb17f23
Add "response_mode=form_post" to CLI client.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 12:08:42 -05:00
Matt Moyer 58363bca2c
Merge pull request #705 from mattmoyer/deflake-impersonation-websocket-test
Make TestImpersonationProxy less flaky.
2021-07-09 10:06:14 -07:00
Matt Moyer 3bf39797bb
Merge pull request #706 from mattmoyer/fix-api-doc-comment
Fix typo in generated API docs (s/mode/type/).
2021-07-09 10:05:05 -07:00
Matt Moyer 3a840cee76
Make TestAPIServingCertificateAutoCreationAndRotation less flaky.
This test would occasionally flake for me when running locally. This change moves more of the assertions into the "eventually" loop, so they can temporarily fail as long as they converge on the expected values.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 11:29:02 -05:00
Matt Moyer 04e9897d51
Make TestImpersonationProxy less flaky.
This test did not tolerate this connection failing, which can happen for any number of flaky networking-related reasons. This change moves the connection setup into an "eventually" retry loop so it's allowed to fail temporarily as long as it eventually connects.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 11:28:33 -05:00
Matt Moyer ff9095f9c4
Fix typo in generated API docs (s/mode/type/).
This CredentialIssuer field is called `spec.impersonationProxy.service.type`, not `spec.impersonationProxy.service.mode`.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-07-09 11:28:02 -05:00
Matt Moyer 2e18c88e33
Merge pull request #684 from christianang/oidc-upstream-watcher-supports-proxy
Add IPv6 support to FederationDomain spec.issuer field.
2021-07-09 09:14:39 -07:00
Matt Moyer 9f91c6c884
Merge branch 'main' into oidc-upstream-watcher-supports-proxy 2021-07-09 07:24:52 -07:00
Ryan Richard 59fd1997f4
Merge pull request #703 from vmware-tanzu/ldap-client-int-tests-only-on-kind
Run the LDAP client's integration tests only on Kind
2021-07-08 12:55:03 -07:00
Ryan Richard 74f3ce5dcd
Merge branch 'main' into ldap-client-int-tests-only-on-kind 2021-07-08 12:54:56 -07:00
Ryan Richard d403c8b44b
Merge pull request #702 from vmware-tanzu/supervisor-https-proxy-fix-timeout
Fix broken upstream OIDC discovery timeout added in previous commit
2021-07-08 12:54:39 -07:00
Ryan Richard e130da6daa Add unit test assertion for new OIDC client request timeout 2021-07-08 11:47:49 -07:00
Ryan Richard 2f7dbed321 Try increasing the "eventually" timeouts in one integration test
There were 10 second timeouts in
`TestAPIServingCertificateAutoCreationAndRotation` which fail often
on CI. Maybe increasing the timeouts will help?
2021-07-08 11:17:22 -07:00
Ryan Richard 709c10227f Run the LDAP client's integration tests only on Kind
TestSimultaneousLDAPRequestsOnSingleProvider proved to be unreliable
on AKS due to some kind of kubectl port-forward issue, so only
run the LDAP client's integration tests on Kind. They are testing
the integration between the client code and the OpenLDAP test server,
not testing anything about Kubernetes, so running only on Kind should
give us sufficient test coverage.
2021-07-08 11:10:53 -07:00
Ryan Richard f0d120a6ca Fix broken upstream OIDC discovery timeout added in previous commit
After noticing that the upstream OIDC discovery calls can hang
indefinitely, I had tried to impose a one minute timeout on them
by giving them a timeout context. However, I hadn't noticed that the
context also gets passed into the JWKS fetching object, which gets
added to our cache and used later. Therefore the timeout context
was added to the cache and timed out while sitting in the cache,
causing later JWKS fetchers to fail.

This commit is trying again to impose a reasonable timeout on these
discovery and JWKS calls, but this time by using http.Client's Timeout
field, which is documented to be a timeout for *each* request/response
cycle, so hopefully this is a more appropriate way to impose a timeout
for this use case. The http.Client instance ends up in the cache on
the JWKS fetcher object, so the timeout should apply to each JWKS
request as well.

Requests that can hang forever are effectively a server-side resource
leak, which could theoretically be taken advantage of in a denial of
service attempt, so it would be nice to avoid having them.
2021-07-08 09:44:02 -07:00
Ryan Richard 1f5480cd5c
Merge pull request #701 from vmware-tanzu/supervisor-https-proxy
Add `https_proxy` and `no_proxy` settings for the Supervisor
2021-07-07 14:57:38 -07:00
Ryan Richard f1e63c55d4 Add `https_proxy` and `no_proxy` settings for the Supervisor
- Add new optional ytt params for the Supervisor deployment.
- When the Supervisor is making calls to an upstream OIDC provider,
  use these variables if they were provided.
- These settings are integration tested in the main CI pipeline by
  sometimes setting them on deployments in certain cases, and then
  letting the existing integration tests (e.g. TestE2EFullIntegration)
  provide the coverage, so there are no explicit changes to the
  integration tests themselves in this commit.
2021-07-07 12:50:13 -07:00