Commit Graph

84 Commits

Author SHA1 Message Date
Ryan Richard 9ebf3a5b92
Merge branch 'main' into main 2022-04-13 08:41:04 -07:00
Víctor Martínez Bevià dc24397df4 Use vmware-tanzu/carvel instead of the deprecated k14/tap to install deps with brew 2022-04-05 16:43:22 +02:00
Ryan Richard ae7aac020a Merge branch 'main' into disable_http 2022-03-30 11:30:32 -07:00
Margo Crawford 53597bb824 Introduce FIPS compatibility
Signed-off-by: Margo Crawford <margaretc@vmware.com>
2022-03-29 16:58:41 -07:00
Ryan Richard 488f08dd6e Provide a way to override the new HTTP loopback-only validation
Add new deprecated_insecure_accept_external_unencrypted_http_requests
value in values.yaml. Allow it to be a boolean or a string to make it
easier to use (both --data-value and --data-value-yaml will work).

Also:
- Consider "ip6-localhost" and "ip6-loopback" to be loopback addresses
  for the validation
- Remove unused env.SupervisorHTTPAddress var
- Deprecate the `service_http_*` values in values.yaml by renaming them
  and causing a ytt render error when the old names are used
2022-03-28 17:03:23 -07:00
Mo Khan 61c8d54527
Fix typo in concierge deploy step 2022-03-10 09:08:40 -05:00
Jason van Zyl 0ea10c77c7 Consolidate declaration of variables 2022-02-25 11:26:53 -05:00
Jason van Zyl 782157e1df Remove debug output 2022-02-25 06:25:20 -05:00
Jason van Zyl 1e3f3555a4 Add line in help output for --alternate-deploy 2022-02-25 06:22:25 -05:00
Jason van Zyl 6491742c3a Minimal changes to allow an alternate deployment mechanism
The purpose of this change is to allow Helm to be used to deploy Pinniped
into the local KinD cluster for the local integration tests. That said,
the change allows any alternate deployment mechanism, I just happen
to be using it with Helm.

All default behavior is preserved. This won't change how anyone uses the
script today, it just allows me not to copy/paste the whole setup for the
integration tests.

Changes:

1) An option called `--alternate-deploy <path-to-deploy-script>` has been
added, that when enabled calls the specified script instead of using ytt
and kapp. The alternate deploy script is called with the app to deploy
and the tag of the docker image to use. We set the default value of
the alternate_deploy variable to undefined, and there is a check that
tests if the alternate deploy is defined. For the superivsor it looks
like this:

```
if [ "$alternate_deploy" != "undefined" ]; then
  log_note "The Pinniped Supervisor will be deployed with $alternate_deploy pinniped-supervisor $tag..."
  $alternate_deploy pinniped-supervisor $tag
else
  normal ytt/kapp deploy
fi
```

2) Additional log_note entries have been added to enumerate all values passed
into the ytt/kapp deploy. Used while I was trying to reach parity in the integration
tests, but I think they are useful for debugging.

3) The manifests produced by ytt and written to /tmp are now named individually.
This is so an easy comparison can be made between manifests produced by a ytt/kapp
run of integration tests and manifests produced by helm run of the integration tests.
If something is not working I have been comparing the manifests after these runs to
find differences.
2022-02-20 10:15:29 -05:00
Ryan Richard b0c36c6633 Fix int test that was failing on MacOS, and some small doc changes 2022-02-15 11:19:49 -08:00
Monis Khan 1e1789f6d1
Allow configuration of supervisor endpoints
This change allows configuration of the http and https listeners
used by the supervisor.

TCP (IPv4 and IPv6 with any interface and port) and Unix domain
socket based listeners are supported.  Listeners may also be
disabled.

Binding the http listener to TCP addresses other than 127.0.0.1 or
::1 is deprecated.

The deployment now uses https health checks.  The supervisor is
always able to complete a TLS connection with the use of a bootstrap
certificate that is signed by an in-memory certificate authority.

To support sidecar containers used by service meshes, Unix domain
socket based listeners include ACLs that allow writes to the socket
file from any runAsUser specified in the pod's containers.

Signed-off-by: Monis Khan <mok@vmware.com>
2022-01-18 17:43:45 -05:00
Monis Khan cd686ffdf3
Force the use of secure TLS config
This change updates the TLS config used by all pinniped components.
There are no configuration knobs associated with this change.  Thus
this change tightens our static defaults.

There are four TLS config levels:

1. Secure (TLS 1.3 only)
2. Default (TLS 1.2+ best ciphers that are well supported)
3. Default LDAP (TLS 1.2+ with less good ciphers)
4. Legacy (currently unused, TLS 1.2+ with all non-broken ciphers)

Highlights per component:

1. pinniped CLI
   - uses "secure" config against KAS
   - uses "default" for all other connections
2. concierge
   - uses "secure" config as an aggregated API server
   - uses "default" config as a impersonation proxy API server
   - uses "secure" config against KAS
   - uses "default" config for JWT authenticater (mostly, see code)
   - no changes to webhook authenticater (see code)
3. supervisor
   - uses "default" config as a server
   - uses "secure" config against KAS
   - uses "default" config against OIDC IDPs
   - uses "default LDAP" config against LDAP IDPs

Signed-off-by: Monis Khan <mok@vmware.com>
2021-11-17 16:55:35 -05:00
Margo Crawford 6c47c3327a Add hint to hack/prepare-for-integration-tests.sh
I keep forgetting the name of the --get-active-directory-vars flag.
2021-10-26 16:25:34 -07:00
Ryan Richard 7ec0304472 Add offline_access scope for integration tests when using Dex 2021-10-19 12:25:51 -07:00
Matt Moyer c7a8c429ed
Add a dry-run 'kubectl apply' in `prepare-for-integration-tests.sh` so we can be sure that our manifests pass API validation.
We had this for some components, but not the ones that mattered the most.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2021-09-02 16:55:28 -05:00
Margo Crawford 2d32e0fa7d Merge branch 'main' of github.com:vmware-tanzu/pinniped into active-directory-identity-provider 2021-08-26 16:21:08 -07:00
Ryan Richard d20cab10b9 Replace one-off usages of busybox and debian images in integration tests
Those images that are pulled from Dockerhub will cause pull failures
on some test clusters due to Dockerhub rate limiting.

Because we already have some images that we use for testing, and
because those images are already pre-loaded onto our CI clusters
to make the tests faster, use one of those images and always specify
PullIfNotPresent to avoid pulling the image again during the integration
test.
2021-08-25 15:12:07 -07:00
Margo Crawford cc3875f048 PR feedback 2021-07-26 16:03:12 -07:00
Margo Crawford 8ea1bd3dfb Make prepare-for-integration-tests active directory setup accessible for anyone 2021-07-23 13:01:41 -07:00
Margo Crawford 91085e68f9 Refactoring defaulting logic 2021-07-23 13:01:41 -07:00
Margo Crawford b3d0b28bd0 Integration test fixes, fixing objectGUID handling 2021-07-23 13:01:40 -07:00
Margo Crawford 5c283d941c Helper script for running active directory tests 2021-07-23 13:01:40 -07:00
Ryan Richard 8b549f66d4 Add integration test for LDAP StartTLS 2021-05-20 13:39:48 -07:00
Ryan Richard 67a568811a Make prepare-for-integration-tests.sh work on linux too
- The linux base64 command is different, so avoid using it at all.
  On linux the default is to split the output into multiple lines,
  which messes up the integration-test-env file. The flag used to
  disable this behavior on linux ("-w0") does not exist on MacOS's
  base64.
- On debian linux, the latest version of Docker from apt-get still
  requires DOCKER_BUILDKIT=1 or else it barfs.
2021-04-27 10:10:02 -07:00
Ryan Richard f63ded99bc Add a flag for skipping chromedriver version check to hack script 2021-04-15 10:27:00 -07:00
Ryan Richard 923938ab26 Avoid multi-line integration test env vars
Avoid them because they can't be used in GoLand for running integration
tests in the UI, like running in the debugger.

Also adds optional PINNIPED_TEST_TOOLS_NAMESPACE because we need it
on the LDAP feature branch where we are developing the upcoming LDAP
support for the Supervisor.
2021-04-14 17:26:12 -07:00
Andrew Keesler c53507809d Rename dex namespace, add new ytt value to deploy/tools, and remove Tilt
- Rename the test/deploy/dex directory to test/deploy/tools
- Rename the dex namespace to tools
- Add a new ytt value called `pinny_ldap_password` for the tools
  ytt templates
- This new value is not used on main at this time. We intend to use
  it in the forthcoming ldap branch. We're defining it on main so
  that the CI scripts can use it across all branches and PRs.

Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-04-05 15:01:49 -07:00
Ryan Richard 7bb5657c4d Add hack/prepare-supervisor-on-kind.sh
A demo of running the Supervisor and Concierge on
a kind cluster. Can be used to quickly set up an
environment for manual testing.

Also added some missing copyright headers to other
hack scripts.
2021-03-31 11:39:10 -07:00
Margo Crawford cd6e48bfa8 Use a random password for the dex integration test user
Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-03-25 15:12:17 -07:00
Ryan Richard 75cfda0ffe prepare-for-integration-tests.sh: Check Chrome and chromedriver versions
They usually need to match, or at least be close, so added some
code to help us remember to do that.
2021-03-22 16:54:22 -07:00
Andrew Keesler 28d00ce67b
Merge remote-tracking branch 'upstream/main' into impersonation-proxy 2021-03-18 20:13:49 -04:00
Ryan Richard 08c446a3e1 Use openssl to generate the test user password instead of /dev/urandom
Because it's more portable across different operating systems and
it is already pre-installed on MacOS.
2021-03-18 11:20:33 -07:00
Matt Moyer b20a8358d3
Merge branch 'main' of github.com:vmware-tanzu/pinniped into impersonation-proxy 2021-03-08 15:16:40 -06:00
Margo Crawford 4bd68b1fa1 Use LC_ALL=C instead of LC_CTYPE=C because it works on Big Sur
It also works on the slightly older MacOS Catalina.
This script is only used on development laptops, so hopefully
this will work for more laptop OS's now.

Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-03-05 15:25:52 -08:00
Ryan Richard 7b7901af36 Add `-timeout 0` when describing how to run integration tests
Because otherwise `go test` will panic/crash your test if it takes
longer than 10 minutes, which is an annoying way for an integration
test to fail since it skips all of the t.Cleanup's.
2021-03-03 12:53:41 -08:00
Andrew Keesler 26922307ad prepare-for-integration-tests.sh: New cmdline option --api_group_suffix
Makes it easy to deploy Pinniped under a different API group for manual
testing and iterating on integration tests on your laptop.

Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-02-03 12:07:38 -08:00
Andrew Keesler 93d25a349f
hack: fix docker most recent tag check
I think this stopped working when we starting using a specific registry in e0b94f47.

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-02-02 18:01:07 -05:00
Ryan Richard b5cbe018e3
Allow passing multiple redirect URIs to Dex
We need this in CI when we want to configure Dex with the redirect URI for both
primary and secondary deploys at one time (since we only stand up Dex once).

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-01-20 17:06:50 -05:00
Monis Khan 3c3da9e75d
Wire in new env vars for user info testing
Signed-off-by: Monis Khan <mok@vmware.com>
2021-01-12 11:23:25 -05:00
Matt Moyer e0b94f4780
Move our main image references to the VMware Harbor registry.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-12-17 17:51:09 -06:00
Andrew Keesler 93cfd8c93a
Fix prepare-for-integration-tests.sh and Tiltfile for kubectl 1.20
kubectl 1.20 prints "Kubernetes control plane" instead of "Kubernetes master".

Signed-off-by: Andrew Keesler <akeesler@vmware.com>

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2020-12-09 10:15:34 -05:00
Matt Moyer 408fbe4f76
Parameterize the `supervisor_redirect_uri` in the test env Dex.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-12-03 12:45:56 -06:00
Matt Moyer fe0481c304
In integration test env, deploy a ClusterIP service and register that with Dex.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-12-02 15:55:33 -06:00
Matt Moyer bc700d58ae
Split test environment variables so there's a specific supervisor upstream client.
Prior to this we re-used the CLI testing client to test the authorize flow of the supervisor, but they really need to be separate upstream clients. For example, the supervisor client should be a non-public client with a client secret and a different callback endpoint.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-11-20 08:03:06 -06:00
Matt Moyer 6ce2f109bf
Refactor certificate generation for integration test Dex.
Before, we did this in an init container, which meant if the Dex pod restarted we would have fresh certs, but our Tilt/bash setup didn't account for this.

Now, the certs are generated by a Job which runs once and saves the generated files into a Secret. This should be a bit more stable.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-11-17 11:36:36 -06:00
Matt Moyer b17ac6ec0b
Update integration tests to run Dex over HTTPS.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-11-16 20:23:20 -06:00
Matt Moyer c8b17978a9
Convert CLI tests to work through an HTTP forward proxy.
This change deploys a small Squid-based proxy into the `dex` namespace in our integration test environment. This lets us use the cluster-local DNS name (`http://dex.dex.svc.cluster.local/dex`) as the OIDC issuer. It will make generating certificates easier, and most importantly it will mean that our CLI can see Dex at the same name/URL as the supervisor running inside the cluster.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-11-16 17:16:58 -06:00
Andrew Keesler 83a156d72b
Enable debug logging in all testing scenarios
It is really helpful to have verbose logs during test debugging.

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2020-11-11 09:01:43 -05:00
Andrew Keesler 3bc13517b2
prepare-for-integration-tests.sh: add check for chromedriver
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2020-11-04 15:53:32 -08:00