Commit Graph

62 Commits

Author SHA1 Message Date
Ryan Richard
d20cab10b9 Replace one-off usages of busybox and debian images in integration tests
Those images that are pulled from Dockerhub will cause pull failures
on some test clusters due to Dockerhub rate limiting.

Because we already have some images that we use for testing, and
because those images are already pre-loaded onto our CI clusters
to make the tests faster, use one of those images and always specify
PullIfNotPresent to avoid pulling the image again during the integration
test.
2021-08-25 15:12:07 -07:00
Ryan Richard
8b549f66d4 Add integration test for LDAP StartTLS 2021-05-20 13:39:48 -07:00
Ryan Richard
67a568811a Make prepare-for-integration-tests.sh work on linux too
- The linux base64 command is different, so avoid using it at all.
  On linux the default is to split the output into multiple lines,
  which messes up the integration-test-env file. The flag used to
  disable this behavior on linux ("-w0") does not exist on MacOS's
  base64.
- On debian linux, the latest version of Docker from apt-get still
  requires DOCKER_BUILDKIT=1 or else it barfs.
2021-04-27 10:10:02 -07:00
Ryan Richard
f63ded99bc Add a flag for skipping chromedriver version check to hack script 2021-04-15 10:27:00 -07:00
Ryan Richard
923938ab26 Avoid multi-line integration test env vars
Avoid them because they can't be used in GoLand for running integration
tests in the UI, like running in the debugger.

Also adds optional PINNIPED_TEST_TOOLS_NAMESPACE because we need it
on the LDAP feature branch where we are developing the upcoming LDAP
support for the Supervisor.
2021-04-14 17:26:12 -07:00
Andrew Keesler
c53507809d Rename dex namespace, add new ytt value to deploy/tools, and remove Tilt
- Rename the test/deploy/dex directory to test/deploy/tools
- Rename the dex namespace to tools
- Add a new ytt value called `pinny_ldap_password` for the tools
  ytt templates
- This new value is not used on main at this time. We intend to use
  it in the forthcoming ldap branch. We're defining it on main so
  that the CI scripts can use it across all branches and PRs.

Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-04-05 15:01:49 -07:00
Ryan Richard
7bb5657c4d Add hack/prepare-supervisor-on-kind.sh
A demo of running the Supervisor and Concierge on
a kind cluster. Can be used to quickly set up an
environment for manual testing.

Also added some missing copyright headers to other
hack scripts.
2021-03-31 11:39:10 -07:00
Margo Crawford
cd6e48bfa8 Use a random password for the dex integration test user
Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-03-25 15:12:17 -07:00
Ryan Richard
75cfda0ffe prepare-for-integration-tests.sh: Check Chrome and chromedriver versions
They usually need to match, or at least be close, so added some
code to help us remember to do that.
2021-03-22 16:54:22 -07:00
Andrew Keesler
28d00ce67b
Merge remote-tracking branch 'upstream/main' into impersonation-proxy 2021-03-18 20:13:49 -04:00
Ryan Richard
08c446a3e1 Use openssl to generate the test user password instead of /dev/urandom
Because it's more portable across different operating systems and
it is already pre-installed on MacOS.
2021-03-18 11:20:33 -07:00
Matt Moyer
b20a8358d3
Merge branch 'main' of github.com:vmware-tanzu/pinniped into impersonation-proxy 2021-03-08 15:16:40 -06:00
Margo Crawford
4bd68b1fa1 Use LC_ALL=C instead of LC_CTYPE=C because it works on Big Sur
It also works on the slightly older MacOS Catalina.
This script is only used on development laptops, so hopefully
this will work for more laptop OS's now.

Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-03-05 15:25:52 -08:00
Ryan Richard
7b7901af36 Add -timeout 0 when describing how to run integration tests
Because otherwise `go test` will panic/crash your test if it takes
longer than 10 minutes, which is an annoying way for an integration
test to fail since it skips all of the t.Cleanup's.
2021-03-03 12:53:41 -08:00
Andrew Keesler
26922307ad prepare-for-integration-tests.sh: New cmdline option --api_group_suffix
Makes it easy to deploy Pinniped under a different API group for manual
testing and iterating on integration tests on your laptop.

Signed-off-by: Ryan Richard <richardry@vmware.com>
2021-02-03 12:07:38 -08:00
Andrew Keesler
93d25a349f
hack: fix docker most recent tag check
I think this stopped working when we starting using a specific registry in e0b94f47.

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-02-02 18:01:07 -05:00
Ryan Richard
b5cbe018e3
Allow passing multiple redirect URIs to Dex
We need this in CI when we want to configure Dex with the redirect URI for both
primary and secondary deploys at one time (since we only stand up Dex once).

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2021-01-20 17:06:50 -05:00
Monis Khan
3c3da9e75d
Wire in new env vars for user info testing
Signed-off-by: Monis Khan <mok@vmware.com>
2021-01-12 11:23:25 -05:00
Matt Moyer
e0b94f4780
Move our main image references to the VMware Harbor registry.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-12-17 17:51:09 -06:00
Andrew Keesler
93cfd8c93a
Fix prepare-for-integration-tests.sh and Tiltfile for kubectl 1.20
kubectl 1.20 prints "Kubernetes control plane" instead of "Kubernetes master".

Signed-off-by: Andrew Keesler <akeesler@vmware.com>

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2020-12-09 10:15:34 -05:00
Matt Moyer
408fbe4f76
Parameterize the supervisor_redirect_uri in the test env Dex.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-12-03 12:45:56 -06:00
Matt Moyer
fe0481c304
In integration test env, deploy a ClusterIP service and register that with Dex.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-12-02 15:55:33 -06:00
Matt Moyer
bc700d58ae
Split test environment variables so there's a specific supervisor upstream client.
Prior to this we re-used the CLI testing client to test the authorize flow of the supervisor, but they really need to be separate upstream clients. For example, the supervisor client should be a non-public client with a client secret and a different callback endpoint.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-11-20 08:03:06 -06:00
Matt Moyer
6ce2f109bf
Refactor certificate generation for integration test Dex.
Before, we did this in an init container, which meant if the Dex pod restarted we would have fresh certs, but our Tilt/bash setup didn't account for this.

Now, the certs are generated by a Job which runs once and saves the generated files into a Secret. This should be a bit more stable.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-11-17 11:36:36 -06:00
Matt Moyer
b17ac6ec0b
Update integration tests to run Dex over HTTPS.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-11-16 20:23:20 -06:00
Matt Moyer
c8b17978a9
Convert CLI tests to work through an HTTP forward proxy.
This change deploys a small Squid-based proxy into the `dex` namespace in our integration test environment. This lets us use the cluster-local DNS name (`http://dex.dex.svc.cluster.local/dex`) as the OIDC issuer. It will make generating certificates easier, and most importantly it will mean that our CLI can see Dex at the same name/URL as the supervisor running inside the cluster.

Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-11-16 17:16:58 -06:00
Andrew Keesler
83a156d72b
Enable debug logging in all testing scenarios
It is really helpful to have verbose logs during test debugging.

Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2020-11-11 09:01:43 -05:00
Andrew Keesler
3bc13517b2
prepare-for-integration-tests.sh: add check for chromedriver
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2020-11-04 15:53:32 -08:00
Ryan Richard
2777c4e9f3 Update prepare-for-integration-tests.sh to use ./hack/kind-{up,down}.sh 2020-10-27 16:56:53 -07:00
Ryan Richard
1f1b6c884e Add integration test: supervisor TLS termination and SNI virtual hosting
- Also reduce the minimum allowed TLS version to v1.2, because v1.3
  is not yet supported by some common clients, e.g. the default MacOS
  curl command
2020-10-27 14:57:25 -07:00
Ryan Richard
8b7c30cfbd Supervisor listens for HTTPS on port 443 with configurable TLS certs
- TLS certificates can be configured on the OIDCProviderConfig using
  the `secretName` field.
- When listening for incoming TLS connections, choose the TLS cert
  based on the SNI hostname of the incoming request.
- Because SNI hostname information on incoming requests does not include
  the port number of the request, we add a validation that
  OIDCProviderConfigs where the issuer hostnames (not including port
  number) are the same must use the same `secretName`.
- Note that this approach does not yet support requests made to an
  IP address instead of a hostname. Also note that `localhost` is
  considered a hostname by SNI.
- Add port 443 as a container port to the pod spec.
- A new controller watches for TLS secrets and caches them in memory.
  That same in-memory cache is used while servicing incoming connections
  on the TLS port.
- Make it easy to configure both port 443 and/or port 80 for various
  Service types using our ytt templates for the supervisor.
- When deploying to kind, add another nodeport and forward it to the
  host on another port to expose our new HTTPS supervisor port to the
  host.
2020-10-26 17:03:26 -07:00
Ryan Richard
397ec61e57 Specify the supervisor NodePort Service's port and nodePort separately
When using kind we forward the node's port to the host, so we only
really care about the `nodePort` value. For acceptance clusters,
we put an Ingress in front of a NodePort Service, so we only really
care about the `port` value.
2020-10-22 15:37:35 -07:00
Ryan Richard
9ba93d66c3
test/integration: prefactoring for testing virtual hosts
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2020-10-20 17:00:36 -04:00
Ryan Richard
f8e461dfc3 Merge branch 'main' into label_every_resource 2020-10-15 10:19:03 -07:00
Ryan Richard
94f20e57b1 Concierge controllers add labels to all created resources 2020-10-15 10:14:23 -07:00
Andrew Keesler
87c7e9a556
hack/prepare-for-integration-tests.sh: default COLORTERM for when not set
Signed-off-by: Andrew Keesler <ankeesler1@gmail.com>
2020-10-14 20:18:10 -04:00
Matt Moyer
33fcc74417
Add Dex to our integration test environment and use it to test the CLI.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-10-13 16:50:38 -05:00
Ryan Richard
d81d395c80 Get ready to deploy Supervisor in CI and run its integration tests
- Also use ./test/integration instead of ./test/... everywhere because
  it will stream the output of the tests while they run
2020-10-09 18:07:13 -07:00
Ryan Richard
354b922e48 Allow creation of different Service types in Supervisor ytt templates
- Tiltfile and prepare-for-integration-tests.sh both specify the
  NodePort Service using `--data-value-yaml 'service_nodeport_port=31234'`
- Also rename the namespaces used by the Concierge and Supervisor apps
  during integration tests running locally
2020-10-09 16:00:11 -07:00
Ryan Richard
34549b779b Make tilt work with the supervisor app and add more uninstall testing
- Also continue renaming things related to the concierge app
- Enhance the uninstall test to also test uninstalling the supervisor
  and local-user-authenticator apps
2020-10-09 14:25:34 -07:00
Ryan Richard
72b2d02777 Rename integration test env variables
- Variables specific to concierge add it to their name
- All variables now start with `PINNIPED_TEST_` which makes it clear
  that they are for tests and also helps them not conflict with the
  env vars that are used in the Pinniped CLI code
2020-10-09 10:11:47 -07:00
Ryan Richard
f5a6a0bb1e Move all three deployment dirs under a new top-level deploy/ dir 2020-10-09 10:00:22 -07:00
Andrew Keesler
20ce142f90
Merge remote-tracking branch 'upstream/main' into supervisor-with-discovery 2020-10-07 11:37:33 -04:00
Andrew Keesler
8a772793b8
supervisor-oidc: fix PINNIPED_SUPERVISOR test env vars?
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2020-10-07 10:51:39 -04:00
Ryan Richard
ae56fcb46a Add integration test for the OIDC discovery endpoint
- Intended to be a red test in this commit; will make it go
  green in a future commit
- Enhance env.go and prepare-for-integration-tests.sh to make it
  possible to write integration tests for the supervisor app
  by setting more env vars and by exposing the service to the kind
  host on a localhost port
- Add `--clean` option to prepare-for-integration-tests.sh
  to make it easier to start fresh
- Make prepare-for-integration-tests.sh advise you to run
  `go test -v -count 1 ./test/integration` because this does
  not buffer the test output
- Make concierge_api_discovery_test.go pass by adding expectations
  for the new OIDCProviderConfig type
2020-10-06 17:53:29 -07:00
Ryan Richard
78cc49d658 Revert "supervisor-oidc: create dynamic config in YTT templates"
This reverts commit 006d96ab92.
2020-10-06 13:35:05 -07:00
Andrew Keesler
006d96ab92
supervisor-oidc: create dynamic config in YTT templates
Signed-off-by: Andrew Keesler <akeesler@vmware.com>
2020-10-06 11:16:57 -04:00
Ryan Richard
76bd462cf8 Implement very rough skeleton of the start of a supervisor server
- This is just stab at a starting place because it felt easier to
  put something down on paper than to keep staring at a blank page
2020-10-05 17:28:19 -07:00
Matt Moyer
b0a4ae13c5
Add Tilt-based local dev workflow.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-10-05 16:34:33 -05:00
Matt Moyer
9beb3855b5
Create webhooks per-test and explicitly in demo.md instead of with ytt in ./deploy.
Signed-off-by: Matt Moyer <moyerm@vmware.com>
2020-09-22 10:03:32 -05:00