ContainerImage.Pinniped/proposals/xxxx_carvel-package

title	authors	status	sponsor	approval_date
Carvel Package Management for Pinniped	@benjaminapetersen	in-review	@cfryanr @joshuatcasey

Disclaimer: Proposals are point-in-time designs and decisions. Once approved and implemented, they become historical documents. If you are reading an old proposal, please be aware that the features described herein might have continued to evolve since.

Carvel Package Management for Pinniped

Problem Statement

There are a number of tools available to the Kubernetes ecosystem for deploying complex software to a Kubernetes cluster. The Carvel toolchain provides a set of APIs, Custom Resources and CLI tools that can aid a user in the configuration and lifecycle management of software deployed to a cluster. We should enhance our deployment options by providing Carvel Packages that may be installed on a cluster configured with kapp-controller to manage the software on the cluster.

How Pinniped Works Today (as of version v0.25.0)

The ./deploy directory in the root of the Pinniped repository contains a set of ytt templates that:

• Are pre-rendered into installable templates listed with each Pinniped relese:
  • v0.25.0
• Can optionally be customized and rendered by a consumer of the Pinniped project by cloning down the github repository, making changes to the values.yaml file and then rendered via ytt.

Terminology / Concepts

• Carvel is an open-source project that provides tools for managing software build, configuration and deployment on a Kubernetes cluster. For more information read the Carvel docs.
• kapp is a Carvel provided CLI tool for deploying software onto a Kubernetes cluster. See the docs for more information.
• imgpkg is a Carvel provided CLI tool that provides a mechanism for collecting configuration and OCI images into a bundle that can be deployed on a cluster.
• kapp-controller is a server side component managing software in the form of Carvel Apps delivered via Carvel Packages. See the docs for more information.
• PackageRepository is a custom resource that configures kapp-controller with a set of Packages on a cluster. See the docs for more information.
• Package is a custom resource that represents configuration in the form of metadata and OCI images that may be used to deliver software onto a Kuberentes cluster. See the docs for more information.
• PackageMetadata is a custom resource describing attributes for a Package. See the docs for more information.

Proposal

Allow Pinniped to be deployed onto a Kuberentes cluster through the mechanism of two Carvel Packages, a Supervisor and a Concierge package. These may be delivered via a PackageRepository resource.

Goals and Non-goals

Goals

• Provide an additional deployment option to deliver Pinniped software to a Kubernetes cluster in the form of the Package apis provided by the Carvel toolchain.

Non-Goals

• Provide additional deployment alternatives, such as official Helm charts

API Changes

No changes or additions to Pinniped APIs, this proposal represents a second, alternative method for deployment utilising Carvel APIs.

Upgrades

Upgrading Pinniped via the Carvel Package mechanism will look something like this:

• We deliver a PackageRepository that lists:
  • Pinniped Supervisor package at a number of versions (ex: 0.23.0,0.24.0,0.25.0, etc)
  • Pinniped Concierge package at a number of versions (ex: 0.23.0,0.24.0,0.25.0, etc)
• The user installs the Pinniped PackageRepository.
• The user creates:
  • A Supervisor and Concierge PackageInstall Custom Resource (and Secret) with the following:

spec:  
  packageRef:
    refName: "supervisor.pinniped.dev"
    versionSelection:
      # - Constraints control the version and upgrades of the package
      #   if the constraint is pinned to a specific version, then only 
      #   that version is installed on the cluster
      # - If the PackageResource no longer serves a version that matches
      #   the constraint, then the PackageInstall will enter an error state
      #   until the constraint is updated      
      constraints: "0.25.0"  
      # - Alternatively, a constraint can be based on a semver range and can 
      #   control automatic updates.  
      constraints: ">0.25.0"  

Tests

Our current integration test suite uses the ./deploy directory to deploy Pinniped onto a variety of clusters. We should continue to test this for the majority of users who do not integrate with Carvel. In addition, we should update at least 1 of our tests to make use of the new Package mechanism to ensure it functions correctly.

Optionally we can:

• Change a single test to deploy via the Package mechanism
• Change several tests to deploy via the Package mechanism
• Provide a flag to our ./prepare-for-integration-test.sh and cycle all of our tests, perhaps randomly back and forth between the simple deploy and the Package deploy.

New Dependencies

The Carvel toolset is not strickly a dependency for Pinniped itself. This proposal is an optional method for delivering the Pinniped software to a kubernetes cluster. Therefore, kapp, kapp-controller, and the custom resources such as PackageRepository, Package, PackageMetadata, PackageInstall as well as imgpkg, vendir and ytt are all optional dependencies for a consumer of Pinniped.

Performance Considerations

None. It is up to the user to determine if the adoption of the Carvel toolset is the correct decision for their application lifecycle needs.

Observability Considerations

Package, App and PackageInstall custom resources contain a status field like many Kubernetes resources. These are thoroughly detailed with relevant bits of information that may aid the user in understanding the state of their applications. For example, a Carvel App Custom Resource (disambiguation, a Carvel app (lowercase) from kapp vs an App (uppercase) from kapp-controller are entirely different resources) contains a detailed status referencing all resources owned by the App. This is very helpful when attempting to understand the state of a complex multi-component application.

Security Considerations

Carvel is a toolset separate from Pinniped. This feature is optional, users who choose to use Carvel should assess Carvel for its risks and treadeoffs.

Usability Considerations

As of today the ./deploy directory of Pinniped is implemented via the use of a subset of the Carvel toolchain, namely, ytt. However, it is implemented in such a way that consumers of Pinniped have choice, they may opt-in to the use of the Carvel toolchain, or simply kubectl apply -f our pre-rendered yaml files, there is no requirement to use the Carvel toolchain.

This feature serves the community users who have deeply adopted Carvel into their management, such that kapp-controller is installed on their cluster and used to manage software lifecycle.

Documentation Considerations

This design doc serves as an announcement that the feature will be implemented. It would be helpful to provide a blog post describing how the feature was validated. Also include in release notes.

Other Approaches Considered

None.

Open Questions

A list of questions that need to be answered.

Answered Questions

Implementation Plan

Implementation PRs

• TBD