[Feature Request] Allow using different charts and/or deploy different Dapr versions

[ItalyPaleAle]

This is based on #45

Currently, the operator is configured to install a Helm chart that is embedded into the binary. This chart is updated every time there's a new Dapr release.

We would like to propose the ability of specifying the Dapr version to deploy, and/or being able to point the operator to a different chart.

The goals would be:

1. Being able to install a different version of Dapr. For example, we could install Dapr 1.11 or Dapr 1.12. (We would maintain a list of allowed versions)
2. Being able to use the CRD to upgrade Dapr in the cluster. For example, users would change the version from "1.11.3" to "1.12.0" to perform an upgrade after the new resource is deployed.
3. (Possible future idea) Perform auto-upgrades. For example, the CRD could specify version "1.11" (no patch) which can be used to auto-upgrade the Dapr control plane to patch versions. Users can specify "1.11.4" to pin a specific patch version.

This could be done by embedding multiple versions of the Dapr's Helm chart. Alternatively (or as an additional option), the Helm chart could be fetched from a remote repository.

Stretch goal: allow using this to install pre-release versions of Dapr, such as from the "master" branch

[lburgazzoli]

I was originally looking into doing so however within OLM I think it would be better to leverage its capabilities such the channels [1] so we would have a 1.11.x, 1.12.x, etc channels. This would allow the user to pick the Dapr version to use to satisfy the point you mentioned by selecting the appropriate channel.

An advantage of this approach is that each operator knows how to upgrade from the previous release (or range of release) so we don't have to validate the operator against a number of Dapr releases and we can leverage OLM update graph [2] to perform incremental upgrades. Also, dedicated conversion web hook conversion can be added to perform any migration required by newer APIs.

[1] https://olm.operatorframework.io/docs/best-practices/channel-naming/
[2] https://olm.operatorframework.io/docs/concepts/olm-architecture/operator-catalog/creating-an-update-graph/

[salaboy]

@ItalyPaleAle Alessandro, do you think that would work for your use cases? I would definitely go with channels for that, as they were designed to make sure that the operator and dapr releases are properly validated.

[ItalyPaleAle]

This would not work for us, no :( We would need the ability to install any (supported) Dapr version just by changing one YAML, we cannot change the operator that is installed.

[salaboy]

@ItalyPaleAle ok.. let's talk about this.. maybe we can have two modes or something like that

[ItalyPaleAle]

The problem with such an approach is that:

• It assumes that versions always go forward. We have seen cases where people also want to roll back versions.
• We also need a way to deploy a Helm chart from the "master" branch
• We need to be able to allow users to set specific patch versions

[lburgazzoli]

@ItalyPaleAle @salaboy I think the fundamental question here is to understand what role this operator is expected to have.

In my experience, having an operator that is only meant to deploy something to Kubernetes is often over-killing especially if it wraps other installation options such as Helm, however, it is also true that the operator has to deploy something to Kubernetes before operating it.

Giving this context, I'm proposing to encode two distinct behavior in the operator:

• install
• cruise control

This can be achieved by:

1. deprecate the DaprControlPlane CRD
2. introduce a new DaprInstance CRD and related controller whose role is to set up a dapr instance (install) in a way it cope with the cases mentioned by @ItalyPaleAle
3. introduce a new opinionated DaprCruiseControl CRD and related controller which role is to encode operational behavior (cruise control) to manage a dapr instance.

To put this in the context of the operator's capability levels:

the DaprInstance is meant to implement levels 1 and 2 whereas the others are delegated to the DaprCruiseControl.

Note that those two resources are meant to be complementary so:

• if a user creates a DaprCruiseControl, then a DaprInstance will be created but such instance will be owned by the DaprCruiseControl CR which dictates i.e. the Dapr release to use.
• if a user creates a DaprInstance, then the operator will just install a dapr instance according to the spec.
• if a user creates a DaprInstance and a DaprCruiseControl, then the DaprCruiseControl will be moved to the error phase until the DaprInstance is removed

Now in relation to #45, I'd propose to:

• make the DaprInstance not strongly typed (at least as initial step) as I think that, if the goal is to support a broad range of Dapr releases, a user should be able to define any of the configuration options supported by the related chart so, i.e. if a new option is available in the main branch, then it will be usable by the user without requiring a new operator release (this does not prevent to perform some specific behavior, like setting the imagePullPolicy)
• make the DaprCruiseControl strongly typed and opinionated so it would expose only the minimum subset of configuration options we want to make configurable

[ItalyPaleAle]

I think this could work.

1. How would we install and upgrade Dapr versions with Install or Cruise Control?
2. Are charts still embedded in the binary? Or can they optionally be fetched from a remote repo?

[salaboy]

@ItalyPaleAle for the DaprInstance I don't see an issue with fetching any version.. as that will not be managed by the DaprCruiseControl

[lburgazzoli]

The DaprInstance would let users to define any version/repo and for what concern upgrades, then it is up to the user to set the new target version they want the operator to deploy (If the operator has any knowledge about steps that must be performed, then it will do them but that's not guaranteed).

For what concern the DaprCruiseControl, the operator dictates the version as it must have knowledge of what it is being installed which means that the operator will automatically upgrade dapr to the version it tracks.

There will be a single version embedded in the container but that is consumed only by the DaprCruiseControl, the DaprInstance won't have such limitation.

@ItalyPaleAle, @salaboy if the proposal is ok with you, I can start work on a POC over the next weeks.

[ItalyPaleAle]

I think that sounds very reasonable. Thanks for suggesting.

[lburgazzoli]

ok, so I'll work on a POC over the next weeks.

[salaboy]

@ItalyPaleAle @lburgazzoli if we are all in the same page.. let's work on a PoC for sure..
but let's make sure that @ItalyPaleAle also review to make sure that the proposal is covering his use cases..
@ItalyPaleAle is there something else that we should know about your requirements?

[ItalyPaleAle]

I think from what I understand, this makes sense.

[lburgazzoli]

I will definitely share my work as soon as I have something ready, like end of next week