OPRUN-4099: OLMv1 Deployment Configuration API

[oceanc80]

Enhancement extending OLMv1's ClusterExtension API to support deployment configuration in order to provide feature parity with OLMv0's SubscriptionConfig.

[openshift-ci-robot]

@oceanc80: This pull request references OPRUN-4099 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

Enhancement extending OLMv1's ClusterExtension API to support deployment configuration in order to provide feature parity with OLMv0's SubscriptionConfig.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[perdasilva]

I wonder what the story for the Selector is. I wonder if its to handle changes in the pod label selector in the operator's controller deployment between versions (the label selector in the deployment spec is immutable). This configuration could provide upgrade continuity across this type of breaking change.

[oceanc80]

I could also see it being used for blue/green deployments or other similar deployment strategies.

[joelanford]

The selector of a deployment is immutable, iirc. I dove into the history of this field, and it looks like it was basically there from the beginning with no real explanation that I could find, and it has never been honored as far as I can tell.

Chalk it up to how fast and loose the early days of OLM were.

[kuiwang02]

@oceanc80 I know the PR is in WIP, and I also am not sure if the following comment is the scope of this EP.

If the following comment is not the scope of this EP or it is not correct time to raise the comments, you could ignore the following the comment:

---

The deploymentConfig API design looks well thought out. I noticed that the proposal currently focuses on initial installation scenarios, and I was wondering if you could clarify the behavior for runtime configuration updates, which I expect will be a common operational workflow.

Question 1: Modifying Existing deploymentConfig Values

Scenario: After creating a ClusterExtension with deploymentConfig, a user wants to update some values (e.g., changing memory limits from 256Mi to 512Mi, or adding a new nodeSelector).

Could you clarify:

1. Will users be able to modify spec.config.inline.deploymentConfig values after ClusterExtension creation?
2. If supported, will the changes automatically reconcile and apply to the existing Deployment?
3. Which configuration field changes will trigger a pod rolling update?
4. Are there any fields that don't support runtime updates?

Example:

  # Initial configuration
  deploymentConfig:
    resources:
      limits:
        memory: "256Mi"

  # User updates to:
  deploymentConfig:
    resources:
      limits:
        memory: "512Mi"      # ← modified
    nodeSelector:            # ← added
      infrastructure: "dedicated"

Question 2: Adding deploymentConfig After Creation

Scenario: A user creates a ClusterExtension without defining deploymentConfig initially, then later wants to add deployment configuration.

Could you clarify:

1. Is it supported to add deploymentConfig to an existing ClusterExtension that was created without it?
2. If supported, will the changes automatically reconcile and apply to the existing Deployment?
3. Which new configuration field will trigger a pod rolling update?
4. Are there any fields that don't support runtime added?
5. Similarly, what happens if a user removes deploymentConfig entirely?

Example:

  # Initial creation (no deploymentConfig)
  apiVersion: olm.operatorframework.io/v1
  kind: ClusterExtension
  metadata:
    name: my-operator
  spec:
    source:
      sourceType: Catalog
      catalog:
        packageName: my-operator
    # Note: no config.inline.deploymentConfig

  ---

  # Later, user adds deploymentConfig
  spec:
    config:
      inline:
        deploymentConfig:    # ← newly added
          nodeSelector:
            infrastructure: "dedicated"

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign jmguzik for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[perdasilva]

@kuiwang02 let me try to reply to your questions

Question 1: Modifying Existing deploymentConfig Values

1. Will users be able to modify spec.config.inline.deploymentConfig values after ClusterExtension creation?

From the perspective of OLMv1, bundle configuration is opaque. It will take user input, validated it against the configuration schema provided by the bundle, and apply it to generate the final manifests. So, any configuration can
be changed at runtime. This does mean that some user configurations might generate manifests that cannot be applied, or could lead to unintended or bad consequences. If there are errors, the extension will be in a broken state until the configuration is fixed.

2. If supported, will the changes automatically reconcile and apply to the existing Deployment?

Yes. The Deployment will be regenerated with the new values and applied to the cluster.

3. Which configuration field changes will trigger a pod rolling update?

I'd say so, yes. Any changes to the pod template should trigger a new replicaset and the deployment will transition towards that.

4. Are there any fields that don't support runtime updates?

This is a good question. I know there are fields in the Deployment spec that are immutable (e.g. the label selector). That the only one I can think of. I think the configuration options under the deployment config are mutable.

Question 2: Adding deploymentConfig After Creation

1. Is it supported to add deploymentConfig to an existing ClusterExtension that was created without it?

Yes. For the same reasons in Q1.1

2. If supported, will the changes automatically reconcile and apply to the existing Deployment?

Yes.

3. Which new configuration field will trigger a pod rolling update?

Yes.

4. Are there any fields that don't support runtime added?

I don't think so.

5. Similarly, what happens if a user removes deploymentConfig entirely?

Then we are back to the Deployment spec defined in the bundle by the author.

The mental model here is really no different than:

1. Create a Deployment
2. Modify the deployment

AFAIK only the pod label selector is immutable once set.

[kuiwang02]

immutable

@perdasilva Thanks for your great reply. I got it.

[anik120]

@JoelSpeed PTAL, thanks!

[openshift-ci]

@oceanc80: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/markdownlint	fd68cdf	link	true	/test markdownlint

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[JoelSpeed]

Sharing types between APIs like this is generally discouraged. Someone may make a change to the SubscriptionConfig that you didn't realise and then you end up either supporting (or breaking) functionality inadvertently in the DeploymentConfig usage.

Is the code shared between v0 and v1 for this part of the deployment strategy, or are you just trying to make sure you support the same set of customizations? Is there anything in the alpha API that you might want to make better from lessons learned as you promote this to v1?

[anik120]

@JoelSpeed we had this discussion upstream where I'd proposed a new, completely separate structure for v1, but it was vetoed in favor of keeping v0 and v1 in sync.

[joelanford]

The purpose of this enhancement is to provide a feature that is needed for the migration pathway from OLMv0-installed operators to OLMv1 ClusterExtensions. We explicitly need all subscription.spec.config fields and behaviors to be supported in OLMv1 to ensure migration from OLMv0 to OLMv1 is not blocked on a diff between the configuration APIs.

The idea here is that a migration tool would be able to take the subscription.spec.config and copy it into the clusterextension.spec.config.inline.deploymentConfig.

So one way or another the OLMv1 bundle-schema for deploymentConfig needs to include everything that OLMv0 includes for equivalent OCP versions.

[JoelSpeed]

What do you mean by json schema validation here?

[anik120]

These two PRs can shed more light on this:

1. ✨ Add Bundle-Agnostic Configuration Validation Using JSON Schema operator-framework/operator-controller#2316
2. ✨ Add automated schema generation and validation for registry+v1 bundle configuration operator-framework/operator-controller#2454

[JoelSpeed]

Why can't you reject invalid configuration at admission time instead of leaving a controller to check this asynchronously?

[perdasilva]

Hey Joel, this enhancement is related to this previous one, where all of this was discussed. The TL;DR is, OLMv1 aims to get out of the bundle configuration game, therefore there won't but a one-size-fits-all configuration schema to validate against at ingress time. We want to shift the configuration surface definition to the authors and allow them to provide content-specific knobs. OLMv1 will only validate configuration against a bundle provided schema.

[JoelSpeed]

Yes it does? You're adding a new field with significant structure to the ClusterExtension API as far as I can tell from reading this?

Even if this is a new structure within something that is effectively unstructured data, you are still introducing a new API contract, this should be explained here and reviewed

Can you link to the existing schema and show where it is updated to allow this deploymentConfig field?

[joelanford]

The CRD doesn't change. inline remains as is:

https://github.com/operator-framework/operator-controller/blob/1ef820f0ca56126586fca2dc7a422c71edd7deef/api/v1/clusterextension_types.go#L193

There are two lenses to this (and I feel like a lot of this was already discussed in the original EP):

1. OLMv1 conceptually claims no control over schemas provided by bundles. Bundles define the schemas of their APIs, OLM validates the provided configurations using the bundle-provided schema.
2. Technically, the registry+v1 schema definition is driven by two things:
   • The install modes of the operator (this dictates the schema of the watch namespace configuration)
   • OLMv0's subscription.spec.config

So this EP is not technically a new contract. It is establishing the OLMv1 implementation of an existing contract (OLMv0's subscription.spec.config)

Follow-up question: Is OCP okay with every single layered product being able to define their own configuration schema in the future? If so, will every layered product be expected to bring their configuration schema to OpenShift's API review? This API change is essentially the equivalent that.

If that seems untenable, perhaps we should drop the configuration API from OLMv1 entirely and require that layered products have a one-size-fits-all deployment (similar to core payload operators). But that would have major implications on migration from OLMv0 to OLMv1 that PM would have to weigh in on.

[JoelSpeed]

These generated schemas are not particularly reliable sources of information, why can you not validate using the internal validation packages within these APIs?

[anik120]

They are reliable sources of information in the sense that they're generated from the source itself. ref: operator-framework/operator-controller#2454

I'm not sure what you mean by internal validation packages. I'm guessing the PR makes things more clear?

[JoelSpeed]

Accepting but ignoring a field is bad practice. Why not create a new type for the deployment config? It doesn't look like it'll be particularly complex to implement

[anik120]

Same answer as above:

we had this discussion upstream where I'd proposed a new, completely separate structure for v1, but it was vetoed in favor of keeping v0 and v1 in sync.

It was discussed that reusing the v0 structure would mean carrying over debts, but the cost of it was assessed to be acceptable for long term maintainability

[joelanford]

I agree with @JoelSpeed 's point here. Even if we re-use the type, we are also in control of the schema generation for that type, right? So at a minimum we could specifically exclude that field from the generated schema.

[JoelSpeed]

How do you know that the schema generated by your v1 tooling is the same as the v0 tooling? What if there's drift between the generators?

[anik120]

The nature of the generators in v0 and v1 are completely different. In v0 they were part of the API surface, in v1 they are not. ie different validation structure in v1 and v0, but both are tied together by the underlying SubscritionConfig structure.

And the v1 generator uses SubscriptionConfig struct to generate the schema.

In other words, this is why it was decided to use the v0 SubscripitonConfig directly, so that any drift in v0, automatically gets tracked in v1.

Note that the schema is not expected to change in v1 (thereby requiring a tracking back of change to v0), as this effort is solely to support old registry+v1 bundles in v1 so that they can be migrated over.

[JoelSpeed]

Will you have checks in place so that people keep this up to date?

[anik120]

Yes upstream runs verify every time and this check has been included in that.

[JoelSpeed]

Are you doing any processing of these fields, or just setting them directly on the deployment that you're rendering and applying? If you aren't processing them and are just passing them through, then this is probably fine

[anik120]

Just passing them through without any processing. See https://docs.google.com/document/d/18O4qBvu5I4WIJgo5KU1opyUKcrfgk64xsI3tyXxmVEU/edit?tab=t.0#bookmark=kix.1y89z5sf1akf for more details.