Add support for composition functions to request OpenAPI schemas by negz · Pull Request #7022 · crossplane/crossplane

negz · 2026-01-15T02:24:18Z

Description of your changes

Composition functions sometimes need OpenAPI schemas for resource kinds - for example to validate resources, generate resources with correct field types, or build schema-aware tooling. Since functions can't assume network access per the function specification, they have no way to fetch schemas directly.

This PR extends the existing required resources pattern to support schemas. Functions return requirements.schemas specifying GVKs they need, and Crossplane populates required_schemas on the next request. If a schema isn't found, Crossplane returns an empty Schema{} to indicate it tried, matching the pattern used for required resources.

This PR also adds capability advertisement. Functions have no way to know which protocol features Crossplane supports - when a function uses a newer feature with an older Crossplane, the request is silently ignored. Crossplane now populates meta.capabilities with all supported features. Functions can check for a capability before relying on a feature, and fall back gracefully when absent. See design/one-pager-function-capabilities.md for details.

I have:

Read and followed Crossplane's contribution process.
Run earthly +reviewable to ensure this PR is ready for review.
Added or updated unit tests.
Added or updated e2e tests.
Linked a PR or a docs tracking issue to document this change.
- Document composition function required schemas docs#1040
- Document composition function capability advertisement docs#1041
~~Added backport release-x.y labels to auto-backport this PR.~~
~~Followed the API promotion workflow if this PR introduces, removes, or promotes an API.~~

negz · 2026-01-15T02:27:50Z

cmd/crank/render/render.go

 	}()

-	runner := xfn.NewFetchingFunctionRunner(runtimes, NewFilteringFetcher(append(in.ExtraResources, in.RequiredResources...)...))
+	runner := xfn.NewFetchingFunctionRunner(runtimes, NewFilteringFetcher(append(in.ExtraResources, in.RequiredResources...)...), xfn.NopRequiredSchemasFetcher{})


Not entirely sure how we'd pass schemas in crossplane render. We could add --required-schemas as a flag, but what would the file format be?

A flag makes sense to me. I think I'd make the flag value either <name>=<file> or <g/k>=<file> so the user explicitly specifies which file is for which requirement. Probably makes sense to accept both JSON and YAML OpenAPI specs.

How're users going to get those OpenAPI schemas conveniently though? Will they need to manually extract them from CRDs? What about built in types?

I'm wondering if we should support some combo of:

Plain OpenAPI v3 file

CRDs

Point it at a Kubernetes API server?

Maybe the last one is overkill considering you can presumably curl the OpenAPI endpoint and then use that.

Yes, seems like a good idea to support CRDs as well as plain OpenAPI. Maybe also XRDs (which we can easily convert to a CRD internally) so users don't have to jump through hoops to get specs for their own APIs.

Adding support for fetching from the k8s API in render feels a little weird, since render doesn't otherwise depend on a real cluster at all. But, it would probably be a simple change, and might make life easier for users.

just a side note for the render package to make RequiredSchemasFetcher configurable via the Inputs struct: c350330

as we know we using the render package by external CLIs (up-cli, future crossplane-cli split), they cannot import from internal/xfn, so the hardcoded xfn.NopRequiredSchemasFetcher{} blocked any schema fetching functionality - I know this is just a draft for discussion, just wanted to flag it, was we need to make sure its still working with the cli🙂

in my commit is a public render.RequiredSchemasFetcher interface that external tools can implement. For example, in up-cli i have a draft PR with a fetcher that uses the apiDependencies from our internal up-cli cache along with XRDs and CRDs from dependencies, we just needed the ability to inject it as an input parameter.

Note on OpenAPI schema handling: The hard part with OpenAPI schemas are $refs - they need to be flattened before they can be used as input ...

This should make it straightforward to wire up whatever flag format we settle on (OpenAPI files, CRDs, XRDs, or API server)...

Yes, seems like a good idea to support CRDs as well as plain OpenAPI.

@adamwg For now I've just added support for plain OpenAPI files, in the format you get if you curl the API server, so you could do that to grab them and save them to disk. I agree CRDs, XRDs, etc would be nice, but if we take that path we'd still need to support built-in types some other way, and also inject the ObjectMeta somehow (hardcode it?). With OpenAPI JSON files read from the API server you can include the component the ObjectMeta $ref points to, so clients can resolve the ref.

make RequiredSchemasFetcher configurable via the Inputs struct

@haarchri We don't currently do this for required resources, right? With that in mind let's handle this (making these configurable) separately from his PR, since we need to do something for both.

Crossplane PR #7022 adds support for composition functions to request OpenAPI schemas for resource kinds. When a function needs to validate or introspect a resource's schema, it can request the schema from Crossplane and receive it in subsequent invocations. This commit updates the protobuf definitions from PR #7022 and adds helper functions to work with schema requirements: - response.require_schema() declares a schema requirement - request.get_required_schema() retrieves the resolved schema The schema is returned as a plain dict, allowing callers to use it directly with validation libraries like openapi-schema-validator or jsonschema. Both get_required_resources() and get_required_schema() docstrings now document how to distinguish between "not yet resolved" and "resolved but not found" using the `name in req.required_*` pattern. Ref: crossplane/crossplane#7022 Signed-off-by: Nic Cope <[email protected]>

coderabbitai · 2026-01-16T01:19:09Z

📝 Walkthrough

Walkthrough

Adds per-step required OpenAPI schema selection and fetching: new selector types and protobuf fields, OpenAPI discovery cache and schema fetchers (online/offline), wiring through controllers and the function-runner, CRD/OpenAPI schema updates, unit and e2e tests, and capability advertisement for function requests.

Changes

Cohort / File(s)	Summary
API Types `apis/apiextensions/v1/composition_common.go`, `apis/ops/v1alpha1/operation_types.go`	Add `RequiredSchemaSelector` type and getters; add `RequiredSchemas []RequiredSchemaSelector` to `FunctionRequirements`.
Protobuf `proto/fn/v1/run_function.proto`, `proto/fn/v1beta1/zz_generated_run_function.proto`	Add `required_schemas` map to `RunFunctionRequest`; add `schemas` to `Requirements`; add `SchemaSelector`/`Schema` messages; introduce `Capability` enum and `RequestMeta.capabilities`.
Server-side OpenAPI fetcher & cache `internal/xfn/required_schemas.go`, `internal/xfn/cached_discovery.go`, `internal/xfn/required_schemas_test.go`	New `RequiredSchemasFetcher` interface and OpenAPI v3 fetcher; cached discovery wrapper with CRD-change invalidation; $ref collection and protobuf conversion; comprehensive tests.
Function-runner / middleware `internal/xfn/required_resources.go`, `internal/xfn/required_resources_test.go`, `internal/xfn/capabilities.go`, `internal/xfn/selector.go`	Extend FetchingFunctionRunner to accept a schemas fetcher, preserve/reset bootstrap schemas, fetch/populate `req.RequiredSchemas`; add capability helpers and selector conversions; tests updated.
Controller wiring `cmd/crossplane/core/core.go`, `internal/controller/.../options.go`, `internal/controller/apiextensions/composite/composition_functions.go`, `internal/controller/apiextensions/definition/reconciler.go`, `internal/controller/ops/operation/constructor.go`, `internal/controller/ops/operation/reconciler.go`	Create and expose cached OpenAPI client; add `OpenAPIClient` options; wire `WithRequiredSchemasFetcher` into reconcilers/composers; bootstrap now fetches required schemas and sets `RequestMeta.Capabilities`.
CRD manifests `cluster/crds/.../apiextensions.crossplane.io_compositions.yaml`, `..._compositionrevisions.yaml`, `ops.crossplane.io_operations.yaml`, `ops.crossplane.io_cronoperations.yaml`, `ops.crossplane.io_watchoperations.yaml`	Add `requiredSchemas` to pipeline step `requirements`: list-map keyed by `requirementName` with `apiVersion`, `kind`, `requirementName` and validation metadata.
CLI / offline schemas `cmd/crank/render/schemas.go`, `cmd/crank/render/schemas_test.go`, `cmd/crank/alpha/render/op/render.go`, `cmd/crank/render/render.go`, `cmd/crank/alpha/render/op/cmd.go`, `cmd/crank/render/cmd.go`	Add offline OpenAPIV3Schema loader and `FilteringSchemaFetcher`; CLI flags to load/pass schemas into Render; pass schema fetcher into runner; tests and examples updated.
E2E tests & manifests `test/e2e/apiextensions_compositions_test.go`, `test/e2e/manifests/apiextensions/composition/required-schemas/`, `test/e2e/manifests/apiextensions/composition/required-resources/`	Add e2e tests and manifests exercising required-schemas and required-resources flows (CRDs, compositions, functions, ConfigMap, XR manifests).
Docs / design `design/one-pager-required-schemas.md`, `design/one-pager-function-capabilities.md`	Add design docs describing required-schemas behavior, schema mapping, capability negotiation, and usage examples.

Sequence Diagram(s)

sequenceDiagram
    rect rgba(220,230,241,0.5)
    participant Controller as Controller
    participant Reconciler as Reconciler
    participant SchemasFetcher as SchemasFetcher
    participant Discovery as Discovery
    participant FunctionRunner as FunctionRunner
    participant Function as Function
    end

    Controller->>Reconciler: Build pipeline / bootstrap
    Reconciler->>SchemasFetcher: Fetch(ctx, SchemaSelector) for each requiredSchemas
    SchemasFetcher->>Discovery: Paths() / get OpenAPI v3 doc
    Discovery-->>SchemasFetcher: OpenAPI document / paths
    SchemasFetcher->>SchemasFetcher: Parse & collect $ref schemas
    SchemasFetcher-->>Reconciler: Return Schema (per requirementName)
    Reconciler->>FunctionRunner: RunFunction(req with RequiredSchemas, RequiredResources, Meta.Capabilities)
    FunctionRunner->>Function: Deliver RunFunctionRequest
    Function-->>FunctionRunner: RunFunctionResponse
    FunctionRunner-->>Reconciler: Results / status

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

fix(fn-cache): prevent caching responses with unfulfilled requirements #6961 — overlaps changes to function-runner middleware ordering and requirement-fetching wiring.

Suggested reviewers

bobh66
phisco

Would you like me to highlight the highest-risk files for focused review (schema collection, cached discovery invalidation, and fetching-runner tests)?

Important

Pre-merge checks failed

Please resolve all errors before merging. Addressing warnings is optional.

❌ Failed checks (1 error)

Check name	Status	Explanation	Resolution
Feature Gate Requirement	❌ Error	PR adds RequiredSchemas field to stable v1 API and core composition logic without feature gate, diverging from Crossplane's established pattern for experimental features.	Add feature flag (e.g., EnableBetaRequiredSchemas) to gate functionality, or confirm stability status and ensure all E2E tests pass with production-ready images before merging.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: adding support for composition functions to request OpenAPI schemas.
Description check	✅ Passed	The description comprehensively relates to the changeset, explaining the motivation, implementation approach, capability advertisement feature, and checklist items.
Breaking Changes	✅ Passed	All modifications in apis/ and cmd/ are purely additive with no field/flag removals, renames, or new required fields. New fields use omitempty JSON tags and optional CLI defaults, maintaining full backward compatibility.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)

cmd/crank/alpha/render/op/render.go (1)
81-146: Render doesn’t populate bootstrap required schemas yet. Thanks for wiring the new fetcher, but crank render still only fills req.RequiredResources from fn.Requirements. Should Render also mirror controller behavior by populating req.RequiredSchemas (even if it’s empty via the no-op fetcher) so function behavior matches between render and controllers?
🛠️ Proposed fix (mirror bootstrap required schemas)
-	runner := xfn.NewFetchingFunctionRunner(runtimes, render.NewFilteringFetcher(in.RequiredResources...), xfn.NopRequiredSchemasFetcher{})
+	schemaFetcher := xfn.NopRequiredSchemasFetcher{}
+	runner := xfn.NewFetchingFunctionRunner(runtimes, render.NewFilteringFetcher(in.RequiredResources...), schemaFetcher)
@@
 	if fn.Requirements != nil {
 		req.RequiredResources = map[string]*fnv1.Resources{}
 		for _, sel := range fn.Requirements.RequiredResources {
 			resources, err := render.NewFilteringFetcher(in.RequiredResources...).Fetch(ctx, xfn.ToProtobufResourceSelector(&sel))
 			if err != nil {
 				return Outputs{}, errors.Wrapf(err, "cannot fetch bootstrap required resources for requirement %q", sel.RequirementName)
 			}
 			req.RequiredResources[sel.RequirementName] = resources
 		}
+
+		req.RequiredSchemas = map[string]*fnv1.Schema{}
+		for _, sel := range fn.Requirements.RequiredSchemas {
+			schema, err := schemaFetcher.Fetch(ctx, xfn.ToProtobufSchemaSelector(&sel))
+			if err != nil {
+				return Outputs{}, errors.Wrapf(err, "cannot fetch bootstrap required schema for requirement %q", sel.RequirementName)
+			}
+			req.RequiredSchemas[sel.RequirementName] = schema
+		}
 	}
cluster/crds/ops.crossplane.io_operations.yaml (1)

126-133: Minor typo: "unction" should be "function".

I noticed a small typo in line 129-130: "will be passed to the unction" should be "will be passed to the function". Since this is a generated CRD, the fix would need to be in the source file (apis/ops/v1alpha1/operation_types.go, line 71-72 in the PipelineStep.Input field comment).

This appears to be pre-existing rather than introduced by this PR, so feel free to address it separately if you prefer!

🤖 Fix all issues with AI agents

In `@internal/xfn/cached_discovery.go`:
- Around line 51-64: The error messages returned from
CachedOpenAPIClient.InvalidateOnCRDChanges when ca.GetInformer or
i.AddEventHandler fail are too generic; update the wrapped errors to state that
schema cache invalidation for CRDs couldn't be set up and suggest checking CRD
RBAC/permissions (e.g., change the errors.Wrap calls around ca.GetInformer and
i.AddEventHandler to include text like "failed to set up CRD informer; schema
cache invalidation not configured — check RBAC/permissions for
CustomResourceDefinitions"). Reference CachedOpenAPIClient and
InvalidateOnCRDChanges, and keep the original wrapped error appended for
diagnostics.

In `@internal/xfn/required_resources.go`:
- Around line 134-141: Improve the operator-facing error message when
c.schemas.Fetch(ctx, selector) fails in the loop over
newRequirements.GetSchemas(): update the errors.Wrapf call to include actionable
guidance (e.g., suggest verifying the CRD is installed and that OpenAPI
discovery is reachable) while keeping context (include the schema name from the
loop), so operators know next steps; locate this inside the function iterating
newRequirements.GetSchemas() where c.schemas.Fetch is called and
req.RequiredSchemas is populated and replace the terse "fetching schema for %s"
wrap with a message that combines the schema name and a brief remediation hint.

🧹 Nitpick comments (3)

apis/ops/v1alpha1/operation_types.go (1)

205-232: RequiredSchemaSelector is duplicated across API groups.

This type is defined identically in both apis/ops/v1alpha1/operation_types.go and apis/apiextensions/v1/composition_common.go. I see that a conversion function exists in zz_generated.conversion.go to handle the separation. Understanding the reasoning here would be helpful: is this duplication intentional to keep the alpha ops API self-contained, or is there an opportunity to consolidate? If keeping them separate is a deliberate design choice (e.g., allowing independent evolution of the alpha and stable APIs), that makes sense!

The implementation itself is solid—the getters correctly satisfy the RequiredSchemaSelector interface from internal/xfn/selector.go.

test/e2e/manifests/apiextensions/composition/required-schemas/setup/functions.yaml (1)

6-8: TODO tracking for temporary function build.

Thanks for documenting this clearly! Could you confirm whether there's an issue or tracking item to remind the team to update this to the official function-python release once schema support is merged upstream? This would help ensure it doesn't get forgotten.

Would you like me to help create a tracking issue for this follow-up task?

cmd/crossplane/core/core.go (1)

34-35: OpenAPI discovery wiring looks good—please confirm RBAC/failure mode.

Thanks for adding the cached OpenAPI client and invalidation hook. Could you confirm the Crossplane ClusterRole includes GET on /openapi/v3 (non‑resource URL) so schema fetches won’t be RBAC‑blocked, and that failing to set up invalidation should be fatal? If it’s optional, a log‑and‑continue path might be safer.

Also applies to: 447-460

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between cdab813 and 383b465.

⛔ Files ignored due to path filters (5)

apis/apiextensions/v1/zz_generated.conversion.go is excluded by !**/zz_generated*.go and included by **/*.go
apis/apiextensions/v1/zz_generated.deepcopy.go is excluded by !**/zz_generated*.go and included by **/*.go
apis/ops/v1alpha1/zz_generated.deepcopy.go is excluded by !**/zz_generated*.go and included by **/*.go
proto/fn/v1/run_function.pb.go is excluded by !**/*.pb.go, !**/*.pb.go and included by **/*.go
proto/fn/v1beta1/zz_generated_run_function.pb.go is excluded by !**/*.pb.go, !**/zz_generated*.go, !**/*.pb.go and included by **/*.go

📒 Files selected for processing (34)

apis/apiextensions/v1/composition_common.go
apis/ops/v1alpha1/operation_types.go
cluster/crds/apiextensions.crossplane.io_compositionrevisions.yaml
cluster/crds/apiextensions.crossplane.io_compositions.yaml
cluster/crds/ops.crossplane.io_cronoperations.yaml
cluster/crds/ops.crossplane.io_operations.yaml
cluster/crds/ops.crossplane.io_watchoperations.yaml
cmd/crank/alpha/render/op/render.go
cmd/crank/render/render.go
cmd/crossplane/core/core.go
internal/controller/apiextensions/composite/composition_functions.go
internal/controller/apiextensions/controller/options.go
internal/controller/apiextensions/definition/reconciler.go
internal/controller/ops/controller/options.go
internal/controller/ops/operation/constructor.go
internal/controller/ops/operation/reconciler.go
internal/xfn/cached_discovery.go
internal/xfn/required_resources.go
internal/xfn/required_resources_test.go
internal/xfn/required_schemas.go
internal/xfn/required_schemas_test.go
internal/xfn/selector.go
proto/fn/v1/run_function.proto
proto/fn/v1beta1/zz_generated_run_function.proto
test/e2e/apiextensions_compositions_test.go
test/e2e/manifests/apiextensions/composition/required-resources/setup/composition.yaml
test/e2e/manifests/apiextensions/composition/required-resources/setup/configmap.yaml
test/e2e/manifests/apiextensions/composition/required-resources/setup/definition.yaml
test/e2e/manifests/apiextensions/composition/required-resources/setup/functions.yaml
test/e2e/manifests/apiextensions/composition/required-resources/xr.yaml
test/e2e/manifests/apiextensions/composition/required-schemas/setup/composition.yaml
test/e2e/manifests/apiextensions/composition/required-schemas/setup/definition.yaml
test/e2e/manifests/apiextensions/composition/required-schemas/setup/functions.yaml
test/e2e/manifests/apiextensions/composition/required-schemas/xr.yaml

🧰 Additional context used

📓 Path-based instructions (10)

{apis,cmd}/**

📄 CodeRabbit inference engine (Custom checks)

{apis,cmd}/**: Public fields/flags in apis/** or cmd/** must not be removed or renamed without a 'breaking-change' label
New required public fields/flags in apis/** or cmd/** must not be added without a 'breaking-change' label
Behavior removals in apis/** or cmd/** must be labeled with 'breaking-change'

Files:

apis/ops/v1alpha1/operation_types.go
cmd/crank/alpha/render/op/render.go
apis/apiextensions/v1/composition_common.go
cmd/crank/render/render.go
cmd/crossplane/core/core.go

apis/**

📄 CodeRabbit inference engine (Custom checks)

New experimental features affecting apis/** must have feature flag implementations

Files:

apis/ops/v1alpha1/operation_types.go
apis/apiextensions/v1/composition_common.go

**/*.go