Error during access to the item of the list #78

khunafin · 2024-03-18T20:59:06Z

What happened?

During the access to the list item by index or by the first function inside the template, I got the error:

      at <index .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs
      0>: error calling index: index of untyped nil

log data

2024-03-18T20:45:49.992Z	DEBUG	fn/fn.go:60	template	{"template": "---\napiVersion: example.acme.io/v1beta1\nkind: Debug\nmetadata:\n  name: test-debug\n  annotations:\n    gotemplating.fn.crossplane.io/composition-resource-name: test-debug\nspec:\n  ip: {{ (first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip  }}\n"}
2024-03-18T20:45:49.993Z	DEBUG	fn/fn.go:74	constructed request map	{"request": {"context":{"apiextensions.crossplane.io/environment":{}},"desired":{"composite":{"resource":{"apiVersion":"example.acme.io/v1beta1","kind":"XR"}},"resources":{"pod":{"resource":{"apiVersion":"kubernetes.crossplane.io/v1alpha2","kind":"Object","spec":{"forProvider":{"manifest":{"apiVersion":"v1","kind":"Pod","metadata":{"name":"nginx","namespace":"default"},"spec":{"containers":[{"image":"nginx:1.14.0","name":"nginx"}]}}},"providerConfigRef":{"name":"kubernetes-provider"}}}}}},"input":{"apiVersion":"gotemplating.fn.crossplane.io/v1beta1","inline":{"template":"---\napiVersion: example.acme.io/v1beta1\nkind: Debug\nmetadata:\n  name: test-debug\n  annotations:\n    gotemplating.fn.crossplane.io/composition-resource-name: test-debug\nspec:\n  ip: {{ (first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip  }}\n"},"kind":"GoTemplate","source":"Inline"},"observed":{"composite":{"resource":{"apiVersion":"example.acme.io/v1beta1","kind":"XR","metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"example.acme.io/v1beta1\",\"kind\":\"XR\",\"metadata\":{\"annotations\":{},\"name\":\"test\"},\"spec\":{\"data\":\"top-secret\"}}\n"},"creationTimestamp":"2024-03-18T20:44:44Z","finalizers":["composite.apiextensions.crossplane.io"],"generation":3,"labels":{"crossplane.io/composite":"test"},"managedFields":[{"apiVersion":"example.acme.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:finalizers":{".":{},"v:\"composite.apiextensions.crossplane.io\"":{}},"f:labels":{".":{},"f:crossplane.io/composite":{}}},"f:spec":{"f:compositionRef":{".":{},"f:name":{}},"f:compositionRevisionRef":{".":{},"f:name":{}}}},"manager":"crossplane","operation":"Update","time":"2024-03-18T20:44:44Z"},{"apiVersion":"example.acme.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:status":{".":{},"f:conditions":{".":{},"k:{\"type\":\"Synced\"}":{".":{},"f:lastTransitionTime":{},"f:message":{},"f:reason":{},"f:status":{},"f:type":{}}}}},"manager":"crossplane","operation":"Update","subresource":"status","time":"2024-03-18T20:44:44Z"},{"apiVersion":"example.acme.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubectl.kubernetes.io/last-applied-configuration":{}}},"f:spec":{".":{},"f:compositionUpdatePolicy":{},"f:data":{}}},"manager":"kubectl-client-side-apply","operation":"Update","time":"2024-03-18T20:44:44Z"}],"name":"test","resourceVersion":"654357","uid":"1aa8b1d7-b30f-4d3b-be9f-05f8192232fa"},"spec":{"compositionRef":{"name":"function-go-template"},"compositionRevisionRef":{"name":"function-go-template-1737ec3"},"compositionUpdatePolicy":"Automatic","data":"top-secret"},"status":{"conditions":[{"lastTransitionTime":"2024-03-18T20:44:44Z","message":"cannot compose resources: pipeline step \"go-templating\" returned a fatal result: cannot execute template: template: manifests:9:10: executing \"manifests\" at <first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs>: error calling first: runtime error: invalid memory address or nil pointer dereference","reason":"ReconcileError","status":"False","type":"Synced"}]}}}}}}

How can we reproduce it?

composition.yaml

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: function-go-template
spec:
  compositeTypeRef:
    apiVersion:  example.acme.io/v1beta1
    kind: XR
  mode: Pipeline
  pipeline:
    - step: patch-and-transform
      functionRef:
        name: function-patch-and-transform
      input:
        apiVersion: pt.fn.crossplane.io/v1beta1
        kind: Resources
        resources:
          - name: pod
            base:
              apiVersion: kubernetes.crossplane.io/v1alpha2
              kind: Object
              spec:
                providerConfigRef:
                  name: kubernetes-provider
                forProvider:
                  manifest:
                    apiVersion: v1
                    kind: Pod
                    metadata:
                      name: nginx
                      namespace: default
                    spec:
                      containers:
                        - name: nginx
                          image: nginx:1.14.0
    - step: go-templating
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            ---
            apiVersion: example.acme.io/v1beta1
            kind: Debug
            metadata:
              name: test-debug
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: test-debug
            spec:
              ip: {{ (first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip  }}

definition.yaml

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xrs.example.acme.io
spec:
  group: example.acme.io
  names:
    kind: XR
    plural: xrs
  versions:
    - name: v1beta1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                data:
                  type: string

---
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: debugs.example.acme.io
spec:
  group: example.acme.io
  names:
    kind: Debug
    plural: debugs
  versions:
    - name: v1beta1
      served: true
      referenceable: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                ip:
                  type: string

xr.yaml

apiVersion: example.acme.io/v1beta1
kind: XR
metadata:
  name: test
spec:
  data: top-secret

What environment did it happen in?

function-go-templating v0.4.1
Crossplane v0.15.1

The text was updated successfully, but these errors were encountered:

phisco · 2024-03-19T13:30:38Z

That's how first works in sprig, https://masterminds.github.io/sprig/lists.html. You can use mustFirst which counter intuitively returns an error instead of panicking

khunafin · 2024-03-19T14:08:43Z

The question is how can I get a particular list item?

I tried to use {{ (index .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs 0).ip }}
instead of first and got the same error.

khunafin · 2024-03-19T14:16:36Z

Also, I have run locally the Crossplane CLI.

crossplane beta render -o observed.yaml xr.yaml composition.yaml functions.yaml -r

---
apiVersion: example.crossplane.io/v1beta1
kind: XR
metadata:
  name: example
status:
  conditions:
  - lastTransitionTime: "2024-01-01T00:00:00Z"
    message: 'Unready resources: test-debug'
    reason: Creating
    status: "False"
    type: Ready
---
apiVersion: example.acme.io/v1beta1
kind: Debug
metadata:
  annotations:
    crossplane.io/composition-resource-name: test-debug
  generateName: example-
  labels:
    crossplane.io/composite: example
  name: test-debug
  ownerReferences:
  - apiVersion: example.crossplane.io/v1beta1
    blockOwnerDeletion: true
    controller: true
    kind: XR
    name: example
    uid: ""
spec:
  ip: 192.168.65.3 # <---- the ip value

log data:

2024-03-19T15:08:09.617+0100    INFO    function-go-templating/fn.go:44 Running Function        {"tag": ""}
2024-03-19T15:08:09.618+0100    DEBUG   function-go-templating/fn.go:60 template        {"template": "---\napiVersion: example.acme.io/v1beta1\nkind: Debug\nmetadata:\n  name: test-debug\n  annotations:\n    gotemplating.fn.crossplane.io/composition-resource-name: test-debug\nspec:\n  ip: {{ (first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip }}\n"}
2024-03-19T15:08:09.619+0100    DEBUG   function-go-templating/fn.go:74 constructed request map {"request": {"context":{},"desired":{},"input":{"apiVersion":"gotemplating.fn.crossplane.io/v1beta1","inline":{"template":"---\napiVersion: example.acme.io/v1beta1\nkind: Debug\nmetadata:\n  name: test-debug\n  annotations:\n    gotemplating.fn.crossplane.io/composition-resource-name: test-debug\nspec:\n  ip: {{ (first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip }}\n"},"kind":"GoTemplate","source":"Inline"},"observed":{"composite":{"resource":{"apiVersion":"example.crossplane.io/v1beta1","kind":"XR","metadata":{"name":"example"},"spec":{"count":2}}},"resources":{"pod":{"resource":{"apiVersion":"kubernetes.crossplane.io/v1alpha2","kind":"Object","metadata":{"annotations":{"crossplane.io/composition-resource-name":"pod"}},"spec":{"forProvider":{"manifest":{"apiVersion":"v1","kind":"Pod"}}},"status":{"atProvider":{"manifest":{"apiVersion":"v1","kind":"Pod","metadata":{"name":"nginx","namespace":"default"},"spec":{"containers":[{"image":"nginx:1.14.2"}]},"status":{"hostIPs":[{"ip":"192.168.65.3"}],"phase":"Running"}}}}}}}}}}
2024-03-19T15:08:09.621+0100    DEBUG   function-go-templating/fn.go:83 rendered manifests      {"manifests": "---\napiVersion: example.acme.io/v1beta1\nkind: Debug\nmetadata:\n  name: test-debug\n  annotations:\n    gotemplating.fn.crossplane.io/composition-resource-name: test-debug\nspec:\n  ip: 192.168.65.3\n"}
2024-03-19T15:08:09.621+0100    DEBUG   function-go-templating/fn.go:201        desired composite resource      {"desiredComposite:": {"Resource":{},"ConnectionDetails":{}}}
2024-03-19T15:08:09.621+0100    DEBUG   function-go-templating/fn.go:202        constructed desired composed resources  {"desiredComposed:": {"test-debug":{"Resource":{"apiVersion":"example.acme.io/v1beta1","kind":"Debug","metadata":{"annotations":{},"name":"test-debug"},"spec":{"ip":"192.168.65.3"}},"Ready":""}}}
2024-03-19T15:08:09.621+0100    INFO    function-go-templating/fn.go:214        Successfully composed desired resources {"source": "Inline", "count": 1}

composition.yaml

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: example-function
spec:
  compositeTypeRef:
    apiVersion: example.crossplane.io/v1beta1
    kind: XR
  mode: Pipeline
  pipeline:
    - step: render-templates
      functionRef:
        name: function-go-templating
      input:
        apiVersion: gotemplating.fn.crossplane.io/v1beta1
        kind: GoTemplate
        source: Inline
        inline:
          template: |
            ---
            apiVersion: example.acme.io/v1beta1
            kind: Debug
            metadata:
              name: test-debug
              annotations:
                gotemplating.fn.crossplane.io/composition-resource-name: test-debug
            spec:
              ip: {{ (first .observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip }}

functions.yaml

apiVersion: pkg.crossplane.io/v1beta1
kind: Function
metadata:
  name: function-go-templating
  annotations:
    render.crossplane.io/runtime: Development
spec:
  package: function-go-templating

observed.yaml

Some fields were removed for brevity.

apiVersion: kubernetes.crossplane.io/v1alpha2
kind: Object
metadata:
  annotations:
    crossplane.io/composition-resource-name: pod
spec:
  forProvider:
    manifest:
      apiVersion: v1
      kind: Pod
status:
  atProvider:
    manifest:
      apiVersion: v1
      kind: Pod
      metadata:
        name: nginx
        namespace: default
      spec:
        containers:
          - image: nginx:1.14.2
      status:
        hostIPs:
          - ip: 192.168.65.3
        phase: Running

xr.yaml

apiVersion: example.crossplane.io/v1beta1
kind: XR
metadata:
  name: example
spec:
  count: 2

khunafin · 2024-04-29T16:52:26Z

I solved the problem in this way

....
{{ if  $.observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs }}
value: {{ (first $.observed.resources.pod.resource.status.atProvider.manifest.status.hostIPs).ip  }}
{{ end }}

In any case how fast should the reconciliation loop fail after getting an error?
hostIPs will eventually show up, but the loop already has failed.

laseanb · 2024-05-14T18:45:25Z

I think I was coming across the same problem - When needing the status of another managed resource that was created in the same composition - any reference would fail ie index $.observed.resources "name-of-resource" because it seemed like the reconciliation loop never picked up the updated $.observed.resources context.

bobh66 · 2024-08-31T13:15:25Z

If one resource in the template depends on status information from another resource, that information will never be present on the first pass through reconciliation, and may not show up for several iterations depending on how long it takes the resource to be created. It is necessary to check for the existence of the data before trying to access it (or use dig) to allow the template to create successfully so the dependency can be created.

khunafin added the bug Something isn't working label Mar 18, 2024

jbw976 added the good first issue Good for newcomers label Apr 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error during access to the item of the list #78

Error during access to the item of the list #78

khunafin commented Mar 18, 2024

phisco commented Mar 19, 2024

khunafin commented Mar 19, 2024

khunafin commented Mar 19, 2024 •

edited

Loading

khunafin commented Apr 29, 2024

laseanb commented May 14, 2024

bobh66 commented Aug 31, 2024

Error during access to the item of the list #78

Error during access to the item of the list #78

Comments

khunafin commented Mar 18, 2024

What happened?

How can we reproduce it?

What environment did it happen in?

phisco commented Mar 19, 2024

khunafin commented Mar 19, 2024

khunafin commented Mar 19, 2024 • edited Loading

khunafin commented Apr 29, 2024

laseanb commented May 14, 2024

bobh66 commented Aug 31, 2024

khunafin commented Mar 19, 2024 •

edited

Loading