Support pluggable waypoint

[linsun]

Describe the feature request

A design from @louiscryan is being socialized in next week's ambient WG meeting on how vendors can plug its own waypoint with what contracts.

Opening this issue to start some tasks for it. Immediate tasks after discussing with @ilrudie:

1. Ensure our istio waypoint object has proper address to inform ztunnel the waypoint service addr: A status field on the Gateway resource identifying the Waypoint endpoints using GatewayStatus.Addresses
2. Allow user to deploy its custom waypoint and being recognized as its custom waypoints. Provide a sample custom waypoint type for testing.

• Need consensus if istio-waypoint is the default Istio waypoint impl and anything waypoint means custom waypoint.

3. For istiod/ztunnel - able to detect the custom waypoint and use its address to configure ztunnel via xDS to the waypoint's address to when the destination has a waypoint.

@ilrudie feel free to add anything i missed.

Affected product area (please put an X in all that apply)

[ x ] Ambient
[ ] Docs
[ ] Dual Stack
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure

Affected features (please put an X in all that apply)

[ ] Multi Cluster
[ ] Virtual Machine
[ ] Multi Control Plane

Additional context

[ilrudie]

Right now we have some requirements for how our own waypoint controller communicates readiness with the ambient controllers. Depending on how folks feel this should probably be included in any formalized specification because if these rules are followed waypoint configuration in zt can be done only after the waypoint is ready to handle traffic which makes adding L7 less disruptive. It additionally includes some requirements about not removing addresses since this is the readiness mechanism right now. As 3rd party waypoint become a formalized thing we may want to track this readiness internally rather than relying on proper 3rd party controller status updates but we can also think about formalizing the requirements and non-conformant waypoint controllers are the vendor/3rd party's problem.

gateway.Status.Addresses should only be populated once the Waypoint's deployment has at least 1 ready pod, it should never be removed after going ready

We also at present require that this field be populated by something which can be parsed into an IPv4 or IPv6 address. This is a subset of the K8s Gateway API spec's valid addresses so it may be something we want to consider relaxing as we iterate on 3rd party interoperations.

[ilrudie]

Re: able to detect waypoint, 2 options come to mind

1. gateway class name in a specific format, waypoint- prefix for instance
2. presence of a label on the gateways.gateway... resource, ambient.istio.io/waypoint: "true" or something along these lines

[ilrudie]

Implementable items

• Discovery of gateways which are being used as waypoints
• Status on gateways (possibly need to begin looking for "programmed: instead of just addresses)
• Status on policy resources

[linsun]

Opened an API PR for discussion: istio/api#3005

[linsun]

Potential idea - create some conformance tests.

Or check small tasks in the community.

[ilrudie]

istio/api#3014 opened to track WI for policy status enhancements

[linsun]

Working on a PR to add the status and condition for APIs.

[linsun]

Need to sync with Louis and agree on scope for L4 beta.

[ilrudie]

Related : https://docs.google.com/document/d/1EDnk9cyji8GtpK7tF6czqiStd7UTujoIhhBUi1XYAQM/edit#heading=h.stll71jzvhaf

Nailing this down would help with the logic for what goes into status.

[ilrudie]

not stale

[howardjohn]

I think this is done and tested in TestSimpleHTTPSandwich