Network clustering #1053

edlerd · 2025-01-11T12:52:00Z

Done

adjust network list to show cluster member specific networks
split network list entries and detail pages to be one entry/page per cluster member for physical interfaces
add pattern for cluster specific inputs for a physical managed networks parent. This is to be reused for other cluster specific inputs like in server settings or for storage pool configuration

QA

Run the LXD-UI:
- On the demo server via the link posted by @webteam-app below. This is only available for PRs created by collaborators of the repo. Ask @mas-who or @edlerd for access.
- With a local copy of this branch, build and run as described in the docs.
Perform the following QA steps:
- Browse the network list in an unclustered backend, check the filters
- Browse the network list in a clustered backend, use the filters and clicking on the cluster member chips applies filtering
- Create and edit a physical network in an unclustered backend
- Create and edit a physical network in a clustered backend, ensure the connections diagram is updating and the chips in it linking correctly. Ensure the chips in the "parent" selector link correctly. Try changing and breaking the parent selector in the clustered backend when creating or editing a physical network.
- Browse a physical unmanaged network in a clustered and unclustered backend

webteam-app · 2025-01-11T12:52:03Z

Demo

Jenkins

demos.haus

MasWho

Some interim code comments. The network list with filtering looks pretty good (cluster and non-cluster conditions) from QA perspective. Will go through the other QA items as well.

src/pages/networks/NetworkSearchFilter.tsx

src/api/networks.tsx

MasWho · 2025-01-16T11:09:20Z

src/pages/networks/NetworkList.tsx

+    isLoading: isClusterNetworksLoading,
+  } = useQuery({
+    queryKey: [queryKeys.networks, "default", queryKeys.cluster],
+    queryFn: () => fetchClusterMemberNetworks("default", clusterMembers),


I think there is a specific scenario we need to take into consideration here. It is possible to create OVN networks for specific restricted projects (even with ip ranges that overlaps other exisitng ovn networks), not sure if fetching networks from the default project will surface those?

See this video ref, watch from around 17:00

Yes, good point. We should use the current project here and not "default". We want to fetch all real interfaces for each cluster member. Those are either available on the current project (if has features.networks set to false) or not available. In the later case, we should not show them.

It is a pretty niche case though, afaik this only applies to OVN networks due to their virtual nature

The uplink network for OVN needs to be in the default project. In the uplink selector we do fetch networks from the default project regardless of which project the user is currently in. I think we should not show those in the network list, though. So I think we are doing the right thing even for all cases.

src/pages/networks/NetworkList.tsx

mas-who · 2025-01-16T12:26:29Z

Some QA observations below:

When clicking on the cluster member resource links in the network list table, all existing search params gets cleared. Why not add the the member search to the existing search params? Is it because it would be a bit weird in the case that the member search is already present, and it would look like nothing is happening?
Noticed that on smaller screen sizes the action buttons and the search filter are positioned weirdly. Maybe we can add the actions in a contextual menu similar to how we do it in the instance detail page. Then there will be more space for the search and filter as well. wdyt?
When creating a clustered physical network with the following parent configs

On submission I get the following error:

Trying to create the network with the same name fails at this point as it already exist in LXD.

I think we should disable the submit button if parents are not selected for all members?
NOTE: the edit case seems fine, the backend seems to block the operation and the configs does not get persisted.
After creating a physical network with some parent interface across cluster members, trying to create another network using the same parent interfaces results in a creation error indicating they are in use. However, the network still gets created in an "Errored" state.
Clicking on the member resource link on a network detail page does not result in search params being set after redirect to the network list page. Is that intended?
After creating an OVN network with a physical uplink, it is possible to delete the physical uplink. However, on the network topology for the OVN network, the deleted uplink still shows up some how.
Observations for when a cluster member is down:
a. Trying to create a physical network results in an error message "peer node 10.94.160.130:8443 is down". However, the network gets created and shows up in the network list with the "Cluster-wide" member category.
b. It is not possible to edit a physical network with the same error message as above.
c. It is not possible to delete the physical network with the same error message as above.
d. trying to visit the physical network detail page for the vm that is down results in a 500 error "Missing event connection with target cluster member". Currently the detail page loads for a long time then shows a blank page, this should probably be handled by displaying the error message.

edlerd · 2025-01-16T17:38:57Z

Resolved issues 1-3 and 5.

After creating a physical network with some parent interface across cluster members, trying to create another network using the same parent interfaces results in a creation error indicating they are in use. However, the network still gets created in an "Errored" state.

This sounds like the expected behaviour. Wdyt?

After creating an OVN network with a physical uplink, it is possible to delete the physical uplink. However, on the network topology for the OVN network, the deleted uplink still shows up some how.

This shows up, because the uplink config of the OVN network doesn't change when deleting the uplink network. It might be that LXD should refuse to delete the uplink, but that should be in the API then, not in the UI itself.

Observations for when a cluster member is down:
a. Trying to create a physical network results in an error message "peer node 10.94.160.130:8443 is down". However, the network gets created and shows up in the network list with the "Cluster-wide" member category.
b. It is not possible to edit a physical network with the same error message as above.
c. It is not possible to delete the physical network with the same error message as above.
d. trying to visit the physical network detail page for the vm that is down results in a 500 error "Missing event connection with target cluster member". Currently the detail page loads for a long time then shows a blank page, this should probably be handled by displaying the error message.

This needs future work.

mas-who · 2025-01-17T07:29:33Z

Resolved issues 1-3 and 5.

After creating a physical network with some parent interface across cluster members, trying to create another network using the same parent interfaces results in a creation error indicating they are in use. However, the network still gets created in an "Errored" state.

This sounds like the expected behaviour. Wdyt?

My main concern is that the network gets created even when it's not valid. Would it be possible to check upon parent selection if it is already used by another network and reflect that as an error message on the creation / edit form? If that's too complex I think the current behaviour is also fine.

After creating an OVN network with a physical uplink, it is possible to delete the physical uplink. However, on the network topology for the OVN network, the deleted uplink still shows up some how.

This shows up, because the uplink config of the OVN network doesn't change when deleting the uplink network. It might be that LXD should refuse to delete the uplink, but that should be in the API then, not in the UI itself.

Noted, perhaps we should raise this with the core team?

Observations for when a cluster member is down:
a. Trying to create a physical network results in an error message "peer node 10.94.160.130:8443 is down". However, the network gets created and shows up in the network list with the "Cluster-wide" member category.
b. It is not possible to edit a physical network with the same error message as above.
c. It is not possible to delete the physical network with the same error message as above.
d. trying to visit the physical network detail page for the vm that is down results in a 500 error "Missing event connection with target cluster member". Currently the detail page loads for a long time then shows a blank page, this should probably be handled by displaying the error message.

This needs future work.
Noted 👍

edlerd · 2025-01-17T09:38:14Z

Improved the error handling, that should resolve 7.

After creating a physical network with some parent interface across cluster members, trying to create another network using the same parent interfaces results in a creation error indicating they are in use. However, the network still gets created in an "Errored" state.

This sounds like the expected behaviour. Wdyt?

My main concern is that the network gets created even when it's not valid. Would it be possible to check upon parent selection if it is already used by another network and reflect that as an error message on the creation / edit form? If that's too complex I think the current behaviour is also fine.

I am not 100% sure we can never reuse an interface as parent. Reported this edge case to lxd: canonical/lxd#14810

After creating an OVN network with a physical uplink, it is possible to delete the physical uplink. However, on the network topology for the OVN network, the deleted uplink still shows up some how.

This shows up, because the uplink config of the OVN network doesn't change when deleting the uplink network. It might be that LXD should refuse to delete the uplink, but that should be in the API then, not in the UI itself.

Noted, perhaps we should raise this with the core team?

I tried to reproduce it, but was getting an error. I couldn't delete the uplink in use by the OVN network. So this might not be an issue after all.

…stered backend Signed-off-by: David Edler <[email protected]>

mas-who

This is looking really good! Couldn't actually find that many issues QA wise, left some code comments

mas-who · 2025-01-17T12:59:01Z

src/api/networks.tsx

+export const fetchNetworksFromClusterMembers = (
+  project: string,
+  clusterMembers: LxdClusterMember[],
+): Promise<LXDNetworkOnClusterMember[]> => {


love how simple things look now with the latest type update! 👍

mas-who · 2025-01-17T13:23:04Z

src/api/networks.tsx

+  project: string,
+  parentsPerClusterMember?: ClusterSpecificValues,
+): Promise<void> => {
+  if (!parentsPerClusterMember) {


not quite sure why we need this here, for this scenario, can't we just use updateNetwork directly where ever we are calling updateClusterNetwork? The handling of undefined parentsPerClusterMember perhaps should be in the component instead?

mas-who · 2025-01-17T13:39:35Z

src/pages/networks/NetworkList.tsx

-  if (error) {
-    notify.failure("Loading networks failed", error);
-  }
+  useEffect(() => {


why does this need to be in an useEffect?

mas-who · 2025-01-17T14:00:17Z

src/util/intersection.tsx

+export const intersection = (lists: string[][]): string[] => {
+  const result = [];
+
+  for (let i = 0; i < lists.length; i++) {


a potential way to improve the efficiency and readability of this could be to hold an object with keys being unique strings from lists, and values indicate the count of occurrence for each unique string. As we iterate the lists, we convert each list into a set, then increment the count for each unique string. At the end, all unique string items with a count that equates to the length of the input lists would have appeared in all.

mas-who · 2025-01-17T14:02:56Z

src/util/networkForm.tsx

+  networkOnMembers?: LXDNetworkOnClusterMember[],
+): NetworkFormValues => {
+  const parentPerClusterMember: ClusterSpecificValues = {};
+  networkOnMembers?.map(


Suggested change

networkOnMembers?.map(

networkOnMembers?.forEach(

mas-who · 2025-01-17T14:23:23Z

src/components/ClusterSpecificSelect.tsx

+
+  const setValueForAllMembers = (value: string) => {
+    const update: ClusterSpecificValues = {};
+    options.map((item) => (update[item.memberName] = value));


Suggested change

options.map((item) => (update[item.memberName] = value));

options.forEach((item) => (update[item.memberName] = value));

mas-who · 2025-01-17T14:26:52Z

src/components/ClusterSpecificSelect.tsx

+          label="Same for all cluster members"
+          checked={!isSpecific}
+          onChange={() => {
+            if (isSpecific) {


shouldn't we place the setValueForAllMembers logic inside a useEffect?

mas-who · 2025-01-17T14:31:43Z

src/components/ClusterSpecificSelect.tsx

+                  <ResourceLink
+                    type="cluster-member"
+                    value={item.memberName}
+                    to={`/ui/project/${project}/networks?member=${item.memberName}`}


do we always want to redirect to the networks page with member filter? This feels a little strange, not sure what's the best way to approach it though, maybe expose this as a function prop?

mas-who · 2025-01-17T14:40:47Z

src/pages/networks/forms/NetworkParentSelector.tsx

+            onChange={(value) => {
+              void formik.setFieldValue("parentPerClusterMember", value);
+            }}


Suggested change

onChange={(value) => {

void formik.setFieldValue("parentPerClusterMember", value);

}}

onChange={(value) => void formik.setFieldValue("parentPerClusterMember", value)}

mas-who · 2025-01-17T14:56:10Z

src/pages/networks/NetworkClusterMemberChip.tsx

+  network: LXDNetworkOnClusterMember;
+}
+
+const NetworkClusterMemberChip: FC<Props> = ({ network }) => {


nice extraction!

mas-who · 2025-01-17T14:57:59Z

QA comments:

On medium screen size, the network list table gets cut off, should we make the table width adjust with viewport width until they turn into cards?
When creating a physical network, should we pre-select a network interface when the "Same for all cluster members" option is checked?

edlerd force-pushed the network-clustering branch 6 times, most recently from 54c1afa to e2e4075 Compare January 15, 2025 15:44

edlerd changed the title ~~Network clustering (wip)~~ Network clustering Jan 15, 2025

edlerd marked this pull request as ready for review January 15, 2025 15:46

edlerd force-pushed the network-clustering branch 2 times, most recently from add1260 to 2379d77 Compare January 15, 2025 16:49

edlerd added Review: Design needed Review: Code needed Review: QA needed labels Jan 15, 2025

edlerd assigned piperdeck Jan 15, 2025

MasWho reviewed Jan 16, 2025

View reviewed changes

edlerd force-pushed the network-clustering branch 3 times, most recently from 8f766bf to b0927a5 Compare January 16, 2025 17:22

edlerd force-pushed the network-clustering branch 2 times, most recently from 016a2d5 to 5b8158b Compare January 16, 2025 17:48

edlerd force-pushed the network-clustering branch from 5b8158b to 9c42ee6 Compare January 17, 2025 08:45

edlerd force-pushed the network-clustering branch 2 times, most recently from bf5ade3 to fb1b4e9 Compare January 17, 2025 11:55

fix(network) ensure network configuration is fully supported in a clu…

b52956e

…stered backend Signed-off-by: David Edler <[email protected]>

edlerd force-pushed the network-clustering branch from fb1b4e9 to b52956e Compare January 17, 2025 12:22

mas-who reviewed Jan 17, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Network clustering #1053

Network clustering #1053

edlerd commented Jan 11, 2025 •

edited

Loading

webteam-app commented Jan 11, 2025

MasWho left a comment

MasWho Jan 16, 2025

edlerd Jan 16, 2025

mas-who Jan 16, 2025

edlerd Jan 16, 2025

mas-who commented Jan 16, 2025

edlerd commented Jan 16, 2025

mas-who commented Jan 17, 2025

edlerd commented Jan 17, 2025

mas-who left a comment

mas-who Jan 17, 2025

mas-who Jan 17, 2025

mas-who Jan 17, 2025

mas-who Jan 17, 2025

mas-who Jan 17, 2025

mas-who Jan 17, 2025

mas-who Jan 17, 2025

mas-who Jan 17, 2025

mas-who Jan 17, 2025

mas-who Jan 17, 2025

mas-who commented Jan 17, 2025

	options.map((item) => (update[item.memberName] = value));
	options.forEach((item) => (update[item.memberName] = value));

Network clustering #1053

Are you sure you want to change the base?

Network clustering #1053

Conversation

edlerd commented Jan 11, 2025 • edited Loading

Done

QA

webteam-app commented Jan 11, 2025

MasWho left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mas-who commented Jan 16, 2025

edlerd commented Jan 16, 2025

mas-who commented Jan 17, 2025

edlerd commented Jan 17, 2025

mas-who left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mas-who commented Jan 17, 2025

edlerd commented Jan 11, 2025 •

edited

Loading