-
Notifications
You must be signed in to change notification settings - Fork 89
Open
Description
My Linstor Satellite and Linstor Controller are installed on the same device, the version numbers are as follows:
root@nas:~# /usr/share/linstor-server/bin/Controller --version
LINSTOR, Module Controller
Version: 1.31.1 (24dff0468692884b946e0204438105b537f551d6)
Build time: 2025-05-21T07:18:26+00:00 Log v2
Java Version: 21
Java VM: Ubuntu, Version 21.0.7+6-Ubuntu-0ubuntu124.04
Operating system: Linux, Version 6.8.0-60-generic
Environment: amd64, 32 processors, 8192 MiB memory reserved for allocations
System components initialization in progress
LINSTOR Controller 1.31.1
root@nas:~# /usr/share/linstor-server/bin/Satellite --version
LINSTOR, Module Satellite
Version: 1.31.1 (24dff0468692884b946e0204438105b537f551d6)
Build time: 2025-05-21T07:18:26+00:00 Log v2
Java Version: 21
Java VM: Ubuntu, Version 21.0.7+6-Ubuntu-0ubuntu124.04
Operating system: Linux, Version 6.8.0-60-generic
Environment: amd64, 32 processors, 2048 MiB memory reserved for allocations
System components initialization in progress
LINSTOR Satellite 1.31.1
My linstor satellite now won't start properly
I can see the following log with journalctl -u linstor-satellite.service:
ay 29 20:49:45 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:45.423 [MainWorkerPool-2] ERROR LINSTOR/Satellite/000004 SYSTEM - While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length. [Report number 683857E0-F3666-000000]
May 29 20:49:51 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:51.963 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DRBD9_KERNEL: supported (9.2.13)
May 29 20:49:51 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:51.963 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DRBD9_UTILS: supported (9.31.0)
May 29 20:49:51 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:51.967 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DRBD_PROXY: NOT supported:
May 29 20:49:51 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:51.974 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for CRYPT_SETUP: supported (2.7.0)
May 29 20:49:51 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:51.982 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for LVM: supported (2.3.16)
May 29 20:49:51 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:51.987 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for LVM_THIN: supported (0.9.0)
May 29 20:49:51 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:51.990 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for THIN_SEND_RECV: NOT supported:
May 29 20:49:51 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:51.994 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for ZFS_KMOD: supported (2.3.2)
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.001 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for ZFS_UTILS: supported (2.3.2-1)
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.003 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for NVME: NOT supported:
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.006 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for SPDK: NOT supported:
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.006 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for EBS_TARGET: NOT supported:
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.009 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for EBS_INIT: supported ()
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.009 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DM_WRITECACHE: supported ()
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.009 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DM_CACHE: supported ()
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.013 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for BCACHE_TOOLS: supported ()
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.018 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for LOSETUP: supported (2.39)
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.022 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for ZSTD: supported (1.5)
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.024 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for SOCAT: NOT supported:
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.028 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for COREUTILS_LINUX: supported (9.4)
May 29 20:49:52 nas.hxhome.local systemd[1]: linstor-satellite.service: Got notification message from PID 4240, but reception only permitted for main PID 3534
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.033 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for UDEVADM: supported (255)
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.035 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for LSSCSI: NOT supported:
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.041 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for SAS_PHY: NOT supported:
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.046 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for SAS_DEVICE: NOT supported:
May 29 20:49:52 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:52.047 [MainWorkerPool-3] INFO LINSTOR/Satellite/000001 SYSTEM - Controller connected and authenticated (172.25.1.1:32984/2)
May 29 20:49:53 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:53.035 [MainWorkerPool-4] ERROR LINSTOR/Satellite/000002 SYSTEM - While parsing a protocol message, the input ended unexpectedly in the middle of a field. This could mean either that the input has been truncated or that an embedded message misreported its own length. [Report number 683857E0-F3666-000001]
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.955 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DRBD9_KERNEL: supported (9.2.13)
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.956 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DRBD9_UTILS: supported (9.31.0)
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.958 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DRBD_PROXY: NOT supported:
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.965 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for CRYPT_SETUP: supported (2.7.0)
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.971 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for LVM: supported (2.3.16)
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.976 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for LVM_THIN: supported (0.9.0)
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.978 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for THIN_SEND_RECV: NOT supported:
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.982 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for ZFS_KMOD: supported (2.3.2)
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.989 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for ZFS_UTILS: supported (2.3.2-1)
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.991 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for NVME: NOT supported:
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.993 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for SPDK: NOT supported:
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.994 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for EBS_TARGET: NOT supported:
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.997 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for EBS_INIT: supported ()
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.998 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DM_WRITECACHE: supported ()
May 29 20:49:59 nas.hxhome.local Satellite[3534]: 2025-05-29 20:49:59.998 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for DM_CACHE: supported ()
May 29 20:50:00 nas.hxhome.local Satellite[3534]: 2025-05-29 20:50:00.001 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for BCACHE_TOOLS: supported ()
May 29 20:50:00 nas.hxhome.local Satellite[3534]: 2025-05-29 20:50:00.005 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for LOSETUP: supported (2.39)
May 29 20:50:00 nas.hxhome.local Satellite[3534]: 2025-05-29 20:50:00.009 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for ZSTD: supported (1.5)
May 29 20:50:00 nas.hxhome.local Satellite[3534]: 2025-05-29 20:50:00.011 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for SOCAT: NOT supported:
May 29 20:50:00 nas.hxhome.local Satellite[3534]: 2025-05-29 20:50:00.014 [MainWorkerPool-5] INFO LINSTOR/Satellite/000001 SYSTEM - Checking support for COREUTILS_LINUX: supported (9.4)
And the following log with journalctl -u linstor-controller.service:
May 29 20:51:21 nas.hxhome.local Controller[3063]: 2025-05-29 20:51:21.939 [TaskScheduleService] INFO LINSTOR/Controller/13d45e SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:51:31 nas.hxhome.local Controller[3063]: 2025-05-29 20:51:31.940 [TaskScheduleService] INFO LINSTOR/Controller/1f2ef1 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:51:41 nas.hxhome.local Controller[3063]: 2025-05-29 20:51:41.939 [TaskScheduleService] INFO LINSTOR/Controller/0895cb SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:51:51 nas.hxhome.local Controller[3063]: 2025-05-29 20:51:51.939 [TaskScheduleService] INFO LINSTOR/Controller/1fdb84 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:01 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:01.940 [TaskScheduleService] INFO LINSTOR/Controller/d494d5 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:11 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:11.940 [TaskScheduleService] INFO LINSTOR/Controller/30e4b3 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:21 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:21.940 [TaskScheduleService] INFO LINSTOR/Controller/709025 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:31 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:31.939 [TaskScheduleService] INFO LINSTOR/Controller/37ca77 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:41 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:41.944 [TaskScheduleService] INFO LINSTOR/Controller/4c70f7 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:51 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:51.940 [TaskScheduleService] INFO LINSTOR/Controller/8f74ef SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:01 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:01.939 [TaskScheduleService] INFO LINSTOR/Controller/3c91df SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:11 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:11.939 [TaskScheduleService] INFO LINSTOR/Controller/6fcf90 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:21 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:21.940 [TaskScheduleService] INFO LINSTOR/Controller/1cc65d SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:31 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:31.940 [TaskScheduleService] INFO LINSTOR/Controller/f8c5e5 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:41 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:41.940 [TaskScheduleService] INFO LINSTOR/Controller/7fb6c5 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:51 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:51.939 [TaskScheduleService] INFO LINSTOR/Controller/f88055 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:54:01 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:01.940 [TaskScheduleService] INFO LINSTOR/Controller/147bf5 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:54:11 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:11.940 [TaskScheduleService] INFO LINSTOR/Controller/8aa726 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:54:11 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:11.942 [SslConnector] INFO LINSTOR/Controller/368b53 SYSTEM - Sending authentication to satellite 'nas.hxhome.local'
May 29 20:54:12 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:12.227 [MainWorkerPool-6] INFO LINSTOR/Controller/000001 SYSTEM - Changing connection state of node 'nas.hxhome.local' from OFFLINE -> CONNECTED
May 29 20:54:12 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:12.229 [MainWorkerPool-6] INFO LINSTOR/Controller/000001 SYSTEM - Satellite 'nas.hxhome.local' authenticated
May 29 20:54:12 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:12.231 [MainWorkerPool-6] INFO LINSTOR/Controller/000001 SYSTEM - Sending full sync to Node: 'nas.hxhome.local'.
May 29 20:54:13 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:13.390 [MainWorkerPool-9] ERROR reactor.core.publisher.Operators/000002 Operator called default onErrorDropped
May 29 20:54:13 nas.hxhome.local Controller[3063]: reactor.core.Exceptions$ErrorCallbackNotImplemented: com.linbit.linstor.core.apicallhandler.response.ApiRcException: (nas.hxhome.local) Controller couldn't parse message.
May 29 20:54:13 nas.hxhome.local Controller[3063]: Caused by: com.linbit.linstor.core.apicallhandler.response.ApiRcException: (nas.hxhome.local) Controller couldn't parse message.
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleAnswer(CommonMessageProcessor.java:344)
May 29 20:54:13 nas.hxhome.local Controller[3063]: Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
May 29 20:54:13 nas.hxhome.local Controller[3063]: Error has been observed at the following site(s):
May 29 20:54:13 nas.hxhome.local Controller[3063]: *__checkpoint ⇢ Send full sync
May 29 20:54:13 nas.hxhome.local Controller[3063]: *__checkpoint ⇢ authResponse
May 29 20:54:13 nas.hxhome.local Controller[3063]: Original Stack Trace:
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleAnswer(CommonMessageProcessor.java:344)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleDataMessage(CommonMessageProcessor.java:297)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.doProcessInOrderMessage(CommonMessageProcessor.java:245)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.lambda$doProcessMessage$4(CommonMessageProcessor.java:230)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:46)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.Flux.subscribe(Flux.java:8773)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:427)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManyUnicast.drainFused(SinkManyUnicast.java:319)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManyUnicast.drain(SinkManyUnicast.java:362)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManyUnicast.tryEmitNext(SinkManyUnicast.java:237)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:100)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.netcom.TcpConnectorPeer.processInOrder(TcpConnectorPeer.java:444)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.doProcessMessage(CommonMessageProcessor.java:228)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.lambda$processMessage$2(CommonMessageProcessor.java:165)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:185)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:440)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:527)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.lang.Thread.run(Thread.java:1583)
May 29 20:54:19 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:19.940 [TaskScheduleService] INFO LINSTOR/Controller/36b38a SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:54:19 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:19.944 [SslConnector] INFO LINSTOR/Controller/dcfc3f SYSTEM - Sending authentication to satellite 'nas.hxhome.local'
May 29 20:54:20 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:20.055 [MainWorkerPool-12] INFO LINSTOR/Controller/000001 SYSTEM - Changing connection state of node 'nas.hxhome.local' from OFFLINE -> CONNECTED
May 29 20:54:20 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:20.056 [MainWorkerPool-12] INFO LINSTOR/Controller/000001 SYSTEM - Satellite 'nas.hxhome.local' authenticated
May 29 20:54:20 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:20.057 [MainWorkerPool-12] INFO LINSTOR/Controller/000001 SYSTEM - Sending full sync to Node: 'nas.hxhome.local'.
May 29 20:54:21 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:21.003 [MainWorkerPool-15] ERROR reactor.core.publisher.Operators/000002 Operator called default onErrorDropped
May 29 20:54:21 nas.hxhome.local Controller[3063]: reactor.core.Exceptions$ErrorCallbackNotImplemented: com.linbit.linstor.core.apicallhandler.response.ApiRcException: (nas.hxhome.local) Controller couldn't parse message.
May 29 20:54:21 nas.hxhome.local Controller[3063]: Caused by: com.linbit.linstor.core.apicallhandler.response.ApiRcException: (nas.hxhome.local) Controller couldn't parse message.
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleAnswer(CommonMessageProcessor.java:344)
May 29 20:54:21 nas.hxhome.local Controller[3063]: Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
May 29 20:54:21 nas.hxhome.local Controller[3063]: Error has been observed at the following site(s):
May 29 20:54:21 nas.hxhome.local Controller[3063]: *__checkpoint ⇢ Send full sync
May 29 20:54:21 nas.hxhome.local Controller[3063]: *__checkpoint ⇢ authResponse
May 29 20:54:21 nas.hxhome.local Controller[3063]: Original Stack Trace:
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleAnswer(CommonMessageProcessor.java:344)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleDataMessage(CommonMessageProcessor.java:297)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.doProcessInOrderMessage(CommonMessageProcessor.java:245)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.lambda$doProcessMessage$4(CommonMessageProcessor.java:230)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:46)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.Flux.subscribe(Flux.java:8773)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:427)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManyUnicast.drainFused(SinkManyUnicast.java:319)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManyUnicast.drain(SinkManyUnicast.java:362)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManyUnicast.tryEmitNext(SinkManyUnicast.java:237)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:100)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.netcom.TcpConnectorPeer.processInOrder(TcpConnectorPeer.java:444)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.doProcessMessage(CommonMessageProcessor.java:228)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.lambda$processMessage$2(CommonMessageProcessor.java:165)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:185)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:440)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:527)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
May 29 20:51:21 nas.hxhome.local Controller[3063]: 2025-05-29 20:51:21.939 [TaskScheduleService] INFO LINSTOR/Controller/13d45e SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:51:31 nas.hxhome.local Controller[3063]: 2025-05-29 20:51:31.940 [TaskScheduleService] INFO LINSTOR/Controller/1f2ef1 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:51:41 nas.hxhome.local Controller[3063]: 2025-05-29 20:51:41.939 [TaskScheduleService] INFO LINSTOR/Controller/0895cb SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:51:51 nas.hxhome.local Controller[3063]: 2025-05-29 20:51:51.939 [TaskScheduleService] INFO LINSTOR/Controller/1fdb84 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:01 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:01.940 [TaskScheduleService] INFO LINSTOR/Controller/d494d5 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:11 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:11.940 [TaskScheduleService] INFO LINSTOR/Controller/30e4b3 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:21 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:21.940 [TaskScheduleService] INFO LINSTOR/Controller/709025 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:31 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:31.939 [TaskScheduleService] INFO LINSTOR/Controller/37ca77 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:41 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:41.944 [TaskScheduleService] INFO LINSTOR/Controller/4c70f7 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:52:51 nas.hxhome.local Controller[3063]: 2025-05-29 20:52:51.940 [TaskScheduleService] INFO LINSTOR/Controller/8f74ef SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:01 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:01.939 [TaskScheduleService] INFO LINSTOR/Controller/3c91df SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:11 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:11.939 [TaskScheduleService] INFO LINSTOR/Controller/6fcf90 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:21 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:21.940 [TaskScheduleService] INFO LINSTOR/Controller/1cc65d SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:31 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:31.940 [TaskScheduleService] INFO LINSTOR/Controller/f8c5e5 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:41 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:41.940 [TaskScheduleService] INFO LINSTOR/Controller/7fb6c5 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:53:51 nas.hxhome.local Controller[3063]: 2025-05-29 20:53:51.939 [TaskScheduleService] INFO LINSTOR/Controller/f88055 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:54:01 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:01.940 [TaskScheduleService] INFO LINSTOR/Controller/147bf5 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:54:11 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:11.940 [TaskScheduleService] INFO LINSTOR/Controller/8aa726 SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:54:11 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:11.942 [SslConnector] INFO LINSTOR/Controller/368b53 SYSTEM - Sending authentication to satellite 'nas.hxhome.local'
May 29 20:54:12 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:12.227 [MainWorkerPool-6] INFO LINSTOR/Controller/000001 SYSTEM - Changing connection state of node 'nas.hxhome.local' from OFFLINE -> CONNECTED
May 29 20:54:12 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:12.229 [MainWorkerPool-6] INFO LINSTOR/Controller/000001 SYSTEM - Satellite 'nas.hxhome.local' authenticated
May 29 20:54:12 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:12.231 [MainWorkerPool-6] INFO LINSTOR/Controller/000001 SYSTEM - Sending full sync to Node: 'nas.hxhome.local'.
May 29 20:54:13 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:13.390 [MainWorkerPool-9] ERROR reactor.core.publisher.Operators/000002 Operator called default onErrorDropped
May 29 20:54:13 nas.hxhome.local Controller[3063]: reactor.core.Exceptions$ErrorCallbackNotImplemented: com.linbit.linstor.core.apicallhandler.response.ApiRcException: (nas.hxhome.local) Controller couldn't parse message.
May 29 20:54:13 nas.hxhome.local Controller[3063]: Caused by: com.linbit.linstor.core.apicallhandler.response.ApiRcException: (nas.hxhome.local) Controller couldn't parse message.
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleAnswer(CommonMessageProcessor.java:344)
May 29 20:54:13 nas.hxhome.local Controller[3063]: Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
May 29 20:54:13 nas.hxhome.local Controller[3063]: Error has been observed at the following site(s):
May 29 20:54:13 nas.hxhome.local Controller[3063]: *__checkpoint ⇢ Send full sync
May 29 20:54:13 nas.hxhome.local Controller[3063]: *__checkpoint ⇢ authResponse
May 29 20:54:13 nas.hxhome.local Controller[3063]: Original Stack Trace:
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleAnswer(CommonMessageProcessor.java:344)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleDataMessage(CommonMessageProcessor.java:297)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.doProcessInOrderMessage(CommonMessageProcessor.java:245)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.lambda$doProcessMessage$4(CommonMessageProcessor.java:230)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:46)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.Flux.subscribe(Flux.java:8773)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:427)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drain(FluxFlattenIterable.java:724)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.onNext(FluxFlattenIterable.java:256)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManyUnicast.drainFused(SinkManyUnicast.java:319)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManyUnicast.drain(SinkManyUnicast.java:362)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManyUnicast.tryEmitNext(SinkManyUnicast.java:237)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:100)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.netcom.TcpConnectorPeer.processInOrder(TcpConnectorPeer.java:444)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.doProcessMessage(CommonMessageProcessor.java:228)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.lambda$processMessage$2(CommonMessageProcessor.java:165)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:185)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:440)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:527)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
May 29 20:54:13 nas.hxhome.local Controller[3063]: at java.base/java.lang.Thread.run(Thread.java:1583)
May 29 20:54:19 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:19.940 [TaskScheduleService] INFO LINSTOR/Controller/36b38a SYSTEM - Establishing connection to node 'nas.hxhome.local' via /172.25.1.1:3366 ...
May 29 20:54:19 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:19.944 [SslConnector] INFO LINSTOR/Controller/dcfc3f SYSTEM - Sending authentication to satellite 'nas.hxhome.local'
May 29 20:54:20 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:20.055 [MainWorkerPool-12] INFO LINSTOR/Controller/000001 SYSTEM - Changing connection state of node 'nas.hxhome.local' from OFFLINE -> CONNECTED
May 29 20:54:20 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:20.056 [MainWorkerPool-12] INFO LINSTOR/Controller/000001 SYSTEM - Satellite 'nas.hxhome.local' authenticated
May 29 20:54:20 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:20.057 [MainWorkerPool-12] INFO LINSTOR/Controller/000001 SYSTEM - Sending full sync to Node: 'nas.hxhome.local'.
May 29 20:54:21 nas.hxhome.local Controller[3063]: 2025-05-29 20:54:21.003 [MainWorkerPool-15] ERROR reactor.core.publisher.Operators/000002 Operator called default onErrorDropped
May 29 20:54:21 nas.hxhome.local Controller[3063]: reactor.core.Exceptions$ErrorCallbackNotImplemented: com.linbit.linstor.core.apicallhandler.response.ApiRcException: (nas.hxhome.local) Controller couldn't parse message.
May 29 20:54:21 nas.hxhome.local Controller[3063]: Caused by: com.linbit.linstor.core.apicallhandler.response.ApiRcException: (nas.hxhome.local) Controller couldn't parse message.
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleAnswer(CommonMessageProcessor.java:344)
May 29 20:54:21 nas.hxhome.local Controller[3063]: Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
May 29 20:54:21 nas.hxhome.local Controller[3063]: Error has been observed at the following site(s):
May 29 20:54:21 nas.hxhome.local Controller[3063]: *__checkpoint ⇢ Send full sync
May 29 20:54:21 nas.hxhome.local Controller[3063]: *__checkpoint ⇢ authResponse
May 29 20:54:21 nas.hxhome.local Controller[3063]: Original Stack Trace:
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleAnswer(CommonMessageProcessor.java:344)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.handleDataMessage(CommonMessageProcessor.java:297)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.doProcessInOrderMessage(CommonMessageProcessor.java:245)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at com.linbit.linstor.proto.CommonMessageProcessor.lambda$doProcessMessage$4(CommonMessageProcessor.java:230)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:46)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.Flux.subscribe(Flux.java:8773)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:427)
May 29 20:54:21 nas.hxhome.local Controller[3063]: at reactor.core.publisher.FluxFlattenIterable$FlattenIterableSubscriber.drainAsync(FluxFlattenIterable.java:453)
And linstor err list showed nothing.
╭──────────────────────────────────╮
┊ Id ┊ Datetime ┊ Node ┊ Exception ┊
╞══════════════════════════════════╡
╰──────────────────────────────────╯
Metadata
Metadata
Assignees
Labels
No labels