chore: Upgrade test runner plugins #15102

Artur- · 2022-11-11T11:00:55Z

This is to potentially resolve VM crashes when running tests with Java 19

github-actions · 2022-11-11T11:21:19Z

Unit Test Results

  958 files ±0   958 suites ±0 55m 58s ⏱️ - 2m 18s
6 113 tests ±0 6 059 ✔️ ±0 54 💤 ±0 0 ❌ ±0
6 377 runs +9 6 315 ✔️ +9 62 💤 ±0 0 ❌ ±0

Results for commit f83832a. ± Comparison against base commit f5bb060.

♻️ This comment has been updated with latest results.

pom.xml

MarcinVaadin · 2022-11-21T13:46:05Z

Downgraded surefire to check latest changes specially one bothers me:

[[SUREFIRE-2058] Corrupted STDOUT by directly writing to native stream in forked JVM 1 with UTF-8]]([https://issues.apache.org/jira/browse/SUREFIRE-2058]](https://issues.apache.org/jira/browse/SUREFIRE-2058%5D) Corrupted STDOUT by directly writing to native stream in forked JVM 1 with UTF-8) - console logging (apache/maven-surefire#518) @zoltanmeze

pom.xml

zoltanmeze · 2022-11-22T10:16:33Z

Downgraded surefire to check latest changes specially one bothers me:

[[SUREFIRE-2058] Corrupted STDOUT by directly writing to native stream in forked JVM 1 with UTF-8]]([https://issues.apache.org/jira/browse/SUREFIRE-2058]](https://issues.apache.org/jira/browse/SUREFIRE-2058%5D) Corrupted STDOUT by directly writing to native stream in forked JVM 1 with UTF-8) - console logging (apache/maven-surefire#518) @zoltanmeze

Try to grab dump files generated by surefire/failsafe to see if there is more information
https://maven.apache.org/surefire/maven-surefire-plugin/faq.html#dumpfiles

pom.xml

github-actions · 2022-11-22T12:13:11Z

Test Results

  479 files -   461 479 suites - 461 47m 2s ⏱️ - 9m 35s
  759 tests - 5 190 716 ✔️ - 5 188 43 💤 - 2 0 ❌ ±0
1 003 runs - 5 209 953 ✔️ - 5 207 50 💤 - 2 0 ❌ ±0

Results for commit aad525d. ± Comparison against base commit 99f8cfe.

♻️ This comment has been updated with latest results.

Artur- · 2022-11-23T07:05:26Z

The dumpstream file contains

# Created at 2022-11-22T16:34:34.644
[SUREFIRE] std/in stream corrupted
java.io.IOException: Resource temporarily unavailable
        at java.base/java.io.FileInputStream.readBytes(Native Method)
        at java.base/java.io.FileInputStream.read(FileInputStream.java:276)
        at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:282)
        at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:343)
        at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:244)
        at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
        at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:343)
        at org.apache.maven.surefire.api.util.internal.Channels$3.readImpl(Channels.java:217)
        at org.apache.maven.surefire.api.util.internal.AbstractNoninterruptibleReadableChannel.read(AbstractNoninterruptibleReadableChannel.java:54)
        at org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:484)
        at org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:470)
        at org.apache.maven.surefire.api.stream.AbstractStreamDecoder.readMessageType(AbstractStreamDecoder.java:118)
        at org.apache.maven.surefire.booter.stream.CommandDecoder.decode(CommandDecoder.java:87)
        at org.apache.maven.surefire.booter.spi.CommandChannelDecoder.decode(CommandChannelDecoder.java:67)
        at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:345)
        at java.base/java.lang.Thread.run(Thread.java:833)

The only thing that stands out in the thread dump is that there are two instances of DevServerOutputTracker running

Files here https://github.com/vaadin/flow/suites/9449041828/artifacts/447046345

This is to potentially resolve VM crashes when running tests with Java 19

Artur- · 2022-11-25T11:59:27Z

Now the unit tests pass for some reason. Nothing has been fixed but some tests have been removed...

But the "slow tests" build fails and that can be reproduced locally as

mvn -Pslow-tests -pl flow-server clean verify

which results in a similar dumpstream file

# Created at 2022-11-25T13:48:24.443
[SUREFIRE] std/in stream corrupted
java.io.IOException: Resource temporarily unavailable
        at java.base/java.io.FileInputStream.readBytes(Native Method)
        at java.base/java.io.FileInputStream.read(FileInputStream.java:276)
        at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:282)
        at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:343)
        at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:244)
        at java.base/java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
        at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:343)
        at org.apache.maven.surefire.api.util.internal.Channels$3.readImpl(Channels.java:217)
        at org.apache.maven.surefire.api.util.internal.AbstractNoninterruptibleReadableChannel.read(AbstractNoninterruptibleReadableChannel.java:54)
        at org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:484)
        at org.apache.maven.surefire.api.stream.AbstractStreamDecoder.read(AbstractStreamDecoder.java:470)
        at org.apache.maven.surefire.api.stream.AbstractStreamDecoder.readMessageType(AbstractStreamDecoder.java:118)
        at org.apache.maven.surefire.booter.stream.CommandDecoder.decode(CommandDecoder.java:87)
        at org.apache.maven.surefire.booter.spi.CommandChannelDecoder.decode(CommandChannelDecoder.java:67)
        at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:345)
        at java.base/java.lang.Thread.run(Thread.java:833)


# Created at 2022-11-25T13:48:24.447
[SUREFIRE] std/in stream corrupted

# Created at 2022-11-25T13:50:12.398
Surefire is going to kill self fork JVM. The exit has elapsed 30 seconds after System.exit(0).

and a not very meaningful dump file

# Created at 2022-11-25T13:50:12.466
Thread dump for process (21712@...) after 30 seconds shutdown timeout:
"main" 
   java.lang.Thread.State: TIMED_WAITING
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
        at [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:717)
        at [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1074)
        at [email protected]/java.util.concurrent.Semaphore.tryAcquire(Semaphore.java:415)
        at app//org.apache.maven.surefire.booter.ForkedBooter.acquireOnePermit(ForkedBooter.java:618)
        at app//org.apache.maven.surefire.booter.ForkedBooter.acknowledgedExit(ForkedBooter.java:442)
        at app//org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:190)
        at app//org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:595)
        at app//org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:581)

"Reference Handler" 
   java.lang.Thread.State: RUNNABLE
        at [email protected]/java.lang.ref.Reference.waitForReferencePendingList(Native Method)
        at [email protected]/java.lang.ref.Reference.processPendingReferences(Reference.java:253)
        at [email protected]/java.lang.ref.Reference$ReferenceHandler.run(Reference.java:215)

"Finalizer" 
   java.lang.Thread.State: WAITING
        at [email protected]/java.lang.Object.wait(Native Method)
        at [email protected]/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
        at [email protected]/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:176)
        at [email protected]/java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:172)

"Signal Dispatcher" 
   java.lang.Thread.State: RUNNABLE

"Notification Thread" 
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" 
   java.lang.Thread.State: TIMED_WAITING
        at [email protected]/java.lang.Object.wait(Native Method)
        at [email protected]/java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:155)
        at [email protected]/jdk.internal.ref.CleanerImpl.run(CleanerImpl.java:140)
        at [email protected]/java.lang.Thread.run(Thread.java:833)
        at [email protected]/jdk.internal.misc.InnocuousThread.run(InnocuousThread.java:162)

"surefire-forkedjvm-stream-flusher" 
   java.lang.Thread.State: TIMED_WAITING
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
        at [email protected]/java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:1672)
        at [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1182)
        at [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:899)
        at [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1062)
        at [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
        at [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at [email protected]/java.lang.Thread.run(Thread.java:833)

"process reaper" 
   java.lang.Thread.State: TIMED_WAITING
        at [email protected]/jdk.internal.misc.Unsafe.park(Native Method)
        at [email protected]/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:252)
        at [email protected]/java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:401)
        at [email protected]/java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:903)
        at [email protected]/java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1061)
        at [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1122)
        at [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at [email protected]/java.lang.Thread.run(Thread.java:833)

"Attach Listener" 
   java.lang.Thread.State: RUNNABLE

"surefire-forkedjvm-last-ditch-daemon-shutdown-thread-30s" 
   java.lang.Thread.State: RUNNABLE
        at [email protected]/sun.management.ThreadImpl.getThreadInfo1(Native Method)
        at [email protected]/sun.management.ThreadImpl.getThreadInfo(ThreadImpl.java:197)
        at app//org.apache.maven.surefire.booter.ForkedBooter.generateThreadDump(ForkedBooter.java:683)
        at app//org.apache.maven.surefire.booter.ForkedBooter.access$600(ForkedBooter.java:80)
        at app//org.apache.maven.surefire.booter.ForkedBooter$7.run(ForkedBooter.java:499)
        at [email protected]/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
        at [email protected]/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at [email protected]/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
        at [email protected]/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at [email protected]/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at [email protected]/java.lang.Thread.run(Thread.java:833)

I wonder how anybody is even supposed to debug this. Isn't the stdin/stdout stream used internally by surefire an... internal... thing that we cannot affect?

Artur- · 2022-11-25T13:08:39Z

I deleted all tests but one in flow-server and now it reproduces around 30% of the time when running mvn -pl flow-server test.

The log when it reproduces looks like

[INFO] Running com.vaadin.flow.server.frontend.TaskRunPnpmInstallTest
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.Platform - Unable to determine version information. No vaadin-core-versions.json found
[main] INFO com.vaadin.flow.server.frontend.TaskRunPnpmInstallTest$1 - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.TaskRunPnpmInstallTest$1 - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[ERROR] [SUREFIRE] std/in stream corrupted

but the corruption can also happen much later on

[INFO] Running com.vaadin.flow.server.frontend.TaskRunPnpmInstallTest
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.Platform - Unable to determine version information. No vaadin-core-versions.json found
[main] INFO com.vaadin.flow.server.frontend.TaskRunPnpmInstallTest$1 - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.TaskRunPnpmInstallTest$1 - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.TaskRunPnpmInstallTest$1 - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Skipping `pnpm install` because the frontend packages are already installed in the folder '/var/folders/3r/3_0g1bhn44j1vvvpfksrz1z40000gn/T/junit14583221484416158835/junit10708200394253933036/node_modules' and the hash in the file '/var/folders/3r/3_0g1bhn44j1vvvpfksrz1z40000gn/T/junit14583221484416158835/junit10708200394253933036/node_modules/.vaadin/vaadin.json' is the same as in 'package.json'
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Frontend dependencies resolved successfully.
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - Running `pnpm install` to resolve and optionally download frontend dependencies. This may take a moment, please stand by...
[main] INFO com.vaadin.flow.server.frontend.NodeUpdater - using '/Users/artur/.nvm/versions/node/v18.12.1/bin/npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install' for frontend package installation
[ERROR] [SUREFIRE] std/in stream corrupted

Artur- · 2022-11-25T13:11:27Z

When it works, it just runs through all tests

[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.366 s - in com.vaadin.flow.server.frontend.TaskRunPnpmInstallTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 16, Failures: 0, Errors: 0, Skipped: 0

vaadin-bot · 2022-11-25T13:13:01Z

TC Format Checker Report - 14:47 - 25 - Nov

There are 1 files with format errors

To see a complete report of formatting issues, download the differences file
To fix the build, please run mvn formatter:format in your branch and commit the changes.
Optionally you might add the following line in your .git/hooks/pre-commit file:
```
mvn formatter:format
```

Here is the list of files with format issues in your PR:

flow-server/src/test/java/com/vaadin/flow/server/frontend/TaskRunPnpmInstallTest.java

Artur- · 2022-11-25T14:42:18Z

So this seems to happen almost always when I have a test case that just runs

npx --yes --quiet pnpm --shamefully-hoist=true --ignore-scripts install

and nothing else. It does not reproduce if I just run some node command like node -e 'console.log("hello")' nor does it reproduce if I run npm instead of pnpm..

sonarcloud · 2022-11-25T14:49:08Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
20 Code Smells

No Coverage information
0.0% Duplication

Artur- · 2022-11-28T08:51:28Z

This is potentially caused by nodejs/node#42826 as Node seems to change the stdin filedescriptor to non-blocking in some cases. If this happens when running pnpm then it would explain the Resource temporarily unavailable message which apparently corresponds to EAGAIN https://stackoverflow.com/a/4058377/3336733 or in other words "The file descriptor you are trying to read from does not have any data right now and you have requested me not to wait for data to become available"

Artur- · 2022-11-28T11:27:47Z

Created https://issues.apache.org/jira/browse/SUREFIRE-2127

Artur- · 2022-12-13T14:31:25Z

Let's close this until the issue has a solution

vaadin-bot added the +0.0.1 label Nov 11, 2022

Artur- closed this Nov 17, 2022

MarcinVaadin reopened this Nov 21, 2022

MarcinVaadin self-assigned this Nov 21, 2022

MarcinVaadin requested changes Nov 21, 2022

View reviewed changes

pom.xml Show resolved Hide resolved

pom.xml Show resolved Hide resolved

MarcinVaadin previously approved these changes Nov 21, 2022

View reviewed changes

MarcinVaadin dismissed their stale review via 72847e0 November 21, 2022 13:44

vaadin-bot added +1.0.0 and removed +0.0.1 labels Nov 21, 2022

MarcinVaadin reviewed Nov 21, 2022

View reviewed changes