java.nio.channels.ClosedChannelException

[boswellp]

On Ubuntu 18.04, running ETL with the usual command line:

mvn exec:java -Dexec.mainClass="org.heigit.bigspatialdata.oshdb.tool.importer.extract.Extract" -Dexec.args="--pbf /home/ertert/oshdb-new/oshdb-tool/etl/files/switzerland-internal.osh.pbf -tmpDir ./tmpFiles -workDir ./results --timevalidity_from 2017-01-01" -Dexec.cleanupDaemonThreads=false

when execution starts I get:

joining on thread Thread[org.heigit.bigspatialdata.oshdb.tool.importer.extract.Extract.main(),5,org.heigit.bigspatialdata.oshdb.tool.importer.extract.Extract]
extracting key tables ...[WARNING]
java.lang.RuntimeException: java.nio.channels.ClosedChannelException

which becomes :

BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.887 s
[INFO] Finished at: 2020-11-26T08:24:50Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:java (default-cli) on project etl: An exception occured while executing the Java class. java.nio.channels.ClosedChannelException -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:exec-maven-plugin:3.0.0:java (default-cli) on project etl: An exception occured while executing the Java class. java.nio.channels.ClosedChannelException

java.nio.channels.ClosedChannelException is a fairly common issue and generally related to programming. Any thoughts?

[boswellp]

I should perhaps add that I reinstalled OSHDB 0.5.9 and it works fine for all three ETL stages. The ClosedChannelException issue is with version 0.5.10.

[rtroilo]

Hi Peter,

thank you for reporting this bug.
One question, could you reproduce this bug with the same dataset and oshdb version. Because there should not be a major change between 0.5.9 and 0.5.10 for the ETL stages.

But we should definitely catch this exception.

[boswellp]

Rafael, hello, from memory, I used the same database for 0.5.9 and 0.5.10. 0.5.9 ran OK; 0.5.10 threw the error. I used one of my own exports as the database. And I think I also tried 0.5.10 with a Geofabrik history download which also gave the error. But I will check again. Best, Peter

On November 26, 2020 2:14 PM Rafael Troilo ***@***.***> wrote: Hi Peter, thank you for reporting this bug. One question, could you reproduce this bug with the same dataset and oshdb version. Because there should not be a major change between 0.5.9 and 0.5.10 for the ETL stages. But we should definitely catch this exception. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub #286 (comment) , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXQNH55TGLAGVRUMJGYMHLSRZICVANCNFSM4UDNCWLQ .

..................................................................................... Peter Boswell PeterBoswell.com Business Development Mobile: +41 ***** Email: ***** Address: ***** .....................................................................................

[boswellp]

Hello,

I was wondering if there is any solution for this issue.

For both Oracle java and OpenJDK I have exactly the same issue for OSHDB 0.6.1 to 0.6.4. etl 0.5.11 works fine in all cases. I've tried several osh files, all of which work with 0.5.11 and by all accounts are history files (checked using osmium fileinfo).

[rtroilo]

Hi Peter,

thank you for your report. The closed channel exception at the end should no corrupt your generated oshdb. Could you confirm that your generated oshdb does not work?
And sorry for this annoying error message, we will remove this.

[boswellp]

Rafael, hello, the extraction does not run at all. The extraction starts with the "extracting key tables ... " then the [WARNING] and no extract files are created. So one cannot move to the next ETL stage. Perhaps I should say that we are on own-hosted Ubuntu 18.04 LTS servers. Best, Peter

On June 2, 2021 11:04 AM Rafael Troilo ***@***.***> wrote: Hi Peter, thank you for your report. The closed channel exception at the end should no corrupt your generated oshdb. Could you confirm that your generated oshdb does not work? And sorry for this annoying error message, we will remove this. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub #286 (comment) , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABXQNH6W764UWLYC7KQZMSLTQXX3VANCNFSM4UDNCWLQ .

..................................................................................... Peter Boswell PeterBoswell.com Business Development Mobile: +41 ***** Email: ***@***.*** ***@***.*** Address: ***** .....................................................................................

[Hagellach37]

This will be fixed by @rtroilo when re-integrating the new importer.