UnsatisfiedLinkError: Native Library /tmp/libJOCL_0_1_9-linux-x86_64.so already loade

Piotr · 21. Mai 2015 um 16:41

Hi Marco,

I’m working on my pet project that uses JOCL: https://github.com/tarsa/SortAlgoBox

I have problem as I’m not able to run build from command line. The problem is, I think, because my project has many modules and SBT (build tool) runs them in some sort of separation. That’s a stacktrace which is given during execution of OpenCL test from a different module than the first OpenCL test:

[info] Exception encountered when attempting to run a suite with class name: pl.tarsa.sortalgobox.random.GpuMwc64xTest *** ABORTED ***
[info]   java.lang.UnsatisfiedLinkError: Error while loading native library "JOCL_0_1_9-linux-x86_64" with base name "JOCL_0_1_9"
[info] Operating system name: Linux
[info] Architecture         : amd64
[info] Architecture bit size: 64
[info] ---(start of nested stack traces)---
[info] Stack trace from the attempt to load the library as a file:
[info] java.lang.UnsatisfiedLinkError: no JOCL_0_1_9-linux-x86_64 in java.library.path
[info] 	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1865)
[info] 	at java.lang.Runtime.loadLibrary0(Runtime.java:870)
[info] 	at java.lang.System.loadLibrary(System.java:1122)
[info] 	at org.jocl.LibUtils.loadLibrary(LibUtils.java:80)
[info] 	at org.jocl.CL.<clinit>(CL.java:47)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextDescription$.enumerate(CLContextDescription.scala:44)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsManager$.contextsDescriptions$lzycompute(CLContextsManager.scala:6)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsManager$.contextsDescriptions(CLContextsManager.scala:6)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsManager$.createGpuContext(CLContextsManager.scala:15)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsCache$.gpuContext$lzycompute(CLContextsCache.scala:25)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsCache$.gpuContext(CLContextsCache.scala:25)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsCache$.withGpuContext(CLContextsCache.scala:32)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64x.generate(GpuMwc64x.scala:48)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$mcVI$sp$1.apply$mcVI$sp(GpuMwc64xTest.scala:32)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$mcVI$sp$1.apply(GpuMwc64xTest.scala:31)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$mcVI$sp$1.apply(GpuMwc64xTest.scala:31)
[info] 	at scala.collection.immutable.List.foreach(List.scala:381)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply$mcVI$sp(GpuMwc64xTest.scala:31)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply(GpuMwc64xTest.scala:30)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply(GpuMwc64xTest.scala:30)
[info] 	at scala.collection.immutable.List.foreach(List.scala:381)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(GpuMwc64xTest.scala:30)
[info] 	at pl.tarsa.sortalgobox.tests.CommonUnitSpecBase.guardedOpenCLTest(CommonUnitSpecBase.scala:40)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1.apply$mcV$sp(GpuMwc64xTest.scala:28)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1.apply(GpuMwc64xTest.scala:28)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1.apply(GpuMwc64xTest.scala:28)
[info] 	at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] 	at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] 	at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] 	at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] 	at org.scalatest.Transformer.apply(Transformer.scala:20)
[info] 	at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1647)
[info] 	at org.scalamock.scalatest.AbstractMockFactory$$anonfun$withFixture$1.apply(AbstractMockFactory.scala:35)
[info] 	at org.scalamock.scalatest.AbstractMockFactory$$anonfun$withFixture$1.apply(AbstractMockFactory.scala:34)
[info] 	at org.scalamock.MockFactoryBase$class.withExpectations(MockFactoryBase.scala:41)
[info] 	at pl.tarsa.sortalgobox.tests.CommonUnitSpecBase.withExpectations(CommonUnitSpecBase.scala:28)
[info] 	at org.scalamock.scalatest.AbstractMockFactory$class.withFixture(AbstractMockFactory.scala:34)
[info] 	at pl.tarsa.sortalgobox.tests.CommonUnitSpecBase.withFixture(CommonUnitSpecBase.scala:28)
[info] 	at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1644)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
[info] 	at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[info] 	at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1656)
[info] 	at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
[info] 	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
[info] 	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
[info] 	at scala.collection.immutable.List.foreach(List.scala:381)
[info] 	at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info] 	at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:390)
[info] 	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:427)
[info] 	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
[info] 	at scala.collection.immutable.List.foreach(List.scala:381)
[info] 	at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info] 	at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
[info] 	at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
[info] 	at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1714)
[info] 	at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683)
[info] 	at org.scalatest.Suite$class.run(Suite.scala:1424)
[info] 	at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
[info] 	at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
[info] 	at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1760)
[info] 	at org.scalatest.FlatSpec.run(FlatSpec.scala:1683)
[info] 	at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
[info] 	at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
[info] 	at sbt.TestRunner.runTest$1(TestFramework.scala:76)
[info] 	at sbt.TestRunner.run(TestFramework.scala:85)
[info] 	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[info] 	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[info] 	at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
[info] 	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[info] 	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[info] 	at sbt.TestFunction.apply(TestFramework.scala:207)
[info] 	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
[info] 	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
[info] 	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[info] 	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[info] 	at sbt.std.Transform$$anon$4.work(System.scala:63)
[info] 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[info] 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[info] 	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[info] 	at sbt.Execute.work(Execute.scala:235)
[info] 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[info] 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[info] 	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[info] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[info] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[info] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[info] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[info] 	at java.lang.Thread.run(Thread.java:745)
[info] Stack trace from the attempt to load the library as a resource:
[info] java.lang.UnsatisfiedLinkError: Native Library /tmp/libJOCL_0_1_9-linux-x86_64.so already loaded in another classloader
[info] 	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1903)
[info] 	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1822)
[info] 	at java.lang.Runtime.load0(Runtime.java:809)
[info] 	at java.lang.System.load(System.java:1086)
[info] 	at org.jocl.LibUtils.loadLibraryResource(LibUtils.java:188)
[info] 	at org.jocl.LibUtils.loadLibrary(LibUtils.java:91)
[info] 	at org.jocl.CL.<clinit>(CL.java:47)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextDescription$.enumerate(CLContextDescription.scala:44)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsManager$.contextsDescriptions$lzycompute(CLContextsManager.scala:6)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsManager$.contextsDescriptions(CLContextsManager.scala:6)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsManager$.createGpuContext(CLContextsManager.scala:15)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsCache$.gpuContext$lzycompute(CLContextsCache.scala:25)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsCache$.gpuContext(CLContextsCache.scala:25)
[info] 	at pl.tarsa.sortalgobox.opencl.common.CLContextsCache$.withGpuContext(CLContextsCache.scala:32)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64x.generate(GpuMwc64x.scala:48)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$mcVI$sp$1.apply$mcVI$sp(GpuMwc64xTest.scala:32)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$mcVI$sp$1.apply(GpuMwc64xTest.scala:31)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2$$anonfun$apply$mcVI$sp$1.apply(GpuMwc64xTest.scala:31)
[info] 	at scala.collection.immutable.List.foreach(List.scala:381)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply$mcVI$sp(GpuMwc64xTest.scala:31)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply(GpuMwc64xTest.scala:30)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply(GpuMwc64xTest.scala:30)
[info] 	at scala.collection.immutable.List.foreach(List.scala:381)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(GpuMwc64xTest.scala:30)
[info] 	at pl.tarsa.sortalgobox.tests.CommonUnitSpecBase.guardedOpenCLTest(CommonUnitSpecBase.scala:40)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1.apply$mcV$sp(GpuMwc64xTest.scala:28)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1.apply(GpuMwc64xTest.scala:28)
[info] 	at pl.tarsa.sortalgobox.random.GpuMwc64xTest$$anonfun$1.apply(GpuMwc64xTest.scala:28)
[info] 	at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info] 	at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info] 	at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info] 	at org.scalatest.Transformer.apply(Transformer.scala:22)
[info] 	at org.scalatest.Transformer.apply(Transformer.scala:20)
[info] 	at org.scalatest.FlatSpecLike$$anon$1.apply(FlatSpecLike.scala:1647)
[info] 	at org.scalamock.scalatest.AbstractMockFactory$$anonfun$withFixture$1.apply(AbstractMockFactory.scala:35)
[info] 	at org.scalamock.scalatest.AbstractMockFactory$$anonfun$withFixture$1.apply(AbstractMockFactory.scala:34)
[info] 	at org.scalamock.MockFactoryBase$class.withExpectations(MockFactoryBase.scala:41)
[info] 	at pl.tarsa.sortalgobox.tests.CommonUnitSpecBase.withExpectations(CommonUnitSpecBase.scala:28)
[info] 	at org.scalamock.scalatest.AbstractMockFactory$class.withFixture(AbstractMockFactory.scala:34)
[info] 	at pl.tarsa.sortalgobox.tests.CommonUnitSpecBase.withFixture(CommonUnitSpecBase.scala:28)
[info] 	at org.scalatest.FlatSpecLike$class.invokeWithFixture$1(FlatSpecLike.scala:1644)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$runTest$1.apply(FlatSpecLike.scala:1656)
[info] 	at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[info] 	at org.scalatest.FlatSpecLike$class.runTest(FlatSpecLike.scala:1656)
[info] 	at org.scalatest.FlatSpec.runTest(FlatSpec.scala:1683)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$runTests$1.apply(FlatSpecLike.scala:1714)
[info] 	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:413)
[info] 	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
[info] 	at scala.collection.immutable.List.foreach(List.scala:381)
[info] 	at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info] 	at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:390)
[info] 	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:427)
[info] 	at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:401)
[info] 	at scala.collection.immutable.List.foreach(List.scala:381)
[info] 	at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401)
[info] 	at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:396)
[info] 	at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:483)
[info] 	at org.scalatest.FlatSpecLike$class.runTests(FlatSpecLike.scala:1714)
[info] 	at org.scalatest.FlatSpec.runTests(FlatSpec.scala:1683)
[info] 	at org.scalatest.Suite$class.run(Suite.scala:1424)
[info] 	at org.scalatest.FlatSpec.org$scalatest$FlatSpecLike$$super$run(FlatSpec.scala:1683)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
[info] 	at org.scalatest.FlatSpecLike$$anonfun$run$1.apply(FlatSpecLike.scala:1760)
[info] 	at org.scalatest.SuperEngine.runImpl(Engine.scala:545)
[info] 	at org.scalatest.FlatSpecLike$class.run(FlatSpecLike.scala:1760)
[info] 	at org.scalatest.FlatSpec.run(FlatSpec.scala:1683)
[info] 	at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:462)
[info] 	at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:671)
[info] 	at sbt.TestRunner.runTest$1(TestFramework.scala:76)
[info] 	at sbt.TestRunner.run(TestFramework.scala:85)
[info] 	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[info] 	at sbt.TestFramework$$anon$2$$anonfun$$init$$1$$anonfun$apply$8.apply(TestFramework.scala:202)
[info] 	at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:185)
[info] 	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[info] 	at sbt.TestFramework$$anon$2$$anonfun$$init$$1.apply(TestFramework.scala:202)
[info] 	at sbt.TestFunction.apply(TestFramework.scala:207)
[info] 	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
[info] 	at sbt.Tests$$anonfun$9.apply(Tests.scala:216)
[info] 	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[info] 	at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:44)
[info] 	at sbt.std.Transform$$anon$4.work(System.scala:63)
[info] 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[info] 	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
[info] 	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
[info] 	at sbt.Execute.work(Execute.scala:235)
[info] 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[info] 	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
[info] 	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
[info] 	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
[info] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[info] 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[info] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[info] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[info] 	at java.lang.Thread.run(Thread.java:745)
[info] ---(end of nested stack traces)---
[info]   at org.jocl.LibUtils.loadLibrary(LibUtils.java:122)
[info]   at org.jocl.CL.<clinit>(CL.java:47)
[info]   at pl.tarsa.sortalgobox.opencl.common.CLContextDescription$.enumerate(CLContextDescription.scala:44)
[info]   at pl.tarsa.sortalgobox.opencl.common.CLContextsManager$.contextsDescriptions$lzycompute(CLContextsManager.scala:6)
[info]   at pl.tarsa.sortalgobox.opencl.common.CLContextsManager$.contextsDescriptions(CLContextsManager.scala:6)
[info]   at pl.tarsa.sortalgobox.opencl.common.CLContextsManager$.createGpuContext(CLContextsManager.scala:15)
[info]   at pl.tarsa.sortalgobox.opencl.common.CLContextsCache$.gpuContext$lzycompute(CLContextsCache.scala:25)
[info]   at pl.tarsa.sortalgobox.opencl.common.CLContextsCache$.gpuContext(CLContextsCache.scala:25)
[info]   at pl.tarsa.sortalgobox.opencl.common.CLContextsCache$.withGpuContext(CLContextsCache.scala:32)
[info]   at pl.tarsa.sortalgobox.random.GpuMwc64x.generate(GpuMwc64x.scala:48)

Is this a problem with JOCL, JNI or the project structure/ build tool itself?

Marco13 · 22. Mai 2015 um 01:21

Hi

After dozens of reports of “UnsatisfiedLinkErrors”, this issue finally seemed to be resolved by packing the natives into the JAR and managing them completely transparently for the user.

However, the one with the message “…already loaded in another classloader” is actually new for me…

And due to a lack of knowledge about SBT and Scala, I couldn’t do much more than wbesearching for “already loaded in another classloader” SBT: The first result refers to JOCL as well, but I guess (or hope) that this is just a coincidence (it also refers to an old version of JOCL, where the native library was still separate). Among the other results (e.g. in this one, but in others as well), people say something about an option fork in run := true (but I have no idea where this belongs… :o )

For short: Currently I can’t say where exactly the problem comes from, or how to avoid it. Did you try any of the suggestions from the first search results?

Sorry that I can’t give a more focussed advice here…

Piotr · 22. Mai 2015 um 04:32

Thank you for your quick response.

I’ve tried some of the solutions, but they didn’t work (maybe I don’t yet know how to use them). I’ll probably ask on SBT forums. However, thankfully for the time being, tests are working fine from within IntelliJ IDEA (they weren’t working before).

Nonetheless, I think there could be a quick and dirty solution - add some random string to extracted library names. Eg instead of ‘libOpenCL-XXX.so’ make ‘libOpenCL-XXX-random.so’. Does that make sense? If that change doesn’t involve too much hassle then I would appreciate if you provide me with modified build of JOCL (based on version 0.1.9 as I don’t have OpenCL 2.0 drivers).

Marco13 · 22. Mai 2015 um 04:45

So these tests ran after setting „fork in run := true“? Sorry, I don’t know SBT at all, so when you post this question somewhere (there seems to be no official SBT forum, is there?), then it might be interesting to have this link here as well.

Most likely, this would work. But originally (in the first version of JOCL that contained the natives in the JAR), this was done like this, implicitly, by creating a temporary file wtih File#createTempFile. The result was that one user complained that his temp directory was flooded with hundreds of DLLs, because under Windows, Java is not capable of deleting the file at program exit (the DLL can not be unloaded by the JVM, even when „deleteOnExit“ was set!). (Although… I think that there in fact was a tricky solution for this… wasn’t it @Fancy who proposed this somewhere…?)

(EDIT: Found it: http://forum.byte-welt.net/java-forum-erste-hilfe-vom-java-welt-kompetenz-zentrum/deployment/4600-wohin-am-besten-mit-der-native-library.html#post21313 , but the link seems to be dead…)

However, I can provide a „patched“ version of the JAR. I’m not sure whether I’ll be able to do this today (it might in the worst case take until tuesday), but in any case, I’d strongly prefer to avoid hacks like this. I’ll probably have to take a closer look at SBT (or read what the SBT guys say about the reason) in order to resolve this properly…

Piotr · 22. Mai 2015 um 05:10

It turns out SBT has a forum - it’s just StackOverflow and you only need to add ‘sbt’ tag to the question.

According to this answer: scala - UnsatisfiedLinkError with native library under sbt - Stack Overflow SBT uses some trickery under the hood which could require trickery on JOCL side. Or maybe not? I’ll ask the SBT guys.

Update:
Here’s the question: sbt - UnsatisfiedLinkError: Native Library already loaded in another classloader - Stack Overflow
Probably the title is a bit ugly.

Fancy · 22. Mai 2015 um 05:54

@Marco13 : Yes, the tumbler site is death. Unfortunately I don’t have a copy of my own site. But I think it was something like this:



import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.util.HashSet;
import java.util.List;
import java.util.Set;


public final class Loader {

    private final Set<File> resources = new HashSet<>();


    public Loader(final String prefix, final String suffix, final String... resources) throws IOException {

        for (final String resource : resources)
            System.load(unpack(prefix, suffix, resource).getAbsolutePath());

    }


    public void dispose() throws Exception {

        freeNativeLibraries();
        deleteNativeLibraries();

    }


    private File unpack(final String prefix, final String suffix, final String resource) throws IOException {

        try (InputStream is = getClass().getClassLoader().getResourceAsStream(resource)) {

            final File file = File.createTempFile(prefix, "." + suffix);

            resources.add(file);

            try (OutputStream os = new FileOutputStream(file)) {

                int read = 0;
                final byte[] bytes = new byte[1024];

                while ((read = is.read(bytes)) != -1)
                    os.write(bytes, 0, read);

            }

            return file;

        }

    }


    @SuppressWarnings("unchecked")
    private void freeNativeLibraries() throws Exception {

        final ClassLoader cl = getClass().getClassLoader();
        final Field field = ClassLoader.class.getDeclaredField("nativeLibraries");

        field.setAccessible(true);

        final List<Object> libs = (List<Object>) field.get(cl);

        if (libs != null)
            for (final Object lib : libs)
                freeNativeLibrary(lib);

    }


    private void freeNativeLibrary(final Object lib) throws Exception {

        final Field name = lib.getClass().getDeclaredField("name");
        name.setAccessible(true);

        final File file = new File((String) name.get(lib));

        if (resources.contains(file)) {

            final Method method = lib.getClass().getDeclaredMethod("finalize", new Class[0]);
            method.setAccessible(true);
            method.invoke(lib, new Object[0]);

        }

    }


    private void deleteNativeLibraries() {

        for (final File file : resources)
            if (file.exists())
                file.delete();

        resources.clear();

    }


    public static void main(final String[] args) throws Exception {

        final Loader loader = new Loader("lwjgl-", "so", "liblwjgl64.so");

        loader.dispose();


    }


}

Gruß
Fancy

Marco13 · 22. Mai 2015 um 06:17

@Fancy Oh yes, that was it (involving some reflection hacks) - any attribution required IFF I use (a derived version of) this?
@Piotr The hints about the “magic renaming stuff” that is done in SBT brought some other keywords into play - I think this one might be related: https://github.com/sbt/sbt-assembly/issues/141 (although I’m not sure…)

Fancy · 22. Mai 2015 um 07:58

@Marco13 : For me, no attribution is necessary. The idea behind this floating around the web since several years and I derived it from someone else. Who itself derived it from someone else, too. And maybe so on…

Marco13 · 22. Mai 2015 um 09:40

@Fancy OK, I just considered adding this to JOCL (so that it might become part of the “official” 0.2.0 release), but … where is dispose supposed to be called? I guess this has to be done via a shutdown hook, right?
@Piotr I put a temporarily, quickly patched-together version at http://jocl.org/JOCL-0.1.9-uniqueTempNativeNames.jar : It appends a random UUID to the names of the temporary native files.

Hopefully, I can quickly resolve this properly, either by coping with the SBT ClassLoader magic, or by adopting the “Loader” accordingly…

Fancy · 22. Mai 2015 um 11:07

Yes, shutdown hook will be fine. Additional I use a cleanup when the library starts e.g.:

        
        final File dir = new File(System.getProperty("java.io.tmpdir"));

        final File[] files = dir.listFiles(new FilenameFilter() {

            @Override
            public boolean accept(final File f, final String s) {

                return new File(f, s).isFile() && s.startsWith(prefix);

            }

        });

        for (final File file : files)
            file.delete();
        
    }```

If the JVM terminate unexpected and the shutdown hook is not called, the death files will be removed on the next run. No bad things happen if another application instance is running. Files can not be deleted or exists in memory until the corresponding instance is terminated.

Gruß
Fancy

Marco13 · 22. Mai 2015 um 11:18

OK, I’ll consider this for JOCL 0.2.0 (it’s still a SNAPSHOT due to missing Linux/Mac libs, anyhow)

Piotr · 22. Mai 2015 um 12:08

I’ve tried using the JAR from http://jocl.org/JOCL-0.1.9-uniqueTempNativeNames.jar and it works. In /tmp there are two copies of JOCL created per test run when running from SBT (as there are two modules with tests involving OpenCL), but only one when running from IntelliJ (seems that IntelliJ is handling tests differently, but they work as they should anyway). It seems it’s okay.

I’m pretty sure the github issue about sbt-assembly and duplicate libraries is a different problem than mine.

Thanks for your help.

Marco13 · 25. Mai 2015 um 09:43

I added a bounty to the stackoverflow question, maybe someone who knows SBT better than me finds a “simple” solution. Otherwise, I’ll consider the Reflection/ShutdownHook solution, but I’ll have to carefully check whether this might have undesired side effects. Otherwise, I also considered an alternative, like adding a JVM parameter that allows selecting the behavior (e.g. “uniqueLibNames=true” for creating random temp file names), but I’m still not sure what’s the best approach here…

Piotr · 27. Mai 2015 um 15:56

Alternatively you could define a place where filename for library is generated so it could be swapped before loading of JOCL. Like this:

    protected File defaultJoclFile = new File(
            System.getProperty("java.io.tmpdir"), "libOpenCL.so");

    public File getJoclFile() {
        return defaultJoclFile;
    }

    public static JoclFileProvider instance = new JoclFileProvider();
}

class Main {
    public static void main(String[] args) {
        System.out.println(JoclFileProvider.instance.getJoclFile());
    }
}

That would be the most flexible solution.

Anyway, if you settle on something then post a candidate version so I can test it.
Thanks.

Marco13 · 28. Mai 2015 um 03:11

I’m not sure how this should solve the SBT issue, but probably did just not understand how this is intended to be used. Should this be something like

// In run/project A:
JOCL.setFileProvider(new JoclFileProvider("foo.dll"));

// In run/project B:
JOCL.setFileProvider(new JoclFileProvider("bar.dll"));

?

Piotr · 28. Mai 2015 um 08:16

No, the “instance” variable should be overwritten before loading of JOCL library. So it should be:

    static {
        JoclFileProvider.instance = new JoclFileProvider {
            File joclFile = generateRandomFileName();
            @Override
            public File getJoclFile() {
                return joclFile;
            }
        }
    }
    public static void initialize() {} // invoking this for the first time causes the static block to run, further invocation do nothing
}```
Then make a base test class that overrides a method that is invoked before every test and put JoclInitializer.initialize() there.

That would ensure every classloader gets its own unique JOCL library file name.

Marco13 · 28. Mai 2015 um 09:34

I see. Still, I’ll have to think about this more throughoutly (I’m a bit slow, suffering from sleep depreviation). I’m still not sure how this would be integrated (i.e. at which point the decision about the behavior should be made). In any case, I’d be hesitant to expose this as an interface, but would rather consider something like JOCLLibraryPolicy.setRandomNaming(true); to limit the possibilities for misuse. But even this is subtle, because it has to be called before the CL class is loaded. (The fact that the library in the CL class is loaded in a static initializer was not the best idea in hindsight - today, I would solve this differently).

Piotr · 28. Mai 2015 um 10:23

I’m still not sure how this would be integrated (i.e. at which point the decision about the behavior should be made).

Since the problem exists during testing then the decision should be made before every test. If I understand correctly, each classloader has its own copy of classes and their static state so with my idea each classloader would execute the static block in JoclInitializer once so each classloader should end up having it’s own single copy of JOCL library.

JOCLLibraryPolicy.setRandomNaming(true); would work also, but would also need to be invoked before each test as static state is different for different classloaders.

Environmental variables shouldn’t cause a lot of hassle for end user, provided that I can set them up from SBT easily (should be doable if I enable JVM forking).

I’m not strongly biased towards any of the solutions. If something works and doesn’t break existing programs then it’s OK for now.

Marco13 · 28. Mai 2015 um 15:36

Yes, I think we understood this correctly, but to emphasize: I want to make sure that the „default“ behavior is that there are no random names.

I’m not so much a fan of defining environment variables, but the same can be achieved by passing a „-DsomeFlag=true“ to the JVM at startup. But I also still have to „develop a bias“ (-> „to make a decision“ ;-))

Marco13 · 1. Juni 2015 um 11:29

OK, the Bounty ran out (and I would just be a „Trusted User“ by now :o) ), but nobody answered…

However, I just pushed the corresponding change to GitHub - gpu/JOCL: Java bindings for OpenCL

The default behavior is still the old one, where the library names only differ in the version number. Maybe I’m overly careful with changes here, but I didn’t want to introduce any bugs here. But now, it is possible to start the application with

java -DuniqueLibraryNames=true TheApplication

In this case, it will generate random library names, and attempt to delete these files at program exit (roughly based on the code that @Fancy provided. This code is a bit TOO hacky, admittedly, so it is not a public/default solution now…)

This will likely be part of 0.2.0 (final).