Jcuda on Hadoop exception

Byte-Welt - Projekte / Projects JCuda
1

Hello everyone, I 've got a problem when I write some Jcuda program based on Hadoop MapReduce.

I use the jcudaUtill. The KernelLauncherSample can be successfully executed on my worker node. However, When I submit a program containing jcuda to Hadoop MapReduce. I got following errors. Any reply will be appreciated!

10/12/09 15:41:39 INFO mapred.JobClient: Running job: job_201012091523_0002
10/12/09 15:41:40 INFO mapred.JobClient: map 0% reduce 0%
10/12/09 15:41:53 INFO mapred.JobClient: Task Id : attempt_201012091523_0002_m_000000_0, Status : FAILED
Error: Could not load native library
attempt_201012091523_0002_m_000000_0: [GC 7402K->1594K(12096K), 0.0045650 secs]
attempt_201012091523_0002_m_000000_0: [GC 108666K->104610K(116800K), 0.0106860 secs]
attempt_201012091523_0002_m_000000_0: [Full GC 104610K->104276K(129856K), 0.0482530 secs]
attempt_201012091523_0002_m_000000_0: Error while loading native library with base name “JCudaDriver”
attempt_201012091523_0002_m_000000_0: Operating system name: Linux
attempt_201012091523_0002_m_000000_0: Architecture : amd64
attempt_201012091523_0002_m_000000_0: Architecture bit size: 64
10/12/09 15:42:00 INFO mapred.JobClient: Task Id : attempt_201012091523_0002_m_000000_1, Status : FAILED
Error: Could not load native library
attempt_201012091523_0002_m_000000_1: [GC 7373K->1573K(18368K), 0.0045230 secs]
attempt_201012091523_0002_m_000000_1: Error while loading native library with base name “JCudaDriver”
attempt_201012091523_0002_m_000000_1: Operating system name: Linux
attempt_201012091523_0002_m_000000_1: Architecture : amd64
attempt_201012091523_0002_m_000000_1: Architecture bit size: 64

It looks like The jcuda library file can not be sucessfully loaded. Actually, I tried many combinations.

  1. I include all the jcuda library files in my TaskTracker’s classpath, and also include jcuda library file into my mapreduce program.
  2. TaskTracker’s classpath w/o jcuda library, but my program contains them
  3. TaskTracker’s classpath w/ jcuda librara, but my program w/o them

All of them report the same error above.

2

I’m not familiar with Hadoop - but assume it may be using some tricky mechanisms for loading the classes…
In any case, I have just scrolled quickly over the “Map Reduce Tutorial”, and the “java.library.path” in this section jumped into my eye - but I have NO idea whether this may be used to specify the location of the JCuda libraries in your case.
I’ll have to read this more thoroughly…

3

problem solved. It is the java.library.path setting.

Thanks