SCM and Maven

Hey. :slight_smile:

Are the source packages on the download page the only way to get to the source, or is there a publicly available SCM?

It would be nice if the project would be available through and buildable by maven. the first would allow projects using your lib to define a dependency to your lib, and the second would make it easier for other devs to contribute.
If you’d use a distributed SCM, you could allow devs to contribute, and still keep full control over what ends up in the main repository.

If you like any of these ideas and need help, i should be able to spare some time. :slight_smile:
i am also in #jocl on freenode (at least today).

Hello

At the moment, JOCL is not officially available in a public repository. I hope that I will soon find the time to move the source code to http://code.google.com/p/jocl/ , but at the moment, this page is still empty.

Recently, I noticed that the source code of JOCL is in this repository (and maybe some others I’m not aware of), but I see that it would have benefits to make the source accessible via an own public repository.

The project structure of JOCL is… as simple as it can be :wink: so it should not be too complex to get it running with Maven. But I’m not sure when I’ll have the chance to get deeper involved in Maven, so any support in this direction would be appreciated.

bye
Marco

ahh nice :slight_smile:

the java part of the project surely is no problem (copy&paste maven hello-world + one or two line changes maybe), but i never used maven for a native project, so i do not know how well the support would be for that. though there are other projects like JNA, that are fully maven-ized, and also ship native libs for various distributions. maybe some inspiration can be found there.
and yeah, i can help of course. :slight_smile:

i would strongly suggest using a distributed SCM though. my favourite is git (+ github), but some prefer Mercurial or bazaar. In my opinion, it is nearly a must for any open-source project. it allows for much easier contribution, without you having to give commit access to everyone → more control for you.
i am also willing to help you with that (setting up, learning how to use it…)

First of all, thanks for your support :slight_smile:

Yes, the native part might be the tricky one, but even if it’s not possible to describe the build process with Maven (apart from putting the makefiles into the repository), it might at least help to manage the dependencies.

Do you already have an idea about how the repository could be set up?

i suggest maven default structure, which would be:
/pom.xml (maven project file)
/src/main/java//*.java
/src/main/native/
/.c
/src/main/native/**/.h

if you want, and are ok with git, i can make the repo on github, and try my luck with the pom.xml file.
you could then fork it (= copy the repo) under your account, and use that as you official repo.

… i forgot.
for the native part, this might be useful (in case you want to try yourself):
http://mojo.codehaus.org/maven-native/native-maven-plugin/index.html

… looks like you would need two maven projects, one for the native, and one for the java part.
that means it is also possible to keep the structure as you have it now.

The linked site contains a dedicated section with instructions for JNI DLLs, so this looks promising.

I’m not familar with GIT - and there might be another collision with Michael Bien’s JOCL at github :wink: But I’ll try to have a look at Maven and Git, and how all this may work. BTW, would there be any difficulties with Maven and SVN?

no, no difficulties with Maven and SVN.
the problem with SVN is not of an imminent technical nature, but of a practical one. the problem is, that it is not distributed, and thus makes it impossible to use some workflows which make it much easier and secure for other devs to contribute to your code, which is especially important for open-source projects.
i work in quite some open-source projects, and all the ones that encourage contribution are using or moving to distributed SCMs. this is one of the things which is quite hard to see how it works when never having experienced it. i did use SVN exclusively in the past, and for me, moving from SVN to git was like the one between having a small plant in your office and walking the forest.
The only possible problem i could see for you (as i guess you are a windows based dev, seeing the VS project file in the source), is that there are not as many GUI tools for git like for SVN yet. i personally used SmartSVN, and did not miss it a bit, even though i only use the most basic git GUI tools. That is something you have to find out yourself though. :slight_smile:
it sure can’t hurt to get to know git anyway.

btw… are you swiss? (not that it matters for this, really. Aargau here.)

Hello,

I want to at least gain some basic experience with GIT, but I’m not sure if this basic experience should be achieved by using it for a public project (and possibly messing something up very badly :o ). At the moment I’m mainly using SVN through Eclipse and Tortoise, but since there are GIT Plugins for Eclipse and even a TortioseGIT, it should not be too hard to get started - except, maybe, due to the slightly different philosophy of both approaches. I’ll try to set up a small „sandbox“ GIT project locally for my first tests, probably during next week, to get familiar with the basic GIT workflow.

BTW: I’m from Germany. Grüße nach Aargau :slight_smile:
Marco

:slight_smile:
soo… have you been experienced yet? :smiley:
the good thing when experimenting with git is, you can do very fast copies of the whole repository locally (faster then copying the folder). under linux, even much much faster.
so whenever you want to do something you are unsure about, just make a clone of the repo as backup (git clone origDir backupDir).

by accident, i also found this plugin:
http://java.freehep.org/freehep-nar-plugin/
just mentioning it here for possible further reference, is also used for native compilation in maven.

Yes, during this week I played a little with GIT, although I did not have much time, I’ll probably continue this week and see how it works with non-local repositories, but have to set up a github account first. Concerning Maven, it’s still pending, but I’ll hopefully find the time for that too.

ahh nice :slight_smile:
i am having a try with maven on JOCL myself right now. the java part was no problem of course, but am still workign on the native one. i tried with the second plugin i linked, but i found out it is deprecated (and i did not find it on any maven repositories). i found the project that continues along its line here:
http://duns.github.com/maven-nar-plugin/

it is on its way to get one of the core maven plugins, and it seems to be quite comfortable to use and well abstracted and so on. though, it is not currently on any repositories either, so you have to compile it yourself. instructions are here (you need maven and git installed):
cd ~/src/
git clone https://github.com/duns/cpptasks-parallel.git
cd cpptasks-parallel
mvn
cd …
https://github.com/duns/maven-nar-plugin.git
cd maven-nar-plugin
mvn

you can now use that plugin in your maven project. i will post again in case i got it all working.

…update

it packaged successfully now!

to get OpenCL libs and headers under Gentoo:
emerge -av dev-util/nvidia-cuda-toolkit

i also had to change something in the source:
JOCL.cpp:2213
-#elif
+#else

native sources have to be in
src/main/java/
src/main/native/

i was able to use the result in an other (also maven based) project, and it compiled fine. i did not test if it would run… do not have code for that, but the only problem i could see is that i may have to adjust some native compile parameters.

Oh, thanks for all your efforts and detailed information :slight_smile:
I was quite busy this week, but should find more time for this now. I’ll dig through this during the weekend.

uploaded what i have so far:
http://filebin.ca/rozohw/JOCL-0.1.4-SNAPSHOT-hoijuiMaven-src.zip
that java source was not changed at all, just moved, and in C++, only that one line i mentioned (#elif#else, saw it mentioned in an other thread as well).
i also edited the two Makefiles and the VS project file (just changed source location there, don’t did not find anything else related to my changes).

changes in Makefile_common.mk:

-SRCDIR ?= ./src/JOCLJNI/
+SRCDIR ?= ./src/main/native/

-CXX := g++
+CXX := g++ -fPIC

in Makefile, it are some more… i guess you’ll see it yourself.

if you put things on git, maybe it would be good to have one commit per each release that you still have around.
if you give me the name and email you want to use as git committer (see http://help.github.com/git-email-settings/), i could setup the git repo with that basic history in master, and some of my changes in a branch or two. i could then put it on github („inofficially“ of course) so you can get it from there (to your account or local machine).
should be quite easy to do for me, in case you feel not yet comfortable with git.

i am on freenode #jocl again, in case you want to talk :wink:

ah yeah, nearly forgot:
pom.xml is the maven project file, and you have to adjust the OpenCL/CUDA include path (the rest might be fine).

test with (needs Maven installed):
mvn package

OK, I was not here during the weekend, but had the chance to read a little.

I’ll check this esle/elif issue, and fix it asap.

Concerning the first import, I think it should be possible to convert a local SVN repository into a GIT repository, to save the comments and tags, but it does not seem to be so easy…

I’m still procrastinating the maven setup and the plugin stuff (admittedly, I’ve never been so keen on deployment systems … the trends are changing frequently, and I’d rather consider myself as a programmer and not a deployer ;)). However, I’ll see if I get it running…

But in general: Will it be possible to remove the dependency to a specific include path? Otherwise, I’m not sure how to maintain the POM properly…

ahaa…
yeah of course, SVN to git conversion is possible, i did it some times already… you never mentioned you have it in SVN. :wink:

you should create an authors conversion file with this format:
svnAccount = Your Full Name your@email.com
svnAccount2 = His Full Name his@email.com

containing everyone that ever committed. you should use the same „Your Full Name your@email.com“ you want to use to commit later on.

then i recommend using svn2git:

use like this:
svn2git --authors authorsFile.txt /local/svn/repo/path/

in the end, use gitk to look at the history of the git repo, to see if everything went fine.

OK, I’m working on that (sorry for the slow progress, there are way too many tasks in my queue - will be a busy christmas vacation…).

It reported some errors in the nar plugin:


-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.maven.plugin.nar.test.TestLinkerVersion
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.156 sec <<< FA
ILURE!
Running org.apache.maven.plugin.nar.test.TestNarLayout21
[debug] Using org.apache.maven.plugin.nar.NarLayout21
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec
Running org.apache.maven.plugin.nar.test.TestNarLayout20
[debug] Using org.apache.maven.plugin.nar.NarLayout20
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec
Running org.apache.maven.plugin.nar.test.TestNarFileLayout10
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec

Results :

Tests in error:
  testVersion(org.apache.maven.plugin.nar.test.TestLinkerVersion)

Tests run: 12, Failures: 0, Errors: 1, Skipped: 0

...

But I’ll see in how far it works anyhow.

I’ll also give svn2git a try ASAP. I have created a GIT account, but didn’t want to use my name as the „root“ name, so I grabbed the rather general and catchy name ‚gpu‘ :wink: (If GIT convinces me, I’ll possibly host JCuda there as well…). Additionally, I have to see how I can adjust the user names from my (local) SVN repository when converting it to GIT.

nice to see progress! :slight_smile:

I explained that in my last post.

Hello, I would also like to add my encouragement to adding JOCL to Maven. I am the project leader of the Encog Machine Learning Framework project. We make use of JOCL for GPU acceleration of some of our neural network algorithms. We just added to Maven, and the only flaw in our setup is that we use JOCL, which is not registered. I think adding Maven would encourage wider adoption of your framework.

By the way, we are quite happy with JOCL, it does everything we need and works quite well. Good job!

Thanks,

Jeff Heaton