SCM and Maven

I think maven needs just a little real world maven experience until it makes “klick”. But then, you will think: “Holy shit how I could live without this!” I suppose you have already found the sonatype books?

And the pom.xml is just the beginning. Maybe at some time you wish to use unit test with maven, then integration test, then continuous integration, then…
And then, at this point, you will use all this stuff for any little piece of code. (Also for “hello world” ;))

Regards,
Fancy

You’re probably right. I’m not concerned with larger projects which use sophisticated build systems. And as I mentioned above, I see the necessity and potential benefits of Maven for building larger pure-Java-projects with nested dependencies and all that. But for a single-JAR library without dependencies, the process of building is not simplified, and for a JAR that requires native libs, Maven is not realistically applicable as a build tool at all. So its application is (for me, in this simple project) reduced to a ridiculously complex way of deploying a JAR, and the only thing that can be built on the target platform is … the platform-independent part of the library :wink:

Apart from that, until now I think Maven mainly imposes a set of rules, and shifts the effort of configuration according to these rules from the one who wants to use a library to the one who creates the library. Everybody could download the JAR from the website, sign it with GPG, „stage it as a snapshot“ (or whatever…) in his own repository, define a POM for it, and add it as a dependency in his main POM - and if this is too much effort for the user, … well, this is not my fault…

Of course, a library should be easy to use. I’m trying to make the usage of JOCL as simple as possible, and hope that will be useful for others. And admittedly, I assume that one of the main reasons for many people to use JOCL instead of other OpenCL bindings is NOT that JOCL is „better“, but that the build process and integration of these other OpenCL bindings are described in multi-page manuals, whereas the manual for the installation of JOCL consists of the single word: „Unzip“ :slight_smile:

But anyhow, once this is up and running, I can at least say that „I have Maven experience“ in job interviews :wink:

Soo… there is … something … on
https://oss.sonatype.org/content/repositories/snapshots/org/jocl/jocl/0.1.4-SNAPSHOT/
The JAR should contain the native libraries for Win32+64, Linux64 and Mac64. The JAR itself seems to work, but… since I’m not so sure what I just did there :o : Should this be “working” now, or does it have to be “staged” and “released” first or … what? Any hints about the next steps…?

At the moment, it is a snapshot release at the sonatype repository. If you will use this in a maven project, you must add this to a pom:

    <repositories>
        <repository>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
            <id>sonatype</id>
            <url>https://oss.sonatype.org/content/repositories/snapshots</url>
        </repository>
    </repositories>

    <dependencies>
        <dependency>
            <groupId>org.jocl</groupId>
            <artifactId>jocl</artifactId>
            <version>0.1.4-SNAPSHOT</version>
        </dependency>
    </dependencies>

For a snapshot this will be ok. But snapshots can not be synchronized with the central maven repository. So, the next step in the maven adventure will be a release. If you have this, you can request sync with the central maven repository via your jira ticket. (see point 8. and 9.). And then finally, all your users can add

<dependency>
     <groupId>org.jocl</groupId>
     <artifactId>jocl</artifactId>
     <version>0.1.4</version>
</dependency>

to their pom, if they wish to use jocl via maven. :wink:

I think that gives you 100% more money! :smiley:

Regards,
Fancy

Maven wants to talk to git and fails, git wants to talk to ssh and fails, “Mojo exceptions” are flying like bees, nexus is complaining about missing .asc files which it explicitly reported to be present … This is such a mess… :twisted:

Now was “staged” and “closed”, but of course, this version will not work, because it does not contain the native libs - these would have to be added to the GIT repository? Is there any way to convince this somatype-maven-nexus-crap-thing that there is just a JAR to be uploaded, and that it should NOT read and write around in some GIT repository?

Hm, I’m surprised. Why can sonatype write to your git? Write to git should only happen at the mvn release:* phase. But this is a “normal” maven thing, nothing sonatype specific. If you mean this, you can do this by hand. Simply use your snapshot version and remove all behind the version number.

I think you need:

jocl-0.1.4.pom
jocl-0.1.4.jar
jocl-0.1.4-javadoc.jar
jocl-0.1.4-sources.jar

and the corresponding .asc files.

In the sonatype docs at point 7b. is described how to bundle them via jar and upload via nexus UI. But I have never tried this. (I don’t sync with central, only with a private repository. (My lib is fare away from release)).

Nevertheless, what happens if you run a “mvn clean install”? Will this produce a valid artefact? I miss something that fetch your native libs and package it into the jar?

But yes, I think there is no reason why you should not check in your natives into git. Like any other binary resource. It is a little unusual, but since your native build is not maven organized I think it will be ok.

Regards,
Fancy

DO NOT but binaries into the git repo!
if you really have to do it, use a git submodule and put them there, so they wont pollute the history of your main repository. i don’t think they have to be there to get them into a maven repo though (but i am also tired :wink: ).

quote from encog forum (main dev (jeff) speaking):
Encog handles the native lib by not calling the JOCL JAR until the program specifically requests OpenCL functionality. Since I am sure most users do not use OpenCL, it does not make it too cumbersome. It is up to the user to obtain and install the correct native lib for their operating system, and we have a page that describes how to do that.

so… having even only the java part on maven is already good for them :slight_smile:

@Fancy: Well, initially it did not work because I had created the GIT repository in an own directory (because only parts of my develoment should be GITed, with a different directory structure), and the Maven Test again was in another directory - I simply was not aware of the fact that maven has to be run from within the local git repository… But the main problem was that I used a password for my SSH key, and … well, try to run this on a Windows PC, and we’ll soon see the rearely used :grr: - Smiley :wink:

I also tried bundeling the file into a „bundle.jar“, and it worked: The bundle contained all required files, and sonatype reported that all files and the corresponding .ASC files had been staged. And when closing it, it reported that the .ASC files were missing. Annoying, really… :frowning:

Concerning the libs-into-GIT-question:
@hoijui:
I don’t think it’s good to put many, large, freqently changing binaries into a source code repository (I did this locally with DLLs in SVN, and it turned out to blow up the repo uselessly - everything that is required to build the DLL is in the Repo anyhow).

That for Encog the JAR but not the native lib is installed may be considered as a simplification assuming that most peope won’t use OpenCL, but assuming that for future versions more people would like to use it, it would be nice to avoid the manual installation. It would be OK if there was a mechanism like „If there is the JAR, then assume that the native lib is present“, but having the JAR but NOT the native may be considered confusing and inconvienient, and the manual installation will lot unlikely be a potential source of the famous „UnsatisfiedLinkError“. That’s one of the main reasons why I wanted to put the native libs into the JAR, where they can be found at any time, and in the spirit of „Write once, run everywhere“.

I assume that it is somehow possible to configure Maven so that it does NOT need the binaries in the GIT repo, but pulls them from any other location and puts them into the target directory (some copy-dependencies thing or so). But again, setting this up properly is tremendously complicated, which means that there is something wrong about it. (Everybody who thinks that it is easy and all right: Send me the working POM via mail :slight_smile: )

I’am still surprised why the natives are in your snapshot and in the release there are not. However, you can configure the resources plugin to fetch resources from non standard places. Like this:

        <plugin>
                <artifactId>maven-resources-plugin</artifactId>
                <version>2.4.3</version>
                <executions>
                    <execution>
                        <id>copy-resources</id>
                        <phase>prepare-package</phase>
                        <goals>
                            <goal>copy-resources</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.outputDirectory}</outputDirectory>
                            <resources>
                                <resource>
                                    <directory>${basedir}/../dll/</directory>
                                    <filtering>false</filtering>
                                    <includes>
                                        <include>*.dll</include>
                                    </includes>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

Regards,
Fancy

The snapshot-building does not do any GIT access, I had built it from the local version (which contained the natives in the “resources” dir), but for the release, a sync with the GIT repo is required (or done automatically or something like that?), which does not contain the natives. Thanks for the example, I’ll try it early next week.

Well… this … might (!) have worked now. At least, I did a lot of things, and there have impressingly few error messages (which means: none :slight_smile: ).

Now there is something on https://oss.sonatype.org/content/repositories/releases/org/jocl/jocl/0.1.4c/ - this should probably be … „synced“ with the central maven repo soon ?!

Thanks to Hoijui for … well, not really for recommending Maven … :confused: or maybe? :confused: I’m not yet sure :wink: … but for providing the initial setup and support in this thread. And thanks to Fancy for the hint with the copy-resources-thing. I still had to mess around in the POM, because the „release : prepare“ seems to work on the target directory, and the „release : perform“ seems to work on a „/checkout/target“ SUB(!)directory of the target directory … ?! But may have worked now … in the end.

I’ll probably not get used to Maven so quickly. It may work great as long as your POM just consists of a long sequence of standard dependencies - but when there is anything not fitting into the Brave New Maven World (like native libraries), you just can hope that you get it running somehow and never-ever have to touch this part of your POM again…

Currently I don’t see it on central and your jira ticket seems to confirm this. Maybe the sonatype review needs just a little more time.

(Just to make sure, the point 9. did you see? (I do not know if I can see all comments on your ticket.))

Regards,
Fancy

No, this is still missing, maybe it was too-late yesterday :wink: I’ll do this asap…

Congratulation, you are on central now! :slight_smile:

At first try, it seems to work like a charm.
(Only JOCLSample and JOCLDeviceQuery at win64 tested).

Thank you, from a maven user to another maven user! :smiley:

Regards,
Fancy

::mantanz

Hope that it works for others as well :slight_smile:

I’m still not sure how to cope with the native libs in general (and for future versions). At the moment, the JAR contains the natives for Win32, Win64, Linux64 and Mac64. When someone now kindly provides Mac32 binaries, I’d have to make another release…

Well, not necessarily: The loading mecahnism uses a „fallback“: When it does not find the matching Lib inside the JAR, it still tries to load it from the java.library.path, so it would still be possible to also use the Mac32 binaries when they are provided as an external file.

But maybe this is an acceptable „compromise“: I’ll publish possible updates on the website, wait a while until contributors provide the most important libs which I can not build on my own, and when they are available, they are packed into a JAR and put into the Maven Repo…

Thanks again to all for your support and feedback!

The maven think for this will be snapshots. If your user wants the newest, he can add something like this to his settings.xml:

      <profiles>
            <profile>

            <activation>
                    <activeByDefault>true</activeByDefault>
            </activation>

                  <repositories>
                    <repository>
                            <snapshots>
                                <enabled>true</enabled>
                                <!-- for the heavy user (default:daily):  <updatePolicy>always</updatePolicy>  -->
                            </snapshots>
                            <id>sonatype</id>
                            <url>https://oss.sonatype.org/content/groups/public/</url>
                    </repository>
                  </repositories>

            </profile>
      </profiles>

Then the snapshots are directly visible via sonatype. To fetch the newest 0.1.5, he need in his pom:

<groupId>org.jocl</groupId>
<artifactId>jocl</artifactId>
<version>0.1.5-SNAPSHOT</version>

(At the moment you don’t have deployed a 0.1.5 snapshot, but your pom suggest this will be the next ;))

If you want, you can deploy every 5 minutes a new snapshot, without make a new release. And if you think you are final, you can „simply“ use the release plugin to make a release and deploy to central :wink:

And then, the same begins with 0.1.6-SNAPSHOT :wink:

Regards,
Fancy

jocl 0.1.4c still seems not to be on maven central, could you please check what is missing?

Hm - maybe I missed something there, I thought it should be http://repo1.maven.org/maven2/org/jocl/jocl/0.1.4c/ (or http://repo1.maven.org/maven2/org/jocl/jocl/0.1.4d/ for the latest version, respectively…) …?