Thoughts on the Tomcat's build system discussion

There is a huge thread about "Moving Tomcat to Maven" and in this post there will be thoughts from Linux packager POV (with Fedora specifics).

The Big Question - " To Ant or To Maven"  ?

It really doesn't matter from a Linux distribution POV. We build everything from source and the simple truth is that none of them is usable for us without the other. Maven dependencies are built with Ant and Ant dependencies are built with Maven. So we have both and support both.

What will be the benefits for us?

  

Maven support is getting better and better with every release and a lot more simplifications are in the work. All of these thanks to standardizations brought by Maven, to name a few there is work to automate rpm dependencies based on the Maven metadata, automation of packaging work and etc. Another possible simplification can be if mavenization is made in such a way that the implemented specifications (servlet, jsp, el ) can be build without the rest of Tomcat in a simple and obvious way. Currently in Fedora land we use Tomcat's implementation everywhere and the fact we have to build the whole Tomcat creates a number of unneeded build cycles.

What will we lose?

Usage of Maven will create additional build cycles because of the specifications jars and theirs excessive usage in Maven and friends. This will make packagers life even harder. If switch to Maven happens please consider the specification jars separation.

OSGification?

Making Tomcat's jars proper OSGi bundles is not something Geronimo specific.In the Eclipse land we use Tomcat artifacts too but they are always with added OSGi metadata. I really don't see how can this influence build system choice because we are really speaking about few tags in MANIFEST.MF. It's so easy to write them manually or if you want to automate the task there are both tools for Maven(maven-bundle-plugin) and Ant(osgi-bundle-ant-task). It would be really helpful in Tomcat devs maintain this part.

Other build systems?

Someone mentioned gradle as possible alternative or even as the best of both worlds. I guess it depends on the POV but in mine it's the worst of both worlds and there is a reason why it's not available in Fedora. Tomcat is critical enough piece of the Java infrastructure to stay on something common and proven and there isn't anything else but Ant and Maven that is widely spread.

P.S. If any Tomcat developer reads this one - feel free to ignore my thoughts but please post it to mailing list so other developers can consider it.

RIP Jakarta - you have done so much

Jakarta project is retired. We won't miss you as you still live in the form of Ant, Apache Commons, Lucene, Maven, Tomcat and numerous others.

This project(and it's contributors) deserves a lot of respect as this was the front office of open source in the Java world. While there were numerous other project spending their time on pure opensourcing of Java in the corporate Java world it was Jakarta that enlightened people.

There are so many Java developers for whom:

• ant is the first build system they have used
• tomcat is the first web container they have deployed to
• struts is the first web framework they have used after pure jsps
• log4j is the first logging tool they have used

I'm definitely one of these guys!

Hats off to Jakarta contributors - you have changed the Java world!

The 5th Pillar of Fedora

Recent thread on fedora-devel provoked this blog post.

Fedora's "Freedom, Friends, Features, First" pillars seems to be missing the most important one FUN. (Courtesy of  Pierre-Yves aka pingou )

Hopefully noone would question that the most important thing for a volunteer based project is "to be fun for the contributors". And being fun for contributors means that no contributor can give orders nor to treat another contributor to remove his access or something like this. 

This is the way down. Everyone should do whatever is fun for him and if it's not fun for him/her and doesn't consider something as important he/she is not obligated to do. If someone else consider it's important please stand up and do it and stop thinking that others are slaves having to obey orders.

Most people I know started working on FOSS because of the FUN part. 

To speak personally I haven't started because I want to give someone something for free but because there are people we can share knowledge, work together and have fun doing it.

Every effort to give people orders is ruining the 1st PILLAR of FOSS - FUN.

Please everyone stop trying to do it!!!

ShellEd 2.0.1 is out

I forgot to announce the Linux Tools 0.9 release. Please check the New and Noteworthy for details. There a number of fixes and enhancements in the RPM related tools. As part of it we changed Manpage viewer plugin's bundle id and forced me to release new ShellEd.

Latest ShellEd is restoring compatibility with Manpage viewer and fixes an outline issue with DLTK 3.x. Also you can try the new update site.
Note that this release will not work with Linux Tools releases prior to 0.9.

ShellEd 2.0.0 escaped yesterday!

After really long wait ShellEd got really tired waiting for me to release it and run out in the wild ! Now seriously - I pushed the tag by mistake while playing with maven-release-plugin and byte the bullet to finish the release after that
Read the release notes, go download it and install it. If we continue receiving bug-reports with detailed analysis and/or patches 2.0.1 will be out soon too.
Fedora users expect the update in Fedora 16, the brave ones already have it in rawhide.

The awesomeness of Eclipse Development Process (part 2)

Chris has asked for "concrete example" showing the real story behind my previous post. My example will be Tycho IP review. (Note: I'm an outsider to the whole process but I'm dealing with a lot of Maven/Eclipse/Tycho issues in Fedora).

Here is the bug with a lot of info in it. And now let's show the amount of changes that happened to Tycho codebase after this bug was open. I'm not sure that all of these changes are thanks to the project moving to Eclipse.org but I'm pretty sure that most of them are provoked by this and commit messages as this one seems to prove me right.

1. Even the first few comments are showing that with the start of the review licensing has been fixed in a number of places. This might not look important to pure Java guys but issues like that are showstoppers if one wants to use this tool for something shipped with Fedora or Debian (the two distros with biggest influence on Linux developers). Examples here, here, here, here and here.
2. No more checked binaries in the scm. Examples here and here.
3. Cleanup dependency chain. The biggest sin of almost every java project! Examples here, here, here and here.
4. Do not duplicate code/functionality. Examples here, here and here.

Some of the examples are falling in more than one group but they are mentioned in one place only, decide for yourself whether they apply for others too. Also note that these examples are taken only from git log so one may even think that this is just the top of the iceberg or that could have happened without Eclipse being involved at all. But there is evidence that at least few of them are thanks to it.

P.S. Tycho devs - keep up the good work - you have really made a huge change in the Eclipse ecosystem and I really appreciate your efforts. You are just the most recent show case for this post. :)

P.S. 2 If someone tries to say that these problems are only in certain projects I would ask him/her in advance to give us the URL of his project so others can point similar problems. 

We all do these mistakes and that's where we need help of entities like the Eclipse Foundation.

Let he who is without sin, cast the first stone

The awesomeness of Eclipse Development Process

The Hudson proposal to move to Eclipse started a big discussion about the Eclipse Development Process, IPLog/IP team and etc. A lot of people are complaining about the amount of work they have to do but having done few Eclipse Linux Tools releases I don't find this work that much, probably it hasn't costed me more than a day with non technical issues.
Now I plan to explain why is to so awesome to follow this process and respect it (note that I'm a Linux distro guy so I'm heavy biased):

• every single bit is properly licensed - One can not imagine how many times we(Fedora) have reported/asked upstreams to at least put some license to their work! And this is a blocker for inclusion in the distribution and we have spend a lot of time patching out such bits and dealing with the consequenses of these actions. +1 for the Eclipse process here

• incompatible licenses are not used/shipped together - One might think that this is just to make lawyers happy but this is only half the truth. It is also about respecting authors wish - if I choose to license my work under BSD I want people to use my work whenever they found it possible but if I choose to license under GPLv3 my primary goal would definetely not be the same and I would care most for people giving back. Most probably library developers would not gonna sue you for using incompatible licenses but this is pure DISRESPECT TO THEIR WORK. I can name a huge number of Java projects that don't even understand that. +1 for the Eclipse process too

•  reduce duplication (side effect) - One of the recent reviews of the IP team made one project remove code that was already done in one of its dependencies. Most probably just to not bother with getting it IP clean. I don't know what would readers think but this is pure technical win!

• remove unneded dependencies (side effect) - Same review as the previous point - a number of unneeded dependencies were removed just to not bother with getting it IP clean (if possible at all - read the first point). And the build was failing without these deps despite them being unused/unneded. Another pure technical win thanks to the IP team!

There are a lot of other benefits that can be mentioned but just the first two are making us - Java packagers(I'm sure I speak for at least few others) want every Java project to go through similar process. This would make packaging Java software a lot easier, a lot more acceptable and not that allien on Linux systems.

Kudos to Eclipse's IP Team 

Everyone actively working on some Java project - think whether you want your software to be included/shipped to a lot of users. You may achieve that yourself but having a team helping you in this is definetely helpfull.

P.S. Links and people/project names are intentionally not used to prevent someone getting it personal. :)

Maven and RPM - one problem less

Are you packaging Java software?

Does your package use Maven as a build system?

Have you wondered what package provides certain Maven artifact?

If you answered yes to at least one of the questions there is a good news for you:

Fedora's build system is going to automatically generate RPM Provides in a sensible way for Maven developers.

Now the details. Big thanks goes to the RPM developers for the new Dependency Generator which has made our work so much easier.

With just these 2 simple files maven.attr and maven.provwe get it to the state where every package that installs pom files according to Fedora Java Packaging Guidelineswill get provides added for every pom installed in the form mvn(groupId:artifactId)where groupId and artifactId are the same things you know from your Maven experience. Look at this recent maven2 rebuildfor example what will be generated for a package with many pom files installed.

It might look a small thing but I'm really looking for the day when I'll be able to write BuildRequires: mvn(org.apache.maven:maven-script-ant) and not having to guess whether this is part of maven package or maven-plugin-tools-ant or maven-shared-ant. And these days are near for rawhide when we get the critical mass of rebuilds done so the provides are in place.

This opens an enourmous opportunity for improving a number of things like:

* when you get a build failure in your maven build - you should be able to just write yum install mvn(groupId:artifactId) and have the package ready. Who knows we might even find someone to create a PackageKit extension in the way bash command-not-found is acting?

* tools like RPMStubby will be able to generate proper BuildRequires section without guessing how did the distribution renamed the package

* use your imagination

There is a possibility to add similar support for the requires generation but lets first do all the fixes/simplification this one gives us.

Happy packaging to all Java and Maven packagers!!!

Next generation :)

Can we retire when we see next generation picking up our work?

Rpm distros unite (at least their Java stacks)

As an Eclipse developer I'm really proud to say that Eclipse happens to be the driving force for improving the Java stack in various Linux RPM distros. Kudos to all Eclipse developers for making the platform that happens to be the single Java based application that Linux packagers are willing to spend time getting it shipped with their distro!!!

Eclipse dependencies are build using the two most popular Java build systems - Ant and Maven. And that's how the whole work on getting their latest versions packaged to work properly on Fedora. A number of other people joined the effort and we wouldn't have made it without their great help. So what we have now is Maven 3.0.3 on the upstream release date (thanks to the hard work done by Stanislav Ochotnický) - hey this is the first time EVER! Mageia and openSUSE are also working hard to get their stacks updated to the state of Fedora one.

Cross-distros collaboration happens in the best possible way - dev-to-dev directly on #fedora-java or #eclipse-linux at irc.freenode.org . Let me put a small cite to show it:

[08:33] <akurtakov> sshaw: did you fix the icu4j
[08:34] <sshaw> akurtakov: yeah
[08:34] <sshaw> well dmorgan did
[08:34] <sshaw> he finished it up for me when I fell asleep
[08:34] <sshaw> akurtakov: with + sed -i -e 's|tomcat5-jsp-2.0-api|tomcat6-jsp-2.1-api|g' dependencies.properties
[08:34] <sshaw> can I drop tomcat 5?
[08:34] <akurtakov> heh, crossdistro collaboration to the rescue
[08:34] <sshaw> akurtakov: rpm distros unite

Disclaimer:

• Fedora - akurtakov - Alexander Kurtakov
• openSUSE - sshaw - Stephen Shaw
• Mageia - dmorgan - Dexter Morgan

As I have started with Eclipse and it's role - Mageia already has Eclipse and OpenSuse is really close to getting it packaged. All of this is of course based on the Eclipse Linux Tools eclipse-build project (there is Debian and Ubuntu helping there) but the RPM distros have gone one step further - even the RPM specfile is kind of a shared one.

When Andrew Overholt pointed me into starting eclipse-build I have never thought that this project will happen to be a hub for real cross-distro collaboration bigger than it's Eclipse influence. There are a number of other nice chats/ideas for collaboration that happened thanks to the personal talks that started around eclipse-build. Thanks to Niels Thykier who told us about Debian work for creating Java webapps packaging guidelines and helper tools this was brought to Fedora Java SIG meetings for discussion. There is no fruit from this discussion yet but at least we are trying to be aware of each other's work so we(all Linux distros) can come with a single position that can influence upstream Java developers - note that they are not mandatory Linux developers so they are not aware of our problems.

Collaboration is not something that can be pushed to people top-down in the FOSS world, it depends on every single contributor to make it happen bottom-up. And it is already happening you have to just join us.

Announcement: Eclipse Linux Tools 0.7 release

I'm especially happy to announce the Eclipse Linux Tools 0.7 release.

You can download it here (Fedora packages will be pushed together with the Eclipse Helios SR2 release at the end of this month or early next one).

Read the New&Noteworthy or use the wiki to read all the documentation. For the first time all our documetation is consolidated in a single place. Everyone is more than welcome in helping us improve it - just edit the wiki page and we will fetch it for the next release.

Going to Brno this weekend

I'm heading to Brno tomorrow for DeveloperConference2011.

If someone wants to have a chat about the Java stack in Fedora, anything Eclipse related or wants to see Eclipse Fedorapackager in action and make it part of his/her workflow just drop me a mail and we will find when and where to do it.
In addition to the conference days Thursday and Sunday are fine too.

Announcement: Maven 3 lands in Fedora

Maven 3 is in Fedora for some time but no package was using it to build.

Things are changing we have the very first package using it to build in Fedora approved. Thus we consider it ready for general usage.

Everyone please try using maven instead of maven2 package.

All you need to do is change mvn-jpp to mvn-local and switching parameters having maven2 in their names to just maven e.g. -Dmaven2.jpp.depmap.file becomes -Dmaven.jpp.depmap.file.

We are looking for your bug reports or even better success stories. Let's make Fedora 15 rock for Maven users and let's hope that we will be able to kill Maven 2.x not later than Fedora 16.