Sonatype Blog

Yesterday, in Central Maven Repository Traffic: Investigation and Analysis, I wrote about the analysis involved in tracking down the increasing load on Central. By identifiying some misbehaving tools, we were able to reduce the traffic from a 98 Mbps average down to 60-80 Mbps.  In this post, I discuss the next step toward a Central Maven repository that can scale to meet the load generated by the millions of developers using an ecosystem of tools which rely on the Central Maven Repository. Read more…

Community, Maven, Nexus central, Maven, Nexus, repository

For several months starting in August, the use of the Maven Central Repository has seen a dramatic increase in traffic.  This is a great indicator of the adoption of Maven and other tools that recognize the value of a shared binary repository. It also set in motion a series of upgrades and changes required to keep the system going strong for everyone.   We’re written about some of the steps we’ve taken in the last few months to preserve Maven Central as a public resource, and I wanted to provide more details for those of you who are interested in the numbers behind these changes.

The first change occurred in August, when we moved Central to a new 100 Mbps connection. This temporarily solved most of our availability problems, but the load continued to increase and Central started running out of httpd worker threads. For a while we played a cat and mouse game of bumping up the workers.  Read on to find out how we ultimately solved this problem and made Central more stable and available for the world of developers it serves.

Read more…

Community, Maven central, Maven, Nexus, repository

The Nexus 1.2.0.2 release is available for download. This release improves upon the architectural enhancements started in 1.1 for plugins.  1.2.0 also provides for greater manageability and customization of the embedded Jetty server, support for incremenetal Nexus indexes, and support for plugins.  To download this 1.2.0 release go to http://nexus.sonatype.org.  Here is an quick overview of some of the more interesting changes in this release.

Read more…

Maven, News, Nexus Maven, Nexus, nexus index, plugins, release

This is a visual follow on to my previous post about the Maven version shuffling. As they say, a picture is worth a thousand words:

Need I say more?

Maven

The Nexus team has been hard at work producing a new plugin model for Nexus that allows us to create some new and exciting Professional features on top of the OSS core. The first release of Nexus Pro will contain 4 new plugins:

Staging and Promotion

Staging and promotion is designed to allow projects to be deployed to independent, temporary repositories. The contents of this repository is then evaluated and eventually either promoted or deleted. When promoted, it is merged in with a target repository and all appropriate metadata is updated. This implementation of Staging and Promotion requires no changes on the Maven side other than updating your distributionManagementUrl. Read more…

Maven, News, Nexus

We would like to announce the release of Nexus 1.1. This release was primarily focused on architecture improvements in the form of ExtJs and Restlet framework upgrades as well as improvements aimed at allowing the introduction of plugins. It also marks the first release of Nexus that is available as a War.

The War has been successfully tested on Resin, Tomcat and Jetty but is considered Beta. It is currently known to not work on Glassfish due to that container’s conflict with Log4j. You can find more information regarding the war support at Nexus-58 and Nexus-1055. Please report any issues to the nexus-user list or Jira.

Also included are several enhancements aimed at improving the user experience:

Index Browsing

Previously the browse repository screens only included items already cached by Nexus. The new screen allows you to browse the tree view for a remote repository when the remote index has been downloaded:

Log Viewing

Tail functionality has been added to the log screen:

Admin can directly set a user password

It is now possible to set a password directly without requiring an SMTP server:

Many other improvements and bug fixes are included. The full list can be retrieved from Jira.

Download your copy of Nexus here.

Maven, News, Nexus

The load on Central has been increasing steadily over the past few months. Coincidence or not, it started shortly after Maven: The Definitive Guide was translated into Chinese. First, httpd was running out of connections, then as we adjusted those higher, the load on the system was getting out of hand. Most recently we were getting 15 minute load averages above 200 on a regular basis. To solve this and provide more redundancy, we started the process of installing a load balancer to share the load with a second machine. On a tip from James, we looked into Nginx “engine x”, which is supposed to be high performance http server. The central repository is currently not doing anything dynamic, the files are just sitting there on disk, updated occasionally via rsync. This means we are not leveraging the flexibility of httpd mods and the overhead was killing us.

To check it out, we installed Nginx in parallel to httpd and once everything looked good, swapped the ports to put it into production. How did it work out? Simply amazing. Since the swap, the loads are averging around .30. This is a quad core machine, so that means that it went from being 28x overloaded to using a third of ONE cpu. Being suspicious, I checked the bandwidth logs and response times. The average response time is 8x faster (no doubt because there’s 3 cpus sipping Mai Tais) and the total throughput shows no discernable change. So we’re now serving the same bandwidth that was swamping the cpus under httpd with next to no cpu with Nginx.

During the ongoing process of tracking the load on Central, we found a surprising number of large organizations out there with a very high connection usage. Please, if you’re using Maven in an enterprise, do us both a favor and install a repository manager such as Nexus. The entire community will benefit because the Central repo will be able to serve the needed artifacts in a timely fashion. You will benefit because it will give you centralized control over your artifacts, less external bandwidth use, less dependency on an external connection, indexing, searching and storage of your internal artifacts, and a ton of other functionality.

Maven

One of the most requested features of Nexus has been to bundle it as a war. Well, i just finished testing 1.1 as a war in Jetty and Tomcat and it looks good. So stay tuned, the 1.1 release is just around the corner and the war bundle will be included.

News, Nexus

It’s been a long time coming, but “Maven: The Definitive Guide” has finally hit the shelves. This was a collaborative effort with many authors and editors involved. I’ll admit that in reviewing the book, I learned a thing or two, even though I’ve been a maven committer for 3 years and on the PMC for 1.5.

The book is comprised of two parts, the first is a learn by example step by step with sample projects, and the second is more in depth discussion of advanced topics like profiles, repository managers, eclipse ide integration, lifecycle, etc.

Since each chapter was written by experts on the topic and reviewed by the whole team, you won’t find a better resource on Maven anytime soon, so grab your copy here

Maven, News

Now you can use the Nexus UI on the Iphone. Perfect for administering the system at the beach, or if you just have to know the latest version of a dependency on the train.

Maven, News, Nexus