Updates on JBM 2.0 progress

Posted on 2008-08-28 08:55:00.0 by Tim Fox [ View original post ]

JBoss Messaging 2.0 is making further great progress... here's an update with what's been going on recently:

I have just completed some further changes to our remoting layer, which allows new transports to plugged in even easier.

So far JBM 2.0 has plug-ins for Netty and MINA, and I'm looking forward to creating one for XNIO soon. Users will be able to easily configure at deploy-time which transports they wish to use.

Clebert about to finish our "paging" functionality. "Paging" in JBoss Messaging, means the ability for the system to page messages between RAM and disk, so JBM can support very large queues - much larger than can fit it in memory at one time. This allows the server to run with a small memory footprint but still handle very large queues.

The difficult bit for successful paging is doing performantly. Clebert reports he can depage around 40K 1K messages per sec on his simple home server! :)

Jeff has been finishing off our management API and he's almost there. JBM 2.0 will boast an enhanced management api with JMX and POJO support, notifications, and an ability to manage over the connection.

---

Redeployment leaks fixed on Apache BeanUtils

Posted on 2008-08-20 18:23:00.0 by Clebert Suconic [ View original post ]

I'm really glad as I could use some of my spare time to help this project. Especially because I found a lot of people with redeployment problems, not only on JBAS but in several other projects. Just google it and you will see what I mean.

We have identified some hard to solve memory leaks on BeanUtils caused by very complex circular references on the Reflection metadata which is now fixed.

The use of the jvmti library from jboss-profiler (the same library we use to chase memory leaks on the JBAS testsuite) simplified the process a lot, and even better we have added some testcases preventing regressions, using the same techniques we have applied to solve our own redeployment leakages on JBAS.

You can find more information about what was done here:

BEANUTILS-291

---

Shiny new remoting layer for JBM

Posted on 2008-07-31 09:53:00.0 by Tim Fox [ View original post ]

I've just completed a refactoring of our JBoss Messaging remoting layer.

This is a thin layer that sits between the majority of the JBM core code and the actual transport provider (E.g. Apache MINA).

Most of the leg-work is done in our remoting layer - ordering of sessions, pinging (heartbeats) etc.

The core remoting code now has zero dependencies on any particular transport provider. This means we can easily plug-in different providers with the minimum of pain - e.g. Apache MINA, Trustin Lee's Netty 3, and David Lloyd's XNIO.

Integrating a new transport provider is simple. JBM provides a spi (service provider interface) set of interfaces that need to implemented for each transport provider. Once that is done, you just add the ConnectorFactory and AcceptorFactory class names to the config file and you're done.

The implementations of the spi interfaces are typically very simple, since all the generic code is in our remoting layer and doesn't change.

---

JBM 2 is getting even faster and better

Posted on 2008-07-30 18:21:00.0 by Clebert Suconic [ View original post ]

When we released JBM 2 Alpha1, we released some performance results and we thought they were already great.

I have recently spent a lot of time working on the journal, testing and improving it, making it as reliable as possible, and because of one of the improvements we achieved the Journal is at least 40% faster on persistent messages than it was on JBM 2 alpha. Depending how you configured Alpha that could mean 100% improvement.

We are aways reusing files on the journal, putting them back as fast as we can. Because of the data format we are now using we don't need to refill the file, and that saved a lot of time on I/O as records were being added on the journal.

And what's great is, we still have room for improvements in performance.

JBoss Messaging 2 rocks!

---

Invoking Greek goddesses to get your test suite in order

Posted on 2008-06-23 08:30:00.0 by Tim Fox [ View original post ]

One thing we're really proud of in JBoss Messaging is our testing. Very soon our unit test suite will be giving close to 100% coverage of our code, and, what's more it runs very fast.

Having a solid unit test suite that runs like lightning is critical for fast effective refactoring of the code.

Long running test suites are a bad test smell. More than likely you've experienced this yourself (I certainly have), you're working on a project and it takes about three hours to run the test suite. So before every commit, you have to wait three hours to see if your changes have broken anything. Or even worse, the test suite takes so long that it's not practical to run it every time you commit, so developers are constantly breaking each others code and progress slows to a crawl. Sounds familiar?

This kind of situation is very expensive to fix retrospectively, and comes down to poor testing techniques from the offset, and too much reliance on integration testing. Familiar bad test smells here include having to start and stop the entire server on each test, usage of Thread.sleep, and start and restart the third party components e.g. databases, jgroups... All these times add up.

In JBM 2.0 we're making sure all our classes are thoroughly unit tested. We use mock or fake objects where appropriate, these nullifies any startup/shutdown times associated with starting any real implementations of the third party components. Then we have a smaller number of integration tests which tests how the objects interact together.

One other thing we've done here is to invoke the law of Demeter

This basically means that you're avoiding doing things like this in your code:

int id = connectionFactory.getConnection().getSession().getConsumer().getID();

Avoiding ugly stuff like the above does two things:

1) Makes your code more maintainable and simpler to refactor since each component does not have such a deep knowledge of other components

2) Make writing mocks a lot easier, since you're not constantly having to tell the mock to expect these nasty chains of invocations.

---

Playing Music, Coding and JBoss Messaging :-)

Posted on 2008-06-02 11:46:00.0 by Clebert Suconic [ View original post ]

Probably not many people know that but I'm also a musician, a Jazz and Bossa Nova piano player and if I invested just a little bit more of my time (which I don't have now :-) ) I could become a professional musician. Some of the feelings I have while playing in a band I'm also getting this week after the JBoss Messaging Alpha release:

I - It's cool to improvise: You feel great by inventing new things and you got that feeling! I have been part of writing a totally cool Asynchronous IO module and helped improving the Persistent of JBoss Messaging. I totally get that feeling here.

II - It's cool when a band (or team) member also improvise: When Jeff Mesnil was working on the new transport, Andy Taylor doing a lot of hard work or Tim Fox orchestrating or playing the whip.. it's all the same... I'm part of the band, and we are jazzing a beautiful song.

III - Applauses! Yeah... musician needs that.. and we are getting that this week.


And the main point is: JBoss Messaging Alpha it's just a rehearsal. It will be much better when we are in concert :-)

It is really cool to be part of this Messaging revolution :-)

---

JBM 2.0.0 alpha released with blistering performance

Posted on 2008-06-01 13:32:00.0 by Tim Fox [ View original post ]

JBoss messaging 2.0.0 alpha is released!

The goal of JBM 2.0 is simple and uncompromising- to bring unrivalled levels of performance to reliable messaging, and to be the fastest, best featured and most scalable open source messaging system. Period.

JBM 2.0 alpha is a technology preview of our next generation messaging system and demonstrates that we are well on the way to achieving that goal, with class-beating performance results.

We don't like to make unfalsifiable claims about our performance in our marketing literature. We prefer science to religion. So, in this release, we back up our assertions with a set of simple performance measurements comparing JBM 2.0 alpha with a selection of other open source messaging systems.

The tests are all reproducable by you, using only the tools shipped in the distribution. I think you'll agree that the facts speak for themselves.

Remember that JBM 2.0 is only an alpha release but we demonstrate results that outstrip competing production, or near production releases.

The success of JBM 2.0 comes down to a few factors:

1) An elegant simple architecture - 100% POJO based, allowing embeddable messaging in any product. The JMS API is provided as a thin optional facade on the client side.

2) An ultra high performance intelligent journal which can automatically flip into using pure Java NIO, or Linux asynchronous IO (via a thin native layer) depending on whether AIO is available on the system. This gives us astonishing persist rates even on commodity hardware. (See performance results). The journal is written in 95% Java, deliberately keeping the native layer to a minimum, making it far simpler to manage and port to other operating systems in the future.

3) A high performance new transport based on Apache MINA.

And not forgetting that none of this would be possible without our team. We have a small, but highly experienced, highly motivated team who have worked their butts off all hours of the day and night for as long as I can remember.

Many thanks to the "dream team", in no particular order, Andy Taylor, Clebert Suconic and Jeff Mesnil. Also thanks to Trustin Lee for working closely with us to build our new transport.

The release is available here:

Enjoy...

---

New Linux asynchronous IO layer for JBM 2.0

Posted on 2008-05-20 06:40:00.0 by Tim Fox [ View original post ]

Clebert has just posted a blog on the new asynchronous IO layer he has written for JBoss Messaging 2.0.

This is very cool technology.

---

Speeding UP IO with Asynchronous IO

Posted on 2008-05-14 19:40:00.0 by Clebert Suconic [ View original post ]

Have you ever wondered how to make IO faster?

We have this dilemma on the messaging world where we need to provide a framework where users are able to send thousands messages per second and we have found our way on doing it, and we are quite excited about it.

In our tests we have been able to send 25,000 persistent messages per second being very close to the the limits of the disk not consuming a lot of CPU on this process, on a computer with 2 QuadCores and a SAS/SCSI disk which is a very modest workstation. And about 10,000 messages/sec on my crappy Laptop because of my slow disk.

We have implemented a very generic JNI layer in top of AIO where you send a Java Callback Interface on every write, and you get informed when the data is on the disk.

With that we avoided requiring to send a sync on every commit we perform. We can just wait for the commit completion so a transaction commit will not affect other transactions.

This JNI layer is also using DMA (Direct Memory Access) when sending the data to the disk, so the CPU will be free doing other things while the data is being written.

At this point we are focusing on making this libaio layer very performatic and stable on JBoss Messaging and depending on the demand we might do it as a separate project very easily since it is ready.

Consider this blog just a teaser. I can't wait the release of JBoss Messaging Alpha showing these numbers so everyone will be able to produce these numbers, and we are almost there. It's amazing to be part of this!

If you want a very fast and reliable Java Messaging System, JBoss Messaging 2 will be your only choice :-)

---

Get the latest news on JBoss Messaging 2.0 at the Munich JBUG

Posted on 2008-04-03 08:40:00.0 by Tim Fox [ View original post ]

As well as the London JBUG next week, I'll also be talking at the next Munich JBUG in Germany on the 2nd June.

Anyone who wants to hear about the exciting developments with JBM 2.0, it would be great to see you there :)

More details here

---