• Subscribe  |  
  • Register  |  
  • Issue Tracking  |  
  • Login   |  
  • Search:

View feed: Bryan's Blog

Navigation

Posts of this feed on this page

Recent posts from all feeds

Red Hat Enterprise MRG Presentations From the 2008 Red Hat Summit

Posted on 2008-06-23 07:09:00.0 by Bryan Che [ View original post ]

I've posted online the presentations that we just did at the 2008 Red Hat Summit about Red Hat Enterprise MRG. You can download them at:

Red Hat Enterprise MRG v1 is Released

Posted on 2008-06-19 12:10:00.0 by Bryan Che [ View original post ]

Today marks the release of version 1 of Red Hat Enterprise MRG, our high performance distributed computing platform that integrates Messaging, Realtime, and Grid technologies. Red Hat has been working across each of these technologies for years, so we're excited to be launching the initial release at the Red Hat Summit.

We've got some pretty impressive performance results, customers, partners, and use cases for MRG. For details, see:
Additionally, if you happen to be at the Red Hat Summit, we're featuring MRG pretty prominently:
  • Our CEO, Jim Whitehurst, highlighted MRG Messaging and AMQP yesterday in his keynote as an example of a customer (JPMC) contributing to open source
  • Our CTO, Brian Stevens, featured MRG in this morning's keynote
  • We have several sessions on MRG
  • We are doing MRG demos at the Red Hat booth in the Expo Hall
  • Cisco is a sponsor at the Summit and is demonstrating their AON Message Bus Interconnect (MBI) solution. Cisco is debuting support for Red Hat Enterprise MRG Messaging in their AON MBI product at the Summit and demonstrating this in the Expo Hall.
  • IBM is a sponsor at the Summit and is demonstrating their WebSphere Real Time, which is an RTSJ-compliant realtime JVM. IBM supports WebSphere Real Time exclusively on Red Hat Enterprise MRG. They have also been a strong development partner with Red Hat around realtime, and they are a winner in this year's Red Hat Innovation Awards for this work. IBM is demonstrating WebSphere Real Time in the Expo Hall.
Congratulations to the entire MRG team for this fantastic release!

Red Hat Summit 2008 (and FUDCon!)

Posted on 2008-06-16 15:46:00.0 by Bryan Che [ View original post ]

Tomorrow is the start of the 2008 Red Hat Summit in Boston. There are going to be several sessions related to Red Hat Enterprise MRG there:
  • Thursday 1:30pm: Realtime Linux: Who, What, When, Where and Why by Clark Williams. Clark is the tech lead for realtime at Red Hat, so he'll have a lot of good stuff to say about performance results, how we've developed realtime, what's happening in the open source community, what's planned for the future, and so on.
  • Thursday 4:00pm: Red Hat Enterprise MRG Overview by Carl Trieloff. Carl is the technical director and visionary behind MRG, so this will be a great opportunity to hear first-hand about the origins, successes, and benefits of MRG. Way back, I spent over a year working to get Carl into Red Hat to launch and drive our MRG initiatives. Now, after creating AMQP, starting new open source projects, bringing realtime to maturity, and signing our partnership with the University of Wisconsin around Condor, we are starting to see significant traction around MRG.
  • Friday 9:00am: Dynamic Grid Computing With Red Hat Enterprise MRG & Amazon EC2 by Bryan Che. That's me! I hope you can get up early enough to attend my session. I'll be presenting on the work we've been doing to enable dynamically provisioning grid capacity at Amazon EC2's cloud infrastructure right from your MRG Grid's Condor scheduler. This will enable enterprises to add capacity dynamically to existing data centers or even to provision entire grids on-demand in the cloud. Cloud computing is hot these days, and we are seeing a lot of customer interest in MRG's integration with EC2.
This week also marks the start of FUDCon 2008 in Boston. Matt Farrellee, who is our tech lead for Condor and MRG Grid, will be coming to town to help lead discussions on implementing Fedora Nightlife. Of course, I'll be there too.

Fedora Nightlife Article on lwn.net

Posted on 2008-06-05 18:55:00.0 by Bryan Che [ View original post ]

There's a nice, detailed article about Fedora Nightlife on lwn.net: http://lwn.net/SubscriberLink/284887/b05744ca15f41a52/.

Fedora Nightlife and Energy Usage

Posted on 2008-06-01 16:48:00.0 by Bryan Che [ View original post ]

Wow, lots of response to my blog post about Nightlife! It's great to see so much interest right at the start. There are a lot of questions, but many of the conversations around these should happen on the Fedora Nightlife mailing lists as they're not just for me to answer. Also, I've now created an initial Wiki page for Nightlife (https://fedoraproject.org/wiki/Nightlife), so a lot of information will ultimately go over there. I will, however, blog about some of the topics that have stirred more discussion.

I'll start with one of the questions that always seems to arise when people talk about harvesting idle computing capacity: energy usage. Specifically, isn't it a waste of energy to leave your computer running when you're not using it so that others can leverage it for distributed computation? This is a complex issue with a complicated answer: it sometimes is a waste of energy, but it doesn't have to be a waste and can even save energy in the long run.

Cycle harvesting is sometimes a waste of energy
Let's start with the obvious: harvesting idle computer capacity across many--perhaps millions--of computers can definitely waste energy. Computers that might otherwise have been turned off are now running at full power crunching data for projects that may not be useful. Furthermore, these computers won't all be fully utilized 100% of the time, so there will be many instances of computers running with nothing to do but waste energy. Yes, unfortunately, cycle harvesting can and often does waste energy.

Cycle harvesting doesn't have to be a waste of energy
Cycle harvesting can waste energy, but it doesn't have to do so. I hope that as we work on Nightlife, this will prove to be the case.

There are many worthwhile tasks which can only be accomplished by heavy computation. A lot of fundamental research today in biology or healthcare, for example, requires access to large computer grids. If you believe that this type of research is a worthy use of energy, then the issue of wasting energy becomes an engineering problem of utilization and efficiency. That is, there are certain tasks for which it is worthwhile to let others use your available computing power. As long as these tasks fully utilize your computer when it is idle and they do so in the most efficient manner, then they aren't really wasting energy. More concretely, what if finding a cure to cancer required a lot of computational modeling? Would it be a waste of energy if there was a good project devoted to harvesting idle capacity to find such a cure, and it did so in a way that fully utilized all the computers which were donating capacity in an efficient manner?

If the keys to preventing energy waste in cycle harvesting are utilization and efficiency for worthwhile projects, then this is a problem we can address for Nightlife in a variety of ways. For example, the Condor scheduler is highly adept at maximizing resources efficiently. Given enough tasks or projects, we should be able to use all the resources available to Nightlife efficiently. The challenges will come from finding enough good projects and work for which people can donate their computing capacity. As long as we've got a good queue of work, we should be able to ensure that all the computers donating to Nightlife are doing something worthwhile and not just sitting around.

There are also many things that we can do at Fedora to increase further our ability to utilize resources efficiently. For example, we could explore waking computers to execute tasks that the owners of those computers deem worthwhile; otherwise, those computers will be in a suspended or low/no-power mode. At Fedora, we can drive the Linux operating system to be much more efficient in how it uses power while doing computations. And, Fedora's patron, Red Hat, has strong relationships with and influence over major hardware manufacturers and customers of grids. As commercial enterprises also look at how to save energy while doing their own grid computations, we have an opportunity to lead the way in developing and demonstrating the best techniques for doing this in an earth-friendly way.

Cycle Harvesting Can Save Energy
Much of the work that would run on Fedora Nightlife is going to be computed one way or another. A project like Nightlife, however, can not only help speed these computations by providing additional processing power, it can also help save total energy usage in the long run.

If you've ever visited a large data center, then you know that the energy usage of of the individual computers in that data center is only a fraction of the total energy the data center consumes. When you put tens of thousands of computers together in a single room, then many other energy hogs come into play. Foremost is cooling--large data centers require massive amounts of redundant air conditioning systems to prevent the computers from overheating as they process in close proximity to each other. There are also many other devices that draw power: the numerous network switches and routers connecting the computers, all the devices that monitor the data center's health and security, the redundant power supplies that keep the data center operating in the event of a power failure, and so on.

If a project were to distribute its computations over many individual, geographically dispersed computers and didn't need to build out a large data center for all its work, then it would no longer have to use as large a cooling center or provide as much backup power or do any of the other energy-consuming things that putting so many computers close together requires. Instead, by distributing its work over a number of individual machines through Nightlife, a project could cut down on its total energy required per computation.

Another way that Nightlife can provide energy benefits over a dedicated data center is by avoiding a concentrated usage of power in a single geographic location. I once visited a large Internet company in a power crisis because its host city's power grid could provide it with no additional electricity to grow--the company had totally maxed out the available electricity to it. Even if Nightlife didn't save total energy usage but increased the amount of energy required per calculation (which, as I've argued above, it doesn't have to do), this could still provide a better overall environmental impact. Rather than concentrating all its energy use in one place, a project could distribute and amortize its energy impact across a much larger area by leveraging Nightlife.

You don't have to participate, but you can contribute
Finally, maybe you fundamentally believe that there is nothing worthwhile for running on a large computer grid--no matter how noble the task--because of the energy required for running a grid. That's fine--you don't have to donate capacity to Nightlife. But, pragmatically, you have to agree that people are going to compute certain things one way or another. At Fedora, we have a tremendous opportunity to improve the power usage of grid computing in general. So, even if you don't donate idle capacity to Nightlife, please consider helping Fedora as a whole become the most energy-efficient platform for computation.

Introducing Fedora Nightlife

Posted on 2008-05-28 22:47:00.0 by Bryan Che [ View original post ]

I've recently started a new project at Fedora called Nightlife. Here's the text of the e-mail I sent to the newly-created Fedora Nightlife mailing list:

Fedora Nightlife is a new project for creating a Fedora community grid. People will be able to donate idle capacity from their own computers to an open, general-purpose Fedora-run grid for processing socially beneficial work and scientific research that requires access to large amounts of computing power. Given the large number of Fedora users, I hope that we will eventually be able to build a community grid of over a million nodes at Fedora. This will be a great example of the power of the Fedora community, give people new and meaningful ways to contribute to Fedora, advance the development of large-scale grid software, and lead to real benefits for the world.

Fedora Nightlife will leverage the Condor project, which was (http://www.cs.wisc.edu/condor/) created and hosted by the University of Wisconsin Madison, for scheduling and harnessing donated computing power. Last year, Red Hat and the University of Wisconsin signed a strategic partnership around Condor. Part of this partnership entailed releasing Condor's source code under an OSI-approved open source license. As a result, we now have Condor packaged at Fedora, and upstream development continues to happen at the University of Wisconsin repository in an open manner. 

Some of the immediate next steps for Fedora Nightlife are:
-create a Wiki page for this project
-get a Condor scheduler hosted at Fedora up and running
-work out what are the requirements for a project to be able to run on Fedora Nightlife (e.g. its software must be open source, it must be safe, it should have some kind of open policy around its results, etc)

We've already started working on getting a scheduler up and running. I should have a wiki setup relatively soon so that we can start mapping out more plans there.

Then, we can focus on growing the Nightlife community of projects and solicit Fedora users to donate capacity. Hopefully, enabling donation of compute power to Nightlife can eventually become a first-boot option for Fedora installs.

I welcome everyone to contribute to and participate in the Fedora Nightlife project!
If you'd like to subscribe to the Fedora Nightlife mailing list, you can do so at https://www.redhat.com/mailman/listinfo/fedora-nightlife-list

Fedora Nightlife is going to build on the work I do in my day job at Red Hat around Red Hat Enterprise MRG--it'll be based on the same technology we use for MRG's grid capabilities. This is great for me for a couple reasons: Fedora Nightlife will be a powerful and public example of the scale that's possible with Red Hat Enterprise MRG, and the work we do to drive Nightlife/Condor to the 1 million node count will directly benefit MRG. Also, it's great to be at a place like Red Hat where I can work on my product management job and do some good in the world at the same time.

Creating a Wireless 3G Network Connection on Fedora 9 With a Novatel Ovation U727

Posted on 2008-05-17 15:33:00.0 by Bryan Che [ View original post ]

One of my favourite features of Fedora 9 is NetworkManager's new support for wireless 3G modems. I just recently got a Novatel Ovation U727 EVDO Rev A USB Modem for Sprint's network because this modem explicitly supports Linux. Sprint provides instructions for using the U727 on Linux from its Web site. But, since Fedora 9 makes this process much easier, here is how to do it:

  1. Have your laptop always load the proper drivers for the modem by adding the following lines to /etc/rc.local:
    #load driver for Sprint Novatel u727 wireless modem
    rmmod usbserial
    modprobe -v usbserial vendor=0x1410 product=0x4100
  2. Insert your U727 into your laptop.

  3. The U727 has built-in flash storage for which Fedora will mount on your machine, launch a file browser, and show a link on your desktop. Close the file browser, and right-click the link to your flash storage and select to eject the device (note that unless you eject the flash storage, your modem won't work).

  4. Right-click on NetworkManager and select Edit Connections.

  5. Click on the Add button to create a new connection for your U727.
  6. Fill in the new dialog box with the name of your modem's connection and the number to dial. In my case, I named the connection Sprint Novatel U727. Type in #777 for the number to dial.


  7. Hit OK and then close the dialog box
  8. Now, click on NetworkManager, and you should see your new USB modem connection show up as a connection option.
  9. Select your new connection, and you'll be online!
Here are instructions for using your modem in the future now that it's setup:
  1. Insert your U727 into your laptop
  2. Eject the mounted flash storage device in your modem (unless you eject the flash storage, your modem won't work)
  3. Go to NetworkManager to select your U727 connection

I'm currently writing this post while online with my Sprint U727 modem. As a side-note, I selected Sprint's 3G service for mobile broadband even though I don't use Sprint for my cell phone service because Sprint has truly unlimited data usage (Verizon and AT&T cap at 5GB/month) with good terms of service (Verizon restricts things like streaming media), and its EVDO rev A network is fairly fast. Here are the results of a speed test that I just ran:


Update 5/19/2008:

Sprint is updating its Terms of Service to cap data usage at 5GB/month too. Given that, I will likely switch from Sprint to Verizon, which I prefer for cell phone service. Verizon also sells a U727 modem, so these instructions will work for Verizon too


JBoss Developer Studio For Mac OS Now Available

Posted on 2008-03-12 08:23:00.0 by Bryan Che [ View original post ]

Hi, I'm happy to announce that JBoss Developer Studio for Mac OS is now available. So, now you can get all the benefits of fantastic certified tools and an integrated JBoss Enterprise Application Platform with native Mac support. Many JBoss developers use Macs, and we know that many in our community use Macs, so we're excited to make this available. And, of course, if you're a Windows or Linux user, JBoss Developer Studio has been available for those platforms as well.

You can get JBoss Developer Studio for Mac OS from http://www.jboss.com/products/devstudio.

Note that right now, due to an issue in Eclipse, we are only supporting JBoss Developer Studio on Mac OS Tiger 10.4.x and earlier right now. There is a workaround available for Leopard, but this isn't a supported configuration. We'll add formal Leopard support in our next update.

Enjoy!

We're not going to release Red Hat Developer Studio anymore. Introducing JBoss Developer Studio 1.0!

Posted on 2007-12-10 08:00:00.0 by Bryan Che [ View original post ]

I'm pleased to announce today the General Availability of JBoss Developer Studio 1.0 for Windows and Linux. JBoss Developer Studio provides a certified open source development environment that includes and integrates: JBoss Developer Studio provides a host of powerful features, such as Seam tools, powerful Ajax capabilities, a Visual Page Editor with WYSIWYG editing of JSF pages and RichFaces Ajax components, robust Hibernate capabilities, and much more.

One of the main benefits of using JBoss Developer Studio is that it pre-integrates and certifies tooling and runtime components together. When you use JBoss Developer Studio:
  • You don't have to worry about whether all the plugins you use will work together or require incompatible dependencies.
  • You'll have the assurance that all your runtime libraries like Hibernate or Seam are properly matched with each other and already installed into JBoss Application Server.
  • You'll know that the particular Eclipse plugins you have work precisely with the runtime libraries and containers in your development environment.
  • You can easily upgrade to new technologies because all the matching tooling, runtime components, and dependencies will be provided to you in an integrated installer.
  • You can deploy your development platform with confidence because Red Hat supports JBoss Enterprise Application Platform releases for 5 years.
In addition to Eclipse, Eclipse Tooling, and JBoss Enterprise Application Platform, JBoss Developer Studio also includes a copy of Red Hat Enterprise Linux and Red Hat Network access for development use. Even if you're a Windows-based developer (and we know that a lot of you are!), you may want to take advantage of Red Hat Enterprise Linux's built-in virtualization capabilities to run multiple Windows guests for different development and test environments.

Many of you know that we had released a couple betas under the name, Red Hat Developer Studio. During the beta process, we found that it would be advantageous to leverage the powerful JBoss brand more clearly. So, as we release this new offering, we are officially christening it JBoss Developer Studio.

JBoss Developer Studio 1.0 is available now as a subscription offering for $99. To summarize, JBoss Developer Studio includes:
  • Eclipse
  • Eclipse Tooling
  • JBoss Enterprise Application Platform
  • Red Hat Enterprise Linux
  • RHN Access
  • Support for Windows and Linux

To learn more about JBoss Developer Studio, visit: http://www.jboss.com/products/devstudio

Red Hat Enterprise MRG: Red Hat, Customer-Driven Innovation, and Open Source Leadership

Posted on 2007-12-04 08:00:00.0 by Bryan Che [ View original post ]

Red Hat has shown that open source is one of the best ways to bring customer-driven innovation and leadership to the market. Today’s announcement of Red Hat Enterprise MRG provides a perfect example of this in many respects.

Spreading the Message of Open Source and Open Standards

Red Hat Enterprise MRG includes Red Hat’s implementation of AMQP-based ( Advanced Message Queuing Protocol) enterprise messaging. Both the MRG Messaging implementation and AMQP itself highlight Red Hat’s leadership and customer-driven innovation.

Red Hat is developing its AMQP messaging implementation in various open source projects and communities. One of the most notable aspects of these communities is that there are many messaging users from financial services and other industries contributing major pieces of code. These users are working to make sure that this messaging implementation will meet their specific needs when they ultimately consume it as customers. The results of this collaboration are noteworthy: MRG Messaging provides breakthrough features and performance and can reach durable messaging throughputs two orders of magnitude higher than other solutions.

Open source developers are not alone in recognizing the value of collaborating with others. The AMQP working group, of which Red Hat is a founding member, is developing the AMQP specification to be an open, interoperable standard for messaging. This particular working group is especially effective because its membership contains not only technology companies but also many end-users of messaging technology—including several investment banking giants. Of course, all contributions in the AMQP working group are valuable, no matter who provides them. But, AMQP is developing into a broadly accepted standard in many ways because there are so many end-users working to ensure that AMQP meets their own needs. Truly, this is customer-driven innovation.

Deterministic Success

In 2005, Red Hat began working on its realtime kernel technology in response to a request by the US Navy for the DDG 1000 Zumwalt Class Destroyer project. Red Hat engineer, Ingo Molnar, developed a realtime patch set which brought highly deterministic response times to the Linux kernel. However, Red Hat did not just release a product around this work. Instead, Red Hat has been working and continues to work with the Linux community to bring this realtime technology into the upstream Linux kernel. To date, Red Hat has incorporated about two-thirds of its realtime code base upstream and is working to push the rest of this code upstream. One notable recent achievement was the acceptance of Ingo’s Completely Fair Scheduler (CFS) into the mainline kernel this summer.

Why is Red Hat working so hard to push its realtime work into the mainline kernel? By having features implemented upstream, these capabilities “carry forward” into future versions of the kernel, so MRG Realtime has the product longevity that proprietary realtime extensions do not.

Trying to support extensions to Linux that are not accepted upstream is a losing battle. Red Hat recognized this long ago and thus pursued the long task of writing realtime extensions and pushing them upstream. Sure, this is hard work. But, at the end of the day, Red Hat will be in an optimal position to support this technology for the long term, since Red Hat wrote and led the work upstream.

Broad-Scale Innovation

Red Hat Enterprise MRG’s High Throughput Computing and grid capabilities are based on the Condor project created by and hosted at the University of Wisconsin, Madison. First developed in the late 1980’s, Condor has been under continuous active research and use and possess features and capabilities that far exceed those of any commercial, proprietary grid product. However, Condor has not seen significant industry usage to date because it does not provide all the enterprise features, manageability and supportability that customers require. For example, one of the first pieces of work Red Hat performed on Condor was to break it up from one large, statically linked program into separate RPM packages that are robust, manageable, upgradeable and can be discreetly patched.

Red Hat and the University of Wisconsin have signed a unique partnership around Condor. Under this agreement, the University of Wisconsin will release Condor’s source code under an OSI-approved open source license so that Red Hat may include Condor in its open source distributions, and Red Hat will jointly fund and staff Condor development on-campus at the University of Wisconsin.

Condor has a large community of users and researchers in the academic space. Through its agreement with the University of Wisconsin, Red Hat will be able to bring this innovation from academia to the enterprise. Furthermore, Red Hat and the University of Wisconsin will work to strengthen Condor with additional features and enterprise strength and also enhance Linux for High Throughput Computing to the benefit of both scientists and enterprises. Red Hat believes that this will lead to great advances in infrastructure technology and a great partnership between industry and academia. This is the best kind of customer-driven, open source innovation of all: one that not only advances technology but improves the way we do things.

For more information on Red Hat Enterprise MRG, see here.


Red Hat.com|JBoss.com|Contact Us|Careers

Hosted on JBoss Labs. Powered by JBoss Portal, JBoss AS and Shotoku.

Experiencing a problem or have a suggestion to make jboss.org better?