There is shift in the tech community towards grids. Oracle came out with its 10G database, now Sun has released the Sun Grid, and is offering $1 per CPU hour to run apps on their grid. At the moment, grids are still a novelty. Berkeley introduced the grid to masses through the search for aliens (SETI). Their system, BOINC, is a very nice and simple grid, but is very University geared. There are a number of vendors and a couple of enterprise strength open-source solutions.
Vendor Tools
http://www.entropia.com
http://www.datasynapse.com/
http://www.avaki.com
http://www.platform.com/
http://www.ud.com
http://www.gridsystems.com/
Open-Source
http://www.zetagrid.net
http://www.globus.org
http://www.cs.wisc.edu/condor/
http://gridengine.sunsource.net/
http://boinc.berkeley.edu/
http://www.globus.org/cog/java/
The other interesting development is that people are talking about Grids in the same breath as web-services. Personally, I am not a big fan of web-services, and has always considered web-services a solution to a very specific and narrow problem space. A grid can be thought of as a single computer with a whole lot of CPU's. I guess running apps on the grid is really just a service, and web-services, in theory, is attempting to standardize that communication. A service oriented world is interesting, but I am not convinced the world works that way.
On top of all this bable, I am proposing to build a PSG (Pretty Simple Grid) as an open-source project. The grid, unlike all other grids, will be geared for small teams whether in large organizations or not. The grid must be simple to install and simple to use. It's features will be limited to what is required and what doesn't require much setup. It will be written in java; client download-able via internet; the client will run as service, screen-saver, etc... The goal is to allow individual technology teams to leverage the grid in their offices. Most of us working for the Man probably work in pretty small teams, but are a part of a large entity that is impossible to convince to install a large grid. A small grid used solely in the small team with their 10 pc, is simple to install and provides a lot of value.
Showing posts with label distributed computing. Show all posts
Showing posts with label distributed computing. Show all posts
Friday, December 30, 2005
Friday, October 21, 2005
Monday, July 11, 2005
Distributed Groove
I spent my day troubleshooting JGroups. For those unfamiliar, JGroups is a open source middleware system. The problem was that sometimes when a client system is restarted, the client would fail to see the cluster and instead become a coordinator of its own cluster. After a bit of research, the problem ended up being connected with long garbage collection delays on the real coordinator. This caused the server to not respond to heart beat requests, which in turn caused the client to think that it was alone and therefore become its own coordinator in its own cluster.
To solve the problem, I started looking through the JGroups source code and I stumbled upon a reference to the Lamport timestamp algorithm. I remember studing the algorithm in school. The basic premise is an ability to understand logical order of messages in a distributed environment based on a concept of time. His paper goes into quite a bit of detail, some of it awfully trivial, and other awfully complicated. This brought me to his website, where I discovered a stock pile of research papers.
He covers a lot of very interesting concepts, most of it some what over my heard. The bizantine systems are very interesting: the ability to write systems that can react to any type of error. He also goes in to describe a truly parallel garbage collection algorithm, something that would be quite nice in Java.
It's very interesting. Most of the code being written these days, myself included, is written to get it out quickly, and cheaply. The code does what it supposed to do, but is by no means very efficient or bullet proof. It does the work, but is at best a temporary solution. Definately not elegant. Then I get to see all these research papers dealing explicitely with the elegance of programming. It is a nice feeling: raw computer science.
To solve the problem, I started looking through the JGroups source code and I stumbled upon a reference to the Lamport timestamp algorithm. I remember studing the algorithm in school. The basic premise is an ability to understand logical order of messages in a distributed environment based on a concept of time. His paper goes into quite a bit of detail, some of it awfully trivial, and other awfully complicated. This brought me to his website, where I discovered a stock pile of research papers.
He covers a lot of very interesting concepts, most of it some what over my heard. The bizantine systems are very interesting: the ability to write systems that can react to any type of error. He also goes in to describe a truly parallel garbage collection algorithm, something that would be quite nice in Java.
It's very interesting. Most of the code being written these days, myself included, is written to get it out quickly, and cheaply. The code does what it supposed to do, but is by no means very efficient or bullet proof. It does the work, but is at best a temporary solution. Definately not elegant. Then I get to see all these research papers dealing explicitely with the elegance of programming. It is a nice feeling: raw computer science.
Subscribe to:
Posts (Atom)