Joe Williams home
Over at they found a link on DanT's Sun blog that has a sweet tutorial on setting up Hadoop using SGE's parallel environments with loose integration.
Here we are relying on master node to start othe daemons ( [rs]sh the machine and start daemons) and distribute jobs , and we donot have control on the TaskTracker threads. This way of setting a pe in Grid Engine is called loose-integration With some more effort one could also achieve a tighter integration wherein the task of starting daemons and tasks on other slaves could be done by SGE. But this would require further understanding of Hadoop internals.
Pretty dope.
Fork me on GitHub