When the Job Tracker goes down, Hue cannot display the Jobs in File Browser or submit to the correct cluster.
In MR1, Hadoop can support two Job Trackers, a master Job Tracker that can fail over to a standby Job Tracker and hence provide Job Tracker HA. Let’s see how Hue 3.5 and CDH5beta1 (and probably CDH4.6) can take advantage of this.
Note: in MR1 Hue is using a plugin to communicate with the Job Tracker. This can be configured in CDH or Hadoop 0.23 / 1.2.0 (MAPREDUCE-461).
We configure two Job Trackers in the hue.ini:
[hadoop]
...
[[mapred_clusters]]
[[[default]]]
\# Enter the host on which you are running the Hadoop JobTracker
jobtracker_host=host-1
\# Whether to submit jobs to this cluster
submit_to=True
[[[ha-standby]]]
\# Enter the host on which you are running the Hadoop JobTracker
jobtracker_host=host-2
\# Whether to submit jobs to this cluster
submit_to=True
And that’s it! Hue will communicate with the available Job Tracker automatically!
Notice that in the case of Oozie jobs, Oozie will try to re-submit the job but will need a logical name (HUE-1631). To enable this in Hue, specify it in each MapReduce cluster, e.g.:
[hadoop]
[[mapred_clusters]]
[[[default]]]
\# JobTracker logical name.
\## logical_name=MY_NAME
As usual feel free to comment on the hue-user list or@gethue!