Search Hadoop in Hue

This post talks about Hue, a UI for making Apache Hadoop easier to use.

Coming in Hue 2.4, on top of more than 150 fixes a new app was added: Search. You can now search Hadoop like you do with Google or Yahoo!. In addition a wizard lets you tweak the result snippets and tailors the search experience to your needs.

Here is a video demoing the querying and the results customization. The demo is based on the Twitter Streaming data collected with Apache Flume and indexed in real time.

The new Hue Search app is adding an impressive list of features to the already enterprise grade, industry standard list of features Solr provides. More specifically, here are the most significant ones:

• Based on standard Solr and Solr Cloud
• Optimized for Cloudera Search for searching Hadoop
• Result snipped editor with live preview
• Field, range and date facets
• Sorting
• Highlighting
• Layout and functions templates
• Custom CSS/Javascript placeholders

Code from Snippet Editor

Here are the final templates used in the Hue Search app demo. They are used for customizing the look and feel of the search results. With this HTML and CSS, we inserted the index fields we wanted to display, added Twitter profile images, icons, links and changed the font and colors of the text.

HTML from the Source tab

<div class="row-fluid">
  <div class="row-fluid">
    <div class="row-fluid">
      <div class="span1">
        <img src="http://twitter.com/api/users/profile_image/{{user_screen_name}}" class="avatar" />
      </div>
      <div class="span11">
        <a href="https://twitter.com/{{user_screen_name}}/status/{{id}}" class="btn openTweet">
          <i class="icon-twitter"></i>
        </a>
        <b>{{user_name}}</b>
        <br/>
        {{text}}
        <br/>
        <div class="created">{{#fromnow}}{{created_at}}{{/fromnow}}</div>
      </div>
    </div>
    <br/>
  </div>
</div>

CSS from the advanced tab

<style>
em {
  font-weight: bold;
  background-color: yellow;
}

.avatar {
  margin: 10px;
}

.created {
  margin-top: 10px;
  color: #CCC;
}

.openTweet {
  float: right;
  margin: 10px;
}
</style>

The release for Hue 2.4 is available here.

The new Hue Search app is using the regular Solr API underneath the hood, yet adds a remarkable list of UI features that makes using search over data stored in Hadoop a breeze. It integrates with the other Hue apps like File Browser for looking at the index file in a few clicks. More advanced features are on the way like fine-grained security of indexes, multi-shard search or even saving results.

We welcome any feedback on hue-user and Solr-specific requests on search-user!