Hadoop search: Dynamic search dashboards with Solr

Hadoop search: Dynamic search dashboards with Solr

Hue 3.6 and a team retreat in Curacao bring the second version of the previous Search App to some higher standards. The user experience has been greatly improved as now the app provides a very easy way to build custom dashboards and visualizations.

Here is a video demoing how to interactively explore some real Apache Log data coming from the live Hue of demo.gethue.com. In just a few clicks, we look for pages with errors, what are the most used Hue apps, the top Web Browsers or inspect the user traffic on a gradient colored World map:

Update!

New widgets are available in Hue 3.7: http://gethue.com/search-app-enhancements-explore-even-more-data/

The main features are:

  • Dynamic interface updating in live
  • Drag & Drop dashboard builder
  • Text, Timeline, Pie, Line, Bar, Map, Filters, Grid and HTML widgets
  • Solr Index creation wizard from a file

 

Example of dynamic dashboard

Build your own dynamic dashboard

 

More is on the roadmap, like integration with other Hue apps like Hive/HBase, export/import of results to Hadoop, more data types to plot. A following tutorial presents how to index the Apache Log into Solr and start doing your own analytics. In the meantime, feel free to give the search dashboards a try with Hue 3.6  or the upcoming C5.1 package!

As usual, we welcome any feedback on @gethue or hue-user!

50 Comments

  1. mingxue 3 years ago

    Nice work, but Why I have to add a zookeeper for using the Solr search in Hue?

  2. Ali 2 years ago

    I am using Hue 3.7.1.

    1-I have 2 columns (say column x and column y). Using Solr search, I didn’t find a way how to plot y as a function of x by using the curve widget.
    Well, It just takes one column (either x or y) and then Hue plots it.
    2-If I use Heatmap to plot a 2 dimensions data (x and y). the y axis is bounded between 0 and 1, while my y-axis should reach up to 28984. Is there a way to set the scale of the y axis ?

  3. Hue Team 2 years ago

    #1 is a great point, tracking this feature request here: https://issues.cloudera.org/browse/HUE-2609
    #2 the y-axis should top to the sum of the counts in the biggest column. is y a bunch of unique values? if you have only one 28984 value it will count for 1 only, cf. http://demo.gethue.com/search/?collection=20000000 . Are you looking at doing a custom bucket range for each x and get the top N?

    • Ali 2 years ago

      #1 Great! Thanks.
      #2 In order to circumvent #1 I thought that Heatmap could be the solution.
      It turns out that I want to plot histograms (or lines) of a bunch of y individual values as a function of x. Like: f(3)=10000, …, f(6)=28984. It looks like it is not supported yet.

      • Hue Team 2 years ago

        Yes, heatmap is something else. You will need #1 but might notor a scattered plot (not there yet in Search).

  4. Kim 2 years ago

    Is it possible to pre-build dashboards? I see that you can share a dashboard, but I’m wondering if you can create default or pre-built dashboards without creating one manually and sharing it.

  5. Ulrich 2 years ago

    Is it possible to have autocompletion on the search fields? The only time I have autocompletion of search terms is, if I take a facet and then type in the name of the field.
    But why is there no autocompletion when I type in the field name in the fulltext search box?
    Another question is, whether it is possible to save only the query and not the whole dashboard?

  6. Dan Arias 2 years ago

    It appears I can save multiple dashboards against a single index. Each dashboard gets the name of the index. How can I give them different names to differentiate them?

    • Hue Team 2 years ago

      Go in edit mode, then just click on the name of the dashboard near the search box, and press enter and save.

  7. Josemy 2 years ago

    Hi.. I read this: http://gethue.com/how-to-build-hue-on-ubuntu-14-04-trusty/
    But i want to know if there is any way to build/”make” ONLY the app to search in Solr?
    because I have some issues with “make apps” and only need the app to search in Solr.

    Thanks..

    • Hue Team 2 years ago

      What issues are you encountering with make apps? This command also builds Hue’s core desktop libraries and modules, many of which are required by the search app. After building Hue, you can then enable only the search app by blacklisting the other apps in the hue.ini configuration.

      • Josemy 2 years ago

        Thanks for your answer! I have installed some other dependencies that I read in other forums and it worked.

  8. Andrew 1 year ago

    Is it possible to leverage solr API through this?

    • Hue Team 1 year ago

      This is entirely based on the standard Solr API

  9. Andrew 1 year ago

    Sorry if my questions come off as….not very educated on this matter, but is there a way to access it though HUE with a restful API?

  10. ben 1 year ago

    Hi,
    Its possible export tu excel the grafic?.

    Best regard

    • Hue Team 1 year ago

      No, it’s not possible, but you can export the search results to Excel and build a chart in there 🙂

  11. Aleks 1 year ago

    Hallo Hue Team,
    Which version of solr should I install for the hue 3.7.1?
    Best regards,

    • Hue Team 1 year ago

      It was mainly tested on Solr 4, but 5 and 6 should work the same

  12. Luiz 1 year ago

    I would like to create public dashboard with Solr Search, but there is no such option.
    These dashboards would be available to any user without logging in, which is quite recommended to open data.
    These users without login certainly could not edit the Dashboard, having only read access.

  13. Vivian 1 year ago

    My search dashboard misses lots of functions, such as count, group, metrics? Anybody can help for this? How could I add all these functions to the dashboard setting? Thanks.

  14. Parvesh 12 months ago

    Is there a way we can build a dashboard from multiple search indexes and apply filters that cascade down to all the search indexes.

    Thanks
    Parvesh

  15. Eugenio 12 months ago

    Hello,

    I need add calculated fields for each record returned from Solr search. It’s possible ?

    Is possible change RequestHandler/SearchHandler/ResponseWriter used by Hue ?

    Can I edit a manually a Text Facet after defined it?

    Thanks
    Eugenio

  16. tsorig 12 months ago

    Hello,

    I’m trying to add the timeline widget to the dashboard but I can’t grab it (it is greyed out). Any idea why I can’t add this widget ?

    Thanks in advance
    Best
    Laetitia

    • Hue Team 12 months ago

      It means that there’s no field in your collection set up as date/time.

  17. Johan 11 months ago

    I’m trying to build a new dashboard with 2 different indexes. It seems that this functionality it’s not possible to implement right now.

    Anyone can help me with that?

    Kind regards,
    Johan

    • Hue Team 11 months ago

      Dashboards are singled indexed currently.

      Could you explain your use case a bit more? Trying to do some {!join} ?

  18. Prateek medi 2 months ago

    Is there a way to make the Hue search components default to q.op = AND?
    Would like to allow users to filter down based on searches on multiple fields with AND logic.

    • Author
      Hue Team 2 months ago

      Currently it is not possible. First wondering if q.op could be set at the Solr config level? (so that Hue gets it for free, or propose to allow to edit it in the indexer app)

      • Prateek Medi 2 months ago

        I changed solr config to the following but did not see any change. it looks like these will be overridden by whatever is conducting the rest call to solrapi.

        Even tried defaulting one requestHandler with this feature.
        This is a pretty important requirement because we would like to implement other request handler components like clustering and “MoreLikeThis” in Hue. Any advice or tips/workarounds?

        Thanks very much!

Leave a reply

Your email address will not be published. Required fields are marked *

*