The HBase app is an elegant way to visualize and search a lot of data. Apache HBase tables can be tricky to update as they require lower level API. Some good alternative for simplifying the data management or access is to use Apache Pig or Hive.
Access HBase with Hive
First, let’s use Beeswax to create a Hive table that is persisted as a HBase table. The script works as intended when using HiveServer2 as the Hive backend. Some HBase jar need to be registered, as shown in the video.
Here is the create table statement for creating a table that is going to store the top N coolest restaurants for everyday:
set hbase.zookeeper.quorum my-hbase.com CREATE TABLE top_cool_hbase (key string, value map<string, int>) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,review:") TBLPROPERTIES ("hbase.table.name" = "top_cool");
In order to allow Hive to use HBase some jars need to be registered (one by session). Upload them on HDFS and add them as resources in the first create table query:
/usr/lib/hive/lib/zookeeper.jar; /usr/lib/hive/lib/hbase.jar; /usr/lib/hive/lib/hive-hbase-handler-0.XX.0-cdhX.X.X.jar /usr/lib/hive/lib/guava-11.0.2.jar;
Then lets add data to our new table. We copy it from our top_cool table of the previous episode.
INSERT OVERWRITE TABLE top_cool_hbase SELECT name, map(`date`, cast(coolness as int)) FROM top_cool
If you don’t have the table from the past episode, you can still use the one from episode one as a workaround:
INSERT OVERWRITE TABLE top_cool_hbase SELECT name, map(`date`, cast(r.stars as int)) FROM review r JOIN business b ON r.business_id = b.business_id;
Access HBase with HBase Browser
As seen in the video, the HBase app provides a slick new Web interface to HBase.
Access HBase with Pig
REGISTER /usr/lib/zookeeper/zookeeper-3.4.5-cdhX.X.X.jar REGISTER /usr/lib/hbase/hbase-0.94.6-cdhX.X.X-security.jar set hbase.zookeeper.quorum 'localhost' data = LOAD 'hbase://top_cool' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage('review:*', '-loadKey true') as (name:CHARARRAY, dates:MAP); counts = FOREACH data GENERATE name, dates#'2012-12-02'; DUMP counts;
Hive and Pig are excellent tools for manipulating HBase data. All combinations are possible, the sky is the limit! For example you could load from HBase and save into Hive table with Pig or use Hive SQL to query HBase tables. You can even pull HDFS or Hive data from Pig with Hcatalog, save it into HBase (or vice versa) and browse it with HBase Browser!
Next time, let’s see how to create a search engine from the Yelp data!