10 January 2021

SQL Querying a log stream and outputting Calculations to another stream

Initially published on https://medium.com/data-querying/live-sql-querying-live-logs-and-sending-live-updates-easily-e6297150cf92 Log analysis tutorial from an Apache Kafka data stream via Flink SQL, ksqlDB & Hue Editor. Real time queries on streams on data is a modern way to perform powerful analyses as demoed in the previous post. This time we will see a more personalized scenario by querying our own logs generated in the Web Query Editor. First, thank you to the community for all the improvements on the open source projects mentioned below, with in particular Flink Version 1.…

3 minutes read - Tutorial / Version 4.9 / Flink SQL / ksqlDB

31 December 2020

A Spark SQL Editor via Hue and the Spark SQL Server

Write and Execute some Spark SQL quickly in your own Web Editor. Initially published on https://medium.com/data-querying/a-sparksql-editor-via-hue-and-the-spark-sql-server-f82e72bbdfc7 Apache Spark is popular for wrangling/preparing data, especially when embedding some SQL snippets to keep the data manipulation programs declarative and simpler. One good news is that the SQL syntax is very similar to Apache Hive so the very powerful Hive autocomplete of Hue works very well. Here we will describe how to integrate with the Spark SQL Thrift Server interface that might be already available in your stack.…

3 minutes read - Tutorial / Version 4.9 / Spark SQL

17 November 2020

Easy Querying of live Kafka data in a Big Table like HBase with Phoenix SQL

Phoenix brings SQL to HBase and let you query Kafka data streams Initially published on https://medium.com/data-querying/phoenix-brings-sql-to-hbase-and-let-you-query-kafka-data-streams-8fd2edda1401 Apache HBase is a massive key value database in the Big Table family. It excels in random read/write and is distributed. The Hue Query Assistant is a versatile SQL compose Web application with a goal of making database querying easy and ubiquitous within organizations. In this post we will demo the recent integration of Apache Phoenix which provides a SQL interfacing layer to HBase, hence making it easy to query.…

4 minutes read - Tutorial / Version 4.9 / Phoenix

20 October 2020

Tutorial on querying live streams of data with Flink SQL

Real time data querying is becoming a modern standard. Who wants to wait until the next day or week when needing to take decision now? Apache Flink SQL is an engine now offering SQL on bounded/unbounded streams of data. The streams can come from various sources and here we picked the popular Apache Kafka. This tutorial is based on the great Flink SQL demo Building an End-to-End Streaming Application but focuses on the end user querying experience.…

4 minutes read - Tutorial / Version 4.9 / Flink SQL

20 October 2020

Tutorial on querying live streams of data with ksql (Kafka SQL)

Real time data querying is becoming a modern standard. Who wants to wait until the next day or week when needing to take decision now? The stream of data comes from an Apache Kafka topic which can be queried via ksqlDB. Components To keep things simple, all the pieces have been put together in a “one-click” Docker Compose project which contains: ksqlDB from the ksqlDB quickstart A Hue Editor already configured with the ksqlDB Editor One-line setup For fetching the configurations and starting everything:…

3 minutes read - Tutorial / Version 4.9 / ksqlDB

10 January 2018

Self Service BI: doing a Customer 360 by querying and joining Salesforce, Marketing and log datasets

In this demo we use the Editor to query credit card transaction data that is saved in an object store in the cloud (here S3) and in a Kudu table. The demos leverages the Data Catalog search and tagging as well as the Query Assistant. Note: Do it Yourself! The queries and data are freely available on demo.gethue.com. Scenario: Digital Services International You recently launched a new streaming service: VP wants to understand support impact of this launch Marketing wants to use this to better target campaigns Goal: Build a 360-degree view of your customers to understand the support costs, product usage, time-to-resolution, and current activity in marketing channels…

1 minute read - Version 4 / Tutorial

24 August 2017

Importing data from traditional databases into HDFS/Hive in just a few clicks

There are exciting new features coming in Hue 4.1 and later in CDH 6 next year. One of which is Hue’s brand new tool to import data from relational databases to HDFS file or Hive table using Apache Sqoop 1. It enables us to bring large amount of data into the cluster in just few clicks via interactive UI. This Sqoop connector was added to the existing import data wizard of Hue.…

5 minutes read - Version 4 / Tutorial

22 August 2016

Easy indexing of data into Solr with ETL operations

Creating Solr Collections from Data files in a few clicks There are exciting new features coming in Hue 3.11 week and later in CDH 5.9 this Fall. One of which is Hue’s brand new tool to create Apache Solr Collections from file data. Hue’s Solr dashboards are great for visualizing and learning more about your data so being able to easily load data into Solr collections can be really useful.…

7 minutes read - Tutorial

07 July 2015

Bay Area BikeShare Data Analysis with Search and Spark Notebook

In this tutorial, we use public data from Bay Area BikeShare and visualize bike trips patterns and their users to understand more the usage of the platform. Hue provides a Dynamic Search dashboard as well as the new Spark Notebook for enriching the data. We recommend to start with the Trip dataset from http://www.bayareabikeshare.com/datachallenge and index it into Solr. For impatient people, we provide a subset of trips ready to be indexed as well as the weather data to be processed later with Spark.…

1 minute read - Tutorial

21 May 2015

Build a Real Time Analytic dashboard with Solr Search and Spark Streaming

Search is a great way to interactively explore your data. The Search App is continuously improving and now comes with a better support for real time! In this video, we are collecting tweets with Spark Streaming and directly indexing them into Solr with the Spark Solr app. Note that we are using a slightly modified version that adds more tweet information.  You can see the tweets rolling in!…

2 minutes read - Tutorial