Big Data Scala by the Bay: Interactive Spark in your Browser

Published on 19 August 2015 in - 1 minute read - Last modified on 06 March 2021

Interactive Spark in your Browser - Scala by the Bay

Supporting running Spark scripts directly from a browser would bring the user experience up. Indeed, everybody has a Web navigator, the command line can be avoided, built-in graphing and visualization make it easy to explore and understand data with just a few clicks. This also simplifies the administration as now everything becomes centralized in a service and is accessible by non native clients. For this purpose, an open source Spark Job Server was developed in order to provide Scala, SQL and Python in a Web shell. The main Hadoop components of the platform are also integrated in the same interface. This talk describes the architecture of the Spark Server and its main features: # Scala, Python, SQL submissions # Impersonation # Security # Job progress / canceling # YARN / HDFS / Hive integration The server also ships with a friendly user interface built as a Hue app. We will focus on explaining how they were built, how to use the API and which lessons were learned. The final end user interaction will be live demoed.




comments powered by Disqus

More recent stories

15 November 2021
Create SQL tables from excel files
Read More
21 September 2021
Access your data in ABFS without any credential keys!
Read More
14 September 2021
SSO for REST APIs with your custom JWT authentication
Read More