How to connect SQL workbench to SparkSQL
Steps to setup sql workbench for accessing spark-sql datases:
- Start sparkSql on Namenode as:
/opt/spark/bin/spark-sql --verbose --master yarn --driver-memory 5G --executor-memory 5G --executor-cores 2 --num-executors 5
- Download SQL workbench, for macOs download from: http://www.sql-workbench.net/Workbench-Build117-MacJava7.tgz
- Extract the downloaded tgz file and launch SQLWorkbenchJ
- Copy the jar /opt/spark/lib/spark-assembly-1.2.1-hadoop2.4.0.jar [Or, equivalent as per the hadoop version] from Namnode(spark-sql server)
- On SQLWorkbench, from menu go to file-> Manage drivers.
- Click on 'Create new entry' button on top left corner.
- Provide the driver name such as spark-sql_driver.
- In Library section, select the jar (needed for jdbc driver) copied from name node in step 3 above.
- In the classname section, click on the 'Search button'.
- From the pop up window, select the driver 'org.apache.hive.jdbc.HiveDriver' and click 'Ok'
- From the menu select file-> connect window.
- Select the Driver as 'spark-sql_driver' provided in step 6 above.
- Provide the URL as, jdbc:hive2://
: / - Provide the username as 'admin' and click ok.
Comments