-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy path11_spark_sql.scala
28 lines (21 loc) · 1.61 KB
/
11_spark_sql.scala
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
"""#######################################################################################################################
Acessando prompt de comando do Spark SQL
#######################################################################################################################"""
/*
Você pode acessar o prompt de comando do Spark SQL por meio do comando
spark-sql
Uma vez dentro do spark-sql, você tem praticamente todas as principais operações de um banco de dados relacional,
como criação de banco de dados, tabelas, views, etc. Não veremos estes comandos neste curso, pois seria necessário um curso
específico este módulo do Spark.
Outra informação importante é que você pode conectar outras ferramentas (por exemplo Tableau) no seu banco de dados/tabelas
do spark-sql por meio de uma conexão Thrift JDBC
*/
"""#######################################################################################################################
Carregando os dados
#######################################################################################################################"""
val dados = spark.read.option("header","true").option("inferSchema","true").option("delimiter",";").format("csv").load("data/bank-additional-full.csv")
"""#######################################################################################################################
Registrando a tabela e executando consultas SQL
#######################################################################################################################"""
dados.createOrReplaceTempView("dados")
val teste = spark.sql("select * from dados limit 10")