Spark SQL Hudi配置与使用指南：核心参数及Thrift Server启动方法.pdfVIP

配置参数

参数:

•–conf‘spark.serializerorg.apache.spark.serializer.KryoSerializer’

•–conf‘spark.sql.extensionsorg.apache.spark.sql.hudi.HoodieSparkSessionExtension’

使用

三种方式使用HudiSparkSQL

SparkThriftServer

启动hudi‑spark‑thrift‑server

spark‑submit‑‑masteryarn‑‑deploy‑modeclient‑‑executor‑memory2G‑‑

num‑executors3‑‑executor‑cores2‑‑driver‑memory2G‑‑driver‑cores2‑‑class

org.apache.spark.sql.hive.thriftserver.HiveThriftServer2‑‑name

hudi‑spark‑thrift‑server‑‑confspark.serializer=

org.apache.spark.serializer.KryoSerializer‑‑confspark.sql.extensions=

org.apache.spark.sql.hudi