We are excited to announce the release of Apache Spark Connector in two new versions:
- 3.1.0 for support with Apache Spark 3.1
- 3.0.0 for support with Apache Spark 3.0
Highlights include (see migration guide in the docs for more info):
- Using our Scala SDK underneath for first-class Scala features
- Support for Scopes and Collections
- Support for Search RDDs
- Support for SparkSQL on top of Analytics in addition to Query
- Writing DataFrames and Datasets either via Query or Key Value