Create Spark SQL DataFrame: Pass bucket name/password to DF not to Session builder

Hi @mnmami. You need to specify the bucket parameters in the SparkSession config, as in your first example. The dataframe.option("bucket","user") exists to choose between multiple buckets that were previous configured in the SparkSession config, and cannot be used to configure the bucket.

This is because opening a connection to a Couchbase bucket is a relatively expensive operation, so it’s not something you want to be doing every time you create a Dataframe.

I hope that helps :slight_smile:

1 Like