Skip to main content
Version: 3.18
Features and pricing

ScalarDB Analytics Configurations

This page provides a comprehensive reference for configuring all components of ScalarDB Analytics.

Overview​

ScalarDB Analytics consists of three main components that require configuration:

  1. ScalarDB Analytics server - The server that hosts the catalog information and metering services
  2. CLI client - The command-line interface for managing catalogs and data sources
  3. Spark integration - Configuration for using ScalarDB Analytics with Apache Spark

ScalarDB Analytics server configuration​

The server is configured using a standard Java properties file (for example, scalardb-analytics-server.properties) that defines database connections, network settings, licensing, and optional features.

Metadata database configurations​

Configure the metadata database that stores catalog information.

db.contact_points​

  • Field: scalar.db.analytics.server.db.contact_points
  • Description: The JDBC URL for the metadata database used by ScalarDB Analytics.

db.username​

  • Field: scalar.db.analytics.server.db.username
  • Description: The username for connecting to the metadata database.

db.password​

  • Field: scalar.db.analytics.server.db.password
  • Description: The password for the metadata database user.

Server network configuration​

Configure network settings including service ports and TLS/SSL encryption.

catalog.port​

  • Field: scalar.db.analytics.server.catalog.port
  • Description: Port for the catalog service.
  • Default value: 11051

metering.port​

  • Field: scalar.db.analytics.server.metering.port
  • Description: Port for the metering service.
  • Default value: 11052

tls.enabled​

  • Field: scalar.db.analytics.server.tls.enabled
  • Description: Enable TLS/SSL for secure communication.
  • Default value: false

tls.cert_chain_path​

  • Field: scalar.db.analytics.server.tls.cert_chain_path
  • Description: Path to the server certificate chain file. Required when tls.enabled is true.

tls.private_key_path​

  • Field: scalar.db.analytics.server.tls.private_key_path
  • Description: Path to the server private key file. Required when tls.enabled is true.

License configuration​

Configure your ScalarDB Analytics license.

licensing.license_key​

  • Field: scalar.db.analytics.server.licensing.license_key
  • Description: Your ScalarDB Analytics license key.

licensing.license_check_cert_pem​

  • Field: scalar.db.analytics.server.licensing.license_check_cert_pem
  • Description: License verification certificate as PEM string. Either this or license_check_cert_path must be specified.

licensing.license_check_cert_path​

  • Field: scalar.db.analytics.server.licensing.license_check_cert_path
  • Description: Path to license verification certificate file. Either this or license_check_cert_pem must be specified.

Authentication and authorization configurations​

Configure authentication and authorization for the ScalarDB Analytics server. For more details, see Authenticate and Authorize Users in ScalarDB Analytics.

auth.enabled​

  • Field: scalar.db.analytics.server.auth.enabled
  • Description: Whether authentication and authorization are enabled.
  • Default value: false

auth.initial_admin_username​

  • Field: scalar.db.analytics.server.auth.initial_admin_username
  • Description: The username of the initial admin user who will be assigned the SUPERADMIN role when no users have already been assigned to the SUPERADMIN role.

auth.password.backend​

  • Field: scalar.db.analytics.server.auth.password.backend
  • Description: The authentication backend to use (internal or scalardb-cluster).
  • Default value: internal

auth.password.token_ttl_seconds​

  • Field: scalar.db.analytics.server.auth.password.token_ttl_seconds
  • Description: The time-to-live (TTL) of access tokens in seconds.
  • Default value: 86400 (24 hours)

auth.password.internal.initial_admin_password​

  • Field: scalar.db.analytics.server.auth.password.internal.initial_admin_password
  • Description: The initial password for the admin user. Only used when auth.password.backend is internal.

auth.password.scalardb_cluster.host​

  • Field: scalar.db.analytics.server.auth.password.scalardb_cluster.host
  • Description: The hostname or IP address of the ScalarDB Cluster instance. Required when auth.password.backend is scalardb-cluster.
  • Default value: localhost

auth.password.scalardb_cluster.port​

  • Field: scalar.db.analytics.server.auth.password.scalardb_cluster.port
  • Description: The port number of the ScalarDB Cluster instance. Required when auth.password.backend is scalardb-cluster.
  • Default value: 60053

auth.password.scalardb_cluster.deadline_millis​

  • Field: scalar.db.analytics.server.auth.password.scalardb_cluster.deadline_millis
  • Description: The gRPC deadline in milliseconds for authentication requests to ScalarDB Cluster.
  • Default value: 5000

auth.password.scalardb_cluster.acl_delegation​

  • Field: scalar.db.analytics.server.auth.password.scalardb_cluster.acl_delegation
  • Description: Whether to delegate namespace-level and table-level authorization for ScalarDB data sources to ScalarDB Cluster. For details, see Authenticate and Authorize Users in ScalarDB Analytics.
  • Default value: false

auth.password.scalardb_cluster.tls.enabled​

  • Field: scalar.db.analytics.server.auth.password.scalardb_cluster.tls.enabled
  • Description: Whether to enable TLS for the connection to ScalarDB Cluster.
  • Default value: false

auth.password.scalardb_cluster.tls.ca_root_cert_path​

  • Field: scalar.db.analytics.server.auth.password.scalardb_cluster.tls.ca_root_cert_path
  • Description: Path to the CA root certificate file for verifying the ScalarDB Cluster server certificate.

auth.password.scalardb_cluster.tls.override_authority​

  • Field: scalar.db.analytics.server.auth.password.scalardb_cluster.tls.override_authority
  • Description: Override the server authority for TLS verification of the ScalarDB Cluster connection.

Metering storage configuration​

Configure storage for metering data.

metering.storage.provider​

  • Field: scalar.db.analytics.server.metering.storage.provider
  • Description: Storage provider for metering data (filesystem, aws-s3, azureblob, google-cloud-storage).

metering.storage.containerName​

  • Field: scalar.db.analytics.server.metering.storage.containerName
  • Description: Container/bucket name for cloud storage.
  • Default value: metering

metering.storage.path​

  • Field: scalar.db.analytics.server.metering.storage.path
  • Description: Local directory path. Required when provider is filesystem.

metering.storage.accessKeyId​

  • Field: scalar.db.analytics.server.metering.storage.accessKeyId
  • Description: Access key ID for cloud storage providers. Required for aws-s3, azureblob, and google-cloud-storage.

metering.storage.secretAccessKey​

  • Field: scalar.db.analytics.server.metering.storage.secretAccessKey
  • Description: Secret access key for cloud storage providers. Required for aws-s3, azureblob, and google-cloud-storage.

metering.storage.prefix​

  • Field: scalar.db.analytics.server.metering.storage.prefix
  • Description: Optional prefix for all storage paths.

CLI client configuration​

The CLI client requires connection settings to communicate with the ScalarDB Analytics server using a Java properties file (for example, client.properties).

Configuration properties​

This section describes the configuration properties.

Server connection configuration​

The following is a list of configurations for connecting to the server.

server.host​

  • Field: scalar.db.analytics.client.server.host
  • Description: Hostname or IP address of the ScalarDB Analytics server.

server.catalog.port​

  • Field: scalar.db.analytics.client.server.catalog.port
  • Description: Port number for the catalog service.
  • Default value: 11051

server.metering.port​

  • Field: scalar.db.analytics.client.server.metering.port
  • Description: Port number for the metering service.
  • Default value: 11052

Authentication configuration​

The following is a list of configurations for authentication. For more details, see Authenticate and Authorize Users in ScalarDB Analytics.

auth.username​

  • Field: scalar.db.analytics.client.auth.username
  • Description: The username for authenticating with the ScalarDB Analytics server.

auth.password​

  • Field: scalar.db.analytics.client.auth.password
  • Description: The password for authenticating with the ScalarDB Analytics server.
tip

You can also set the credentials by using environment variables, which take precedence over the configuration file properties.

Environment variableDescription
SCALAR_DB_ANALYTICS_CLIENT_AUTH_USERNAMEThe username for authentication.
SCALAR_DB_ANALYTICS_CLIENT_AUTH_PASSWORDThe password for authentication.

TLS configuration​

The following is a list of configurations for TLS.

server.tls.enabled​

  • Field: scalar.db.analytics.client.server.tls.enabled
  • Description: Enable TLS/SSL for server connections.
  • Default value: false

server.tls.ca_root_cert_path​

  • Field: scalar.db.analytics.client.server.tls.ca_root_cert_path
  • Description: Path to the CA certificate file for verifying server certificates. Required when tls.enabled is true.

server.tls.override_authority​

  • Field: scalar.db.analytics.client.server.tls.override_authority
  • Description: Override the server authority for TLS verification (useful for testing).

Spark integration configuration​

To use ScalarDB Analytics with Apache Spark, configure your Spark application by adding the necessary settings to your Spark configuration file (spark-defaults.conf).

Spark Core configuration​

The following is a list of configurations for Spark Core.

spark.jars.packages​

  • Field: spark.jars.packages
  • Description: Maven coordinates for ScalarDB Analytics dependencies.

spark.extraListeners​

  • Field: spark.extraListeners
  • Description: Register the ScalarDB Analytics metering listener.

Catalog configuration​

The following is a list of configurations for the catalog.

spark.sql.catalog.<catalog-name>​

  • Field: spark.sql.catalog.<catalog-name>
  • Description: Register the ScalarDB Analytics catalog implementation. Replace <catalog-name> with the exact name of the catalog created in ScalarDB Analytics server. Use com.scalar.db.analytics.spark.ScalarDbAnalyticsCatalog as the value.
important

The <catalog-name> must match the catalog name created in ScalarDB Analytics server using the CLI. For example, if you created a catalog named production in the server, use spark.sql.catalog.production.

Server connection configurations​

The following is a list of configurations for the server connection.

spark.sql.catalog.<catalog-name>.server.host​

  • Field: spark.sql.catalog.<catalog-name>.server.host
  • Description: Hostname or IP address of the ScalarDB Analytics server.

spark.sql.catalog.<catalog-name>.server.catalog.port​

  • Field: spark.sql.catalog.<catalog-name>.server.catalog.port
  • Description: Port number for the catalog service.
  • Default value: 11051

spark.sql.catalog.<catalog-name>.server.metering.port​

  • Field: spark.sql.catalog.<catalog-name>.server.metering.port
  • Description: Port number for the metering service.
  • Default value: 11052

TLS/SSL configurations​

The following is a list of configurations for TLS/SSL.

spark.sql.catalog.<catalog-name>.server.tls.enabled​

  • Field: spark.sql.catalog.<catalog-name>.server.tls.enabled
  • Description: Enable TLS/SSL for server connections.
  • Default value: false

spark.sql.catalog.<catalog-name>.server.tls.ca_root_cert_path​

  • Field: spark.sql.catalog.<catalog-name>.server.tls.ca_root_cert_path
  • Description: Path to the CA certificate file for verifying server certificates. Required when tls.enabled is true.

spark.sql.catalog.<catalog-name>.server.tls.override_authority​

  • Field: spark.sql.catalog.<catalog-name>.server.tls.override_authority
  • Description: Override the server authority for TLS verification.

Replace <catalog-name> with your chosen catalog name (for example, analytics).

Authentication configurations​

The following is a list of configurations for authentication when connecting to a server with authentication enabled. For more details, see Authenticate and Authorize Users in ScalarDB Analytics.

spark.sql.catalog.<catalog-name>.server.auth.username​

  • Field: spark.sql.catalog.<catalog-name>.server.auth.username
  • Description: The username for authenticating the Spark application with the ScalarDB Analytics server.

spark.sql.catalog.<catalog-name>.server.auth.password​

  • Field: spark.sql.catalog.<catalog-name>.server.auth.password
  • Description: The password for authenticating the Spark application with the ScalarDB Analytics server.

Configuration examples​

This section provides some configuration examples.

Basic development configuration​

The following are examples of configurations for the server, CLI client, and Spark.

Server configuration (scalardb-analytics-server.properties)​

# Metadata database
scalar.db.analytics.server.db.contact_points=jdbc:postgresql://localhost:5432/scalardb_analytics
scalar.db.analytics.server.db.username=dev_user
scalar.db.analytics.server.db.password=dev_password

# License
scalar.db.analytics.server.licensing.license_key=YOUR_DEV_LICENSE_KEY
scalar.db.analytics.server.licensing.license_check_cert_path=/path/to/license_cert.pem

# Metering storage (filesystem for development)
scalar.db.analytics.server.metering.storage.provider=filesystem
scalar.db.analytics.server.metering.storage.path=/tmp/scalardb-analytics-metering

CLI client configuration (client.properties)​

scalar.db.analytics.client.server.host=localhost

Spark configuration (spark-defaults.conf)​

spark.jars.packages                         com.scalar-labs:scalardb-analytics-spark-all-3.5_2.12:3.18.0
spark.extraListeners                        com.scalar.db.analytics.spark.metering.ScalarDbAnalyticsListener
spark.sql.catalog.analytics                 com.scalar.db.analytics.spark.ScalarDbAnalyticsCatalog
spark.sql.catalog.analytics.server.host     localhost

Configuration with authentication enabled​

The following are examples of configurations with authentication and authorization enabled.

Server configuration (scalardb-analytics-server.properties)​

# Metadata database
scalar.db.analytics.server.db.contact_points=jdbc:postgresql://localhost:5432/scalardb_analytics
scalar.db.analytics.server.db.username=dev_user
scalar.db.analytics.server.db.password=dev_password

# Authentication and authorization
scalar.db.analytics.server.auth.enabled=true
scalar.db.analytics.server.auth.initial_admin_username=<YOUR_ADMIN_USERNAME>
scalar.db.analytics.server.auth.password.internal.initial_admin_password=<YOUR_ADMIN_PASSWORD>

# TLS (required when authentication is enabled)
scalar.db.analytics.server.tls.enabled=true
scalar.db.analytics.server.tls.cert_chain_path=/path/to/server.crt
scalar.db.analytics.server.tls.private_key_path=/path/to/server.key

# License
scalar.db.analytics.server.licensing.license_key=YOUR_DEV_LICENSE_KEY
scalar.db.analytics.server.licensing.license_check_cert_path=/path/to/license_cert.pem

# Metering storage (filesystem for development)
scalar.db.analytics.server.metering.storage.provider=filesystem
scalar.db.analytics.server.metering.storage.path=/tmp/scalardb-analytics-metering

CLI client configuration (client.properties)​

scalar.db.analytics.client.server.host=localhost
scalar.db.analytics.client.server.tls.enabled=true
scalar.db.analytics.client.server.tls.ca_root_cert_path=/path/to/ca.crt
scalar.db.analytics.client.auth.username=<YOUR_ADMIN_USERNAME>
scalar.db.analytics.client.auth.password=<YOUR_ADMIN_PASSWORD>

Configuration with ScalarDB Cluster authentication backend​

The following is an example of a server configuration that uses ScalarDB Cluster as the authentication backend with ACL delegation enabled.

Server configuration (scalardb-analytics-server.properties)​

# Metadata database
scalar.db.analytics.server.db.contact_points=jdbc:postgresql://db.internal:5432/scalardb_analytics
scalar.db.analytics.server.db.username=analytics_user
scalar.db.analytics.server.db.password=your_secure_password

# Authentication and authorization
scalar.db.analytics.server.auth.enabled=true
scalar.db.analytics.server.auth.initial_admin_username=admin
scalar.db.analytics.server.auth.password.backend=scalardb-cluster
scalar.db.analytics.server.auth.password.scalardb_cluster.host=cluster.example.com
scalar.db.analytics.server.auth.password.scalardb_cluster.port=60053
scalar.db.analytics.server.auth.password.scalardb_cluster.acl_delegation=true

# TLS (required when authentication is enabled)
scalar.db.analytics.server.tls.enabled=true
scalar.db.analytics.server.tls.cert_chain_path=/path/to/server.crt
scalar.db.analytics.server.tls.private_key_path=/path/to/server.key

# License
scalar.db.analytics.server.licensing.license_key=YOUR_LICENSE_KEY
scalar.db.analytics.server.licensing.license_check_cert_path=/path/to/license_cert.pem

# Metering storage
scalar.db.analytics.server.metering.storage.provider=filesystem
scalar.db.analytics.server.metering.storage.path=/tmp/scalardb-analytics-metering

Production configuration with TLS​

The following are examples of configurations for TLS, CLI client, and Spark in production environments.

Server configuration (scalardb-analytics-server.properties)​

# Metadata database
scalar.db.analytics.server.db.contact_points=jdbc:postgresql://db.internal:5432/scalardb_analytics_prod
scalar.db.analytics.server.db.username=analytics_prod
scalar.db.analytics.server.db.password=your_secure_password

# gRPC ports
scalar.db.analytics.server.catalog.port=11051
scalar.db.analytics.server.metering.port=11052

# TLS
scalar.db.analytics.server.tls.enabled=true
scalar.db.analytics.server.tls.cert_chain_path=/path/to/server.crt
scalar.db.analytics.server.tls.private_key_path=/path/to/server.key

# License
scalar.db.analytics.server.licensing.license_key=YOUR_LICENSE_KEY
scalar.db.analytics.server.licensing.license_check_cert_pem=-----BEGIN CERTIFICATE-----\nMIID...certificate content...\n-----END CERTIFICATE-----

# Metering storage (S3)
scalar.db.analytics.server.metering.storage.provider=aws-s3
scalar.db.analytics.server.metering.storage.containerName=analytics-metering
scalar.db.analytics.server.metering.storage.accessKeyId=AKIAIOSFODNN7EXAMPLE
scalar.db.analytics.server.metering.storage.secretAccessKey=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
scalar.db.analytics.server.metering.storage.prefix=prod/

CLI Client configuration (client.properties)​

scalar.db.analytics.client.server.host=analytics.example.com
scalar.db.analytics.client.server.tls.enabled=true
scalar.db.analytics.client.server.tls.ca_root_cert_path=/path/to/cert.pem

Spark configuration (spark-defaults.conf)​

spark.jars.packages                                     com.scalar-labs:scalardb-analytics-spark-all-3.5_2.12:3.18.0
spark.extraListeners                                    com.scalar.db.analytics.spark.metering.ScalarDbAnalyticsListener
spark.sql.catalog.analytics                             com.scalar.db.analytics.spark.ScalarDbAnalyticsCatalog
spark.sql.catalog.analytics.server.host                 analytics.example.com
spark.sql.catalog.analytics.server.tls.enabled          true
spark.sql.catalog.analytics.server.tls.ca_root_cert_path /path/to/cert.pem

Next steps​

Contact technical support
Available only to customers with a commercial license.
Report doc issue
If you have any feedback about this page, please submit an issue.