ScalarDB Analytics Configurations
This page provides a comprehensive reference for configuring all components of ScalarDB Analytics.
Overview​
ScalarDB Analytics consists of three main components that require configuration:
- ScalarDB Analytics server - The server that hosts the catalog information and metering services
- CLI client - The command-line interface for managing catalogs and data sources
- Spark integration - Configuration for using ScalarDB Analytics with Apache Spark
ScalarDB Analytics server configuration​
The server is configured using a standard Java properties file (for example, scalardb-analytics-server.properties
) that defines database connections, network settings, licensing, and optional features.
Metadata database configurations​
Configure the metadata database that stores catalog information.
db.url
​
- Field:
scalar.db.analytics.server.db.url
- Description: The JDBC URL for the metadata database used by ScalarDB Analytics.
db.username
​
- Field:
scalar.db.analytics.server.db.username
- Description: The username for connecting to the metadata database.
db.password
​
- Field:
scalar.db.analytics.server.db.password
- Description: The password for the metadata database user.
Server network configuration​
Configure network settings including service ports and TLS/SSL encryption.
catalog.port
​
- Field:
scalar.db.analytics.server.catalog.port
- Description: Port for the catalog service.
- Default value:
11051
metering.port
​
- Field:
scalar.db.analytics.server.metering.port
- Description: Port for the metering service.
- Default value:
11052
tls.enabled
​
- Field:
scalar.db.analytics.server.tls.enabled
- Description: Enable TLS/SSL for secure communication.
- Default value:
false
tls.cert_chain_path
​
- Field:
scalar.db.analytics.server.tls.cert_chain_path
- Description: Path to the server certificate chain file. Required when
tls.enabled
istrue
.
tls.private_key_path
​
- Field:
scalar.db.analytics.server.tls.private_key_path
- Description: Path to the server private key file. Required when
tls.enabled
istrue
.
License configuration​
Configure your ScalarDB Analytics license.
licensing.license_key
​
- Field:
scalar.db.analytics.server.licensing.license_key
- Description: Your ScalarDB Analytics license key.
licensing.license_check_cert_pem
​
- Field:
scalar.db.analytics.server.licensing.license_check_cert_pem
- Description: License verification certificate as PEM string. Either this or
license_check_cert_path
must be specified.
licensing.license_check_cert_path
​
- Field:
scalar.db.analytics.server.licensing.license_check_cert_path
- Description: Path to license verification certificate file. Either this or
license_check_cert_pem
must be specified.
Metering storage configuration​
Configure storage for metering data.
metering.storage.provider
​
- Field:
scalar.db.analytics.server.metering.storage.provider
- Description: Storage provider for metering data (
filesystem
,aws-s3
,azureblob
,google-cloud-storage
).
metering.storage.containerName
​
- Field:
scalar.db.analytics.server.metering.storage.containerName
- Description: Container/bucket name for cloud storage.
- Default value:
metering
metering.storage.path
​
- Field:
scalar.db.analytics.server.metering.storage.path
- Description: Local directory path. Required when provider is
filesystem
.
metering.storage.accessKeyId
​
- Field:
scalar.db.analytics.server.metering.storage.accessKeyId
- Description: Access key ID for cloud storage providers. Required for
aws-s3
,azureblob
, andgoogle-cloud-storage
.
metering.storage.secretAccessKey
​
- Field:
scalar.db.analytics.server.metering.storage.secretAccessKey
- Description: Secret access key for cloud storage providers. Required for
aws-s3
,azureblob
, andgoogle-cloud-storage
.
metering.storage.prefix
​
- Field:
scalar.db.analytics.server.metering.storage.prefix
- Description: Optional prefix for all storage paths.
CLI client configuration​
The CLI client requires connection settings to communicate with the ScalarDB Analytics server using a Java properties file (for example, client.properties
).
Configuration properties​
This section describes the configuration properties.
Server connection configuration​
The following is a list of configurations for connecting to the server.
server.host
​
- Field:
scalar.db.analytics.client.server.host
- Description: Hostname or IP address of the ScalarDB Analytics server.
server.catalog.port
​
- Field:
scalar.db.analytics.client.server.catalog.port
- Description: Port number for the catalog service.
- Default value:
11051
server.metering.port
​
- Field:
scalar.db.analytics.client.server.metering.port
- Description: Port number for the metering service.
- Default value:
11052
TLS configuration​
The following is a list of configurations for TLS.
server.tls.enabled
​
- Field:
scalar.db.analytics.client.server.tls.enabled
- Description: Enable TLS/SSL for server connections.
- Default value:
false
server.tls.ca_root_cert_path
​
- Field:
scalar.db.analytics.client.server.tls.ca_root_cert_path
- Description: Path to the CA certificate file for verifying server certificates. Required when
tls.enabled
istrue
.
server.tls.override_authority
​
- Field:
scalar.db.analytics.client.server.tls.override_authority
- Description: Override the server authority for TLS verification (useful for testing).
Spark integration configuration​
To use ScalarDB Analytics with Apache Spark, configure your Spark application by adding the necessary settings to your Spark configuration file (spark-defaults.conf
).
Spark Core configuration​
The following is a list of configurations for Spark Core.
spark.jars.packages
​
- Field:
spark.jars.packages
- Description: Maven coordinates for ScalarDB Analytics dependencies.
spark.extraListeners
​
- Field:
spark.extraListeners
- Description: Register the ScalarDB Analytics metering listener.
Catalog configuration​
The following is a list of configurations for the catalog.
spark.sql.catalog.<catalog-name>
​
- Field:
spark.sql.catalog.<catalog-name>
- Description: Register the ScalarDB Analytics catalog implementation. Replace
<catalog-name>
with the exact name of the catalog created in ScalarDB Analytics server. Usecom.scalar.db.analytics.spark.catalog.ScalarDBAnalyticsCatalog
as the value.
The <catalog-name>
must match the catalog name created in ScalarDB Analytics server using the CLI. For example, if you created a catalog named production
in the server, use spark.sql.catalog.production
.
Server connection configurations​
The following is a list of configurations for the server connection.
spark.sql.catalog.<catalog-name>.server.host
​
- Field:
spark.sql.catalog.<catalog-name>.server.host
- Description: Hostname or IP address of the ScalarDB Analytics server.
spark.sql.catalog.<catalog-name>.server.catalog.port
​
- Field:
spark.sql.catalog.<catalog-name>.server.catalog.port
- Description: Port number for the catalog service.
- Default value:
11051
spark.sql.catalog.<catalog-name>.server.metering.port
​
- Field:
spark.sql.catalog.<catalog-name>.server.metering.port
- Description: Port number for the metering service.
- Default value:
11052
TLS/SSL configurations​
The following is a list of configurations for TLS/SSL.
spark.sql.catalog.<catalog-name>.server.tls.enabled
​
- Field:
spark.sql.catalog.<catalog-name>.server.tls.enabled
- Description: Enable TLS/SSL for server connections.
- Default value:
false
spark.sql.catalog.<catalog-name>.server.tls.ca_root_cert_path
​
- Field:
spark.sql.catalog.<catalog-name>.server.tls.ca_root_cert_path
- Description: Path to the CA certificate file for verifying server certificates. Required when
tls.enabled
istrue
.
spark.sql.catalog.<catalog-name>.server.tls.override_authority
​
- Field:
spark.sql.catalog.<catalog-name>.server.tls.override_authority
- Description: Override the server authority for TLS verification.
Replace <catalog-name>
with your chosen catalog name (for example, analytics
).
Configuration examples​
This section provides some configuration examples.
Basic development configuration​
The following are examples of configurations for the server, CLI client, and Spark.
Server configuration (scalardb-analytics-server.properties
)​
# Metadata database
scalar.db.analytics.server.db.url=jdbc:postgresql://localhost:5432/scalardb_analytics
scalar.db.analytics.server.db.username=dev_user
scalar.db.analytics.server.db.password=dev_password
# License
scalar.db.analytics.server.licensing.license_key=YOUR_DEV_LICENSE_KEY
scalar.db.analytics.server.licensing.license_check_cert_path=/path/to/license_cert.pem
# Metering storage (filesystem for development)
scalar.db.analytics.server.metering.storage.provider=filesystem
scalar.db.analytics.server.metering.storage.path=/tmp/scalardb-analytics-metering
CLI client configuration (client.properties
)​
scalar.db.analytics.client.server.host=localhost
Spark configuration (spark-defaults.conf
)​
spark.jars.packages com.scalar-labs:scalardb-analytics-spark-all-3.5_2.12:3.16.2
spark.extraListeners com.scalar.db.analytics.spark.metering.ScalarDbAnalyticsListener
spark.sql.catalog.analytics com.scalar.db.analytics.spark.catalog.ScalarDBAnalyticsCatalog
spark.sql.catalog.analytics.server.host localhost
Production configuration with TLS​
The following are examples of configurations for TLS, CLI client, and Spark in production environments.
Server configuration (scalardb-analytics-server.properties
)​
# Metadata database
scalar.db.analytics.server.db.url=jdbc:postgresql://db.internal:5432/scalardb_analytics_prod
scalar.db.analytics.server.db.username=analytics_prod
scalar.db.analytics.server.db.password=your_secure_password
# gRPC ports
scalar.db.analytics.server.catalog.port=11051
scalar.db.analytics.server.metering.port=11052
# TLS
scalar.db.analytics.server.tls.enabled=true
scalar.db.analytics.server.tls.cert_chain_path=/path/to/server.crt
scalar.db.analytics.server.tls.private_key_path=/path/to/server.key
# License
scalar.db.analytics.server.licensing.license_key=YOUR_LICENSE_KEY
scalar.db.analytics.server.licensing.license_check_cert_pem=-----BEGIN CERTIFICATE-----\nMIID...certificate content...\n-----END CERTIFICATE-----
# Metering storage (S3)
scalar.db.analytics.server.metering.storage.provider=aws-s3
scalar.db.analytics.server.metering.storage.containerName=analytics-metering
scalar.db.analytics.server.metering.storage.accessKeyId=AKIAIOSFODNN7EXAMPLE
scalar.db.analytics.server.metering.storage.secretAccessKey=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
scalar.db.analytics.server.metering.storage.prefix=prod/
CLI Client configuration (client.properties
)​
scalar.db.analytics.client.server.host=analytics.example.com
scalar.db.analytics.client.server.tls.enabled=true
scalar.db.analytics.client.server.tls.ca_root_cert_path=/path/to/cert.pem
Spark configuration (spark-defaults.conf
)​
spark.jars.packages com.scalar-labs:scalardb-analytics-spark-all-3.5_2.12:3.16.2
spark.extraListeners com.scalar.db.analytics.spark.metering.ScalarDbAnalyticsListener
spark.sql.catalog.analytics com.scalar.db.analytics.spark.catalog.ScalarDBAnalyticsCatalog
spark.sql.catalog.analytics.server.host analytics.example.com
spark.sql.catalog.analytics.server.tls.enabled true
spark.sql.catalog.analytics.server.tls.ca_root_cert_path /path/to/cert.pem
Next steps​
- Catalog management - Learn how to manage catalogs and data sources
- Run analytical queries - Start running queries with your configuration
- Deployment guide - Deploy ScalarDB Analytics in production