Skip to main content
Version: 3.16

Data Source Reference

This reference guide provides detailed information about data source configuration formats, provider-specific settings, and data type mappings for ScalarDB Analytics.

warning

You need to have a license key (trial license or commercial license) to use ScalarDB Analytics. If you don't have a license key, please contact us.

Data source registration file format​

Data sources are registered to catalogs using the CLI with data source registration files. These files have the following structure. For CLI command details, see CLI command reference.

{
"catalog": "<catalog-name>", // The catalog to register the data source in
"name": "<data-source-name>", // A unique name for this data source
"type": "<database-type>", // Database type: postgres, mysql, scalardb, sqlserver, oracle, dynamodb
"provider": {
// Type-specific connection configuration
// Configuration varies by database type
}
}

The provider section contains data source-specific connection settings that vary based on the type field.

Provider configuration by type​

The following sections show the provider configuration for each supported database type:

Configurations

The following configuration is for ScalarDB.

configPath

  • Field: configPath
  • Description: Path to the ScalarDB configuration file.

Example

{
"catalog": "production",
"name": "scalardb_source",
"type": "scalardb",
"provider": {
"configPath": "/path/to/scalardb.properties"
}
}

Catalog information reference​

This section describes catalog structure mappings by data source and data type mappings.

Catalog structure mappings by data source​

When registering a data source to ScalarDB Analytics, the catalog structure of the data source, that is, namespaces, tables, and columns, are resolved and registered to the universal data catalog. To resolve the catalog structure of the data source, a particular object on the data sources side are mapped to the universal data catalog object.

Catalog-level mappings​

The catalog-level mappings are the mappings of the namespace names, table names, and column names from the data sources to the universal data catalog. To see the catalog-level mappings in each data source, select a data source.

The catalog structure of ScalarDB is automatically resolved by ScalarDB Analytics. The catalog-level objects are mapped as follows:

  • The ScalarDB namespace is mapped to the namespace. Therefore, the namespace of the ScalarDB data source is always single level, consisting of only the namespace name.
  • The ScalarDB table is mapped to the table.
  • The ScalarDB column is mapped to the column.

Data type mappings​

The following sections show how native types from each data source are mapped to ScalarDB Analytics types:

warning

Columns with data types that are not included in the mapping tables below will be ignored during data source registration. These columns will not appear in the ScalarDB Analytics catalog and cannot be queried. Information about ignored columns is logged in the ScalarDB Analytics server logs.

ScalarDB Data TypeScalarDB Analytics Data Type
BOOLEANBOOLEAN
INTINT
BIGINTBIGINT
FLOATFLOAT
DOUBLEDOUBLE
TEXTTEXT
BLOBBLOB
DATEDATE
TIMETIME
TIMESTAMPTIMESTAMP
TIMESTAMPTZTIMESTAMPTZ