ScalarDB Cluster Java API Guide
The ScalarDB Cluster Java API is composed of the Administrative API and Transaction API, which are part of ScalarDB Core, as well as additional APIs specific to ScalarDB Cluster. This guide explains what kinds of APIs exist, how to use them, and related topics like how to handle exceptions.
Administrative API
This section explains how to execute administrative operations programmatically by using the Administrative API in ScalarDB.
When an Administrative API call writes to the underlying databases, it triggers several write operations. However, these operations are not executed atomically, meaning that if the call fails midway, you may encounter inconsistent states. To resolve this inconsistency issue, you can repair the namespace or table. For details, see the following pages:
- Repair a namespace and Repair a table by using the Java API
- Repair namespaces and tables by using ScalarDB Schema Loader
Another method for executing administrative operations is to use Schema Loader.
Get a DistributedTransactionAdmin instance
You first need to get a DistributedTransactionAdmin instance to execute administrative operations.
To get a DistributedTransactionAdmin instance, you can use TransactionFactory as follows:
TransactionFactory transactionFactory = TransactionFactory.create("<CONFIGURATION_FILE_PATH>");
DistributedTransactionAdmin admin = transactionFactory.getTransactionAdmin();
For details about configurations, see ScalarDB Configurations.
After you have executed all administrative operations, you should close the DistributedTransactionAdmin instance as follows:
admin.close();
Create a namespace
Before creating tables, namespaces must be created since a table belongs to one namespace.
You can create a namespace as follows:
// Create the namespace "ns". If the namespace already exists, an exception will be thrown.
admin.createNamespace("ns");
// Create the namespace only if it does not already exist.
boolean ifNotExists = true;
admin.createNamespace("ns", ifNotExists);
// Create the namespace with options.
Map<String, String> options = ...;
admin.createNamespace("ns", options);
Creation options
In the namespace creation operations, you can specify options that are maps of option names and values (Map<String, String>). By using the options, you can set storage adapter–specific configurations.
Select your database to see the options available:
- JDBC databases
- DynamoDB
- Cosmos DB for NoSQL
- Cassandra
- Object Storage
No options are available.
No options are available.
| Name | Description | Default |
|---|---|---|
| ru | Base resource unit. | 400 |
| no-scaling | Disable auto-scaling for Cosmos DB for NoSQL. | false |
| Name | Description | Default |
|---|---|---|
| replication-strategy | Cassandra replication strategy. Must be SimpleStrategy or NetworkTopologyStrategy. | SimpleStrategy |
| replication-factor | Cassandra replication factor. | 3 |
No options are available.
Create a table
When creating a table, you should define the table metadata and then create the table.
To define the table metadata, you can use TableMetadata. The following shows how to define the columns, partition key, clustering key including clustering orders, and secondary indexes of a table:
// Define the table metadata.
TableMetadata tableMetadata =
TableMetadata.newBuilder()
.addColumn("c1", DataType.INT)
.addColumn("c2", DataType.TEXT)
.addColumn("c3", DataType.BIGINT)
.addColumn("c4", DataType.FLOAT)
.addColumn("c5", DataType.DOUBLE)
.addPartitionKey("c1")
.addClusteringKey("c2", Scan.Ordering.Order.DESC)
.addClusteringKey("c3", Scan.Ordering.Order.ASC)
.addSecondaryIndex("c4")
.build();
For details about the data model of ScalarDB, see Data Model.
Then, create a table as follows:
// Create the table "ns.tbl". If the table already exists, an exception will be thrown.
admin.createTable("ns", "tbl", tableMetadata);
// Create the table only if it does not already exist.
boolean ifNotExists = true;
admin.createTable("ns", "tbl", tableMetadata, ifNotExists);
// Create the table with options.
Map<String, String> options = ...;
admin.createTable("ns", "tbl", tableMetadata, options);
Creation options
In the table creation operations, you can specify options that are maps of option names and values (Map<String, String>). By using the options, you can set storage adapter–specific configurations.
Select your database to see the options available:
- JDBC databases
- DynamoDB
- Cosmos DB for NoSQL
- Cassandra
- Object Storage
| Name | Description | Default |
|---|---|---|
| transaction-metadata-decoupling | Enable transaction metadata decoupling when using Consensus Commit, which manages the transaction metadata in a separate table from application data. | false |
| Name | Description | Default |
|---|---|---|
| no-scaling | Disable auto-scaling for DynamoDB. | false |
| no-backup | Disable continuous backup for DynamoDB. | false |
| ru | Base resource unit. | 10 |
No options are available.
| Name | Description | Default |
|---|---|---|
| compaction-strategy | Cassandra compaction strategy, Must be LCS, STCS, or TWCS. | STCS |
No options are available.
Create a secondary index
You can create a secondary index as follows:
// Create a secondary index on column "c5" for table "ns.tbl". If a secondary index already exists, an exception will be thrown.
admin.createIndex("ns", "tbl", "c5");
// Create the secondary index only if it does not already exist.
boolean ifNotExists = true;
admin.createIndex("ns", "tbl", "c5", ifNotExists);
// Create the secondary index with options.
Map<String, String> options = ...;
admin.createIndex("ns", "tbl", "c5", options);
When using Consensus Commit, createIndex() on a non-primary-key column also creates a companion before-image secondary index. For details, see Correctness of index-based reads.
Creation options
In the secondary index creation operations, you can specify options that are maps of option names and values (Map<String, String>). By using the options, you can set storage adapter–specific configurations.
Select your database to see the options available:
- JDBC databases
- DynamoDB
- Cosmos DB for NoSQL
- Cassandra
- Object Storage
No options are available for JDBC databases.
| Name | Description | Default |
|---|---|---|
| no-scaling | Disable auto-scaling for DynamoDB. | false |
| ru | Base resource unit. | 10 |
No options are available.
No options are available.
No options are available.
Add a new column to a table
You can add a new, non-partition key column to a table as follows:
// Add a new column "c6" with the INT data type to the table "ns.tbl".
admin.addNewColumnToTable("ns", "tbl", "c6", DataType.INT);
// Add the new column only if it does not already exist.
boolean ifNotExists = true;
admin.addNewColumnToTable("ns", "tbl", "c6", DataType.INT, false, ifNotExists);
You should carefully consider adding a new column to a table because the execution time may vary greatly depending on the underlying storage. Please plan accordingly and consider the following, especially if the database runs in production:
- For Cosmos DB for NoSQL and DynamoDB: Adding a column is almost instantaneous as the table schema is not modified. Only the table metadata stored in a separate table is updated.
- For Cassandra: Adding a column will only update the schema metadata and will not modify the existing schema records. The cluster topology is the main factor for the execution time. Changes to the schema metadata are shared to each cluster node via a gossip protocol. Because of this, the larger the cluster, the longer it will take for all nodes to be updated.
- For relational databases (MySQL, Oracle, etc.): Adding a column can cause a table rebuild depending on the database engine. In such cases, it can take a long time to execute.
Drop a column from a table
You can drop a column from a table as follows:
// Drop the column "c6" from the table "ns.tbl".
admin.dropColumnFromTable("ns", "tbl", "c6");
// Drop the column only if it exists.
boolean ifExists = true;
admin.dropColumnFromTable("ns", "tbl", "c6", ifExists);
You cannot drop a column from a table in the following cases:
- The column is part of the partition key or clustering key.
- The table is on a non-JDBC database except for Cassandra.
You should carefully consider dropping a column from a table because the execution time may vary greatly depending on the underlying storage. Please plan accordingly and consider the following, especially if the database runs in production:
- For Cassandra: Dropping a column will only update the schema metadata and the actual data will be removed during the next compaction. The cluster topology is the main factor for the execution time. Changes to the schema metadata are shared to each cluster node via a gossip protocol. Because of this, the larger the cluster, the longer it will take for all nodes to be updated.
- For relational databases (MySQL, Oracle, etc.): Dropping a column issues
ALTER TABLE ... DROP COLUMNto the underlying relational databases and could trigger a table rebuild depending on the database. In such cases, it can take a long time to execute.
Rename a column of a table
You can rename a column of a table as follows:
// Rename the column "c6" to "c66" in the table "ns.tbl".
admin.renameColumnInTable("ns", "tbl", "c6", "c66");
You cannot rename a column of a table in the following cases:
- The table is on a non-JDBC database except for Cassandra.
- For Cassandra, the column is not part of the partition key or clustering key.
- For Db2, the column is part of the partition key, clustering key, or secondary index key.
Rename a table
You can rename a table as follows:
// Rename the table "ns.tbl" to "ns.new_tbl".
admin.renameTable("ns", "tbl", "new_tbl");
You cannot rename a table on non-JDBC databases.
Alter a column data type of a table
You can alter a column data type of a table as follows:
// Alter the data type of the column "c6" to BIGINT in the table "ns.tbl".
admin.alterColumnDataType("ns", "tbl", "c6", DataType.BIGINT);
You cannot alter a column data type of a table in the following cases:
- The column is part of the partition key, clustering key, or secondary index key.
- The table is on a non-JDBC database or on SQLite.
- The conversions other than from INT to BIGINT, FLOAT to DOUBLE, and from any data to TEXT are specified.
- For Oracle, the conversions except for from INT to BIGINT are specified.
- For Db2 and TiDB, the conversion from BLOB to TEXT is specified.
You should carefully consider altering a column type because the execution time may vary greatly depending on the underlying storage. Please plan accordingly and consider the following, especially if the database runs in production:
- For relational databases (MySQL, Oracle, etc.): Altering a column issues
ALTER TABLE ... ALTER COLUMNorALTER TABLE ... MODIFYto the underlying relational databases and could trigger a table rebuild depending on the database. In such cases, it can take a long time to execute.
Truncate a table
You can truncate a table as follows:
// Truncate the table "ns.tbl".
admin.truncateTable("ns", "tbl");
Drop a secondary index
You can drop a secondary index as follows:
// Drop the secondary index on column "c5" from table "ns.tbl". If the secondary index does not exist, an exception will be thrown.
admin.dropIndex("ns", "tbl", "c5");
// Drop the secondary index only if it exists.
boolean ifExists = true;
admin.dropIndex("ns", "tbl", "c5", ifExists);
When using Consensus Commit, dropIndex() also drops the companion before-image secondary index. For details, see Correctness of index-based reads.
Drop a table
You can drop a table as follows:
// Drop the table "ns.tbl". If the table does not exist, an exception will be thrown.
admin.dropTable("ns", "tbl");
// Drop the table only if it exists.
boolean ifExists = true;
admin.dropTable("ns", "tbl", ifExists);
Drop a namespace
You can drop a namespace as follows:
// Drop the namespace "ns". If the namespace does not exist, an exception will be thrown.
admin.dropNamespace("ns");
// Drop the namespace only if it exists.
boolean ifExists = true;
admin.dropNamespace("ns", ifExists);
Get existing namespaces
You can get the existing namespaces as follows:
Set<String> namespaces = admin.getNamespaceNames();
Get the tables of a namespace
You can get the tables of a namespace as follows:
// Get the tables of the namespace "ns".
Set<String> tables = admin.getNamespaceTableNames("ns");
Get table metadata
You can get table metadata as follows:
// Get the table metadata for "ns.tbl".
TableMetadata tableMetadata = admin.getTableMetadata("ns", "tbl");
Repair a namespace
If a namespace is in an unknown state, such as the namespace exists in the underlying storage but not its ScalarDB metadata or vice versa, this method will re-create the namespace and its metadata if necessary.
You can repair the namespace as follows:
// Repair the namespace "ns" with options.
Map<String, String> options = ...;
admin.repairNamespace("ns", options);
Repair a table
If a table is in an unknown state, such as the table exists in the underlying storage but not its ScalarDB metadata or vice versa, this method will re-create the table, its secondary indexes, and their metadata if necessary.
You can repair the table as follows:
// Repair the table "ns.tbl" with options.
TableMetadata tableMetadata =
TableMetadata.newBuilder()
...
.build();
Map<String, String> options = ...;
admin.repairTable("ns", "tbl", tableMetadata, options);
When using Consensus Commit, after upgrading to ScalarDB 3.16.5, 3.17.3, or 3.18.0 from an earlier version, you must run repairTable() on each existing table to create the companion before-image secondary indexes that index-based reads require. For details, see Correctness of index-based reads.
Upgrade the environment to support the latest ScalarDB API
You can upgrade the ScalarDB environment to support the latest version of the ScalarDB API. Typically, as indicated in the release notes, you will need to run this method after updating the ScalarDB version that your application environment uses.
// Upgrade the ScalarDB environment.
Map<String, String> options = ...;
admin.upgrade(options);
Specify operations for the Coordinator table
The Coordinator table is used by the Transaction API to track the statuses of transactions.
When using a transaction manager, you must create the Coordinator table to execute transactions. In addition to creating the table, you can truncate and drop the Coordinator table.
Create the Coordinator table
You can create the Coordinator table as follows:
// Create the Coordinator table.
admin.createCoordinatorTables();
// Create the Coordinator table only if one does not already exist.
boolean ifNotExist = true;
admin.createCoordinatorTables(ifNotExist);
// Create the Coordinator table with options.
Map<String, String> options = ...;
admin.createCoordinatorTables(options);
Truncate the Coordinator table
You can truncate the Coordinator table as follows:
// Truncate the Coordinator table.
admin.truncateCoordinatorTables();
Drop the Coordinator table
You can drop the Coordinator table as follows:
// Drop the Coordinator table.
admin.dropCoordinatorTables();
// Drop the Coordinator table if one exist.
boolean ifExist = true;
admin.dropCoordinatorTables(ifExist);
Import a table
You can import an existing table to ScalarDB as follows:
// Import the table "ns.tbl". If the table is already managed by ScalarDB, the target table does not
// exist, or the table does not meet the requirements of the ScalarDB table, an exception will be thrown.
admin.importTable("ns", "tbl", options, overrideColumnsType);
When using Consensus Commit, you should carefully plan to import a table to ScalarDB in production because it will add transaction metadata columns to your database tables and the ScalarDB metadata tables. In this case, there would also be several differences between your database and ScalarDB, as well as some limitations. For details, see Importing Existing Tables to ScalarDB by Using ScalarDB Schema Loader.
You can also enable transaction metadata decoupling by specifying the transaction-metadata-decoupling option to true to store transaction metadata in a separate table from application data. For details, see Transaction Metadata Decoupling.
Authentication and authorization API
ScalarDB Cluster supports authentication and authorization. To manage users, roles, and privileges programmatically via Java, use ClusterClientTransactionAdmin, which implements AuthAdmin.
For authentication and authorization concepts, including users, roles, and privileges, see Authenticate and authorize users.
Manage users
The following operations let you create, modify, and retrieve users.
When creating or altering a user, you can specify the following options by using the UserOption enum:
| Option | Description |
|---|---|
SUPERUSER | Creates or sets the user as a superuser. |
NO_SUPERUSER | Creates or sets the user as a non-superuser. This is the default if neither option is specified. |
You can also specify one or more authentication methods for a user by using the AuthenticationMethod enum:
| Authentication method | Description |
|---|---|
USERPASS | Username and password-based authentication. |
OIDC | OpenID Connect (OIDC) authentication. |
Create a user
You can create a user as follows:
// Create a regular (non-superuser) user with a password.
admin.createUser("username", "password");
// Create a superuser.
admin.createUser("username", "password", AuthAdmin.UserOption.SUPERUSER);
// Create a user without a password for OIDC authentication.
admin.createUser("username", null, ImmutableSet.of(AuthAdmin.AuthenticationMethod.OIDC));
// Create a user with specific authentication methods.
admin.createUser("username", "password", ImmutableSet.of(AuthAdmin.AuthenticationMethod.USERPASS, AuthAdmin.AuthenticationMethod.OIDC));
Alter a user
You can alter (modify) an existing user as follows:
// Change the password of an existing user.
admin.alterUser("username", "newpassword");
// Remove the password of an existing user (pass an empty string to delete the password).
admin.alterUser("username", "");
// Promote an existing user to superuser.
admin.alterUser("username", null, AuthAdmin.UserOption.SUPERUSER);
// Change the authentication methods of an existing user.
admin.alterUser("username", null, ImmutableSet.of(AuthAdmin.AuthenticationMethod.OIDC));
When null is passed as the password in alterUser, the password is not changed. When an empty string is passed, the password is deleted.
Drop a user
You can drop (delete) a user as follows:
admin.dropUser("username");
Get a user
You can get an existing user as follows:
Optional<AuthAdmin.User> user = admin.getUser("username");
user.ifPresent(u -> {
System.out.println("Name: " + u.getName());
System.out.println("Superuser: " + u.isSuperuser());
System.out.println("Authentication methods: " + u.getAuthenticationMethods());
});
Get all users
You can get all users as follows:
List<AuthAdmin.User> users = admin.getUsers();
for (AuthAdmin.User user : users) {
System.out.println("Name: " + user.getName());
System.out.println("Superuser: " + user.isSuperuser());
}
Get the current user
You can get the currently logged-in user as follows:
AuthAdmin.User currentUser = admin.getCurrentUser();
System.out.println("Current user: " + currentUser.getName());
System.out.println("Superuser: " + currentUser.isSuperuser());
Manage privileges for users
The following operations let you grant, revoke, and check privileges for users.
The Privilege enum defines the following values:
| Privilege | Description |
|---|---|
READ | Read operations (Get and Scan) |
WRITE | Write operations (Put, Insert, Upsert, Update) |
DELETE | Delete operations (Delete) |
CREATE | Creating tables and indexes |
DROP | Dropping tables and indexes |
TRUNCATE | Truncating tables |
ALTER | Altering tables |
GRANT | Granting and revoking privileges on tables |
These privilege names differ from the SQL-level privilege names (such as SELECT, INSERT, and UPDATE) used by DCL in ScalarDB SQL. When using the Java API directly, use the Privilege enum values listed above.
Grant privileges to a user
You can grant privileges to a user for all tables in a namespace or for a specific table as follows:
// Grant READ and WRITE privileges to a user for all tables in a namespace.
admin.grant("username", "namespace", AuthAdmin.Privilege.READ, AuthAdmin.Privilege.WRITE);
// Grant READ and WRITE privileges to a user for a specific table.
admin.grant("username", "namespace", "table", AuthAdmin.Privilege.READ, AuthAdmin.Privilege.WRITE);
Revoke privileges from a user
You can revoke privileges from a user for all tables in a namespace or for a specific table as follows:
// Revoke READ privilege from a user for all tables in a namespace.
admin.revoke("username", "namespace", AuthAdmin.Privilege.READ);
// Revoke READ privilege from a user for a specific table.
admin.revoke("username", "namespace", "table", AuthAdmin.Privilege.READ);
A privilege can only be revoked at the scope it was granted at. Revoking a privilege at a scope where it wasn't granted is a no-op—it doesn't raise an error, and it doesn't remove the privilege granted at a different scope.
For example, revoking a READ privilege on a specific table has no effect if that privilege was granted at the namespace level. Because table-level privilege checks also consider namespace-level privileges, as described in Check if a user has a privilege, a namespace-wide grant continues to grant access to the table even after a table-scoped revoke.
Get privileges for a user
You can get the privileges a user has for all tables in a namespace or for a specific table as follows:
// Get privileges for a user for all tables in a namespace.
Set<AuthAdmin.Privilege> nsPrivileges = admin.getPrivileges("username", "namespace");
// Get privileges for a user for a specific table.
Set<AuthAdmin.Privilege> tablePrivileges = admin.getPrivileges("username", "namespace", "table");
getPrivileges() returns directly granted privileges only. It doesn't include role-inherited privileges, namespace-level privileges that apply to a table, or superuser access. To check a user's effective access, use hasPrivilege().
Check if a user has a privilege
You can check whether a user has a specific privilege for a namespace or for a specific table as follows:
// Check if a user has READ privilege on the namespace.
boolean hasNsPrivilege = admin.hasPrivilege("username", "namespace", AuthAdmin.Privilege.READ);
// Check if a user has READ privilege for a specific table.
boolean hasTablePrivilege = admin.hasPrivilege("username", "namespace", "table", AuthAdmin.Privilege.READ);
When checking a privilege on a specific table, both table-level and namespace-level privileges are considered, including privileges granted transitively via roles. When checking a privilege on a namespace, only namespace-level privileges (including those granted transitively via roles) are considered. Superusers always return true for any privilege check.
Manage roles and privileges
The following operations let you create, modify, and retrieve roles, as well as grant and revoke roles to users and other roles.
Create a role
You can create a role as follows:
admin.createRole("rolename");
Drop a role
You can drop (delete) a role as follows:
admin.dropRole("rolename");
Get a role
You can get an existing role as follows:
Optional<AuthAdmin.Role> role = admin.getRole("rolename");
role.ifPresent(r -> {
System.out.println("Name: " + r.getName());
System.out.println("Granted roles: " + r.getGrantedRoles());
});
Get all roles
You can get all roles as follows:
List<AuthAdmin.Role> roles = admin.getRoles();
for (AuthAdmin.Role role : roles) {
System.out.println("Name: " + role.getName());
}
Get roles for a user
You can get the roles granted to a user as follows:
List<AuthAdmin.RoleForUser> roles = admin.getRolesForUser("username");
for (AuthAdmin.RoleForUser role : roles) {
System.out.println("Role: " + role.getName());
System.out.println("Has admin option: " + role.hasAdminOptionOnUser());
}
Grant a role to a user
You can grant a role to a user as follows:
// Grant a role to a user without admin option.
admin.grantRoleToUser("username", "rolename", false);
// Grant a role to a user with admin option (the user can then grant this role to others).
admin.grantRoleToUser("username", "rolename", true);
Revoke a role from a user
You can revoke a role from a user as follows:
admin.revokeRoleFromUser("username", "rolename");
Revoke admin option from a user for a role
You can revoke only the admin option (without revoking the role itself) from a user as follows:
admin.revokeAdminOptionFromUser("username", "rolename");
Get grantee users for a role
You can get the users who have been granted a specific role as follows:
List<AuthAdmin.GranteeUserRef> grantees = admin.getGranteeUsersForRole("rolename");
for (AuthAdmin.GranteeUserRef grantee : grantees) {
System.out.println("User: " + grantee.getName());
System.out.println("Has admin option: " + grantee.hasAdminOption());
}
Grant a member role to a role
You can grant a member role to a role so that users with that role inherit the member role's privileges as follows:
// Grant a member role to a role without admin option.
admin.grantRoleToRole("rolename", "memberrolename", false);
// Grant a member role to a role with admin option.
admin.grantRoleToRole("rolename", "memberrolename", true);
Revoke a member role from a role
You can revoke a member role from a role as follows:
admin.revokeRoleFromRole("rolename", "memberrolename");
Revoke admin option from a role for another role
You can revoke only the admin option for a role-to-role grant as follows:
admin.revokeAdminOptionFromRole("rolename", "memberrolename");
Grant privileges to a role
You can grant privileges to a role for all tables in a namespace or for a specific table as follows:
// Grant READ and WRITE privileges to a role for all tables in a namespace.
admin.grantPrivilegeToRole("rolename", "namespace", AuthAdmin.Privilege.READ, AuthAdmin.Privilege.WRITE);
// Grant READ and WRITE privileges to a role for a specific table.
admin.grantPrivilegeToRole("rolename", "namespace", "table", AuthAdmin.Privilege.READ, AuthAdmin.Privilege.WRITE);
Revoke privileges from a role
You can revoke privileges from a role for all tables in a namespace or for a specific table as follows:
// Revoke READ privilege from a role for all tables in a namespace.
admin.revokePrivilegeFromRole("rolename", "namespace", AuthAdmin.Privilege.READ);
// Revoke READ privilege from a role for a specific table.
admin.revokePrivilegeFromRole("rolename", "namespace", "table", AuthAdmin.Privilege.READ);
A privilege can only be revoked at the scope it was granted at. Revoking a privilege at a scope where it wasn't granted is a no-op—it doesn't raise an error, and it doesn't remove the privilege granted at a different scope.
For example, revoking a READ privilege on a specific table has no effect if that privilege was granted at the namespace level. Because table-level privilege checks also consider namespace-level privileges, as described in Check if a user has a privilege, a namespace-wide grant continues to grant access to the table even after a table-scoped revoke.
Get privileges for a role
You can get the privileges a role has for all tables in a namespace or for a specific table as follows:
// Get privileges for a role for all tables in a namespace.
Set<AuthAdmin.Privilege> nsPrivileges = admin.getRolePrivileges("rolename", "namespace");
// Get privileges for a role for a specific table.
Set<AuthAdmin.Privilege> tablePrivileges = admin.getRolePrivileges("rolename", "namespace", "table");
getRolePrivileges() returns privileges granted directly to the role only. It doesn't include privileges inherited from member roles or namespace-level privileges that apply to a table.
For details about handling failures from these APIs, see Handle authentication and authorization exceptions.
Transaction API
This section explains how to execute transactional operations by using the Transaction API in ScalarDB.
Get a DistributedTransactionManager instance
You first need to get a DistributedTransactionManager instance to execute transactional operations.
To get a DistributedTransactionManager instance, you can use TransactionFactory as follows:
TransactionFactory transactionFactory = TransactionFactory.create("<CONFIGURATION_FILE_PATH>");
DistributedTransactionManager transactionManager = transactionFactory.getTransactionManager();
After you have executed all transactional operations, you should close the DistributedTransactionManager instance as follows:
transactionManager.close();
Execute transactions
This subsection explains how to execute transactions with multiple CRUD operations.
Begin or start a transaction
Before executing transactional CRUD operations, you need to begin or start a transaction.
You can begin a transaction as follows:
// Begin a transaction.
DistributedTransaction transaction = transactionManager.begin();
Or, you can start a transaction as follows:
// Start a transaction.
DistributedTransaction transaction = transactionManager.start();
Alternatively, you can use the begin method for a transaction by specifying a transaction ID as follows:
// Begin a transaction by specifying a transaction ID.
DistributedTransaction transaction = transactionManager.begin("<TRANSACTION_ID>");
Or, you can use the start method for a transaction by specifying a transaction ID as follows:
// Start a transaction by specifying a transaction ID.
DistributedTransaction transaction = transactionManager.start("<TRANSACTION_ID>");
Specifying a transaction ID is useful when you want to link external systems to ScalarDB. Otherwise, you should use the begin() method or the start() method.
When you specify a transaction ID, make sure you specify a unique ID (for example, UUID v4) throughout the system since ScalarDB depends on the uniqueness of transaction IDs for correctness.
Begin or start a transaction in read-only mode
You can also begin or start a transaction in read-only mode. In this case, the transaction will not allow any write operations, and it will be optimized for read operations.
Using read-only transactions for read-only operations is strongly recommended to improve performance and reduce resource usage.
You can begin or start a transaction in read-only mode as follows:
// Begin a transaction in read-only mode.
DistributedTransaction transaction = transactionManager.beginReadOnly();
// Start a transaction in read-only mode.
DistributedTransaction transaction = transactionManager.startReadOnly();
Alternatively, you can use the beginReadOnly and startReadOnly methods by specifying a transaction ID as follows:
// Begin a transaction in read-only mode by specifying a transaction ID.
DistributedTransaction transaction = transactionManager.beginReadOnly("<TRANSACTION_ID>");
// Start a transaction in read-only mode by specifying a transaction ID.
DistributedTransaction transaction = transactionManager.startReadOnly("<TRANSACTION_ID>");
Specifying a transaction ID is useful when you want to link external systems to ScalarDB. Otherwise, you should use the beginReadOnly() method or the startReadOnly() method.
When you specify a transaction ID, make sure you specify a unique ID (for example, UUID v4) throughout the system since ScalarDB depends on the uniqueness of transaction IDs for correctness.
Begin or start a transaction with attributes
You can specify a map of transaction-scoped attributes when beginning or starting a transaction. The specified attributes are merged into every operation issued within the transaction, so configuration that should apply uniformly across the whole transaction can be specified once at begin time. If an operation already has the same attribute key, the attribute on the operation takes precedence.
You can begin or start a transaction with attributes as follows:
// Specify attributes for the transaction.
Map<String, String> attributes = ...;
// Begin a transaction with attributes.
DistributedTransaction transaction = transactionManager.begin(attributes);
// Start a transaction with attributes.
DistributedTransaction transaction = transactionManager.start(attributes);
You can also specify a transaction ID together with attributes as follows:
// Begin a transaction with attributes by specifying a transaction ID.
DistributedTransaction transaction = transactionManager.begin("<TRANSACTION_ID>", attributes);
// Start a transaction with attributes by specifying a transaction ID.
DistributedTransaction transaction = transactionManager.start("<TRANSACTION_ID>", attributes);
The beginReadOnly and startReadOnly methods also have overloads that accept attributes:
// Begin a transaction in read-only mode with attributes.
DistributedTransaction transaction = transactionManager.beginReadOnly(attributes);
// Start a transaction in read-only mode with attributes.
DistributedTransaction transaction = transactionManager.startReadOnly(attributes);
// Begin a transaction in read-only mode with attributes by specifying a transaction ID.
DistributedTransaction transaction = transactionManager.beginReadOnly("<TRANSACTION_ID>", attributes);
// Start a transaction in read-only mode with attributes by specifying a transaction ID.
DistributedTransaction transaction = transactionManager.startReadOnly("<TRANSACTION_ID>", attributes);
For the list of available attributes, see Operation attributes.
Join a transaction
Joining a transaction is particularly useful in a stateful application where a transaction spans multiple client requests. In such a scenario, the application can start a transaction during the first client request. Then, in subsequent client requests, the application can join the ongoing transaction by using the join() method.
You can join an ongoing transaction that has already begun by specifying the transaction ID as follows:
// Join a transaction.
DistributedTransaction transaction = transactionManager.join("<TRANSACTION_ID>");
To get the transaction ID with getId(), you can specify the following:
tx.getId();
Resume a transaction
Resuming a transaction is particularly useful in a stateful application where a transaction spans multiple client requests. In such a scenario, the application can start a transaction during the first client request. Then, in subsequent client requests, the application can resume the ongoing transaction by using the resume() method.
You can resume an ongoing transaction that you have already begun by specifying a transaction ID as follows:
// Resume a transaction.
DistributedTransaction transaction = transactionManager.resume("<TRANSACTION_ID>");
To get the transaction ID with getId(), you can specify the following:
tx.getId();
Implement CRUD operations
The following sections describe key construction and CRUD operations.
Although all the builders of the CRUD operations can specify consistency by using the consistency() methods, those methods are ignored. Instead, the LINEARIZABLE consistency level is always used in transactions.
Key construction
Most CRUD operations need to specify Key objects (partition-key, clustering-key, etc.). So, before moving on to CRUD operations, the following explains how to construct a Key object.
For a single column key, you can use Key.of<TYPE_NAME>() methods to construct the key as follows:
// For a key that consists of a single column of INT.
Key key1 = Key.ofInt("col1", 1);
// For a key that consists of a single column of BIGINT.
Key key2 = Key.ofBigInt("col1", 100L);
// For a key that consists of a single column of DOUBLE.
Key key3 = Key.ofDouble("col1", 1.3d);
// For a key that consists of a single column of TEXT.
Key key4 = Key.ofText("col1", "value");
For a key that consists of two to five columns, you can use the Key.of() method to construct the key as follows. Similar to ImmutableMap.of() in Guava, you need to specify column names and values in turns:
// For a key that consists of two to five columns.
Key key1 = Key.of("col1", 1, "col2", 100L);
Key key2 = Key.of("col1", 1, "col2", 100L, "col3", 1.3d);
Key key3 = Key.of("col1", 1, "col2", 100L, "col3", 1.3d, "col4", "value");
Key key4 = Key.of("col1", 1, "col2", 100L, "col3", 1.3d, "col4", "value", "col5", false);
For a key that consists of more than five columns, we can use the builder to construct the key as follows:
// For a key that consists of more than five columns.
Key key = Key.newBuilder()
.addInt("col1", 1)
.addBigInt("col2", 100L)
.addDouble("col3", 1.3d)
.addText("col4", "value")
.addBoolean("col5", false)
.addInt("col6", 100)
.build();
Get operation
Get is an operation to retrieve a single record specified by a primary key.
You need to create a Get object first, and then you can execute the object by using the transaction.get() method as follows:
// Create a `Get` operation.
Key partitionKey = Key.ofInt("c1", 10);
Key clusteringKey = Key.of("c2", "aaa", "c3", 100L);
Get get =
Get.newBuilder()
.namespace("ns")
.table("tbl")
.partitionKey(partitionKey)
.clusteringKey(clusteringKey)
.projections("c1", "c2", "c3", "c4")
.where(ConditionBuilder.column("c1").isNotEqualToInt(10))
.build();
// Execute the `Get` operation.
Optional<Result> result = transaction.get(get);
You can specify projections to choose which columns are returned.
Use the WHERE clause
You can also specify arbitrary conditions by using the where() method. If the retrieved record does not match the conditions specified by the where() method, Option.empty() will be returned. As an argument of the where() method, you can specify a condition, an AND-wise condition set, or an OR-wise condition set. After calling the where() method, you can add more conditions or condition sets by using the and() method or or() method as follows:
// Create a `Get` operation with condition sets.
Get get =
Get.newBuilder()
.namespace("ns")
.table("tbl")
.partitionKey(partitionKey)
.clusteringKey(clusteringKey)
.where(
ConditionSetBuilder.condition(ConditionBuilder.column("c1").isLessThanInt(10))
.or(ConditionBuilder.column("c1").isGreaterThanInt(20))
.build())
.and(
ConditionSetBuilder.condition(ConditionBuilder.column("c2").isLikeText("a%"))
.or(ConditionBuilder.column("c2").isLikeText("b%"))
.build())
.build();
In the where() condition method chain, the conditions must be an AND-wise junction of ConditionalExpression or OrConditionSet (known as conjunctive normal form) like the above example or an OR-wise junction of ConditionalExpression or AndConditionSet (known as disjunctive normal form).
For more details about available conditions and condition sets, see the ConditionBuilder and ConditionSetBuilder pages in the Javadoc.
Handle Result objects
The Get operation and Scan operation return Result objects. The following shows how to handle Result objects.
You can get a column value of a result by using get<TYPE_NAME>("<COLUMN_NAME>") methods as follows:
// Get the BOOLEAN value of a column.
boolean booleanValue = result.getBoolean("<COLUMN_NAME>");
// Get the INT value of a column.
int intValue = result.getInt("<COLUMN_NAME>");
// Get the BIGINT value of a column.
long bigIntValue = result.getBigInt("<COLUMN_NAME>");
// Get the FLOAT value of a column.
float floatValue = result.getFloat("<COLUMN_NAME>");
// Get the DOUBLE value of a column.
double doubleValue = result.getDouble("<COLUMN_NAME>");
// Get the TEXT value of a column.
String textValue = result.getText("<COLUMN_NAME>");
// Get the BLOB value of a column as a `ByteBuffer`.
ByteBuffer blobValue = result.getBlob("<COLUMN_NAME>");
// Get the BLOB value of a column as a `byte` array.
byte[] blobValueAsBytes = result.getBlobAsBytes("<COLUMN_NAME>");
// Get the DATE value of a column as a `LocalDate`.
LocalDate dateValue = result.getDate("<COLUMN_NAME>");
// Get the TIME value of a column as a `LocalTime`.
LocalTime timeValue = result.getTime("<COLUMN_NAME>");
// Get the TIMESTAMP value of a column as a `LocalDateTime`.
LocalDateTime timestampValue = result.getTimestamp("<COLUMN_NAME>");
// Get the TIMESTAMPTZ value of a column as a `Instant`.
Instant timestampTZValue = result.getTimestampTZ("<COLUMN_NAME>");
And if you need to check if a value of a column is null, you can use the isNull("<COLUMN_NAME>") method.
// Check if a value of a column is null.
boolean isNull = result.isNull("<COLUMN_NAME>");
For more details, see the Result page in the Javadoc.
Execute Get by using a secondary index
You can execute a Get operation by using a secondary index.
Instead of specifying a partition key, you can specify an index key (indexed column) to use a secondary index as follows:
// Create a `Get` operation by using a secondary index.
Key indexKey = Key.ofFloat("c4", 1.23F);
Get get =
Get.newBuilder()
.namespace("ns")
.table("tbl")
.indexKey(indexKey)
.projections("c1", "c2", "c3", "c4")
.where(ConditionBuilder.column("c1").isNotEqualToInt(10))
.build();
// Execute the `Get` operation.
Optional<Result> result = transaction.get(get);
You can also specify arbitrary conditions by using the where() method. For details, see Use the WHERE clause.
If the result has more than one record, transaction.get() will throw an exception. If you want to handle multiple results, see Execute Scan by using a secondary index.
Scan operation
Scan is an operation to retrieve multiple records within a partition. You can specify clustering-key boundaries and orderings for clustering-key columns in Scan operations. To execute a Scan operation, you can use the transaction.scan() method or the transaction.getScanner() method:
transaction.scan():- This method immediately executes the given
Scanoperation and returns a list of all matching records. It is suitable when the result set is expected to be small enough to fit in memory.
- This method immediately executes the given
transaction.getScanner():- This method returns a
Scannerobject that allows you to iterate over the result set lazily. It is useful when the result set may be large, as it avoids loading all records into memory at once.
- This method returns a
You need to create a Scan object first, and then you can execute the object by using the transaction.scan() method or the transaction.getScanner() method as follows:
// Create a `Scan` operation.
Key partitionKey = Key.ofInt("c1", 10);
Key startClusteringKey = Key.of("c2", "aaa", "c3", 100L);
Key endClusteringKey = Key.of("c2", "aaa", "c3", 300L);
Scan scan =
Scan.newBuilder()
.namespace("ns")
.table("tbl")
.partitionKey(partitionKey)
.start(startClusteringKey, true) // Include startClusteringKey
.end(endClusteringKey, false) // Exclude endClusteringKey
.projections("c1", "c2", "c3", "c4")
.orderings(Scan.Ordering.desc("c2"), Scan.Ordering.asc("c3"))
.where(ConditionBuilder.column("c1").isNotEqualToInt(10))
.limit(10)
.build();
// Execute the `Scan` operation by using the `transaction.scan()` method.
List<Result> results = transaction.scan(scan);
// Or, execute the `Scan` operation by using the `transaction.getScanner()` method.
try (TransactionCrudOperable.Scanner scanner = transaction.getScanner(scan)) {
// Fetch the next result from the scanner
Optional<Result> result = scanner.one();
// Fetch all remaining results from the scanner
List<Result> allResults = scanner.all();
}
You can omit the clustering-key boundaries or specify either a start boundary or an end boundary. If you don't specify orderings, you will get results ordered by the clustering order that you defined when creating the table.
In addition, you can specify projections to choose which columns are returned and use limit to specify the number of records to return in Scan operations.