ScalarDL Deployment Sample on Kubernetes (Auditor mode)
Version
- ScalarDL Ledger v3.5.3
- ScalarDL Auditor v3.5.3
- Scalar Envoy v1.3.0
- ScalarDL Schema Loader v3.5.0
- PostgreSQL v14.4
- Helm Chart: scalar-labs/scalardl v4.3.3
- Helm Chart: scalar-labs/scalardl-audit v2.3.3
- Helm Chart: scalar-labs/envoy v2.2.0
- Helm Chart: scalar-labs/schema-loading v2.6.0
- Helm Chart: bitnami/postgresql v11.6.26
Environment
This sample creates the following environment on Kubernetes cluster.
+-----------------------------------------------------------------------------------------------------------------------------+
| [Kubernetes Cluster] |
| [Pod] [Pod] [Pod] |
| |
| +-------+ +---------+ |
| +---> | Envoy | ---+ +---> | Ledger | ---+ |
| | +-------+ | | +---------+ | |
| | | | | |
| +---------+ | +-------+ | +-----------+ | +---------+ | +---------------+ |
| +---> | Service | ---+---> | Envoy | ---+---> | Service | ---+---> | Ledger | ---+---> | PostgreSQL | |
| | | (Envoy) | | +-------+ | | (Ledger) | | +---------+ | | (For Ledger) | |
| | +---------+ | | +-----------+ | | +---------------+ |
| | | +-------+ | | +---------+ | |
| | +---> | Envoy | ---+ +---> | Ledger | ---+ |
| +--------+ | +-------+ +---------+ |
| | Client | ---+ |
| +--------+ | +-------+ +---------+ |
| | +---> | Envoy | ---+ +---> | Auditor | ---+ |
| | | +-------+ | | +---------+ | |
| | | | | | |
| | +---------+ | +-------+ | +-----------+ | +---------+ | +---------------+ |
| +---> | Service | ---+---> | Envoy | ---+---> | Service | ---+---> | Auditor | ---+---> | PostgreSQL | |
| | (Envoy) | | +-------+ | | (Auditor) | | +---------+ | | (For Auditor) | |
| +---------+ | | +-----------+ | | +---------------+ |
| | +-------+ | | +---------+ | |
| +---> | Envoy | ---+ +---> | Auditor | ---+ |
| +-------+ +---------+ |
| |
+-----------------------------------------------------------------------------------------------------------------------------+
Note: To make Byzantine fault detection with auditing work properly, Ledger and Auditor should be deployed and managed in different administrative domains. However, for this samples, we deploy them in the same Kubernetes cluster.
Preparation
- Get sample files.
git clone https://github.com/scalar-labs/helm-charts.git cd helm-charts/docs/samples/scalardl/scalardl-auditor-mode-sample/ - Add Helm repositories.
helm repo add bitnami https://charts.bitnami.com/bitnamihelm repo add scalar-labs https://scalar-labs.github.io/helm-charts - Create a secret resource to access private container registry (GitHub Packages).
kubectl create secret docker-registry reg-docker-secrets \ --docker-server=ghcr.io \ --docker-username=<USERNAME> \ --docker-password=<GITHUB_PERSONAL_ACCESS_TOKEN> - Deploy PostgreSQL for Ledger.
helm install postgresql-ledger bitnami/postgresql \ --set auth.postgresPassword=postgres \ --set primary.persistence.enabled=false \ --version 11.6.26 - Deploy PostgreSQL for Auditor.
helm install postgresql-auditor bitnami/postgresql \ --set auth.postgresPassword=postgres \ --set primary.persistence.enabled=false \ --version 11.6.26
Deploy ScalarDL Ledger
- Create a secret resource that includes DB credentials.
kubectl create secret generic ledger-credentials-secret \ --from-literal=SCALAR_DB_USERNAME=postgres \ --from-literal=SCALAR_DB_PASSWORD=postgres - Create a secret resource that includes private key file.
kubectl create secret generic ledger-key-secret \ --from-file=ledger-key-file=./ledger-key.pem - Deploy ScalarDL Schema Loader to create schema on PostgreSQL for Ledger.
helm install schema-ledger scalar-labs/schema-loading \ -f ./schema-loader-ledger-custom-values.yaml \ --version 2.6.0 - Deploy ScalarDL Ledger.
helm install scalardl-ledger scalar-labs/scalardl \ -f ./scalardl-ledger-custom-values.yaml \ --version 4.3.3
Deploy ScalarDL Auditor
- Create a secret resource that includes DB credentials.
kubectl create secret generic auditor-credentials-secret \ --from-literal=SCALAR_DB_USERNAME=postgres \ --from-literal=SCALAR_DB_PASSWORD=postgres - Create a secret resource that includes private key and certificate file.
kubectl create secret generic auditor-key-secret \ --from-file=auditor-key-file=./auditor-key.pem \ --from-file=auditor-cert-file=./auditor.pem - Deploy ScalarDL Schema Loader to create schema on PostgreSQL for Auditor.
helm install schema-auditor scalar-labs/schema-loading \ -f ./schema-loader-auditor-custom-values.yaml \ --version 2.6.0 - Deploy ScalarDL Auditor.
helm install scalardl-auditor scalar-labs/scalardl-audit \ -f ./scalardl-auditor-custom-values.yaml \ --version 2.3.3
Deploy Client
- Create secret resources that include each private key and certificate file.
kubectl create secret generic client-ledger-key-secret \ --from-file=ledger-key-file=./ledger-key.pem \ --from-file=ledger-cert-file=./ledger.pemkubectl create secret generic client-auditor-key-secret \ --from-file=auditor-key-file=./auditor-key.pem \ --from-file=auditor-cert-file=./auditor.pemkubectl create secret generic client-key-secret \ --from-file=client-key-file=./client-key.pem \ --from-file=client-cert-file=./client.pem - Create configmap resources that include each properties file.
kubectl create configmap ledger-as-client-properties \ --from-file=./ledger.as.client.propertieskubectl create configmap auditor-as-client-properties \ --from-file=./auditor.as.client.propertieskubectl create configmap client-properties \ --from-file=./client.properties - Deploy client.
kubectl apply -f ./scalardl-client.yaml
Run sample contracts
- Attach to client container with bash.
kubectl exec -it scalardl-client -- bash - Install some tools to build and run the sample contracts.
apt update && DEBIAN_FRONTEND="noninteractive" TZ="Etc/UTC" apt install -y git openjdk-8-jdk curl unzip - Clone ScalarDL Java Client SDK git repository and build sample contracts.
git clone https://github.com/scalar-labs/scalardl-java-client-sdk.git cd /scalardl-java-client-sdk/ git checkout -b v3.5.3 refs/tags/v3.5.3 ./gradlew assemble - Download CLI tools of ScalarDL and unzip them.
curl -OL https://github.com/scalar-labs/scalardl-java-client-sdk/releases/download/v3.5.3/scalardl-java-client-sdk-3.5.3.zip unzip ./scalardl-java-client-sdk-3.5.3.zip - Register the certificate file of Ledger, Auditor, and client.
./scalardl-java-client-sdk-3.5.3/bin/register-cert --properties /conf/ledger/ledger.as.client.properties ./scalardl-java-client-sdk-3.5.3/bin/register-cert --properties /conf/auditor/auditor.as.client.properties ./scalardl-java-client-sdk-3.5.3/bin/register-cert --properties /conf/client/client.properties - Register the sample contract
StateUpdater../scalardl-java-client-sdk-3.5.3/bin/register-contract --properties /conf/client/client.properties --contract-id StateUpdater --contract-binary-name com.org1.contract.StateUpdater --contract-class-file ./build/classes/java/main/com/org1/contract/StateUpdater.class - Register the sample contract
StateReader../scalardl-java-client-sdk-3.5.3/bin/register-contract --properties /conf/client/client.properties --contract-id StateReader --contract-binary-name com.org1.contract.StateReader --contract-class-file ./build/classes/java/main/com/org1/contract/StateReader.class - Register the contract
ValidateLedger../scalardl-java-client-sdk-3.5.3/bin/register-contract --properties /conf/client/client.properties --contract-id validate-ledger --contract-binary-name com.scalar.dl.client.contract.ValidateLedger --contract-class-file ./build/classes/java/main/com/scalar/dl/client/contract/ValidateLedger.class - Execute the contract
StateUpdater. This sample contract updates thestate(value) of the asset namedtest_assetto3../scalardl-java-client-sdk-3.5.3/bin/execute-contract --properties /conf/client/client.properties --contract-id StateUpdater --contract-argument '{"asset_id": "test_asset", "state": 3}' - Execute the contract
StateReader../scalardl-java-client-sdk-3.5.3/bin/execute-contract --properties /conf/client/client.properties --contract-id StateReader --contract-argument '{"asset_id": "test_asset"}' - Execute a validation request of the asset.
./scalardl-java-client-sdk-3.5.3/bin/validate-ledger --properties /conf/client/client.properties --asset-id "test_asset"