Getting Started with Helm Charts (ScalarDL Ledger and Auditor / Auditor mode)
This document explains how to get started with ScalarDL Ledger and Auditor using Helm Chart on a Kubernetes cluster as a test environment. Here, we assume that you already have a Mac or Linux environment for testing. We use Minikube in this document, but the steps we will show should work in any Kubernetes cluster.
Requirement
You need to subscribe to ScalarDL Ledger and ScalarDL Auditor in the AWS Marketplace or Azure Marketplace to get the following container images.
- AWS Marketplace
- scalar-ledger
- scalar-ledger-envoy
- scalardl-schema-loader-ledger
- scalar-auditor
- scalar-auditor-envoy
- scalardl-schema-loader-auditor
- Azure Marketplace
- scalar-ledger
- scalar-auditor
- scalardl-envoy
- scalardl-schema-loader
Please refer to the following documents for more details.
- How to install Scalar products through AWS Marketplace
- How to install Scalar products through Azure Marketplace
Note
To make Byzantine fault detection with auditing work properly, Ledger and Auditor should be deployed and managed in different administrative domains. However, in this guide, we will deploy Ledger and Auditor in the same Kubernetes cluster to make the test easier.
What we create
We will deploy the following components on a Kubernetes cluster as follows.
+-----------------------------------------------------------------------------------------------------------------------------+
| [Kubernetes Cluster] |
| [Pod] [Pod] [Pod] |
| |
| +-------+ +---------+ |
| +---> | Envoy | ---+ +---> | Ledger | ---+ |
| | +-------+ | | +---------+ | |
| | | | | |
| +---------+ | +-------+ | +-----------+ | +---------+ | +---------------+ |
| +---> | Service | ---+---> | Envoy | ---+---> | Service | ---+---> | Ledger | ---+---> | PostgreSQL | |
| | | (Envoy) | | +-------+ | | (Ledger) | | +---------+ | | (For Ledger) | |
| | +---------+ | | +-----------+ | | +---------------+ |
| | | +-------+ | | +---------+ | |
| | +---> | Envoy | ---+ +---> | Ledger | ---+ |
| +--------+ | +-------+ +---------+ |
| | Client | ---+ |
| +--------+ | +-------+ +---------+ |
| | +---> | Envoy | ---+ +---> | Auditor | ---+ |
| | | +-------+ | | +---------+ | |
| | | | | | |
| | +---------+ | +-------+ | +-----------+ | +---------+ | +---------------+ |
| +---> | Service | ---+---> | Envoy | ---+---> | Service | ---+---> | Auditor | ---+---> | PostgreSQL | |
| | (Envoy) | | +-------+ | | (Auditor) | | +---------+ | | (For Auditor) | |
| +---------+ | | +-----------+ | | +---------------+ |
| | +-------+ | | +---------+ | |
| +---> | Envoy | ---+ +---> | Auditor | ---+ |
| +-------+ +---------+ |
| |
+-----------------------------------------------------------------------------------------------------------------------------+
Step 1. Start a Kubernetes cluster
First, you need to prepare a Kubernetes cluster. If you use a minikube environment, please refer to the Getting Started with Scalar Helm Charts. If you have already started a Kubernetes cluster, you can skip this step.
Step 2. Start PostgreSQL containers
ScalarDL Ledger and Auditor use some kind of database system as a backend database. In this document, we use PostgreSQL.
You can deploy PostgreSQL on the Kubernetes cluster as follows.
-
Add the Bitnami helm repository.
helm repo add bitnami https://charts.bitnami.com/bitnami
-
Deploy PostgreSQL for Ledger.
helm install postgresql-ledger bitnami/postgresql \
--set auth.postgresPassword=postgres \
--set primary.persistence.enabled=false -
Deploy PostgreSQL for Auditor.
helm install postgresql-auditor bitnami/postgresql \
--set auth.postgresPassword=postgres \
--set primary.persistence.enabled=false -
Check if the PostgreSQL containers are running.
kubectl get pod
[Command execution result]
NAME READY STATUS RESTARTS AGE
postgresql-auditor-0 1/1 Running 0 11s
postgresql-ledger-0 1/1 Running 0 16s
Step 3. Create a working directory
We will create some configuration files and key/certificate files locally. So, create a working directory for them.
- Create a working directory.
mkdir -p ~/scalardl-test/certs/
Step 4. Create key/certificate files
Note: In this guide, we will use self-sign certificates for the test. However, it is strongly recommended that these certificates NOT be used in production.
-
Change the working directory to
~/scalardl-test/certs/
directory.cd ~/scalardl-test/certs/
-
Create a JSON file that includes Ledger information.
cat << 'EOF' > ~/scalardl-test/certs/ledger.json
{
"CN": "ledger",
"hosts": ["example.com","*.example.com"],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"O": "ledger",
"OU": "test team",
"L": "Shinjuku",
"ST": "Tokyo",
"C": "JP"
}
]
}
EOF -
Create a JSON file that includes Auditor information.
cat << 'EOF' > ~/scalardl-test/certs/auditor.json
{
"CN": "auditor",
"hosts": ["example.com","*.example.com"],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"O": "auditor",
"OU": "test team",
"L": "Shinjuku",
"ST": "Tokyo",
"C": "JP"
}
]
}
EOF -
Create a JSON file that includes Client information.
cat << 'EOF' > ~/scalardl-test/certs/client.json
{
"CN": "client",
"hosts": ["example.com","*.example.com"],
"key": {
"algo": "ecdsa",
"size": 256
},
"names": [
{
"O": "client",
"OU": "test team",
"L": "Shinjuku",
"ST": "Tokyo",
"C": "JP"
}
]
}
EOF -
Create key/certificate files for the Ledger.
cfssl selfsign "" ./ledger.json | cfssljson -bare ledger
-
Create key/certificate files for the Auditor.
cfssl selfsign "" ./auditor.json | cfssljson -bare auditor
-
Create key/certificate files for the Client.
cfssl selfsign "" ./client.json | cfssljson -bare client
-
Confirm key/certificate files are created.
ls -1
[Command execution result]
auditor-key.pem
auditor.csr
auditor.json
auditor.pem
client-key.pem
client.csr
client.json
client.pem
ledger-key.pem
ledger.csr
ledger.json
ledger.pem
Step 5. Create DB schemas for ScalarDL Ledger using Helm Charts
We will deploy two ScalarDL Schema Loader pods on the Kubernetes cluster using Helm Charts.
The ScalarDL Schema Loader will create the DB schemas for ScalarDL Ledger and Auditor in PostgreSQL.
-
Change the working directory to
~/scalardl-test/
.cd ~/scalardl-test/
-
Add the Scalar helm repository.
helm repo add scalar-labs https://scalar-labs.github.io/helm-charts
-
Create a secret resource to pull the ScalarDL container images from AWS/Azure Marketplace.
- AWS Marketplace
kubectl create secret docker-registry reg-ecr-mp-secrets \
--docker-server=709825985650.dkr.ecr.us-east-1.amazonaws.com \
--docker-username=AWS \
--docker-password=$(aws ecr get-login-password --region us-east-1) - Azure Marketplace
kubectl create secret docker-registry reg-acr-secrets \
--docker-server=<your private container registry login server> \
--docker-username=<Service principal ID> \
--docker-password=<Service principal password>
Please refer to the following documents for more details.
- AWS Marketplace
-
Create a custom values file for ScalarDL Schema Loader for Ledger (schema-loader-ledger-custom-values.yaml).
-
AWS Marketplace
cat << 'EOF' > ~/scalardl-test/schema-loader-ledger-custom-values.yaml
schemaLoading:
schemaType: "ledger"
image:
repository: "709825985650.dkr.ecr.us-east-1.amazonaws.com/scalar/scalardl-schema-loader-ledger"
version: "3.6.0"
imagePullSecrets:
- name: "reg-ecr-mp-secrets"
databaseProperties: |
scalar.db.contact_points=jdbc:postgresql://postgresql-ledger.default.svc.cluster.local:5432/postgres
scalar.db.username={{ default .Env.SCALAR_DL_LEDGER_POSTGRES_USERNAME "" }}
scalar.db.password={{ default .Env.SCALAR_DL_LEDGER_POSTGRES_PASSWORD "" }}
scalar.db.storage=jdbc
secretName: "ledger-credentials-secret"
EOF -
Azure Marketplace
cat << 'EOF' > ~/scalardl-test/schema-loader-ledger-custom-values.yaml
schemaLoading:
schemaType: "ledger"
image:
repository: "<your private container registry>/scalarinc/scalardl-schema-loader"
version: "3.6.0"
imagePullSecrets:
- name: "reg-acr-secrets"
databaseProperties: |
scalar.db.contact_points=jdbc:postgresql://postgresql-ledger.default.svc.cluster.local:5432/postgres
scalar.db.username={{ default .Env.SCALAR_DL_LEDGER_POSTGRES_USERNAME "" }}
scalar.db.password={{ default .Env.SCALAR_DL_LEDGER_POSTGRES_PASSWORD "" }}
scalar.db.storage=jdbc
secretName: "ledger-credentials-secret"
EOF
-
-
Create a custom values file for ScalarDL Schema Loader for Auditor (schema-loader-auditor-custom-values.yaml).
-
AWS Marketplace
cat << 'EOF' > ~/scalardl-test/schema-loader-auditor-custom-values.yaml
schemaLoading:
schemaType: "auditor"
image:
repository: "709825985650.dkr.ecr.us-east-1.amazonaws.com/scalar/scalardl-schema-loader-auditor"
version: "3.6.0"
imagePullSecrets:
- name: "reg-ecr-mp-secrets"
databaseProperties: |
scalar.db.contact_points=jdbc:postgresql://postgresql-auditor.default.svc.cluster.local:5432/postgres
scalar.db.username={{ default .Env.SCALAR_DL_AUDITOR_POSTGRES_USERNAME "" }}
scalar.db.password={{ default .Env.SCALAR_DL_AUDITOR_POSTGRES_PASSWORD "" }}
scalar.db.storage=jdbc
secretName: "auditor-credentials-secret"
EOF -
Azure Marketplace
cat << 'EOF' > ~/scalardl-test/schema-loader-auditor-custom-values.yaml
schemaLoading:
schemaType: "auditor"
image:
repository: "<your private container registry>/scalarinc/scalardl-schema-loader"
version: "3.6.0"
imagePullSecrets:
- name: "reg-acr-secrets"
databaseProperties: |
scalar.db.contact_points=jdbc:postgresql://postgresql-auditor.default.svc.cluster.local:5432/postgres
scalar.db.username={{ default .Env.SCALAR_DL_AUDITOR_POSTGRES_USERNAME "" }}
scalar.db.password={{ default .Env.SCALAR_DL_AUDITOR_POSTGRES_PASSWORD "" }}
scalar.db.storage=jdbc
secretName: "auditor-credentials-secret"
EOF
-
-
Create a secret resource that includes a username and password for PostgreSQL for Ledger.
kubectl create secret generic ledger-credentials-secret \
--from-literal=SCALAR_DL_LEDGER_POSTGRES_USERNAME=postgres \
--from-literal=SCALAR_DL_LEDGER_POSTGRES_PASSWORD=postgres -
Create a secret resource that includes a username and password for PostgreSQL for Auditor.
kubectl create secret generic auditor-credentials-secret \
--from-literal=SCALAR_DL_AUDITOR_POSTGRES_USERNAME=postgres \
--from-literal=SCALAR_DL_AUDITOR_POSTGRES_PASSWORD=postgres -
Deploy the ScalarDL Schema Loader for Ledger.
helm install schema-loader-ledger scalar-labs/schema-loading -f ./schema-loader-ledger-custom-values.yaml
-
Deploy the ScalarDL Schema Loader for Auditor.
helm install schema-loader-auditor scalar-labs/schema-loading -f ./schema-loader-auditor-custom-values.yaml
-
Check if the ScalarDL Schema Loader pods are deployed and completed.
kubectl get pod
[Command execution result]
NAME READY STATUS RESTARTS AGE
postgresql-auditor-0 1/1 Running 0 2m56s
postgresql-ledger-0 1/1 Running 0 3m1s
schema-loader-auditor-schema-loading-dvc5r 0/1 Completed 0 6s
schema-loader-ledger-schema-loading-mtllb 0/1 Completed 0 10sIf the ScalarDL Schema Loader pods are ContainerCreating or Running, wait for the process will be completed (The STATUS will be Completed).
Step 6. Deploy ScalarDL Ledger and Auditor on the Kubernetes cluster using Helm Charts
-
Create a custom values file for ScalarDL Ledger (scalardl-ledger-custom-values.yaml).
-
AWS Marketplace
cat << 'EOF' > ~/scalardl-test/scalardl-ledger-custom-values.yaml
envoy:
image:
repository: "709825985650.dkr.ecr.us-east-1.amazonaws.com/scalar/scalar-ledger-envoy"
version: "1.3.0"
imagePullSecrets:
- name: "reg-ecr-mp-secrets"
ledger:
image:
repository: "709825985650.dkr.ecr.us-east-1.amazonaws.com/scalar/scalar-ledger"
version: "3.6.0"
imagePullSecrets:
- name: "reg-ecr-mp-secrets"
ledgerProperties: |
scalar.db.contact_points=jdbc:postgresql://postgresql-ledger.default.svc.cluster.local:5432/postgres
scalar.db.username={{ default .Env.SCALAR_DL_LEDGER_POSTGRES_USERNAME "" }}
scalar.db.password={{ default .Env.SCALAR_DL_LEDGER_POSTGRES_PASSWORD "" }}
scalar.db.storage=jdbc
scalar.dl.ledger.proof.enabled=true
scalar.dl.ledger.auditor.enabled=true
scalar.dl.ledger.proof.private_key_path=/keys/private-key
secretName: "ledger-credentials-secret"
extraVolumes:
- name: "ledger-keys"
secret:
secretName: "ledger-keys"
extraVolumeMounts:
- name: "ledger-keys"
mountPath: "/keys"
readOnly: true
EOF -
Azure Marketplace
cat << 'EOF' > ~/scalardl-test/scalardl-ledger-custom-values.yaml
envoy:
image:
repository: "<your private container registry>/scalarinc/scalardl-envoy"
version: "1.3.0"
imagePullSecrets:
- name: "reg-acr-secrets"
ledger:
image:
repository: "<your private container registry>/scalarinc/scalar-ledger"
version: "3.6.0"
imagePullSecrets:
- name: "reg-acr-secrets"
ledgerProperties: |
scalar.db.contact_points=jdbc:postgresql://postgresql-ledger.default.svc.cluster.local:5432/postgres
scalar.db.username={{ default .Env.SCALAR_DL_LEDGER_POSTGRES_USERNAME "" }}
scalar.db.password={{ default .Env.SCALAR_DL_LEDGER_POSTGRES_PASSWORD "" }}
scalar.db.storage=jdbc
scalar.dl.ledger.proof.enabled=true
scalar.dl.ledger.proof.private_key_path=/keys/private-key
secretName: "ledger-credentials-secret"
extraVolumes:
- name: "ledger-keys"
secret:
secretName: "ledger-keys"
extraVolumeMounts:
- name: "ledger-keys"
mountPath: "/keys"
readOnly: true
EOF
-
-
Create a custom values file for ScalarDL Auditor (scalardl-auditor-custom-values.yaml).
-
AWS Marketplace
cat << 'EOF' > ~/scalardl-test/scalardl-auditor-custom-values.yaml
envoy:
image:
repository: "709825985650.dkr.ecr.us-east-1.amazonaws.com/scalar/scalar-auditor-envoy"
version: "1.3.0"
imagePullSecrets:
- name: "reg-ecr-mp-secrets"
auditor:
image:
repository: "709825985650.dkr.ecr.us-east-1.amazonaws.com/scalar/scalar-auditor"
version: "3.6.0"
imagePullSecrets:
- name: "reg-ecr-mp-secrets"
auditorProperties: |
scalar.db.contact_points=jdbc:postgresql://postgresql-auditor.default.svc.cluster.local:5432/postgres
scalar.db.username={{ default .Env.SCALAR_DL_AUDITOR_POSTGRES_USERNAME "" }}
scalar.db.password={{ default .Env.SCALAR_DL_AUDITOR_POSTGRES_PASSWORD "" }}
scalar.db.storage=jdbc
scalar.dl.auditor.ledger.host=scalardl-ledger-envoy.default.svc.cluster.local
scalar.dl.auditor.cert_path=/keys/certificate
scalar.dl.auditor.private_key_path=/keys/private-key
secretName: "auditor-credentials-secret"
extraVolumes:
- name: "auditor-keys"
secret:
secretName: "auditor-keys"
extraVolumeMounts:
- name: "auditor-keys"
mountPath: "/keys"
readOnly: true
EOF -
Azure Marketplace
cat << 'EOF' > ~/scalardl-test/scalardl-auditor-custom-values.yaml
envoy:
image:
repository: "<your private container registry>/scalarinc/scalardl-envoy"
version: "1.3.0"
imagePullSecrets:
- name: "reg-acr-secrets"
auditor:
image:
repository: "<your private container registry>/scalarinc/scalar-auditor"
version: "3.6.0"
imagePullSecrets:
- name: "reg-acr-secrets"
auditorProperties: |
scalar.db.contact_points=jdbc:postgresql://postgresql-auditor.default.svc.cluster.local:5432/postgres
scalar.db.username={{ default .Env.SCALAR_DL_AUDITOR_POSTGRES_USERNAME "" }}
scalar.db.password={{ default .Env.SCALAR_DL_AUDITOR_POSTGRES_PASSWORD "" }}
scalar.db.storage=jdbc
scalar.dl.auditor.ledger.host=scalardl-ledger-envoy.default.svc.cluster.local
scalar.dl.auditor.cert_path=/keys/certificate
scalar.dl.auditor.private_key_path=/keys/private-key
secretName: "auditor-credentials-secret"
extraVolumes:
- name: "auditor-keys"
secret:
secretName: "auditor-keys"
extraVolumeMounts:
- name: "auditor-keys"
mountPath: "/keys"
readOnly: true
EOF
-
-
Create secret resource
ledger-keys
.kubectl create secret generic ledger-keys --from-file=certificate=./certs/ledger.pem --from-file=private-key=./certs/ledger-key.pem
-
Create secret resource
auditor-keys
.kubectl create secret generic auditor-keys --from-file=certificate=./certs/auditor.pem --from-file=private-key=./certs/auditor-key.pem
-
Deploy the ScalarDL Ledger.
helm install scalardl-ledger scalar-labs/scalardl -f ./scalardl-ledger-custom-values.yaml
-
Deploy the ScalarDL Auditor.
helm install scalardl-auditor scalar-labs/scalardl-audit -f ./scalardl-auditor-custom-values.yaml
-
Check if the ScalarDL Ledger and Auditor pods are deployed.
kubectl get pod
[Command execution result]
NAME READY STATUS RESTARTS AGE
postgresql-auditor-0 1/1 Running 0 14m
postgresql-ledger-0 1/1 Running 0 14m
scalardl-auditor-auditor-5b885ff4c8-fwkpf 1/1 Running 0 18s
scalardl-auditor-auditor-5b885ff4c8-g69cb 1/1 Running 0 18s
scalardl-auditor-auditor-5b885ff4c8-nsmnq 1/1 Running 0 18s
scalardl-auditor-envoy-689bcbdf65-5mn6v 1/1 Running 0 18s
scalardl-auditor-envoy-689bcbdf65-fpq8j 1/1 Running 0 18s
scalardl-auditor-envoy-689bcbdf65-lsz2t 1/1 Running 0 18s
scalardl-ledger-envoy-547bbf7546-n7p5x 1/1 Running 0 26s
scalardl-ledger-envoy-547bbf7546-p8nwp 1/1 Running 0 26s
scalardl-ledger-envoy-547bbf7546-pskpb 1/1 Running 0 26s
scalardl-ledger-ledger-6db5dc8774-5zsbj 1/1 Running 0 26s
scalardl-ledger-ledger-6db5dc8774-vnmrw 1/1 Running 0 26s
scalardl-ledger-ledger-6db5dc8774-wpjvs 1/1 Running 0 26s
schema-loader-auditor-schema-loading-dvc5r 0/1 Completed 0 11m
schema-loader-ledger-schema-loading-mtllb 0/1 Completed 0 11mIf the ScalarDL Ledger and Auditor pods are deployed properly, you can see the STATUS are Running.
-
Check if the ScalarDL Ledger and Auditor services are deployed.
kubectl get svc
[Command execution result]
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 47d
postgresql-auditor ClusterIP 10.107.9.78 <none> 5432/TCP 15m
postgresql-auditor-hl ClusterIP None <none> 5432/TCP 15m
postgresql-ledger ClusterIP 10.108.241.181 <none> 5432/TCP 15m
postgresql-ledger-hl ClusterIP None <none> 5432/TCP 15m
scalardl-auditor-envoy ClusterIP 10.100.61.202 <none> 40051/TCP,40052/TCP 55s
scalardl-auditor-envoy-metrics ClusterIP 10.99.6.227 <none> 9001/TCP 55s
scalardl-auditor-headless ClusterIP None <none> 40051/TCP,40053/TCP,40052/TCP 55s
scalardl-auditor-metrics ClusterIP 10.108.1.147 <none> 8080/TCP 55s
scalardl-ledger-envoy ClusterIP 10.101.191.116 <none> 50051/TCP,50052/TCP 61s
scalardl-ledger-envoy-metrics ClusterIP 10.106.52.103 <none> 9001/TCP 61s
scalardl-ledger-headless ClusterIP None <none> 50051/TCP,50053/TCP,50052/TCP 61s
scalardl-ledger-metrics ClusterIP 10.99.122.106 <none> 8080/TCP 61sIf the ScalarDL Ledger and Auditor services are deployed properly, you can see private IP addresses in the CLUSTER-IP column. (Note:
scalardl-ledger-headless
andscalardl-auditor-headless
have no CLUSTER-IP.)
Step 7. Start a Client container
We will use certificate files in a Client container. So, we create a secret resource and mount it to a Client container.
-
Create secret resource
client-keys
.kubectl create secret generic client-keys --from-file=certificate=./certs/client.pem --from-file=private-key=./certs/client-key.pem
-
Start a Client container on the Kubernetes cluster.
cat << 'EOF' | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
name: "scalardl-client"
spec:
containers:
- name: scalardl-client
image: eclipse-temurin:8
command: ['sleep']
args: ['inf']
volumeMounts:
- name: "ledger-keys"
mountPath: "/keys/ledger"
readOnly: true
- name: "auditor-keys"
mountPath: "/keys/auditor"
readOnly: true
- name: "client-keys"
mountPath: "/keys/client"
readOnly: true
volumes:
- name: "ledger-keys"
secret:
secretName: "ledger-keys"
- name: "auditor-keys"
secret:
secretName: "auditor-keys"
- name: "client-keys"
secret:
secretName: "client-keys"
restartPolicy: Never
EOF -
Check if the Client container is running.
kubectl get pod scalardl-client
[Command execution result]
NAME READY STATUS RESTARTS AGE
scalardl-client 1/1 Running 0 4s