`, as explained in the topology management overview. The public key is used to ultimately verify authorizations with respect to identities, as explained below.
How a party is exactly set up is defined by the set of topology transactions. These transactions need to be signed by all affected parties and validators and submitted to the ledger. The sum of all topology transactions defines the topology state, which is the shared state of knowledge about parties, keys, validators and packages on the ledger.
The topology state can evolve. A party is not tied to its initial configuration. However, replicating parties across nodes is an operational procedure between validator operators, not covered in this section.
### Hosting Relationships
To set up a party, a user must establish a **hosting relationship** with one or more validators. This hosting relationship is expressed through a topology transaction named **party to participant mapping** which need to be signed by the party and the validators involved. As part of this hosting relationship, the party defines which private key it will use to authorize transactions affecting its contracts. This private key of the party can be managed by the user, or one can delegate signing to the validator. This allows to distinguish two types of hosting setups, depending on how the authorizing key of the party is managed:
* **External parties**, where the authorizing private key is owned and operated by the user of the party, which may be an end user wallet or a backend application.
* **Internal parties**, where the validators key is used to authorize transactions on behalf of the party, while the user authenticates itself using a JWT token on the Ledger API.
Internal parties are easier to use from an operational perspective, as they don't require safe-keeping of an additional private key, but ultimately, they entrust the validators to not misuse the private key to authorize transactions. The hosting relationship between a party and a participant is defined using three different categories of validator permissions:
* **Submission**, where the validators signing key is used to authorize transactions
* **Confirmation**, where the validators signing key is only used to confirm transactions.
* **Observation**, where the validator is only informed and validates the state, but is not required to confirm.
Granting a validator **Submission** rights means that the party is an internal party. If the validators are granted **Confirmation** or **Observation** rights, then the party is an external party for which the authorizing key needs to be defined within the **party to participant mapping**.
Please continue with the following tutorial to learn how to onboard an external party using the Ledger API.
Internal parties are covered in the party management documentation.
### Multi-Hosted Parties
To reduce the need to fully trust any single validator, a party can configure its hosting relationship to require a specific number of validators to approve a transaction before it is considered valid. This defines another dimension of deployments:
* **Single hosted parties**, where a party is hosted by only a single validator. This is the simplest setup and is suitable for scenarios where the party owner fully trusts the validator.
* **Multi hosted parties**, where a party is hosted by multiple validators. This setup enhances security and availability by distributing trust across several validators.
Please follow the following tutorial to learn how to setup a multi-hosted party using the Ledger API
To summarize, in order for a party to be configured to use the ledger, the following topology state needs to be defined:
* The name of the party and the signing key the party will use to authorize transactions.
* Which set of validators will host the party with which permissions and threshold.
* If necessary, any namespace delegations required to verify the signatures on the topology transactions.
Note that the hosting choices have a significant impact on the security and availability of the party. Please refer to the trust assumptions outlined in the party trust model.
### Managing Topology via Admin API
The topology system is very flexible and powerful, but also complex. For topology transactions that require multiple independent actors to sign, the process of building, signing and submitting these transactions can be tedious. However, topology transactions can also be distributed as proposals through the ledger itself. A proposal is a topology transaction that has not yet been signed by all the necessary actors.
How to use the Admin API for various topology management tasks related to parties is covered in the following tutorials:
* Use the Admin API to onboard an external party
* Build, sign and submit topology transactions
### Transaction Submission
Once a party is set up, it can start using the ledger by submitting transactions that create contracts or exercise choices. For parties that authorize using an external key, the transaction submission process involves the following steps:
* Submit a command to a validator node of choice to interpret the command and produce the resulting transaction. The transaction is not sent to the ledger yet, but returned to the user.
* Validate and sign the transaction hash which is a commitment to the entire resulting transaction (command and result) with the party's private key.
* Submit the pre-computed transaction and the signature to a validator node for submission to the ledger.
* Observe the result of the transaction through the validator nodes hosting the party.
Please refer to the external signing overview for a more detailed explanation.
Note that the signature is a commitment to the entire transaction output, not just the command. If the interpreter was faulty, then the validators will reject the transaction. Submission can happen through any node, not just one that hosts the party.
How to submit commands using an external signing key is covered in the following tutorials: - Create a contract using external signing - Exercise a choice on a contract using external signing
# External Signing: Hashing Algorithm
Source: https://docs.canton.network/appdev/deep-dives/external-signing-hashing-algorithm
Deterministic hashing specification for prepared transactions used by external signing.
# External Signing Hashing Algorithm
## Introduction
This document specifies the encoding algorithm used to produce a deterministic hash of a `com.daml.ledger.api.v2.interactive.PreparedTransaction`. The resulting hash is signed by the holder of the external party's private key. The signature authorizes the ledger changes described by the transaction on behalf of the external party.
The specification can be implemented in any language, but certain encoding patterns are biased due to Canton being implemented in a JVM-based language and using the Java protobuf library. Those biases are made explicit in the specification.
Protobuf serialization is unsuitable for signing cryptographic hashes because it is not canonical. We must define a more precise encoding specification that can be re-implemented deterministically across languages and provide the required cryptographic guarantees. See [https://protobuf.dev/programming-guides/serialization-not-canonical/](https://protobuf.dev/programming-guides/serialization-not-canonical/) for more information on the topic.
## Versioning
### Hashing Scheme Version
The hashing algorithm as a whole is versioned. This enables updates to accommodate changes in the underlying Daml format, or, for instance, to the way the protocol verifies signatures. The implementation must respect the specification of the version it implements.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
// The hashing scheme version used when building the hash of the PreparedTransaction
enum HashingSchemeVersion {
HASHING_SCHEME_VERSION_UNSPECIFIED = 0;
reserved 1; // Hashing Scheme V1 - unsupported
HASHING_SCHEME_VERSION_V2 = 2;
HASHING_SCHEME_VERSION_V3 = 3;
}
```
The hashing algorithm is tied to the protocol version of the synchronizer used to synchronize the transaction. Specifically, each hashing scheme version is supported on one or several protocol versions. Implementations must use a hashing scheme version supported on the synchronizer on which the transaction is submitted.
| Protocol Version | Supported Hashing Schemes |
| ---------------- | ------------------------- |
| v34 | V2 |
| v35 | V2, V3 |
### Transaction Nodes
Transaction nodes are additionally individually versioned with a Daml version (also called LF version). The encoding version is decoupled from the LF version and implementations should only focus on the hashing version. However, new LF versions may introduce new fields in nodes or new node types. For that reason, the protobuf representation of a node is versioned to accommodate those future changes. In practice, every new Daml language version results in a new hashing version.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
message Node {
// Required
string node_id = 1;
// Versioned node
//
// Required
oneof versioned_node {
// Start at 1000 so we can add more fields before if necessary
// When new versions will be added, they will show here
//
// Required
interactive.transaction.v1.Node v1 = 1000;
}
}
```
## V3
### General approach
The hash of the `PreparedTransaction` is computed by encoding every protobuf field of the messages to byte arrays, and feeding those encoded values into a `SHA-256` hash builder. The rest of this section details how to deterministically encode every proto message into a byte array. Sometimes during the process, partially encoded results are hashed with SHA-256, and the resulting hash value serves as the encoding in messages further up. This is explicit when necessary.
Big Endian notation is used for numeric values. Furthermore, protobuf numeric values are encoded according to their Java type representation. Refer to the official protobuf documentation for more information about protobuf to Java type mappings: [https://protobuf.dev/programming-guides/proto3/#scalar](https://protobuf.dev/programming-guides/proto3/#scalar) In particular:
In Java, unsigned 32-bit and 64-bit integers are represented using their signed counterparts, with the top bit simply being stored in the sign bit
Additionally, this is the java library used under the hood in Canton to serialize and deserialize protobuf: [https://github.com/protocolbuffers/protobuf/tree/v3.25.5/java](https://github.com/protocolbuffers/protobuf/tree/v3.25.5/java)
### Changes from V2
* Addition of an `max_record_time` field in metadata to make maximum record time explicit in the signed metadata.
### Changes from V1
* Addition of an `interface_id` field in Fetch nodes for support of Daml interfaces.
* Addition of the hashing scheme version in the final hash to make the hash more robust to cross version collisions.
* Replace `ledger_effective_time` in the metadata with `min_ledger_effective_time` and `max_ledger_effective_time`.
> * These effectively replace a fixed ledger time with time bounds, allowing Daml Models to make assertions based on time without restricting the signing window as was required with a fixed set ledger time.
V3 introduces support for contract keys. Usage of contract keys in externally signed transactions requires usage of V3. Contract keys will not work on V2. Also note that V3 is only supported on protocol version 35.
### Notation and Utility Functions
* `encode`: Function that takes a protobuf message or primitive type `T` and transforms it into an array of bytes: `encode: T => byte[]`
e.g:
```
encode(false) = [0x00]
```
* `to_utf_8`: Function converting a Java `String` to its UTF-8 encoded version: `to_utf_8: string => byte[]`
e.g:
```
to_utf_8("hello") = [0x68, 0x65, 0x6c, 0x6c, 0x6f]
```
* `len`: Function returning the size of a collection (`array`, `list` etc...) as a signed 4 bytes integer: `len: Col => Int`
e.g:
```
len([4, 2, 8]) = 3
```
* `split`: Function converting a Java `String` to a list of `String`, by splitting the input using the provided delimiter: `split: (string, char) => byte[]`
e.g:
```
split("com.digitalasset.canton", '.') = ["com", "digitalasset", "canton"]
```
* `||`: Symbol representing concatenation of byte arrays
e.g:
```
[0x00] || [0x01] = [0x00, 0x01]
```
* `[]`: Empty byte array. Denotes that the value should not be encoded.
* `from_hex_string`: Function that takes a string in the hexadecimal format as input and decodes it as a byte array: `from_hex_string: string => byte[]`
e.g:
```
from_hex_string("08020a") = [0x08, 0x02, 0x0a]
```
* `int_to_string`: Function that takes an int and converts it to a string : `int_to_string: int => string`
e.g:
```
int_to_string(42) = "42"
```
* `some`: Value wrapped in a defined optional. Should be encoded as a defined optional value: `some: T => optional T`
e.g:
```
encode(some(5)) = 0x01 || encode(5)
```
See encoding of optional values below for details.
### Primitive Types
Unless otherwise specified, this is how primitive protobuf types should be encoded.
Not all protobuf types are described here, only the ones necessary to encode a `PreparedTransaction` message.
Even default values must be included in the encoding. For instance if an int32 field is not set in the serialized protobuf, its default value (0) should be encoded. Similarly, an empty repeated field still results in a `0x00` byte encoding (see the `repeated` section below for more details)
#### google.protobuf.Empty
```
fn encode(empty): 0x00
```
#### bool
```
fn encode(bool):
if (bool)
0x01
else
0x00
```
#### int64 - uint64 - sint64 - sfixed64
```
fn encode(long):
long # Java `Long` value equivalent: 8 bytes
```
e.g:
```
31380 (base 10) == 0x0000000000007a94
```
#### int32 - uint32 - sint32 - sfixed32
```
fn encode(int):
int # Java `Int` value equivalent: 4 bytes
```
e.g:
```
5 (base 10) == 0x00000005
```
#### bytes / byte\[]
```
fn encode(bytes):
encode(len(bytes)) || bytes
```
e.g
```
0x68656c6c6f ->
0x00000005 || # length
0x68656c6c6f # content
```
#### string
```
fn encode(string):
encode(to_utf8(string))
```
e.g
```
"hello" ->
0x00000005 || # length
0x68656c6c6f # utf-8 encoding of "hello"
```
### Collections / Wrappers
#### repeated
`repeated` protobuf fields represent an ordered collection of values of a specific message of type `T`. It is critical that the order of values in the list is not modified, both for the encoding process and in the protobuf itself when submitting the transaction for execution. Below is the pseudocode algorithm encoding a protobuf value `repeated T list;`
```
fn encode(list):
# prefix the result with the serialized length of the list
result = encode(len(list)) # (result is mutable)
# successively add encoded elements to the result, in order
for each element in list:
result = result || encode(element)
return result
```
This encoding function also applies to lists generated from utility functions (e.g: `split`).
#### optional
```
fn encode(optional):
if (is_set(optional))
0x01 || encode(optional.value)
else
0x00
```
`is_set` returns `true` if the value was set in the protobuf, `false` otherwise.
#### map
The ordering of `map` entries in protobuf serialization is not guaranteed, making it problematic for deterministic encoding. To address this, `repeated` values are used instead of `map` throughout the protobuf definitions.
### gRPC Ledger API Value
Encoding for the `Value` message defined in `com.daml.ledger.api.v2.value.proto` For clarity, all value types are exhaustively listed here. Each value is prefixed by a tag unique to its type, which is explicitly specified for each value below.
#### Unit
```
fn encode(unit):
0X00 # Unit Type Tag
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L27)
#### Bool
```
fn encode(bool):
0X01 || # Bool Type Tag
encode(bool) # Primitive boolean encoding
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L30)
#### Int64
```
fn encode(int64):
0X02 || # Int64 Type Tag
encode(int64) # Primitive int64 encoding
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L32)
#### Numeric
```
fn encode(numeric):
0X03 || # Numeric Type Tag
encode(numeric) # Primitive string encoding
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L52)
#### Timestamp
```
fn encode(timestamp):
0X04 || # Timestamp Type Tag
encode(timestamp) # Primitive sfixed64 encoding
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L45)
#### Date
```
fn encode(date):
0X05 || # Date Type Tag
encode(date) # Primitive int32 encoding
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L37)
#### Party
```
fn encode(party):
0X06 || # Party Type Tag
encode(party) # Primitive string encoding
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L56)
#### Text
```
fn encode(text):
0X07 || # Text Type Tag
encode(text) # Primitive string encoding
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L59)
#### Contract\_id
```
fn encode(contract_id):
0X08 || # Contract Id Type Tag
from_hex_string(contract_id) # Contract IDs are hexadecimal strings, so they need to be decoded as such. They should not be encoded as classic strings
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L63)
#### Optional
```
fn encode(optional):
if (optional.value is set)
0X09 || # Optional Type Tag
0x01 || # Defined optional
encode(optional.value)
else
0X09 || # Optional Type Tag
0x00 || # Undefined optional
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L66)
Note this is conceptually the same as for the primitive `optional` protobuf modifier, with the addition of the type tag prefix.
#### List
```
fn encode(list):
0X0a || # List Type Tag
encode(list.elements)
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L156-L160)
#### TextMap
```
fn encode(text_map):
0X0b || # TextMap Type Tag
encode(text_map.entries)
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L169-L176)
**TextMap.Entry**
```
fn encode(entry):
encode(entry.key) || encode(entry.value)
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L170C3-L173C4)
#### Record
```
fn encode(record):
0X0c || # Record Type Tag
encode(some(record.record_id)) ||
encode(record.fields)
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L87-L95)
**RecordField**
```
fn encode(record_field):
encode(some(record_field.label)) || encode(record_field.value)
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L98-L109)
#### Variant
```
fn encode(variant):
0X0d || # Variant Type Tag
encode(some(variant.variant_id)) ||
encode(variant.constructor) || encode(variant.value)
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L79)
#### Enum
```
fn encode(enum):
0X0e || # Enum Type Tag
encode(some(enum.enum_id)) ||
encode(enum.constructor)
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L82)
#### GenMap
```
fn encode(gen_map):
0X0f || # GenMap Type Tag
encode(gen_map.entries)
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L178-L185)
**GenMap.Entry**
```
fn encode(entry):
encode(entry.key) || encode(entry.value)
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L179-L182)
#### Identifier
```
fn encode(identifier):
encode(identifier.package_id) || encode(split(identifier.module_name, '.')) || encode(split(identifier.entity_name, '.')))
```
[Protobuf Definition](https://github.com/digital-asset/daml/blob/b5698e2327b83f7d9a5619395cd9b2de21509ab3/sdk/daml-lf/ledger-api-value/src/main/protobuf/com/daml/ledger/api/v2/value.proto#L112-L125)
## Transaction
A transaction is a forest (list of trees). It is represented with a following protobuf message found [here](https://github.com/digital-asset/daml/blob/ba14c4430b8345e7f0f8b20c3feead2b88c90fb8/sdk/canton/community/ledger-api-proto/src/main/protobuf/com/daml/ledger/api/v2/interactive/interactive_submission_service.proto#L283-L315).
The encoding function for a transaction is
```
fn encode(transaction):
encode(transaction.version) || encode_node_ids(transaction.roots)
```
`encode_node_ids(node_ids)` encodes lists in the same way as described before, except the encoding of a `node_id` is NOT done by encoding it as a string, but instead uses the following `encode(node_id)` function:
```
fn encode(node_id):
for node in nodes:
if node.node_id == node_id:
return sha_256(encode_node(node))
fail("Missing node") # All node ids should have a unique node in the nodes list. If a node is missing it should be reported as a bug.
```
`encode(node_id)` effectively finds the corresponding node in the list of nodes and encodes the node. The `node_id` is an opaque value only used to reference nodes and is itself never encoded. Additionally, each node's encoding is **hashed using the sha\_256 hashing algorithm**. This is relevant when encoding root nodes here as well as when recursively encoding sub-nodes of `Exercise` and `Rollback` nodes as seen below.
### Node
Each node's encoding is prefixed with additional meta-information about the node, this is made explicit in the encoding of each node.
`Exercise` and `Rollback` nodes both have a `children` field that references other nodes by their `NodeId`.
The following `find_seed: NodeId => optional bytes` function is used in the encoding:
```
fn find_seed(node_id):
for node_seed in node_seeds:
if int_to_string(node_seed.node_id) == node_id
return some(node_seed.seed)
return none
# There's no need to prefix the seed with its length because it has a fixed length. So its encoding is the identity function
fn encode_seed(seed):
seed
# Normal optional encoding, except the seed is encoded with `encode_seed`
fn encode_optional_seed(optional_seed):
if (is_some(optional_seed))
0x01 || encode_seed(optional_seed.get)
else
0x00
```
`some` represents a set optional field, `none` an empty optional field.
#### Create
```
fn encode_node(create):
0x01 || # Node encoding version
encode(create.lf_version) || # Node LF version
0x00 || # Create node tag
encode_optional_seed(find_seed(node.node_id)) ||
encode(create.contract_id) ||
encode(create.package_name) ||
encode(create.template_id) ||
encode(create.argument) ||
encode(create.signatories) ||
encode(create.stakeholders)
```
#### Exercise
```
fn encode_node(exercise):
0x01 || # Node encoding version
encode(exercise.lf_version) || # Node LF version
0x01 || # Exercise node tag
encode_seed(find_seed(node.node_id).get) ||
encode(exercise.contract_id) ||
encode(exercise.package_name) ||
encode(exercise.template_id) ||
encode(exercise.signatories) ||
encode(exercise.stakeholders) ||
encode(exercise.acting_parties) ||
encode(exercise.interface_id) ||
encode(exercise.choice_id) ||
encode(exercise.chosen_value) ||
encode(exercise.consuming) ||
encode(exercise.exercise_result) ||
encode(exercise.choice_observers) ||
encode(exercise.children)
```
For Exercise nodes, the node seed **MUST** be defined. Therefore it is encoded as a **non** optional field, as noted via the `.get` in `find_seed(node.node_id).get`. If the seed of an exercise node cannot be found in the list of `node_seeds`, encoding must be stopped and it should be reported as a bug.
The last encoded value of the exercise node is its `children` field. This recursively traverses the transaction tree.
#### Fetch
```
fn encode_node(fetch):
0x01 || # Node encoding version
encode(fetch.lf_version) || # Node LF version
0x02 || # Fetch node tag
encode(fetch.contract_id) ||
encode(fetch.package_name) ||
encode(fetch.template_id) ||
encode(fetch.signatories) ||
encode(fetch.stakeholders) ||
encode(fetch.interface_id) ||
encode(fetch.acting_parties)
```
#### Rollback
```
fn encode_node(rollback):
0x01 || # Node encoding version
0x03 || # Rollback node tag
encode(rollback.children)
```
Rollback nodes do not have an lf version.
#### Transaction Hash
Once the transaction is encoded, the hash is obtained by running `sha_256` over the encoded byte array, with a hash purpose prefix:
```
fn hash(transaction):
sha_256(
0x00000030 || # Hash purpose
encode(transaction)
)
```
## Metadata
The final part of `PreparedTransaction` is metadata. Note that all fields of the metadata need to be signed. Only some fields contribute to the ledger change triggered by the transaction. The rest of the fields are required by the Canton protocol but either have no impact on the ledger change, or have already been signed indirectly by signing the transaction itself.
```
fn encode(metadata, prepare_submission_request):
0x01 || # Metadata Encoding Version
encode(metadata.submitter_info.act_as) ||
encode(metadata.submitter_info.command_id) ||
encode(metadata.transaction_uuid) ||
encode(metadata.mediator_group) ||
encode(metadata.synchronizer_id) ||
encode(metadata.min_ledger_effective_time) ||
encode(metadata.max_ledger_effective_time) ||
encode(metadata.submission_time) ||
encode(metadata.disclosed_events) ||
encode(metadata.max_record_time)
```
### ProcessedDisclosedContract
```
fn encode(processed_disclosed_contract):
encode(processed_disclosed_contract.created_at) ||
encode(processed_disclosed_contract.contract)
```
### Metadata Hash
Once the metadata is encoded, the hash is obtained by running `sha_256` over the encoded byte array, with a hash purpose prefix:
```
fn hash(metadata):
sha_256(
0x00000030 || # Hash purpose
encode(metadata)
)
```
## Final Hash
Finally, compute the hash that needs to be signed to commit to the ledger changes.
```
fn encode(prepared_transaction):
0x00000030 || # Hash purpose
0x03 || # Hashing Scheme Version
hash(transaction) ||
hash(metadata)
```
```
fn hash(prepared_transaction):
sha_256(encode(prepared_transaction))
```
This resulting hash must be signed with the protocol signing private key(s) used to onboard the external party. Both the signature along with the `PreparedTransaction` must be sent to the API to submit the transaction to the ledger.
## Example
Example V3 implementation in Python
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Copyright (c) 2026 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Implements V3 of the transaction hashing specification.
# V3 removes node/metadata encoding version prefixes, adds key_opt to Create/Exercise/Fetch,
# adds by_key to Exercise/Fetch, introduces QueryByKey node type, and adds max_record_time to metadata.
import com.daml.ledger.api.v2.interactive.interactive_submission_service_pb2 as interactive_submission_service_pb2
from daml_transaction_hashing_common import (
PREPARED_TRANSACTION_HASH_PURPOSE,
encode_bool,
encode_hex_string,
encode_identifier,
encode_int32,
encode_int64,
encode_node_id,
encode_optional,
encode_proto_optional,
encode_repeated,
encode_string,
encode_hash,
encode_value,
encode_transaction,
find_seed,
sha256,
create_nodes_dict,
)
# Version of the hashing scheme implemented in this file as a byte
HASHING_SCHEME_VERSION_V3 = (
interactive_submission_service_pb2.HashingSchemeVersion.HASHING_SCHEME_VERSION_V3
)
# Byte version for the encoding (\x03)
HASHING_SCHEME_VERSION = HASHING_SCHEME_VERSION_V3.to_bytes(
length=1, byteorder="big", signed=False
)
def encode_key_with_maintainers(key_with_maintainers):
"""Encodes a GlobalKeyWithMaintainers composite value."""
return (
encode_string(key_with_maintainers.key.package_name)
+ encode_identifier(key_with_maintainers.key.template_id)
+ encode_value(key_with_maintainers.key.key)
+ encode_hash(key_with_maintainers.key.hash)
+ encode_repeated(key_with_maintainers.maintainers, encode_string)
)
def encode_prepared_transaction(
prepared_transaction: interactive_submission_service_pb2.PreparedTransaction,
nodes_dict: dict,
):
transaction_hash = hash_transaction(prepared_transaction.transaction, nodes_dict)
metadata_hash = hash_metadata(prepared_transaction.metadata)
return sha256(
PREPARED_TRANSACTION_HASH_PURPOSE
+ HASHING_SCHEME_VERSION
+ transaction_hash
+ metadata_hash
)
def hash_transaction(
transaction: interactive_submission_service_pb2.DamlTransaction, nodes_dict: dict
):
encoded_transaction = encode_transaction(
transaction, nodes_dict, transaction.node_seeds, encode_node
)
return sha256(PREPARED_TRANSACTION_HASH_PURPOSE + encoded_transaction)
def encode_node(
node,
nodes_dict: dict,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
node_id = node.node_id
if node.HasField("v1"):
return encode_node_v1(node.v1, node_id, nodes_dict, node_seeds)
raise ValueError("Unsupported node version")
def encode_node_v1(
node,
node_id,
nodes_dict: dict,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
if node.HasField("create"):
return encode_create_node(node.create, node_id, node_seeds)
elif node.HasField("exercise"):
return encode_exercise_node(node.exercise, node_id, nodes_dict, node_seeds)
elif node.HasField("fetch"):
return encode_fetch_node(node.fetch, node_id)
elif node.HasField("rollback"):
return encode_rollback_node(node.rollback, node_id, nodes_dict, node_seeds)
elif node.HasField("query_by_key"):
return encode_query_by_key_node(node.query_by_key)
raise ValueError("Unsupported node type")
def encode_create_node(
create,
node_id,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
return (
encode_string(create.lf_version)
+ b"\x00" # Create node tag
+ encode_optional(find_seed(node_id, node_seeds), encode_hash)
+ encode_hex_string(create.contract_id)
+ encode_string(create.package_name)
+ encode_identifier(create.template_id)
+ encode_value(create.argument)
+ encode_repeated(create.signatories, encode_string)
+ encode_repeated(create.stakeholders, encode_string)
+ encode_proto_optional(create, "key", create.key, encode_key_with_maintainers)
)
def encode_exercise_node(
exercise,
node_id,
nodes_dict: dict,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
return (
encode_string(exercise.lf_version)
+ b"\x01" # Exercise node tag
+ encode_hash(find_seed(node_id, node_seeds))
+ encode_hex_string(exercise.contract_id)
+ encode_string(exercise.package_name)
+ encode_identifier(exercise.template_id)
+ encode_repeated(exercise.signatories, encode_string)
+ encode_repeated(exercise.stakeholders, encode_string)
+ encode_repeated(exercise.acting_parties, encode_string)
+ encode_proto_optional(
exercise, "interface_id", exercise.interface_id, encode_identifier
)
+ encode_string(exercise.choice_id)
+ encode_value(exercise.chosen_value)
+ encode_bool(exercise.consuming)
+ encode_proto_optional(
exercise, "exercise_result", exercise.exercise_result, encode_value
)
+ encode_repeated(exercise.choice_observers, encode_string)
+ encode_bool(exercise.by_key)
+ encode_proto_optional(exercise, "key", exercise.key, encode_key_with_maintainers)
+ encode_repeated(exercise.children, encode_node_id(nodes_dict, node_seeds, encode_node))
)
def encode_fetch_node(fetch, node_id):
return (
encode_string(fetch.lf_version)
+ b"\x02" # Fetch node tag
+ encode_hex_string(fetch.contract_id)
+ encode_string(fetch.package_name)
+ encode_identifier(fetch.template_id)
+ encode_repeated(fetch.signatories, encode_string)
+ encode_repeated(fetch.stakeholders, encode_string)
+ encode_proto_optional(
fetch, "interface_id", fetch.interface_id, encode_identifier
)
+ encode_repeated(fetch.acting_parties, encode_string)
+ encode_bool(fetch.by_key)
+ encode_proto_optional(fetch, "key", fetch.key, encode_key_with_maintainers)
)
def encode_rollback_node(
rollback,
node_id,
nodes_dict: dict,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
return (
b"\x03" # Rollback node tag
+ encode_repeated(rollback.children, encode_node_id(nodes_dict, node_seeds, encode_node))
)
def encode_query_by_key_node(query_by_key):
return (
encode_string(query_by_key.lf_version)
+ b"\x04" # QueryByKey node tag
+ encode_string(query_by_key.package_name)
+ encode_identifier(query_by_key.template_id)
+ encode_bool(query_by_key.exhaustive)
+ encode_key_with_maintainers(query_by_key.key)
+ encode_repeated(query_by_key.result, encode_hex_string)
)
def hash_metadata(metadata):
encoded_metadata = encode_metadata(metadata)
return sha256(PREPARED_TRANSACTION_HASH_PURPOSE + encoded_metadata)
def encode_metadata(metadata):
return (
encode_repeated(metadata.submitter_info.act_as, encode_string)
+ encode_string(metadata.submitter_info.command_id)
+ encode_string(metadata.transaction_uuid)
+ encode_int32(metadata.mediator_group)
+ encode_string(metadata.synchronizer_id)
+ encode_proto_optional(
metadata,
"min_ledger_effective_time",
metadata.min_ledger_effective_time,
encode_int64,
)
+ encode_proto_optional(
metadata,
"max_ledger_effective_time",
metadata.max_ledger_effective_time,
encode_int64,
)
+ encode_int64(metadata.preparation_time)
+ encode_repeated(metadata.input_contracts, encode_input_contract)
+ encode_proto_optional(metadata, "max_record_time", metadata.max_record_time, encode_int64)
)
def encode_disclosed_contract(contract):
return encode_int64(contract.created_at) + sha256(
encode_create_node(contract.contract, "unused_node_id", [])
)
def encode_input_contract(contract):
return encode_int64(contract.created_at) + sha256(
encode_create_node(contract.v1, "unused_node_id", [])
)
```
Example V2 implementation in Python
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Copyright (c) 2026 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Implements V2 of the transaction hashing specification defined in the README.md at https://github.com/digital-asset/canton/blob/main/community/ledger-api/src/release-line-3.2/protobuf/com/daml/ledger/api/v2/interactive/README.md
import com.daml.ledger.api.v2.interactive.interactive_submission_service_pb2 as interactive_submission_service_pb2
from daml_transaction_hashing_common import (
PREPARED_TRANSACTION_HASH_PURPOSE,
encode_bool,
encode_hex_string,
encode_identifier,
encode_int32,
encode_int64,
encode_node_id,
encode_optional,
encode_proto_optional,
encode_repeated,
encode_string,
encode_hash,
encode_value,
encode_transaction,
find_seed,
sha256,
create_nodes_dict,
)
# Version of the hashing scheme implemented in this file as a byte
HASHING_SCHEME_VERSION_V2 = (
interactive_submission_service_pb2.HashingSchemeVersion.HASHING_SCHEME_VERSION_V2
)
# Byte version for the encoding (\x02)
HASHING_SCHEME_VERSION = HASHING_SCHEME_VERSION_V2.to_bytes(
length=1, byteorder="big", signed=False
)
# Version of the protobuf encoding the transaction nodes
NODE_ENCODING_VERSION = b"\x01"
def encode_prepared_transaction(
prepared_transaction: interactive_submission_service_pb2.PreparedTransaction,
nodes_dict: dict,
):
transaction_hash = hash_transaction(prepared_transaction.transaction, nodes_dict)
metadata_hash = hash_metadata(prepared_transaction.metadata)
return sha256(
PREPARED_TRANSACTION_HASH_PURPOSE
+ HASHING_SCHEME_VERSION
+ transaction_hash
+ metadata_hash
)
def hash_transaction(
transaction: interactive_submission_service_pb2.DamlTransaction, nodes_dict: dict
):
encoded_transaction = encode_transaction(
transaction, nodes_dict, transaction.node_seeds, encode_node
)
return sha256(PREPARED_TRANSACTION_HASH_PURPOSE + encoded_transaction)
def encode_node(
node,
nodes_dict: dict,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
node_id = node.node_id
if node.HasField("v1"):
return encode_node_v1(node.v1, node_id, nodes_dict, node_seeds)
raise ValueError("Unsupported node version")
def encode_node_v1(
node,
node_id,
nodes_dict: dict,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
if node.HasField("create"):
return encode_create_node(node.create, node_id, node_seeds)
elif node.HasField("exercise"):
return encode_exercise_node(node.exercise, node_id, nodes_dict, node_seeds)
elif node.HasField("fetch"):
return encode_fetch_node(node.fetch, node_id)
elif node.HasField("rollback"):
return encode_rollback_node(node.rollback, node_id, nodes_dict, node_seeds)
raise ValueError("Unsupported node type")
def encode_create_node(
create,
node_id,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
return (
NODE_ENCODING_VERSION
+ encode_string(create.lf_version)
+ b"\x00" # Create node tag
+ encode_optional(find_seed(node_id, node_seeds), encode_hash)
+ encode_hex_string(create.contract_id)
+ encode_string(create.package_name)
+ encode_identifier(create.template_id)
+ encode_value(create.argument)
+ encode_repeated(create.signatories, encode_string)
+ encode_repeated(create.stakeholders, encode_string)
)
def encode_exercise_node(
exercise,
node_id,
nodes_dict: dict,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
return (
NODE_ENCODING_VERSION
+ encode_string(exercise.lf_version)
+ b"\x01" # Exercise node tag
+ encode_hash(find_seed(node_id, node_seeds))
+ encode_hex_string(exercise.contract_id)
+ encode_string(exercise.package_name)
+ encode_identifier(exercise.template_id)
+ encode_repeated(exercise.signatories, encode_string)
+ encode_repeated(exercise.stakeholders, encode_string)
+ encode_repeated(exercise.acting_parties, encode_string)
+ encode_proto_optional(
exercise, "interface_id", exercise.interface_id, encode_identifier
)
+ encode_string(exercise.choice_id)
+ encode_value(exercise.chosen_value)
+ encode_bool(exercise.consuming)
+ encode_proto_optional(
exercise, "exercise_result", exercise.exercise_result, encode_value
)
+ encode_repeated(exercise.choice_observers, encode_string)
+ encode_repeated(exercise.children, encode_node_id(nodes_dict, node_seeds, encode_node))
)
def encode_fetch_node(fetch, node_id):
return (
NODE_ENCODING_VERSION
+ encode_string(fetch.lf_version)
+ b"\x02" # Fetch node tag
+ encode_hex_string(fetch.contract_id)
+ encode_string(fetch.package_name)
+ encode_identifier(fetch.template_id)
+ encode_repeated(fetch.signatories, encode_string)
+ encode_repeated(fetch.stakeholders, encode_string)
+ encode_proto_optional(
fetch, "interface_id", fetch.interface_id, encode_identifier
)
+ encode_repeated(fetch.acting_parties, encode_string)
)
def encode_rollback_node(
rollback,
node_id,
nodes_dict: dict,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
):
return (
NODE_ENCODING_VERSION
+ b"\x03" # Rollback node tag
+ encode_repeated(rollback.children, encode_node_id(nodes_dict, node_seeds, encode_node))
)
def hash_metadata(metadata):
encoded_metadata = encode_metadata(metadata)
return sha256(PREPARED_TRANSACTION_HASH_PURPOSE + encoded_metadata)
def encode_metadata(metadata):
return (
b"\x01" # Metadata encoding version
+ encode_repeated(metadata.submitter_info.act_as, encode_string)
+ encode_string(metadata.submitter_info.command_id)
+ encode_string(metadata.transaction_uuid)
+ encode_int32(metadata.mediator_group)
+ encode_string(metadata.synchronizer_id)
+ encode_proto_optional(
metadata,
"min_ledger_effective_time",
metadata.min_ledger_effective_time,
encode_int64,
)
+ encode_proto_optional(
metadata,
"max_ledger_effective_time",
metadata.max_ledger_effective_time,
encode_int64,
)
+ encode_int64(metadata.preparation_time)
+ encode_repeated(metadata.input_contracts, encode_input_contract)
)
def encode_input_contract(contract):
return encode_int64(contract.created_at) + sha256(
encode_create_node(contract.v1, "unused_node_id", [])
)
```
Both versions make use of the following common code:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Copyright (c) 2026 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
# Common encoding utilities shared between hashing scheme versions V2 and V3.
import com.daml.ledger.api.v2.interactive.interactive_submission_service_pb2 as interactive_submission_service_pb2
import hashlib
import struct
# Hash purpose reserved for prepared transaction
PREPARED_TRANSACTION_HASH_PURPOSE = b"\x00\x00\x00\x30"
def encode_bool(value):
return b"\x01" if value else b"\x00"
def encode_int32(value):
if not (-(2**31) <= value < 2**31):
raise ValueError(f"Value {value} out of range for int32")
return struct.pack(">i", value)
def encode_int64(value):
return struct.pack(">q", value)
def encode_string(value):
utf8_bytes = value.encode("utf-8")
return encode_bytes(utf8_bytes)
def encode_bytes(value):
length = encode_int32(len(value))
return length + value
# Like encode_bytes but without the length prefix, as hashes have a fixed size
def encode_hash(value):
return value
def encode_hex_string(value):
return encode_bytes(bytes.fromhex(value))
def encode_optional(value, encode_fn):
if value is not None:
return b"\x01" + encode_fn(value)
else:
return b"\x00"
def encode_proto_optional(parent_value, field_name, value, encode_fn):
if parent_value.HasField(field_name):
return b"\x01" + encode_fn(value)
else:
return b"\x00"
def encode_repeated(values, encode_fn):
length = encode_int32(len(values))
encoded_values = b"".join(encode_fn(v) for v in values)
return length + encoded_values
def sha256(data):
return hashlib.sha256(data).digest()
def find_seed(
node_id, node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed]
):
for node_seed in node_seeds:
if str(node_seed.node_id) == node_id:
return node_seed.seed
return None
def encode_identifier(identifier):
return (
encode_string(identifier.package_id)
+ encode_repeated(identifier.module_name.split("."), encode_string)
+ encode_repeated(identifier.entity_name.split("."), encode_string)
)
def encode_node_id(
nodes_dict: dict,
node_seeds: [interactive_submission_service_pb2.DamlTransaction.NodeSeed],
encode_node_fn,
):
def encode(node_id):
node = nodes_dict[node_id]
return sha256(encode_node_fn(node, nodes_dict, node_seeds))
return encode
def encode_transaction(transaction, nodes_dict, node_seeds, encode_node_fn):
version = encode_string(transaction.version)
roots = encode_repeated(
transaction.roots, encode_node_id(nodes_dict, node_seeds, encode_node_fn)
)
return version + roots
def encode_value(value):
if value.HasField("unit"):
return b"\x00"
elif value.HasField("bool"):
return b"\x01" + encode_bool(value.bool)
elif value.HasField("int64"):
return b"\x02" + encode_int64(value.int64)
elif value.HasField("numeric"):
return b"\x03" + encode_string(value.numeric)
elif value.HasField("timestamp"):
return b"\x04" + encode_int64(value.timestamp)
elif value.HasField("date"):
return b"\x05" + encode_int32(value.date)
elif value.HasField("party"):
return b"\x06" + encode_string(value.party)
elif value.HasField("text"):
return b"\x07" + encode_string(value.text)
elif value.HasField("contract_id"):
return b"\x08" + encode_hex_string(value.contract_id)
elif value.HasField("optional"):
return b"\x09" + encode_proto_optional(
value.optional, "value", value.optional.value, encode_value
)
elif value.HasField("list"):
return b"\x0a" + encode_repeated(value.list.elements, encode_value)
elif value.HasField("text_map"):
return b"\x0b" + encode_repeated(value.text_map.entries, encode_text_map_entry)
elif value.HasField("record"):
return (
b"\x0c"
+ encode_proto_optional(
value.record, "record_id", value.record.record_id, encode_identifier
)
+ encode_repeated(value.record.fields, encode_record_field)
)
elif value.HasField("variant"):
return (
b"\x0d"
+ encode_proto_optional(
value.variant, "variant_id", value.variant.variant_id, encode_identifier
)
+ encode_string(value.variant.constructor)
+ encode_value(value.variant.value)
)
elif value.HasField("enum"):
return (
b"\x0e"
+ encode_proto_optional(
value.enum, "enum_id", value.enum.enum_id, encode_identifier
)
+ encode_string(value.enum.constructor)
)
elif value.HasField("gen_map"):
return b"\x0f" + encode_repeated(value.gen_map.entries, encode_gen_map_entry)
raise ValueError("Unsupported value type")
def encode_text_map_entry(entry):
return encode_string(entry.key) + encode_value(entry.value)
def encode_record_field(field):
return encode_optional(field.label, encode_string) + encode_value(field.value)
def encode_gen_map_entry(entry):
return encode_value(entry.key) + encode_value(entry.value)
def create_nodes_dict(prepared_transaction):
nodes_dict = {}
for node in prepared_transaction.transaction.nodes:
nodes_dict[node.node_id] = node
return nodes_dict
```
# External Signing: Party Onboarding
Source: https://docs.canton.network/appdev/deep-dives/external-signing-onboarding
Onboard external parties for external signing using the Ledger API
# Onboard External Party
This tutorial demonstrates how to onboard an **external party** using the Ledger API.
## Prerequisites
This tutorial uses a script which is included as an example in the Canton artifact. Please note that the script uses openssl to create keys on the file system, which is not secure for production use.
To obtain a Canton artifact refer to the getting started section. From the artifact directory, start Canton using the command:
```
./bin/canton -c examples/08-interactive-submission/interactive-submission.conf --bootstrap examples/08-interactive-submission/bootstrap.canton
```
## Run The Script
The steps of this tutorial are included in the script `external_party_onboarding.sh` located in the `examples/08-interactive-submission` directory of the artifact. The steps covered by the script are:
* Create a private key using openssl for the external party.
* Determine the synchronizer-id available.
* Create a set of topology transactions to define a new external party.
* Sign the topology transactions.
* Upload the signed topology transactions to the Ledger API.
Make sure to run the script from the same directory where you started Canton such that the script can find the `canton_ports.json` file which contains the port configuration of the running Canton instance, or invoke the script with the hostname and port of the Ledger API using the command line argument `-p1 :`.
Once you start it, you will see:
```
./examples/08-interactive-submission/external_party_onboarding.sh
Fetching localhost:7374/v2/state/connected-synchronizers
Detected synchronizer-id "da::1220682ef8618b4425e8b1c5d7104260d5340eb4140509e99050a6bc9c5e8898d7b4"
Requesting generate topology transactions
Signing hash EiAfdSBLNQswwxUq9LyAYqHj8C5FzeZNLVvUJSgyrtORWg== for MyParty::1220ad82d8863893d65f10e2275a2f7b7af5c26cca97a761cb7cdc77d68e1ba20dc5 using ED25519
Submitting onboarding transaction to participant1
Onboarded party "MyParty::1220ad82d8863893d65f10e2275a2f7b7af5c26cca97a761cb7cdc77d68e1ba20dc5"
```
Note that the script supports a few command line arguments, which you can see by inspecting the code.
## The Details of the Script
First, the script determines the available synchronizer-ids using the `v2/connected-synchronizers` endpoint, assuming that there is exactly one. The party allocation must be repeated for each synchronizer-id the party should be hosted on.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
SYNCHRONIZER_ID=$(curl -f -s -L ${PARTICIPANT1}/v2/state/connected-synchronizers | jq .connectedSynchronizers.[0].synchronizerId)
```
Next, openssl is used to create a private Ed25519 key for the external party (other types of keys are supported as well). The public key is then extracted in DER format and convert the binary DER format to base64.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Generate an ed25519 private key and extract its public key
openssl genpkey -algorithm ed25519 -outform DER -out $PRIVATE_KEY_FILE
# Extract the public key from the private key
openssl pkey -in private_key.der -pubout -outform DER -out public_key.der 2> /dev/null
# Convert public key to base64
PUBLIC_KEY_BASE64=$(base64 -w 0 -i public_key.der)
```
The script uses the convenience endpoint `/v2/parties/external/generate-topology` to generate the topology transactions required to onboard the external party. This is fine if the node is trusted. In other scenarios, the transactions should be built manually or inspected before signing, including recomputing the hash.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Create the JSON payload to generate the onboarding transaction
# Note: otherConfirmingParticipantUids is optional but can be used to add other participants
# as confirming nodes. confirmationThreshold allows to configure the number of required confirmations.
# If not set, all confirming nodes must confirm.
GENERATE=$(cat << EOF
{
"synchronizer" : $SYNCHRONIZER_ID,
"partyHint" : "$PARTY_NAME",
"publicKey" : {
"format" : "CRYPTO_KEY_FORMAT_DER_X509_SUBJECT_PUBLIC_KEY_INFO",
"keyData": "$PUBLIC_KEY_BASE64",
"keySpec" : "SIGNING_KEY_SPEC_EC_CURVE25519"
},
"otherConfirmingParticipantUids" : [$OTHER_PARTICIPANT_UIDS]
}
EOF
)
# Submit it to the JSON API
ONBOARDING_TX=$(curl -f -s -d "$GENERATE" -H "Content-Type: application/json" \
-X POST ${PARTICIPANT1}/v2/parties/external/generate-topology)
```
The convenience endpoint returns the generated topology transactions together with the computed party-id for the new party and the fingerprint of the public key. In addition, it also returns a multi-hash, which is a commitment to the entire set of transactions.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
PARTY_ID=$(echo $ONBOARDING_TX | jq -r .partyId)
TRANSACTIONS=$(echo $ONBOARDING_TX | jq '.topologyTransactions | map({ transaction : .})')
PUBLIC_KEY_FINGERPRINT=$(echo $ONBOARDING_TX | jq -r .publicKeyFingerprint)
MULTI_HASH=$(echo -n $ONBOARDING_TX | jq -r .multiHash)
```
This hash needs to be signed by the private key of the new party. The script uses openssl to sign the hash and then converts the signature to base64.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
echo "Signing hash ${MULTI_HASH} for ${PARTY_ID} using ED25519"
echo -n $MULTI_HASH | base64 --decode > hash_binary.bin
openssl pkeyutl -sign -inkey $PRIVATE_KEY_FILE -rawin -in hash_binary.bin -out signature.bin -keyform DER
SIGNATURE=$(base64 -w 0 < signature.bin)
```
Using the signature and the data from the previous step, the script submits the topology transactions and the signature to the ledger API to complete the onboarding of the new external party:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
ALLOCATE=$(cat << EOF
{
"synchronizer" : $SYNCHRONIZER_ID,
"onboardingTransactions": $TRANSACTIONS,
"multiHashSignatures": [{
"format" : "SIGNATURE_FORMAT_CONCAT",
"signature": "$SIGNATURE",
"signedBy" : "$PUBLIC_KEY_FINGERPRINT",
"signingAlgorithmSpec" : "SIGNING_ALGORITHM_SPEC_ED25519"
}]
}
EOF
)
RESULT=$(curl -f -s -d "$ALLOCATE" -H "Content-Type: application/json" \
-X POST ${PARTICIPANT1}/v2/parties/external/allocate)
```
The transactions can be signed one by one, or together as one hash, as done in the script.
# Onboard Multi-Hosted External Party
This tutorial demonstrates how to onboard an **external party** using the Ledger API which is hosted on multiple validators. It is a simple extension to the onboard external party tutorial.
## Prerequisites
Make sure that you have completed the onboard external party tutorial and still have a running Canton example instance.
## Run The Script
The example script used in the previous tutorial also supports onboarding a multi-hosted external party. It will onboard by default on two nodes if invoked with the `--multi-hosted` command line argument.
```
./examples/08-interactive-submission/external_party_onboarding.sh --multi-hosted
```
## The Details of the Script
The flag `--multi-hosted` will pass the second participant id into the `generate-topology` request through the
```
`"otherConfirmingParticipantUids" : [$OTHER_PARTICIPANT_ID]`
```
field. This will cause the generated topology transaction to include the additional participant id in the hosting relation ship. Other options are fields such as `observingParticipantUids`, `confirmationThreshold` and more. If not configured, then the confirmation threshold will be set to the number of confirming nodes.
The generated topology transactions then just need to be uploaded to the Ledger API of the second participant:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
ALLOCATE=$(cat << EOF
{
"synchronizer" : $SYNCHRONIZER_ID,
"onboardingTransactions": $TRANSACTIONS,
"multiHashSignatures": [{
"format" : "SIGNATURE_FORMAT_CONCAT",
"signature": "$SIGNATURE",
"signedBy" : "$PUBLIC_KEY_FINGERPRINT",
"signingAlgorithmSpec" : "SIGNING_ALGORITHM_SPEC_ED25519"
}]
}
EOF
)
RESULT=$(curl -f -s -d "$ALLOCATE" -H "Content-Type: application/json" \
-X POST ${PARTICIPANT1}/v2/parties/external/allocate)
```
You can try this out on the Canton console if you have two participants connected to the same synchronizer. In the following example, you will use the participant1 to create the hosting proposal for an internal party. This way, you don't need to deal with creating signatures for the topology transactions externally. The approval of the proposal will be done using participant2.
First, create a hosting proposal using participant1:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.topology.party_to_participant_mappings.propose(
com.digitalasset.canton.topology.PartyId.tryCreate("Alice", participant1.id.uid.namespace),
newParticipants = Seq(
(participant1.id, ParticipantPermission.Confirmation),
(participant2.id, ParticipantPermission.Confirmation),
),
)
res1: SignedTopologyTransaction[TopologyChangeOp, PartyToParticipant] = SignedTopologyTransaction(
TopologyTransaction(
PartyToParticipant(
Alice::12201ff69b1d...,
PositiveNumeric(1),
Vector(
HostingParticipant(PAR::participant1::12201ff69b1d..., Confirmation, false),
HostingParticipant(PAR::participant2::1220a4d7463b..., Confirmation, false)
),
None
),
serial = 1,
operation = Replace,
hash = SHA-256:483ecaff7581...
),
signatures = 12201ff69b1d...,
proposal
)
```
Then, list the proposals on participant2. The new proposal should appear shortly:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant2.topology.party_to_participant_mappings.list_hosting_proposals(sequencer1.synchronizer_id, participant2.id)
res2: Seq[com.digitalasset.canton.admin.api.client.data.topology.ListMultiHostingProposal] = Vector(
ListMultiHostingProposal(
txHash = SHA-256:483ecaff7581...,
party = Alice::12201ff69b1d...,
permission = Confirmation$,
others = PAR::participant1::12201ff69b1d... -> Confirmation$,
threshold = 1
)
)
```
This will show the pending proposal, awaiting the signature of the second participant. The proposal is identified by the transaction hash `txHash`, which can be obtained from the output of the previous command:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ val txHash = participant2.topology.party_to_participant_mappings.list_hosting_proposals(sequencer1.synchronizer_id, participant2.id).head.txHash
txHash : TopologyTransaction.TxHash = TxHash(hash = SHA-256:483ecaff7581...)
```
Authorize the proposal using the console command topology.transactions.authorize:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant2.topology.transactions.authorize(sequencer1.synchronizer_id, txHash)
res4: SignedTopologyTransaction[TopologyChangeOp, TopologyMapping] = SignedTopologyTransaction(
TopologyTransaction(
PartyToParticipant(
Alice::12201ff69b1d...,
PositiveNumeric(1),
Vector(
HostingParticipant(PAR::participant1::12201ff69b1d..., Confirmation, false),
HostingParticipant(PAR::participant2::1220a4d7463b..., Confirmation, false)
),
None
),
serial = 1,
operation = Replace,
hash = SHA-256:483ecaff7581...
),
signatures = Seq(12201ff69b1d..., 1220a4d7463b...),
proposal
)
```
This will add the signature of participant2 to the proposal. Because the proposal is now fully signed, the party will appear as being hosted on both nodes:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.parties.hosted("Alice")
res5: Seq[ListPartiesResult] = Vector(
ListPartiesResult(
party = Alice::12201ff69b1d...,
participants = Vector(
ParticipantSynchronizers(
participant = PAR::participant1::12201ff69b1d...,
synchronizers = Vector(
SynchronizerPermission(synchronizerId = local::122032922613..., permission = Confirmation)
)
),
ParticipantSynchronizers(
participant = PAR::participant2::1220a4d7463b...,
synchronizers = Vector(
SynchronizerPermission(synchronizerId = local::122032922613..., permission = Confirmation)
)
)
)
)
)
```
# External Signing: Topology Transactions
Source: https://docs.canton.network/appdev/deep-dives/external-signing-topology
Build, sign, and submit topology transactions with external keys
# Externally Signed Topology Transactions
Canton's [Topology](/overview/reference/topology) formalizes a shared state on a synchronizer and provides a secure, distributed mechanism for modifying this state.
This tutorial demonstrates how to build, sign, and submit topology transactions. It is particularly useful for cases where the signature is provided by a key held externally to the network, such as in the case of external party onboarding, or initialization of the root namespace of a participant. This tutorial goes through the steps of importing a root namespace delegation, but can be generalized to any topology mapping. A root namespace delegation is essentially equivalent to a X509v3 CA root certificate in Canton and creates an associated namespace.
This tutorial is for demo purposes. The code snippets should not be used directly in a production environment.
## Prerequisites
For simplicity, this tutorial assumes a minimal Canton setup consisting of one participant node connected to one synchronizer (which includes both a sequencer node and a mediator node).
### Start Canton
To obtain a Canton artifact refer to the getting started section. From the artifact directory, start Canton using the command:
```
./bin/canton -c examples/01-simple-topology/simple-topology.conf --bootstrap examples/01-simple-topology/simple-ping.canton
```
Once the "Welcome to Canton" message appears, you are ready to proceed.
### Setup
Navigate to the interactive submission example folder located at `examples/08-interactive-submission` in the Canton release artifact.
The code examples in this tutorial are extracted from scripts located in that folder.
To proceed, gather the following information by running the commands below in the Canton console:
* Admin API endpoint
* Synchronizer ID
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.config.adminApi.address
res1: String = "127.0.0.1"
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.config.adminApi.port.unwrap
res2: Int = 30044
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ sequencer1.synchronizer_id.toProtoPrimitive
res3: String = "da::1220a82692abc55c0367abefc4bdbc23df25688230430ddfeef5759845f26d5cc29c"
```
In the rest of the tutorial we use the following values, but make sure to replace them with your own:
* Admin API endpoint: `localhost:4002`
* Synchronizer ID: `da::12207a94aca813c822c6ae10a1b5478c2ba1077447b468cc66dbd255f60f8fa333e1`
### API
This tutorial interacts with the `TopologyManagerWriteService`, a gRPC service available on the **Admin API** of the participant node. It assumes that the Admin API is not authenticated via client certificates.
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Copyright (c) 2025 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
syntax = "proto3";
package com.digitalasset.canton.topology.admin.v30;
import "com/digitalasset/canton/protocol/v30/topology.proto";
import "com/digitalasset/canton/topology/admin/v30/common.proto";
import "google/protobuf/duration.proto";
/**
* Write operations on the local topology manager.
*
* Participants, mediators, and sequencers run a local topology manager exposing the same write interface.
*/
service TopologyManagerWriteService {
rpc Authorize(AuthorizeRequest) returns (AuthorizeResponse);
rpc AddTransactions(AddTransactionsRequest) returns (AddTransactionsResponse);
rpc ImportTopologySnapshot(stream ImportTopologySnapshotRequest) returns (ImportTopologySnapshotResponse);
rpc ImportTopologySnapshotV2(stream ImportTopologySnapshotV2Request) returns (ImportTopologySnapshotV2Response);
rpc SignTransactions(SignTransactionsRequest) returns (SignTransactionsResponse);
/** RPC to generate topology transactions that can be signed */
rpc GenerateTransactions(GenerateTransactionsRequest) returns (GenerateTransactionsResponse);
/** Creates a temporary topology store.
* Trying to create a store with the same name results in an error.
*/
rpc CreateTemporaryTopologyStore(CreateTemporaryTopologyStoreRequest) returns (CreateTemporaryTopologyStoreResponse);
/** Drops a temporary topology store.
* Trying to drop a temporary store that does not exist results in an error.
*/
rpc DropTemporaryTopologyStore(DropTemporaryTopologyStoreRequest) returns (DropTemporaryTopologyStoreResponse);
}
message GenerateTransactionsRequest {
message Proposal {
/** Replace / Remove */
com.digitalasset.canton.protocol.v30.Enums.TopologyChangeOp operation = 1;
/** Optionally, the serial number of this request (auto-determined if omitted)
* NOTE: omitting the serial MAY end up overwriting previous mappings processed concurrently.
* To avoid such cases, First read the state using the TopologyManagerReadService and update the mappings
* accordingly, incrementing the serial by one and setting it here explicitly.
*/
uint32 serial = 2;
/** The mapping to be authorized */
com.digitalasset.canton.protocol.v30.TopologyMapping mapping = 3;
// Target store
StoreId store = 4;
}
// transaction proposals for which to generate topology transactions
repeated Proposal proposals = 1;
}
message GenerateTransactionsResponse {
message GeneratedTransaction {
// Serialized com.digitalasset.canton.protocol.v30.TopologyTransaction
bytes serialized_transaction = 1;
// Hash of the transaction - this should be signed by the submitter to authorize the transaction
bytes transaction_hash = 2;
}
// Generated transactions, in the same order as the mappings provided in the request
repeated GeneratedTransaction generated_transactions = 1;
}
message AuthorizeRequest {
message Proposal {
/** Replace / Remove */
com.digitalasset.canton.protocol.v30.Enums.TopologyChangeOp change = 1;
/** Optionally, the serial number of this request (auto-determined if omitted) */
uint32 serial = 2;
/** The mapping to be authorized */
com.digitalasset.canton.protocol.v30.TopologyMapping mapping = 3;
}
oneof type {
/**
* Propose a transaction and distribute it.
* If authorize if the node has enough signing keys
*/
Proposal proposal = 1;
/**
* Authorize a transaction, meaning the node needs to be able to fully sign it locally.
* Hash is in hexadecimal format.
*/
string transaction_hash = 2;
}
/**
* If true: the transaction is only signed if the new signatures will result in the transaction being fully
* authorized. Otherwise returns as an error.
* If false: the transaction is signed and the signature distributed. The transaction may still not be fully
* authorized and remain as a proposal.
*/
bool must_fully_authorize = 3;
/** Force specific changes even if dangerous */
repeated ForceFlag force_changes = 4;
/**
* Fingerprint of the keys signing the authorization
*
* The signing key is used to identify a particular `NamespaceDelegation` certificate,
* which is used to justify the given authorization.
* Optional, if empty, suitable signing keys available known to the node are automatically selected.
*/
repeated string signed_by = 5;
/**
* The store that is used as the underlying source for executing this request.
* If `store` is a synchronizer store, the resulting topology transaction will only be available on the respective synchronizer.
* If `store` is the authorized store, the resulting topology transaction may or may not be synchronized automatically
* to all synchronizers that the node is currently connected to or will be connected to in the future.
*
* Selecting a specific synchronizers store might be necessary, if the transaction to authorize by hash or the previous
* generation of the submitted proposal is only available on the synchronizers store and not in the authorized store.
*/
StoreId store = 6;
/** Optional timeout to wait for the transaction to become effective in the store. */
google.protobuf.Duration wait_to_become_effective = 7;
}
message AuthorizeResponse {
/** the generated signed topology transaction */
com.digitalasset.canton.protocol.v30.SignedTopologyTransaction transaction = 1;
}
message AddTransactionsRequest {
/**
* The transactions that should be added to the target store as indicated by the parameter `store`.
*/
repeated com.digitalasset.canton.protocol.v30.SignedTopologyTransaction transactions = 1;
/** Force specific changes even if dangerous */
repeated ForceFlag force_changes = 2;
/**
* The store that is used as the underlying source for executing this request.
* If `store` is a synchronizers store, the resulting topology transaction will only be available on the respective synchronizers.
* If `store` is the authorized store, the resulting topology transaction may or may not be synchronized automatically
* to all synchronizers that the node is currently connected to or will be connected to in the future.
*
* Selecting a specific synchronizers store might be necessary, if the transaction to authorize by hash or the previous
* generation of the submitted proposal is only available on the synchronizers store and not in the authorized store.
*/
StoreId store = 3;
/** Optional timeout to wait for the transaction to become effective in the store. */
google.protobuf.Duration wait_to_become_effective = 7;
}
message AddTransactionsResponse {}
/**
* Same message as AddTransactionsRequest, except that transactions are encoded in a byte string
*/
message ImportTopologySnapshotRequest {
bytes topology_snapshot = 1;
StoreId store = 2;
/** Optional timeout to wait for the transaction to become effective in the store. */
google.protobuf.Duration wait_to_become_effective = 3;
}
message ImportTopologySnapshotResponse {}
/**
* Same message as AddTransactionsRequest, except that transactions are encoded in a byte string
*/
message ImportTopologySnapshotV2Request {
bytes topology_snapshot = 1;
StoreId store = 2;
/** Optional timeout to wait for the transaction to become effective in the store. */
google.protobuf.Duration wait_to_become_effective = 3;
}
message ImportTopologySnapshotV2Response {}
message SignTransactionsRequest {
/** The transactions to be signed, but will not be stored in the authorized store */
repeated com.digitalasset.canton.protocol.v30.SignedTopologyTransaction transactions = 1;
/**
* Fingerprint of the keys signing the authorization
*
* The signing key is used to identify a particular `NamespaceDelegation` certificate,
* which is used to justify the given authorization.
* Optional, if empty, suitable signing keys available known to the node are automatically selected.
*/
repeated string signed_by = 2;
// Target store
StoreId store = 3;
/** Force specific changes even if dangerous */
repeated ForceFlag force_flags = 4;
}
message SignTransactionsResponse {
/** The transactions with the additional signatures from this node. */
repeated com.digitalasset.canton.protocol.v30.SignedTopologyTransaction transactions = 1;
}
message CreateTemporaryTopologyStoreRequest {
/** The name of the topology store */
string name = 1;
/** The protocol version that should be used by the store */
uint32 protocol_version = 2;
}
message CreateTemporaryTopologyStoreResponse {
/** The identifier of the topology store that should be used as a store filter string */
StoreId.Temporary store_id = 1;
}
message DropTemporaryTopologyStoreRequest {
/** The identifier of the topology store that should be dropped */
StoreId.Temporary store_id = 1;
}
message DropTemporaryTopologyStoreResponse {}
enum ForceFlag {
FORCE_FLAG_UNSPECIFIED = 0;
/** Required when authorizing adding a topology transaction on behalf of another node. */
FORCE_FLAG_ALIEN_MEMBER = 1;
/* Deprecated, increasing ledger time record time tolerance does not require a force flag for PV >= 32 */
FORCE_FLAG_LEDGER_TIME_RECORD_TIME_TOLERANCE_INCREASE = 2;
// Previously FORCE_FLAG_ALLOW_UNVET_PACKAGE, now always enabled as it is not dangerous anymore
reserved 3;
reserved "FORCE_FLAG_ALLOW_UNVET_PACKAGE";
/** Required when vetting unknown packages (not uploaded). */
FORCE_FLAG_ALLOW_UNKNOWN_PACKAGE = 4;
/** Required when vetting a package with unvetted dependencies */
FORCE_FLAG_ALLOW_UNVETTED_DEPENDENCIES = 5;
/** Required when disabling a party with active contracts */
FORCE_FLAG_DISABLE_PARTY_WITH_ACTIVE_CONTRACTS = 6;
/**
* Required when using a key that is not suitable to sign a topology transaction.
* Using this force flag likely causes the transaction to be rejected at a later stage of the processing.
*/
FORCE_FLAG_ALLOW_UNVALIDATED_SIGNING_KEYS = 7;
// Previously FORCE_FLAG_ALLOW_UNVET_PACKAGE_WITH_ACTIVE_CONTRACTS, now allowed without flag as it is not a dangerous operation anymore.
reserved 8;
reserved "FORCE_FLAG_ALLOW_UNVET_PACKAGE_WITH_ACTIVE_CONTRACTS";
/** Required when increasing the submission time record time tolerance */
FORCE_FLAG_PREPARATION_TIME_RECORD_TIME_TOLERANCE_INCREASE = 9;
/** Required when we want to change all participants' permissions to observation while the party is still a signatory of a contract. */
FORCE_FLAG_ALLOW_INSUFFICIENT_PARTICIPANT_PERMISSION_FOR_SIGNATORY_PARTY = 10;
/** Required when changing the party-to-participant mapping, that would result in insufficient
* signatory-assigning participants and thus the assignment would be stuck.
*/
FORCE_FLAG_ALLOW_INSUFFICIENT_SIGNATORY_ASSIGNING_PARTICIPANTS_FOR_PARTY = 11;
/** Required when vetting a package that fails upgrade checking */
FORCE_FLAG_ALLOW_VET_INCOMPATIBLE_UPGRADES = 12;
/** Required when submitting dynamic synchronizer parameters that have out-of-bounds values */
FORCE_FLAG_ALLOW_OUT_OF_BOUNDS_VALUE = 13;
/** Required when changing the confirming threshold to a value higher than the number of confirming participants */
FORCE_FLAG_ALLOW_CONFIRMING_THRESHOLD_CANNOT_BE_MET = 14;
}
```
### Python
It is recommended to use a dedicated python environment to avoid conflicting dependencies. Considering using [venv](https://docs.python.org/3/library/venv.html).
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
Then run the setup script to generate the necessary python files to interact with Canton's gRPC interface:
```
./setup.sh
```
Finally, the following imports will be needed:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePrivateKey
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives import hashes
from grpc import Channel
from com.digitalasset.canton.topology.admin.v30 import (
topology_manager_write_service_pb2_grpc,
topology_manager_read_service_pb2_grpc,
)
from com.digitalasset.canton.topology.admin.v30 import (
topology_manager_write_service_pb2,
topology_manager_read_service_pb2,
common_pb2,
)
from com.digitalasset.canton.protocol.v30 import topology_pb2
from com.digitalasset.canton.version.v1 import untyped_versioned_message_pb2
from com.digitalasset.canton.crypto.v30 import crypto_pb2
from google.rpc import status_pb2, error_details_pb2
from google.protobuf import empty_pb2
from google.protobuf.json_format import MessageToJson
import hashlib
import grpc
```
### Shell
For a terminal-based approach, install the following tools:
* [openssl](https://www.openssl.org/)
* [buf](https://buf.build/docs/cli/installation/)
* [jq](https://jqlang.org/)
* [xxd](https://linux.die.net/man/1/xxd)
The tutorial uses a buf proto image to (de)serialize proto messages.
> ```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
> CURRENT_DIR=$(cd "$(dirname "${BASH_SOURCE[0]}")" > /dev/null && pwd)
> BUF_PROTO_IMAGE="$CURRENT_DIR/interactive_topology_buf_image.json.gz"
> ```
>
> ```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
> ROOT_PATH="../../protobuf"
> ```
The following functions will be used throughout the tutorial:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Encode bytes read from stdin to base64
encode_to_base64() {
openssl base64 -e -A
}
# Decode base64 string to bytes
decode_from_base64() {
openssl base64 -d
}
# Encode bytes read from stdin to hexadecimal
encode_to_hex() {
xxd -ps -c 0
}
```
### Error Handling
When encountering RPC errors, it may be necessary to perform additional deserialization to get actionable information on the cause of the error. Example of an RPC error:
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"code": "invalid_argument",
"message": "PROTO_DESERIALIZATION_FAILURE(8,0): Deserialization of protobuf message failed",
"details": [
{
"type": "google.rpc.ErrorInfo",
"value": "Ch1QUk9UT19ERVNFUklBTElaQVRJT05fRkFJTFVSRRobCgtwYXJ0aWNpcGFudBIMcGFydGljaXBhbnQxGlQKBnJlYXNvbhJKVmFsdWVDb252ZXJzaW9uRXJyb3Ioc3RvcmUsRW1wdHkgc3RyaW5nIGlzIG5vdCBhIHZhbGlkIHVuaXF1ZSBpZGVudGlmaWVyLikaDQoIY2F0ZWdvcnkSATg"
}
]
}
```
The `type` field specifies the protobuf type in which the error is encoded. In this case, it is a `google.rpc.ErrorInfo` message. The following utility code can be used to deal with errors and extract useful information out of them.
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Make an RPC call with the given request.
# Arguments:
# $1 - JSON request string
# $2 - RPC endpoint URL
make_rpc_call() {
local request=$1
local rpc=$2
echo -n "$request" | buf curl --protocol grpc --http2-prior-knowledge -d @- "$rpc" 2>&1
}
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
handle_rpc_error() {
local response="$1"
local details
local type
echo "Request failed"
# Extract the first element from the details field using jq
details=$(echo "$response" | jq -r '.details[0].value // empty')
type=$(echo "$response" | jq -r '.details[0].type // empty')
if [ -n "$details" ] && [ "$type" = "google.rpc.ErrorInfo" ]; then
# Decode the base64 value and save it to a file
echo "$details" | base64 -d > error_info.bin
# Download the error info proto if it doesn't exist
if [ ! -f "google/rpc/error_details.proto" ]; then
mkdir -p "google/rpc"
curl -s "https://raw.githubusercontent.com/googleapis/googleapis/9415ba048aa587b1b2df2b96fc00aa009c831597/google/rpc/error_details.proto" -o "google/rpc/error_details.proto"
fi
# Deserialize the protobuf message using buf convert
buf convert google/rpc/error_details.proto --from error_info.bin --to - --type google.rpc.ErrorInfo | jq .
else
echo "No details available in the response or type is not google.rpc.ErrorInfo."
fi
}
```
Python
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def handle_grpc_error(func):
"""
Decorator to handle gRPC errors and print detailed error information.
Args:
func (function): The gRPC function to be wrapped.
Returns:
function: Wrapped function with error handling.
"""
def wrapper(*args, **kwargs):
try:
return func(*args, **kwargs)
except grpc.RpcError as e:
print("gRPC error occurred:")
grpc_metadata: grpc.aio.Metadata = grpc.aio.Metadata.from_tuple(
e.trailing_metadata()
)
metadata = grpc_metadata.get("grpc-status-details-bin")
if metadata is None:
raise
status: status_pb2.Status = status_pb2.Status.FromString(metadata)
for detail in status.details:
if detail.type_url == "type.googleapis.com/google.rpc.ErrorInfo":
error: error_details_pb2.ErrorInfo = (
error_details_pb2.ErrorInfo.FromString(detail.value)
)
print(MessageToJson(error))
else:
print(MessageToJson(detail))
raise
return wrapper
```
## 1. Signing Keys
First, generate an external signing key pair to use in the rest of this tutorial.
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Generate an ECDSA private key and extract its public key
openssl ecparam -name prime256v1 -genkey -noout -outform DER -out namespace_private_key.der
openssl ec -inform der -in namespace_private_key.der -pubout -outform der -out namespace_public_key.der 2> /dev/null
```
Python
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
private_key = ec.generate_private_key(curve=ec.SECP256R1())
public_key = private_key.public_key()
```
## 2. Hash
Hashing is required at several steps to compute a hash over a sequence of bytes. The process uses an underlying algorithm, with specific prefixes added to both the input bytes and the final hash:
1. A hash purpose (a 4-byte integer) is prefixed to the byte sequence. Hash purpose values are defined directly in the Canton codebase.
> >
>
> ```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
> // Copyright (c) 2025 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
> // SPDX-License-Identifier: Apache-2.0
>
> package com.digitalasset.canton.crypto
>
> import scala.collection.mutable
>
> /** The purpose of a hash serves to avoid hash collisions due to equal encodings for different
> * objects. It is in general not possible to derive the purpose of the hash from the hash alone.
> *
> * Whenever a hash is computed using [[HashOps]], a [[HashPurpose]] must be specified that gets
> * included in the hash. To reliably prevent hash collisions, every [[HashPurpose]] object should
> * be used only in a single place.
> *
> * All [[HashPurpose]] objects must be created through the [[HashPurpose$.apply]] method, which
> * checks that the id is fresh.
> *
> * @param id
> * The identifier for the [[HashPurpose]]. Every [[HashPurpose]] object must have a unique
> * [[id]].
> */
> class HashPurpose private (val id: Int) extends AnyVal
>
> object HashPurpose {
> private val ids: mutable.Map[Int, String] = mutable.TreeMap.empty[Int, String]
>
> /** Creates a new [[HashPurpose]] with a given description */
> def apply(id: Int, description: String): HashPurpose = {
> ids.put(id, description).foreach { oldDescription =>
> throw new IllegalArgumentException(
> s"requirement failed: HashPurpose with id=$id already exists for $oldDescription"
> )
> }
>
> new HashPurpose(id)
> }
>
> /** Returns the description that was given when the hash purpose was created. */
> def description(hashPurpose: HashPurpose): String =
> ids.getOrElse(
> hashPurpose.id,
> throw new IllegalStateException(
> s"Hash purpose with id ${hashPurpose.id} has been created without going through apply"
> ),
> )
>
> /* HashPurposes are listed as `val` rather than `case object`s such that they are initialized eagerly.
> * This ensures that HashPurpose id clashes are detected eagerly. Otherwise, it may be there are two hash purposes
> * with the same id, but they are never used in the same Java process and therefore the clash is not detected.
> * NOTE: We're keeping around the old hash purposes (no longer used) to prevent accidental reuse.
> */
> val SequencedEventSignature = HashPurpose(1, "SequencedEventSignature")
> val _Hmac = HashPurpose(2, "Hmac")
> val MerkleTreeInnerNode = HashPurpose(3, "MerkleTreeInnerNode")
> val _Discriminator = HashPurpose(4, "Discriminator")
> val SubmitterMetadata = HashPurpose(5, "SubmitterMetadata")
> val CommonMetadata = HashPurpose(6, "CommonMetadata")
> val ParticipantMetadata = HashPurpose(7, "ParticipantMetadata")
> val ViewCommonData = HashPurpose(8, "ViewCommonData")
> val ViewParticipantData = HashPurpose(9, "ViewParticipantData")
> val _MalformedMediatorRequestResult = HashPurpose(10, "MalformedMediatorRequestResult")
> val TopologyTransactionSignature = HashPurpose(11, "TopologyTransactionSignature")
> val PublicKeyFingerprint = HashPurpose(12, "PublicKeyFingerprint")
> val _DarIdentifier = HashPurpose(13, "DarIdentifier")
> val AuthenticationToken = HashPurpose(14, "AuthenticationToken")
> val _AgreementId = HashPurpose(15, "AgreementId")
> val _MediatorResponseSignature = HashPurpose(16, "MediatorResponseSignature")
> val _TransactionResultSignature = HashPurpose(17, "TransactionResultSignature")
> val _TransferResultSignature = HashPurpose(19, "TransferResultSignature")
> val _ParticipantStateSignature = HashPurpose(20, "ParticipantStateSignature")
> val _SynchronizerTopologyTransactionMessageSignature =
> HashPurpose(21, "SynchronizerTopologyTransactionMessageSignature")
> val _AcsCommitment = HashPurpose(22, "AcsCommitment")
> val Stakeholders = HashPurpose(23, "Stakeholders")
> val UnassignmentCommonData = HashPurpose(24, "UnassignmentCommonData")
> val UnassignmentView = HashPurpose(25, "UnassignmentView")
> val AssignmentCommonData = HashPurpose(26, "AssignmentCommonData")
> val AssignmentView = HashPurpose(27, "AssignmentView")
> val _TransferViewTreeMessageSeed = HashPurpose(28, "TransferViewTreeMessageSeed")
> val Unicum = HashPurpose(29, "Unicum")
> val RepairUpdateId = HashPurpose(30, "RepairUpdateId")
> val _MediatorLeadershipEvent = HashPurpose(31, "MediatorLeadershipEvent")
> val _LegalIdentityClaim = HashPurpose(32, "LegalIdentityClaim")
> val DbLockId = HashPurpose(33, "DbLockId")
> val HashedAcsCommitment = HashPurpose(34, "HashedAcsCommitment")
> val SubmissionRequestSignature = HashPurpose(35, "SubmissionRequestSignature")
> val AcknowledgementSignature = HashPurpose(36, "AcknowledgementSignature")
> val DecentralizedNamespaceNamespace = HashPurpose(37, "DecentralizedNamespace")
> val SignedProtocolMessageSignature = HashPurpose(38, "SignedProtocolMessageSignature")
> val AggregationId = HashPurpose(39, "AggregationId")
> val BftOrderingPbftBlock = HashPurpose(40, "BftOrderingPbftBlock")
> val _SetTrafficPurchased = HashPurpose(41, "SetTrafficPurchased")
> val OrderingRequestSignature = HashPurpose(42, "OrderingRequestSignature")
> val TopologyMappingUniqueKey = HashPurpose(43, "TopologyMappingUniqueKey")
> val CantonScript = HashPurpose(44, "CantonScriptHash")
> val BftAvailabilityAck = HashPurpose(45, "BftAvailabilityAck")
> val BftBatchId = HashPurpose(46, "BftBatchId")
> val BftSignedAvailabilityMessage = HashPurpose(47, "BftSignedAvailabilityMessage")
> val PreparedSubmission = HashPurpose(48, "PreparedSubmission")
> val TopologyUpdateId = HashPurpose(49, "TopologyUpdateId")
> val OnlinePartyReplicationId = HashPurpose(50, "OnlinePartyReplication")
> val PartyUpdateId = HashPurpose(51, "PartyUpdateId")
> val BftSignedConsensusMessage = HashPurpose(52, "BftSignedConsensusMessage")
> val BftSignedStateTransferMessage = HashPurpose(53, "BftSignedStateTransferMessage")
> val BftSignedRetransmissionMessage = HashPurpose(54, "BftSignedRetransmissionMessage")
> val MultiTopologyTransaction = HashPurpose(55, "MultiTopologyTransaction")
> val SessionKeyDelegation = HashPurpose(56, "SessionKeyDelegation")
> val ReassignmentId = HashPurpose(57, "ReassignmentId")
> val EncryptedSessionKey = HashPurpose(58, "EncryptedSessionKey")
> val ContractIdAbsolutization = HashPurpose(59, "ContractIdAbsolutization")
> val InitialTopologyStateConsistency = HashPurpose(60, "InitialTopologyStateConsistency")
>
> // Do not use for anything other than testing or "mock" hashes
> // Is not in a testing-only module because it used in traffic cost estimation that requires mock hashes
> val TestHashPurpose: HashPurpose = HashPurpose(-1, "testing")
> }
> ```
2. The resulting data is hashed using the underlying algorithm.
3. The final multihash is prefixed again with two bytes, following the [multi-codec](https://github.com/multiformats/multicodec) specification:
> * The identifier for the hash algorithm used.
> * The length of the hash.
For most practical usages, SHA-256 can be used as the underlying algorithm, and is used in this tutorial as well.
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
compute_canton_hash() {
# The hash purpose integer must be prefixed to the content to be hashed as a 4 bytes big endian
(printf "\\x00\\x00\\x00\\x$(printf '%02X' "$1")"; cat - <(cat)) | \
# Then hash with sha256
openssl dgst -sha256 -binary | \
# And finally prefix with 0x12 (The multicodec code for SHA256 https://github.com/multiformats/multicodec/blob/master/table.csv#L9)
# and 0x20, the length of the hash (32 bytes)
( printf '\x12\x20'; cat - )
}
```
Python
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def compute_sha256_canton_hash(purpose: int, content: bytes):
hash_purpose = purpose.to_bytes(4, byteorder="big")
# Hashed content
hashed_content = hashlib.sha256(hash_purpose + content).digest()
# Multi-hash encoding
# Canton uses an implementation of multihash (https://github.com/multiformats/multihash)
# Since we use sha256 always here, we can just hardcode the prefixes
# This may be improved and simplified in subsequent versions
sha256_algorithm_prefix = bytes([0x12])
sha256_length_prefix = bytes([0x20])
return sha256_algorithm_prefix + sha256_length_prefix + hashed_content
```
## 3. Fingerprint
Canton uses fingerprints to efficiently identify and reference signing keys. A fingerprint is a hash of the public key. Using the hashing algorithm described previously, compute the fingerprint of the public key. For fingerprints, the hash purpose value is `12`.
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
compute_canton_fingerprint() {
# 12 is the hash purpose for public key fingerprints
# https://github.com/digital-asset/canton/blob/main/community/base/src/main/scala/com/digitalasset/canton/crypto/HashPurpose.scala
compute_canton_hash 12 | encode_to_hex
}
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Compute the fingerprint of the public key
fingerprint=$(compute_canton_fingerprint < namespace_public_key.der)
```
Python
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def compute_fingerprint(public_key_bytes: bytes) -> str:
"""
Computes the fingerprint of a public signing key.
Args:
public_key_bytes (bytes): The serialized transaction data.
Returns:
str: The computed fingerprint in hexadecimal format.
"""
# 12 is the hash purpose for public key fingerprints
# https://github.com/digital-asset/canton/blob/main/community/base/src/main/scala/com/digitalasset/canton/crypto/HashPurpose.scala
return compute_sha256_canton_hash(12, public_key_bytes).hex()
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
public_key_bytes: bytes = public_key.public_bytes(
encoding=serialization.Encoding.DER,
format=serialization.PublicFormat.SubjectPublicKeyInfo,
)
public_key_fingerprint = compute_fingerprint(public_key_bytes)
```
The scripts in this tutorial can provide a quick way to verify third party implementations of the hashing and signing logic. For instance, the following script outputs the valid fingerprint of a signing public key passed in a base64 format:
```
> . ./interactive_topology_util.sh && compute_canton_fingerprint_from_base64 "2RwUiIHVUVdulxzD8NKtPmIaaBqMer1A90rDjoklJPY="
1220205057e331cc8929dd217e2f8e63f503b7081773de60d01fb46839700bc5caaa
```
## 4. Namespace Delegation Mapping
There is a number of different mappings available, each modeling a part of the topology state.
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
message TopologyMapping {
oneof mapping {
NamespaceDelegation namespace_delegation = 1;
DecentralizedNamespaceDefinition decentralized_namespace_definition = 3;
OwnerToKeyMapping owner_to_key_mapping = 4;
SynchronizerTrustCertificate synchronizer_trust_certificate = 5;
ParticipantSynchronizerPermission participant_permission = 6;
PartyHostingLimits party_hosting_limits = 7;
VettedPackages vetted_packages = 8;
PartyToParticipant party_to_participant = 9;
SynchronizerParametersState synchronizer_parameters_state = 11;
MediatorSynchronizerState mediator_synchronizer_state = 12;
SequencerSynchronizerState sequencer_synchronizer_state = 13;
DynamicSequencingParametersState sequencing_dynamic_parameters_state = 15;
PartyToKeyMapping party_to_key_mapping = 16;
SynchronizerUpgradeAnnouncement synchronizer_upgrade_announcement = 17;
SequencerConnectionSuccessor sequencer_connection_successor = 18;
}
reserved 2; // was identifier_delegation
reserved 10; // was authority_of
reserved 14; // was purge_topology_txs
}
```
This tutorial illustrates the process of importing a root namespace delegation, represented by the `NamespaceDelegation` mapping, but the same procedure can be applied for any topology mapping.
The Namespace Delegation mapping requires three values:
1. `namespace`: Root key's fingerprint
2. `target_key`: Public key expected to be used by delegation. Root namespace delegations are self-signed.
> * The format (`DER`) and specification (`EC256`) of the key must match those of the key generated in step 1.
3. `is_root_delegation`: `true` for root namespace delegations
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
build_namespace_mapping() {
local namespace="$1"
local format="$2"
local public_key="$3"
local spec="$4"
local restrictions="$5"
cat <
The versioning of protobuf messages is relatively stable and is not expected to change often. The rest of the tutorial assumes the protobuf version used is `30`.
Topology transactions consist of three parts:
### Topology Mapping
See the Namespace Delegation Mapping section
### Serial
The `serial` is a monotonically increasing number, starting from 1. Each transaction creating, replacing, or deleting a unique topology mapping must specify a serial incrementing the serial of the previous accepted transaction for that mapping by 1. Uniqueness is defined differently for each mapping. Refer to the protobuf definition of the mapping for details. This mechanism ensures that concurrent topology transactions updating the same mapping do not accidentally overwrite each other. To obtain the serial of an existing transaction, use the `TopologyManagerReadService` to list relevant mappings and obtain their current serial.
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Copyright (c) 2025 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
syntax = "proto3";
package com.digitalasset.canton.topology.admin.v30;
import "com/digitalasset/canton/protocol/v30/synchronizer_parameters.proto";
import "com/digitalasset/canton/protocol/v30/topology.proto";
import "com/digitalasset/canton/topology/admin/v30/common.proto";
import "google/protobuf/empty.proto";
import "google/protobuf/timestamp.proto";
service TopologyManagerReadService {
rpc ListNamespaceDelegation(ListNamespaceDelegationRequest) returns (ListNamespaceDelegationResponse);
rpc ListDecentralizedNamespaceDefinition(ListDecentralizedNamespaceDefinitionRequest) returns (ListDecentralizedNamespaceDefinitionResponse);
rpc ListOwnerToKeyMapping(ListOwnerToKeyMappingRequest) returns (ListOwnerToKeyMappingResponse);
rpc ListPartyToKeyMapping(ListPartyToKeyMappingRequest) returns (ListPartyToKeyMappingResponse);
rpc ListSynchronizerTrustCertificate(ListSynchronizerTrustCertificateRequest) returns (ListSynchronizerTrustCertificateResponse);
rpc ListParticipantSynchronizerPermission(ListParticipantSynchronizerPermissionRequest) returns (ListParticipantSynchronizerPermissionResponse);
rpc ListPartyHostingLimits(ListPartyHostingLimitsRequest) returns (ListPartyHostingLimitsResponse);
rpc ListVettedPackages(ListVettedPackagesRequest) returns (ListVettedPackagesResponse);
rpc ListPartyToParticipant(ListPartyToParticipantRequest) returns (ListPartyToParticipantResponse);
rpc ListSynchronizerParametersState(ListSynchronizerParametersStateRequest) returns (ListSynchronizerParametersStateResponse);
rpc ListMediatorSynchronizerState(ListMediatorSynchronizerStateRequest) returns (ListMediatorSynchronizerStateResponse);
rpc ListSequencerSynchronizerState(ListSequencerSynchronizerStateRequest) returns (ListSequencerSynchronizerStateResponse);
rpc ListSynchronizerUpgradeAnnouncement(ListSynchronizerUpgradeAnnouncementRequest) returns (ListSynchronizerUpgradeAnnouncementResponse);
rpc ListSequencerConnectionSuccessor(ListSequencerConnectionSuccessorRequest) returns (ListSequencerConnectionSuccessorResponse);
rpc ListAvailableStores(ListAvailableStoresRequest) returns (ListAvailableStoresResponse);
rpc ListAll(ListAllRequest) returns (ListAllResponse);
rpc ExportTopologySnapshot(ExportTopologySnapshotRequest) returns (stream ExportTopologySnapshotResponse);
rpc ExportTopologySnapshotV2(ExportTopologySnapshotV2Request) returns (stream ExportTopologySnapshotV2Response);
// Fetch the genesis topology state.
// The returned bytestring can be used directly to initialize a sequencer.
rpc GenesisState(GenesisStateRequest) returns (stream GenesisStateResponse);
rpc GenesisStateV2(GenesisStateV2Request) returns (stream GenesisStateV2Response);
// Fetch the topology state
// The returned bytestring can be used directly to initialize a successor sequencer
rpc LogicalUpgradeState(LogicalUpgradeStateRequest) returns (stream LogicalUpgradeStateResponse);
}
message BaseQuery {
StoreId store = 1;
// whether to query only for proposals instead of approved topology mappings
bool proposals = 2;
com.digitalasset.canton.protocol.v30.Enums.TopologyChangeOp operation = 3;
reserved 4;
message TimeRange {
google.protobuf.Timestamp from = 1;
google.protobuf.Timestamp until = 2;
}
oneof time_query {
google.protobuf.Timestamp snapshot = 5;
google.protobuf.Empty head_state = 6;
TimeRange range = 7;
}
string filter_signed_key = 8;
optional int32 protocol_version = 9;
}
message BaseResult {
StoreId store = 1;
google.protobuf.Timestamp sequenced = 2;
google.protobuf.Timestamp valid_from = 3;
google.protobuf.Timestamp valid_until = 4;
com.digitalasset.canton.protocol.v30.Enums.TopologyChangeOp operation = 5;
bytes transaction_hash = 6;
int32 serial = 7;
repeated string signed_by_fingerprints = 8;
}
message ListNamespaceDelegationRequest {
BaseQuery base_query = 1;
string filter_namespace = 2;
string filter_target_key_fingerprint = 3;
}
message ListNamespaceDelegationResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.NamespaceDelegation item = 2;
}
repeated Result results = 1;
}
message ListDecentralizedNamespaceDefinitionRequest {
BaseQuery base_query = 1;
string filter_namespace = 2;
}
message ListDecentralizedNamespaceDefinitionResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.DecentralizedNamespaceDefinition item = 2;
}
repeated Result results = 1;
}
message ListOwnerToKeyMappingRequest {
BaseQuery base_query = 1;
string filter_key_owner_type = 2;
string filter_key_owner_uid = 3;
}
message ListOwnerToKeyMappingResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.OwnerToKeyMapping item = 2;
}
repeated Result results = 1;
}
message ListPartyToKeyMappingRequest {
BaseQuery base_query = 1;
string filter_party = 2;
}
message ListPartyToKeyMappingResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.PartyToKeyMapping item = 2;
}
repeated Result results = 1;
}
message ListSynchronizerTrustCertificateRequest {
BaseQuery base_query = 1;
string filter_uid = 2;
}
message ListSynchronizerTrustCertificateResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.SynchronizerTrustCertificate item = 2;
}
repeated Result results = 1;
}
message ListParticipantSynchronizerPermissionRequest {
BaseQuery base_query = 1;
string filter_uid = 2;
}
message ListParticipantSynchronizerPermissionResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.ParticipantSynchronizerPermission item = 2;
}
repeated Result results = 1;
}
message ListPartyHostingLimitsRequest {
BaseQuery base_query = 1;
string filter_uid = 2;
}
message ListPartyHostingLimitsResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.PartyHostingLimits item = 2;
}
repeated Result results = 1;
}
message ListVettedPackagesRequest {
BaseQuery base_query = 1;
string filter_participant = 2;
}
message ListVettedPackagesResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.VettedPackages item = 2;
}
repeated Result results = 1;
}
message ListPartyToParticipantRequest {
BaseQuery base_query = 1;
string filter_party = 2;
string filter_participant = 3;
}
message ListPartyToParticipantResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.PartyToParticipant item = 2;
}
repeated Result results = 2;
}
message ListSynchronizerParametersStateRequest {
BaseQuery base_query = 1;
string filter_synchronizer_id = 2;
}
message ListSynchronizerParametersStateResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.DynamicSynchronizerParameters item = 2;
}
repeated Result results = 1;
}
message ListMediatorSynchronizerStateRequest {
BaseQuery base_query = 1;
string filter_synchronizer_id = 2;
}
message ListMediatorSynchronizerStateResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.MediatorSynchronizerState item = 2;
}
repeated Result results = 1;
}
message ListSequencerSynchronizerStateRequest {
BaseQuery base_query = 1;
string filter_synchronizer_id = 2;
}
message ListSequencerSynchronizerStateResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.SequencerSynchronizerState item = 2;
}
repeated Result results = 1;
}
message ListSynchronizerUpgradeAnnouncementRequest {
BaseQuery base_query = 1;
string filter_synchronizer_id = 2;
}
message ListSynchronizerUpgradeAnnouncementResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.SynchronizerUpgradeAnnouncement item = 2;
}
repeated Result results = 1;
}
message ListSequencerConnectionSuccessorRequest {
BaseQuery base_query = 1;
string filter_sequencer_id = 2;
}
message ListSequencerConnectionSuccessorResponse {
message Result {
BaseResult context = 1;
com.digitalasset.canton.protocol.v30.SequencerConnectionSuccessor item = 2;
}
repeated Result results = 1;
}
message ListAvailableStoresRequest {}
message ListAvailableStoresResponse {
repeated StoreId store_ids = 1;
}
message ListAllRequest {
BaseQuery base_query = 1;
/** The list of topology mappings to exclude from the result.*/
repeated string exclude_mappings = 2;
string filter_namespace = 3;
}
message ListAllResponse {
com.digitalasset.canton.topology.admin.v30.TopologyTransactions result = 1;
}
message ExportTopologySnapshotRequest {
BaseQuery base_query = 1;
repeated string exclude_mappings = 2;
string filter_namespace = 3;
}
message ExportTopologySnapshotResponse {
bytes chunk = 1;
}
message ExportTopologySnapshotV2Request {
BaseQuery base_query = 1;
repeated string exclude_mappings = 2;
string filter_namespace = 3;
}
message ExportTopologySnapshotV2Response {
bytes chunk = 1;
}
message GenesisStateRequest {
// Must be specified if the genesis state is requested from a participant node.
optional StoreId synchronizer_store = 1;
// Optional - the effective time used to fetch the topology transactions. If not provided the effective time of the last topology transaction is used.
google.protobuf.Timestamp timestamp = 2;
}
message GenesisStateResponse {
// versioned stored topology transactions
bytes chunk = 1;
}
message GenesisStateV2Request {
// Must be specified if the genesis state is requested from a participant node.
optional StoreId synchronizer_store = 1;
// Optional - the effective time used to fetch the topology transactions. If not provided the effective time of the last topology transaction is used.
google.protobuf.Timestamp timestamp = 2;
}
message GenesisStateV2Response {
// versioned stored topology transactions
bytes chunk = 1;
}
message LogicalUpgradeStateRequest {}
message LogicalUpgradeStateResponse {
// versioned stored topology transactions
bytes chunk = 1;
}
```
In this tutorial, it is assumed that the `NamespaceDelegation` created is new, in particular there is no pre-existing root namespace delegation with the key created in step 1. The serial is therefore set to 1.
### Operation
There are two operations possible:
* `ADD_REPLACE`: Adds a new mapping or replaces an existing one.
* `REMOVE`: Remove an existing mapping
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
build_topology_transaction() {
local mapping="$1"
local serial="$2"
local operation="${3:-TOPOLOGY_CHANGE_OP_ADD_REPLACE}"
cat < "$serialized_versioned_transaction_file"
```
Python
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def build_versioned_transaction(
data: bytes,
):
"""
Builds a versioned transaction wrapper for the given data.
Args:
data (bytes): Serialized transaction data.
Returns:
untyped_versioned_message_pb2.UntypedVersionedMessage: The versioned transaction object.
"""
return untyped_versioned_message_pb2.UntypedVersionedMessage(
data=data,
version=30,
)
```
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def serialize_topology_transaction(
mapping: topology_pb2.TopologyMapping,
serial: int = 1,
):
"""
Serializes a topology transaction.
Args:
mapping (topology_pb2.TopologyMapping): The topology mapping to serialize.
serial (int): The serial of the topology transaction. Defaults to 1.
Returns:
bytes: The serialized topology transaction.
"""
topology_transaction = build_topology_transaction(mapping, serial)
versioned_topology_transaction = build_versioned_transaction(
topology_transaction.SerializeToString()
)
return versioned_topology_transaction.SerializeToString()
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
serialized_versioned_topology_transaction = serialize_topology_transaction(mapping)
```
## 7. Transaction Hash
The next step is to compute the hash of the transaction. It is computed from the serialized protobuf of the versioned transaction. Simply reuse the hashing function defined earlier in the tutorial. This time, the hash purpose value is `11`.
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
compute_topology_transaction_hash() {
# 11 is the hash purpose for topology transaction signatures
# https://github.com/digital-asset/canton/blob/main/community/base/src/main/scala/com/digitalasset/canton/crypto/HashPurpose.scala
compute_canton_hash 11
}
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
topology_transaction_hash_file="topology_transaction_hash.bin"
compute_topology_transaction_hash < $serialized_versioned_transaction_file > $topology_transaction_hash_file
```
Python
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def compute_topology_transaction_hash(serialized_versioned_transaction: bytes) -> bytes:
"""
Computes the hash of a serialized topology transaction.
Args:
serialized_versioned_transaction (bytes): The serialized transaction data.
Returns:
bytes: The computed hash.
"""
# 11 is the hash purpose for topology transaction signatures
# https://github.com/digital-asset/canton/blob/main/community/base/src/main/scala/com/digitalasset/canton/crypto/HashPurpose.scala
return compute_sha256_canton_hash(11, serialized_versioned_transaction)
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
transaction_hash = compute_topology_transaction_hash(
serialized_versioned_topology_transaction
)
```
To facilitate steps `5` to `7`, the topology API offers a `GenerateTransactions` RPC to generate the serialized versioned transaction and its hash. When using the `GenerateTransactions` API, it is strongly recommended to deserialize the returned transaction, validate its content and re-compute its hash, to prevent any accidental misuse or adversarial behavior of the participant generating the transaction.
## 8. Signature
The hash is now ready to be signed. For root namespace transactions, there is only one key involved, and it therefore needs only one signature. Other topology mappings may require additional signatures, either because the mappings themselves contain additional public keys (e.g `OwnerToKeyMapping`), or because the authorization rules of the mapping require signatures from several entities (e.g `PartyToParticipant`). All transactions, however, require a signature either from the root namespace key of the namespace the transaction is targeting or from a delegated key of that namespace registered via a (non-root) `NamespaceDelegation`. The authorization rules vary by mapping and are out of the scope of this tutorial, but can be found on their protobuf definition.
Sign the hash with the private key:
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
sign_hash() {
local private_key_file="$1"
local transaction_hash_file="$2"
openssl pkeyutl -rawin -inkey "$private_key_file" -keyform DER -sign < "$transaction_hash_file" | encode_to_base64
}
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
signature=$(sign_hash namespace_private_key.der $topology_transaction_hash_file)
canton_signature=$(build_canton_signature "SIGNATURE_FORMAT_DER" "$signature" "$fingerprint" "SIGNING_ALGORITHM_SPEC_EC_DSA_SHA_256")
```
Python
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def sign_hash(
private_key: EllipticCurvePrivateKey,
data: bytes,
):
"""
Signs the given data using an elliptic curve private key.
Args:
private_key (EllipticCurvePrivateKey): The private key used for signing.
data (bytes): The data to be signed.
Returns:
bytes: The generated signature.
"""
return private_key.sign(
data=data,
signature_algorithm=ec.ECDSA(hashes.SHA256()),
)
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
signature = sign_hash(private_key, transaction_hash)
```
## 9. Submit the transaction
Submit the transaction and its signature. This is done via the `AddTransactions` RPC of the `TopologyManagerWriteService`:
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
build_canton_signature() {
local format="$1"
local signature="$2"
local signed_by="$3"
local spec="$4"
cat < (EllipticCurvePrivateKey, str):
"""
Submits signed topology transactions to the Canton topology API.
Args:
channel (Channel): The gRPC channel used to communicate with the topology service.
signed_transactions (list[topology_pb2.SignedTopologyTransaction]):
A list of signed topology transactions to be submitted.
synchronizer_id (str): The identifier of the synchronizer to target.
Raises:
grpc.RpcError: If there is an issue communicating with the topology API.
"""
add_transactions_request = build_add_transaction_request(
signed_transactions,
synchronizer_id,
)
topology_write_client = (
topology_manager_write_service_pb2_grpc.TopologyManagerWriteServiceStub(channel)
)
topology_write_client.AddTransactions(add_transactions_request)
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
canton_signature = build_canton_signature(
signature,
public_key_fingerprint,
crypto_pb2.SignatureFormat.SIGNATURE_FORMAT_DER,
crypto_pb2.SigningAlgorithmSpec.SIGNING_ALGORITHM_SPEC_EC_DSA_SHA_256,
)
signed_transaction = build_signed_transaction(
serialized_versioned_topology_transaction,
[canton_signature],
)
submit_signed_transactions(channel, [signed_transaction], synchronizer_id)
print(f"Transaction submitted successfully")
```
### Proposal
The `SignedTopologyTransaction` message contains a boolean `proposal` field. When set to true, it allows submitting topology transactions without attaching all the signatures required for the transaction to be fully authorized. This is especially useful in cases where signatures from multiple entities of the network are necessary, that would be tedious and difficult to gather offline.
## 10. Observe the transaction
The last step of the tutorial is to observe the `NamespaceDelegation` on the topology state of the synchronizer. Note that the submission is asynchronous, which means it may take some time before the submission is accepted.
Bash
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
build_list_namespace_delegations_request() {
local synchronizer_id="$1"
local namespace="$2"
cat <
This tutorial is for demo purposes. The code snippets should not be used directly in a production environment.
## Prerequisites
For simplicity, this tutorial assumes a minimal Canton setup consisting of one participant node connected to one synchronizer (which includes both a sequencer node and a mediator node).
If you already have such an instance running or have completed the onboarding tutorial, proceed to the Setup section.
### Start Canton
To obtain a Canton artifact refer to the getting started section. First, navigate to the interactive submission example folder located at `examples/08-interactive-submission` in the Canton release artifact.
All commands in this tutorial are expected to be run from that folder.
From the artifact directory, start Canton using the command:
```
../../bin/canton -c examples/08-interactive-submission/interactive-submission.conf --bootstrap examples/08-interactive-submission/bootstrap.canton
```
Once the Welcome to Canton message appears, you are ready to proceed.
### Setup
Navigate to the interactive submission example folder located at `examples/08-interactive-submission` in the Canton release artifact.
This tutorial demonstrates external signing with two external parties: Alice and Bob. If you haven't onboarded an external party yet, refer to the onboarding tutorial.
To proceed, gather the following information:
* `Alice`'s Party Id, protocol signing private key, and protocol signing public key fingerprint
* `Bob`'s Party Id
* Synchronizer Id to which the participant is connected
* gRPC Ledger API endpoint
The Party IDs and key-related information should already be known from the onboarding tutorial. To retrieve the participant and synchronizer IDs, as well as the gRPC Ledger API ports, run the following commands in the Canton console:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ sequencer1.synchronizer_id.filterString
res1: String = "local::122032922613929d67857e621fb13e3da49ec13883e24908404520319eee6d31fb4d"
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.config.ledgerApi.address
res2: String = "127.0.0.1"
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.config.ledgerApi.port.unwrap
res3: Int = 30225
```
For this tutorial, the following values will be used (replace them with actual values):
* `Alice` Party Id: `alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e`
* `Bob` Party Id: `bob::1220254d06095b407f8c6a378b6fc443a67d3356ab8edfbf1378cb3e44218de32c8a`
* Synchronizer Id: `da::12203c0ecb446b35b0efa78e0bda9fd91716855866150a5eb7611a2ed5d418129de3`
* gRPC Ledger API endpoint: `localhost:4001`
### API
This tutorial interacts with the `InteractiveSubmissionService`, a service available on the gRPC Ledger API of the participant node.
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Copyright (c) 2025 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
// SPDX-License-Identifier: Apache-2.0
syntax = "proto3";
package com.daml.ledger.api.v2.interactive;
import "com/daml/ledger/api/v2/commands.proto";
import "com/daml/ledger/api/v2/crypto.proto";
import "com/daml/ledger/api/v2/interactive/interactive_submission_common_data.proto";
import "com/daml/ledger/api/v2/interactive/transaction/v1/interactive_submission_data.proto";
import "com/daml/ledger/api/v2/package_reference.proto";
import "com/daml/ledger/api/v2/transaction.proto";
import "com/daml/ledger/api/v2/transaction_filter.proto";
import "com/daml/ledger/api/v2/value.proto";
import "google/protobuf/duration.proto";
import "google/protobuf/timestamp.proto";
option csharp_namespace = "Com.Daml.Ledger.Api.V2.Interactive";
option java_outer_classname = "InteractiveSubmissionServiceOuterClass";
option java_package = "com.daml.ledger.api.v2.interactive";
// Service allowing interactive construction of command submissions
//
// The prepare and execute endpoints allow to submit commands in 2-steps:
//
// 1. prepare transaction from commands,
// 2. submit the prepared transaction
//
// This gives callers the ability to sign the daml transaction with their own signing keys
service InteractiveSubmissionService {
// Requires `readAs` scope for the submitting party when LAPI User authorization is enabled
rpc PrepareSubmission(PrepareSubmissionRequest) returns (PrepareSubmissionResponse);
// Execute a prepared submission _asynchronously_ on the ledger.
// Requires a signature of the transaction from the submitting external party.
rpc ExecuteSubmission(ExecuteSubmissionRequest) returns (ExecuteSubmissionResponse);
// Similar to ExecuteSubmission but _synchronously_ wait for the completion of the transaction
// IMPORTANT: Relying on the response from this endpoint requires trusting the Participant Node to be honest.
// A malicious node could make a successfully committed request appeared failed and vice versa
rpc ExecuteSubmissionAndWait(ExecuteSubmissionAndWaitRequest) returns (ExecuteSubmissionAndWaitResponse);
// Similar to ExecuteSubmissionAndWait but additionally returns the transaction
// IMPORTANT: Relying on the response from this endpoint requires trusting the Participant Node to be honest.
// A malicious node could make a successfully committed request appear as failed and vice versa
rpc ExecuteSubmissionAndWaitForTransaction(ExecuteSubmissionAndWaitForTransactionRequest) returns (ExecuteSubmissionAndWaitForTransactionResponse);
// A preferred package is the highest-versioned package for a provided package-name
// that is vetted by all the participants hosting the provided parties.
//
// Ledger API clients should use this endpoint for constructing command submissions
// that are compatible with the provided preferred package, by making informed decisions on:
// - which are the compatible packages that can be used to create contracts
// - which contract or exercise choice argument version can be used in the command
// - which choices can be executed on a template or interface of a contract
//
// Can be accessed by any Ledger API client with a valid token when Ledger API authorization is enabled.
//
// Provided for backwards compatibility, it will be removed in the Canton version 3.4.0
rpc GetPreferredPackageVersion(GetPreferredPackageVersionRequest) returns (GetPreferredPackageVersionResponse);
// Compute the preferred packages for the vetting requirements in the request.
// A preferred package is the highest-versioned package for a provided package-name
// that is vetted by all the participants hosting the provided parties.
//
// Ledger API clients should use this endpoint for constructing command submissions
// that are compatible with the provided preferred packages, by making informed decisions on:
// - which are the compatible packages that can be used to create contracts
// - which contract or exercise choice argument version can be used in the command
// - which choices can be executed on a template or interface of a contract
//
// If the package preferences could not be computed due to no selection satisfying the requirements,
// a `FAILED_PRECONDITION` error will be returned.
//
// Can be accessed by any Ledger API client with a valid token when Ledger API authorization is enabled.
//
// Experimental API: this endpoint is not guaranteed to provide backwards compatibility in future releases
rpc GetPreferredPackages(GetPreferredPackagesRequest) returns (GetPreferredPackagesResponse);
}
// Hints to improve cost estimation precision of a prepared transaction
message CostEstimationHints {
// Disable cost estimation
// Default (not set) is false
bool disabled = 1;
// Details on the keys that will be used to sign the transaction (how many and of which type).
// Signature size impacts the cost of the transaction.
// If empty, the signature sizes will be approximated with threshold-many signatures (where threshold is defined
// in the PartyToKeyMapping of the external party), using keys in the order they are registered.
// Optional (empty list is equivalent to not providing this field)
repeated SigningAlgorithmSpec expected_signatures = 2;
}
// Estimation of the cost of submitting the prepared transaction
// The estimation is done against the synchronizer chosen during preparation of the transaction
// (or the one explicitly requested).
// The cost of re-assigning contracts to another synchronizer when necessary is not included in the estimation.
message CostEstimation {
// Timestamp at which the estimation was made
google.protobuf.Timestamp estimation_timestamp = 1;
// Estimated traffic cost of the confirmation request associated with the transaction
uint64 confirmation_request_traffic_cost_estimation = 2;
// Estimated traffic cost of the confirmation response associated with the transaction
// This field can also be used as an indication of the cost that other potential confirming nodes
// of the party will incur to approve or reject the transaction
uint64 confirmation_response_traffic_cost_estimation = 3;
// Sum of the fields above
uint64 total_traffic_cost_estimation = 4;
}
message PrepareSubmissionRequest {
// Uniquely identifies the participant user that prepares the transaction.
// Must be a valid UserIdString (as described in ``value.proto``).
// Required unless authentication is used with a user token.
// In that case, the token's user-id will be used for the request's user_id.
// Optional
string user_id = 1;
// Uniquely identifies the command.
// The triple (user_id, act_as, command_id) constitutes the change ID for the intended ledger change,
// where act_as is interpreted as a set of party names.
// The change ID can be used for matching the intended ledger changes with all their completions.
// Must be a valid LedgerString (as described in ``value.proto``).
// Required
string command_id = 2;
// Individual elements of this atomic command. Must be non-empty.
// Limitation: Only single command transaction are currently supported by the API.
// The field is marked as repeated in preparation for future support of multiple commands.
// Required
repeated Command commands = 3;
// Optional
MinLedgerTime min_ledger_time = 4;
// Maximum timestamp at which the transaction can be recorded onto the ledger via the synchronizer specified in the `PrepareSubmissionResponse`.
// If submitted after it will be rejected even if otherwise valid, in which case it needs to be prepared and signed again
// with a new valid max_record_time.
// Use this to limit the time-to-life of a prepared transaction,
// which is useful to know when it can definitely not be accepted
// anymore and resorting to preparing another transaction for the same
// intent is safe again.
// Optional
optional google.protobuf.Timestamp max_record_time = 11;
// Set of parties on whose behalf the command should be executed, if submitted.
// If ledger API authorization is enabled, then the authorization metadata must authorize the sender of the request
// to **read** (not act) on behalf of each of the given parties. This is because this RPC merely prepares a transaction
// and does not execute it. Therefore read authorization is sufficient even for actAs parties.
// Note: This may change, and more specific authorization scope may be introduced in the future.
// Each element must be a valid PartyIdString (as described in ``value.proto``).
// Required, must be non-empty.
repeated string act_as = 5;
// Set of parties on whose behalf (in addition to all parties listed in ``act_as``) contracts can be retrieved.
// This affects Daml operations such as ``fetch``, ``fetchByKey``, ``lookupByKey``, ``exercise``, and ``exerciseByKey``.
// Note: A command can only use contracts that are visible to at least
// one of the parties in ``act_as`` or ``read_as``. This visibility check is independent from the Daml authorization
// rules for fetch operations.
// If ledger API authorization is enabled, then the authorization metadata must authorize the sender of the request
// to read contract data on behalf of each of the given parties.
// Optional
repeated string read_as = 6;
// Additional contracts used to resolve contract & contract key lookups.
// Optional
repeated DisclosedContract disclosed_contracts = 7;
// Must be a valid synchronizer id
// If not set, a suitable synchronizer that this node is connected to will be chosen
// Optional
string synchronizer_id = 8;
// The package-id selection preference of the client for resolving
// package names and interface instances in command submission and interpretation
// Optional
repeated string package_id_selection_preference = 9;
// When true, the response will contain additional details on how the transaction was encoded and hashed
// This can be useful for troubleshooting of hash mismatches. Should only be used for debugging.
// Optional, default to false
bool verbose_hashing = 10;
// Fetches the contract keys into the caches to speed up the command processing.
// Should only contain contract keys that are expected to be resolved during interpretation of the commands.
// Keys of disclosed contracts do not need prefetching.
//
// Optional
repeated PrefetchContractKey prefetch_contract_keys = 15;
// Hints to improve the accuracy of traffic cost estimation.
// The estimation logic assumes that this node will be used for the execution of the transaction
// If another node is used instead, the estimation may be less precise.
// Request amplification is not accounted for in the estimation: each amplified request will
// result in the cost of the confirmation request to be charged additionally.
//
// Optional - Traffic cost estimation is enabled by default if this field is not set
// To turn off cost estimation, set the CostEstimationHints#disabled field to true
optional CostEstimationHints estimate_traffic_cost = 16;
}
// [docs-entry-start: HashingSchemeVersion]
// The hashing scheme version used when building the hash of the PreparedTransaction
enum HashingSchemeVersion {
HASHING_SCHEME_VERSION_UNSPECIFIED = 0;
reserved 1; // Hashing Scheme V1 - unsupported
HASHING_SCHEME_VERSION_V2 = 2;
}
// [docs-entry-end: HashingSchemeVersion]
message PrepareSubmissionResponse {
// The interpreted transaction, it represents the ledger changes necessary to execute the commands specified in the request.
// Clients MUST display the content of the transaction to the user for them to validate before signing the hash if the preparing participant is not trusted.
PreparedTransaction prepared_transaction = 1;
// Hash of the transaction, this is what needs to be signed by the party to authorize the transaction.
// Only provided for convenience, clients MUST recompute the hash from the raw transaction if the preparing participant is not trusted.
// May be removed in future versions
bytes prepared_transaction_hash = 2;
// The hashing scheme version used when building the hash
HashingSchemeVersion hashing_scheme_version = 3;
// Optional additional details on how the transaction was encoded and hashed. Only set if verbose_hashing = true in the request
// Note that there are no guarantees on the stability of the format or content of this field.
// Its content should NOT be parsed and should only be used for troubleshooting purposes.
optional string hashing_details = 4;
// Traffic cost estimation of the prepared transaction
// Optional
optional CostEstimation cost_estimation = 5;
}
// Signatures provided by a single party
message SinglePartySignatures {
// Submitting party
// Required
string party = 1;
// Signatures
// Required
repeated Signature signatures = 2;
}
// Additional signatures provided by the submitting parties
message PartySignatures {
// Additional signatures provided by all individual parties
// Required
repeated SinglePartySignatures signatures = 1;
}
message ExecuteSubmissionRequest {
// the prepared transaction
// Typically this is the value of the `prepared_transaction` field in `PrepareSubmissionResponse`
// obtained from calling `prepareSubmission`.
// Required
PreparedTransaction prepared_transaction = 1;
// The party(ies) signatures that authorize the prepared submission to be executed by this node.
// Each party can provide one or more signatures..
// and one or more parties can sign.
// Note that currently, only single party submissions are supported.
// Required
PartySignatures party_signatures = 2;
// Specifies the deduplication period for the change ID (See PrepareSubmissionRequest).
// If omitted, the participant will assume the configured maximum deduplication time.
// Optional
oneof deduplication_period {
// Specifies the length of the deduplication period.
// It is interpreted relative to the local clock at some point during the submission's processing.
// Must be non-negative. Must not exceed the maximum deduplication time.
google.protobuf.Duration deduplication_duration = 3;
// Specifies the start of the deduplication period by a completion stream offset (exclusive).
// Must be a valid absolute offset (positive integer).
int64 deduplication_offset = 4;
}
// A unique identifier to distinguish completions for different submissions with the same change ID.
// Typically a random UUID. Applications are expected to use a different UUID for each retry of a submission
// with the same change ID.
// Must be a valid LedgerString (as described in ``value.proto``).
//
// Required
string submission_id = 5;
// See [PrepareSubmissionRequest.user_id]
// Optional
string user_id = 6;
// The hashing scheme version used when building the hash
// Required
HashingSchemeVersion hashing_scheme_version = 7;
// If set will influence the chosen ledger effective time but will not result in a submission delay so any override
// should be scheduled to executed within the window allowed by synchronizer.
// Optional
MinLedgerTime min_ledger_time = 8;
}
message ExecuteSubmissionResponse {}
message ExecuteSubmissionAndWaitRequest {
// the prepared transaction
// Typically this is the value of the `prepared_transaction` field in `PrepareSubmissionResponse`
// obtained from calling `prepareSubmission`.
// Required
PreparedTransaction prepared_transaction = 1;
// The party(ies) signatures that authorize the prepared submission to be executed by this node.
// Each party can provide one or more signatures..
// and one or more parties can sign.
// Note that currently, only single party submissions are supported.
// Required
PartySignatures party_signatures = 2;
// Specifies the deduplication period for the change ID (See PrepareSubmissionRequest).
// If omitted, the participant will assume the configured maximum deduplication time.
// Optional
oneof deduplication_period {
// Specifies the length of the deduplication period.
// It is interpreted relative to the local clock at some point during the submission's processing.
// Must be non-negative. Must not exceed the maximum deduplication time.
google.protobuf.Duration deduplication_duration = 3;
// Specifies the start of the deduplication period by a completion stream offset (exclusive).
// Must be a valid absolute offset (positive integer).
int64 deduplication_offset = 4;
}
// A unique identifier to distinguish completions for different submissions with the same change ID.
// Typically a random UUID. Applications are expected to use a different UUID for each retry of a submission
// with the same change ID.
// Must be a valid LedgerString (as described in ``value.proto``).
//
// Required
string submission_id = 5;
// See [PrepareSubmissionRequest.user_id]
// Optional
string user_id = 6;
// The hashing scheme version used when building the hash
// Required
HashingSchemeVersion hashing_scheme_version = 7;
// If set will influence the chosen ledger effective time but will not result in a submission delay so any override
// should be scheduled to executed within the window allowed by synchronizer.
// Optional
MinLedgerTime min_ledger_time = 8;
}
message ExecuteSubmissionAndWaitResponse {
// The id of the transaction that resulted from the submitted command.
// Must be a valid LedgerString (as described in ``value.proto``).
// Required
string update_id = 1;
// The details of the offset field are described in ``community/ledger-api/README.md``.
// Required
int64 completion_offset = 2;
}
message ExecuteSubmissionAndWaitForTransactionRequest {
// the prepared transaction
// Typically this is the value of the `prepared_transaction` field in `PrepareSubmissionResponse`
// obtained from calling `prepareSubmission`.
// Required
PreparedTransaction prepared_transaction = 1;
// The party(ies) signatures that authorize the prepared submission to be executed by this node.
// Each party can provide one or more signatures..
// and one or more parties can sign.
// Note that currently, only single party submissions are supported.
// Required
PartySignatures party_signatures = 2;
// Specifies the deduplication period for the change ID (See PrepareSubmissionRequest).
// If omitted, the participant will assume the configured maximum deduplication time.
// Optional
oneof deduplication_period {
// Specifies the length of the deduplication period.
// It is interpreted relative to the local clock at some point during the submission's processing.
// Must be non-negative. Must not exceed the maximum deduplication time.
google.protobuf.Duration deduplication_duration = 3;
// Specifies the start of the deduplication period by a completion stream offset (exclusive).
// Must be a valid absolute offset (positive integer).
int64 deduplication_offset = 4;
}
// A unique identifier to distinguish completions for different submissions with the same change ID.
// Typically a random UUID. Applications are expected to use a different UUID for each retry of a submission
// with the same change ID.
// Must be a valid LedgerString (as described in ``value.proto``).
//
// Required
string submission_id = 5;
// See [PrepareSubmissionRequest.user_id]
// Optional
string user_id = 6;
// The hashing scheme version used when building the hash
// Required
HashingSchemeVersion hashing_scheme_version = 7;
// If set will influence the chosen ledger effective time but will not result in a submission delay so any override
// should be scheduled to executed within the window allowed by synchronizer.
// Optional
MinLedgerTime min_ledger_time = 8;
// If no ``transaction_format`` is provided, a default will be used where ``transaction_shape`` is set to
// TRANSACTION_SHAPE_ACS_DELTA, ``event_format`` is defined with ``filters_by_party`` containing wildcard-template
// filter for all original ``act_as`` and ``read_as`` parties and the ``verbose`` flag is set.
// When the ``transaction_shape`` TRANSACTION_SHAPE_ACS_DELTA shape is used (explicitly or is defaulted to as explained above),
// events will only be returned if the submitting party is hosted on this node.
// Optional
TransactionFormat transaction_format = 9;
}
message ExecuteSubmissionAndWaitForTransactionResponse {
// The transaction that resulted from the submitted command.
// The transaction might contain no events (request conditions result in filtering out all of them).
// Required
Transaction transaction = 1;
}
message MinLedgerTime {
oneof time {
// Lower bound for the ledger time assigned to the resulting transaction.
// The ledger time of a transaction is assigned as part of command interpretation.
// Important note: for interactive submissions, if the transaction depends on time, it **must** be signed
// and submitted within a time window around the ledger time assigned to the transaction during the prepare method.
// The time delta around that ledger time is a configuration of the ledger, usually short, around 1 minute.
// If however the transaction does not depend on time, the available time window to sign and submit the transaction is bound
// by the preparation time, which is also assigned in the "prepare" step (this request),
// but can be configured with a much larger skew, allowing for more time to sign the request (in the order of hours).
// Must not be set at the same time as min_ledger_time_rel.
// Optional
google.protobuf.Timestamp min_ledger_time_abs = 1;
// Same as min_ledger_time_abs, but specified as a duration, starting from the time this request is received by the server.
// Must not be set at the same time as min_ledger_time_abs.
// Optional
google.protobuf.Duration min_ledger_time_rel = 2;
}
}
/**
* Prepared Transaction Message
*/
message PreparedTransaction {
// Daml Transaction representing the ledger effect if executed. See below
DamlTransaction transaction = 1;
// Metadata context necessary to execute the transaction
Metadata metadata = 2;
}
// Transaction Metadata
// Refer to the hashing documentation for information on how it should be hashed.
message Metadata {
message SubmitterInfo {
repeated string act_as = 1;
string command_id = 2;
}
message GlobalKeyMappingEntry {
interactive.GlobalKey key = 1;
optional Value value = 2;
}
message InputContract {
oneof contract {
// When new versions will be added, they will show here
interactive.transaction.v1.Create v1 = 1;
}
uint64 created_at = 1000;
reserved 1001; // Used to contain driver_metadata, now contained in event_blob
bytes event_blob = 1002;
}
/* ************************************************** */
/* ** Metadata information that needs to be signed ** */
/* ************************************************** */
// this used to contain the ledger effective time
reserved 1;
SubmitterInfo submitter_info = 2;
string synchronizer_id = 3;
uint32 mediator_group = 4;
string transaction_uuid = 5;
uint64 preparation_time = 6;
repeated InputContract input_contracts = 7;
/*
* Where ledger time constraints are imposed during the execution of the contract they will be populated
* in the fields below. These are optional because if the transaction does NOT depend on time, these values
* do not need to be set.
* The final ledger effective time used will be chosen when the command is submitted through the [execute] RPC.
* If the ledger effective time is outside of any populated min/max bounds then a different transaction
* can result, that will cause a confirmation message rejection.
*/
optional uint64 min_ledger_effective_time = 9;
optional uint64 max_ledger_effective_time = 10;
/* ********************************************************** */
/* ** Metadata information that does NOT need to be signed ** */
/* ********************************************************** */
// Contextual information needed to process the transaction but not signed, either because it's already indirectly
// signed by signing the transaction, or because it doesn't impact the ledger state
repeated GlobalKeyMappingEntry global_key_mapping = 8;
// Maximum timestamp at which the transaction can be recorded onto the ledger via the synchronizer `synchronizer_id`.
// If submitted after it will be rejected even if otherwise valid, in which case it needs to be prepared and signed again
// with a new valid max_record_time.
// Unsigned in 3.3 to avoid a breaking protocol change
// Will be signed in 3.4+
// Set max_record_time in the PreparedTransactionRequest to get this field set accordingly
optional uint64 max_record_time = 11;
}
/*
* Daml Transaction.
* This represents the effect on the ledger if this transaction is successfully committed.
*/
message DamlTransaction {
message NodeSeed {
int32 node_id = 1;
bytes seed = 2;
}
// A transaction may contain nodes with different versions.
// Each node must be hashed using the hashing algorithm corresponding to its specific version.
// [docs-entry-start: DamlTransaction.Node]
message Node {
string node_id = 1;
// Versioned node
oneof versioned_node {
// Start at 1000 so we can add more fields before if necessary
// When new versions will be added, they will show here
interactive.transaction.v1.Node v1 = 1000;
}
}
// [docs-entry-end: DamlTransaction.Node]
// serialization version, will be >= max(nodes version)
string version = 1;
// Root nodes of the transaction
repeated string roots = 2;
// List of nodes in the transaction
repeated Node nodes = 3;
// Node seeds are values associated with certain nodes used for generating cryptographic salts
repeated NodeSeed node_seeds = 4;
}
message GetPreferredPackageVersionRequest {
// The parties whose participants' vetting state should be considered when resolving the preferred package.
// Required
repeated string parties = 1;
// The package-name for which the preferred package should be resolved.
// Required
string package_name = 2;
// The synchronizer whose vetting state should be used for resolving this query.
// If not specified, the vetting states of all synchronizers to which the participant is connected are used.
// Optional
string synchronizer_id = 3;
// The timestamp at which the package vetting validity should be computed
// on the latest topology snapshot as seen by the participant.
// If not provided, the participant's current clock time is used.
// Optional
google.protobuf.Timestamp vetting_valid_at = 4;
}
message GetPreferredPackageVersionResponse {
// Not populated when no preferred package is found
// Optional
PackagePreference package_preference = 1;
}
message PackagePreference {
// The package reference of the preferred package.
// Required
PackageReference package_reference = 1;
// The synchronizer for which the preferred package was computed.
// If the synchronizer_id was specified in the request, then it matches the request synchronizer_id.
// Required
string synchronizer_id = 2;
}
// Defines a package-name for which the commonly vetted package with the highest version must be found.
message PackageVettingRequirement {
// The parties whose participants' vetting state should be considered when resolving the preferred package.
// Required
repeated string parties = 1;
// The package-name for which the preferred package should be resolved.
// Required
string package_name = 2;
}
message GetPreferredPackagesRequest {
// The package-name vetting requirements for which the preferred packages should be resolved.
//
// Generally it is enough to provide the requirements for the intended command's root package-names.
// Additional package-name requirements can be provided when additional Daml transaction informees need to use
// package dependencies of the command's root packages.
//
// Required
repeated PackageVettingRequirement package_vetting_requirements = 1;
// The synchronizer whose vetting state should be used for resolving this query.
// If not specified, the vetting states of all synchronizers to which the participant is connected are used.
// Optional
string synchronizer_id = 2;
// The timestamp at which the package vetting validity should be computed
// on the latest topology snapshot as seen by the participant.
// If not provided, the participant's current clock time is used.
// Optional
google.protobuf.Timestamp vetting_valid_at = 3;
}
message GetPreferredPackagesResponse {
// The package references of the preferred packages.
// Must contain one package reference for each requested package-name.
//
// If you build command submissions whose content depends on the returned
// preferred packages, then we recommend submitting the preferred package-ids
// in the ``package_id_selection_preference`` of the command submission to
// avoid race conditions with concurrent changes of the on-ledger package vetting state.
//
// Required
repeated PackageReference package_references = 1;
// The synchronizer for which the package preferences are computed.
// If the synchronizer_id was specified in the request, then it matches the request synchronizer_id.
// Required
string synchronizer_id = 2;
}
```
### Python
It is recommended to use a dedicated python environment to avoid conflicting dependencies. Considering using [venv](https://docs.python.org/3/library/venv.html).
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
```
Then run the setup script to generate the necessary python files to interact with Canton's gRPC interface:
```
./setup.sh
```
### Shell
For a terminal-based approach, install the following tools:
* [grpcurl](https://github.com/fullstorydev/grpcurl)
* [openssl](https://www.openssl.org/)
* [jq](https://jqlang.org/)
## 1. Prepare the transaction
Transform `Ledger Command` into a `Daml Transaction`.
Bash
Request:
> ```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> echo '{
> "user_id": "demo_app",
> "command_id": "f2ec4d8f-ccc1-402b-b278-7556fdd2b412",
> "act_as": ["alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e"],
> "synchronizer_id": "da::12203c0ecb446b35b0efa78e0bda9fd91716855866150a5eb7611a2ed5d418129de3",
> "commands": [
> {
> "create": {
> "template_id": {
> "package_id": "#canton-builtin-admin-workflow-ping",
> "module_name": "Canton.Internal.Ping",
> "entity_name": "Ping"
> },
> "create_arguments": {
> "record_id": null,
> "fields": [
> {
> "label" :"id",
> "value": { "text": "ping_id" }
> },
> {
> "label" :"initiator",
> "value": { "party": "alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e" }
> },
> {
> "label" :"responder",
> "value": { "party": "bob::1220254d06095b407f8c6a378b6fc443a67d3356ab8edfbf1378cb3e44218de32c8a" }
> }
> ]
> }
> }
> }
> ]
> }' > create_ping_prepare_request.json
>
> cat "create_ping_prepare_request.json" | grpcurl -emit-defaults -plaintext -d @ localhost:4001 com.daml.ledger.api.v2.interactive.InteractiveSubmissionService/PrepareSubmission > create_ping_prepare_response.json
> ```
Record the response in `create_ping_prepare_response.json` to make it easier to submit the transaction afterwards. Now inspect the response with
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
cat create_ping_prepare_response.json
```
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"prepared_transaction": {
"transaction": {
"version": "2.1",
"roots": [
"0"
],
"nodes": [
{
"node_id": "0",
"v1": {
"create": {
"lf_version": "2.1",
"contract_id": "004c3409aa2e8f8e22604d58ea6211f667df2bae4abc7984a95d76b3d120b8bd85",
"package_name": "canton-builtin-admin-workflow-ping",
"template_id": {
"packageId": "9a19e9cc152538d3ad3b99b933ccf881e53b193ee6af17bdd9a65905a6e1f8ab",
"moduleName": "Canton.Internal.Ping",
"entityName": "Ping"
},
"argument": {
"record": {
"recordId": {
"packageId": "9a19e9cc152538d3ad3b99b933ccf881e53b193ee6af17bdd9a65905a6e1f8ab",
"moduleName": "Canton.Internal.Ping",
"entityName": "Ping"
},
"fields": [
{
"label": "id",
"value": {
"text": "ping_id"
}
},
{
"label": "initiator",
"value": {
"party": "alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e"
}
},
{
"label": "responder",
"value": {
"party": "bob::1220254d06095b407f8c6a378b6fc443a67d3356ab8edfbf1378cb3e44218de32c8a"
}
}
]
}
},
"signatories": [
"alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e"
],
"stakeholders": [
"alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e",
"bob::1220254d06095b407f8c6a378b6fc443a67d3356ab8edfbf1378cb3e44218de32c8a"
]
}
}
}
],
"node_seeds": [
{
"seed": "Gv8neKcoUyIvsa5vdfjUxwGQLGuJOUeVO3j26YB4vOQ="
}
]
},
"metadata": {
"submitter_info": {
"act_as": [
"alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e"
],
"command_id": "f2ec4d8f-ccc1-402b-b278-7556fdd2b412"
},
"synchronizer_id": "da::12203c0ecb446b35b0efa78e0bda9fd91716855866150a5eb7611a2ed5d418129de3",
"transaction_uuid": "0c36b6ea-a5a8-40ee-8708-d47589f34db7",
"submission_time": "1739897973772660"
}
},
"prepared_transaction_hash": "lafpRryDAe5lA8sBONBv0u2umlGKtnJXnhec/7AN+Ro=",
"hashing_scheme_version": "HASHING_SCHEME_VERSION_V2"
}
```
Python
Ensure you have followed the setup instructions for Python before proceeding.
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
import argparse
import sys
import grpc
import uuid
from google.protobuf.json_format import MessageToJson
from com.daml.ledger.api.v2.interactive import interactive_submission_service_pb2_grpc
from com.daml.ledger.api.v2.interactive import interactive_submission_service_pb2
from com.daml.ledger.api.v2 import commands_pb2, value_pb2, completion_pb2, crypto_pb2
from external_party_onboarding_admin_api import onboard_external_party
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePrivateKey
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.primitives import hashes
from daml_transaction_hashing_v2 import (
create_nodes_dict,
encode_prepared_transaction,
HASHING_SCHEME_VERSION_V2,
)
from com.daml.ledger.api.v2 import (
command_completion_service_pb2,
command_completion_service_pb2_grpc,
update_service_pb2,
update_service_pb2_grpc,
event_pb2,
state_service_pb2_grpc,
state_service_pb2,
transaction_filter_pb2,
event_query_service_pb2_grpc,
event_query_service_pb2,
)
import os
import json
```
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
initiator = "alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e"
responder = "bob::1220254d06095b407f8c6a378b6fc443a67d3356ab8edfbf1378cb3e44218de32c8a"
synchronizer_id = "da::12203c0ecb446b35b0efa78e0bda9fd91716855866150a5eb7611a2ed5d418129de3"
user_id = "demo_app"
party = "alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e"
lapi_port="4001"
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
ping_template_id = value_pb2.Identifier(
package_id="#canton-builtin-admin-workflow-ping",
module_name="Canton.Internal.Ping",
entity_name="Ping",
)
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
lapi_channel = grpc.insecure_channel(f"localhost:{lapi_port}")
# Interactive submission service client - used to submit externally signed transactions
iss_client = interactive_submission_service_pb2_grpc.InteractiveSubmissionServiceStub(
lapi_channel
)
# Command completion service client - used to observe command execution results
ccs_client = command_completion_service_pb2_grpc.CommandCompletionServiceStub(
lapi_channel
)
# Update service client - used to query transactions once they've completed
us_client = update_service_pb2_grpc.UpdateServiceStub(lapi_channel)
# State service client - used to query active contracts
state_client = state_service_pb2_grpc.StateServiceStub(lapi_channel)
# Event query service client - used to retrieve event information of a completed transaction
eqs_client = event_query_service_pb2_grpc.EventQueryServiceStub(lapi_channel)
```
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def prepare_create_ping_contract(
initiator: str,
responder: str,
synchronizer_id: str,
) -> interactive_submission_service_pb2.PrepareSubmissionResponse:
ping_create_command = commands_pb2.Command(
create=commands_pb2.CreateCommand(
template_id=ping_template_id,
create_arguments=value_pb2.Record(
record_id=None,
fields=[
value_pb2.RecordField(
label="id", value=value_pb2.Value(text="ping_id")
),
value_pb2.RecordField(
label="initiator", value=value_pb2.Value(party=initiator)
),
value_pb2.RecordField(
label="responder", value=value_pb2.Value(party=responder)
),
],
),
)
)
print("Preparing create ping transaction")
# Prepare the submission request
prepare_create_request = (
interactive_submission_service_pb2.PrepareSubmissionRequest(
user_id=user_id,
command_id=str(uuid.uuid4()),
act_as=[initiator],
read_as=[initiator],
synchronizer_id=synchronizer_id,
commands=[ping_create_command],
)
)
# Call the PrepareSubmission RPC
prepare_create_response = iss_client.PrepareSubmission(prepare_create_request)
return prepare_create_response
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
prepare_create_response = prepare_create_ping_contract(
initiator, responder, synchronizer_id
)
if not auto_accept:
inspect_and_validate_transaction(prepare_create_response.prepared_transaction)
print(
"Returned transaction hash is "
+ prepare_create_response.prepared_transaction_hash.hex()
)
prepared_create_transaction = prepare_create_response.prepared_transaction
```
### Request
* `user_id`: Identifier for the application interacting with the ledger.
* `command_id`: Unique, random string identifying this specific command. Each command submission must have a new and unique `command_id`.
* `act_as`: ID of the party issuing the command.
* `synchronizer_id`: ID of the synchronizer that processes the transaction upon submission.
* `commands`: Ledger commands for submission. In this case, it shows the creation of a Ping contract with `Alice` as the initiator, `Bob` as the responder, and a `ping_id` value. See the [Ledger API command reference](/reference/grpc-ledger-api-reference/com-daml-ledger-api-v2) for details.
### Response
#### Transaction
Represents the explicit ledger changes upon successful commitment.
* `version`: Version of the transaction. This is also called `LF Version`.
* `roots`: List of root node ids. A Daml transaction is a list of trees. The nodes are flattened in a single list (see below). The root node ids design the root node of each individual tree in the transaction.
A current limitation of externally signed transactions is that they can only contain a single root node, and therefore a single transaction tree.
* `nodes`: List of all nodes in the transaction. There are 4 types of nodes: `Create`, `Exercise`, `Fetch` and `Rollback`. The number, type and content of each node depends on the Daml model and the state of the ledger.
* `node_seeds`: List of seeds used by the Canton protocol to generate cryptographically secure salts. They can be ignored.
#### Metadata
Additional information required for transaction processing.
* `ledger_effective_time`: Time picked during the interpretation of the command into a transaction. Set if and only if the daml model makes use of time.
* `submitter_info`: Contains the `act_as` party and `command_id`
* `synchronizer_id`: Synchronizer that will be used for transaction processing
* `transaction_uuid`: Unique value generated by the prepare endpoint to uniquely identify this transaction
> * The transaction UUID is randomly selected during the `prepare` step and is fixed from that point forward. This allows the mediator node to de-duplicate transactions and prevent replays.
* `mediator_group`: Group of mediators that will gather confirmation responses for the transaction. Can be ignored.
* `submission_time`: The timestamp that the Canton protocol will use as a submission time to perform validations (e.g for de-duplication)
* `disclosed_events`: Existing input contracts used in the transaction
* `global_key_mapping`: Unused in the current version, can be ignored.
#### Hash
* `prepared_transaction_hash`: Pre-computed transaction hash. For security reasons the hash should be re-computed client-side as mentioned in the Compute transaction hash section.
The `prepare` API can return additional details on how the Canton node is hashing the transaction to help troubleshoot hash-related errors (for example: pre-computed and re-computed hash mismatch).
To enable it:
1. Enable verbose hashing on the participant config `ledger-api.interactive-submission-service.enable-verbose-hashing = true`.
2. Set the `verbose_hashing` field in the `PrepareSubmissionRequest` to `true`.
#### Hashing scheme version
Version of the hashing scheme:
| Protocol Version | Hashing Scheme Version |
| ---------------- | ---------------------------- |
| 33 | HASHING\_SCHEME\_VERSION\_V2 |
If the gRPC Ledger API authorization is enabled, the user must have the `readAs` claim on behalf of `Alice` to call the `prepare` endpoint.
#### Traffic cost estimation
Estimates the traffic cost for the Participant Node that submits the transaction to the synchronizer. See the API documentation for details on the estimation response fields.
The precision of the estimation is influenced by several factors that cannot be known when the transaction is being prepared. Additionally, the traffic cost will be incurred by the node submitting the transaction, not the one preparing it. When they are the same, the cost estimation will be more accurate. The following factors contribute to the uncertainty of the cost estimation:
> * Hosting relationship of the submitting party with the executing node
> * Number and type of external signatures provided upon submission of the transaction
> * Topology state of the network when the transaction will be submitted
> * Request amplification during submission
> * Part of the transaction that will be confirmed (root view or sub views)
> * Whether nodes approve or reject the transaction
> * IDs of contract created within the transaction are not suffixed in the cost estimation, whereas they are suffixed in the actual submission
> * Whether session signing keys are enabled on the submitting or confirming nodes
In most cases the impact of those factors is low and the variance they cause can generally be expected to be 10% or less. Hints can be specified in the prepare request to help improve the precision of the estimation.
It's worth noting that request amplification can significantly increase traffic cost when triggered. The estimation provided is valid for a single confirmation request submission. Subsequent amplified requests sent will cost additional traffic cost as they correspond to a new confirmation request being sent.
Traffic cost estimation is enabled by default. To disable it, set the disabled field in the `CostEstimationHints` message to `true`.
For details on traffic management, read the related explanation page.
## 2. Validate the transaction
Deserialize and inspect the transaction to verify its correctness before proceeding. The initiator of the transaction must be able to inspect and validate it to ensure it matches their intent before proceeding. See the Trust Model for guidance.
## 3. Compute the transaction hash
It is strongly recommended that the transaction hash be recomputed from the transaction and metadata to verify correctness. The pre-computed hash provided in the `Prepare` step is for debugging purposes.
* The hashing algorithm specification is available here as well as in the release artifact under `protobuf/ledger-api/com/daml/ledger/api/v2/interactive/README.md`
* An example implementation in python is available in the release articact under `examples/08-interactive-submission/daml_transaction_hashing_v2.py`
## 4. Sign the transaction hash
Using `Alice`'s protocol signing private key, sign the hash.
Technically what is needed is the ability to sign with `Alice`'s key, not the key itself. The management of the key can be delegated to a wallet, HSM or crypto custody provider. In this tutorial the key is managed locally and explicitly to demonstrate the signing process. Refer to the onboarding tutorial for details on how to generate a key for this tutorial.
Bash
Assuming `Alice's` private key is stored in a file called `alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e-private-key.der`, he hash can be signed using `openssl`.
In this tutorial the hash retrieved from the response of step 1 will be signed, without re-computing it as suggested in step 3. For an example of how to re-compute the hash, see the `Python` example.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
TRANSACTION_HASH=$(cat create_ping_prepare_response.json | jq -r .prepared_transaction_hash)
PREPARED_TRANSACTION=$(cat create_ping_prepare_response.json | jq -r .prepared_transaction)
SIGNATURE=$(echo -n "$TRANSACTION_HASH" | base64 --decode | openssl pkeyutl -rawin -inkey alice::1220d466a5d96a3509736c821e25fe81fc8a73f226d92e57e94a65170e58b07fc08e-private-key.der -keyform DER -sign | openssl base64 -e -A)
```
Python
The Python example demo includes an implementation of the transaction hashing algorithm. In this example, `party_private_key` is assumed to be an `EllipticCurvePrivateKey` Python object containing Alice's private key. If the onboarding tutorial was followed, this key should already be available.
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
transaction_hash = encode_prepared_transaction(
prepared_transaction, create_nodes_dict(prepared_transaction)
)
print("Computed hash: " + transaction_hash.hex())
# Sign it
signature = party_private_key.sign(
transaction_hash, signature_algorithm=ec.ECDSA(hashes.SHA256())
)
```
## 5. Execute the transaction
Submit the transaction and its signature to the ledger.
Bash
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
grpcurl -emit-defaults -plaintext -d @ localhost:4001 com.daml.ledger.api.v2.interactive.InteractiveSubmissionService/ExecuteSubmission < * Because `submission_id` is not part of the signature, a command can be re-submitted with a different `submission_id` without requiring a new signature.
* `signatures`: Object containing the signature of the transaction hash, along with metadata. In particular:
> * `signing_algorithm_spec`: Will vary depending on the key used during onboarding.
> * `signed_by`: Fingerprint of the protocol signing *public* key of `Alice`. This tutorial assumes the same key was used to create `Alice`'s namespace and her protocol signing key. This is why the fingerprint of the signing key matches the second part of her Party Id (after `::`). For more details check out the onboarding tutorial and the [Daml parties guide](/appdev/deep-dives/manage-daml-parties).
If the gRPC Ledger API authorization is enabled, the user must have the `actAs` claim on behalf of `Alice` to call the `execute` endpoint.
## 6. Observe the transaction outcome
Monitor the completion stream for transaction confirmation, then retrieve the contract ID and binary blob representation.
Bash
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
grpcurl -emit-defaults -plaintext -d @ localhost:4001 com.daml.ledger.api.v2.CommandCompletionService/CompletionStream <
You may need to interrupt the command with `Ctrl-C` as the completion stream is a gRPC server streaming RPC which waits for updates from the server until interrupted.
Let's now retrieve the corresponding transaction:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
grpcurl -emit-defaults -plaintext -d @ localhost:4001 com.daml.ledger.api.v2.UpdateService/GetUpdateById <
The execution and extraction of the `contract_id` above are summarized in the following function, which is reused in Part 2 of the tutorial:
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def execute_and_get_contract_id(
prepared_transaction: interactive_submission_service_pb2.PreparedTransaction,
party: str,
party_private_key: EllipticCurvePrivateKey,
pub_fingerprint: str,
):
# [Compute transaction hash]
transaction_hash = encode_prepared_transaction(
prepared_transaction, create_nodes_dict(prepared_transaction)
)
print("Computed hash: " + transaction_hash.hex())
# Sign it
signature = party_private_key.sign(
transaction_hash, signature_algorithm=ec.ECDSA(hashes.SHA256())
)
# [Signed hash]
# Create the execute request
execute_request = interactive_submission_service_pb2.ExecuteSubmissionRequest(
prepared_transaction=prepared_transaction,
user_id=user_id,
party_signatures=interactive_submission_service_pb2.PartySignatures(
signatures=[
interactive_submission_service_pb2.SinglePartySignatures(
party=party,
signatures=[
crypto_pb2.Signature(
format=crypto_pb2.SignatureFormat.SIGNATURE_FORMAT_CONCAT,
signature=signature,
signed_by=pub_fingerprint,
signing_algorithm_spec=crypto_pb2.SigningAlgorithmSpec.SIGNING_ALGORITHM_SPEC_ED25519,
)
],
)
]
),
hashing_scheme_version=HASHING_SCHEME_VERSION_V2,
submission_id=str(uuid.uuid4()),
)
# Submit the transaction to the ledger
iss_client.ExecuteSubmission(execute_request)
# [Submitted request]
# [Waiting for the transaction to show on the completion stream]
update_request = command_completion_service_pb2.CompletionStreamRequest(
user_id=user_id, parties=[party]
)
completion_stream = ccs_client.CompletionStream(update_request)
for update in completion_stream:
if (
update.HasField("completion")
and update.completion.submission_id == execute_request.submission_id
):
completion: completion_pb2.Completion = update.completion
break
update_response: update_service_pb2.GetUpdateResponse = (
us_client.GetUpdateById(
update_service_pb2.GetUpdateByIdRequest(
update_id=completion.update_id,
update_format=transaction_filter_pb2.UpdateFormat(
include_transactions=transaction_filter_pb2.TransactionFormat(
event_format=get_event_format(party),
transaction_shape=transaction_filter_pb2.TransactionShape.TRANSACTION_SHAPE_ACS_DELTA
)
)
)
)
)
for event in update_response.transaction.events:
if event.HasField("created"):
contract_id = event.created.contract_id
break
if event.HasField("archived"):
contract_id = event.archived.contract_id
break
# [Got Contract Id from Transaction]
return contract_id
```
To complete this part, the next step is to retrieve the binary blob of the creation event for the Ping contract. This serialized representation will be required in Part 2 when executing a choice on the contract.
```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
def get_active_contracts(party: str):
ledger_end_response: state_service_pb2.GetLedgerEndResponse = (
state_client.GetLedgerEnd(state_service_pb2.GetLedgerEndRequest())
)
active_contracts_response = state_client.GetActiveContracts(
state_service_pb2.GetActiveContractsRequest(
event_format=get_event_format(party),
active_at_offset=ledger_end_response.offset,
)
)
return active_contracts_response
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
ping_created_event: event_pb2.CreatedEvent
initiator_active_contracts = get_active_contracts(initiator)
# Find the contract in the active contract set
for active_contract_response in initiator_active_contracts:
if (
active_contract_response.HasField("active_contract")
and active_contract_response.active_contract.created_event.contract_id
== contract_id
):
ping_created_event = active_contract_response.active_contract.created_event
break
```
This concludes Part 1 of the tutorial. In Part 2, `Bob` exercises the `Respond` choice to archive the contract.
# Submit Externally Signed Transactions - Part 2
Complete Part 1 before proceeding.
The tutorial illustrates the external signing process using two external parties, `Alice` and `Bob`, leveraging the same Ping Daml Template used in Part 1 of the tutorial.
* In Part 1 `Alice` created a `Ping` contract.
* In Part 2 `Bob` exercises the `Respond` choice on the contract and archives it.
The majority of the work involved in external transaction signing was completed in Part 1. The key addition in Part 2 is utilizing the Ping contract created earlier through **explicit disclosure** and executing the Respond choice on that contract. The overall process remains similar to Part 1.
This tutorial is for demo purposes. The code snippets should not be used directly in a production environment.
## Setup
To proceed, gather the following information:
* `Bob`'s Party ID, protocol signing private key, and protocol signing public key fingerprint
* Synchronizer ID of the synchronizer to which the participant is connected
* gRPC Ledger API endpoint
* `ping_created_event`: Event retrieved in the last step of Part 1.
* `contract_id`: ID of the contract created in Part 1.
This information should already be known from the onboarding tutorial and the first part of the external signing tutorial.
## Python
If you are following this tutorial in Python, generate gRPC Python classes by following the setup instructions in the `README` in the example folder.
## Exercise `Respond` Choice
This tutorial does not repeat the material covered in Part 1 regarding transaction preparation, validation, signing, and execution, as these steps remain largely the same. Instead, it highlights the key differences from Part 1.
### Prepare the transaction
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
ping_exercise_command = commands_pb2.Command(
exercise=commands_pb2.ExerciseCommand(
template_id=ping_template_id,
contract_id=contract_id,
choice="Respond",
choice_argument=value_pb2.Value(
record=value_pb2.Record(record_id=None, fields=[])
),
)
)
```
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
prepare_exercise_request = interactive_submission_service_pb2.PrepareSubmissionRequest(
user_id=user_id,
command_id=str(uuid.uuid4()),
act_as=[responder],
read_as=[responder],
synchronizer_id=synchronizer_id,
commands=[ping_exercise_command],
# We need to explicitly disclosed the ping contract we created earlier
disclosed_contracts=[
commands_pb2.DisclosedContract(
template_id=template_id,
contract_id=contract_id,
created_event_blob=created_event_blob,
synchronizer_id=synchronizer_id,
)
],
)
prepare_exercise_response = iss_client.PrepareSubmission(prepare_exercise_request)
```
The `Prepare` request is very similar to the one from Part 1, with the following differences:
* `act_as`: Now the `responder`, `Bob`, instead of the `initiator`, `Alice`. This makes sense because `Bob` is the one exercising the choice on the contract.
* `commands`: The command is now an `Exercise`command instead of a `Create` command. Notably it requires the `contract_id` from Part 1.
* `disclosed_contracts`: The serialized representation of contracts required to process the transaction.
#### Metadata
The only significant difference with Part 1 in the metadata is: `disclosed_events`. This field now contains the input `Ping` contract. It is also included in the hash of the transaction.
Like in Part 1, the transaction must be validated, hashed and signed. The hash computation and signature is performed by the `execute_and_get_contract_id` function provided at the end of Part 1, as shown in the next section.
### Submit and observe archived contract
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
execute_and_get_contract_id(
prepared_exercise_transaction,
responder,
responder_private_key,
responder_fingerprint,
)
# The contract was archived by exercising the choice, we get an archived event this time
contract_events = get_events(responder, contract_id)
if contract_events.HasField("archived"):
print(
f"Ping contract with ID {contract_events.archived.archived_event.contract_id} has been archived"
)
else:
raise Exception("Expected an archive event")
```
By querying the event service and filtering for the contract ID, an archived event is observed, confirming that the contract has been successfully archived.
This concludes the external signing tutorial. The code used in this tutorial is available in the `examples/08-interactive-submission` folder and can be run with
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
python interactive_submission.py run-demo
```
## Tooling
The scripts mentioned in this tutorial can be used as tools for testing and development purposes
### Decode base64 encoded prepared transaction to JSON
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
./setup.sh
python daml_transaction_util.py --decode --base64
```
### Compute hash of base64 encoded prepared transaction
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
./setup.sh
python daml_transaction_util.py --hash --base64
```
# How to allocate and query Daml parties
Source: https://docs.canton.network/appdev/deep-dives/manage-daml-parties
Allocate new parties on a participant and query party metadata.
# How to allocate and query Daml parties
Canton Participant Node exposes a party management service that allows creation and discovery of the Daml parties. This guide explains how to programmatically manipulate the parties using the JSON Ledger API, which is described using OpenAPI specifications.
To learn about the Daml parties and users, see Daml parties and users in the key concepts section.
Refer to the party management section in the operational guide to learn how the parties can be created using the Canton console.
## Start Canton Participant Node
Ensure that your Canton Participant Node opens a JSON Ledger API HTTP port by adding a flag to the Canton Participant startup
```sh theme={"theme":{"light":"github-light","dark":"github-dark"}}
-C canton.participants.participant1.http-ledger-api.port=7575
```
Alternatively, enable the [JSON Ledger API](/appdev/quickstart/json-api) in the [Canton config](/global-synchronizer/reference/canton-configuration-guide) file.
Start the Canton Participant Node and connect it to the Synchronizer. If you are unfamiliar with the procedure, review [Connecting the Participant nodes and synchronizers](/global-synchronizer/canton-console/getting-started-tutorial#connecting-the-participant-nodes-and-synchronizers).
## How to query for existing parties
To list all parties known to the participant, issue a GET request towards the `v2/parties` endpoint.
```sh theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl http://localhost:7575/v2/parties
```
The participant responds with a message containing all the known parties
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"partyDetails": [
{
"party": "Alice::122091f5d8d174bc0d624616d4f366904f8d4c56d56e33508878db3156c3dd9b8ae9",
"isLocal": true,
"localMetadata": {
"resourceVersion": "0",
"annotations": {}
},
"identityProviderId": ""
},
{
"party": "Bob::122091f5d8d174bc0d624616d4f366904f8d4c56d56e33508878db3156c3dd9b8ae9",
"isLocal": true,
"localMetadata": {
"resourceVersion": "0",
"annotations": {}
},
"identityProviderId": ""
},
{
"party": "ee1d49e9-fa52-480a-8e85-033738a1fc75::122091f5d8d174bc0d624616d4f366904f8d4c56d56e33508878db3156c3dd9b8ae9",
"isLocal": true,
"localMetadata": {
"resourceVersion": "",
"annotations": {}
},
"identityProviderId": ""
}
],
"nextPageToken": ""
}
```
The `isLocal` attribute is set to true if the participant hosts the party and the party shares the same identity provider as the user issuing the request.
## How to create a new local party
To create a new party, issue a POST request towards the `v2/parties` endpoint.
```sh theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -d '{"partyIdHint":"Alice"}' http://localhost:7575/v2/parties
```
The resulting party id is composed of the supplied `partyIdHint` and the namespace fingerprint of the entity overseeing that party. Typically, it is the fingerprint associated with the Canton Participant Node.
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"partyDetails": {
"party": "Alice::122091f5d8d174bc0d624616d4f366904f8d4c56d56e33508878db3156c3dd9b8ae9",
"isLocal": true,
"localMetadata": {
"resourceVersion": "0",
"annotations": {}
},
"identityProviderId": ""
}
}
```
If you omit the `partIdHint` in your request, the Participant Node selects a random hint string.
# Multi-Hosting and Resilience
Source: https://docs.canton.network/appdev/deep-dives/multi-hosting
Distributing parties across multiple validators for high availability, failover, and data resilience
Multi-hosting lets a single party be hosted on multiple validators simultaneously. If one validator goes down, the party's operations continue on the others — without changing any Daml logic. This deep dive covers how multi-hosting works, when to use it, and how to set it up.
## Why Multi-Host?
A party hosted on a single validator has a single point of failure. If that validator goes offline, the party can't submit transactions or receive updates until it recovers. Multi-hosting addresses this by distributing the party across multiple validators.
Common reasons to multi-host:
* **High availability** — Business-critical parties that can't tolerate downtime
* **Geographic redundancy** — Validators in different regions to survive regional outages
* **Validator migration** — Gradually shifting a party from one validator to another without downtime
* **Organizational resilience** — Separating operational risk across independently managed validators
* **Protect against a malicious operator** — Using a threshold value higher than one prevents a single malicious operator from invalid activity
## How Multi-Hosting Works
Multi-hosting is managed through Canton's topology system. A party-to-participant mapping declares which validators host a given party and what permission each validator has (Submission, Confirmation, or Observation).
When a local party is hosted on multiple validators with Submission permission, any of those validators can submit commands on behalf of the party. For external parties, submission happens on any validator where the party is hosted with Confirmation permission. The synchronizer delivers transaction views to all hosting validators, so each one maintains a consistent view of the party's contracts.
### Confirmation Thresholds
You can configure a confirmation threshold that determines how many hosting validators must confirm a transaction before it proceeds. With a threshold of 1 (the default when not specified), any single hosting validator can confirm. With a higher threshold, multiple validators must independently validate and confirm, providing stronger integrity guarantees at the cost of additional latency. A threshold greater than 1 protects against a malicious validator.
### Permission Levels
Each hosting validator is assigned one of three permission levels:
* **Submission** — The validator can submit commands on behalf of the party. This permission is only available for local parties (parties whose keys are managed by the validator).
* **Confirmation** — The validator can confirm transactions for the party. This is the standard permission for active hosting.
* **Observation** — The validator receives transaction data for the party but cannot submit or confirm. Useful for read-only replicas, audit nodes, or pre-staging a migration target.
## Setting Up Multi-Hosting for New Parties
Multi-hosting requires a topology transaction that maps the party to multiple validators. All hosting validators must sign the mapping — it's a proposal that becomes active only when all parties agree.
The instructions below apply to **new external parties**. Adding hosting nodes to existing parties is called [party replication](/global-synchronizer/production-operations/party-management#simple-party-replication) and is a different, more involved workflow. For external parties, the external party must authorize (sign) the party to participant mapping with its own key. See the [external signing onboarding documentation](/appdev/deep-dives/external-signing-onboarding) for details.
### Via the Ledger API
When onboarding an external party, include additional validators in the topology request:
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"otherConfirmingParticipantUids": ["PAR::participant2::1220a4d7..."],
"confirmationThreshold": 1
}
```
The generated topology transactions need to be uploaded to each hosting validator's Ledger API. When a party-to-participant mapping is uploaded through the allocate endpoint which mentions the local validator, it is automatically signed by the local validator and forwarded to the network. If the topology transaction is not fully authorized (some signatures are still missing), it is treated as a proposal.
If the proposal already exists on the network, the new signatures are merged into the proposal. Once enough signatures are present, the topology transaction is accepted and added to the state.
### Via the Canton Console
You can also set up multi-hosting directly through the Canton Console. This approach is useful for internal parties where you have console access to both validators.
Create a hosting proposal on the first validator:
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.topology.party_to_participant_mappings.propose(partyId, newParticipants = Seq((participant1.id, ParticipantPermission.Confirmation), (participant2.id, ParticipantPermission.Confirmation)), store = synchronizerId)
res1: SignedTopologyTransaction[TopologyChangeOp, PartyToParticipant] = SignedTopologyTransaction(
TopologyTransaction(
PartyToParticipant(
partyId = Alice::12201ff69b1d...,
participants = Map(
PAR::participant1::12201ff69b1d... -> Confirmation,
PAR::participant2::1220a4d7463b... -> Confirmation
)
),
serial = 2,
operation = Replace,
hash = SHA-256:f77e8ccd88df...
),
signatures = 12201ff69b1d...,
proposal
)
```
On the second validator, list pending proposals and authorize:
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ val proposals = participant2.topology.party_to_participant_mappings.list_hosting_proposals(synchronizerId, participant2.id)
proposals : Seq[com.digitalasset.canton.admin.api.client.data.topology.ListMultiHostingProposal] = Vector(
ListMultiHostingProposal(
txHash = SHA-256:f77e8ccd88df...,
party = Alice::12201ff69b1d...,
permission = Confirmation$,
others = PAR::participant1::12201ff69b1d... -> Confirmation$,
threshold = 1
)
)
```
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ proposals.map { p => participant2.topology.transactions.authorize(synchronizerId, p.txHash); p.txHash}
res3: Seq[TopologyTransaction.TxHash] = Vector(TxHash(hash = SHA-256:f77e8ccd88df...))
```
Once both validators have signed, the party appears as hosted on both nodes. You can verify with:
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.parties.hosted("Alice")
res4: Seq[ListPartiesResult] = Vector(
ListPartiesResult(
partyResult = Alice::12201ff69b1d...,
participants = Vector(
ParticipantSynchronizers(
participant = PAR::participant1::12201ff69b1d...,
synchronizers = Vector(
SynchronizerPermission(synchronizerId = local::122032922613..., permission = Submission)
)
)
)
)
)
```
## Data Resilience Patterns
Multi-hosting addresses compute resilience (can I submit transactions?), but you also need data resilience (can I query my party's contract state?). The Participant Query Store (PQS) is the primary query layer for Canton applications, and its availability matters as much as the validator itself.
For PQS deployment patterns — including high availability across hosting validators and sharing the PQS database with application tables — see [PQS (Participant Query Store)](/sdks-tools/development-tools/pqs#high-availability-and-database-sharing).
## When to Use Multi-Hosting vs. Other Resilience Strategies
Multi-hosting is one of several resilience approaches. Choose based on your requirements:
* **Single validator with database backups** — Sufficient when some downtime is acceptable and you can restore from backup. Simplest to operate.
* **Multi-hosting (two or more validators)** — Good for parties that need continuous availability and need to protect against a malicious validator. Both validators independently maintain state, so failover is immediate.
* **Multi-hosting with observation nodes** — Add Observation-permission validators as read-only replicas for query scaling or audit purposes without increasing confirmation overhead.
* **Multi-hosting across synchronizers** — For workflows that span multiple synchronizers, each hosting validator can connect to different synchronizers, providing both resilience and multi-synchronizer access.
## Operational Considerations
* **Cost** — Each hosting validator with Confirmation permission independently processes and confirms transactions, which increases traffic consumption proportionally. The exact cost increase depends on the number of confirming validators and the confirmation threshold.
* **Consistency** — All hosting validators see the same transactions (the synchronizer ensures this), but PQS queries may need to take into account offsets being a little different.
* **Key management** — Each hosting validator holds its own signing keys. Compromising one validator's keys doesn't compromise the others, but the party's security depends on the threshold configuration.
* **Removing a host** — To stop hosting a party on a validator, submit a new topology transaction that removes that validator from the mapping. The party's contracts remain accessible on the other hosting validators.
## Next Steps
* [Decentralization](/appdev/deep-dives/decentralization) — How multi-hosting fits into Canton's decentralization spectrum
* [Composition and Multi-Party Workflows](/appdev/deep-dives/composition-multi-party) — Daml patterns for multi-party interactions
# Open Tracing in Ledger API Client Applications
Source: https://docs.canton.network/appdev/deep-dives/open-tracing
Adding OpenTelemetry-based distributed tracing to Daml applications interacting with the Ledger API.
## Introduction
Distributed tracing is a technique used for troubleshooting performance issues in a microservices environment like Daml Enterprise. Tracing in Canton has been described in a page dedicated to monitoring (Canton Monitoring / Tracing). This guide describes how to write **Ledger API** client applications so that distributed traces and spans can seamlessly continue between the client and Canton software components.
To study a **Ledger API** client application with OpenTelemetry support in detail, see this [example on GitHub](https://github.com/digital-asset/ex-java-bindings-with-opentelemetry).
The example implements a variation of the already familiar `PingPong` application where every call to the **Ledger API** is decorated with an OpenTelemetry trace context and demonstrates how to retrieve the trace context from past transactions.
To familiarize yourself with the broader topic of open tracing, consult the official pages of [the OpenTelemetry project](https://opentelemetry.io/). To find out more about open tracing in Java, the documentation on [Java OpenTelemetry instrumentation](https://opentelemetry.io/docs/instrumentation/java/) is an excellent source of references and examples.
## Set Up an OpenTelemetry Environment
To observe distributed tracing in action, you first need to start an OpenTelemetry backend server. Canton supports Jaeger, Zipkin, or OTLP formats. To start a Jaeger server you can use the following docker command:
docker run --rm -it --name jaeger\
-p 16686:16686 \
-p 14250:14250 \
jaegertracing/all-in-one:1.22.0
You also have to start Canton with OpenTelemetry exporting enabled. You can achieve it by defining a new `jaeger.conf` configuration file:
```
canton.monitoring.tracing.tracer.exporter {
type = jaeger
address = "localhost" // it's the default, so can be omitted
port = 14250 // it's the default, so can be omitted
}
```
Next, launch a small Canton installation combining the `jaeger.conf` into the configuration mix:
bin/canton -c examples/01-simple-topology/simple-topology.conf -c jaeger.conf
## Add Project Dependencies
To use the OpenTelemetry libraries, add the following **Maven** dependencies to your project's `pom.xml`:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
io.opentelemetry
opentelemetry-api
1.29.0
io.opentelemetry
opentelemetry-exporter-jaeger
1.29.0
io.opentelemetry
opentelemetry-sdk
1.29.0
io.opentelemetry.instrumentation
opentelemetry-grpc-1.6
1.29.0-alpha
```
Replace the version number in each dependency with the version you want to use. To find available versions, check the [Maven Central Repository](https://search.maven.org/artifact/io.opentelemetry/opentelemetry-api).
## Initialize
An application that wants to use OpenTelemetry must initialize a number of global controller objects that orchestrate different aspects of the distributed tracing process such as span creation, propagation, and export. The exact set of controllers needed may vary from application to application. You may draw some inspiration from the selection used in the example inside [the OpenTelemetryUtil.createOpenTelemetry method](https://github.com/digital-asset/ex-java-bindings-with-opentelemetry/blob/master/src/main/java/examples/pingpong/codegen/OpenTelemetryUtil.java). This is the minimum set required for a fully functional Jaeger trace reporting.
The next step is to initialize the GRPCTelemetry controller, which is responsible for the propagation of the trace contexts inside the HTTP2 headers of the gRPC communication.
The example wraps the necessary initialization steps in the constructor of the OpenTelemetryUtil class. All you have to do is call:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
OpenTelemetryUtil openTelemetry = new OpenTelemetryUtil(APP_ID);
```
The GRPCTelemetry controller can construct client call interceptors that need to be mounted on top of the **Netty** channels used in the gRPC communication. The example provides a useful helper method called `withClientInterceptor` that injects an interceptor at the channel builder level:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
ManagedChannel channel = openTelemetry.withClientInterceptor(
ManagedChannelBuilder
.forAddress(host, port)
.usePlaintext()
)
.build();
```
And with that, you are all set to start generating own spans, reporting them to the **Jaeger** server and also propagating them transparently to the **Ledger API**.
## Start New Spans
Before making a gRPC call, you must generate a new span to cover the multi-component interaction that is about to be initiated. The example provides a useful combinator called `runInNewSpan` that wraps the execution of an arbitrary function in a newly generated span:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
public R runInNewSpan(String spanName, Supplier body) {
Span span = tracer.spanBuilder(spanName).startSpan();
try(Scope ignored = span.makeCurrent()) {
return body.get();
} finally {
span.end();
}
}
```
You can use it on a command submission as follows:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
openTelemetry.runInNewSpan("createInitialContracts", () -> submissionService.submit(request));
```
The gRPC interceptors that were mounted at the initialization stage do the rest of the work behind the scenes making sure that the spans make it across to the Canton.
## Continue Spans Across Different Applications
Sometimes you may wish to continue the same span across multiple Daml transactions forming a single workflow. This may be especially interesting when different client application instances interact through the ledger and yet their entire conversation should be seen as a single coherent succession of spans. In that case, it is possible to extract the trace context associated with the past transactions from the Transaction, TransactionTree, or Completion records that are returned from the following **Ledger API** calls:
* `UpdateService.GetUpdates`
* `UpdateService.GetUpdateTrees`
* `UpdateService.GetTransactionByOffset`
* `UpdateService.GetTransactionById`
* `UpdateService.GetTransactionTreeByOffset`
* `UpdateService.GetTransactionTreeById`
* `UpdateService.GetUpdateByOffset`
* `UpdateService.GetUpdateById`
* `CommandCompletionService.CompletionStream`
You can extract the context by using a helper function implemented in the example:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
Context extractedContext = openTelemetry.contextFromDamlTraceContext(tx.getTraceContext());
```
The extracted context then has to be elevated to the status of the current context. Doing this allows the continuation of the original trace context into the present operation. Again the example provides a convenient combinator for that:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
openTelemetry.runInOpenTelemetryScope(extractedContext, () -> ... );
```
Finally, you generate a new span within the original context. You can use the already familiar `runInNewSpan` method:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
openTelemetry.runInNewSpan("follow", () ->
submissionService.submit(SubmitRequest.toProto(ledgerId, commandsSubmission))
)
```
## Put It All Together
When the client applications follow the rules and pass the trace contexts without interruption, it becomes possible to witness the entire workflow as one long succession of spans in Jaeger UI. The span diagram collected while running the example application is shown below:
![images/jaegerPingSpans.png]()
# Performance Optimization
Source: https://docs.canton.network/appdev/deep-dives/performance-optimization
Scale Canton applications for throughput and latency: network and node scaling, contention avoidance, ACS sizing, and batching.
# Scaling and Performance
## Network Scaling
The scaling and performance characteristics of a Canton-based system are determined by many factors. The simplest approach is to deploy Canton as a simple monolith where vertical scaling would add more CPUs, memory, etc. to the compute resource. However, the most frequent and expected deployment of Canton is as a distributed, micro-service architecture, running in different data centers of different organizations, with many opportunities to incrementally increase throughput. This is outlined below.
The ledger state in Canton does not exist globally so there is no single node that, by design, hosts all contracts. Instead, participant nodes are involved in transactions that operate on the ledger state on a strict need-to-know basis (data minimization), only exchanging (encrypted) information on the synchronizers used as coordination points for the given input contracts. For example, if participants Alice and Bank transact on an i-owe-you contract on synchronizer A, another participant Bob, or another synchronizer B, does not receive a single bit related to this transaction. This is in contrast to blockchains, where each node has to process each block regardless of how active or directly affected they are by a given transaction. This lends itself to a micro-service approach that can scale horizontally.
The micro-services deployment of Canton includes the set of participant nodes (hereafter, "participant" or "participants") and synchronizers, as well as the services internal to the synchronizer (e.g., Topology Manager). In general, each Canton micro-service follows the best practice of having its own local database which increases throughput. Deploying a service to its own compute server increases throughput because of the additional CPU and disk capacity. A vertical scaling approach can be used to increase throughput if a single service becomes a bottleneck, along with the option of horizontal scaling that is discussed next.
An initial Canton deployment can increase its scaling in multiple ways that build on each other. If a single participant node has many parties, then throughput can be increased by migrating parties off to a new, additional participant node (currently supported as a manual early access feature). For example, if 100 parties are performing multi-lateral transactions with each other, then the system can reallocate parties to 10 participants with 10 parties each, or 100 participants with 1 party each. As most of the computation occurs on the participants, a synchronizer can sustain a very substantial load from multiple participants. If the synchronizer were to be a bottleneck then the sequencer(s), topology manager, and mediator can be run on their own compute server which increases the synchronizer throughput. Therefore, new compute servers with additional Canton nodes can be added to the network when needed, allowing the entire system to scale horizontally.
If even more throughput is needed then the multi-synchronizer feature of Canton can be leveraged to increase throughput. In a large and active network where a synchronizer reaches the capacity limit, additional synchronizers can be rolled out, such that the workflows can be sharded over the available synchronizers (early access). This is a standard technique for load balancing where the client application does the load balancing via sharding.
If a single party is a bottleneck then the throughput can be increased by sharding the workflow across multiple parties hosted on separate participants. If a workflow is involving some large operator (i.e. an exchange), then an option would be to shard the operator by creating two operator parties and distribute the workflows evenly over the two operators (eventually hosted on different participants), and by adding some intermediate steps for the few cases where the workflows would span across the two shards.
Some anti-patterns need to be avoided for the maximum scaling opportunity. For example, having almost all of the parties on a single participant is an anti-pattern to be avoided since that participant will be a bottleneck. Similarly, the design of the Daml model has a strong impact on the degree to which sharding is possible. For example, having a Daml application that introduces a synchronization party through which all transactions need to be validated introduces a bottleneck so it is also an anti-pattern to avoid.
The bottom line is that a Canton system can scale out horizontally if commands involve only a small number of participants and synchronizers.
## Node Scaling
Canton supports the following scaling of nodes:
* The database-backed drivers (Postgres and Oracle) can run in an active-active setup with parallel processing, supporting multiple writer and reader processes. Thus, such nodes can scale horizontally.
* Canton processes make use of multiple CPUs and will detect the number of available CPUs automatically (conflict detection between transactions must run sequentially, but interpretation of each transaction can run in parallel). The number of parallel threads can be controlled by setting the JVM properties `scala.concurrent.context.numThreads` to the desired value.
Generally, the performance of Canton nodes is currently storage I/O bound. Therefore, their performance depends on the scaling behavior and throughput performance of the underlying storage layer, which can be a database or a distributed ledger for some drivers. Therefore, appropriately sizing the database is key to achieving the necessary performance.
On a related note: the Daml interpretation is a pure operation, without side-effects. Therefore, the interpretation of each transaction can run in parallel, and only the conflict detection between transactions must run sequentially.
## Performance and Sizing
A Daml workflow can be computationally arbitrarily complex, performing lots of computation (cpu!) or fetching many contracts (io!), and involve different numbers of parties, participants, and synchronizers. Canton nodes store their entire data in the storage layer (database), with additional indexes. Every workflow and topology is different, and therefore, sizing requirements depend on the Daml application that is going to run, and on the resource requirements of the storage layer. Therefore, to obtain sizing estimates you must measure the resource usage of dominant workflows using a representative topology and setup of your use case.
## Batching
As every transaction comes with an overhead (signatures, symmetric encryption keys, serialization and wrapping into messages for transport, HTTP headers, etc), we recommend designing the applications submitting commands in a way that batches smaller requests together into a single transaction.
Optimal batch sizes depend on the workflow and the topology and need to be determined experimentally.
## Asynchronous Submissions
In order to achieve the best performance, we suggest that you use asynchronous command submissions. However, please note that the async submission is only partially asynchronous, as the initial command interpretation and transaction building are included in that step, while the transaction validation and result finalization are not. This means that an async submission takes between 50 to 1000 ms, depending on command size and complexity. In the extreme case with a single thread submitting transactions, this would mean that you would only achieve a rate of one command per second.
If you use synchronous command submissions, the system will wait for the entire transaction to complete, which will require even more threads. Also, please note that the synchronous command submission has a default upper limit of 256 in flight commands, which can be reconfigured using
```conf theme={"theme":{"light":"github-light","dark":"github-dark"}}
canton.participants.participant1.ledger-api.command-service.max-commands-in-flight = 256 // default value
```
## Storage Estimation
A priori storage estimation of a Canton installation is tricky. As explained above, storage usage depends on topology, payload, Daml models used, and what type of storage layer is configured. However, the following example may help you understand the storage usage for your use case:
First, a command submitted through the gRPC Ledger API is sent to the participant as a serialized gRPC request.
This command is first interpreted and translated into a Daml-LF transaction. The interpreted transaction is next translated into a Canton transaction view-decomposition, which is a privacy-preserving representation of the full transaction tree structure. A transaction typically consists of several transaction views; in the worst case, every action node in the transaction tree becomes a separate transaction view. Each view contains the full set of arguments required by that view, including the contract arguments of the input contracts. So the data representation can be multiplied quite a bit. Here, we cannot estimate the resulting size without having a concrete example. For simplicity, let us consider the simple case where a participant is exercising a simple "Transfer" choice on an typical "Iou" contract to a new owner, preserving the other contract arguments. We assume that the old and new owners of the IOU are hosted on the same participant whereas the IOU issuer is hosted on a second participant.
The resulting Canton transaction consists of two views (one for the **Exercise** node of the Transfer choice and one for the **Create** node of the transferred IOU). Both views contain some metadata such as the package and template identifiers, contract keys, stakeholders, and involved participants. The view for the **Exercise** node contains the contract arguments of the input IOU, say of size `Y`. The view for the **Create** node contains the updated contract arguments for the created contract, again of size `Y`. Note that there is no fixed relation between the command size `X` and the size of the input contracts `Y`. Typically `X` only contains the receiver of the transfer, but not the contract arguments that are stored on the ledger.
Then, we observe the following storage usage:
* Two encrypted envelopes with payload `Y` each, one view seed per view, one symmetric key per informee group, two root hashes for each participant and the participant IDs as recipients at the sequencer store, and the informee tree for the mediator (informees and transaction metadata, but no payload), together with the sequencer database indexes.
* Two encrypted envelopes with payload `Y` each and the symmetric keys for the views, in the participant events table of each participant (as both receive the data)
* Decrypted new resulting contract of size `Y` in the private contract store and some status information of that contract on the active contract journal of the sync service.
* The full decrypted transaction with a payload of size `Y` for the created contract, in the sync service linear event log. This transaction does not contain the input contract arguments.
* The full decrypted transaction with `Y` in the indexer events table, excluding input contracts, but including newly divulged input contracts.
If we assume that payloads dominate the storage requirements, we conclude that the storage requirement is given by the payload multiplication due to the view decomposition. In our example, the transaction requires `5\*Y` storage on each participant and `2\*Y` on the sequencer. For the two participants and the sequencer, this makes `12\*Y` in total.
Additionally to this, some indexes have to be built by the database to serve the contracts and events efficiently. The exact estimation of the size usage of such indexes for each database layer is beyond the scope of our documentation.
Please note that we do have plans to remove the storage duplication between the sync service and the indexer. Ideally, will be able to reduce the storage on the participant for this example from `5\*Y` down to \`3\*Y\`: once for the unencrypted created contract and twice for the two encrypted transaction views.
Generally, to recover used storage, a participant and a synchronizer can be pruned. Pruning is available through a set of console commands and allows removal of past events and archived contracts based on a timestamp. The storage usage of a Canton deployment can be kept constant by continuously removing obsolete data. Non-repudiation and auditability of the unpruned history are preserved due to the bilateral commitments.
## Set Up Canton to Get the Best Performance
In this section, the findings from internal performance tests are outlined to help you achieve optimal performance for your Canton application.
### System Design / Architecture
We recommend running the latest supported Canton release for optimal performance.
Plan your topology such that your Daml parties can be partitioned into independent blocks. That means most of your Daml commands involve parties of a single block only. It is ok if some commands involve parties of several (or all) blocks, as long as this happens only very rarely. In particular, avoid having a single master party that is involved in every command, because that party bottlenecks the system.
If your participants are becoming a bottleneck, add more participant nodes to your system. Make sure that each block runs on its own participant. If your synchronizer(s) are becoming a bottleneck, add more synchronizer nodes and distribute the load evenly over all synchronizers.
Prefer sending big commands with multiple actions (creates / exercises) over sending numerous small commands. Avoid sending unnecessary commands through the gRPC Ledger API. Try to minimize the payload of commands.
Further information can be found in Section `scaling_and_performance`.
### Hardware and Database
Do not run Canton nodes with an in-memory storage or with an H2 storage in production or during performance tests. You may observe very good performance in the beginning, but performance can degrade substantially once the data stores fill up.
Measure memory usage, CPU usage and disk throughput and improve your hardware as needed. For simplicity, it makes sense to start on a single machine. Once the resources of a machine are becoming a bottleneck, distribute your nodes and databases to different machines.
Try to make sure that the latency between a Canton node and its database is very low (ideally in the order of microseconds). The latency between Canton nodes has a much lower impact on throughput than the latency between a Canton node and its database.
Please check the Postgres persistence section for tuning instructions.
### Configuration
In the following, we go through the parameters with known impact on performance.
**Timeouts.** Under high load, you may observe that commands timeout. This will negatively impact throughput, because the commands consume resources without contributing to the number of accepted commands. To avoid this situation increase timeout parameters from the Canton console:
\#22917: Fix broken literalinclude literalinclude:: CANTON/scripts/canton-testing/config/run-synchronizers.canton start-after: user-manual-entry-begin: BumpSynchronizerTimeouts end-before: user-manual-entry-end: BumpSynchronizerTimeouts
If timeouts keep occurring, change your setup to submit commands at a lower rate. In addition, take the next paragraph on resource limits into account.
**Tune resource limits at the Canton protocol level.** Resource limits are used to prevent ledger applications from overloading Canton by sending commands at an excessive rate. While resource limits are necessary to protect the system from denial of service attacks in a production environment, they can prevent Canton from achieving maximum throughput. Resource limits at the Canton protocol level can be configured as follows from the Canton console:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
participant1.resources.set_resource_limits(
ResourceLimits(
// Allow for submitting at most 200 commands per second
maxSubmissionRate = Some(200),
// Limit the number of in-flight requests to 500.
// A "request" includes every transaction that needs to be validated by participant1:
// - transactions originating from commands submitted to participant1
// - transaction originating from commands submitted to different participants.
// The chosen configuration allows for processing up to 100 requests per second
// with an average latency of 5 seconds.
maxInflightValidationRequests = Some(500),
// Allow submission bursts of up to `factor * maxSubmissionRate`
maxSubmissionBurstFactor = 0.5,
)
)
```
As a rule of thumb, configure `maxDirtyRequests` to be slightly larger than `throughput * latency`, where
* `throughput` is the number of requests per second Canton needs to handle and
* `latency` is the time taken to process a single request while Canton is receiving requests at rate `throughput`.
You should run performance tests to ensure that `throughput` and `latency` are actually realistic. Otherwise, an application may overload Canton by submitting more requests than Canton can handle.
Configure the `maxRate` parameter to be slightly higher than the expected maximal `throughput`.
If you need to support command bursts, configure the `maxBurstFactor` accordingly. Then, the `maxRate` limitation will only start to enforce the rate after having received the initial burst of `maxBurstFactor * maxRate`.
To find optimal resource limits you need to run performance tests. The `maxDirtyRequest` parameter will protect Canton from being overloaded, if requests are arriving at a constant rate. The `maxRate` parameter offers additional protection, if requests are arriving at a variable rate.
If you choose higher resource limits, you may observe a higher throughput, at the risk of a higher latency. In the extreme case however, latency grows so much that commands will timeout; as a result, the command processing consumes resources even though some commands are not committed to the ledger.
If you choose lower resource limits, you may observe a lower latency, at the cost of lower throughput and commands getting rejected with the error code `PARTICIPANT_BACKPRESSURE`.
**Tune resource limits at the gRPC Ledger API level.** Resource limits can also be imposed on the gRPC Ledger API level. As these settings are applied closer to the ledger applications, they can be used for protecting the resources of individual participants rather than the entire Canton system.
You can modify the following configuration options:
```
canton.participants..ledger-api.rate-limit {
max-streams = 333
max-api-services-queue-size = 444
max-api-services-index-db-queue-size = 555
max-used-heap-space-percentage = 66
min-free-heap-space-bytes = 7777777
}
```
You can cap the number of gRPC Ledger API streams open at any given time by modifying the `max-streams` parameter. When the number of simultaneously open transaction, transaction-tree, completion, or acs streams reaches the maximum, it doesn't accept any additional get stream requests and returns a `MAXIMUM_NUMBER_OF_STREAMS` error code instead.
You can cap the number of items pending in the thread pools serving the Ledger API and the index database read requests by modifying the `max-api-services-queue-size` and `max-api-services-index-db-queue-size` respectively. When the CPU worker thread pool or the database communication thread pool is overloaded, the server responds with a `THREADPOOL_OVERLOADED` error code.
You can cap the percentage of the memory heap used by changing the `max-used-heap-space-percentage` parameter. If this percentage is exceeded following a garbage collection of the `tenured` memory pool, the system is rate-limited until additional space is freed up.
Similarly, you can set the minimum heap space in absolute terms by changing the `min-free-heap-space-bytes` parameter. If the amount of free space is below this value following a garbage collection of the `tenured` memory pool, the system is rate-limited until additional space is freed up. When the maximum memory thresholds are exceeded the server responds to gRPC Ledger API requests with a `HEAP_MEMORY_OVER_LIMIT` error code.
The following configuration values are the defaults:
```
max-streams = 1000
max-api-services-queue-size = 10000
max-api-services-index-db-queue-size = 1000
max-used-heap-space-percentage = 100
min-free-heap-space-bytes = 0
```
The memory-related settings of 100 for `max-used-heap-space-percentage` and 0 for `min-free-heap-space-bytes` render them effectively inactive. This is done on purpose. They are highly sensitive to the operating environment and should only be configured where memory profiling has highlighted spikes in memory usage that need to be flattened.
It is possible to turn off rate limiting at the gRPC Ledger API level:
```
canton.participants..ledger-api {
rate-limit = null
}
```
**Number of Open Streams**: Similarly to the Ledger API, the number of open streams on the Admin API and the Sequencer API can be configured using the `limits.active` section of the configuration:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
canton.sequencers.sequencer.public-api.limits.active = {
"com.digitalasset.canton.sequencer.api.v30.SequencerService/DownloadTopologyStateForInit" : 10,
"com.digitalasset.canton.sequencer.api.v30.SequencerService/Subscribe" : 1000,
}
```
**Size of connection pools.** Make sure that every node uses a connection pool to communicate with the database. This avoids the extra cost of creating a new connection on every database query. Canton chooses a suitable connection pool by default, but for performance sensitive applications you may want to optimize. Configure the maximum number of connections such that the database is fully loaded, but not overloaded. Allocating too many database connections will lead to resource waste (each thread costs), context switching and contention on the database system, slowing the overall system down. If this occurs, the query latencies reported by Canton go up (check the troubleshooting guide).
Detailed instructions on handling this issue can also be found in `max_connection_settings`.
**Size of database task queue.** If you are seeing frequent `RejectedExecutionExceptions` when Canton queries the database, increase the size of the task queue, as described in `database_task_queue_full`. The rejection is otherwise harmless. It just points out that the database is overloaded.
**Database Latency.** Ensure that the database latency is low. The higher the database latency, the lower the actual bandwidth and the lower the throughput of the system.
**Throttling configuration for SequencerClient.** The `SequencerClient` is the component responsible for managing the connection of any member (participant, mediator, or topology manager) in a Canton network to the synchronizer. Each synchronizer can have multiple sequencers, and the `SequencerClient` connects to one of them. However, there is a possibility that the `SequencerClient` can become overwhelmed and struggle to keep up with the incoming messages. To address this issue, a configuration parameter called `maximum-in-flight-event-batches` is available:
```conf theme={"theme":{"light":"github-light","dark":"github-dark"}}
canton.participants.participant1.sequencer-client.maximum-in-flight-event-batches = 100
canton.mediators.mediator1.sequencer-client.maximum-in-flight-event-batches = 100
canton.sequencers.sequencer1.sequencer-client.maximum-in-flight-event-batches = 100
canton.monitoring.metrics.qualifiers = [debug, traffic, errors, saturation, latency]
```
By setting the `maximum-in-flight-event-batches` parameter, you can control the maximum number of event batches that the system processes concurrently. This configuration helps prevent overload and ensures that the system can handle the workload effectively.
It's important to note that the value you choose for `maximum-in-flight-event-batches` impacts the `SequencerClient`'s performance in several ways. A higher value can potentially increase the `SequencerClient`'s throughput, allowing it to handle more events simultaneously. However, this comes at the cost of higher memory consumption and longer processing times for each batch.
On the other hand, a lower value for `maximum-in-flight-event-batches` might limit the throughput, as it can process fewer events concurrently. However, this approach can result in more stable and predictable `SequencerClient` behavior.
To monitor the performance of the `SequencerClient` and ensure it is operating within the desired limits, you can observe the metric `sequencer-client.handler.actual-in-flight-event-batches`. This metric provides the current value of the in-flight event batches, indicating how close it is to the configured limit. Additionally, you can also reference the metric `sequencer-client.handler.max-in-flight-event-batches` to determine the configured maximum value.
By monitoring these metrics, you can gain insights into the actual workload being processed and assess whether it is approaching the specified limit. This information is valuable for maintaining optimal `SequencerClient` performance and preventing any potential bottlenecks or overload situations.
**Turn on High-Throughput Sequencer.** The database sequencer has a number of parameters that can be tuned. The trade-off is low-latency or high-throughput. In the low-latency setting, every submission will be immediately processed as a single item. In the high-throughput setting, the sequencer will accumulate a few events before writing them together at once. While the latency added is only a few ms, it does make a difference during development and testing of your Daml applications. Therefore, the default setting is `low-latency`. A production deployment with high throughput demand should choose the `high-throughput` setting by configuring:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
// example setting for synchronizers nodes. database sequencer nodes have the exact same settings.
canton.sequencers.sequencer1.sequencer {
type = BFT
block {
writer = {
// choose between high-throughput or low-latency
type = high-throughput
}
}
}
```
There are additional parameters that can in theory be fine-tuned, but we recommend to leave the defaults and use either high-throughput or low-latency. In our experience, a high-throughput sequencer can handle several thousand submissions per second.
**JVM heap size.** In case you observe `OutOfMemoryErrors` or high overhead of garbage collection, you must increase the heap size of the JVM, as described in Section `jvm_arguments`. Use tools of your JVM provider (such as VisualVM) to monitor the garbage collector to check whether the heap size is tight.
**Size of thread pools.** Every Canton process has a thread pool for executing internal tasks. By default, the size of the thread-pool is configured as the number of (virtual) cores of the underlying (physical) machine. If the underlying machine runs other processes (e.g., a database) or if Canton runs inside of a container, the thread-pool may be too big, resulting in excessive context switching. To avoid that, configure the size of the thread pool explicitly like this:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
"bin/canton -Dscala.concurrent.context.numThreads=12 --config examples/01-simple-topology/simple-topology.conf"
```
As a result, Canton will log the following line:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
"INFO c.d.canton.environment.Environment - Deriving 12 as number of threads from '-Dscala.concurrent.context.numThreads'."
```
**Asynchronous commits.** If you are using a Postgres database, configure the Participant's Ledger API server to commit database transactions asynchronously by including the following line into your Canton configuration:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
canton.participants.participant1.ledger-api.postgres-data-source.synchronous-commit = off
```
**Logging Settings.** Make sure that Canton outputs log messages only at level INFO and above and turn off immediate log flushing using the `--log-immediate-flush=false` commandline flag, at the risk of missing log entries during a host system crash.
**Replication.** If (and **only if**) using single nodes for participant, sequencer, and/or mediator, replication can be turned off by setting `replication.enabled = false` in their respective configuration.
While replication can be turned off to try to obtain performance gains, it must **not** be disabled when running multiple nodes for HA.
**Caching Configuration.** In some cases, you might also want to tune caching configurations and either reduce or increase them, depending on your situation. This can also be helpful if you need to reduce the memory foot-print of Canton, which can be large, as the default cache configurations are tailored for high-throughput, high-memory and small transaction sizes.
Generally, the caches that usually matter with respect to size are the contract caches and the in-memory fan-out event buffer. You can tune these using the following configurations. The values depicted here are the ones recommended for smaller memory-footprints and are therefore also helpful if you run into out-of-memory issues:
```conf theme={"theme":{"light":"github-light","dark":"github-dark"}}
canton.participants.participant1 {
// tune caching configs of the ledger api server
ledger-api {
index-service {
max-contract-state-cache-size = 1000 // default 1e4
max-contract-key-state-cache-size = 1000 // default 1e4
// The in-memory fan-out will serve the transaction streams from memory as they are finalized, rather than
// using the database. Therefore, you should choose this buffer to be large enough such that the likeliness of
// applications having to stream transactions from the database is low. Generally, having a 10s buffer is
// sensible. Therefore, if you expect e.g. a throughput of 20 tx/s, then setting this number to 200 is sensible.
// The default setting assumes 100 tx/s.
max-transactions-in-memory-fan-out-buffer-size = 200 // default 1000
}
}
// tune the synchronisation protocols contract store cache
parameters.caching {
contract-store {
maximum-size = 1000 // default 1e6
expire-after-access = 120s // default 10 minutes
}
}
}
```
## Model Tuning
How you write your Daml model has a large impact on the performance of your system. There are instances of good models running with 3000 ledger events/second (v2.7) on a single participant node. A bad model will reach a fraction of that. Therefore, it is important to understand the connection between the model and the performance implications. This section aims to give a few guidelines.
### Reduce the Number of Views
One key performance driver in Canton is the transaction structure resulting from the model. A Daml command is effectively a "program" that computes a transaction structure. This transaction structure is then broken up by Canton into pieces called "transaction views". The transaction views are then encrypted and sent to the participants who then confirm each view. Each view requires cryptography and creates additional payload that needs to be processed and validated.
As a result, the performance of your system directly depends on how many views the transaction creates. The number of views of a transaction is logged on the participant side as a DEBUG log message:
```
Computed transaction tree with total=27 for #root-nodes=8
```
If you submit a Ledger API command (which can have multiple Daml commands), then every Daml command will create one so-called root-node, (two for the `CreateAndExercise` command). Each root node creates one view.
Just putting all Daml commands into a single batch command does not help, as you might still create lots of views.
Currently, Canton creates a view for every action node in the transaction tree if the **participants that host their informees are not a subset of their parent view's informee participants**. A view's informee participants are all the participants that must be informed about the view. Therefore, to reduce the number of views, you should try to reduce either the number of times the set of informees grows or make sure most of your actions share the same pool of participants. Alternatively, you can add informees to the action nodes close to the root (e.g., by including choice observers) so that their children nodes are aggregated in the initial view, since these children nodes target only a subset of those participants.
One concrete way to reduce the number of views being generated by a Daml model is to write batch commands and group the operations on the ledger based on their informees' participants. In other words, you can create a single choice and batch all the operations that share the same informee participant group.
You can also reduce the number of views being generated for the children of a node (i.e., parent-child views) when creating your Daml model. With the below example as reference, instead of sending a batch with 20 exercise commands `Foo`, you can group your contracts by a set of stakeholders that cover the exercise's informee participants and send one exercise command on a generator contract with the list of contracts that in turn exercises the Foo choice per stakeholder group. This way you can perform these operations in a single view, instead of having them spawn multiple different views.
For example, if the `doUpdate` below is called from a choice visible to the `owner` party alone, then whether the `efficient` flag is set or not has a huge impact on the number of views created. In the choice `Foo` of the `Example` contract, even though the participants to be informed are a subset of the contract's informee participants they are not merged into a single view if this choice is called independently each time. On the contrary, you will produce multiple root actions each belonging to their own view. Finally, aligning the choice observer of `Run` to its template contract means that either the choice observer is the same in each case or the new observer is hosted by any of the contract's informee participants.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
doUpdate : Bool -> Party -> Party -> [ContractId Example] -> Update ()
doUpdate efficient owner obs batch = do
if efficient then do
batcher <- create BatchFoo with owner
-- This only works out if obs is the same observer on all the exercised Example contracts
batcher `exercise` Run with batch = batch; obs = obs
else do
-- Canton will create one view per exercise
forA_ batch (`exercise` Foo)
pure ()
template Example with
owner : Party
obs: Party
where
signatory owner
observer obs
choice Foo : ()
controller owner
do
return ()
template BatchFoo with
owner : Party
where
signatory owner
choice Run : () with
batch : [ContractId Example]
obs : Party
-- The observer here is a choice observer. Therefore, Canton will
-- ship a single view with the top node of this choice if the observer of the
-- choice aligns with the observer of the contract.
observer obs
controller owner
do
forA_ batch (`exercise` Foo)
```
As a rule, the number of views should depend on the number of groups of informee participants you have in your batch choice, not the number of "batches" you process in parallel within one command. An informee's participants group is formed by aggregating the participants that host each stakeholder, or in other words, the set of participants that need to see a particular view.
The informees for the different type of transaction tree nodes are (also see `da-model-projections`):
* create: signatories, observers
* consuming exercise: signatories, observers, stakeholders of the choice (controller, choice observers)
* nonconsuming exercise: signatories, stakeholders of the choice (controller, choice observers), but not the observers of the contract
* fetch: signatories + actors of the fetch, which are all stakholders which are in the authorization context that the fetch executed in
* lookupByKey: only key maintainers
### Reduce Ledger Events
The best optimisation is always to just not do something. A ledger is meant to store relevant data and coordinate different parties. If you use the ledger as processing queue, you run into performance issues, as each transaction view is cryptographically secured and processed extensively to ensure privacy and integrity, which is unnecessary for intermediate steps. The rule is: if the output of a transaction submitted by a party is immediately consumed by the same party, then you are using the ledger as a processing queue. Instead, restructure your command such that the two steps happen as one.
Furthermore, each `create` creates a contract, causing data to be written to the ledger. Each `fetch` causes the interpreter to halt interpretation, asking the ledger for a contract, which in the worst case requires a lookup in the database. As the interpretation must happen sequentially, this means a one-by-one lookup of contracts, causing load and latency. Fetching data is important and a key feature, but you should apply the same reasoning as you would for a database: A database lookup is expensive and should only be done if necessary, ideally caching repetive computing results.
As an example, if you have a high throughput process that always resolves some data by a chain `A -\> B -\> C -\> D` (four fetches), then change your model to use a single cache contract `ABCD`, only fetch that contract and ensure that there is a process that whenever one of the contracts changes, `ABCD` is updated.
Also, avoid unnecessary transient contracts, as they may cause additional views. `createAndExercise` is not a single command, but translated to one create and an exercise. Use a `nonconsuming` choice instead, possibly with a choice observer if you need to leave a trace on the ledger for audit purposes.
Generally, using `lookupKey` is also discouraged. If you use `lookupByKey` for on-ledger deduplication, the lookup requires a database lookup, as the lookupByKey will resolve to `None` in most cases. Instead, use command deduplication on the Ledger API. Second, the current `lookupByKey` will not be supported in a multi-synchronizer deployment, as the current semantic only works on a single-synchronizer deployment and can not be translated 1:1 to multi-synchronizer.
# Managing Latency and Throughput
## Problem Definition
Latency is a measure of how long a business transaction takes to complete. Throughput measures, on average, the number of business transactions possible per second while taking into account any lower or upper bounds which may point to bottlenecks. Defense against latency and throughput issues can be written into the Daml application during design.
First we need to identify the potential bottlenecks in a Daml application. We can do this by analyzing the sync-domain-specific transactions.
Each Daml business transaction kicks off when a Ledger API client sends the commands `create` or `exercise` to a participant.
* Ledger transactions are not synonymous with business transactions.
* Often a complete business transaction spans multiple workflow steps and thus multiple ledger transactions.
* Multiple business transactions can be processed in a single ledger transaction through batching.
* Expected ledger transaction latencies are on the order of 0.5-1 seconds on database sequencers, and multiple seconds on blockchain sequencers.
Refer to the [Daml execution model](/appdev/modules/m3-design-patterns#daml’s-execution-model) that describes a ledger transaction processed by the Canton ledger. The table below highlights potential resource-intensive activities at each step.
| Step | Participant | Resources used | Possible bottleneck drivers |
| -------------- | ------------------------------------------- | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Interpretation | Submitting participant node | CPU
Memory
DB read access | Calculation complexity
Size and number of variables
Number of contract fetches |
| Blinding | Submitting participant node | CPU/memory | Number and size of views |
| Submission | Submitting participant node
Sequencer | CPU
Memory | Serialization/deserialization
Transaction size/number of views |
| Sequencing | Sequencer | Backend storage
Network bandwidth | Transaction size
Transaction size/number of views |
| Validation | Receiving participant nodes | Network bandwidth
CPU
Memory
DB read throughput | Transaction size -> download, deserialization, storage costs
Computation complexity
Number of contract fetch reads
Number and size of variables |
| Confirmation | Validating participant nodes
Sequencer | Network bandwidth
Sequencer network
Backend write throughput | Number of confirming parties |
| Mediation | Mediator nodes | Network throughput
CPU
Memory | Number of confirming parties |
| Commit | Mediator nodes
Sequencer | CPU
Memory
DB
Network bandwidth | Number of confirming parties |
### Possible Throughput Bottlenecks in Order of Likelihood
1. Transaction size causing high serialization/deserialization and encryption/decryption costs on participant nodes.
2. Transaction size causing sequencer backend overload, especially on blockchains.
3. High interpretation and validation cost due to calculation complexity or memory use.
4. Large number of involved nodes and associated network bandwidth on sequencer.
Latency can also be affected by the above factors. However, baseline latency usually has more to do with system set-up issues (DB or blockchain latency) rather than Daml modeling problems.
### Solutions
1. **Minimize transaction size.**
Each of the following actions in Daml adds a node to the transaction containing the payload of the contract being acted on. A large number of such operations, and/or operations of this kind on large contracts, are the most common cause of performance bottlenecks.
> * `create`
> * `fetch`
> * `fetchByKey`
> * `lookupByKey`
> * `exercise`
> * `exerciseByKey`
> * `archive`
Use the above actions sparingly. For example, if contracts have intermediary states within a transaction, you can often skip them by writing only the end state. For example:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Incrementor
with
p : Party
n : Int
where
signatory p
choice Increment : ContractId Incrementor
controller p
do create this with n = n+1
-- This adds all m-1 intermediary versions of
-- the contract to the transaction tree
choice BadIncrementMany : ContractId Incrementor
with m : Int
controller p
do foldlA (\self' _ -> exercise self' Increment) self [1..m]
-- This only adds the end result to the transaction
choice GoodIncrementMany : ContractId Incrementor
with m : Int
controller p
do create this with n = n+m
```
When you need to read a contract, or act on a single contract in multiple ways, you can often bundle those operations into a single action. For example:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Asset
with
issuer : Party
owner : Party
quantity : Decimal
where
signatory [issuer, owner]
-- BadMerge acts on each of the otherCids three times:
-- Once for validation
-- Once to extract the quantities
-- Once to archive
choice BadMerge : ContractId Asset
with otherCids : [ContractId Asset]
controller owner
do
-- validate the cids.
forA_ otherCids (\cid -> do
other <- fetch cid
assert (other.issuer == issuer && other.owner == owner))
-- extract the quantities
quantities <- forA otherCids (\cid -> do
other <- fetch cid
return other.quantity)
-- archive the others
forA_ otherCids archive
create this with quantity = quantity + sum quantities
-- Allow us to do a fetch and an archive in one action
choice ConsumingFetch : Asset
controller owner
do return this
-- GoodMerge only acts on each of the other assets once.
choice GoodMerge : ContractId Asset
with otherCids : [ContractId Asset]
controller owner
do
-- Get and archive the others
others <- forA otherCids (`exercise` ConsumingFetch)
-- validate
forA_ others (\other -> do
assert (other.issuer == issuer && other.owner == owner))
-- extract the quantities
let quantities = map (.quantity) others
create this with quantity = quantity + sum quantities
```
Separate templates for large payloads that change rarely and require minimum access from those for fields that change with almost every action. This optimizes resource consumption for multiple business transactions.
This batching approach makes updates in one transaction submission rather than requiring separate transactions for each update. Note: this option can cause a small increase in latency and may increase the possibility of command failure but this can be avoided. For example:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
where
signatory p
choice Foo : ()
controller p
do return ()
batching : Script ()
batching = do
p <- allocateParty "p"
-- without batching we have 10 ledger
-- transactions.
cid1 <- submit p do createCmd T with ..
cid2 <- submit p do createCmd T with ..
cid3 <- submit p do createCmd T with ..
cid4 <- submit p do createCmd T with ..
cid5 <- submit p do createCmd T with ..
submit p do exerciseCmd cid1 Foo
submit p do exerciseCmd cid2 Foo
submit p do exerciseCmd cid3 Foo
submit p do exerciseCmd cid4 Foo
submit p do exerciseCmd cid5 Foo
-- With batching, there are only two ledger transactions.
cids <- submit p do
replicateA 5 $ createCmd T with ..
submit p do
forA_ cids (`exerciseCmd` Foo)
```
2. CPU and memory issues: Use the Daml profiler to analyze Daml code execution.
3. Once you feel interpretation is not the bottleneck, scale up your machine.
Profile the JVM and monitor your databases to see where the bottlenecks occur.
# Managing Active Contract Set (ACS) Size
## Problem Definition
The Active Contract Set (ACS) size makes up the load related to the number of active contracts in the system at any one time. It means the totality of all the contracts that have been created but not yet archived. ACS size may come from a deliberate Daml workflow design, but the size may also be unexpected when insufficient care is given to supporting and auxiliary contract lifetimes.
See the documentation on Daml contracts for more information.
When the ACS size is in the high 100s GBs or TBs, local database access performance may deteriorate. We will look at potential issues around large ACS size and possible solutions.
## Relational Databases
Large ACS can have a negative impact on many aspects of system performance in relational databases. The following points focus on PostgreSQL as the underlying database; the details differ in the case of Oracle but the results are similar.
* Large ACS size directly affects the resource consumption and performance of a Ledger API client application dealing with a large data set that may not fit into the memory or the application database.
* ACS size directly affects the speed at which the ACS can be transmitted from the Ledger API server using the StateService. In extreme cases, it could take hours to transfer the complete set requested by the application due to the limits imposed by the gRPC channel capacity and the speed of storage queries.
* Increased latency is a less direct impact which shows up wherever a query is issued to the database index to make progress. Large ACS size means that the corresponding indices are also large, and at a certain point they will no longer fit into the shared-buffer space. It then takes increasingly longer for the database engine to produce query results. This affects activities such as contract lookups during the command submission, transaction tree streaming, or pointwise transaction lookups.
* Large ACS size may affect the speed at which the database underpinning the participant ingests new transactions. Normally, as new updates pour in the write-ahead log commits the table and index changes immediately. Those updates come in two shapes; full-page writes or differential writes. With large volumes, many are full-page writes.
* Finally, many dirty pages also translate into prolonged and expensive flushes to the disk as part of the checkpointing process.
### Solutions
* Pay attention to the lifetime of the contracts. Make sure that the supporting and auxiliary contracts don’t clutter the ACS and archive them as soon as it is practical to do so.
* Set up a frequent pruning schedule. Be aware that pruning is only effective if there are archived contracts available for pruning. If all contracts are still active, pruning has limited success. Refer to our [pruning documentation](/overview/reference/pruning) for more information.
* Implement an ODS in your ledger client application to limit reliance on read access to the ACS. Do this whenever you notice that the time to initialize the application from the ACS exceeds your pain level.
* Monitor database performance.
* Monitor the disk read and write activity. Look for sudden changes in the operation patterns. For instance, a sudden increase in the disk’s read activity may be a sign of indices no longer fitting into the shared buffers.
* Observe the performance of the database queries. Check our [monitoring documentation](/global-synchronizer/extension-synchronizers/synchronizer-monitoring) for query [metrics](/global-synchronizer/reference/canton-metrics#daml-sequencer-db-storage-write-executor-waittime) that can assist. You may also consider setting up a [log\_min\_duration\_statement parameter](https://www.postgresql.org/docs/current/runtime-config-logging.html#GUC-LOG-MIN-DURATION-STATEMENT) in the PostgreSQL configuration.
* Set up [autovacuum](https://www.postgresql.org/docs/current/routine-vacuuming.html#AUTOVACUUM) on the PostgreSQL database. Note that, after pruning, a lot of dead tuples will need removing.
# Avoid Contention Issues
Measuring the performance of business applications involves more than considering the transactions per second and transaction latency of the underlying blockchain and Distributed Ledger Technology (DLT). Blockchains are distributed systems; even the highest-performance blockchains have considerably higher transaction latencies than traditional databases. These factors make the systems prone to contention, which can stifle the performance of applications when not handled appropriately.
It is, unfortunately, easy to design low-performance applications even on a high-performance blockchain system. Applications that initially perform well may fail under pressure. It is better to plan around contention in your application design than to fix issues later. The marginal cost of including extra business logic within a blockchain transaction is often small.
Contention is expected in distributed systems. The aim is to reduce it to acceptable levels and handle it gracefully, not to eliminate it at all costs. If contention only occurs rarely, it may be cheaper for both performance and complexity to simply let the occasional allocation fail and retry, rather than implement an advanced technique to avoid it.
As an added benefit to reducing contention issues, carefully bundling or batching strategic business logic can improve performance by yielding business transaction throughput that far exceeds the blockchain transaction throughput.
## Contention in Daml
Daml uses an unspent transaction output (UTXO) ledger model. UTXO models enable higher performance by supporting parallel transactions. This means that you can send new transactions while other transactions are still processing. The downside is that contention can occur if a second transaction arrives while a conflicting earlier transaction is still pending.
Daml guarantees that there can only be one consuming choice exercised per contract. If you try to commit two transactions that would consume the same contract, you have write-write contention.
Contention can also result from incomplete or stale knowledge. For example, a contract may have been archived, but a client hasn’t yet been notified due to latencies or a privacy model might prevent the client from ever knowing. If you try to commit two transactions on the same contract where one transaction reads and the other one consumes an input, you run the risk of a read-write contention.
A contract is considered pending when you do not know if the output has been consumed. It is best to assume that your transactions will go through and to treat pending ones as probably consumed. You must also assume that acting on a pending contract will fail.
You need to wait while the sequencer is processing a transaction in order to confirm that an input was consumed from a consuming input request. If you do not get confirmation back from the first transaction before submitting a second transaction on the same contract, the sequence is not guaranteed. The only way to avoid this conflict is to control the sequence of those two transactions.
Ledger state is read in the following places within the Daml Execution Model :
1. A client submits a command based on the client’s latest view of the state of the shared ledger. The command might include references to `ContractIds` that the client believes are active.
2. During interpretation, ledger state is used to look up active contracts.
3. During validation, ledger state is again used to look up contracts and to validate the transaction by reinterpreting it.
Contention can occur both between #1 and #2 and between #2 and #3:
* The client is constructing the command in #1 based on contracts it believes to be active. But by the time the participant performs interpretation in #2, it has processed the commit of another transaction that consumed those contracts. The participant node rejects the command due to contention.
* The participant successfully constructs a transaction in #2 based on contracts it believes to be active. But by the time validation happens in #3, another transaction that consumes the same contracts has already been sequenced. The validating participants reject the command due to contention.
The complete and relevant ledger state at the time of the transaction is known only after sequencing, which happens between #2 and #3. That ledger state takes precedence to ensure double spend protection.
Contention slows performance significantly. While you cannot avoid contention completely, you can design logic to minimize it. The same considerations apply to any UTXO ledger.
# Reduce Contention
Contention is natural and expected when programming within a distributed system like Daml in which every action is asynchronous. It is important to understand the different causes of contention, be able to diagnose the root cause if errors of this type occur, and be able to avoid contention by designing contracts appropriately.
You can use different techniques to manage contention and to improve performance by increasing throughput and decreasing latency. These techniques include the following:
* Add retry logic.
* Run transactions that have causality in series.
* Bundle or batch business logic to increase business transaction throughput.
* Maximize parallelism with techniques such as sharding, while ensuring no contention between shards.
* Split contracts across natural lines to reduce single, high-contention contracts.
* Avoid write-write and write-read contention on contracts. This type of contention occurs when one requester submits a transaction with a consuming exercise on a contract while another requester submits another exercise or a fetch on the same contract. This type of contention cannot be eliminated entirely, since there will always be some latency between a client submitting a command to a participant and other clients learning of the committed transaction. Here are a few scenarios and specific measures you can take to reduce this type of collision:
* Shard data. Imagine you want to store a user directory on the ledger. At the core, this is of type `[(Text, Party)]`, where `Text` is a display name and `Party` is the associated Party. If you store this entire list on a single contract, any two users wanting to update their display name at the same time will cause a collision. If you instead keep each `(Text, Party)` on a separate contract, these write operations become independent from each other.
A helpful analogy when structuring your data is to envision that a template defines a table, where a contract is a row in that table. Keeping large pieces of data on a contract is like storing big blobs in a database row. If these blobs can change through different actions, you have write conflicts.
* Use non-consuming choices, where possible, as they do not collide. Non-consuming choices can be used to model events that have occurred, so instead of creating a short-lived contract to hold some data that needs to be referenced, that data could be recorded as a ledger event using a non-consuming choice.
* Avoid workflows that encourage multiple parties to simultaneously exercise a consuming choice on the same contract. For example, imagine an auction contract containing a field `highestBid : (Party, Decimal)`. If Alice tries to bid \$100 at the same time that Bob tries to bid \$90, it does not matter that Alice’s bid is higher. The sequencer rejects the second because it has a write collision with the first transaction. It is better to record the bids in separate Bid contracts, which can be updated independently. Think about how you would structure this data in a relational database to avoid data loss due to race conditions.
* Think carefully about storing `ContractIds`. Imagine that you create a sharded user directory according to the first bullet in this list. Each user has a `User` contract that stores their display name and party. Now assume that you write a chat application, where each `Message` contract refers to the sender by `ContractId` User.
If a user changes the display name, that reference goes stale. You either have to modify all messages that the user ever sent, or you cannot use the sender contract in Daml.
Contract keys can be used to make this link inside Daml. If the only place you need to link `Party` to `User` is in the user interface, it might be best to not store contract references in Daml at all.
# Example Application with Techniques for Reducing Contention
The example application below illustrates the relationship between blockchain and business application performance, as well as the impact of design choices. Trading, settlement, and related systems are core use cases of blockchain technology, so this example demonstrates different ways of designing such a system within a UTXO ledger model and how the design choices affect application performance.
## The Example Minimal Settlement System
This section defines the requirements that the example application should fulfill, as well as how to measure its performance and where contention might occur. Assume that there are initial processes already in place to issue assets to parties. All of the concrete numbers in the example are realistic order-of-magnitude figures that are for illustrative purposes only.
### Basic functional requirements for the example application
A trading system is a system that allows parties to swap assets. In this example, the parties are Alice and Bob, and the assets are shares and dollars. The basic settlement workflow could be:
1. **Proposal**: Alice offers Bob to swap one share for \$1.
2. **Acceptance**: Bob agrees to the swap.
3. **Settlement**: The swap is settled atomically, meaning that at the same time Alice transfers \$1 to Bob, Bob transfers one share to Alice.
### Practical and security requirements for the example application
The following list adds some practical matters to complete the rough functional requirements of an example minimal trading system.
* Parties can hold *asset positions* of different asset types which they control.
* An asset position consists of the type, owner, and quantity of the asset.
* An asset type is usually the combination of an on-ledger issuer and a symbol (such as currency, CUSIP, or ISIN).
* Parties can transfer an asset position (or part of a position) to another party.
* Parties can agree on a settlement consisting of a swap of one position for another.
* Settlement happens atomically.
* There are no double spends.
* It is possible to constrain the *total asset position* of an owner to be non-negative. In other words, it is possible to ensure that settlements are funded. The total asset position is the sum of the quantities of all assets of a given type by that owner.
### Performance measurement in the example application
Performance in the example can be measured by latency and throughput; specifically, settlement latency and settlement throughput. Another important factor in measuring performance is the ledger transaction latency.
* **Settlement latency**: the time it takes from one party wanting to settle (just before the proposal step) to the time that party receives final confirmation that the settlement was committed (after the settlement step). For this example, assume that the best possible path occurs and that parties take zero time to make decisions.
* **Settlement throughput**: the maximum number of settlements per second that the system as a whole can process over a long period.
* **Transaction latency**: the time it takes from when a client application submits a command or transaction to the ledger to the time it receives the commit confirmation. The length of time depends on the command. A transaction settling a batch of 100 settlements will take longer than a transaction settling a single swap. For this example, assume that transaction latency has a simple formula of a fixed cost `fixed_tx` and a variable processing cost of `var_tx` times the number of settlements, as shown here:
`transaction latency = fixed_tx + (var_tx * #settlements)`
* Note that the example application does not assign any latency cost to settlement proposals and acceptances.
* For the example application, assume that:
* `fixed_tx = 250ms`
* `var_tx = 10ms`
To set a baseline performance measure for the example application, consider the simplest possible settlement workflow, consisting of one proposal transaction plus one settlement transaction done back-to-back. The following formula approximates the settlement latency of the simple workflow:
> `(2 * fixed_tx) + var_tx`
`=` `(2 * 250ms) + 10ms`
`=` `510ms`
To find out how many settlements per second are possible if you perform them in series, throughput evaluates to the following formula (there are 1,000ms in one second):
> `1000ms / (fixed_tx + var_tx) settlements per second`
`=` `1000ms / (250ms + 10ms)`
`=` `1000 / 260`
`=` `3.85 or ≈ 4 settlements per second`
These calculations set the optimal baselines for a high performance system.
The next goal is to increase throughput without dramatically increasing latency. Assume that the underlying DLT has limits on total throughput and on transaction size. Use a simple cost model in a unit called `dlt_min_tx` representing the minimum throughput unit in the DLT system. An empty transaction has a fixed cost `dlt_fixed_tx` which is:
> `dlt_fixed_tx = 1 dlt_min_tx`
Assume that the ratio of the marginal throughput cost of a settlement to the throughput cost of a transaction is roughly the same as the ratio of marginal latency to transaction latency (shown previously). A marginal settlement throughput cost `dlt_var_tx` can then be determined by this calculation:
> `dlt_var_tx = ratio * dlt_fixed_tx`
`=` `dlt_var_tx = (var_tx / fixed_tx) * dlt_fixed_tx`
`=` `dlt_var_tx = 10sm/250ms * dlt_fixed_tx`
`=` `dlt_var_tx = 0.04 * dlt_fixed_tx`
and, since from previously
> `dlt_fixed_tx = 1 dlt_min_tx`
then
> `dlt_var_tx = 0.04 * dlt_min_tx`
Even with good parallelism, ledgers have limitations. The limitations might involve CPUs, databases, or networks. Calculate and design for whatever ceiling you hit first. Specifically, there is a maximum throughput `max_throughput` (measured in `dlt_min_tx/second`) and a maximum transaction size `max_transaction` (measured in `dlt_min_tx`). For this example, assume that `max_throughput` is limited by being CPU-bound. Assume that there are 10 CPUs available and that an empty transaction takes 10ms of CPU time. For each second:
> `max_throughput = 10 * each CPU’s capacity`
Each `dlt_min_tx` takes 10ms and there are 1,000 ms in a second. The capacity for each CPU is then 100 `dlt_min_tx` per second. The throughput calculation becomes:
> `max_throughput = 10 * 100 dlt_min_tx/second`
`=` `max_throughput = 1,000 dlt_min_tx/second`
Similarly, `max_transaction` could be limited by message size limit. For this example, assume that the message size limit is 3 MB and that an empty transaction `dlt_min_tx` is 1 MB. So
> `max_transaction = 3 * dlt_min_tx`
One of the three transactions needs to hold an approval with no settlements. That leaves the equivalent of `(2 * dlt_min_tx)` available to hold many settlements in the biggest possible transaction. Using the ratio described earlier, each marginal settlement `dlt_var_tx` takes `0.04 * dlt_min_tx`. So the maximum number of settlements per second is:
> `(2 * dlt_min_tx)/(0.04 * dlt_min_tx)`
`=` `50 settlements/second`
Using the same assumptions, if you process settlements in parallel rather than in series (with only one settlement per transaction), latency stays constant while settlement throughput increases. Earlier, it was noted that a simple workflow can be `(2 * fixed_tx) + var_tx`. In the DLT system, the simple workflow calculation is:
> `(2 * dlt_min_tx) + dlt_var_tx`
`=` `(2 * dlt_min_tx) + (0.04 * dlt_min_tx)`
`=` `2.04 * dlt_min_tx`
It was assumed earlier that max\_throughput is `1,000 dlt_min_tx/second`. So the maximum number of settlements per second possible through parallel processing alone in the example DLT system is:
> `1,000/2.04 settlements per second`
`=` `490.196 or ~490 settlements per second`
These calculations provide a baseline when comparing various techniques that can improve performance. The techniques are described in the following sections.
## Prepare Transactions for Contention-Free Parallelism
This section examines which aspects of UTXO ledger models can be processed in parallel to improve performance. In UTXO ledger models, the state of the system consists of a set of immutable contracts, sometimes also called UTXOs.
Only two things can happen to a contract: it is created and later it is consumed (or spent). Each transaction is a set of input contracts and a set of output contracts, which may overlap. The transaction creates any output contracts that are not also consumed in the same transaction. It also consumes any input contracts, unless they are defined as non-consumed in the smart contract logic.
Other than smart contract logic, the execution model is the same for all UTXO ledger systems:
1. **Interpretation**: the submitting party precalculates the transaction, which consists of input and output contracts.
2. **Submission**: the submitting party submits the transaction to the network.
3. **Sequencing**: the consensus algorithm for the network assigns the transaction a place in the total order of all transactions.
4. **Validation**: the transaction is validated and considered valid if none of the inputs were already spent by a previous transaction.
5. **Commitment**: the transaction is committed.
6. **Response**: the submitting party receives a response that the transaction was committed.
The only step in this process which has a sequential component is sequencing. All other stages of transaction processing are parallelizable, which makes UTXO a good model for high-performance systems. However, the submitting party has a challenge. The interpretation step relies on knowing possible input contracts, which are by definition unspent outputs from a previous transaction. Those outputs only become known in the response step, after a minimum delay of `fixed_tx`.
For example, if a party has a single \$1,000 contract and wants to perform 1,000 settlements of \$1 each, sequencing in parallel for all 1,000 settlements leads to 1,000 transactions, each trying to consume the same contract. Only one succeeds, and all the others fail due to contention. The system could retry the remaining 999 settlements, then the remaining 998, and so on, but this does not lead to a performant system. On the other hand, using the example latency of 260ms per settlement, processing these in series would take 260s or four minutes 20s, instead of the theoretical optimum of one second given by `max_throughput`. The trading party needs a better strategy. Assume that:
> `max_transaction > dlt_fixed_tx + 1,000 * dlt_var_tx = 41 dlt_min_tx`
The trading party could perform all 1,000 settlements in a single transaction that takes:
> `fixed_tx + 1,000 * var_tx = 10.25s`
If the latency limit is too small or this latency is unacceptable, the trading party could perform three steps to split \$1,000 into:
* 10 \* \$100
* 100 \* \$10
* 1,000 \* \$1
and perform the 1,000 settlements in parallel. Latency would then be theoretically around:
> `3 * fixed_tx + (fixed_tx + var_tx) = 1.01s`
However, since the actual settlement starts after 750 ms, and the `max_throughput` is `1,000 dlt_min_tx/s`, it would actually be:
> `0.75s + (1,000 * (dlt_fixed_tx + dlt_var_tx)) / 1,000 dlt_min_tx/s = 1.79s`
These strategies apply to one particular situation with a very static starting state. In a real-world high performance system, your strategy needs to perform with these assumptions:
* There are constant incoming settlement requests, which you have limited ability to predict. Treat this as an infinite stream of random settlements from some distribution and maximize settlement throughput with reasonable latency.
* Not all settlements are successful, due to withdrawals, rejections, and business errors.
To compare between different techniques, assume that the settlement workflow consists of the steps previously illustrated with Alice and Bob:
1. **Proposal**: proposal of the settlement
2. **Acceptance**: acceptance of the settlement
3. **Settlement**: actual settlement
These steps are usually split across two transactions by bundling the acceptance and settlement steps into one transaction. Assume that the first two steps, proposal and acceptance, are contention-free and that all contention is on settlement in the last step. Note that the cost model allocates the entire latency and throughput costs `var_tx` and `dlt_var_tx` to the settlement, so rather than discussing performant trading systems, the concern is for performant settlement systems. The following sections describe some strategies for trading under these assumptions and their tradeoffs.
## Non-UTXO Alternative Ledger Models
As an alternative to a UTXO ledger model, you could use a replicated state machine ledger model, where the calculation of the transaction only happens after the sequencing.
The steps would be:
1. **Submission**: the submitting party submits a command to the network.
2. **Sequencing**: the consensus algorithm of the network assigns the command a place in the total order of all commands.
3. **Validation**: the command is evaluated to a transaction and then validated.
4. **Response**: the submitting party receives a response about the effect of the command.
**Pros**
This technique has a major advantage for the submitting party: no contention. The party pipes the stream of incoming transactions into a stream of commands to the ledger, and the ledger takes care of the rest.
**Cons**
The disadvantage of this approach is that the submitting party cannot predict the effect of the command. This makes systems vulnerable to attacks such as frontrunning and reordering.
In addition, the validation step is difficult to optimize. Command evaluation may still depend on the effects of previous commands, so it is usually done in a single-threaded manner. Transaction evaluation is at least as expensive as transaction validation. Simplifying and assuming that `var_tx` is mostly due to evaluation and validation cost, a single-threaded system would be limited to `1s / var_tx = 100` settlements per second. It could not be scaled further by adding more hardware.
## Simple Strategies for UTXO Ledger Models
To attain high throughput and scalability, UTXO is the best option for a ledger model. However, you need strategies to reduce contention so that you can parallelize settlement processing.
### Batch transactions sequentially
Since `(var_tx << fixed_tx)`, processing two settlements in one transaction is much cheaper than processing them in two transactions. One strategy is to batch transactions and submit one batch at a time in series.
**Pros**
This technique completely removes contention, just as the replicated state machine model does. It is not susceptible to reordering or frontrunning attacks.
**Cons**
As in the replicated state machine technique, each batch is run in a single-threaded manner. However, on top of the evaluation time, there is transaction latency. Assuming a batch size of `N < max_settlements`, the latency is:
> `fixed_tx + N * var_tx`
and transaction throughput is:
> `N / (fixed_tx + N * var_tx)`
As `N` goes up, this tends toward `1 / var_tx = 100`, which is the same as the throughput of replicated state machine ledgers.
In addition, there is the `max_settlements` ceiling. Assuming `max_settlements = 50`, you are limited to a throughput of `50 / 0.75 = 67` settlement transactions per second, with a latency of 750ms. Assuming that the proposal and acceptance steps add another transaction before settlement, the settlement throughput is 67 settlements per second, with a settlement latency of one second. This is better than the original four settlements per second, but far from the 490 settlements per second that is achievable with full parallelism.
Additionally, the success or failure of a whole batch of transactions is tied together. If one transaction fails in any way, all will fail, and the error handling is complex. This can be somewhat mitigated by using features such as Daml exception handling, but contention errors cannot be handled. As long as there is more than one party acting on the system and contention is possible between parties (which is usually the case), batches may fail. The larger the batch is, the more likely it is to fail, and the more costly the failure is.
### Use sequential processing or batching per asset type and owner
In this technique, assume that all contention is within the asset allocation steps. Imagine that there is a single contract on the ledger that takes care of all bookkeeping, as shown in this Daml code snippet:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template AllAssets
with
-- A map from owner and type to quantity
holdings : Map Party (Map AssetType Decimal)
where
signatory (keys holdings)
```
This is a typical pattern in replicated state machine ledgers, where contention does not matter. On a UTXO ledger, however, this pattern means that any two operations on assets experience contention. With this representation of assets, you cannot do better than sequential batching. There are many additional issues with this approach, including privacy and contract size.
Since you typically only need to touch one owner’s asset of one type at a time and constraints such as non-negativity are also at that level, assets are usually represented by asset positions in UTXO ledgers, as shown in this Daml code snippet:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template
with
assetType : AssetType
owner : Party
quantity : Decimal
where
signatory assetType.issuer, owner
```
An asset position is a contract containing a triple (owner, asset type, and quantity). The total asset position of an asset type for an owner is the sum of the quantities for all asset positions with that owner and asset type. If the settlement transaction touches two total asset positions for the buy-side and two total asset positions for the sell-side, batching by asset type and owner does not help much.
Imagine that Alice wants to settle USD for EUR with Bob, Bob wants to settle EUR for GBP with Carol, and Carol wants to settle GBP for USD with Alice. The three settlement transactions all experience contention, so you cannot do better than sequential batching.
However, if you could ensure that each transaction only touches one total asset position, you could then apply sequential processing or batching per total asset position. This is always possible to do by decomposing the settlement step into the following:
1. **Buy-side allocation**: the buy-side splits out an asset position from their total asset position and allocates it to the settlement.
2. **Sell-side allocation**: the sell-side splits out an asset position from their total asset position and allocates it to the settlement.
3. **Settlement**: the asset positions change ownership.
4. **Buy-side merge**: the buy-side merges their new position back into the total asset position.
5. **Sell-side merge**: the sell-side merges their new position back into the total asset position.
This does not need to result in five transactions.
* Buy-side allocation is usually done as part of a settlement proposal.
* Sell-side allocation is typically handled as part of the settlement.
* Buy-side merge and sell-side merge technically do not need any action. By definition of total asset positions, merging is an optional step. It is easy to keep things organized without extra transactions. Every time a total asset position is touched as part of buy-side allocation or sell-side allocation above, you merge all positions into a single one. As long as there is a similar amount of inbound and outbound traffic on the total asset position, the number of individual positions stays low.
**Pros**
Assuming that a settlement is considered complete after the settlement step and that you bundle the allocation steps above into the proposal and settlement steps, the system performance will stay at the optimum settlement latency of 510ms.
Also, if there are enough open settlements on distinct total asset positions, the total throughput may reach up to the optimal 490 settlements per second.
With batch sizes of `N=50` for both proposals and settlements and sufficient total asset positions with open settlements, the maximum theoretical settlement throughput is:
`50 stls * 1,000 dlt_min_tx/s / (2 * dlt_fixed_tx + 50 * dlt_var_tx) = 12,500 stls/s`
**Cons**
Without batching, you are limited to the original four outgoing settlements per second, per total asset position. If there are high-traffic assets, such as the USD position of a central counterparty, this can bottleneck the system as a whole.
Using higher batch sizes, you have the same tradeoffs as for sequential batching, except that it is at a total asset position level rather than a global level. Latency also scales exactly as it does for sequential batching.
Using a batch size of 50, you would get settlement latencies of around 1.5s and a maximum throughput per total asset position of 67 settlements per second, per total asset position.
Another disadvantage is that allocating the buy-side asset in a transaction before the settlement means that asset positions can be locked up for short periods.
Additionally, if the settlement fails, the already allocated asset needs to be merged back into the total asset position.
## Shard Asset Positions for UTXO Ledger Models
In systems where peak loads on a single total asset position is in the tens or hundreds of settlements per second, more sophisticated strategies are needed. The total asset positions in question cannot be made up of a single asset position. They need to be sharded.
### Shard total asset positions without global constraints
Consider a total asset position that represents a bookkeeping position without any on-ledger constraints. For example, the trading system may deal with fiat settlement off-ledger, and you simply want to record a balance, whether it is positive or negative. In this situation, you can easily get rid of contention altogether by assigning all allocations an arbitrary amount. To allocate \$1 to a settlement, write two new asset positions of \$1 and -\$1 to the ledger, then use the \$1 to allocate. The total asset position is unchanged.
**Pros**
This approach removes all contention on a total asset position.
Trading between two such total asset positions without global constraints can run at the theoretically optimal latency and throughput. Combining this with batching of batch size 50, it is possible to achieve settlements per second up to the same 12,500 settlements per second per total asset position that are possible globally.
**Cons**
Besides the inability to enforce any global constraints on the total asset position, this creates many new contracts. At 500 settlements per second, two allocations per settlement, and two new assets per allocation, that results in 2,000 new asset positions per second, which adds up quickly.
This effect has to be mitigated by a netting automation that nets them up into a single position once a period (for example, every time it sees >= 100 asset positions for a total position). This automation does not contend with the trading, but it adds up to 20 large transactions per second to the system and slightly reduces total throughput.
### Shard total asset positions with global constraints
As an example of a global constraint, assume that the total asset position has to stay positive. This is usually done by ensuring that each individual asset position is positive. If that is the case, the strategy is to define a sharding scheme where the total position is decomposed into `N` smaller shard positions and then run sequential processing or batching per shard position.
Each asset position has to be clearly assignable to a single shard position so that there is no contention between shards. The partitioning of the total asset position does not have to be done on-ledger. If the automation for all shards can communicate off-ledger, it is possible to run a sharding strategy where you simply set the total number of desired asset positions.
For example, assume that there should be 100 asset positions for a total asset position with some minimal value.
* The automation keeps track of a synchronized pending set of asset positions, which marks asset positions that are in use.
* Every time the automation triggers (which may happen concurrently), it looks at how many asset positions there are relative to the desired 100 and how much quantity is needed to allocate the open settlements.
* It then selects an appropriate set of non-pending asset positions so that it can allocate the open settlements and return new asset positions to move the total number closer to 100.
* Before sending the transaction, it adds those positions to the pending set to make sure that another thread does not also use them.
Alternatively, if you have a sufficiently large total position compared to settlement values, you can pick the 99th percentile `p_99` of settlement values and maintain `N-1` positions of value between `p_99` and `2 * p_99` and one of the (still large) remainder. 99% of transactions will be processed in the `N-1` shard positions, and the remaining 1% will be processed against the remaining pool. Whenever a shard moves out of the desired range, it is balanced against the pool.
**Pros**
Assuming that there is always enough liquidity in the total asset position, the performance can be the same as without global constraints: up to 12,500 settlements per second on a single total asset position.
**Cons**
If settlement values are large compared to total asset holdings, this technique helps little. In an extreme case, if every settlement needs more than 50% of the total holding, it does not perform any better than the sequential processing or batching per asset type and owner technique.
In realistic scenarios where settlement values are distributed on a broad range relative to total asset position and those relativities change as holdings go up and down, developing strategies that perform optimally is complex. There are competing priorities that need to be balanced carefully:
* Keeping the total number of asset positions limited so that the number of active contracts does not impact system performance.
* Having sufficient large asset positions so that frequent small settlements can be processed in parallel.
* Having a mechanism that ensures large settlements, possibly requiring as much as 100% of the available total asset position, are not blocked.
## Application development performance practice
*Material relocated from Module 7 — Performance Best Practices.*
Performance optimization for Canton applications means understanding the boundary between on-ledger and off-ledger operations, and making deliberate choices about what crosses that boundary. Ledger operations carry synchronization overhead that database operations don't, so the goal is to minimize unnecessary ledger interaction while keeping business-critical state on-ledger.
## On-Ledger vs. Off-Ledger
Canton Network application design should account for performance characteristics, carefully considering on-ledger versus off-ledger implementation. When deciding whether information should be stored on-ledger:
* **State** — Essential facts in the workflow. When these facts require agreement across organizations, store them on-ledger.
* **Data** — Non-essential facts. For efficiency, use off-ledger technologies for sharing non-critical data.
* **Workflows** — Only workflows that cross organizational boundaries should be on-ledger.
A practical example: in a licensing application, the license itself (who holds it, what it covers, whether it's active) belongs on-ledger because both parties need to agree on its status. The license's detailed description text, marketing materials, or usage analytics belong off-ledger in a conventional database.
## Contract Design for Performance
### Avoid contract fan-out
When a single transaction creates or archives many contracts, it requires all stakeholders of all those contracts to participate in the transaction. Design your templates to avoid creating large numbers of contracts in a single transaction.
Instead of creating one contract per item in a batch:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Avoid: creates N contracts in one transaction
choice ProcessBatch : [ContractId Item]
controller processor
do mapA (\item -> create Item with ...) items
```
Consider batching into a single contract with a list field, or processing items in separate transactions:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Better: one contract representing the batch
choice ProcessBatch : ContractId BatchResult
controller processor
do create BatchResult with items = processedItems
```
### Minimize stakeholders per transaction
Each additional signatory or observer on a contract adds a party that must participate in the transaction protocol. Keep the stakeholder set as small as the business logic requires.
### Use contract keys judiciously
Contract keys enable efficient lookups but introduce contention. If multiple transactions try to create or exercise contracts with the same key concurrently, they will conflict. Design keys that partition naturally by party or business entity to reduce contention.
## PQS Query Optimization
PQS stores contract data in JSONB columns in PostgreSQL. For read-heavy workloads, proper indexing is critical.
### Creating indexes
Use the PQS helper function to create JSONB indexes:
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
call create_index_for_contract('token_wallet_holder_idx',
'register:DA.Register:Token',
'(payload->''wallet''->>''holder'')',
'hash');
```
This index allows efficient lookups on the wallet holder field without scanning the entire JSONB payload.
### PostgreSQL tuning
The default PostgreSQL Docker image ships with minimal settings. For any realistic workload, tune the following:
* `shared_buffers` — Set to 25% of available RAM
* `effective_cache_size` — Set to 75% of available RAM
* `max_connections` — Match to your expected connection pool size
* `autovacuum` settings — Tune for your write patterns
Resource requirements depend on your workload. Consult the [PQS operations guide](/sdks-tools/development-tools/pqs) for sizing guidance specific to your query patterns and contract volumes.
### Query patterns
* Use `explain analyze` to verify that queries use your indexes
* Prefer filtering in SQL over fetching all contracts and filtering in application code
* For time-series queries, use the event tables rather than the active contract set
* Consider materialized views for frequently-run aggregation queries
## Parallel Command Submission
The Ledger API supports parallel command submissions. Commands that affect different contracts can execute concurrently on the synchronizer. Structure your backend to take advantage of this:
* Use async/concurrent command submission from your backend
* Assign unique command IDs to enable deduplication
* Group commands by the contracts they affect to avoid contention
* Monitor command completion times to detect bottleneck patterns
## Traffic Management
On DevNet, TestNet, and MainNet, every ledger transaction consumes traffic credits purchased with Canton Coin (CC). Optimize your traffic costs by:
* Reducing the number and size of on-ledger transactions
* Using batch operations where possible
* Configuring auto-top-up to avoid running out of traffic during peak usage
* Monitoring traffic consumption patterns to forecast costs
# Privacy Model for App Developers
Source: https://docs.canton.network/appdev/deep-dives/privacy-model
How Canton's sub-transaction privacy works in practice, including transaction views, stakeholder consensus, disclosure patterns, and auditability
Canton's defining architectural feature is sub-transaction privacy: the ability to execute a single atomic transaction where each party sees only the parts relevant to them. This page explains how it works and how to use privacy-aware patterns in your applications.
## Transaction Tree Decomposition
When you submit a transaction that involves multiple parties, Canton does not send the full transaction to every validator. Instead, the transaction tree is decomposed into **views**, one per stakeholder group.
Each view contains only the actions (creates, exercises, fetches) where that stakeholder group has a stake. The view is encrypted so that only the validators hosting those stakeholders can decrypt it.
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
subgraph TX[Transaction Tree]
ROOT[Root Action:
Alice exercises Transfer]
ROOT --> C1[Create: Bob receives asset]
ROOT --> C2[Create: Charlie receives fee]
end
TX --> D[Decompose]
D --> VA["Alice's view
Root action + both creates"]
D --> VB["Bob's view
Asset create only"]
D --> VC["Charlie's view
Fee create only"]
```
Bob does not learn about Charlie's fee, and Charlie does not learn about Bob's asset. Alice, as the submitter and signatory, sees the full scope of her action and its consequences.
## Stakeholder Consensus and Privacy
Validators confirm only the views they can decrypt. A validator hosting Alice confirms Alice's view; a validator hosting Bob confirms Bob's. The synchronizer collects these confirmations and determines the transaction outcome.
What each role sees:
* **Validators** see only the views for parties they host. A validator hosting no parties involved in a transaction receives nothing about it.
* **The synchronizer** sees encrypted blobs, routing metadata (which validators to deliver to), and confirmation results. It cannot read transaction content, contract data, party relationships within views, or amounts and terms.
* **Non-stakeholders** receive no information about the transaction payload or metadata. They do not learn that the transaction happened.
## Privacy Guarantees
Canton provides these guarantees at the protocol level:
* Transaction content is visible only to stakeholders (signatories, observers, controllers for the relevant actions)
* Each validator stores data only for its hosted parties -- there is no global state replication
* The synchronizer cannot read any transaction payload
* Parties not entitled to a view learn nothing about it, including the identities of the parties involved
Your validator is the one entity that sees all your party's data. Choose your validator with the same care you would choose a custodian or cloud provider.
## Privacy Patterns
### Bilateral Agreements
When a contract has two signatories and no observers, only those two parties see it. No third party -- not even another party on the same validator -- has visibility.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template NDA
with
partyA : Party
partyB : Party
terms : Text
where
signatory partyA, partyB
-- No observers: maximum privacy
```
Use this pattern for confidential two-party contracts where no third-party visibility is needed.
### Observer Disclosure
You can grant read access to additional parties by declaring them as observers. Observers see the contract but cannot exercise choices on it (unless separately declared as controllers).
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template RegulatedLoan
with
lender : Party
borrower : Party
regulator : Party
principal : Decimal
where
signatory lender, borrower
observer regulator
```
The regulator sees the loan and its lifecycle events but cannot modify it. Use this pattern when compliance or audit requirements demand third-party visibility.
### Divulgence
When a contract is fetched or exercised within another party's transaction, that party automatically gains visibility of the fetched contract. This is called divulgence.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice SettleTrade : ()
controller buyer
do
-- Fetching sellerAsset here divulges it to the buyer
asset <- fetch sellerAssetId
archive sellerAssetId
create Asset with owner = buyer, issuer = asset.issuer, value = asset.value
```
Divulgence happens automatically as a consequence of the transaction structure. Be deliberate about which contracts you fetch inside transactions, because every fetch can widen visibility.
### Explicit Contract Disclosure
Canton also supports explicit disclosure, where a party shares a contract reference with another party outside of the normal stakeholder model. The receiving party can then use that contract in their own transactions. Explicit disclosure gives you fine-grained control over information sharing without modifying the contract's observer list.
## Privacy and Auditability
Privacy does not mean you lose the ability to audit. Several mechanisms support auditability while preserving privacy boundaries.
**PQS (Participant Query Store)** maintains a PostgreSQL database synchronized with your validator's ledger state. You can run SQL queries against your own transaction history, contract state, and archived contracts. PQS sees only what your party sees -- it respects the same privacy boundaries.
**Scan** is a public service for the Global Synchronizer that exposes network-level data: round information, CC minting, and aggregate statistics. Scan does not expose private contract data.
**Auditor-as-observer** is a Daml pattern where you add an auditor party as an observer on contracts that require audit trails. The auditor sees the contract and its lifecycle but cannot act on it.
The key principle: each party can audit everything they are entitled to see, and nothing more.
## Next Steps
* [Privacy Model Explained](/overview/learn/privacy-model) -- Technical overview of sub-transaction privacy at the protocol level
* [Privacy Differences from Ethereum](/appdev/modules/m2-privacy-differences) -- How Canton's model compares to public blockchains
* [Design Patterns](/appdev/modules/m3-design-patterns) -- Patterns for authorization and multi-party workflows
# Smart Contract Upgrade (SCU)
Source: https://docs.canton.network/appdev/deep-dives/smart-contract-upgrade
Develop, deploy, and validate smart contract upgrades across Daml package versions.
# Smart Contract Upgrade
## Overview
### What is Smart Contract Upgrade (SCU)?
Smart Contract Upgrade (SCU) allows Daml models (packages in DAR files) to be updated on Canton transparently, provided some guidelines in making the changes are followed. For example, you can fix an application bug by uploading the DAR of the fixed package.
Smart Contract Upgrade (SCU) is a feature for Daml packages which enable authors to publish new versions of their templates while maintaining compatibility with prior versions, without any downtime for package users and existing contracts.
This feature is well-suited for developing and rolling out incremental template changes. There are guidelines to ensure upgrade compatibility between DAR files. The compatibility is checked at compile time, DAR upload time, and runtime. This is to ensure data backwards upgrade compatibility and forwards compatibility (subject to the guidelines being followed) so that DARs can be safely upgraded to new versions. It also prevents unexpected data loss at clients if a runtime downgrade occurs (e.g., a client is using template version 1.0.0 while the participant node has the newer version 1.1.0).
A general guideline for SCU is that additive application changes are allowed but items cannot be removed. A summary of the allowed changes in templates are:
* A template can add new `Optional` fields at the end of the list of fields;
* A `record` datatype can add new optional fields at the end of the list of fields, and a `variant` or `enum` datatype can add new constructors at the end;
* The `ensure` predicate can be changed and it is reevaluated at interpretation;
* A choice signature can be changed by adding `Optional` parameters at the end;
* The controller of a choice can be changed;
* The observer of a choice can be changed
* The body of a choice can be changed;
* A new choice can be added to a template;
* The implementation of an interface instance can be changed;
The following table summarizes the changes supported by SCU. Consult the sections below for additional information.
| **Daml Model Change** | **Scope** | **Covered by SCU?** | **Comments** |
| ----------------------------------------------------------------- | -------------------- | ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Rename of Module | Template Definition | No | |
| Rename of Template | Template Definition | No | |
| Change of Field Type | Template Definition | No | |
| Change of Field Ordering | Template Definition | No | |
| Change of Field Name | Template Definition | No | |
| Change of Signatories | Template Definition | No | A new template version needs to return the same signatory result for a persisted contract, otherwise the contracts cannot be used by an upgraded package (this fails at runtime).
One can change the function that computes the signatories (for example to take into account new fields) when creating contracts. |
| Change of | Template | No | The key's type cannot be changed, like adding a new field in the record it contains.
One can change the function that computes the key in the same way the function computing the signatories can be changed. A persisted contract can be used by an upgrade only if the result of the function does not change. |
| Change of Observers | Template Definition | No | Same as template signatories |
| Change of Key Maintainers | Template Definition | No | Same as template signatories |
| Change of Ensure Predicate | Template Definition | Yes | Note that the Ensure is recalculated just before usage so any changes need to ensure that a V1 contract is still valid as a V2 contract. |
| Addition of New Definition Fields, Enum, and Variant Constructors | Template Definition | Yes, but they they must be optional | |
| Deletion of Fields from Template | Template Definition | No | |
| Change of Choice Implementation | Template Definition | Yes | |
| Change in Choice Controller | Choice | Yes | |
| Addition of Choice Parameters | Choice | Yes, but they must be optional | |
| Deletion of Choice Parameters | Choice | No | |
| Change of Consuming Choice to Non-Consuming Choice, or Vice-versa | Choice | No | |
| Change of Choice Body | Choice | Yes | |
| Change of Choice Return Type | Choice | Yes, except for Tuple or scalar types | You can change a user data type (Record, Variant, Enum) contained in the return type. In this case you can add a field to the record or a new case to the Variant/Enum.
In particular: you cannot change a collection, Optional, List, or Map; you can change Tuple, (to a triple) if you define your own type. |
| Change of Interface Definition | Interface Definition | No | Interfaces are static and cannot be changed. Separate interface definition and keep the implementation in the template code. |
| Adding Interfaces to Templates | Template Definition | Yes | |
| Removing Interfaces from Templates | Template Definition | No | Though the implementation can be changed. |
| Change or Deletion of Exception | Exception Definition | No | Keep the exception definition separate. Exceptions cannot be upgraded. |
In this way, package authors can publish new package versions that improve contract functionality without requiring any error-prone migrations, without downtime, without requiring any additional network traffic, and without any extensive communication with downstream users.
With SCU, packages may now be referenced by either:
* the package ID
* the package name.
Package names are used to group versions of a package. Any two packages with the same package name must have distinct package versions.
**Note:**
* When defining a Daml package, the `name` field of the package's `daml.yaml` is now used to specify the SCU package name.
The Java and TypeScript codegen allow the use of package name and package ID (if needed).
PQS supports using package-name (specified in `daml.yaml`) instead of package-id. When specifying a template/interface/choice name, simply substitute any package-id with the package-name (eg. now `register:DA.Register:Token`) instead of the old `deadbeefpackageidhex:DA.Register:Token` format. This applies to template filters and SQL queries (eg. via the `active()` function). These functions will always return all versions of a given identifier. Qualified name can be:
* fully qualified, e.g. `::`
* partially qualified, e.g. `:`
Qualified names cannot be ambiguous.
The PQS Read API now returns the package-name, package-id, and package-version for each contract instance, making it easy for users to determine and inspect different versions over time. To reconstruct the old experience (should you need to) of querying one specific version, use a filter predicate in the SQL.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
SELECT *
FROM active('mypackage:My.App:MyTemplate')
WHERE package_id = 'deadbeefpackageidhex'
```
### Smart Contract Upgrade Basics
To upgrade a package the package author modifies their existing package to add new functionality, such as new fields and choices. When the new package is uploaded to a participant with the old version, the participant ensures that every modification to the model in the new version is a valid upgrade of the previous version.
To be able to automatically upgrade a contract or datatype, SCU restricts the kinds of changes that a new package version can introduce over its prior version.
For example, the simplest kind of data transformation that SCU supports is adding a field to a template.
Given the following first version of a template:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template IOU
with
issuer: Party
owner: Party
value: Int
where
signatory issuer
observer owner
```
You can add a new field for currency:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template IOU
with
issuer: Party
owner: Party
value: Int
-- New field:
currency: Optional String
where
signatory issuer
observer owner
```
With SCU, any new template fields must be optional - templates from the old version are automatically upgraded to new versions by setting the new field to `None`. This optional field requirement extends to all records in your package. Conversely, newer contracts with this field set to `None` can be automatically downgraded to previous versions of the template in workflows that have not yet been updated.
#### Automatic Data Upgrades and Downgrades
When extending data in a Daml model, SCU requires the old model to be representable in the new model. For extending a record, we can only add nullable (`Optional`) fields, so that old data can be represented by setting these fields to `None`. Similar constraints hold for Variants and Enums, which only allow adding constructors, with some other restrictions covered in Continuing to Write Your [Upgrades](). This approach is inspired by [Protobuf]() and Typescript's ability to ignore excess [fields]() via `as`.
Automatic data upgrades occur in the following places:
**Submissions to the Ledger API**
When you submit a command, and provide only a package-name instead of a package-id, Canton will automatically upgrade (or downgrade) the payloads you give to the most recent version of the package that is uploaded on the participant. It will also use the most recent implementation of any choices you exercise directly through the Ledger API, by automatically upgrading/downgrading the choice argument. Choice result upgrading/downgrading is handled by Consuming Clients, as discussed later in this section. This behavior can be influenced by the package preference.
**Updates in a choice body**
When Fetching a contract, the contract payload will be automatically upgraded/downgraded to match the version expected by the calling choice body, as compiled into the DAR. When Exercising a choice on a contract, the contract payload will be upgraded/downgraded to match the version of the choice expected by the calling choice body. This means that in a choice body, an exercised choice argument or return type is never upgraded/downgraded.
**Consuming Clients (such as Daml Script, TypeScript/Java codegen)**
When clients query the Ledger API for contracts, the returned event payload format matches the template originally used for generating the event (creating a contract/exercising a choice). It is the responsibility of these clients to upgrade/downgrade the payloads they receive to match what is expected downstream. The same applies to choice results. Daml Script, as well as TypeScript/Java codegen, does this for you to match the Ledger API response to the package versions they were run/built from.
### Upgrading Across the Stack
These are all the components that interact with SCU, and how you as a user should be aware that they interacts.
#### Canton
When considering the Canton ledger nodes, only the Canton participant node is aware of smart contract upgrading.
Below, we provide a brief overview of the interactions with the participant node that have been adapted for supporting the smart contract upgrading feature:
* DAR upload requests go through an additional validation stage to check the contained new packages for upgrade-compatibility with other packages previously uploaded on the participant.
* Ledger API command submissions can be automatically or explicitly up/downgraded if multiple packages exist for the same package-name.
* Ledger API streaming queries are adapted to support fetching events more generically, by package-name.
#### Code Generation
The Java and TypeScript CodeGen have been updated to perform upgrades on retrieved contracts, and now use package-names over package-ids for commands to the participant.
#### JSON Ledger API
To match the changes to the gRPC Ledger API, the JSON Ledger API similarly supports package-name queries and command submission.
#### PQS & Daml Shell
PQS only supports querying contracts via package-name, dropping support for direct package-id queries. See the PQS [section]() for more information and a work-around.
Daml Shell builds on top of PQS, so inherits this behavior.
#### Daml Script
Support for SCU is available in the opt-in LTS version of Daml Script.
This version acts as a drop-in replacement for the previous Daml Script, and enables support for upgrades on all queries and command submissions.
We also expose functions for more advanced interactions with upgrades, as well as to revert to the previous submission behavior.
#### Daml Compiler
The Daml compiler supports the `upgrades:` configuration field - every time `dpm build` is invoked, it validates the current package for upgrade compatibility against the package specified in the `upgrades:` field.
Validation emits custom error messages for common upgrading mistakes, and warns the package author when upgrading a package in a potentially unsafe way. Note however that this validation cannot be complete, as upgrade validity depends on a participant’s package store. The participant’s DAR upload checks have the final say on upgrade validity.
### Limitations
To allow SCU to minimize downtime, and multiple versions of a package to be active at once, we limit the types of transformations that can be performed on live data. Following are some data transformations that cannot be made using SCU upgrades:
* Renaming, removal, or rearrangement of fields in a template
* Conversion of records to variants and vice versa
* Moving templates/datatypes to other modules
* Upgrading interface and exception definitions
* Removing an interface instance from a template
These restrictions are required to give a simple model of runtime upgrades, avoiding ambiguity and non-obvious side effects. If you require any of these types of changes, you may need to consider a redeployment with downtime, either using the approach suggested in Rolling out backwards-incompatible changes.
In this version of SCU, the following functionality has not yet been implemented, but may be implemented in future releases.
* Daml Script does not support SCU or LF1.17, you must use Daml Script LTS.
There are further limitations with respect to managing the packages on a running ledger:
* Once a version of a package is uploaded to the ledger, it cannot be replaced or removed. If a package is uploaded by mistake it can only be overriden by uploading an even newer version.
* As a consequence of the above, if a record is extended by mistake with an optional field, that field must be part of all future versions.
* Package versions cannot be deleted, they can only be unvetted. As a good rule of thumb the highest version for each package name should be vetted.
* If, for whatever reason, a lower version of a package is vetted and preferred, that package version needs to be explicitly mentioned in the command submissions in the specific `package_id_selection_preference` field of the grpc `Commands` message.
* Participants that confirm and observe a certain transaction must have vetted the package version that is preferred by the submitting participant. If that is not the case, the transaction is rejected.
* The contract create events sent as part of the active contracts and transaction streams are always represented according to the template definition in the package version used at time of their creation.
## The Programming Model by Example
### Writing Your First Smart Contract Upgrade
#### Setup
We continue with the example introduced in Smart Contract Upgrade [Basics](). Begin by defining the first (old) version of our package:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> mkdir -p v1/my-pkg
> cd v1/my-pkg
> dpm init
> dpm version
SDK versions:
3.3.0 (project SDK version from daml.yaml)
```
Running `dpm version` should print a line showing that 3.3.0 or higher is the "project SDK version from daml.yaml".
Add `daml-script` to the list of dependencies in `v1/my-pkg/daml.yaml`, as well as `--target=2.1` to the `build-options`:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
name: my-pkg
...
dependencies:
- daml-prim
- daml-stdlib
- daml-script
build-options:
- --target=2.1
```
**Note:** Throughout this tutorial we ignore some best practices in favor of simplicity. In particular, we recommend a package undergoing SCU should contain a version identifier in its name as well, but we omit this - consult the section on package naming best practices to learn more. We also recommend that you do not depend on `daml-script` in any package that is uploaded to the ledger - more on this here.
Then create `v1/my-pkg/daml/Main.daml`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import Daml.Script
template IOU
with
issuer: Party
owner: Party
value: Int
where
signatory issuer
observer owner
key issuer : Party
maintainer key
```
Running dpm build should successfully produce a DAR in `v1/my-pkg/.daml/dist/my-pkg-1.0.0.dar`:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm build
Running single package build of my-pkg as no multi-package.yaml was found.
...
Compiling my-pkg to a DAR.
...
Created .daml/dist/my-pkg-1.0.0.dar
```
Now you can create the second (new) version of this package, which upgrades the first version. Navigate back to the root directory and copy the v1 package into a v2 directory.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd ../..
> cp -r v1 v2
> cd v2/my-pkg
```
Edit the `daml.yaml` to update the package version, and add the `upgrades:` field pointing to v1:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
version: 1.1.0
...
dependencies:
- daml-prim
- daml-stdlib
- daml-script
upgrades: ../../v1/my-pkg/.daml/dist/my-pkg-1.0.0.dar
build-options:
- --target=2.1
```
Any changes you make to v2 are now validated as correct upgrades over v1.
#### Adding a New Field
Begin by adding a new `currency` field to `v2/my-pkg/daml/Main.daml`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
template IOU
with
issuer: Party
owner: Party
value: Int
currency: Text -- New field
where
...
```
Run `dpm build`. An error is emitted:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm build
...
error type checking template Main.IOU :
The upgraded template IOU has added new fields, but those fields are not Optional.
ERROR: Creation of DAR file failed.
```
Any new fields added to a template must be optional - old contracts from the previous version are automatically upgraded by setting new fields to `None`.
Fix the `currency` field to be optional, and re-run `dpm build`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
currency: Optional Text -- New field
...
```
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm build
...
Created .daml/dist/my-pkg-1.0.0.dar
```
The build may produce warnings about expression changes - this is covered in the Continuing to Write Your [Upgrades]() section.
#### Seeing Upgraded Fields in Contracts
Using the Daml Sandbox, we can see our old contracts automatically upgrade.
Add a script to make and get IOUs to `v1/my-pkg/daml/Main.daml`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import Daml.Script
...
mkIOU : Script Party
mkIOU = do
alice <- allocateParty "alice"
alice `submit` createCmd (IOU alice alice 1)
pure alice
getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key
```
Open `v2/my-pkg/daml/Main.daml` and add scripts to make IOUs with and without a currency field, and a script to get any IOU:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import Daml.Script
...
mkIOU : Script Party
mkIOU = do
alice <- allocateParty "alice"
alice `submit` createCmd (IOU alice alice 1 (Some "USD"))
pure alice
mkIOUWithoutCurrency : Script Party
mkIOUWithoutCurrency = do
alice <- allocateParty "alice"
alice `submit` createCmd (IOU alice alice 1 None)
pure alice
getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key
```
Start a new terminal, run `dpm sandbox` to start a simple ledger in which to test upgrades.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm sandbox --json-api-port 7575
Starting Canton sandbox.
Listening at port 6865
Canton sandbox is ready.
```
Start another terminal, separately from the terminal in which the sandbox is running. From inside `v1/my-pkg`, upload and run the `mkIOU` script and place the resulting party for Alice into an output file `alice-v1`:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v1/my-pkg
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.0.0.dar \
--script-name Main:mkIOU \
--output-file alice-v1
...
```
From inside `v2/my-pkg`, upload and run the `getIOU` script, passing in the `alice-v1` file as the script’s input:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd ../../v2/my-pkg
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.1.0.dar \
--script-name Main:getIOU \
--output-file /dev/stdout \
--input-file ../../v1/my-pkg/alice-v1
...
{
"_1": "...",
"_2": {
"issuer": "party-...",
"owner": "party-...",
"value": 1,
"currency": null
}
}
...
```
The returned contract has a field `currency` which is set to `null`. When running the `getIOU` script from v1, this field does not appear.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd ../../v1/my-pkg
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.0.0.dar \
--script-name Main:getIOU \
--output-file /dev/stdout \
--input-file alice-v1
...
{
"_1": "...",
"_2": {
"issuer": "party-...",
"owner": "party-...",
"value": 1
}
}
...
```
#### Downgrading Contracts
New contracts cannot be downgraded if they have a value in their Optional fields. Create a new v2 IOU contract from the `v2/my-pkg` directory, with `USD` as currency:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> # Create a new v2 IOU contract, with USD as currency
> cd ../../v2/my-pkg
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.1.0.dar \
--script-name Main:mkIOU \
--output-file alice-v2
...
```
Query it from a v1 script in the `v1/my-pkg` directory:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> # Query from v1 package
> cd ../../v1/my-pkg
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.0.0.dar \
--script-name Main:getIOU \
--output-file /dev/stdout \
--input-file ../../v2/my-pkg/alice-v2
...
Exception in thread "main" com.daml.lf.engine.script.Script$FailedCmd: Command QueryContractKey failed: Failed to translate create argument:
...
An optional contract field with a value of Some may not be dropped during downgrading.
```
The error states that the optional field may not be dropped during downgrading.
Contracts created in a workflow from a v2 package may be used if the optional, upgraded fields are set to `None`. For example, create an IOU with the currency field set to `None` using `mkIOUWithoutCurrency`:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> # Create a new v2 IOU contract, without USD as currency
> cd ../../v2/my-pkg
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.1.0.dar \
--script-name Main:mkIOUWithoutCurrency \
--output-file alice-v2
...
```
And then query it from v1:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> # Query from v1 package
> cd ../../v1/my-pkg
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.0.0.dar \
--script-name Main:getIOU \
--output-file /dev/stdout \
--input-file ../../v2/my-pkg/alice-v2
...
"issuer": "party-...",
"owner": "party-...",
"value": 1
...
```
In this case, the query from v1 succeeded because all upgraded fields are set to `None`.
#### Adding a Choice
SCU also allows package authors to add new choices - add the example choice `Double` to `v2/my-pkg/daml/Main.daml`, which archives the current contract and produces a new one with twice the value.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
maintainer key
choice Double : ContractId IOU
controller issuer
do create this with value = value * 2
...
```
Along with a script to call it.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
import DA.Optional (fromSome)
...
doubleIOU : Party -> Script IOU
doubleIOU alice = do
oIOU <- getIOU alice
case oIOU of
Some (cid, _) -> do
newCid <- alice `submit` exerciseCmd cid Double
fromSome <$> queryContractId alice newCid
None -> fail "Failed to find IOU"
```
Compiled changes are checked against the previous version and pass:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm build
...
Compiling my-pkg to a DAR.
...
Created .daml/dist/my-pkg-1.1.0.dar
...
```
Restart the sandbox and re-upload both v1 and v2:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v1/my-deps
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> # Make a new IOU
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.0.0.dar \
--script-name Main:mkIOU \
--output-file alice-v1
...
> cd ../../v2/my-deps
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
...
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.1.0.dar \
--script-name Main:doubleIOU \
--output-file /dev/stdout \
--input-file ../../v1/my-pkg/alice-v1
...
"issuer": "party-...",
"owner": "party-...",
"value": 2,
"currency": null
...
```
Contracts made in v1 can now be exercised with choices introduced in v2.
Exercising a v1 choice on a v2 contract is also possible if upgraded fields are set to `None`, but this requires a different script function -replace the use of `exerciseCmd` with `exerciseExactCmd` in the body of `doubleIOU` in v1, and restart your sandbox.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> # Replace exerciseCmd with exerciseExactCmd in v1
> # Do it using your editor, or use `sed`
> sed -i -E 's/exerciseCmd/exerciseExactCmd/g' \
v1/my-pkg/daml/Main.daml
```
Once you’ve restarted your sandbox, create an IOU without a currency in V2 via `mkIOUWithoutCurrency`, then run `doubleIOU` on it from V1:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> # Create a new v2 IOU contract, without USD as currency
> cd v2/my-pkg
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.1.0.dar \
--script-name Main:mkIOUWithoutCurrency \
--output-file alice-v2
> cd ../../v1/my-pkg
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> dpm script \
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.0.0.dar \
--script-name Main:doubleIOU \
--output-file /dev/stdout \
--input-file ../../v2/my-pkg/alice-v2
...
"issuer": "party-...",
"owner": "party-...",
"value": 2
...
```
Existing choices can also be upgraded, as covered in Continuing to Write Your [Upgrades]().
### Deploying Your First Upgrade
#### Configuring Canton to Support Smart Upgrading
When using the feature one must be using Protocol Version 7.
#### Using Smart Contract Upgrading Enabled Packages
Once you have finished development of your smart contract app, use the mentioned upgrade-enabled options in daml.yaml to compile and generate the related DAR. This can be uploaded using the existing gRPC endpoints without modifications and is immediately available for use.
Once a DAR is successfully uploaded, it cannot be safely removed from the participant node. Participant operators must then ensure that uploaded functionality does not break the intended upgrade lineage as newly uploaded DARs affect the upgrading line (i.e. all subsequent uploads must be compatible with this one as well).
Packages stored on the participant must lead to unique package-id -> (package-name, package-version) relationships since runtime package-name -> package-id resolution must be deterministic (see Ledger [API]()). For this reason, once a DAR has been uploaded with its main package having a specific package-name/package-version, this relationship cannot be overridden. Hence, uploading a DAR with different content for the same name/version as an existing DAR on the participant leads to a rejection with error code KNOWN\_DAR\_VERSION.
##### Validate the DAR Against a Running Participant Node
You can validate your DAR against the current participant node state without uploading it to the participant via the `PackageManagementService.validateDar` Ledger API endpoint. This allows participant node operators to first check the DAR before uploading it.
This operation is also available via the Canton Admin API and Console:
```
participant.dars.validate("dars/CantonExamples.dar")
```
##### Upgrading and Package Vetting
Upgradable packages are also subject to package vetting restrictions: in to be able to use a package in Daml transactions with smart contract upgrading, it must be vetted by all participants informed about the transaction. This applies to both the packages used for creating the contracts and the target packages.
**Note:** Package vetting is enabled by default on DAR upload operations.
### Continuing to Write Your Upgrades
SCU allows package authors to change many more aspects of their packages - fields can be extended in templates, choices, and data type definitions. Choice bodies can be changed, and other expressions such as key definitions and signatory lists can be changed with caveats.
#### Setup
Continue the package defined in the Writing Your First [Upgrade]() section above, but overwrite the v1 and v2 IOU modules. The v1 IOU module should be overwritten as follows:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import Daml.Script
template IOU
with
issuer: Party
owner: Party
value: Int
where
signatory issuer
observer owner
key issuer : Party
maintainer key
mkIOU : Script Party
mkIOU = do
alice <- allocateParty "alice"
alice `submit` createCmd (IOU alice alice 1)
pure alice
getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key
```
The v2 IOU module should be overwritten to look like the following:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import Daml.Script
import DA.Optional (fromOptional)
template IOU
with
issuer: Party
owner: Party
value: Int
currency: Optional Text
where
signatory issuer
observer owner
key issuer : Party
maintainer key
mkIOU : Script Party
mkIOU = do
alice <- allocateParty "alice"
alice `submit` createCmd (IOU alice alice 1 (Some "USD"))
pure alice
mkIOUWithoutCurrency : Script Party
mkIOUWithoutCurrency = do
alice <- allocateParty "alice"
alice `submit` createCmd (IOU alice alice 1 None)
pure alice
getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key
```
All other files should remain the same.
#### Changing Choices
Add the following choice, `Duplicate`, to both v1 and v2 versions of IOU:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data DuplicateResult = DuplicateResult with
newCid : ContractId IOU
choice Duplicate : DuplicateResult
controller issuer
do
cid <- create this with value = value * 2
return DuplicateResult with newCid = cid
```
Running `dpm build` should succeed without errors.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v1/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.0.0.dar
> cd ../../v2/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.1.0.dar
```
Next, upgrade the `Duplicate` choice by adding an optional field `amount`, and changing the behavior of the choice to default to a multiple of 3. Also upgrade the `DuplicateResult` data type to include the old value. Replace the definitions of the `DuplicateResult` data type and of the `Duplicate` choice in v2 only:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
-- Add import to top of module
import DA.Optional (fromOptional)
...
-- Replace DuplicateResult definition
data DuplicateResult = DuplicateResult with
newCid : ContractId IOU
oldValue : Optional Int -- New optional oldValue field
...
-- Replace Duplicate choice implementation
choice Duplicate : DuplicateResult
with
amount : Optional Int -- New optional amount
controller issuer
do
let amt = fromOptional 3 amount
cid <- create this with value = value * amt
return DuplicateResult with
newCid = cid
oldValue = Some value
...
```
Add a script called `duplicateIOU` in V1:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
duplicateIOU : Party -> Script (Optional (ContractId IOU, IOU))
duplicateIOU key = do
mbIOU <- getIOU key
case mbIOU of
None -> pure None
Some (iouCid, _) -> do
res <- key `submit` exerciseExactCmd iouCid Duplicate
mbNewIOU <- queryContractId key res.newCid
case mbNewIOU of
Some newIOU -> pure (Some (newCid, newIOU))
None -> pure None
```
A similar script called `duplicateIOU` should be added in V2, supplying an `amount` field:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
duplicateIOU : Party -> Script (Optional (ContractId IOU, Int, IOU))
duplicateIOU key = do
mbIOU <- getIOU key
case mbIOU of
None -> pure None
Some (iouCid, _) -> do
res <- key `submit` exerciseExactCmd iouCid Duplicate { amount = Some 4 }
case res.oldValue of
None -> pure None -- This should never happen
Some oldValue ->
mbNewIOU <- queryContractId key res.newCid
case mbNewIOU of
Some newIOU -> pure (Some (newCid, oldValue, newIOU))
None -> pure None
```
Running the v1 `duplicateIOU` script with `exerciseExactCmd` always runs the v1 implementation for the `Duplicate` choice, and likewise for v2.
#### Modifying Signatory Definitions
Other definitions can be changed, but warnings are emitted to remind the developer that the changes can be unsafe and need to be made with care to preserve necessary invariants.
Signatories and observers are one expression that can be changed. Importantly, SCU assumes that the new definition does not alter the computed values of the signatories. The computed value of the observers is allowed to change in one specific way: observers that are also signatories can be removed. Any other change to the computed value of the observers (losing a non-signatory observer, adding an observer) is not allowed.
For example, add a new field of "outside observers" to the v2 IOU template, and add them to the observer definition.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
-- Add a new outsideObservers field
outsideObservers: Optional [Party]
where
signatory issuer
-- Add outsideObservers to the observer definition
observer owner, fromOptional [] outsideObservers
...
```
The new observer definition allows v2 contracts and beyond to add new observers via the outsideObservers field. However, any existing contracts default to `None` for the `outsideObservers` field, so all existing contracts have the same observer list as before: the single owner.
In the case where a contract's signatories or observers change in during an upgrade/downgrade in a way that doesn't meet the constraints above, the upgrade, and thus full transaction, fails at runtime.
#### Upgrading Enums
Variants and enums can be extended using SCU, either by adding fields to an existing constructor, or by adding a new constructor to the end of the list.
Redefine the IOU package, overwriting the v1 and v2 sections similarly to the previous section. Overwrite the IOU package in both V1 and V2 with the following:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import Daml.Script
template IOU
with
issuer: Party
owner: Party
value: Int
currency: Currency
where
signatory issuer
observer owner
key issuer : Party
maintainer key
data Currency
= USD
| GBP
deriving (Show, Eq, Ord)
mkIOU : Script Party
mkIOU = do
alice <- allocateParty "alice"
alice `submit` createCmd (IOU alice alice 1 USD)
pure alice
getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key
```
Instead of using `Text` for the currency field, here we use an enum data-type `Currency` with two constructors: `USD` and `GBP`.
Running `dpm build` should succeed with no errors:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v1/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.0.0.dar
> cd ../../v2/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.1.0.dar
```
When you want to extend our contract to support new currencies, you can add new entries to the end of our `Currency` enum.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
data Currency
= USD
| GBP
| CHF -- Add a new currency type
deriving (Show, Eq, Ord)
...
```
Upgrades of extended enums from an old version to a new version always succeed. In the case of IOUs, a v1 IOU can always be interpreted as a v2 IOU because the constructors for its `currency` field are a subset of those in a v2 contract.
For example, create an IOU with USD via v1’s `mkIOU` script, and query it via v2’s `getIOU` script:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v1/my-pkg
> dpm script
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.0.0.dar \
--script-name Main:mkIOU \
--output-file alice-v1
...
> cd ../../v2/my-pkg
> dpm script
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.1.0.dar \
--script-name Main:getIOU \
--output-file /dev/stdout \
--input-file ../../v1/my-pkg/alice-v1
...
"issuer": "party-...",
"owner": "party-...",
"value": 1,
"currency": "USD"
...
```
Only constructors that are defined in both v1 and v2 can be downgraded from v2 to v1. Any constructor that does not exist in the v1 package fails to downgrade with a runtime error. In the case of our `IOU`, any `CHF` fails to downgrade, so any v2 contracts with a `CHF` currency cannot be used in v1 workflows.
For example, create a contract with `CHF` as its `currency` field via v2’s `mkIOU` script. Attempting to query it via v1’s `getIOU` script fails with a lookup error for the CHF variant.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v2/my-pkg
> dpm script
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.1.0.dar \
--script-name Main:mkIOU \
--output-file alice-v2
...
> cd ../../v1/my-pkg
> dpm script
--ledger-host localhost --ledger-port 6865 \
--dar .daml/dist/my-pkg-1.0.0.dar \
--script-name Main:getIOU \
--output-file /dev/stdout \
--input-file ../../v2/my-pkg/alice-v2
...
Failed to translate create argument: Lookup(NotFound(DataVariantConstructor(c1543a5c2b7ff03650162e68e03e469d1ecf9f546565d3809cdec2e0255ed902:Main:Currency,CHF),DataEnumConstructor(c1543a5c2b7ff03650162e68e03e469d1ecf9f546565d3809cdec2e0255ed902:Main:Currency,CHF)))
...
```
#### Upgrading Variants
Variants, also known as algebraic data types, are very similar to enums except that they also contain structured data.
For example, the following variant has two constructors, each with unique fields. Overwrite both v1 and v2 modules with the following source:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
data Shape
= Circle
| Polygon { sides : Int }
```
You can extend this variant in two ways. You can add a new constructor, similarly to enums. Modify the v2 module to add a new `Line` constructor with a `len` field:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
data Shape
= Circle
| Polygon { sides : Int }
| Line { len : Numeric 10 } -- New line constructor
```
As before, building should succeed.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v1/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.0.0.dar
> cd ../../v2/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.1.0.dar
```
You can also add a new field to a constructor, similarly to templates -for example, add a `sideLen` field to the `Polygon` constructor, to specify the lengths of the sides of the polygon.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Shape
= Circle
| Polygon
{ sides : Int
, sideLen : Numeric 10 -- New field
}
| Line { len : Numeric 10 }
```
Building this fails because the new `sideLen` field is non-optional.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v2/my-pkg
> dpm build
...
error type checking data type Main.Shape:
The upgraded variant constructor Polygon from variant Shape has added a field.
ERROR: Creation of DAR file failed.
```
Making the new `sideLen` field optional fixes the error:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
, sideLen : Optional (Numeric 10) -- New field
...
```
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v2/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.1.0.dar
```
#### Limitations in Upgrading Variants
Upgrading variants has some limitations - because the `Circle` constructor has been defined without a field in curly braces, it cannot be upgraded with new fields.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
-- Add a field where no fields existed
= Circle { radius : Optional (Numeric 10) }
...
```
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v2/my-pkg
> dpm build
...
error type checking data type Main.Shape:
The upgraded data type Shape has changed the type of a variant.
ERROR: Creation of DAR file failed.
```
The same applies to variants with unnamed fields. If the v1 definition of the `Line` constructor were as follows, it would also not be able to upgrade:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
| Line (Numeric 10)
```
In general, in order to enable future upgrades, it is strongly recommended that all constructors use named fields, and that all constructors have at least one field. If a constructor has no fields in an initial v1 package, one can assign a dummy field.
For example, the correct way to write the v1 `Circle` constructor would be as follows:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
= Circle { dummy : () }
...
```
The subsequent v2 upgrade would succeed:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
= Circle { dummy : (), radius : Optional (Numeric 10) }
...
```
#### Nested Datatypes
If a data type, choice, or template has a field which refers to another data type, the larger data type can be upgraded if the field’s data type is upgradeable.
For example, given the data type `A` with a field referring to data type `B`,
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data A = A { b : B }
data B = B { field : Text }
```
If modifications made to `B` are valid for SCU, then `A` is also valid.
#### Dependencies
Package authors may upgrade the dependencies of a package as well as the package itself. A new version of a package may add new dependencies, and must have all the (non-utility-package) dependencies of the old version. If these dependencies are used in ways that are checked for upgrades, each existing dependency must be either unchanged from the old DAR or an upgrade of its previous version.
For example, given a dependencies folder containing v1, v2, and v3 of a dependency package `dep`:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> ls ./dependencies
dep-1.0.0.dar
dep-2.0.0.dar
dep-3.0.0.dar
```
Then assume a version `1.0.0` of a package `main` that depends on a datatype from version `2.0.0` of `dep`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import qualified Dep
data MyData = MyData
{ depData : Dep.AdditionalData
}
```
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
dependencies:
- daml-prim
- daml-stdlib
- daml3-script
data-dependencies:
- dependencies/dep-2.0.0.dar
...
```
Because a package with a newer version may upgrade any dependency to a newer version (or keep the version the same), version `2.0.0` of the `main` package may keep its dependencies the same, or it may upgrade `dep` to `3.0.0`:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
dependencies:
- daml-prim
- daml-stdlib
- daml3-script
data-dependencies:
- dependencies/dep-3.0.0.dar # Can upgrade dep-2.0.0 to dep-3.0.0, or leave it unchanged
...
```
You cannot downgrade a dependency when using that dependency's datatypes. For example, `main` may not downgrade `dep` to version `1.0.0`. The following `daml.yaml` would be invalid:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
dependencies:
- daml-prim
- daml-stdlib
- daml3-script
data-dependencies:
- dependencies/dep-1.0.0.dar # Cannot downgrade dep-2.0.0 to dep-1.0.0
...
```
If you try to build this package, the typechecker returns an error on a package ID mismatch for the Dep:AdditionalData field, because the Dep:AdditionalData reference in this case has changed to a package that is not a legitimate upgrade of the original.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm build
...
error type checking data type Main.MyData:
The upgraded data type MyData has changed the types of some of its original fields:
Field 'depData' changed type from :Dep:AdditionalData to :Dep:AdditionalData
```
#### Upgrading Interface Instances
SCU also supports changing Interface instances. First, create a new package directory `my-iface`, with `my-iface/daml.yaml` and module `my-iface/daml/MyIface.daml`:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
sdk-version: 3.3.0
name: my-iface
version: 1.0.0
source: daml/MyIface.daml
parties:
- Alice
- Bob
dependencies:
- daml-prim
- daml-stdlib
build-options:
- --target=2.1
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module MyIface where
data HasValueView = HasValueView { hasValueView : Int }
interface HasValue where
viewtype HasValueView
getValue : Int
```
And build the module:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd my-iface
> dpm build
...
Created .daml/dist/my-iface-1.0.0.dar
```
Add references to the newly created DAR in both `v1/my-pkg/daml.yaml` and `v2/my-pkg/daml.yaml`:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
dependencies:
- daml-prim
- daml-stdlib
- daml3-script
- ../../my-iface/.daml/dist/my-iface-1.0.0.dar
...
```
Overwrite both `v1/my-pkg/daml/Main.daml` and `v2/my-pkg/daml/Main.daml` with the following:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import Daml.Script
import MyIface
template IOU
with
issuer: Party
owner: Party
value: Int
where
signatory issuer
observer owner
key issuer : Party
maintainer key
interface instance HasValue for IOU where
view = HasValueView value
getValue = value
```
Interface instances can be changed by an upgrade. For example, v2 can change the definition of `getValue` in the `HasValue` instance.
Add a `quantity` field to the v2 IOU package, and amend the definition of `getValue` to use it:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
import DA.Optional (fromOptional)
template IOU
with
issuer: Party
owner: Party
value: Int
quantity: Optional Int -- new quantity field
where
...
interface instance HasValue for IOU where
view = HasValueView (value * fromOptional 1 quantity)
-- Use quantity field to calculate value
getValue = value * fromOptional 1 quantity
```
Shut down and relaunch the Daml sandbox, then build and upload the two DARs. They should both succeed again:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v1/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> cd ../../v2/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
```
Packages with new versions cannot remove an instance that is already there. For example, the v2 IOU template cannot remove its instance of `HasValue`. Comment out the interface instance for `HasValue` from `v2/my-pkg/daml/Main.daml` completely, then restart the sandbox and try to reupload the two versions:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v1/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> cd ../../v2/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
...
Implementation of interface ...:MyIface:HasValue by template IOU appears in package that is being upgraded, but does not appear in this package.
```
As another example, restore the instance deleted in the previous step and remove the `HasValue` interface from `v2/my-pkg/daml/Main.daml` instead. Then restart the sandbox and try to reupload the two versions.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd v1/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> cd ../../v2/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
```
Similarly to choices, scripts may invoke interface implementations from their own version using `exerciseExactCmd`.
#### Upgrading Interfaces
Interface instances may be upgraded, but interface definitions cannot be upgraded. If an interface definition is present in v1 of a package, it must be removed from all subsequent versions of that package.
Because interfaces definitions may not be defined in subsequent versions, any package that uses an interface definition from a dependency package can never upgrade that dependency to a new version.
For this reason, it is strongly recommended that interfaces always be defined in their own packages separately from templates.
### Developer Workflow
This section contains suggestions on how to set up your development environment for SCU to help you iterate more quickly on your projects.
#### Multi-Package Builds
Following the best practices outlined below and the testing recommendations leads to a proliferation of packages in your project. Use Multi-Package builds to reliably rebuild these packages as you iterate on your project. Multi-package builds also enable cross-package navigation in Daml Studio.
The Multi-package builds for upgrades section goes into more detail on how to set up multi-package builds for SCU and how it can help with testing.
#### Working with a Running Canton Instance
With SCU, it is no longer possible to iterate on a package by uploading it to a running participant after each rebuild. A participant rejects two packages with the same name and version whose content differs.
There are two ways to work around this:
* restart the participant after each rebuild
* change the version name of the package before each rebuild
Environment variable interpolation in DPM can help with the latter. In the `daml.yaml` file of each of your packages, append an environment variable to the name of the package:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
name: my-package-v${UNIQUE_BUILD_ID}
```
Make sure to also append the variable to the name of the DAR file produced by `my-package` in the `daml.yaml` files that depend on it:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
dependencies:
- daml-prim
- daml-stdlib
data-dependencies:
- ../my-package/.daml/dist/my-package-${UNIQUE_BUILD_ID}.dar
```
Then, before invoking `dpm build --all`, increment the `UNIQUE_BUILD_ID` environment variable. This ensures that the build produces unique DAR files that can be uploaded to the participant without conflict.
#### Working with the Daml Sandbox
For the same reason, the Daml sandbox does not support hot-swapping packages.
There are two ways to work around this:
* restart the sandbox after each rebuild
* change the version name of your packages after each rebuild, as outlined in the previous section
### The Upgrade Model in Depth - Reference
You can find the in-depth upgrading model, which can be used as a reference for valid upgrades, here.
## Package Selection in the Ledger API
Until the introduction of SCU, template IDs in requests to the Ledger API were all of the form `::`. For disambiguation, going forward, we refer to this format as **by-package-id template IDs**.
With SCU, we introduce a more generic template reference of the format `#::` (**by-package-name template ID**) that scopes templates by package-name, allowing version-agnostic addressing of templates on the Ledger API. It serves as a reference to all template IDs with the qualified name `:` from all packages with the name `package-name` known to the Ledger API server.
`By-package-name template ID` addressing is supported for all packages and is possible on both the write path (command submission) and read path (Ledger API queries).
The `by-package-name template ID` is an API-level concept of the Ledger API. Internally, Canton uses by-package-id template IDs for all operations. Therefore, when the new format is used in requests to the Ledger API, a dynamic resolution is performed as described in the following sections.
### Package Preference
On processing a command submission, there are scenarios where the Ledger API server needs to resolve `by-package-name template IDs` to `by-package-id template IDs` (see `dynamic-package-resolution-in-command-submission`). For this purpose, the **package preference** concept is introduced as the mapping from package-name to package ID for all known package-names on the participant.
The package preference is needed at each command submission time and is assembled from two sources:
* The **default** package preference of the Ledger API server to which the command is submitted. By default, for each package name the Ledger API picks a package version known by all participants involved in the transaction, if it exists.
* The package-id selection preference list specified in the submitted command's package\_id\_selection\_preference in a command submission. This is package-id resolution list explicitly provided by the client to override the default package preference mentioned above.
> * See here for how to provide this in Daml-Script
> * **Note:** The package\_id\_selection\_preference must not lead to ambiguous resolutions for package-names, meaning that it must not contain two package-ids pointing to packages with the same package-name, as otherwise the submission will fail with an `INVALID_ARGUMENT` error.
### Dynamic Package Resolution in Command Submission
Dynamic package resolution can happen in two cases during command submission:
* For command submissions that use a `by-package-name template ID` in the command’s templateId field (e.g. in a create command here)
* For command submissions whose Daml interpretation requires the execution of interface choices or fetch-by-interface actions.
In these situations, the package preference is used for selecting the target implementation package for these interface actions.
### Dynamic Package Resolution in Ledger API Queries
When subscribing for transaction or active contract streams, users can now use the `by-package-name template ID` format in the template-id request filter field. to specify that they’re interested in fetching events for all templates pertaining to the specified package-name. This template selection set is dynamic and it widens with each uploaded template/package.
#### Example
Given the following packages existing on the participant node:
* Package AppV1
* package-name: `app1`
* package-version: `1.0.0`
* template-ids: `pkgId1:mod:T`
* Package AppV2
* package-name: `app1`
* package-version: `1.1.0`
* template-ids: `pkgId2:mod:T`
If a transaction query is created with a templateId specified as `#app1:mod:T`, then the events stream will include events from both template-ids: `pkgId1:mod:T` and `pkgId2:mod:T`
## Best Practices
To ensure that future upgrades and DAR lifecycling go smoothly, we recommend the following practices:
### Separate Interfaces/Exceptions from Templates
Interface and exception definitions are not upgradable. As such, if you attempt to redefine an interface or exception in version 2 of a package, even if it is unchanged, the package does not type check. Removing an interface from the version 2 package also causes issues, especially if the interface has choices.
This means that template definitions that exist in the same package as interfaces and exception definitions are not upgradeable. To avoid this issue, move interface and exception definitions into a separate package such that subsequent versions of your template package all depend on the same version of the package with interfaces/exceptions.
For example, a single package `main` defined as follows would not be able to upgrade, leaving the template `T` non-upgradeable.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
interface I where
...
template T with
...
```
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
sdk-version: 3.3.0
name: main
version: 1.0.0
source: Main.daml
dependencies:
- daml-prim
- daml-stdlib
build-options:
- --target=2.1
```
The SCU type checker emits an error and refuses to compile this module:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
error type checking :
This package defines both interfaces and templates. This may make this package and its dependents not upgradeable.
It is recommended that interfaces are defined in their own package separate from their implementations.
Downgrade this error to a warning with -Wupgrade-interfaces
Disable this error entirely with -Wno-upgrade-interfaces
```
**Note:** It is very strongly recommended that you do not compile interfaces or exceptions in a package alongside templates. However, you can downgrade this error to a warning by passing the `-Wupgrade-interfaces` flag, or ignore this error entirely with the `-Wno-upgrade-interfaces` flag.
The recommended way to fix this is to split the `main` package by redefining it as two packages, `helper` and `main`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Helper where
interface I where
...
```
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
sdk-version: 3.3.0
name: helper
version: 1.0.0
source: Helper.daml
dependencies:
- daml-prim
- daml-stdlib
build-options:
- --target=2.1
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import qualified Helper
template T with
...
```
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
sdk-version: 3.3.0
name: main
version: 1.0.0
source: Main.daml
dependencies:
- daml-prim
- daml-stdlib
data-dependencies:
-
build-options:
- --target=2.1
```
To manage the complexity of working with multiple packages at once, we recommend using multi-package builds.
### Avoid Contract Metadata Changes
The signatories, stakeholders, contract key, and ensure clauses of a contract should be fixed at runtime for a given contract. Changing their definitions in your Daml code is discouraged and triggers a warning from the SCU typechecker.
We strongly recommend against altering the type of a key. If changing the type of a key cannot be avoided, consider using an off-ledger migration instead of SCU.
### Breaking Changes via Explicit Package Version
To make a breaking change to your package that is not upgrade compatible, you can change the name of your package to indicate a breaking version bump. To enable this, we recommend that your package name contains a version marker for when a breaking change occurs.
For example, for your first iteration of a package, you would name it `main-v1`, starting with package version `1.0.0`. In this case, the `v1` is part of the *package name*, not the package version. You could publish upgrade-compatible versions by changing the `version:` field from `1.0.0` to `2.0.0` to `3.0.0`. These versions would all be upgrade-compatible with one another:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
main-v1-1.0.0
main-v1-2.0.0
main-v1-3.0.0
```
Note how the `v1` in all three packages remains stable - this means the package name has not changed, and ensures that these three packages and their datatypes are considered by the runtime and the type checker to be upgradeable.
To make a breaking change, publish a new package version with package name `main-v2`. Because this package has a different package name from those with `main-v1`, it is not typechecked against those packages and its datatypes are not automatically converted. You would need to manually migrate values from `main-v1` packages to `main-v2`.
### Do not depend on Daml Script Packages
We recommend only depending on Daml Script packages such as `daml-script` in dedicated packages for running tests written in Daml Script. These packages should not be part of your model and should not be uploaded to the ledger.
Daml Script packages are not guaranteed to be upgradeable across SDK versions. If you depend on a Daml Script datatype in a serializable position (e.g. the field of a template), your package may rely on a Daml Script package in a way that can neither be removed nor upgraded to the next SDK version. Your package and any of its SCU upgrades would be stuck on that SDK version.
### Operational Design Guideline for Upgrading Daml Apps
When considering upgrading, we regard each Daml application as composed of:
* **On-ledger components**: The Daml code running the on-ledger logic (i.e. the DARs uploaded to all participant nodes interacting with the app)
* **Off-ledger components** interacting with the ledger via the Ledger API or other supported Canton Ledger API clients (JSON Ledger API or PQS). These are typically Java or TypeScript services implementing off-ledger business logic.
#### Zero-Downtime Upgrades
Upgrading an application without operational downtime can be achieved by designing the above-mentioned components to allow:
* **Asynchronous rollout**: Operators deploy updated software at their own pace, similar to established CI/CD practices in micro-service environments.
* **Synchronous switch-over**: All components switch to using updated Daml code at the same time.
**Designing for asynchronous upgrade roll-outs**
To allow for asynchronous roll-outs, off-ledger components should:
* **use package names in Ledger API requests**: App components interacting with the Ledger API should use `by-package-name template IDs` instead of `by-package-id template IDs` in all their command submissions and queries. This allows:
> * Ledger API server-side version selection of the package preference for command submissions.
> * reading all versions of the templates in queries, even newer versions than the one the component was developed against.
* **handle missing optional fields**: App components reading from the Ledger API or Ledger API clients must be prepared to handle missing optional fields in records, including those in the initial package. The TypeScript and Java codegens for reading Daml values do so by default.
* **use exhaustive package preference**: on each command submission, the `package_id_selection_preference` is set ensuring that:
> * The package ID of every package used in the command and of every package used by all possible interface instances is included in the package preference.
> * Within an application, all submissions use the same package preference.
Following these recommendations decouples the application's behavior from the DAR uploads (see the note regarding Package Preference), as package resolution is fully determined by the explicit and exhaustive package preference provided in command submissions.
**Operational steps for upgrading**
Once a new Daml package version of the application is ready, define an operational window for allowing the asynchronous rollout of the updated components. During this window, Canton node and off-ledger app operators should:
* Upload the upgraded DARs to the participant nodes
* Roll-out the updated off-ledger components
After the operational window passes, the application can switch over to the upgraded logic as such:
* A switch-over time is decided and communicated to all app clients in advance, along with the updated package preference pertaining to the application's upgraded DARs. For example, you may encode the switch-over time in a config value set on all components; or publish it at a well-known HTTP endpoint from which all components read it.
* After the switch-over time, all Ledger API clients update their package preference and use it for subsequent command submissions.
Ledger API clients or participant nodes that do not finish the components rollout before the update switch-over time may not be able to participate in the upgraded workflow.
#### Rolling out backwards-incompatible changes
Some changes to a Daml workflow cannot be made backwards-compatible, such as changing the definition of a template in a breaking way (e.g., extending the observer set).
To handle such changes, you can replace the existing contract with an upgraded one of the target template as follows:
* Introduce the target template as a new template following the Breaking changes via Explicit Package Version
* Add a consuming `OriginalTemplate_UpgradeToNewTemplate` choice to the existing template by rolling out a new version in a backwards-compatible fashion.
* Where required, provide reference data for the `upgrade` choice via additional choice arguments.
* Implement backend automation to migrate all old contracts to the new ones by exercising the `upgrade` choice on the existing contracts.
Rolling out backwards-incompatible changes as described in this section incurs downtime on affected workflows until their contracts have been converted by the automation. In order to not disrupt business, such rollouts should be executed during maintenance windows.
Note that this kind of upgrade requires O(number-of-active-contracts) of transactions to roll out. Depending on the size of your ACS this can take a long time and consume significant compute and storage resources. In contrast, the backwards-compatible upgrades can be rolled out with constant cost, independent of the size of your ACS.
## Testing
### Standalone Upgradeability Checks
We recommend using the `upgrade-check` tool to perform a standalone check that all of the DARs typecheck against one another correctly as further validation of your upgraded packages.
This tool takes a list of DARs and runs Canton's participant-side upgrade typechecking without spinning up an instance of Canton. You should pass the tool the list of DARs constituting your previous model and the list of DARs for your new model.
For example, assume you have a helper package `helper` that does not change, and two packages `main` and `dep`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
main-1.0.0.dar
dep-1.0.0.dar
helper-1.0.0.dar
```
After upgrading your model, you would publish a new DAR `main-2.0.0.dar` for `main` and a new DAR `dep-2.0.0.dar` for `dep`. We would then recommend running the upgrade-check tool as follows:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm upgrade-check --participant helper-1.0.0.dar dep-1.0.0.dar main-1.0.0.dar dep-2.0.0.dar main-2.0.0.dar
...
```
This runs the same upload validation over these DARs that would be run in the event of an upload to the ledger, and prints out the same messages and errors. Because it does not require a ledger to be spun up, the command runs much more quickly.
We can also check that all of the DARs pass compiler-side checks, but this is much less likely to indicate an issue because the DARs are typechecked during compilation.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm upgrade-check --compiler helper-1.0.0.dar dep-1.0.0.dar main-1.0.0.dar dep-2.0.0.dar main-2.0.0.dar
...
```
### Dry Run Uploading to a Test Environment
If you have a test environment with DARs that are not available to you, you may not be able to supply a complete list of DARs for your previous model to the standalone `dpm upgrade-check` tool.
For workflows involving multiple DARs, we recommend more robust testing by running a Canton sandbox with the same version and environment as your in-production participant and uploading all the old and new packages that constitute your Daml app.
If you want to test the validity of your package against a running ledger, but without uploading the dar, you can hit the validateDar gRPC endpoint
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> grpcurl -plaintext -d @ \
localhost:6865 \
com.daml.ledger.api.v2.admin.PackageManagementService.ValidateDarFile < helper-1.0.0.dar
...
```
### Daml Script Testing
Daml Script has been used for demonstrative purposes in this document, however usually the complexities of live upgrades comes with your workflows, not the data transformations themselves. You can use Daml Script (with Canton) to test some isolated simple cases, but for thorough tests of you system using SCU, you should prefer full Workflow [Testing](#testing).
We recommend placing your Daml Script tests in a separate package which depends on all versions of your business logic when testing your upgrades with Daml Script. This testing package should not be uploaded to the ledger if possible, as it depends on the `daml-script` package. This package emits a warning on the participant when uploaded, as it serves no purpose on a participant, cannot be fully removed (as with any package), and may not be uploadable to the ledger in future versions (Daml 3). More information about this limitation here.
Depending on multiple versions of the same package does however face ambiguity issues with imports. You can resolve these issues using module prefixes:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
name: my-testing-package
...
data-dependencies:
- my/v1/main-package.dar
- my/v2/main-package.dar
module-prefixes:
main-1.0.0.dar: V11
main-1.1.0.dar: V12
```
It is important to verify old workflows are still functional under new data and new implementation when writing your tests. You also need to verify that any new workflows intended to be backward compatible can consume old data. You should build your testing structure to cover this how you see fit, but we give the following recommendation as a starting point:
If your new version only includes choice body or interface instance changes (that is, it is a patch release)
* Run your existing test suite for V1 but updated to call V2 choices. This can be done with a rewrite, or by passing down a package preference and calling the test with both the V1 and V2 package ID.
If your new version includes data changes, be that to contract payloads or choices (that is, it is a minor release)
* Assuming your data change affects a template payload, write separate setup code for V1 and V2, populating new fields
`setupV1 : Script V1TestData`
`setupV2 : Script V2TestData`
These new data types should mostly just hold Contract IDs
* Update your existing test suite from V1 to take a package preference, allowing the V2 implementation without additional fields to choices to be called.
`testV1 : [PackageId] -> V1TestData -> Script ()`
* Run the above test suite against V1 data, passing a V1 preference, then a V2 preference.
This ensures your changes haven't broken any existing workflows.
* Next write tests for your new/modified workflows, using the V2 choice implementations. This does not need a package preference.
`testV2 : V2TestData -> Script ()`
* Run these tests against both the V1 setup and the V2 setup, to ensure your new workflows support existing/old templates.
In order to do this, you'll need some way to upcast your `V1TestData`, i.e.
`upcastTestData : V1TestData -> V2TestData`
This function should mostly just call `coerceContractId` on any contract IDs, and fill in any `None` values if needed.
* Finally, you can cover any workflows that require the contract data to already be upgraded (new fields populated), these are written entirely in V2 without any special considerations.
### Multi-package builds for upgrades
When you are developing upgrades, you may have multiple DARs in scope that need to be built together. Tracking these DARs and building them in the right order can be complicated, especially as you develop live and as the project grows.
Multi-package builds help with projects containing multiple DARs, for example, a project using upgrades.
To understand how multi-package builds simplify the development of a project using upgrades, begin by creating a new Daml project with the `upgrades-example` template.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm new upgraded-iou --template upgrades-example
> cd upgraded-iou
> tree
.
├── multi-package.yaml
├── run-test.sh
├── interfaces
│ ├── daml/...
│ └── daml.yaml
├── main-v1
│ ├── daml/...
│ └── daml.yaml
├── main-v2
│ ├── daml/...
│ └── daml.yaml
└── test
├── daml/...
└── daml.yaml
```
The example template contains:
* A package `upgraded-iou-interfaces`, which defines an interface `Asset` and a viewtype `Asset.View`.
* The first version (1.0.0) of a package `upgraded-iou-main`, which defines a template `IOU` with instance of `upgraded-iou-interfaces:Main.Asset`.
* The second version (2.0.0) of `upgraded-iou-main` which upgrades the first. It adds a new `description` field to `IOU`, and uses it (when the field is defined) in an upgraded implementation of `Asset`.
* A testing package `upgraded-iou-test`, which depends on both `upgraded-iou-main-1.0.0` and `upgraded-iou-main-2.0.0`. It defines a script which exercises v1.0.0 and v2.0.0 `IOU`s via their `Asset` interface.
* A script `run-test.sh`, which runs the main test in `upgraded-iou-test`.
* A `multi-package.yaml` file which lists our four packages.
Without multi-package builds you would test your program like this:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> # Run sandbox in the background, wait until the three lines below are shown
> dpm sandbox &
Starting Canton sandbox.
Listening at port 6865
Canton sandbox is ready.
> # Build all, run test
> cd interfaces/; dpm build --enable-multi-package=no
> cd ../main-v1/; dpm build --enable-multi-package=no
> cd ../main-v2/; dpm build --enable-multi-package=no
> cd ../test/; dpm build --enable-multi-package=no
> cd ..
> ./run-test.sh
> # Modify v2, run test
> cd main-v2/
> ... modify main-v2 ...
> dpm build --enable-multi-package=no
> cd ../test/; dpm build --enable-multi-package=no
> cd ..
> # Modify test, run test
> cd test/
> dpm build --enable-multi-package=no
> cd ..
> ./run-test.sh
...
Test output:
...
> kill %1 # Kill backgrounded sandbox process
```
Forgetting to rebuild packages after changing their source would not cause a failure - for example, if you modified the source from `main-v2` in an incompatible way but did not recompile it, the `test` package would still compile successfully against the previous DAR for `main-v2`.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> # Modify main-v2 in an incompatible way
> cd main-v2/
> ... add a non-optional field `currency: Text` to template T in main-v2 ...
> cd ../test/
> dpm build --enable-multi-package=no
...
Created .daml/dist/upgraded-iou-upgrades-template-test-1.0.0.dar
> # Compiling `test` succeeded even though main-v2 was changed incorrectly
```
With Daml multi-package builds, all builds automatically rebuild dependencies if their source has changed:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> cd test/
> dpm build # --enable-multi-package is set to true by default
...
Building /home/dylanthinnes/ex-upgrades-template/main-v2
...
Severity: DsError
Message:
error type checking template Main.IOU :
The upgraded template IOU has added new fields, but the following new fields are not Optional:
Field 'currency' with type Text
...
> # Compiling `test` failed as expected because main-v2 was changed incorrectly
```
The `./run-test.sh` script automatically rebuilds all DARs in the package that need to be rebuilt:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> dpm sandbox & # Start sandbox in background
Starting Canton sandbox.
Listening at port 6865
Canton sandbox is ready.
> ... Fix main-v2 by dropping non-optional `currency` field ...
> # Re-run test
> ./run-test.sh
...
Building /home/dylanthinnes/ex-upgrades-template/main-v2
...
> # Modify test, run test
> ... modify test ...
> dpm build --all
> ./run-test.sh
```
Multi-package builds invoked by `dpm build --all` always recompile stale dependencies and DARs in order. This ensures a fully up-to-date package environment before running `./run-test.sh`.
### Workflow Testing
While testing your workflows is application-specific, we still recommend at least one test for your core workflows that follows this pattern:
1. Start your app using version 2.0 of your DAR, but only upload version 1.0.
2. Initialize the app and start one instance of every core workflow.
3. Upload version 2.0 of your DAR.
4. Switch your backends to start using version 2.0, ideally this should be a flag.
5. Validate that the core workflows are in the same state and advance them to check that they are not stuck.
## SCU Support in Daml Tooling
### Codegen
Generated code uses package names in place of package IDs in template IDs. Retrieved data from the ledger is subject to the upgrade transformations described in previous sections.
Concretely, this is implemented as follows:
#### Java
The classes that are generated for each template and interface contain a `TEMPLATE_ID` field, which, for upgradable packages, now use a package name rather than a package ID. To help you determine the package ID of these packages, we have added a new `PACKAGE_ID` field to all such classes. Upgradable packages also have `PACKAGE_NAME` and `PACKAGE_VERSION` fields.
If you need to identify a template by the specific package ID of the DAR from which the code was generated, you can use the `TEMPLATE_ID_WITH_PACKAGE_ID` field, which is on all generated classes and their companion objects. When submitting commands you may also use the `packageIdSelectionPreference` to explicitly specify which package ID(s) you want the ledger to use to interpret the commands. Note that using these options prevents your application from seamlessly supporting upgrades.
#### TypeScript
The `templateId` field on generated template classes has been updated to use the package name as the package qualifier for upgrade-compatible packages. This is used for command submission and queries. However, note that queries return the package qualifier with the package ID rather than the package name. Generated modules now also give the package "reference", which is the package name for upgrade-compatible packages; for other packages it is the package ID.
To perform package ID-qualified commands/queries in an upgrade compatible package, create a copy of the template object using the following:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
const MyTemplateWithPackageId = {
...pkg.Mod.MyTemplate,
templateId: `${pkg.packageId}:Mod:MyTemplate`,
}
```
### JSON Ledger API
Packages can be addressed both by package ID and by package name in the interaction with the JSON Ledger API.
Note: template IDs in query results always use a package ID. This allows you to distinguish the source of a particular contract. This means that if you use a template with a package name in the request, you can no longer expect the template IDs in the result to exactly match the input template ID.
Package ID selection preference: preferences apply to JSON Ledger API where you can specify your preferred selection of package versions.
### PQS
To match the package-name changes to the Ledger API, PQS has changed how packages are selected for queries. All queries that take a Daml identity in the form `::` now take a package name in place of package ID. Note that this differs from the Ledger API in that the `\#` prefix is not required for PQS, as PQS has dropped direct package ID queries. Queries for package names return all versions of a given contract, alongside the package version and package ID for each contract.
If you still need to perform a query with an explicit package ID, you can either use a previous version of PQS or add the following filter predicate to your query: `SELECT \* FROM active('my_package:My.App:MyTemplate') WHERE package_id = 'my_package_id'`
Given that PQS uses a document-oriented model for ledger content (JSONB), extensions to contract payloads are handled simply by returning the additional data in the blob.
### Daml Shell
Daml Shell builds on PQS by providing a shell interface to inspect the ledger using package name to create an integrated view of all versions of contracts.
### Daml-Script
All commands and queries in this version of Daml Script now use upgrades/downgrades automatically to ensure that they exercise the correct versions of choices and return correct payloads.
The following additional functionality is available for more advanced uses of SCU.
**Exact commands**
Each of the four submission commands now has an "exact" variant: `createExactCmd`, `exerciseExactCmd`, `exerciseByKeyExactCmd`, and `createAndExerciseExactCmd`. These commands force the participant to use the exact version of the package that your script uses, so you can be certain of the choice code you are calling. Note that exact and non-exact commands can be mixed in the same submission.
**Daml Script Package Preference**
A submission can specify a package preference as a list of package IDs:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
(actAs alice <> packagePreference [myPackageId]) `submitWithOptions` createCmd ...
```
Note the use of `submitWithOptions : SubmitOptions -> Commands a -> Script a`. You can build `SubmitOptions` by combining the `actAs` and `packagePreference` functions with `<>`.
The full list of builders for `SubmitOptions` is as follows:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- `p` can be `Party`, `[Party]`, etc.
actAs : IsParties p => p -> SubmiOptions
readAs : IsParties p => p -> SubmitOptions
disclose : Disclosure -> SubmitOptions
discloseMany : [Disclosure] -> SubmitOptions
newtype PackageId = PackageId Text
packagePreference : [PackageId] -> SubmitOptions
```
A `PackageId` can be hard-coded in your script, in which case it must be updated whenever the package changes. Otherwise, it can be provided using the `--input-file` flag of the `dpm script` command line tool.
The following example demonstrates reading the package ID from a DAR and passing it to a script:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Path to the dar you want to pass as package preference.
PACKAGE_DAR=path/to/main/dar.dar
# Path to the dar containing the Daml script for which you want to pass the package-id
SCRIPT_DAR=path/to/script/dar.dar
# Extract the package-id of PACKAGE_DAR's main package.
dpm damlc inspect-dar ${PACKAGE_DAR} --json | jq '.main_package_id' > ./package-id-script-input.json
# replace --ide-ledger with --ledger-host and --ledger-port for deployed Canton
dpm script --dar ${SCRIPT_DAR} --script-name Main:main --ide-ledger --input-file ./package-id-script-input.json
```
Following this, your script would look like:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
import Daml.Script
main : Text -> Script ()
main rawPkgId = do
let pkgId = PackageId rawPkgId
alice <- allocateParty "alice"
-- Commands omitted for brevity
let submitOptions = actAs alice <> packagePreference [pkgId]
submitOptions `submitWithOptions` createCmd ...
```
### Daml Studio support
Daml Studio runs a reference model of Canton called the IDE Ledger. This ledger has been updated to support the relevant parts of the Smart Contract Upgrades feature.
# Smart Contract Upgrading Reference
Source: https://docs.canton.network/appdev/deep-dives/smart-contract-upgrading-reference
Detailed rules for package validation on upload, and runtime upgrade and downgrade of contracts, choice arguments, and results.
This document describes in detail what rules govern package validation upon upload and how contracts, choice arguments and choice results are upgraded or downgraded at runtime. These topics are for a large part covered in Smart Contract Upgrade. This document acts as a thorough reference.
## Static Checks
Upgrade static checks are performed once alongside other validity checks when a DAR is uploaded to a participant. DARs deemed invalid for upgrades are rejected.
DAR upgrade checks are broken down into package-level checks, which are in turn broken down into module, template and data type-level checks.
### Packages
**Definition:** A *utility package* is a package with no template definition, no interface definition, no exception definition, and only non-[serializable](https://github.com/digital-asset/daml/blob/main/sdk/daml-lf/spec/daml-lf-2.rst#serializable-types) data type definitions. A utility package typically consists of helper functions and constants, but no type definitions.
A DAR is checked against previously uploaded DARs for upgrade validity on upload to a participant. Specifically, for every package with name *p* and version *v* present in the uploaded DAR:
1. The participant looks up versions *v\_prev* and *v\_next* of *p* in its package database, such that *v\_prev* is the greatest version of *p* smaller than *v*, and *v\_next* is the smallest version of *p* greater than *v*. Note that they may not always exist.
![image]()
2. The participant checks that version *v* of *p* is a valid upgrade of version *v\_prev* of p, if it exists.
3. The participant checks that version *v\_next* of *p* is a valid upgrade of version *v* of *p*, if it exists.
Therefore "being a valid upgrade" for a DAR is context dependent: it depends on what packages are already uploaded on the participant. It also *modular*: checks are performed at the package level. That is, a new version of a package is rejected as soon as it contains some element which doesn't properly upgrade its counterpart in the old package, even if some other elements do. Similarly, all packages in a DAR must be pass the check for the DAR to be accepted. If one of the packages fails the check, the entire DAR is rejected.
### Modules
The modules of the new version of a package must form a superset of the modules of the prior version of that package. In other words, it is valid to add new modules but deleting a module leads to a validation error.
**Examples**
In the file tree below, package v2 is a potentially valid upgrade of package v1, assuming `v2/A.daml` is a valid upgrade of `v1/A.daml`.
In the file tree below, package v2 cannot possibly be a valid upgrade of v1 because it doesn't define module `B`.
### Templates
The templates of the new version of a package must form a superset of the templates of the prior version of that package. In other words, it is valid to add new templates but deleting a template leads to a validation error.
**Examples**
Below, the module on the right is a valid upgrade of the module on the left. But the module on the left is **not** a valid upgrade of the module on the right because it lacks a definition for template `T2`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
template T1
with
p : Party
where
signatory p
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
template T1
with
p : Party
where
signatory p
template T2
with
p : Party
where
signatory p
```
### Template Parameters
The new version of a template may add new optional parameters at the end of the parameter sequence of the prior version of the template. The types of the parameters that the new template has in common with the prior template must be pairwise valid upgrades of the original types.
Deleting a parameter leads to a validation error.
Adding a parameter in the middle of the parameter sequence leads to a validation error.
As a special case of the two points above, renaming a parameter leads to a validation error.
Adding a non-optional parameter at the end of the parameter leads to a validation error.
**Examples**
Below, the template on the right is a valid upgrade of the template on the left. It adds an optional parameter `x1` at the end of the parameter sequence.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
where
signatory p
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
x1 : Optional Int
where
signatory p
```
Below, the template on the right is **not** a valid upgrade of the template on the left because it adds a new parameter `x1` before `p` instead of adding it at the end of the parameter sequence.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
where
signatory p
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
x1 : Optional Int
p : Party
where
signatory p
```
Below, the template on the right is **not** a valid upgrade of the template on the left because it drops parameter `x1`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
x1 : Int
where
signatory p
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
where
signatory p
```
Below, the template on the right is **not** a valid upgrade of the template on the left because it changes the type of `x1` from `Int` to `Text`. `Text` is not a valid upgrade of `Int`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
x1 : Int
where
signatory p
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
x1 : Text
where
signatory p
```
### Template Choices
The choices of the new version of a template must form a superset of the choices of the prior version of the template template. In other words, it is valid to add new choices but deleting a choice leads to a validation error.
**Examples**
Below, the template on the right is a valid upgrade of the template on the left. It adds a choice `C` to the previous version of the template. But the template on the left is **not** a valid upgrade of the template on the right as it deletes a choice.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
where
signatory p
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
where
signatory p
choice C : ()
controller p
do
return ()
```
### Template Choices - Parameters
As with template parameters, the new version of a choice may add new optional parameters at the end of the parameter sequence of the prior version of that choice. The types of the parameters that the new choice has in common with the prior choice must be pairwise valid upgrades of the original types.
Deleting a parameter leads to a validation error.
Adding a parameter in the middle of the parameter sequence leads to a validation error.
As a special case of the two points above, renaming a parameter leads to a validation error.
Adding a non-optional parameter at the end of the parameter sequence leads to a validation error.
**Example**
Below, the choice on the right is a valid upgrade of the choice on the left. It adds an optional parameter `x2` at the end of the parameter sequence.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
x1 : Int
controller p
do
return ()
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
x1 : Int
x2 : Optional Text
controller p
do
return ()
```
Below, the choice on the right is **not** a valid upgrade of the choice on the left because it adds a new parameter `x2` before `x1` instead of adding it at the end of the parameter sequence.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
x1 : Int
controller p
do
return ()
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
x2 : Optional Text
x1 : Int
controller p
do
return ()
```
Below, the choice on the right is **not** a valid upgrade of the choice on the left because it adds a new field `x2` before `x1` instead of adding it at the end of the parameter sequence.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
x1 : Int
controller p
do
return ()
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
x2 : Optional Text
x1 : Int
controller p
do
return ()
```
Below, the choice on the right is **not** a valid upgrade of the choice on the left because it drops parameter `x1`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
x1 : Int
controller p
do
return ()
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
controller p
do
return ()
```
Below, the choice on the right is **not** a valid upgrade of the choice on the left because it changes the type of `x1` from `Int` to `Text`. `Text` is not a valid upgrade of `Int`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
x1 : Int
controller p
do
return ()
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
with
controller p
do
return ()
```
### Template Choices - Return Type
The return type of the new version of a choice must be a valid upgrade of the return type of the prior version of that choice.
Changing the return type of a choice for a non-valid upgrade leads to a validation error.
**Examples**
Below, the choice on the right is **not** a valid upgrade of the choice on the left because it changes its return type from `()` to `Int`. `Int` is not a valid upgrade of `()`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : ()
controller p
do
return ()
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice C : Int
controller p
do
return 1
```
### Data Types
The serializable data types of the new version of a module must form a superset of the serializable data types of the prior version of that package. In other words, it is valid to add new data types but deleting a data type leads to a validation error.
Changing the variety of a serializable data type leads to a validation error. For instance, one cannot change a record type into a variant type.
Non-serializable data types are inexistent from the point of view of the upgrade validity check. Turning a non-serializable data type into a serializable one amounts to adding a new data type, which is valid. Turning a serializable data type into a non-serializable one amounts to deleting this data type, which is invalid.
**Examples**
Below, the module on the right is a valid upgrade of the module on the left. It defines an additional serializable data type `B`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data A = A
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data A = A
data B = B
```
Below, the module on the right is a valid upgrade of the module on the left. It turns the non-serializable type `A` into a serializable one. The non-serializable type is invisible to the upgrade validity check so this amounts to adding a new data type to the module on the right.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data A = A
with
x : Int -> Int
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data A = A
with
```
Below, the module on the right is **not** a valid upgrade of the module on the left because it changes the variety of `A` from record type to variant type.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data A = A
with
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data A = A | B
```
Below, the module on the right is **not** a valid upgrade of the module on the left because it drops the serializable data type `A`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data A = A
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
```
Below, the module on the right is **not** a valid upgrade of the module on the left because although it adds an optional field to the record type `A`, it also turns `A` into a non-serializable type, which amounts to deleting `A` from the point of view of the upgrade validity check.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data A = A
with
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data A = A
with
x : Optional (Int -> Int)
```
### Data Types - Records
The new version of a record may add new optional fields at the end of the field sequence of the prior version of that record. The types of the fields that the new record has in common with the prior record must be pairwise valid upgrades of the original types.
Deleting a field leads to a validation error.
Adding a field in the middle of the field sequence leads to a validation error.
As a special case of the two points above, renaming a field leads to a validation error.
Adding a non-optional field at the end of the field sequence leads to a validation error.
**Examples**
Below, the record on the right is a valid upgrade of the module on the left. It adds an optional field `x2` at the end of the field sequence.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x1 : Int
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x1 : Int
x2 : Optional Text
```
Below, the record on the right is **not** a valid upgrade of the record on the left because it adds a new field `x2` before `x1` instead of adding it at the end of the field sequence.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x1 : Int
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x2 : Optional Text
x1 : Int
```
Below, the record on the right is **not** a valid upgrade of the record on the left because it drops field `x2`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x1 : Int
x2 : Text
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x1 : Int
```
Below, the record on the right is **not** a valid upgrade of the record on the left because it changes the type of `x1` from `Int` to `Text`. `Text` is not a valid upgrade of `Int`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x1 : Int
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x1 : Text
```
### Data Types - Variants
The new version of a variant may add new constructors at the end of the constructor sequence of the old version of that variant. The argument types of the constructors that the new variant has in common with the prior variant must be pairwise valid upgrades of the original types. This last rule also applies to constructors whose arguments are unnamed records, in which case the rules about record upgrade apply.
Adding an argument to a constructor without arguments leads to a validation error. In particular, adding an optional field to a constructor that previously had no arguments is not allowed.
Adding a constructor in the middle of the constructor sequence leads to a validation error.
Changing the order or the name of the constructor sequence leads to a validation error.
Removing a constructor leads to a validation error.
Enums cannot get upgraded to variants: adding a constructor with an argument at the end of the constructor sequence of an enum leads to a validation error.
**Examples**
Below, the variant on the right is a valid upgrade of the variant on the left. It adds a new constructor `C` at the end of the constructor sequence.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | B Text
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | B Text | C Bool
```
Below, the variant on the right is a valid upgrade of the variant on the left. It adds a new optional field to constructor `B`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A | B { x : Int }
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A | B { x : Int, y : Optional Text }
```
Below, the variant on the right is **not** a valid upgrade of the variant on the left because it adds a new constructor `C` before `B` instead of adding it at the end of the constructor sequence.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | B Text
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | C Bool | B Text
```
Below, the variant on the right is **not** a valid upgrade of the variant on the left because it changes the order of its constructors.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | B Text
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
B Text | A Int
```
Below, the variant on the right is **not** a valid upgrade of the variant on the left because it drops constructor `B`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | B Text
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int
```
Below, the variant on the right is **not** a valid upgrade of the variant on the left because it changes the type of `B`'s argument from `Text` to `Bool`. `Bool` is not a valid upgrade of `Text`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | B Text
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | B Bool
```
Below, the variant on the right is **not** a valid upgrade of the variant on the left because it adds an argument to constructor `B` which didn't have one before.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | B
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A Int | B { x : Optional Text }
```
Below, the variant on the right is **not** a valid upgrade of the enum on the left. Enums cannot get upgraded to variants and `T` as defined on the left is an enum because none of its constructors have arguments.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A | B
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T =
A | B | C Int
```
### Data Types - Enums
For the purpose of upgrade validation, enums can be treated as a special case of variants. The rules of [the section on variants](#data-types-variants) apply, only without constructor arguments.
### Data Types - Type References
A type reference is an identifier that resolves to a type. For instance, consider the following module definitions, from two different packages:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- In package q
module Dep where
data U = U with x : Int
type A = U
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- In package p
module M where
import qualified Dep
data T = T with x : Dep.A
```
In the definition of `T`, `Dep.A` is a type reference that resolves to the type with qualified name `Dep.U` in package `q`.
A reference *r2* to a data type upgrades a reference *r1* to a data type if and only if:
* *r2* resolves to a type *t2* with qualified name *q2* in package *p2;*
* *r1* resolves to a type *t1* with qualified name *q1* in package *p1;*
* The qualified names *q2* and *q1* are the same;
* Package *p2* is a valid upgrade of package *p1*.
It is worth noting that even when *t2* upgrades *t1*, *r2* only upgrades *r1* provided that package *p2* is a valid upgrade of package *p1* as a whole.
**Examples**
In these examples we assume the existence of packages `q-1.0.0` and `q-2.0.0` and that the latter is a valid upgrade of the former.
In q-1.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Dep where
data U = C1
data V = V
```
In q-2.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Dep where
data U = C1 | C2
data V = V
```
Then below, the module on the right is a valid upgrade of the module on the left.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
-- imported from q-1.0.0
import qualified Dep
data T = T Dep.U
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
-- imported from q-2.0.0
import qualified Dep
data T = T Dep.U
```
However below, the module on the right is **not** a valid upgrade of the module on the left because `Dep.V` on the right belongs to package `q-1.0.0` which is not a valid upgrade of package `p-2.0.0`, even though the two definitions of `V` are the same.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
-- imported from q-2.0.0
import qualified Dep
data T = T Dep.V
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
-- imported from q-1.0.0
import qualified Dep
data T = T Dep.V
```
### Data Types - Builtin Types
Builtin scalar types like `Int`, `Text`, `Party`, etc. only upgrade themselves. In other words, it is never valid to replace them with another type.
### Data Types - Parameterized Data Types
The upgrade validation for parameterized data types follows the same rules as non-parameterized data types, but also compares type variables. Type variables may be renamed.
**Example**
Below, the parameterized data type on the right is a valid upgrade of the parameterized data type on the left. As is valid with any record type, it adds an optional field.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Tree a =
Tree with
label : a
children : [Tree a]
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Tree b =
Tree with
label : b
children : [Tree b]
cachedSize : Optional Int
```
### Data Types - Applied Parameterized Data Types
A type constructor application `T' U1' .. Un'` upgrades `T U1 .. Un` if and only if `T'` upgrades `T` and each `Ui'` upgrades the corresponding `Ui`.
**Examples**
Below, the module on the right is a valid upgrade of the module on the left. The record type `T` on the right upgrades the record type `T` on the left. As a result, the type constructor application `List T` on the right upgrades the type constructor application `List T` on the left. Same goes for `List` and `Optional`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data T = T {}
data Demo = Demo with
field1 : List T
field2 : Map T T
field3 : Optional T
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data T = T { i : Optional Int }
data Demo = Demo with
field1 : List T
field2 : Map T T
field3 : Optional T
```
Below, the module on the right is a valid upgrade of the module on the left. The parameterized type `C` on the right upgrades the parameterized type `C` on the left. As a result, the type constructor application `C Int` on the right upgrades the type constructor application `C Int` on the left.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data C a = C { x : a }
data Demo = Demo with
field1 : C T
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data C a = C { x : a, y : Optional Int }
data Demo = Demo with
field1 : C T
```
### Interface and Exception Definitions
Neither interface definitions nor exception definitions can be upgraded. We strongly discourage uploading a package that defines interfaces or exceptions alongside templates, as these templates cannot benefit from smart contract upgrade in the future. Instead, we recommend declaring interfaces and exceptions in a package of their own that defines no template.
### Interface Instances
Interface instances may be upgraded. Note however that the type signature of their methods and view cannot change between two versions of an instance since they are fixed by the interface definition, which is non-upgradable. Hence, the only thing that can change between two versions of an instance is the bodies of its methods and view.
Interface instances may be added to a template.
Deleting an interface instance from a template leads to a validation error.
**Examples**
Assume an interface `I` with view type `IView` and a method `m`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data IView = IView { i : Int }
interface I where
viewtype IView
```
Then, below, the instance of `I` for template `T` on the right is a valid upgrade of the instance on the left. It changes the `view` expression and the body of method `m`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
i : Int
where
signatory p
interface instance I for T where
view = IView i
m = i
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
i : Int
j : Optional Int
where
signatory p
interface instance I for T where
view = IView (fromOptional i j)
m = fromOptional i j
```
Below, the template on the right is a valid upgrade of the template on the left. It adds a new instance of `I` for template `T3`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T3
with
p : Party
i : Int
where
signatory p
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T3
with
p : Party
i : Int
where
signatory p
interface instance I for T3 where
view = IView i
m = i
```
Below, the template on the right is **not** a valid upgrade of the template on the left because it removes the instance of `I` for template `T2`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T2
with
p : Party
i : Int
where
signatory p
interface instance I for T2 where
view = IView i
m = i
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T2
with
p : Party
i : Int
where
signatory p
```
## Data Transformation: Runtime Semantics
A template version is selected whenever a contract is fetched, a choice is exercised, or an interface value is converted to a template value, according to a set of rules detailed below. We call this template the target template.
The contract is then transformed into a value that fits the type of the target template. Then, its metadata (signatories, stakeholders) is recomputed using the code of the target template and compared against the existing metadata stored on the ledger: it is not allowed to change. The ensure clause of the contract is also re-evaluated: it must evaluate to `True`.
In addition, when a choice is exercised, its arguments are transformed into values that fit the type signature of the choice in the target template. The result of the exercise is then possibly transformed back to some other target type by the client (e.g. the generated java client code).
Below, we detail the rules governing target template selection, then explain how transformations are performed, and finally detail the rules of metadata re-computation.
### Static Target Template Selection
In a non-interface fetch or exercise triggered by the body of a choice, the target template is determined by the dependencies of the package that defines the choice. In other words, it is statically known.
Interface fetches and exercises, on the other hand, are subject to dynamic target template selection, as detailed in the next section. However, operations acting on interface *values* — as opposed to IDs — are static. Their mode of operation is detailed below.
Daml contracts are represented by one of two sorts of values at runtime: template values or interface values.
* Template values are those whose concrete template type is statically known. They are obtained by directly constructing a template record, or by a call to `fetch`. Their runtime representation is a simple record.
* Interface values are those whose concrete template type is not fully statically known, aside from the fact that it implements a given interface. They are obtained by applying `toInterface` to a template value. At runtime, they are represented by a pair consisting of:
> * a record: the contract;
> * a template type: the runtime type of that record.
For instance, if `c` is a contract of type `T` and `T` implements the interface `I`, then `toInterface c` evaluates to the pair `(c, T)`.
Note that the type of interface values is opaque: while it is useful to conceptualize interface values as pairs for defining the runtime semantics of the language, their actual implementation may vary and is not exposed to the user.
Let us assume an interface value `iv` = `(c, T)`. Then `fromInterface @U iv` evaluates as follow.
> * If `U` upgrades `T`, then it evaluates to `Some c'` where `c'` is the result of transforming `c` into a value of type `U`.
> * Otherwise, it evaluates to `None`.
Let us assume an interface value `iv` = `(c, T)` and an interface type `I`. Then `create @I iv` evaluates as follow.
> * If `T` does not implement `I` then an error is thrown.
> * Otherwise `create @T c` is evaluated.
**Example 1**
Assume two versions of a package called dep, defining a template U and its upgrade.
In dep-1.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Dep where
template U
with
p : Party
where
signatory p
```
In dep-2.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Dep where
template U
with
p : Party
t : Optional Text
where
signatory p
```
Assume then some package `q` which depends on version `1.0.0` of `dep`.
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
[...]
name: q
version: 1.0.0
data-dependencies:
- dep-1.0.0.dar
```
Package `q` defines a template `S` with a choice that fetches a contract of type `U`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
import qualified Dep
template S
with
p : Party
where
signatory p
choice GetU : Dep.U
with
cid : ContractId Dep.U
where
controller p
do fetch cid
```
Finally assume a ledger that contains a contract of type `S` written by `q` and a contract of type `U` written by `dep-2.0.0`.
| Contract ID | Type | Contract |
| ----------- | ------------- | --------------------------- |
| `4321` | `q:T` | `T { p = 'Alice' }` |
| `8765` | `dep-2.0.0:U` | `U { p = 'Bob', t = None }` |
When exercising choice `GetU 8765` on contract `4321` with package preference `dep-2.0.0`, we trigger a fetch of contract `5678`. Because package `q` depends on version `1.0.0` of `dep`, the target type for `U` is the one defined in package `dep-1.0.0`. Contract `5678` is thus downgraded to `U { p = 'Bob'}` upon retrieval. Note that the command preference for version `2.0.0` of package `dep` bears no incidence here.
**Example 2**
Assume an interface `I` with view type `IView` and a method `m`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data IView = IView {}
interface I where
viewtype IView
```
Assume then two versions of a template `T` that implements `I`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
where
signatory p
interface instance I for T where
view = IView {}
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
i : Optional Int
where
signatory p
interface instance I for T where
view = IView {}
```
Finally, assume that the module defining the first version of `T` is imported as `V1`, and the module defining the second version of `T` is imported as `V2`. The expression `fromInterface @V2.T (toInterface @I (V1.T 'Alice'))` evaluates as follows:
> * `toInterface @I (@V1.T alice)` evaluates to the interface value `(V1.T { p = 'Alice' }, V1.T)`.
> * The type `V2.T` upgrades `V1.T` so `fromInterface` proceeds to transform `(V1.T { p = 'Alice' })` into a value of type `V2.T`
> * The entire expression thus evaluates to `V2.T { p = 'Alice', i = None }`.
### Dynamic Target Template Selection
In a top-level exercise triggered by a Ledger API command, or in an interface fetch or exercise triggered from the body of a choice, the rules of package preference detailed in dynamic package resolution determine the target template at runtime.
**Example 1**
Assume a package `p` with two versions. The new version adds an optional text field.
In p-1.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
where
signatory p
```
In p-2.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
t : Optional Text
where
signatory p
```
Also assume a ledger that contains a contract of type `T` written by `p-1.0.0`, and another contract of written by `p-2.0.0`.
| Contract ID | Type | Contract |
| ----------- | ----------- | ----------------------------------- |
| `1234` | `p-1.0.0:T` | `T { p = 'Alice' }` |
| `5678` | `p-2.0.0:T` | `T { p = 'Bob', t = Some "Hello" }` |
Then
* Fetching contract `1234` with package preference `p-1.0.0` retrieves the contract and leaves it unchanged, returning `T { p = 'Alice' }`.
* Fetching contract `1234` with package preference `p-2.0.0` retrieves the contract and successfully transforms it to the target template type, returning `T { p = 'Alice', t = None }`.
* Fetching contract `5678` with package preference `p-1.0.0` retrieves the contract and fails to downgrade it to the target template type, returning an error.
* Fetching contract `5678` with package preference `p-2.0.0` retrieves the contract and leaves it unchanged, returning `T { p = 'Bob', t = Some "Hello" }`.
**Example 2**
Assume an interface `I` with a choice `GetInt`
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data IView = IView {}
interface I where
viewtype IView
getInt : Int
choice GetInt : Int
with
p : Party
controller p
do
pure (getInt this)
```
Now, assume two versions of a package called `inst`, defining a template `Inst` and its upgrade. The two versions of the template instantiate interface `I`, but their `getInt` method return different values.
In inst-1.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Inst
with
p : Party
where
signatory p
interface instance I for T where
view = IView
getInt = 1
```
In inst-2.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Inst
with
p : Party
where
signatory p
interface instance I for T where
view = IView
getInt = 2
```
Assume then some package `client` which defines a template whose choice `Go` exercises choice `GetInt` by interface.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Client
with
p : Party
icid : ContractId I
where
signatory p
choice Go : Int
controller p
do
exercise icid (GetInt p)
```
Finally assume a ledger that contains a contract of type `Inst` written by `inst-1.0.0`, and a contract of type `Client` written by `client`.
| Contract ID | Type | Contract |
| ----------- | ----------------- | ------------------------ |
| `0123` | `inst-1.0.0:Inst` | `Inst { p = 'Alice' }` |
| `0456` | `client:Client` | `Client { p = 'Alice' }` |
Then:
* When exercising choice `Go` on contract `0456` with package preference `inst-1.0.0`, we trigger an exercise by interface of contract `0123`. Because `inst-1.0.0` is prefered, contract `0123` is upgraded to a value of type `inst-1.0.0::Inst` and its `getInt` method is executed. The result of the exercise is thus the value `1`.
* When exercising choice `Go` on contract `0456` but with package preference `inst-2.0.0` this time, `inst-2.0.0:Inst` is picked as the target template for `0123` and thus the exercise returns the value `2`. Note that the fact that the exercise stored on the ledger is of type `inst-1.0.0:Inst` bears no incidence on the `getInt` method that is eventually executed.
**Example 3**
Assume now a package `r` with two versions. They define a template with a choice, and version `2.0.0` adds an optional field to the parameters of the choice. The return type of the choice is also upgraded.
In r-1.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data Ret = Ret with
template V
with
p : Party
where
signatory p
choice C : Ret
with
i : Int
controller p
do return Ret
```
In r-2.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
module M where
data Ret = Ret with
j : Optional Int
template V
with
p : Party
where
signatory p
choice C : Ret
with
i : Int
j : Optional Int
controller p
do return Ret with j = j
```
Also assume a ledger that contains a contract of type `V` written by `r-1.0.0`.
| Contract ID | Type | Contract |
| ----------- | ----------- | ------------------- |
| `9101` | `r-1.0.0:V` | `V { p = 'Alice' }` |
Then:
* Exercising `C with i=1` on contract `9101` with package preference `r-2.0.0` will execute the code of `C` as defined in `r-2.0.0`. The parameter sequence `i=1` is thus transformed into the parameter sequence `i=1, j=None` to match its parameter types. The exercise then returns the value `Ret with j=None`. It is up to the client code (e.g. the caller of the ledger API) to transform this to a value that fits the return type it expects. For instance, a client which only knows about version `1.0.0` of package `r` would expect a value of type `Ret` and would thus transform the value `Ret with j=None` back to `Ret`.
* Exercising `C with i=1` on contract `9101` with package preference `r-1.0.0` will execute the code of `C` as defined in `r-1.0.0`. The parameter sequence requires therefore no transformation. The exercise returns the value `Ret`.
* Exercising `C with i=1 j=Some 2` on contract `9101` with package preference `r-2.0.0` will execute the code of `C` as defined in `r-2.0.0`. Again, the parameter sequence no transformation. The exercise returns the value `Ret with j=Some 2`.
* Exercising `C with i=1 j=Some 2` on contract `9101` with package preference `r-1.0.0` will fail with a runtime error as the parameter sequence `i=1 j=Some 2` cannot be downgraded to the parameter sequence of `C` as defined in `r-1.0.0`.
### Transformation Rules
Once the target type has been determined, the data transformation rules themselves follow the [upgrading rules of protocol buffers](https://protobuf.dev/programming-guides/proto3/#updating).
#### Records and Parameters
Given a record type and its upgrade, referred to respectively as `T-v1` and `T-v2` in the following,
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x1 : T1
...
xn : Tn
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data T = T with
x1 : T1'
...
xn : Tn'
y1 : Optional U1
...
ym : Optional Um
```
* A `T-v1` value `T { x1 = v1, ..., xn = vn }` is upgraded to a `T-v2` value by setting the additional fields to None and upgrading `v1...vn` recursively. The transformation results in a value `T { x1 = v1', ..., xn = vn', y1 = None, ..., ym = None }`, where `v1'... vn'` is the result of upgrading `v1...vn` to `T1' ... Tn'`.
* A `T-v2` value of the shape `T { x1 = v1, ..., xn = vn, y1 = None, ..., ym = None }` is downgraded to a `T-v1` value by dropping additional fields and downgrading `v1...vn` recursively. The transformation results in a value `T { x1 = v1', ..., xn = vn' }` where `v1'... vn'` is the result of downgrading `v1 ... vn` to `T1 ... Tn`.
* Attempting to downgrade a `T-v2` value where at least one `yi` is a `Some _` results in a runtime error.
The same transformation rules apply to template parameters and choice parameters.
#### Variants and Enums
Given a variant type and its upgrade, referred to respectively as `V-v1` and `V-v2` in the following,
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data V =
= C1 T1
| ...
| Cn Tn
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
data V =
= C1 T1'
| ...
| Cn Tn'
| D1 U1
| ...
| Dm Um
```
* A `V-v1` value `Ci vi` is upgraded to a `V-v2` value by upgrading `vi` recursively. The transformation results in a value `Ci vi'` where `vi'` is the result of upgrading `vi` to `Ti'`.
* A `V-v2` value `Ci vi` is downgraded to a `V-v1` value by downgrading `vi` recursively. The transformation results in a value `Ci vi'` where `vi'` is the result of downgrading `vi` to `Ti`.
* Attempting to downgrade a `V-v2` value of the form `Dj vj` results in a runtime error.
The same transformation rules apply to enum types, constructor arguments aside.
#### Other Types
Types that aren't records or variants are "pass-through" for the upgrade and downgrade transformations:
* Values of scalar types are trivially transformed to themselves.
* The payload of an Optional is recursively transformed.
* The elements of Lists are recursively transformed.
* The keys and values of Maps are recursively transformed.
### Metadata
For a given contract, metadata is every information outside of the contract parameters that is stored on the ledger for this contract. Namely:
* The contract signatories;
* The contract stakeholders (the union of signatories and observers);
The metadata of two contracts are equivalent if and only if:
* their signatories are equal;
* their stakeholders are equal;
Upon retrieval and after conversion, the metadata of a contract is recomputed using the code of the target template. It is a runtime error if the recomputed metadata is not equivalent to that of the original contract.
**Note:** A given implementation may choose to perform the equivalence check differently from what is described above, as long as the result is semantically equivalent.
**Example**
Below the template on the right is a valid upgrade of the template on the left.
In p-1.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
sig : Party
where
signatory sig
```
In p-2.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
sig : Party
additionalSig : Optional Party
where
signatory sig, fromOptional [] additionalSig
```
Assume a ledger that contains a contract of type `T` written by `p-1.0.0`.
| Contract ID | Type | Contract |
| ----------- | ----------- | ----------------------- |
| `1234` | `p-1.0.0:T` | `T { sig = ['Alice'] }` |
Fetching contract `1234` with target type `p-2.0.0:T` retrieves the contract and successfully transforms it into a value of type `p-2.0.0:T`: `T { sig = 'Alice', additionalSig = None }`. The signatories of this transformed contract are then computed using the expression `sig, fromOptional [] additionalSig`, which evaluate to the list `['Alice']`. This list is then compared to signatories of the original contract stored on the ledger: `['Alice']`. They match and thus the upgrade is valid.
On the other hand, below, the template on the right is **not** a valid upgrade of the template on the left.
In p-1.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
sig : Party
where
signatory sig
```
In p-2.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
sig : Party
where
signatory sig, sig
```
Assume the same ledger as above. Fetching contract `1234` with target type `p-2.0.0:T` retrieves the contract and again successfully transforms it into the value `T { sig = 'Alice', additionalSig = None }`. The signatories of this transformed contract are then computed using the expression `sig, sig`, which evaluate to the list `['Alice', 'Alice']`. This list is then compared to signatories of the original contract stored on the ledger: `['Alice']`. They do not match and thus the upgrade is rejected at runtime.
### Ensure Clause
Upon retrieval and after conversion, the ensure clause of a contract is recomputed using the code of the target template. It is a runtime error if the recomputed ensure clause evaluates to `False`.
**Examples**
Below, the template on the right is **not** a valid upgrade of the template on the left because its ensure clause will evaluate to `False` for contracts that have been written using the template on the left with `n = 0`.
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
sig : Party
n : Int
where
signatory sig
ensure n >= 0
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
sig : Party
n : Int
where
signatory sig
ensure n > 0
```
### Interface Views
The view for a given interface instance may change between two versions of a contract. When a contract is fetched or exercised by interface, its view is recopmuted according to the code of the target template.
**Example**
Assume an interface `I` with view type `IView` and a method `m`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data IView = IView { i : Int }
interface I where
viewtype IView
m : Int
```
Below, the template on the right is a valid upgrade of the template on the left.
In p-1.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
i : Int
where
signatory p
interface instance I for T where
view = IView i
m = i
```
In p-2.0.0:
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
i : Int
where
signatory p
interface instance I for T where
view = IView (i+1)
m = i
```
Assume a ledger that contains a contract of type `T` written by `p-1.0.0`.
| Contract ID | Type | Contract |
| ----------- | ----------- | --------------------------- |
| `1234` | `p-1.0.0:T` | `T { p = 'Alice', i = 42 }` |
Fetching contract `1234` by interface with package preference `p-2.0.0` retrieves the contract and transforms it into a value of type `p-2.0.0:T`: `T { sig = 'Alice', i = 42, j = None }`. Then its view is computed time according to `p-2.0.0`: `IView 43`.
# Token Standard
Source: https://docs.canton.network/appdev/deep-dives/token-standard
The Canton Network Token Standard APIs and Daml interfaces
## Overview
* See the [text of the CIP-0056](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md) for an overview of the APIs that are part of the Canton Network Token Standard.
* See the [README in its source-code](https://github.com/canton-network/splice/tree/main/token-standard#readme) for background on how to use the APIs.
## Holding UTXO Management
Analogous to Bitcoin, Canton uses a UTXO model, where the UTXOs are all the active contracts that implement the `Holding` interface.
Active `Holding` contracts incur both storage and compute cost on the validator nodes hosting users that own `Holding` contracts and the validator nodes hosting the token administrator. To make efficient use of the network's resources, we thus recommend that wallet providers aim to keep the number of UTXOs per user low, i.e, below \~10 UTXOs per user on average.
This also optimizes traffic costs, as each UTXO input to a transfer costs extra traffic. Furthermore, this recommendation aligns with the incentives put in place by tokens like Canton Coin, which:
* allows at most 100 input contracts for a single transfer, and thus discourages excessive splitting of holdings
* expires coin UTXOs whose initial amount is lower than the accrued holding fee, and thus discourages the creation of dust UTXOs
We recommend wallet providers to implement a UTXO management strategy that:
* prefers the selection of `Holding` UTXOs with small amounts to provide the input holdings to fund a transfer
* asks the user to setup a `MergeDelegation` contract (see docs) as part of wallet onboarding, which enables the wallet provider to automatically merge small `Holding` UTXOs on behalf of the user
`MergeDelegation` contracts also allow wallet providers to run airdrop campaigns jointly with the auto merging of holdings using a single, batched call to airdrop and merge holdings for multiple users and multiple instruments. Furthermore, featured wallet providers earn featured app rewards for performing merges for their users.
### Setting up MergeDelegations
Assuming you are a wallet provider that runs a validator node for your users, you can set up `MergeDelegation` contracts for your users as follows.
1. Extract the latest version of the `splice-util-token-standard-wallet.dar` file from the release bundle (Download Bundle (DevNet 0.6.12)).
2. Upload the extracted `.dar` file to your validator node.
3. Adjust your user onboarding procedure such that the users signs the creation of a `MergeDelegationProposal` contract (see docs).
4. Accept the `MergeDelegationProposal` contracts by exercising their `Accept` choice using your wallet provider's party.
Assuming you are a wallet provider that runs a validator node for your users, you can set up `MergeDelegation` contracts for your users as follows.
1. Extract the latest version of the `splice-util-token-standard-wallet.dar` file from the release bundle (Download Bundle (TestNet 0.6.11)).
2. Upload the extracted `.dar` file to your validator node.
3. Adjust your user onboarding procedure such that the users signs the creation of a `MergeDelegationProposal` contract (see docs).
4. Accept the `MergeDelegationProposal` contracts by exercising their `Accept` choice using your wallet provider's party.
Assuming you are a wallet provider that runs a validator node for your users, you can set up `MergeDelegation` contracts for your users as follows.
1. Extract the latest version of the `splice-util-token-standard-wallet.dar` file from the release bundle (Download Bundle (MainNet 0.6.10)).
2. Upload the extracted `.dar` file to your validator node.
3. Adjust your user onboarding procedure such that the users signs the creation of a `MergeDelegationProposal` contract (see docs).
4. Accept the `MergeDelegationProposal` contracts by exercising their `Accept` choice using your wallet provider's party.
### Using MergeDelegations
We recommend to use the `MergeDelegation` contracts in a batched fashion as follows.
1. Create a single `BatchMergeUtility` contract (see docs) for your wallet provider's party as part of your validator node's setup.
2. Grant your wallet provider's user the `CanReadAsAnyParty` right on your validator node to allow it to read all users' `Holding` UTXOs.
3. Run a background process that regularly performs the following steps:
> 1. Determines all users that have more than 10 `Holding` UTXOs. For example, using the DB provided by the [Participant Query Store](/appdev/deep-dives/query-with-pqs); or by reading the Holding contracts directly from the Ledger API of your validator node. The former being the more scalable option.
>
> 2. Constructs the transfer choices to merge the extra `Holding` contracts by querying the registry API as explained in `token_standard_usage_executing_factory_choice`.
>
> 3. Lookup the `MergeDelegation` contract for each user and construct the corresponding call to exercise the `MergeDelegation_Merge` choice.
>
> 4. Assemble batches of \~100 merge delegation choices into a single call to the `BatchMergeUtility_MergeHoldings` choice.
>
> 5. Lookup the contract-id of the `BatchMergeUtility` contract that you setup above.
>
> 6. Execute the batched merge by exercising the `BatchMergeUtility_MergeHoldings` choice using your wallet provider user on your validator node. Use the Ledger API to exercise the choice and make sure that you add all disclosed contracts obtained from the previous steps to the single call.
>
> Note that for you can execute multiple batches in parallel for higher throughput.
Optionally, you can add transfers from your operator party to the merge calls to implement airdrop campaigns in a batched fashion.
### Upgrading from custom MergeDelegation implementations
Some wallet providers already implement their own custom merge delegation contracts. They can continue to use them alongside the `MergeDelegation` contracts provided by Splice. There is no requirement to upgrade to the Splice-provided contracts.
However, if you would like to upgrade to the Splice-provided contracts (e.g., to benefit from the additional features), then you can do so as follows.
1. Add a `CustomMergeDelegation_Upgrade` choice to your `CustomMergeDelegation` template that creates a `MergeDelegation` contract for the user. Make the choice `consuming`, so that the old `CustomMergeDelegation` contract is archived as part of exercising the upgrade choice.
2. Bump the version of your custom merge delegation `.dar` file and build a new release.
3. Upload the new `custom-merge-delegation.dar` file to your validator node.
4. Call the `CustomMergeDelegation_Upgrade` choice on all existing `CustomMergeDelegation` contracts to upgrade them to the Splice-provided `MergeDelegation` contracts
## Wallet integration with Token Standard Assets
This section provides wallet developers with guidance on how to integrate with token standard assets. Such an integration works by sending the right read and write requests to the Ledger API of the validator node hosting the wallet user's party. There are five kinds of integration patterns:
> * `token_standard_usage_reading_contracts`
> * `token_standard_usage_reading_tx_history`
> * `token_standard_usage_executing_factory_choice`
> * `token_standard_usage_executing_nonfactory_choice`
> * `token_standard_usage_custom_daml_code`
These integrations patterns have recently been nicely packaged in the [Wallet SDK](https://github.com/canton-network/wallet-gateway/tree/main/sdk/wallet-sdk) maintained in the `canton-network/wallet-gateway` repository and [documented here](/integrations/wallet/guidance).
All of these integration patterns are also demonstrated in the form of executable code as part of the [experimental command-line interface](https://github.com/canton-network/splice/tree/main/token-standard#cli) for token standard assets. The sections below explaining the patterns below thus all start with a link to the code. They then provide additional context for an implementor.
All interaction works via the JSON Ledger API (see its [OpenAPI definition here](/sdks-tools/api-reference/json-api)). This OpenAPI definition is also accessible at `http(s)://${YOUR_PARTICIPANT}/docs/openapi`. We encourage developers to use OpenAPI code generation tools as opposed to manually writing HTTP requests.
Check out the [Authentication docs](/global-synchronizer/reference/security-configuration#configure-api-authentication-and-authorization-with-jwt) for more information on how to authenticate the requests.
### Reading contracts implementing a Token Standard interface for a party
Reference code from the Token Standard CLI to [list contracts by interface](https://github.com/canton-network/splice/blob/main/token-standard/cli/src/commands/listContractsByInterface.ts)
The Token Standard includes several interfaces that are implemented by Daml templates. To list all contracts implementing a particular interface, you have to query the participant's [active-contracts endpoint](/sdks-tools/api-reference/json-api).
The `activeAtOffset` parameter can be set to the result of [the ledger-end endpoint on the participant](/sdks-tools/api-reference/json-api#ledger-end) to get the latest ACS, or an older (non-pruned) one to get the ACS at that point in time.
To filter for a particular party and interface, it should include a `filtersByParty` with an `InterfaceFilter`:
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"filtersByParty": {
"$A_PARTY": {
"cumulative": [
{
"identifierFilter": {
"InterfaceFilter": {
"value": {
"interfaceId": "$AN_INTERFACE_ID",
"includeInterfaceView": true,
"includeCreatedEventBlob": true
}
}
}
}
]
}
}
}
```
For example:
* `"$A_PARTY"` could look like `test::1220a0db3761b3fc919b55e7ff80ad740824336010bfde8829611c0e64477ab7bee5`.
* `"$AN_INTERFACE_ID"` could be `#splice-api-token-holding-v1:Splice.Api.Token.HoldingV1:Holding`.
Additionally, there's three flags that can be set:
* `includeInterfaceView`: to include the interface view of the contract in the response.
* `includeCreatedEventBlob`: to include a binary blob that is required for [explicit disclosure](/appdev/deep-dives/explicit-contract-disclosure).
* `verbose`: to include additional information in the response.
The response for such a query will contain the `createdEvent` of the contract, including the interface views requested (if any). The `viewValue` within it will be the JSON-serialized Daml interface view. If more than one interface is requested, you can distinguish them by checking the `interfaceId` field. You can find an [example response for Holdings here](https://github.com/canton-network/splice/blob/main/token-standard/cli/__tests__/mocks/data/holdings.json).
### Reading and parsing transaction history involving Token Standard contracts
Example code: [Token Standard CLI's code to list transactions](https://github.com/canton-network/splice/blob/main/token-standard/cli/src/commands/listHoldingTransactions.ts)
The participant has an [endpoint to list all transactions](/sdks-tools/api-reference/json-api) involving the provided parties and interfaces.
To filter for a particular party and interface, it should include a `filtersByParty` with an `InterfaceFilter`:
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"filtersByParty": {
"$A_PARTY": {
"cumulative": [
{
"identifierFilter": {
"InterfaceFilter": {
"value": {
"interfaceId": "$AN_INTERFACE_ID",
"includeInterfaceView": true,
"includeCreatedEventBlob": true
}
}
}
}
]
}
}
}
```
For example:
* `"$A_PARTY"` could look like `test::1220a0db3761b3fc919b55e7ff80ad740824336010bfde8829611c0e64477ab7bee5`.
* `"$AN_INTERFACE_ID"` could be `#splice-api-token-holding-v1:Splice.Api.Token.HoldingV1:Holding` to read all `Holding` contracts of the specified party.
To include other transaction nodes that don't directly involve the interfaces (e.g., non-interface-specific children nodes), a `WildcardFilter` can be included in the `cumulative` filter array:
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"identifierFilter": {
"WildcardFilter": {
"value": {
"includeCreatedEventBlob": true
}
}
}
}
```
The `beginExclusive` field is the offset from which to start reading transactions. To paginate, you can start with the `participantPrunedUpToInclusive` from `GET ${PARTICIPANT_URL}/v2/state/latest-pruned-offsets` and continue by passing the offset of the last transaction from the previous response.
#### Parsing the history
Example code: [the parser here](https://github.com/canton-network/splice/blob/main/token-standard/cli/src/txparse/parser.ts). It extracts a user-readable wallet history by parsing transactions involving the `Holding` and `TransferInstruction` interfaces.
The endpoint returns transaction trees as an array. The transactions are ordered as they occur in the ledger. Given an `ExercisedEvent` with `nodeId=X` and `lastDescendantNodeId=Y`, the children of that node are those with `nodeId` in the range `[X+1, Y]`. `CreatedEvent` and `ArchivedEvent` (or equivalently, `ExercisedEvent` where `consuming=true`) do not have children.
Given the above, a tree-like traversal can be performed on the transaction nodes. Generally, a Token Standard parser will focus on the exercise of Token Standard choices and creation of contracts implementing Token Standard interfaces. Where further customization is required, a parser can decide to also focus on internal/specific choices that are not available in the standard, but in some specific implementation.
In each Token Standard exercise node, one can find:
* The choice being executed, useful to distinguish what operation was performed.
* As part of the archival/creation of children, one can find out other relevant operations that happened. For example, creation or archival of `Holdings`.
* Meta key/values, of which part of the standard:
* `splice.lfdecentralizedtrust.org/tx-kind`: the kind of operation happening in the node. This can give more information than the exercised choice does. It can be one of:
* `transfer`
* `merge-split`
* `burn`
* `mint`
* `unlock`
* `expire-dust`
* `splice.lfdecentralizedtrust.org/sender`: which party is the sender in the node.
* `splice.lfdecentralizedtrust.org/reason`: a text specifying the reason for the operation in the node.
* `splice.lfdecentralizedtrust.org/burned`: how much of a holding was burned in the node.
Meta key/values can be specified in several optional fields. For transfers, the values from fields that are present should be merged in last-write-wins order of:
* event.choiceArgument.transfer.meta,
* event.choiceArgument.extraArgs.meta,
* event.choiceArgument.meta,
* event.exerciseResult.meta,
### Executing a factory choice
Example code: [Token Standard CLI's code to create a transfer via TransferFactory](https://github.com/canton-network/splice/blob/main/token-standard/cli/src/commands/transfer.ts)
To execute a choice via a Token Standard factory, first you need need to fetch the factory from the corresponding registry.
The mapping from an instrument's `admin` party-id to the corresponding registry URL needs to be maintained currently by wallets themselves, until a generic solution ([likely based on CNS](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md#off-ledger-api-discovery-and-access)) is implemented.
The registry will return the relevant factory in the corresponding endpoint:
* TransferFactory
* AllocationFactory
The response's payload will include three relevant fields:
* `factoryId`: the contract id of the factory
* `disclosedContracts`: must be provided to the exercise of the factory's choice for it to work
* `choiceContextData`: to be passed as `context` in the `choiceArgument`.
With this data, you can execute a choice on the factory. For external parties you must call the [prepare](/sdks-tools/api-reference/json-api#interactive-submission) and [execute](/sdks-tools/api-reference/json-api#interactive-submission) endpoints of the participant. For non-external parties, you can just use the [submit-and-wait endpoint](/sdks-tools/api-reference/json-api#command-submission).
In both cases, you must include an `ExerciseCommand` in your payload with the following fields:
* `templateId`: the interface id of the factory you want to exercise the choice on. For example, `#splice-api-token-transfer-instruction-v1:Splice.Api.Token.TransferInstructionV1:TransferFactory`.
* `contractId`: the `factoryId` obtained from the registry.
* `choice`: the name of the choice you want to execute. For example, `TransferFactory_Transfer`.
* `choiceArgument`: the arguments that will be passed to the Daml choice. These will be decoded from JSON. For a `TransferFactory_Transfer`, this will include for example the sender, receiver and amount, among other fields.
### Executing a non-factory choice
Example code: [Token Standard CLI's code to accept a transfer instruction](https://github.com/canton-network/splice/blob/main/token-standard/cli/src/commands/acceptTransferInstruction.ts)
To execute a choice on a contract implementing a Token Standard interface for external parties, you must call the [prepare](/sdks-tools/api-reference/json-api#interactive-submission) and [execute](/sdks-tools/api-reference/json-api#interactive-submission) endpoints of the participant. For non-external parties, you can just use the [submit-and-wait endpoint](/sdks-tools/api-reference/json-api#command-submission).
In both cases, you must include an `ExerciseCommand` in your payload with the following fields:
* `templateId`: the interface id of the contract you want to exercise the choice on. For example, `#splice-api-token-transfer-instruction-v1:Splice.Api.Token.TransferInstructionV1:TransferInstruction`.
* `contractId`: the contract id of the contract you want to exercise the choice on. Typically, you'll get this from the current ACS of a party.
* `choice`: the name of the choice you want to execute. For example, `TransferInstruction_Accept`.
* `choiceArgument`: the arguments that will be passed to the Daml choice. These will be decoded from JSON.
Where a `context` is required as part of the `choiceArgument`, it can be fetched from the corresponding registry:
* To accept a TransferInstruction
* To reject a TransferInstruction
* To withdraw a TransferInstruction
* To withdraw an Allocation
* To cancel an Allocation
The response of these endpoints include two fields:
* `choiceContextData`: to be passed as `context` in the `choiceArgument`.
* `disclosedContracts`: to be passed in the submit or prepare request.
Note that `AllocationRequest_Reject` and `AllocationRequest_Withdraw` should be called with an empty choice context. This `ChoiceContext` is present to allow for potential future extensions of the behavior of implementations of these choices.
### Using token standard choices from custom Daml code
Calling the token standard choices from custom Daml code is useful when integrating one's own app workflows with the token standard. Example workflows relevant to a wallet provider are merging of holdings for a user to keep their ACS small, doing bulk transfers, or marking a user's action as activity of the wallet app.
Splice releases the optional `splice-util-token-standard-wallet.dar` file, which packages common workflows that improve the operations of a wallet app. See [splice-util-token-standard-wallet](/sdks-tools/api-reference/splice-daml/splice-util-token-standard-wallet) for the per-template reference.
## API References
Refer to [CIP-0056](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md#details) for more context on the APIs.
### Token Metadata
* Daml reference: [splice-api-token-metadata-v1](/sdks-tools/api-reference/splice-daml/splice-api-token-metadata-v1)
* OpenAPI reference: [Token Metadata Service](/reference/splice-token-metadata-service/registrymetadatav1info)
### Holding
This allows implementation of a [Portfolio View](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md#wallet-client--portfolio-view).
* Daml reference: [splice-api-token-holding-v1](/sdks-tools/api-reference/splice-daml/splice-api-token-holding-v1)
### Transfer Instruction
This allows implementation of [Direct Peer-to-Peer / Free of Payment (FOP) Transfers](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md#direct-peer-to-peer--free-of-payment-fop-transfer-workflow).
* Daml reference: [splice-api-token-transfer-instruction-v1](/sdks-tools/api-reference/splice-daml/splice-api-token-transfer-instruction-v1)
* OpenAPI reference: [Transfer Instruction API](/reference/splice-transfer-instruction-api/registrytransfer-instructionv1transfer-factory)
### Allocation
This allows implementation of [Delivery versus Payment (DVP) Transfer Workflows](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md#delivery-versus-payment-dvp-transfer-workflows), jointly with the Allocation Instruction and Allocation Request APIs below.
* Daml reference: [splice-api-token-allocation-v1](/sdks-tools/api-reference/splice-daml/splice-api-token-allocation-v1)
* OpenAPI reference: [Allocation API](/reference/splice-allocation-api/registryallocationsv1-choice-contextsexecute-transfer)
### Allocation Instruction
* Daml reference: [splice-api-token-allocation-instruction-v1](/sdks-tools/api-reference/splice-daml/splice-api-token-allocation-instruction-v1)
* OpenAPI reference: [Allocation Instruction API](/reference/splice-allocation-instruction-api/registryallocation-instructionv1allocation-factory)
### Allocation Request
* Daml reference: [splice-api-token-allocation-request-v1](/sdks-tools/api-reference/splice-daml/splice-api-token-allocation-request-v1)
# Values in the Ledger API
Source: https://docs.canton.network/appdev/deep-dives/values-in-the-ledger-api
Validation, normalization, and dynamic package resolution rules for values exchanged over the Ledger API.
Commands and queries have relaxed validation rules for ingested values. Returned values are subject to normalization.
### Value Validation in Commands
In the following examples, the *target template* of a command is the template or interface identified by the `template_id` field of the command after dynamic package resolution.
A value featured in a command (e.g. `create_arguments`) has *expected type* `T` if the value needs to type-check against `T` in order to satisfy the type signatures of the target template of the command. Note that this definition necessarily extends to sub-values.
In a record value of the form `Constructor { field1 = v1, ..., fieldn = vn }`, `vi` is a *trailing None* if for all `n >= j >= i`, `vj = None`.
On submission of a command, the validation rules for values are relaxed as follows:
> * The `record_id`, `variant_id`, and `enum_id` fields of values, if present, are only checked against the module and type name of the expected type for that value. The package ID component of these fields is ignored.
> * In record values where all field names are provided, *any* fields of value None may be omitted.
> * In record values where not all field names are provided, fields must be provided in the same order as that of the record type definition, and trailing Nones may be omitted.
These rules apply for all sub-values.
**Example 1**
Assume a package called `example1-1.0.0` which defines a template called `T` in a module called `Main`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
template T
with
p : Party
where
signatory p
```
Assume another package called `other-1.0.0` which defines a different template also called `T` in a module also called `Main`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
template T
with
s : Party
i : Int
where
signatory s
```
Then the ledger API will accept Create commands for `example1-1.0.0:Main.T` whose create arguments are annotated with type `other-1.0.0:T:Main.T`, even though the type annotation is wrong. In other words, the following console commands succeed:
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ val createCmd = Command(
command = Command.Command.Create(
value = CreateCommand(
templateId = Some(
value = Identifier(packageId = packageIdExample1, moduleName = "Main", entityName = "T")),
createArguments = Some(
value = Record(
recordId = Some(
Identifier(
packageId = packageIdOther,
moduleName = "Main",
entityName = "T"
)
),
fields = Seq(
RecordField(
label = "p",
value = Some(
value = Value(sum = Value.Sum.Party(value = sandbox.adminParty.toLf))
)
)
)
)
)
)
)
)
@ sandbox.ledger_api.commands.submit(Seq(sandbox.adminParty), Seq(createCmd))
```
This is because the module and type names of the type annotation, `Main` and `T`, match those of the expected type: `example1-1.0.0:Main.T`.
**Example 2**
Assume a package called `example2-1.0.0` which defines a template with two optional fields: one in leading position, and one in trailing position.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
template T
with
i : Optional Int
p : Party
j : Optional Int
where
signatory p
```
Then submitting a Create command for `example2-1.0.0:Main.T` which only provides `p` by name and no other field succeeds:
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ val createCmd = Command(
command = Command.Command.Create(
value = CreateCommand(
templateId = Some(value = Identifier(packageId = packageIdExample2, moduleName = "Main", entityName = "T")),
createArguments = Some(
value = Record(
recordId = None,
fields = Seq(
RecordField(
label = "p",
value = Some(value = Value(sum = Value.Sum.Party(value = sandbox.adminParty.toLf)))
)
)
)
)
)
)
)
@ sandbox.ledger_api.commands.submit(Seq(sandbox.adminParty), Seq(createCmd))
res13: com.daml.ledger.api.v1.transaction.TransactionTree = TransactionTree(
...
eventsById = Map(
"#122062d3d0b89f011ac651ea0139f381a73fe080ab215e5970a8c7bf804edeec932c:0" -> TreeEvent(
kind = Created(
value = CreatedEvent(
...
createArguments = Some(
value = Record(
recordId = Some(value = Identifier(packageId = "627f4ad4df901b80bae208eded1c03932f38ed9c1f44c50468f27c88ef988e25", moduleName = "Main", entityName = "T")),
fields = Vector(
RecordField(label = "i", value = Some(value = Value(sum = Optional(value = Optional(value = None))))),
RecordField(label = "p", value = Some(value = Value(sum = Party(value = "sandbox::1220077e3366037ffce33cba97d757506fc1c72ad957a9b86c6bf137404637c7fee3")))),
RecordField(label = "j", value = Some(value = Value(sum = Optional(value = Optional(value = None)))))
)
)
),
...
)
)
)
),
...
)
```
Submitting the same command but with no label for field `p` fails:
```
@ val createCmd = Command(
command = Command.Command.Create(
value = CreateCommand(
templateId = Some(value = Identifier(packageId = packageIdExample2, moduleName = "Main", entityName = "T")),
createArguments = Some(
value = Record(
recordId = None,
fields = Seq(
RecordField(
label = "",
value = Some(value = Value(sum = Value.Sum.Party(value = sandbox.adminParty.toLf)))
)
)
)
)
)
)
)
@ sandbox.ledger_api.commands.submit(Seq(sandbox.adminParty), Seq(createCmd))
ERROR c.d.c.e.CommunityConsoleEnvironment - Request failed for sandbox.
GrpcClientError: INVALID_ARGUMENT/COMMAND_PREPROCESSING_FAILED(8,b880be91): Missing non-optional field "p", cannot upgrade non-optional fields.
Request: SubmitAndWaitTransactionTree(
actAs = sandbox::1220077e3366...,
readAs = Seq(),
commandId = '',
workflowId = '',
submissionId = '',
deduplicationPeriod = None(),
applicationId = 'CantonConsole',
commands = ...
)
...
```
However, providing all but the trailing optional field `j` suceeds, even without labels:
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ val createCmd = Command(
command = Command.Command.Create(
value = CreateCommand(
templateId = Some(value = Identifier(packageId = packageIdExample2, moduleName = "Main", entityName = "T")),
createArguments = Some(
value = Record(
recordId = None,
fields = Seq(
RecordField(label = "", value = Some(value = Value(sum = Value.Sum.Optional(value = Optional(value = None))))),
RecordField(
label = "",
value = Some(value = Value(sum = Value.Sum.Party(value = sandbox.adminParty.toLf)))
)
)
)
)
)
)
)
@ sandbox.ledger_api.commands.submit(Seq(sandbox.adminParty), Seq(createCmd))
res22: com.daml.ledger.api.v1.transaction.TransactionTree = TransactionTree(
...
eventsById = Map(
"#12203bb4082d4868c393ca2c969bb639757d21992cbac2a1abad271d688a30dbcae6:0" -> TreeEvent(
kind = Created(
value = CreatedEvent(
...
createArguments = Some(
value = Record(
recordId = Some(value = Identifier(packageId = "627f4ad4df901b80bae208eded1c03932f38ed9c1f44c50468f27c88ef988e25", moduleName = "Main", entityName = "T")),
fields = Vector(
RecordField(label = "i", value = Some(value = Value(sum = Optional(value = Optional(value = None))))),
RecordField(label = "p", value = Some(value = Value(sum = Party(value = "sandbox::1220077e3366037ffce33cba97d757506fc1c72ad957a9b86c6bf137404637c7fee3")))),
RecordField(label = "j", value = Some(value = Value(sum = Optional(value = Optional(value = None)))))
)
)
),
...
)
)
)
),
...
)
```
### Value normalization in Ledger API responses
A Ledger API value (e.g. `create_arguments` in a `CreatedEvent`) is said to be in normal form if none of its sub-values (itself included) has trailing Nones.
Starting with Daml 3.3.0, values in Ledger API non-verbose responses are subject to normalization. The normalization extends to all sub-values.
**Example**
Assume a package called `example1-1.0.0` which defines a template `T` and a record `Record` in a module called `Main`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Main where
data Record = Record { ri : Optional Int, rj : Int, rk : Optional Int }
deriving (Eq, Show)
template T
with
p : Party
i : Optional Int
r : Record
j : Optional Int
where
signatory p
```
Also assume a ledger that contains a contract of type `T` written by `example1-1.0.0` where all the optional fields are set to `None`.
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
val createCmd = ledger_api_utils.create(
packageIdExample1,
"Main",
"T",
Map(
"p" -> sandbox.adminParty,
"i" -> None,
"r" -> Map("ri" -> None, "rj" -> 1, "rk" -> None),
"j" -> None))
sandbox.ledger_api.commands.submit(Seq(sandbox.adminParty), Seq(createCmd))
```
Then querying the ledger's active contract set in non-verbose mode returns the following:
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ sandbox.ledger_api.acs.of_party(sandbox.adminParty, verbose=false)
res15: Seq[com.digitalasset.canton.admin.api.client.commands.LedgerApiTypeWrappers.WrappedCreatedEvent] = List(
WrappedCreatedEvent(
event = CreatedEvent(
...
createArguments = Some(
value = Record(
recordId = None,
fields = Vector(
RecordField(
label = "",
value = Some(value = Value(sum = Party(value = "sandbox::122010fdef685011beecd318f03c9d82bf1e2d45950bdb0fceb3497a112ee17f9476")))
),
RecordField(label = "", value = Some(value = Value(sum = Optional(value = Optional(value = None))))),
RecordField(
label = "",
value = Some(
value = Value(
sum = Record(
value = Record(
recordId = None,
fields = Vector(
RecordField(label = "", value = Some(value = Value(sum = Optional(value = Optional(value = None))))),
RecordField(label = "", value = Some(value = Value(sum = Int64(value = 1L))))
)
)
)
)
)
)
)
)
),
...
)
)
)
```
Note that not only has the third template argument (originally `j`) been omitted from the response, but also the third field of the nested record (originally `rk`). Note also that despite being optional fields of value `None`, the second template argument (originally `i`) and the first nested record field (originally `ri`) are present in the response because they are not in trailing positions.
# Common Issues FAQ
Source: https://docs.canton.network/appdev/faq
Frequently asked questions from Canton Network validators and developers
Answers to frequently asked questions from Canton Network validators and application developers. This FAQ is compiled from actual support interactions and addresses the most common points of confusion.
***
## Getting Started
**Hardware Requirements:**
* 8GB RAM minimum (16GB recommended)
* 4 CPU cores minimum
* 50GB free disk space
**Software Requirements:**
* Docker Desktop with Docker Compose 2.26.0+
* Java 17 or 21 (Java 22+ is not supported)
* Node.js 18.x or higher
* Git
**For Mac users using Colima:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
colima start --memory 8 --cpu 4
```
The prerequisites documentation may not list specific version requirements for all dependencies. If you encounter errors, verify your Docker Compose version first—this is the most common source of quickstart failures.
| Environment | Purpose | Access | Real Value |
| ------------ | --------------------------- | ------------------------- | ----------------- |
| **LocalNet** | Development on your machine | No access needed | No |
| **DevNet** | Integration testing | VPN + SV sponsorship | No |
| **TestNet** | Staging/pre-production | IP whitelist required | No |
| **MainNet** | Production | IP whitelist + onboarding | Yes (Canton Coin) |
**LocalNet** runs entirely on your machine with a local synchronizer. Use for initial development and unit testing.
**DevNet** connects to the public development environment. Requires VPN access and Super Validator sponsorship. Allow 2-4 weeks for approval.
**TestNet** is for staging deployments before production. More stable than DevNet. Requires IP whitelisting.
**MainNet** (Global Synchronizer) is production. Real Canton Coin with real value. Full validator onboarding process required.
1. Contact a Super Validator sponsor listed at [canton.foundation](https://canton.foundation)
2. They will:
* Provide VPN credentials
* Whitelist your validator IP
* Submit sponsorship information
**Allow 2-4 weeks** for the approval process.
DevNet is designed for integration testing and requires an active relationship with a Super Validator sponsor.
***
## Validator Operations
If your validator shows an old version on explorers like ccview\.io or CantonLoop Lighthouse despite successful helm upgrade, the issue is likely using the `--reuse-values` flag.
**Solution:**
Upgrade **without** `--reuse-values`:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
helm upgrade validator splice-validator/splice-validator \
--version 0.5.4 \
-f validator-values.yaml \
--namespace validator
```
**Verify:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
kubectl -n validator get deploy validator-app \
-o "jsonpath={.spec.template.spec.containers[0].image}"
```
The `--reuse-values` flag can cause the old version configuration to persist even when upgrading to a new chart version.
These URLs serve different purposes and should **not** be confused:
**SV URL** (Super Validator URL):
* Used for: Validator onboarding and sponsorship
* Format: `https://sv.sv-2.global.canton.network.digitalasset.com`
* Goes in: `svSponsorAddress` configuration
**Scan URL**:
* Used for: Viewing network data, exploring transactions
* Format: `https://scan.sv-2.global.canton.network.digitalasset.com`
* Used by: Block explorers and public-facing tools
Using the Scan URL in your `svSponsorAddress` will cause onboarding failures with errors like "Gave up getting app version".
Add pruning configuration to your `validator-values.yaml`:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
participantPruningSchedule:
cron: "0 */10 * * * ?" # Every 10 minutes
maxDuration: 30m # Max time per pruning run
retention: 90d # Keep 90 days of history
```
**For first-time pruning on MainNet:**
If you have a large history, increase `maxDuration` or start with a larger `retention`:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
participantPruningSchedule:
cron: "0 */10 * * * ?"
maxDuration: 60m # Longer for initial pruning
retention: 180d # Start high, reduce later
```
**Monitor progress:**
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.pruning.get_schedule()
res1: Option[PruningSchedule] = Some(value = PruningSchedule(cron = "0 */10 * * * ?", maxDuration = 30m, retention = 2160h))
```
Check `/v2/state/latest-pruned-offsets` endpoint to verify pruning is running.
**Via HTTP endpoints:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Validator health
curl http://localhost/api/validator/readyz
# Participant health
curl http://localhost:5003/health
```
**Via Canton Console:**
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ health.status
res2: CantonStatus = Status for Sequencer 'sequencer1':
Sequencer id: sequencer1::1220cb0a22fb0aef9243a11f778497d7cacb19f9c4bcc7606776a109983edfaa6b4a
Synchronizer id: da::122032922613929d67857e621fb13e3da49ec13883e24908404520319eee6d31fb4d::35-0
Uptime: 13.206078s
Ports:
public: 30304
admin: 30308
Connected participants:
PAR::participant1::12201ff69b1d...
Connected mediators:
MED::mediator1::122009299340...
Sequencer: SequencerHealthStatus(active = true)
details-extra: None
Components:
memory_storage : Ok()
sequencer : Ok()
Accepts admin changes: true
Version: 3.6.0-SNAPSHOT
Protocol version: 35
Status for Mediator 'mediator1':
Node uid: mediator1::12200929934059da3e012af672ee8a5d26a7e4b3e5084920be298f791f7619843c78
Synchronizer id: da::122032922613929d67857e621fb13e3da49ec13883e24908404520319eee6d31fb4d::35-0
Uptime: 12.617687s
Ports:
admin: 30302
Active: true
Components:
memory_storage : Ok()
sequencer-client : Ok()
sequencer-connection-pool : Ok()
sequencer-subscription-pool : Ok()
internal-sequencer-connection-sequencer1-0 : Ok()
subscription-sequencer-connection-sequencer1-0 : Ok()
Version: 3.6.0-SNAPSHOT
Protocol version: 35
Status for Participant 'participant1':
Participant id: PAR::participant1::12201ff69b1d24edbf0ee2028a304ea702ee8536790dab1a31e7136e6d90ff6d473c
Uptime: 20.221094s
Ports:
ledger: 30296
admin: 30298
json: 30300
Connected synchronizers:
da::122032922613...::35-0
Unhealthy synchronizers: None
Active: true
Components:
memory_storage : Ok()
connected-synchronizer : Ok()
sync-ephemeral-state : Ok()
sequencer-client : Ok()
acs-commitment-processor : Ok()
sequencer-connection-pool : Ok()
sequencer-subscription-pool : Ok()
internal-sequencer-connection-sequencer1-0 : Ok()
subscription-sequencer-connection-sequencer1-0 : Ok()
Version: 3.6.0-SNAPSHOT
Supported protocol version(s): 35
Status for Participant 'participant2':
Participant id: PAR::participant2::1220a4d7463bd34b2ba3704401b48ab41d8f88cdcbe512fc1ef071aad97fef106161
Uptime: 21.457155s
Ports:
ledger: 30288
admin: 30290
json: 30292
Connected synchronizers: None
Unhealthy synchronizers: None
Active: true
Components:
memory_storage : Ok()
connected-synchronizer : Not Initialized
sync-ephemeral-state : Not Initialized
sequencer-client : Not Initialized
acs-commitment-processor : Not Initialized
Version: 3.6.0-SNAPSHOT
Supported protocol version(s): 35
```
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.synchronizers.list_connected()
res3: Seq[ListConnectedSynchronizersResult] = Vector(
ListConnectedSynchronizersResult(
synchronizerAlias = Synchronizer 'da',
physicalSynchronizerId = da::122032922613...::35-0,
healthy = true
)
)
```
**Via Kubernetes:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
kubectl get pods -n validator
kubectl logs -n validator deployment/validator-app --tail=100
```
**Signs of a healthy validator:**
* All pods in Running state
* Health endpoints return 200
* Connected to synchronizer
* No persistent error logs
* Receiving liveness rewards (MainNet)
Network upgrades follow a coordinated schedule. When an upgrade occurs:
1. **Check the target version** at [canton.foundation/sv-network-status](https://canton.foundation/sv-network-status/)
2. **Review release notes** for breaking changes and migration requirements
3. **For version upgrades** (e.g., 0.4.x → 0.5.x):
* Take backups/snapshots before upgrading
* Update database name if required:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
persistence:
databaseName: participant_4
```
4. **Upgrade your helm charts** or Docker images to match network version
5. **Verify** your validator rejoins the network and resumes operation
Do not upgrade incrementally through intermediate versions. Upgrade directly to the current network version.
***
## Authentication & Security
1. **Set up your OIDC provider** (Auth0, Keycloak, etc.)
2. **Configure environment variables** in your `.env` file:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
AUTH_URL="https://your-tenant.auth0.com"
AUTH_JWKS_URL="https://your-tenant.auth0.com/.well-known/jwks.json"
AUTH_WELLKNOWN_URL="https://your-tenant.auth0.com/.well-known/openid-configuration"
LEDGER_API_AUTH_AUDIENCE="https://ledger_api.your-domain.com"
VALIDATOR_ADMIN_USER="auth0|123456789"
WALLET_ADMIN_USER="auth0|123456789"
```
3. **Start the validator with authentication:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
./start.sh -a
```
**Migrating from non-authenticated to authenticated:**
* Stop validator: `./stop.sh`
* Restart with `-a` flag: `./start.sh -a`
* The validator operator user will be automatically migrated
Your OIDC provider must issue JWTs with the `daml_ledger_api` scope when requested.
This typically occurs when token lifetime is too short. Newer splice versions may require longer token lifetimes.
**Solution:**
Increase access token timeout in your OIDC provider:
**For Auth0:**
1. Applications → Your App → Settings
2. Advanced Settings → Access Token Lifetime
3. Set to 900 seconds (15 minutes) or higher
**For Keycloak:**
1. Realm Settings → Tokens
2. Access Token Lifespan → 900 (15 minutes)
Then restart your validator.
**Recommended settings:**
* Access Token: 15-30 minutes
* Refresh Token: 24 hours
The default 5-minute token lifetime that some OIDC providers use is often insufficient for Canton validators, especially during high-activity periods or network latency.
***
## Transactions & Errors
This error indicates the mediator didn't receive sufficient confirmations from all required parties within the timeout period.
**Common causes:**
1. **Insufficient Canton Coin** - A party doesn't have enough CC for traffic top-ups
2. **Validator offline** - One of the involved validators is down or unreachable
3. **Network latency** - Temporary network issues
**Solution:**
1. Check Canton Coin balances for all involved parties
2. Verify all validators are healthy
3. Top up CC if needed:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -X POST "http://localhost/api/validator/v0/admin/traffic/purchase" \
-H "Authorization: Bearer $TOKEN"
```
The error message includes `unresponsiveParties` which tells you which party(ies) didn't respond.
503 errors typically indicate the participant is overloaded. Check for:
**Database queue overflow:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
grep "DB_STORAGE_DEGRADATION" participant.log
grep "queued tasks = 2000" participant.log
```
**Solutions:**
1. **Enable pruning** to reduce database size
2. **Increase database resources** (IOPS, memory)
3. **Consider PQS** for read-heavy workloads
4. **Implement retry logic** with exponential backoff
If you're submitting many transactions, consider batching or rate limiting your submissions.
This error occurs when multiple transactions are competing for the same locked contracts or resources.
**Solution:**
Implement retry logic with exponential backoff:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
async function submitWithRetry(command, maxRetries = 5) {
for (let i = 0; i < maxRetries; i++) {
try {
return await submit(command);
} catch (e) {
if (e.code === 'ABORTED' && i < maxRetries - 1) {
await sleep(Math.pow(2, i) * 100);
continue;
}
throw e;
}
}
}
```
This error is **expected** in concurrent environments - the retry strategy is the correct solution.
**Steps to debug:**
1. **Get the trace ID** from the error response
2. **Search logs** for the trace ID:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
grep "trace-id\":\"YOUR_TRACE_ID" participant.log validator.log
```
3. **Check common causes:**
* Authorization failures (party not authorized)
* Package not vetted
* Insufficient traffic (Canton Coin)
* Contract already archived
* Timeout issues
4. **Use Canton Console** for deeper investigation:
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.parties.list()
res4: Seq[ListPartiesResult] = Vector(
ListPartiesResult(
partyResult = participant1::12201ff69b1d...,
participants = Vector(
ParticipantSynchronizers(
participant = PAR::participant1::12201ff69b1d...,
synchronizers = Vector(
SynchronizerPermission(synchronizerId = da::122032922613..., permission = Submission)
)
)
)
),
ListPartiesResult(
partyResult = Alice::12201ff69b1d...,
participants = Vector(
ParticipantSynchronizers(
participant = PAR::participant1::12201ff69b1d...,
synchronizers = Vector(
SynchronizerPermission(synchronizerId = da::122032922613..., permission = Submission)
)
)
)
)
)
```
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.packages.list()
res5: Seq[PackageDescription] = Vector(
PackageDescription(
packageId = 9e70a8b3510d...,
name = ghc-stdlib-DA-Internal-Template,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 114
),
PackageDescription(
packageId = 0e4a572ab1fb...,
name = daml-prim-DA-Internal-Erased,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 98
),
PackageDescription(
packageId = 5aee9b21b8e9...,
name = daml-prim-DA-Types,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 17554
),
PackageDescription(
packageId = a1fa18133ae4...,
name = daml-stdlib-DA-Action-State-Type,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 593
),
PackageDescription(
packageId = 60c61c542207...,
name = daml-stdlib-DA-Stack-Types,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 1194
),
PackageDescription(
packageId = d095a2ccf6dd...,
name = daml-stdlib-DA-Semigroup-Types,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 426
),
PackageDescription(
packageId = ee33fb70918e...,
name = daml-prim-DA-Exception-ArithmeticError,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 286
),
PackageDescription(
packageId = c280cc3ef501...,
name = daml-stdlib-DA-Internal-Interface-AnyView-Types,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 826
),
PackageDescription(
packageId = de2cc2f90eb5...,
name = canton-builtin-admin-workflow-ping,
version = 3.4.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 148192
),
PackageDescription(
packageId = e5411f3d75f0...,
name = daml-prim-DA-Internal-NatSyn,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 109
),
PackageDescription(
packageId = 7adc4c2d07fa...,
name = daml-stdlib-DA-Internal-Fail-Types,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 802
),
PackageDescription(
packageId = 86d888f34152...,
name = daml-stdlib-DA-Internal-Down,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 258
),
PackageDescription(
packageId = 99ea07e101ed...,
name = daml-stdlib,
version = 3.4.0.20251020.14338.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 711601
),
PackageDescription(
packageId = 6f8e6085f576...,
name = ghc-stdlib-DA-Internal-Any,
version = 1.0.0,
uploadedAt = 2026-06-04T11:34:11.579923Z,
size = 390
),
...
```
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.ledger_api.state.acs.of_party(alice)
res6: Seq[com.digitalasset.canton.admin.api.client.commands.LedgerApiTypeWrappers.WrappedContractEntry] = List(
WrappedContractEntry(
entry = ActiveContract(
value = ActiveContract(
createdEvent = Some(
value = CreatedEvent(
offset = 12L,
nodeId = 0,
contractId = "00a90f34c0581381cb8f21bc4a7910d1b94a56f5ee13e2d643f960aa3f8f32436cca12122011ec536be02b9b5c6c53a1b12c91710ea39bfbdc42353ff52e828c865f082daa",
templateId = Some(
value = Identifier(
packageId = "2bf40efb6ff32ee400d0f1ade4fbc2aac695c75ed617ccdec57615fabbb4ad38",
moduleName = "Iou",
entityName = "Iou"
)
),
contractKey = None,
contractKeyHash = ,
createArguments = Some(
value = Record(
recordId = Some(
value = Identifier(
packageId = "2bf40efb6ff32ee400d0f1ade4fbc2aac695c75ed617ccdec57615fabbb4ad38",
moduleName = "Iou",
entityName = "Iou"
)
),
fields = Vector(
RecordField(
label = "payer",
value = Some(
value = Value(
sum = Party(
value = "Alice::12201ff69b1d24edbf0ee2028a304ea702ee8536790dab1a31e7136e6d90ff6d473c"
)
)
)
),
RecordField(
label = "owner",
value = Some(
value = Value(
sum = Party(
value = "Alice::12201ff69b1d24edbf0ee2028a304ea702ee8536790dab1a31e7136e6d90ff6d473c"
)
)
)
),
RecordField(
label = "amount",
value = Some(
value = Value(
sum = Record(
value = Record(
recordId = Some(
value = Identifier(
packageId = "2bf40efb6ff32ee400d0f1ade4fbc2aac695c75ed617ccdec57615fabbb4ad38",
moduleName = "Iou",
entityName = "Amount"
)
),
fields = Vector(
RecordField(
label = "value",
value = Some(value = Value(sum = Numeric(value = "100.0000000000")))
),
RecordField(
label = "currency",
value = Some(value = Value(sum = Text(value = "EUR")))
)
)
)
)
)
)
),
RecordField(
label = "viewers",
value = Some(value = Value(sum = List(value = List(elements = Vector()))))
)
)
)
),
createdEventBlob = ,
interfaceViews = Vector(),
witnessParties = Vector(
"Alice::12201ff69b1d24edbf0ee2028a304ea702ee8536790dab1a31e7136e6d90ff6d473c"
),
signatories = Vector(
"Alice::12201ff69b1d24edbf0ee2028a304ea702ee8536790dab1a31e7136e6d90ff6d473c"
),
observers = Vector(),
createdAt = Some(
value = Timestamp(
seconds = 1780572879L,
...
```
***
## Quickstart Issues
**Diagnostic steps:**
1. **Check logs:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
docker logs splice-validator-participant-1
```
2. **Verify resources:**
* Docker memory ≥ 8GB
* Docker CPU ≥ 4 cores
3. **Check for configuration errors** in your `.env` file
**Common solutions:**
For Colima users:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
colima stop
colima start --memory 8 --cpu 4
```
For Docker Desktop:
* Settings → Resources → Memory → 8GB+
* Apply & Restart
Then clean start:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
make clean
make setup && make build
make start
```
**Error:**
```
'env_file[1]' expected type 'string', got unconvertible type 'map[string]interface {}'
```
**Cause:** Docker Compose version is below 2.26.0
**Solution:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Check version
docker compose version
# Upgrade (Mac with Homebrew)
brew install docker-compose
# Or update Docker Desktop
```
Canton Quickstart requires Docker Compose 2.26.0+.
On adequate hardware (8GB RAM, 4 CPU cores):
* **First run:** 10-15 minutes (downloading images, building)
* **Subsequent runs:** 2-5 minutes
If startup exceeds 20 minutes, check:
* Available system resources
* Docker logs for errors
* Network connectivity for image downloads
***
## Backup & Recovery
**Export node ID dump:**
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.health.dump()
res7: String = "canton/canton-dump-2026-06-04T11-34-42.068575Z.zip"
```
This produces a JSON file containing:
* Participant ID
* Cryptographic key pairs (namespace, signing, encryption)
* Authorized store snapshot
* Version
**Store securely** - this backup allows recovery of your validator identity.
The node ID dump contains private keys. Encrypt and store securely, following your organization's key management policies.
Yes, but you may need to update the key names in the JSON file.
**Old format (pre-0.4.x):**
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"keys": [
{ "name": "participant-namespace", ... },
{ "name": "participant-signing", ... },
{ "name": "participant-encryption", ... }
]
}
```
**Current format:**
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"keys": [
{ "name": "namespace", ... },
{ "name": "signing", ... },
{ "name": "encryption", ... }
]
}
```
Update the key names and version field before restoration.
**Before any upgrade:**
1. **Database snapshots** (PostgreSQL dump)
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
pg_dump -h localhost -U cnadmin cantonnet_participant > backup.sql
```
2. **Persistent Volume snapshots** (Kubernetes)
* Validator PV
* Participant PV
3. **Node ID dump**
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ participant1.health.dump()
res8: String = "canton/canton-dump-2026-06-04T11-34-42.673059Z.zip"
```
4. **Configuration files**
* `validator-values.yaml`
* `.env` files
* Custom configuration
5. **Document current state**
* Current version
* Migration ID
* Database names
***
## Performance & Scaling
**Options for improving performance:**
1. **Enable pruning** to reduce ACS size:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
participantPruningSchedule:
cron: "0 */10 * * * ?"
maxDuration: 30m
retention: 90d
```
2. **Use PQS** (Participant Query Store) for read-heavy workloads - moves queries off the main participant
3. **Increase database resources:**
* Upgrade storage (gp2 → gp3 on AWS)
* Increase IOPS
* Add more memory/CPU
4. **Tune connection pools:**
```hocon theme={"theme":{"light":"github-light","dark":"github-dark"}}
canton.participants.participant1.storage.parameters.connection-allocation {
num-ledger-api = 32
}
```
5. **Implement client-side batching** and rate limiting
Large databases are common on MainNet validators that have been running for a while without pruning.
**Solutions:**
1. **Enable pruning** (see pruning FAQ above)
2. **Start with conservative retention** to reduce initial pruning volume:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
retention: 180d # Start high
maxDuration: 60m # Allow longer pruning runs
```
3. **Monitor database growth** and adjust retention as needed
4. **Consider database maintenance:**
* VACUUM ANALYZE on PostgreSQL
* Index optimization
***
## Wallet & Canton Coin
**Via Wallet UI:**
Navigate to the wallet interface and use the top-up functionality.
**Via API:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -X POST "http://localhost/api/validator/v0/admin/traffic/purchase" \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json"
```
**Automatic top-ups:**
Configure automatic traffic purchases in your validator configuration:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
# In .env or validator-values.yaml
TARGET_TRAFFIC_THROUGHPUT=20000
MIN_TRAFFIC_TOPUP_INTERVAL=1m
```
**DevNet/TestNet provide faucet functionality** for obtaining test Canton Coin:
1. Access your wallet UI
2. Use the "Tap" or faucet functionality
3. Test CC will be credited to your wallet
Test CC has no real value and is only for testing purposes on DevNet and TestNet.
No, your CC is likely not lost. This usually indicates a sync issue.
**Steps:**
1. Wait for validator to fully resync (can take hours after a protocol upgrade)
2. Check for errors in logs:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
grep "503\|UNAVAILABLE" logs-validator.log
```
3. Verify all components are healthy
4. If issue persists after 24 hours, contact support with logs
The validator needs to process all historical events to display correct balances.
***
## Training & Certification
**Warning:** Some certification content may teach deprecated patterns.
Courses built for Daml 2.x may not align with Canton Network 3.x architecture. If you're building on Canton Network:
1. **Focus on current documentation** on this site
2. **Use the Canton Quickstart** for hands-on learning
If a course doesn't mention Canton Network or Daml 3.x, or covers only Daml 2.x specifically, the architectural patterns may not apply to current Canton Network development.
**Recommended resources:**
1. **Official Documentation:**
* [Build Documentation](/appdev/get-started/choose-your-path)
* [Operator Documentation](/global-synchronizer/understand/introduction)
2. **Hands-on:**
* [Canton Quickstart](https://github.com/digital-asset/cn-quickstart)
* Work through the quickstart end-to-end
3. **Community:**
* Join the Slack channels (#gsf-global-synchronizer-appdev)
* Ask questions in validator-operations for operational topics
4. **Videos:**
* [Digital Asset YouTube](https://www.youtube.com/@digitalassetcom)
* [Canton Network YouTube](https://www.youtube.com/@CantonNetwork)
***
## Support & Escalation
**Support Channels:**
| Type | Contact | Response |
| -------------------- | ----------------------------------------------------------------- | ----------- |
| **Discretionary** | [da-support@digitalasset.com](mailto:da-support@digitalasset.com) | Best effort |
| **SLA (Enterprise)** | [support@digitalasset.com](mailto:support@digitalasset.com) | SLA-based |
| **Community** | Slack channels | Community |
| **Forum** | [discuss.daml.com](https://discuss.daml.com) | Community |
**When contacting support, include:**
* Validator ID
* Network (DevNet/TestNet/MainNet)
* Splice version
* Infrastructure details (Docker/K8s, cloud provider)
* Relevant logs
* Steps to reproduce
* Timeline of when issue started
**Essential information:**
1. **Environment:**
* Splice/Canton version
* Deployment method (Docker Compose / Kubernetes)
* Cloud provider and infrastructure details
* Database setup
2. **Issue details:**
* Clear description of the problem
* Expected vs actual behavior
* When the issue started
* Any recent changes made
3. **Logs:**
* Participant logs
* Validator logs
* Relevant stack traces
* Timestamps of errors
4. **Identifiers:**
* Validator ID
* Party IDs involved
* Transaction IDs (if applicable)
* Trace IDs from error messages
Redact sensitive information (private keys, passwords, JWTs) before sharing logs.
***
## Network-Specific Questions
**DevNet → TestNet:**
1. Request TestNet IP whitelisting
2. Update configuration:
* Change synchronizer URLs
* Update SV sponsor address
3. Deploy fresh or migrate (depending on use case)
**TestNet → MainNet:**
1. Complete MainNet validator onboarding
2. Request MainNet IP whitelisting
3. Follow [MainNet onboarding documentation](/global-synchronizer/deployment/onboarding-process)
4. Deploy with production configuration
DevNet and TestNet data cannot be migrated to MainNet. Plan for fresh deployment.
**Network status:**
Visit [canton.foundation/sv-network-status](https://canton.foundation/sv-network-status/) for current version information.
**Your validator version:**
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Kubernetes
kubectl -n validator get deploy validator-app -o jsonpath='{.spec.template.spec.containers[0].image}'
# Docker
docker inspect validator-app --format='{{.Config.Image}}'
```
**Via Canton Console:**
```scala theme={"theme":{"light":"github-light","dark":"github-dark"}}
@ health.status
res9: CantonStatus = Status for Sequencer 'sequencer1':
Sequencer id: sequencer1::1220cb0a22fb0aef9243a11f778497d7cacb19f9c4bcc7606776a109983edfaa6b4a
Synchronizer id: da::122032922613929d67857e621fb13e3da49ec13883e24908404520319eee6d31fb4d::35-0
Uptime: 23.570802s
Ports:
public: 30304
admin: 30308
Connected participants:
PAR::participant1::12201ff69b1d...
Connected mediators:
MED::mediator1::122009299340...
Sequencer: SequencerHealthStatus(active = true)
details-extra: None
Components:
memory_storage : Ok()
sequencer : Ok()
Accepts admin changes: true
Version: 3.6.0-SNAPSHOT
Protocol version: 35
Status for Mediator 'mediator1':
Node uid: mediator1::12200929934059da3e012af672ee8a5d26a7e4b3e5084920be298f791f7619843c78
Synchronizer id: da::122032922613929d67857e621fb13e3da49ec13883e24908404520319eee6d31fb4d::35-0
Uptime: 22.980273s
Ports:
admin: 30302
Active: true
Components:
memory_storage : Ok()
sequencer-client : Ok()
sequencer-connection-pool : Ok()
sequencer-subscription-pool : Ok()
internal-sequencer-connection-sequencer1-0 : Ok()
subscription-sequencer-connection-sequencer1-0 : Ok()
Version: 3.6.0-SNAPSHOT
Protocol version: 35
Status for Participant 'participant1':
Participant id: PAR::participant1::12201ff69b1d24edbf0ee2028a304ea702ee8536790dab1a31e7136e6d90ff6d473c
Uptime: 30.582442s
Ports:
ledger: 30296
admin: 30298
json: 30300
Connected synchronizers:
da::122032922613...::35-0
Unhealthy synchronizers: None
Active: true
Components:
memory_storage : Ok()
connected-synchronizer : Ok()
sync-ephemeral-state : Ok()
sequencer-client : Ok()
acs-commitment-processor : Ok()
sequencer-connection-pool : Ok()
sequencer-subscription-pool : Ok()
internal-sequencer-connection-sequencer1-0 : Ok()
subscription-sequencer-connection-sequencer1-0 : Ok()
Version: 3.6.0-SNAPSHOT
Supported protocol version(s): 35
Status for Participant 'participant2':
Participant id: PAR::participant2::1220a4d7463bd34b2ba3704401b48ab41d8f88cdcbe512fc1ef071aad97fef106161
Uptime: 31.819044s
Ports:
ledger: 30288
admin: 30290
json: 30292
Connected synchronizers: None
Unhealthy synchronizers: None
Active: true
Components:
memory_storage : Ok()
connected-synchronizer : Not Initialized
sync-ephemeral-state : Not Initialized
sequencer-client : Not Initialized
acs-commitment-processor : Not Initialized
Version: 3.6.0-SNAPSHOT
Supported protocol version(s): 35
```
***
## Still Have Questions?
If your question isn't answered here:
1. **[Search the documentation](https://docs.canton.network)** on this site
2. **Check the [Troubleshooting Cheat Sheet](/appdev/troubleshooting)** for specific error solutions
3. **Ask in [community Slack](https://docs.canton.network/shared/support-channels)** channels for guidance from other developers
4. **[Contact support](mailto:da-support@digitalasset.com)** with detailed information about your issue
Have a common question that should be added? Let us know via support.
# Choose Your Path
Source: https://docs.canton.network/appdev/get-started/choose-your-path
Find the right learning path based on your background and goals
Whether you're new to blockchain or migrating from another platform, this guide helps you find the most efficient path to building on Canton Network.
## Quick Assessment
**Recommended Path:**
1. [Five-Minute Overview](/overview/understand/five-minute-overview) - Understand what Canton is
2. [Core Concepts](/overview/understand/core-concepts) - Learn the fundamentals
3. [Module 1: Understanding Canton](/appdev/modules/m1-understanding-canton) - Build mental models
4. [Module 3: Daml Smart Contracts](/appdev/modules/m3-dev-environment) - Start coding
5. [Module 4: Building Applications](/appdev/modules/m4-building-apps-intro) - Hands-on practice with the example application
**Recommended Path:**
1. [Canton for Ethereum Developers](/appdev/modules/m2-canton-for-ethereum-devs) - Map your knowledge
2. [Privacy Model](/overview/learn/privacy-model) - Understand the key difference
3. [Module 3: Daml Smart Contracts](/appdev/modules/m3-dev-environment) - Learn Daml syntax
4. [Module 4: Building Applications](/appdev/modules/m4-building-apps-intro) - Hands-on practice building a full-stack Canton app
**Key differences to internalize:**
* Immutable contracts (archive + create, not mutate)
* Explicit authorization (signatory/controller, not msg.sender)
* Privacy by default (declare observers, not hide data)
**Recommended Path:**
1. [Five-Minute Overview](/overview/understand/five-minute-overview) - Canton's approach
2. [Canton for Ethereum Developers](/appdev/modules/m2-canton-for-ethereum-devs) - Concept mapping (still useful)
3. [Architecture Overview](/overview/learn/architecture) - How components work
4. [Module 3: Daml Smart Contracts](/appdev/modules/m3-dev-environment) - Start coding
**Recommended Path:**
1. [Five-Minute Overview](/overview/understand/five-minute-overview)
2. [The Problem Canton Solves](/overview/understand/the-problem)
3. [Canton's Solution](/overview/understand/cantons-solution)
4. [Use Cases](/overview/understand/use-cases)
5. [Architecture Overview](/overview/learn/architecture)
## Learning Modules
The developer documentation is organized into progressive modules:
| Module | Focus | Prerequisites |
| ------------ | ------------------------- | ------------------------------ |
| **Module 1** | Understanding Canton | None |
| **Module 2** | Canton for Ethereum Devs | Ethereum/blockchain experience |
| **Module 3** | Daml Smart Contracts | Module 1 or 2 |
| **Module 4** | Building Applications | Module 3 |
| **Module 5** | Testing & Deployment | Module 4 |
| **Module 6** | Smart Contract Upgrades | Module 3-5 |
| **Module 7** | Production Best Practices | Module 5 |
## Development Stack Overview
Canton Network development involves these components:
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph YourCode[Your Application]
FE[Frontend
React, Vue, etc.]
BE[Backend
TypeScript, Java]
SC[Smart Contracts
Daml]
end
subgraph Canton[Canton Network Infrastructure]
PART[Validator Node]
PQS[PQS
SQL queries]
SYNC[Synchronizer]
end
FE --> BE
BE --> PART
BE --> PQS
PQS --> PART
PART <--> SYNC
SC --> PART
```
## Prerequisites
Before starting development:
### Required
* **Programming experience** in any language
* **Command line** familiarity
* **Git** for version control
### Helpful (but not required)
* **Functional programming** concepts (Haskell, OCaml, F#, or similar)
* **Docker** for running local environments
* **PostgreSQL** for PQS queries
### Development Environment
Install the SDK including Daml compiler and tools.
Install the Daml VS Code extension for syntax highlighting and IDE support.
## Hands-on Practice
Ready to build? [Module 4: Building Applications](/appdev/modules/m4-building-apps-intro) walks you through a full-stack Canton Network application end-to-end — prerequisites, running the demo, backend and frontend development, the JSON Ledger API, and observability.
## Getting Help
Email a request to join the app developer Slack channel.
Technical discussions and Q\&A
Common questions answered
# What's New
Source: https://docs.canton.network/appdev/get-started/whats-new
Pointers to release notes for Canton Network components
For changes shipping in each component, see the corresponding release notes:
* **Splice and the Global Synchronizer** — [Splice release notes](/global-synchronizer/release-notes/splice), [release history](/global-synchronizer/release-notes/release-history), [weekly patch releases](/global-synchronizer/release-notes/weekly-patch-releases)
* **Wallet SDK** — [Wallet SDK release notes](/integrations/release-notes/wallet-sdk)
* **Canton and Daml SDK** — [Canton release notes](/global-synchronizer/release-notes/canton)
* **CIPs** — [Canton Improvement Proposals](https://github.com/canton-foundation/cips)
## Version compatibility
Compatible versions across Canton Network components.
## Upgrading from a previous version
See [Upgrading from Previous Versions](/appdev/get-started/upgrading-from-previous-versions) for guidance on moving applications across SDK series.
## Staying current
* **GitHub releases** — [github.com/digital-asset](https://github.com/digital-asset)
* **Slack** — `#gsf-global-synchronizer-appdev`
* **Forum** — [forum.canton.network](https://forum.canton.network/)
# The Canton Development Stack
Source: https://docs.canton.network/appdev/modules/m1-development-stack
Overview of tools and technologies for building on Canton Network
This page introduces the development stack you'll use to build Canton applications. Understanding these components helps you see how everything fits together.
## Stack Overview
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph App[Your Application]
FE[Frontend
React, Vue, etc.]
BE[Backend
TypeScript, Java, Python]
SC[Smart Contracts
Daml]
end
subgraph Tools[Development Tools]
SDK[Daml SDK]
DPM[dpm
Package Manager]
DAML[Daml Compiler]
SCRIPT[Daml Script]
VSC[VS Code Extension]
SANDBOX[Sandbox]
end
subgraph Infra[Canton Infrastructure]
LOCAL[LocalNet
Development]
PART[Participant Node]
PQS[PQS
Query Service]
end
SC --> DAML
DAML --> SDK
SDK --> LOCAL
FE --> BE
BE --> PART
BE --> PQS
```
## Smart Contract Layer
### Daml
Daml is Canton's smart contract language—a functional language designed for multi-party workflows.
| Aspect | Details |
| --------------- | ---------------------------------------- |
| **Paradigm** | Functional programming |
| **Type system** | Strongly typed with inference |
| **Compiles to** | Daml-LF (ledger format) |
| **Primary use** | Define contracts, choices, authorization |
**Example:**
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Token
with
owner : Party
issuer : Party
amount : Decimal
where
signatory issuer
observer owner
choice Transfer : ContractId Token
with newOwner : Party
controller owner
do create this with owner = newOwner
```
### Daml Compiler
The Daml compiler (`dpm build`) compiles Daml source code into DAR files (Daml Archives) that can be deployed to participant nodes.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Compile Daml code
dpm build
# Output: .dar file containing compiled contracts
```
## Application Layer
### Backend Integration
Your backend connects to Canton via the Ledger API.
| Option | Protocol | Best For |
| ------------ | ------------- | --------------------------------- |
| **gRPC API** | gRPC/Protobuf | High-performance, typed |
| **JSON API** | HTTP/JSON | Simpler integration, web-friendly |
**Language support:**
* TypeScript/JavaScript (code generation available via `dpm codegen-js`)
* Java (code generation available via `dpm codegen-java`)
* Any language via gRPC or JSON API
Community-supported bindings also exist for Python, Rust, and Go.
### Code Generation
Generate type-safe bindings from your Daml code:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Generate TypeScript bindings
dpm codegen-js .dar -o generated
# Generate Java bindings
dpm codegen-java .dar -o generated
```
Generated code provides:
* Type-safe contract representations
* Command submission helpers
* Event handling utilities
### Frontend
Use any web framework. Common choices:
| Framework | Notes |
| ----------- | ------------------------------- |
| **React** | Most common in Canton ecosystem |
| **Vue** | Good alternative |
| **Angular** | Enterprise preference |
The frontend typically connects via your backend, which handles Ledger API communication.
## Development Tools
### Daml SDK
The Daml SDK bundles everything needed for Canton development:
| Component | Purpose |
| ------------------ | ------------------------------------ |
| **Daml compiler** | Compile smart contracts |
| **Daml Script** | Test and interact with contracts |
| **Sandbox** | Run a single Canton node for testing |
| **Canton runtime** | Run local participant nodes |
| **Console** | Interactive administration |
| **Templates** | Project scaffolding |
### dpm (Daml Package Manager)
Manage dependencies and build workflows:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Initialize project
dpm init
# Add dependency by editing dam.yaml
# Build
dpm build
```
### VS Code Extension
The Daml VS Code extension (Daml Studio) provides:
* Syntax highlighting
* Type checking
* Error diagnostics
* Code navigation
* Integrated terminal
The extension is installed automatically with DPM. You need VS Code 1.87 or above. To launch Daml Studio, run `dpm studio` from your project directory.
## Infrastructure Components
### LocalNet
LocalNet is a local Global Synchronizer simulation for development:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Start LocalNet (via QuickStart)
make setup
make build
make start
# Or run a single Canton node via Daml SDK
dpm sandbox
```
LocalNet provides:
* Local synchronizer
* Local participant node(s)
* Test Canton Coin
* No external dependencies
### Participant Node
The participant node is the portion of the validator that hosts the Canton runtime which:
* Hosts your parties
* Stores contract data
* Executes Daml logic
* Exposes the Ledger API
In production, this runs as part of your validator.
### PQS (Participant Query Store)
PQS provides SQL-based querying for complex data access:
| Use Case | Ledger API | PQS |
| -------------------- | ---------- | --------- |
| Simple queries | Good | Good |
| Complex aggregations | Limited | Excellent |
| Reporting | Difficult | Easy |
| Real-time updates | Excellent | Excellent |
PQS maintains a PostgreSQL database synchronized with ledger state.
## Development Workflow
### Typical Flow
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
WRITE[Write Daml] --> COMPILE[Compile]
COMPILE --> TEST[Test locally]
TEST --> ITERATE[Iterate]
ITERATE --> WRITE
TEST --> DEPLOY[Deploy to DevNet]
DEPLOY --> VALIDATE[Validate]
VALIDATE --> PROMOTE[Promote to TestNet]
```
### Steps
1. **Write** Daml contracts defining your business logic
2. **Compile** with `daml build`
3. **Test** locally with Daml Script or LocalNet
4. **Build** backend integration
5. **Deploy** to DevNet for integration testing
6. **Promote** through TestNet to MainNet
## QuickStart Project
The [cn-quickstart](https://github.com/digital-asset/cn-quickstart) repository provides a complete example that includes build tooling:
| Component | Technology |
| ------------------ | ----------------------- |
| **Contracts** | Daml |
| **Backend** | TypeScript |
| **Frontend** | React |
| **Infrastructure** | Docker Compose LocalNet |
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Clone and run
git clone https://github.com/digital-asset/cn-quickstart
cd cn-quickstart
direnv allow
cd quickstart
make install-daml-sdk
make setup
make build
make start
```
## Tool Comparison with Other Platforms
| Purpose | Ethereum | Canton |
| ------------------- | -------------------- | ------------------------ |
| **Smart contracts** | Solidity | Daml |
| **Build tool** | Hardhat/Foundry | daml build/dpm |
| **IDE** | Remix, VS Code | VS Code + Daml extension |
| **Testing** | Mocha, Foundry tests | Daml Script |
| **Local network** | Hardhat node, Anvil | LocalNet, Canton Sandbox |
| **API** | JSON-RPC | Ledger API (gRPC/JSON) |
| **Indexing** | The Graph | PQS |
## Next Steps
Run the example application.
Start writing smart contracts.
# Mental Models for Canton
Source: https://docs.canton.network/appdev/modules/m1-mental-models
Building the right intuition for Canton development
Effective Canton development requires the right mental models. This page helps you build intuition that will make everything else click.
## The Private Database Network
Think of Canton not as a blockchain, but as a network of private blockchains or databases that can synchronize.
### The Model
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph Network[Canton Network]
subgraph VA[Alice's Validator]
DBA[(Alice's Data)]
end
subgraph VB[Bob's Validator]
DBB[(Bob's Data)]
end
subgraph VC[Charlie's Validator]
DBC[(Charlie's Data)]
end
SYNC[Synchronizer
Coordination Only]
end
VA <--> |synchronize| SYNC
VB <--> |synchronize| SYNC
VC <--> |synchronize| SYNC
```
**Key insight:** Each party's data stays in their validator. When Alice and Bob transact, only Alice and Bob's validators participate. Charlie's data is unaffected and Charlie doesn't even know it happened.
### Why This Matters
* **For queries**: You can only query your own data
* **For design**: Think about what data goes where
* **For privacy**: Data stays with the parties who should have it
## Contracts as Facts
Think of contracts not as code that executes, but as **facts** that exist.
### The Model
| Traditional View | Canton View |
| ----------------------- | ---------------------------------------- |
| "Contract has state X" | "Fact: Alice owns 100 tokens" |
| "Update state to Y" | "Archive old fact, create new fact" |
| "Contract at address Z" | "Fact identified by ID, may be archived" |
### Example
```
Action 1: "Alice creates a contract with 100 tokens" (exists)
Action 2: Alice transfers 30 to Bob
Fact 1: "Alice owns 100 tokens" (archived)
Fact 2: "Alice owns 70 tokens" (created)
Fact 3: "Bob owns 30 tokens" (created)
```
**Key insight:** You never modify facts. Old facts become history, new facts are created. This gives you an immutable audit trail and enables privacy guarantees.
## Authorization as Structure
Think of authorization not as code you write, but as **structure** you declare.
### Traditional Authorization
```
function doThing() {
if (msg.sender != owner) revert();
// do thing
}
```
The code checks authorization at runtime. If you forget the check, anyone can call it.
### Canton Authorization
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice DoThing : Result
controller owner -- Declaration, not code
do
-- Only owner can reach here
```
**Key insight:** Authorization is declared in the type system. The protocol enforces it. You can't forget to check because there's nothing to check.
## Views as Windows
Think of transactions as having **windows** (views) that different parties look through.
### The Model
Imagine a transaction as a building with different windows:
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph TX[Transaction Building]
CENTER[Full Transaction
All actions]
end
WA[Alice's Window
Sees: her part]
WB[Bob's Window
Sees: his part]
WC[Charlie's Window
Sees: nothing]
TX --> WA
TX --> WB
TX --> WC
```
Each party looks through their window and sees only what they're entitled to see. The building (transaction) is the same, but the views are different.
**Key insight:** Privacy isn't about hiding data—it's about each party having their own view of shared reality.
## Parties as Stakeholders
Think of parties not as "accounts" but as **stakeholders** with specific roles.
### Roles
| Role | Meaning | Analogy |
| -------------- | --------------------------- | ------------------------------ |
| **Signatory** | Must authorize, always sees | Signer on a legal document |
| **Observer** | Can see, can't act | CC'd on an email |
| **Controller** | Can execute specific action | Has the key to a specific door |
### Example: Loan Agreement
```
Loan Contract:
- Signatory: Bank (must authorize the loan)
- Signatory: Borrower (must authorize the loan)
- Observer: Auditor (can see the loan)
- Controller for Repay: Borrower (only they can repay)
- Controller for Foreclose: Bank (only they can foreclose)
```
**Key insight:** Each party has a specific relationship to the contract. This relationship determines what they can see and do.
## The Synchronizer as a Post Office
Think of the synchronizer not as a blockchain, but as a **post office**.
### The Model
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
A[Alice] --> |sealed letter| PO[Post Office]
PO --> |deliver| B[Bob]
Note[Post office:
- Delivers mail
- Orders delivery
- Doesn't read letters]
```
The post office:
* Receives sealed letters (encrypted messages)
* Puts them in order
* Delivers them to recipients
* Never opens or reads them
**Key insight:** The synchronizer is infrastructure, not a validator. It can't see what you're doing, only that you're doing something.
## Transactions as Proposals
Think of multi-party transactions as **proposals** that require acceptance.
### The Model
When Alice wants to create a contract that Bob must also sign:
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
sequenceDiagram
Alice->>System: Create Proposal
Note over System: Proposal exists
Only Alice signed
System->>Bob: Notify: proposal waiting
Bob->>System: Accept Proposal
Note over System: Agreement created
Both signed
```
**Key insight:** Multi-party authorization requires explicit consent. You can't force someone into a contract—they must accept.
## Putting It Together
When building on Canton, keep these models in mind:
| Situation | Apply Model |
| ---------------------------- | --------------------------- |
| Designing data storage | Private Database Network |
| Designing state changes | Contracts as Facts |
| Designing permissions | Authorization as Structure |
| Designing privacy | Views as Windows |
| Designing parties | Parties as Stakeholders |
| Understanding infrastructure | Synchronizer as Post Office |
| Designing workflows | Transactions as Proposals |
## Common Misconceptions
| Misconception | Reality |
| ----------------------------------- | ------------------------------------------------ |
| "Canton is just another blockchain" | It's a fundamentally different architecture |
| "I can query all contracts" | You can only query your party's data |
| "Smart contracts have addresses" | Contracts have IDs that change on archive/create |
| "Validators see everything" | Validators see only their parties' data |
| "The synchronizer stores my data" | The synchronizer stores nothing |
## Next Steps
Understand the tools you'll use.
See how components work together technically.
# Module 1: Understanding Canton
Source: https://docs.canton.network/appdev/modules/m1-understanding-canton
Build the foundational understanding needed for Canton development
This module provides the conceptual foundation you need before writing Canton code. Even if you're eager to start coding, taking time to understand these concepts will make you more effective.
## Module Overview
| Section | Purpose |
| ---------------------- | ------------------------------------- |
| **Mental Models** | Build intuition for how Canton works |
| **Development Stack** | Understand the tools and technologies |
| **How Canton Differs** | See what makes Canton unique |
## Why This Module Matters
Canton is not just another blockchain with different syntax. It represents a fundamentally different approach to distributed ledgers:
* **Privacy is native**, not bolted on
* **Consensus is targeted**, not global
* **State is distributed**, not replicated
* **Authorization is declared**, not coded
Understanding these principles upfront will save you from fighting against the architecture later.
## Core Insights
### Privacy First
On most blockchains, you build an application and then try to add privacy. On Canton, you start with privacy and choose what to reveal and whom to reveal it to.
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
subgraph Traditional[Traditional Blockchain]
T1[Build app] --> T2[Add privacy layer]
T2 --> T3[Hope it works]
end
subgraph Canton[Canton]
C1[Design privacy] --> C2[Declare visibility]
C2 --> C3[Privacy enforced]
end
```
### No Global State
There is no single "blockchain" you can query for all information. Each party has their own view of the ledger.
| Traditional View | Canton Reality |
| ---------------------- | ----------------------------------------------------- |
| "Query the blockchain" | Query *your* data from *your* validator |
| "Total supply" | Only visible if the application exposes it via an API |
| "All transactions" | *Your* transactions only |
### Immutable Everything
Contracts don't change. When you "update" a contract, you archive the old one and create a new one. This isn't a limitation—it's the foundation of privacy and integrity guarantees.
### Explicit Authorization
You declare what each party can do at compile time, and the protocol enforces it. Unlike traditional systems where you check caller identity at runtime, Canton's authorization is structural.
## Prerequisites Check
Before proceeding, you should:
* **Understand** what Canton is ([Five-Minute Overview](/overview/understand/five-minute-overview))
* **Know** the basic components ([Core Concepts](/overview/understand/core-concepts))
* **Have** programming experience (any language)
No blockchain experience is required—and if you have it, be prepared to unlearn some things.
## What You'll Learn
By the end of this module, you'll understand:
1. How to think about Canton's privacy model
2. The relationship between parties, validators, and synchronizers
3. How transactions flow through the system
4. What tools you'll use for development
## Learning Path
Build intuition for Canton's approach to distributed ledgers.
Understand the tools and technologies you'll use.
After completing this module, continue to:
* **[Module 2](/appdev/modules/m2-canton-for-ethereum-devs)**: If you have Ethereum/blockchain experience
* **[Choose your path](/appdev/get-started/choose-your-path)**: If you're ready to start writing Daml
# Canton for Blockchain Developers
Source: https://docs.canton.network/appdev/modules/m2-canton-for-ethereum-devs
Bridge your Ethereum and blockchain knowledge to Canton concepts
If you're coming from Ethereum, Solana, or another blockchain platform, Canton will feel both familiar and fundamentally different. This section maps concepts you know to their Canton equivalents - and highlights where mental models must shift.
## Core Concept Mapping
The following table maps familiar Ethereum concepts to their Canton equivalents.
| Ethereum Concept | Canton Equivalent | Canton Notes |
| ----------------- | ----------------------- | --------------------------------------------------- |
| Blockchain | Synchronizer | Coordinates consensus without storing state |
| Smart Contract | Template | Defines data schema and executable choices |
| Contract Instance | Contract | Immutable; state changes create new contracts |
| Function | Choice | Actions that archive and/or create contracts |
| EOA (Address) | Party | Cryptographic identity with explicit permissions |
| Transaction | Transaction | Only entitled parties see their relevant views |
| Global State | Distributed State | No global state; each node stores its parties' data |
| Node | Validator (Participant) | Stores data only for its hosted parties |
| Gas | Traffic | Network usage fee paid in Canton Coin |
## The Mental Model Shift
**On Ethereum**: You write code that mutates global state. Everyone sees everything. Your contract sits at an address anyone can call.
**On Canton**: You write templates that define what data exists and what actions are possible. Contracts are created and archived (never mutated). Only relevant parties see the data. Authorization is built into the model, not bolted on.
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph EM[Ethereum Model]
TX1[Transaction] --> SC[Smart Contract]
SC --> GS[Global State]
GS --> N1[Node 1]
GS --> N2[Node 2]
GS --> N3[Node 3]
GS --> NN[Node N]
end
```
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph CM[Canton Model]
TX2[Transaction] --> TV[Transaction Views]
TV --> VA[View for Party A]
TV --> VB[View for Party B]
TV --> VC[View for Party C]
VA --> ValA[Validator hosting A]
VB --> ValB[Validator hosting B]
VC --> ValC[Validator hosting C]
end
```
## Smart Contract Paradigm: Templates vs Solidity
In Solidity, you define contracts with mutable state and functions that modify that state:
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Solidity: Mutable state
contract Token {
mapping(address => uint256) public balances;
function transfer(address to, uint256 amount) public {
balances[msg.sender] -= amount;
balances[to] += amount;
}
}
```
In Daml (Canton's smart contract language), you define templates with immutable contracts and choices that archive existing contracts and create new ones:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Daml: Immutable contracts with choices
template Token
with
owner : Party
issuer : Party
amount : Decimal
where
signatory issuer
observer owner
choice Transfer : ContractId Token
with
newOwner : Party
controller owner
do
create this with owner = newOwner
```
### Key Differences
These are the fundamental differences between Solidity and Daml programming models.
| Aspect | Solidity | Daml |
| ---------------------- | -------------------------------- | ---------------------------------------------------- |
| **State model** | Mutable storage variables | Immutable contracts; changes create new contracts |
| **Authorization** | Runtime `msg.sender` checks | Compile-time `signatory`/`controller` declarations |
| **Default visibility** | Public by default | Private by default; explicit `observer` declarations |
| **Execution control** | Anyone can call public functions | Only declared controllers can exercise choices |
The `Transfer` choice in Daml doesn't mutate the existing contract. It **archives** the current contract and **creates** a new one with the new owner. This immutability is fundamental to Canton's privacy and integrity guarantees.
## Privacy Model Differences
**Ethereum default**: Everything public.
**Canton default**: Everything private. Visibility requires explicit declaration.
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
sequenceDiagram
participant Alice
participant Bob
participant Charlie
participant Network
Note over Alice,Network: Ethereum: Everyone sees everything
Alice->>Network: Transfer 100 ETH to Bob
Network->>Alice: ✓ Confirmed
Network->>Bob: ✓ Confirmed
Network->>Charlie: ✓ Confirmed (sees full tx)
Note over Alice,Network: Canton: Only stakeholders see
Alice->>Network: Transfer 100 CC to Bob
Network->>Alice: ✓ Confirmed (sees full tx)
Network->>Bob: ✓ Confirmed (sees full tx)
Network--xCharlie: (sees nothing)
```
## Reading Data: No Global RPC
On Ethereum, any node can answer queries about any state. On Canton, **you must connect to the validator that hosts the party whose data you want**.
There is no single, all-encompassing blockchain RPC endpoint you can call to retrieve all data. Instead, you use your validator's Ledger API for your parties' data, and potentially an application provider's API for their data.
This is a direct consequence of privacy. If Charlie can't see Alice's data on-ledger, Charlie's node doesn't have Alice's data to query.
## Authorization Model
Authorization works fundamentally differently in Canton compared to Ethereum.
| Ethereum | Canton |
| ------------------------------------- | ------------------------------------------------------------------- |
| `msg.sender` determines authorization | `signatory` and `controller` determine authorization |
| Anyone can call any public function | Only specified parties can exercise choices |
| Authorization is runtime check | Authorization is compile-time guarantee and a run-time double check |
Canton's authorization model uses three key roles:
* **Signatory**: The party or parties that must authorize contract creation. Signatories always see the contract and can exercise choices if also declared as controller.
* **Observer**: A party permitted to see the contract but cannot exercise choices unless also declared as controller.
* **Controller**: The party permitted to execute a particular choice on a contract. Controllers see the contract when exercising.
### Authorization Example
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Asset
with
owner : Party
issuer : Party
auditor : Party
where
signatory issuer -- Must authorize creation; always sees contract
observer owner, auditor -- Can see but cannot act unless also controller
choice Transfer : ContractId Asset
with
newOwner : Party
controller owner -- Only owner can exercise this choice
do
create this with owner = newOwner
```
In this template:
* `issuer` must sign to create the contract (signatory)
* `owner` and `auditor` can see the contract (observers)
* Only `owner` can exercise the `Transfer` choice (controller)
Note that `controller` declarations are per-choice. A template can have multiple choices with different controllers:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Transfer : ContractId Asset
controller owner -- Only owner can transfer
do
create this with owner = newOwner
choice Audit : AuditReport
controller auditor -- Only auditor can audit
do
return AuditReport with ...
```
## Developer Tooling Comparison
Canton has equivalent tooling for most Ethereum development workflows.
| Ethereum Tool | Canton Equivalent | Notes |
| ------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Solidity | Daml | Functional vs imperative paradigm |
| Hardhat / Foundry | Daml SDK + dpm | Build, test, deploy toolchain |
| Remix | VS Code + Daml Extension | IDE with language support |
| MetaMask | Wallet SDK | User wallet integration |
| Web3.js / ethers.js | Ledger API (gRPC/JSON) | Application integration |
| ERC standards | CIPs | [CIP-0056](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md) (tokens), [CIP-0103](https://github.com/canton-foundation/cips/blob/main/cip-0103/cip-0103.md) (dApp standard) |
## Multi-Party Workflows
Canton treats multi-party coordination as a first-class concern. Where Ethereum requires manual coordination patterns, Canton builds them into the language.
### Ethereum Approach: Manual Multi-Sig
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Solidity: Manual signature collection
contract MultiSig {
mapping(address => bool) public approved;
uint256 public approvalCount;
function approve() public {
require(!approved[msg.sender], "Already approved");
approved[msg.sender] = true;
approvalCount++;
}
function execute() public {
require(approvalCount >= 2, "Need 2 approvals");
// ... execute action
}
}
```
### Canton Approach: Built-in Multi-Party
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Daml: Native multi-party agreement
template Agreement
with
partyA : Party
partyB : Party
terms : Text
where
signatory partyA, partyB -- Both must agree to create
choice Execute : ()
controller partyA, partyB -- Both must agree to execute
do
-- ... execute action
return ()
```
The Daml version is enforced at the protocol level. There's no way to create the contract without both signatures, and no way to exercise `Execute` without both parties.
Multi-party authorization in Canton requires collecting signatures from parties that may be hosted on different validators. This typically involves workflow patterns where one party proposes an action and others accept it. See the developer modules for detailed patterns on implementing multi-party workflows.
## What You'll Need to Unlearn
Coming from Ethereum, these habits will need to change.
| Ethereum Habit | Canton Reality |
| ----------------------------------------- | ----------------------------------------------------------- |
| **Query all state from any node** | Query from validators hosting relevant parties |
| **Mutate contract storage** | State changes create new contracts; old ones are archived |
| **Implicit authorization via msg.sender** | Explicit declaration of signatories and controllers |
| **Public by default** | Private by default; explicitly add observers for visibility |
| **Interchangeable nodes** | Validators store their hosted parties' state |
### The Four Mental Shifts
1. **No global state queries**: You can't query "all tokens" across the network
2. **Immutable contracts**: State changes create new contracts; old ones are archived
3. **Explicit authorization**: Every action requires explicit signatory/controller declarations
4. **Privacy by default**: You must opt-in to visibility, not opt-out
## Common Gotchas
These are the most common mistakes blockchain developers make when first building on Canton.
| Gotcha | Why It Happens | How to Avoid |
| ---------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------ |
| **Building public state lookups** | Expecting Ethereum-style global queries | Design for party-scoped queries from the start |
| **Forgetting multi-party authorization** | Ethereum's permissionless model | Always consider: who must sign? who can act? |
| **Trying to mutate contracts** | Solidity's mutable storage model | Embrace create/archive pattern |
| **Designing too many parties** | Ethereum addresses are free | Each party creates storage overhead; design party structure deliberately |
## Next Steps
* **[Architecture Overview](/overview/learn/architecture)** - Deep dive into Canton's component model
* **[Privacy Model Explained](/overview/learn/privacy-model)** - Understand sub-transaction privacy
* **[Developer Track Module 3: Daml Development](/appdev/modules/m3-dev-environment)** - Start writing Daml code
# Concept Translation Tables
Source: https://docs.canton.network/appdev/modules/m2-concept-translation
Comprehensive mapping of Ethereum concepts to Canton equivalents
This reference provides detailed translations from Ethereum concepts to their Canton equivalents. Use this as a quick reference when you encounter familiar terms and need to understand their Canton counterpart.
## Core Protocol Concepts
| Ethereum | Canton | Key Differences |
| ---------------- | ----------------------- | ------------------------------------------------------------------------------- |
| **Blockchain** | Synchronizer | Canton synchronizers coordinate without storing state |
| **Block** | No direct equivalent | Transactions are ordered individually; participants receive only relevant views |
| **Node** | Validator / Participant | Stores only hosted parties' data |
| **Global state** | Distributed state | No global view; each party has their view |
| **Finality** | Confirmation | Deterministic finality after confirmation |
| **Reorg** | Not applicable | Canton has no chain reorganizations |
## Identity & Accounts
| Ethereum | Canton | Key Differences |
| -------------------- | ------------------------- | --------------------------------------------- |
| **EOA (Address)** | Party | Parties have explicit authorization semantics |
| **Private key** | Party keys | Can be local (validator-held) or external |
| **msg.sender** | Controller | Declared at compile-time, not runtime |
| **Contract address** | Contract ID | Unique identifier, not derived from deployer |
| **ENS name** | Canton Name Service (CNS) | Human-readable party identifiers |
### Party vs. Address Deep Dive
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph Ethereum
A1[Address: 0x1234...]
A2[Derived from public key]
A3[No on-chain creation cost]
A4[Anonymous by default]
end
subgraph Canton
B1["Party: alice::fingerprint"]
B2[Registered with validator]
B3[Creates state on validator]
B4[Identity known to validator]
end
```
**Implications:**
* Don't create parties frivolously (unlike Ethereum addresses)
* Party creation requires interaction with a validator
* Parties are not pseudonymous in the same way as Ethereum addresses
## Smart Contract Concepts
| Ethereum | Canton | Key Differences |
| --------------------- | --------------- | --------------------------------------------- |
| **Smart contract** | Template | Daml defines types and choices |
| **Contract instance** | Contract | Immutable; state changes create new contracts |
| **Function** | Choice | Choices are typed and authorized |
| **Constructor** | `create` | Creates contract from template |
| **Storage variables** | Contract fields | Immutable within a contract instance |
| **ABI** | Daml types | Type-safe interface definition |
### State Model Comparison
**Ethereum: Mutable State**
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
contract Token {
mapping(address => uint256) balances;
function transfer(address to, uint256 amount) public {
balances[msg.sender] -= amount; // Mutate in place
balances[to] += amount;
}
}
```
**Canton: Immutable Contracts**
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Token
with
owner : Party
amount : Decimal
where
signatory owner
choice Transfer : ContractId Token
with newOwner : Party
controller owner
do
-- Archive this contract (implicit)
-- Create new contract with new owner
create this with owner = newOwner
```
| Aspect | Ethereum | Canton |
| ---------------- | ----------------------------- | ------------------------------ |
| **Modification** | Update storage variables | Archive old, create new |
| **History** | Implicit in state transitions | Explicit in contract lifecycle |
| **Atomicity** | Per-transaction | Per-transaction |
| **Rollback** | Revert state changes | Hidden as part of the protocol |
## Transaction Concepts
| Ethereum | Canton | Key Differences |
| -------------------- | --------------------- | ------------------------------------------ |
| **Transaction** | Transaction / Command | Canton transactions are privacy-preserving |
| **Transaction hash** | Transaction ID | Unique identifier |
| **Gas** | Traffic | Paid in Canton Coin |
| **Gas price** | Traffic cost | Based on transaction size and complexity |
| **Nonce** | Not required | Canton handles ordering |
| **Receipt** | Completion | Confirmation of command execution |
### Transaction Visibility
| Ethereum | Canton |
| ------------------------------------- | ------------------------------------------- |
| Transaction visible to all nodes | Transaction split into views |
| Everyone sees sender, recipient, data | Each party sees only their view |
| Public mempool | Encrypted submission |
| Block explorer shows everything | Block explorer shows your transactions only |
## Authorization Concepts
| Ethereum | Canton | Key Differences |
| ------------------ | --------------------------------- | ------------------------------ |
| **msg.sender** | Controller | Compile-time vs. runtime check |
| **require()** | Signatory/controller declarations | Enforced by protocol |
| **Ownable** | Signatory pattern | Built into the language |
| **Access control** | Observer/controller declarations | Explicit visibility control |
| **Multi-sig** | Multiple signatories | Native multi-party support |
### Authorization Example
**Ethereum: Runtime Check**
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
function transfer(address to, uint256 amount) public {
require(msg.sender == owner, "Not authorized");
// ... transfer logic
}
```
**Canton: Compile-Time Declaration**
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Transfer : ContractId Asset
controller owner -- Only owner can exercise this choice
do
-- No runtime check needed; protocol enforces
create this with owner = newOwner
```
## Event & Data Concepts
| Ethereum | Canton | Key Differences |
| ---------------------- | ------------------------- | --------------------------------- |
| **Events** | Transaction tree | Structured record of actions |
| **emit** | Implicit in transaction | All creates/archives are recorded |
| **Indexed parameters** | Contract fields | Queryable via Ledger API/PQS |
| **Logs** | Active Contract Set (ACS) | Current state queryable |
| **eth\_getLogs** | GetTransactionTrees | Historical transaction data |
## Network Concepts
| Ethereum | Canton | Key Differences |
| ------------------------- | ---------------------- | -------------------------- |
| **MainNet** | Canton Network MainNet | Production with real value |
| **Testnet (Sepolia)** | DevNet / TestNet | Test environments |
| **Local (Hardhat/Anvil)** | LocalNet | Development environment |
| **RPC endpoint** | Ledger API | gRPC or JSON interface |
| **Infura/Alchemy** | Validator service | Hosted infrastructure |
| **Chain ID** | Synchronizer ID | Network identifier |
### Environment Comparison
| Environment | Ethereum | Canton |
| --------------------- | ----------------------- | ----------------------------- |
| **Local development** | Hardhat, Anvil, Ganache | LocalNet (Daml SDK) |
| **Shared testing** | Sepolia, Goerli | DevNet (requires sponsorship) |
| **Pre-production** | Sepolia | TestNet (requires approval) |
| **Production** | MainNet | MainNet (full onboarding) |
## Development Tooling
| Ethereum | Canton | Notes |
| --------------------- | ------------------------ | --------------------------------------------- |
| **Solidity** | Daml | Different paradigm: functional vs. imperative |
| **Hardhat/Foundry** | dpm + daml build | Build and test toolchain |
| **Remix** | VS Code + Daml extension | IDE support |
| **ethers.js/web3.js** | Ledger API (gRPC/JSON) | Application integration |
| **MetaMask** | Wallet SDK | User wallet integration |
| **Etherscan** | Scan API | Network exploration |
| **The Graph** | PQS | Indexed data queries |
## Common Patterns
| Ethereum Pattern | Canton Equivalent |
| ------------------------ | ---------------------------------------------------------------------------------------------------------- |
| **ERC-20** | Token Standard ([CIP-0056](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md)) |
| **ERC-721/ERC-1155** | dApp Standard ([CIP-0103](https://github.com/mjuchli-da/cips/blob/cip-dapp-standard/cip-0103/cip-0103.md)) |
| **Proxy pattern** | Smart Contract Upgrade (SCU) |
| **Factory pattern** | Template instantiation |
| **Pull payments** | Propose/accept pattern |
| **Access control lists** | Observer lists |
| **Pausable** | Contract lifecycle choices |
| **Reentrancy guard** | Not needed (sequential execution) |
## What Doesn't Translate
Some Ethereum concepts have no direct Canton equivalent:
| Ethereum | Canton Reality |
| -------------------------------------- | ----------------------------- |
| **Public by default** | Private by default |
| **Any node can query any state** | Query only your parties' data |
| **Anonymous participation** | Parties have identity |
| **Permissionless contract deployment** | Packages vetted by validators |
| **Flash loans** | Different execution model |
| **MEV/front-running** | Transactions are encrypted |
## Next Steps
Deep dive into Canton's privacy model vs. Ethereum.
Understand the Daml vs. Solidity programming model.
# Migration Checklist
Source: https://docs.canton.network/appdev/modules/m2-migration-checklist
Practical checklist for Ethereum developers migrating to Canton
Use this checklist when planning and executing a migration from Ethereum to Canton. Each section covers a key area with specific action items.
## Before You Start
* [ ] **Identify state model dependencies**: Does your app rely on global state queries?
* [ ] **Map multi-party relationships**: Who are the parties in your contracts?
* [ ] **Evaluate privacy requirements**: Do you need sub-transaction privacy?
* [ ] **Review integration points**: What systems connect to your smart contracts?
If your app heavily relies on global state queries (e.g., "show all NFTs"), you'll need to redesign this functionality using party-scoped queries or off-ledger indexing.
| Area | Ethereum Assumption | Canton Reality |
| ------------- | -------------------- | ---------------------------- |
| State | Global, queryable | Distributed, party-scoped |
| Contracts | Mutable | Immutable (archive + create) |
| Authorization | Runtime `msg.sender` | Compile-time signatories |
| Privacy | Public by default | Private by default |
| Identity | Anonymous addresses | Named parties |
* [ ] Install the [Daml SDK](/sdks-tools/sdks/daml-sdk)
* [ ] Install VS Code with the [Daml extension](https://marketplace.visualstudio.com/items?itemName=DigitalAssetHoldingsLLC.daml)
* [ ] Clone the [CN Quickstart](/sdks-tools/reference-projects/cn-quickstart)
* [ ] Run `make setup && make build && make start` to verify your setup works
## Smart Contract Migration
### Phase 1: Model Your Parties
List everyone who interacts with your contracts and map them to Canton parties:
* **Contract owner** → `admin : Party` (signatory for admin contracts)
* **Users** → `user : Party` (each user is a distinct party)
* **Operators** → `operator : Party` (may be signatory or controller)
For each contract, determine:
* **Who must agree to create it?** → Signatories
* **Who should see it?** → Observers
* **Who can act on it?** → Controllers (per choice)
Decide where parties will be hosted:
* **User parties**: Often on the application provider's validator
* **Admin parties**: On your organization's validator
* **External parties**: On their own validators
### Phase 2: Translate Contracts
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Solidity
struct Asset {
address owner;
uint256 value;
bool locked;
}
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Daml
data Asset = Asset with
owner : Party
value : Decimal
locked : Bool
deriving (Eq, Show)
```
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Solidity: Global mapping
mapping(address => uint256) public balances;
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Daml: Individual contracts per holding
template TokenHolding
with
owner : Party
issuer : Party
amount : Decimal
where
signatory issuer
observer owner
```
The key insight: Instead of one contract with a mapping, you have many contracts—one per holding.
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Solidity
function transfer(address to, uint256 amount) public {
require(balances[msg.sender] >= amount);
balances[msg.sender] -= amount;
balances[to] += amount;
}
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Daml
choice Transfer : ContractId TokenHolding
with
newOwner : Party
controller owner
do
create this with owner = newOwner
```
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Solidity
modifier onlyOwner() {
require(msg.sender == owner);
_;
}
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Daml: No modifier needed
choice AdminAction : ()
controller admin -- Only admin can exercise
do
-- Protocol enforces this
pure ()
```
### Phase 3: Handle State Differently
* [ ] **Replace global queries**: Design party-scoped query patterns
* [ ] **Plan for contract keys**: Use keys for lookups instead of addresses
* [ ] **Accept contract ID changes**: IDs change on every update; use keys for stable references
Canton has no equivalent to Ethereum's `eth_getLogs` or event filtering across all contracts. Plan your indexing strategy early.
## Integration Migration
### API Changes
| Ethereum | Canton | Notes |
| ------------------- | ------------------------------------------------------------- | --------------------------------- |
| JSON-RPC | Ledger API (gRPC or JSON API) | Different protocol |
| Web3.js / ethers.js | Ledger API client or [dApp SDK](/appdev/modules/m4-sdks-apis) | Language-specific clients |
| Event logs | Transaction streams | Subscribe to party's transactions |
| `eth_call` | Exercise non-consuming choice | Read-only operations |
### Authentication Changes
Ethereum authenticates via private key signatures and derives identity from `msg.sender`. In Canton, identity comes from party authentication with JWT tokens. Where Ethereum allows anyone to call a public function, Canton restricts access to authenticated parties only.
### Indexing Strategy
Since there's no global state query:
* [ ] **Use the Participant Query Store (PQS)** for SQL-based querying
* [ ] **Stream transactions** to your own database
* [ ] **Design contracts with query-friendly keys**
## Testing Migration
### Unit Tests
* [ ] Replace Hardhat/Foundry tests with Daml Script tests
* [ ] Test authorization rules (who can exercise what)
* [ ] Test multi-party workflows (propose-accept patterns)
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Daml Script test
testTransfer = script do
alice <- allocateParty "Alice"
bob <- allocateParty "Bob"
-- Alice creates a token
tokenId <- submit alice do
createCmd Token with owner = alice, issuer = alice, amount = 100.0
-- Alice transfers to Bob
newTokenId <- submit alice do
exerciseCmd tokenId Transfer with newOwner = bob
-- Verify Bob owns it
Some token <- queryContractId bob newTokenId
assert (token.owner == bob)
```
### Integration Tests
* [ ] Test against LocalNet (CN Quickstart provides this)
* [ ] Verify API integration works
* [ ] Test error scenarios and rollback behavior
## Deployment Migration
### Network Selection
* **LocalNet** — development and testing
* **DevNet** — integration testing
* **TestNet** — pre-production validation
* **MainNet** — production
### Deployment Checklist
* [ ] **Package your Daml code**: `dpm build`
* [ ] **Upload to validator**: Via admin or LAPI APIs
* [ ] **Allocate parties**: Create party identities
* [ ] **Initialize contracts**: Create initial state
* [ ] **Configure applications**: Point to Ledger API
## Operational Changes
### Monitoring
Where Ethereum has block explorers, Canton uses validator logs and metrics. Transaction hash lookups become transaction ID queries. Gas tracking maps to traffic unit tracking.
### Key Management
Ethereum EOA private keys correspond to Canton party keys, which can live on the validator or be held externally. Hardware wallets map to HSM and KMS integrations. Multisig wallets are replaced by multi-signatory contracts, where authorization is enforced at the Daml level.
## Common Migration Pitfalls
Avoid these common mistakes when migrating from Ethereum.
| Pitfall | Why It Happens | How to Avoid |
| ------------------------------- | -------------------------------------- | ----------------------------------------------- |
| **Building global state views** | Expecting Ethereum-like queries | Design for party-scoped data from start |
| **Overusing parties** | Treating parties like addresses | Parties have hosting costs; design deliberately |
| **Ignoring propose-accept** | Expecting unilateral contract creation | Multi-party contracts need workflow |
| **Expecting event logs** | Using Ethereum event patterns | Use transaction streams and indexing |
| **Contract ID as identifier** | Treating IDs like addresses | Use contract keys for stable references |
## Migration Phases
| Phase | Focus | Completion Criteria |
| ------------------ | ------------------------- | ---------------------------------- |
| **1. Learning** | Understand Canton model | Team comfortable with Daml basics |
| **2. Design** | Redesign for Canton | Party model and contracts designed |
| **3. Development** | Implement in Daml | Contracts and tests complete |
| **4. Integration** | Connect applications | APIs working on LocalNet |
| **5. Testing** | Validate on test networks | Passed DevNet/TestNet testing |
| **6. Deployment** | Go to production | Running on MainNet |
## Next Steps
Start building with Daml.
Get hands-on with a working example.
# Multi-Party Workflows
Source: https://docs.canton.network/appdev/modules/m2-multi-party-workflows
How Canton handles multi-party workflows differently than Ethereum
Multi-party workflows are where Canton's architecture shines compared to Ethereum. This page covers the key patterns and how to think about them differently.
## The Core Difference
On Ethereum, multi-party agreement is a **pattern you implement**. On Canton, it's a **protocol guarantee**.
| Aspect | Ethereum | Canton |
| ---------------------- | --------------------------------------------- | -------------------------------------------------- |
| **Multi-sig creation** | Deploy contract, collect signatures over time | Collect signatures over time or submit all at once |
| **Authorization** | Runtime mapping checks | Protocol-level enforcement |
| **Atomicity** | Manual state machine | Built-in all-or-nothing |
| **Visibility** | All parties see everything | Each party sees only their view |
## The Propose-Accept Pattern
Since Canton requires all signatories to authorize contract creation, you can't create a multi-party contract unilaterally. The standard pattern is **propose-accept**:
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
sequenceDiagram
participant Alice
participant Bob
participant Ledger
Alice->>Ledger: Create Proposal (signatory: Alice, observer: Bob)
Note over Ledger: Proposal exists
Bob can see it
Bob->>Ledger: Exercise Accept on Proposal
Note over Ledger: Proposal archived
Agreement created (signatory: Alice, Bob)
```
### In Daml
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Step 1: Alice creates a proposal
template TradeProposal
with
proposer : Party
counterparty : Party
asset : Text
price : Decimal
where
signatory proposer
observer counterparty -- Counterparty can see the proposal
choice Accept : ContractId Trade
controller counterparty -- Only counterparty can accept
do
create Trade with
buyer = counterparty
seller = proposer
asset
price
choice Withdraw : ()
controller proposer -- Proposer can cancel
do pure ()
-- Step 2: Acceptance creates the multi-party contract
template Trade
with
buyer : Party
seller : Party
asset : Text
price : Decimal
where
signatory buyer, seller -- Both must have agreed
```
### Compare to Ethereum
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Ethereum: Manual approval tracking
contract TradeEscrow {
address public buyer;
address public seller;
bool public buyerApproved;
bool public sellerApproved;
function approve() public {
if (msg.sender == buyer) buyerApproved = true;
if (msg.sender == seller) sellerApproved = true;
}
function execute() public {
require(buyerApproved && sellerApproved, "Not approved");
// Execute trade...
}
}
```
The Canton version:
* Authorization is enforced by the protocol, not by application-level checks
* State transitions are atomic, so partial or inconsistent states don't arise
* Visibility is automatically scoped to the involved parties
## Delegation Patterns
Canton supports sophisticated delegation where one party grants another the ability to act on their behalf.
### Controller Delegation
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Asset
with
owner : Party
delegate : Optional Party -- Optional delegate
where
signatory owner
choice Transfer : ContractId Asset
with newOwner : Party
controller case delegate of
Some d -> d -- Delegate can act if set
None -> owner -- Otherwise owner acts
do
create this with owner = newOwner
choice SetDelegate : ContractId Asset
with newDelegate : Party
controller owner
do
create this with delegate = Some newDelegate
```
### Delegation via Separate Contract
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Delegation authority as a separate contract
template DelegationAuthority
with
principal : Party -- Who grants authority
agent : Party -- Who receives authority
scope : [Text] -- What actions are allowed
where
signatory principal
observer agent
nonconsuming choice ActOnBehalf : ()
with action : Text
controller agent
do
assertMsg "Action not in scope" (action `elem` scope)
-- Perform delegated action...
pure ()
```
## Multi-Step Workflows
For workflows requiring multiple parties in sequence:
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
subgraph Step1[Step 1: Proposal]
A[Alice proposes]
end
subgraph Step2[Step 2: Approval]
B[Bob approves]
end
subgraph Step3[Step 3: Execution]
C[Charlie settles]
end
Step1 --> Step2 --> Step3
```
### Workflow State Machine
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data WorkflowState
= Proposed
| Approved
| Settled
template Workflow
with
initiator : Party
approver : Party
settler : Party
state : WorkflowState
payload : Text
where
signatory initiator
observer approver, settler
choice Approve : ContractId Workflow
controller approver
do
assertMsg "Must be in Proposed state" (state == Proposed)
create this with state = Approved
choice Settle : ContractId Workflow
controller settler
do
assertMsg "Must be in Approved state" (state == Approved)
create this with state = Settled
```
## Atomic Multi-Contract Operations
Canton can atomically update multiple contracts in a single transaction:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice ExecuteSwap : ()
with
assetA : ContractId Asset
assetB : ContractId Asset
controller buyer, seller
do
-- Both happen atomically or neither does
exercise assetA Transfer with newOwner = buyer
exercise assetB Transfer with newOwner = seller
```
### Why This Matters
On Ethereum, atomic swaps require:
* Escrow contracts
* Time-locked phases
* Failure recovery logic
* Careful reentrancy protection
On Canton, atomicity is **guaranteed by the protocol**. If any part fails, nothing happens.
## Privacy in Multi-Party Workflows
Each party only sees their relevant portion:
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph TX[Transaction]
E1[Alice accepts Proposal from Bob]
E1 --> E2[Create Trade between Alice and Bob]
E2 --> E3[Transfer payment from Alice to Bob]
E3 --> E4[Notify Alice's bank]
end
TX --> VA[Alice sees: all]
TX --> VB[Bob sees: Trade, payment received]
TX --> VK[Bank sees: notification only]
```
## Common Workflow Patterns
| Pattern | Use Case | Key Feature |
| ------------------------- | --------------------- | ----------------------------- |
| **Propose-Accept** | Two-party agreements | Simple, clear consent |
| **Propose-Accept-Settle** | Three-party workflows | Sequential authorization |
| **Delegation** | Acting on behalf | Controlled authority transfer |
| **Escrow** | Conditional execution | Atomic swap guarantee |
| **Voting** | Group decisions | Threshold-based approval |
### Voting Example
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Vote
with
proposal : Text
voters : [Party]
votes : [(Party, Bool)]
threshold : Int
where
signatory (map fst votes)
observer voters
choice CastVote : ContractId Vote
with
voter : Party
approve : Bool
controller voter
do
assertMsg "Not a voter" (voter `elem` voters)
assertMsg "Already voted" (voter `notElem` map fst votes)
create this with votes = (voter, approve) :: votes
choice TallyAndExecute : ()
controller voters
do
let approvals = length [v | (_, v) <- votes, v]
assertMsg "Threshold not met" (approvals >= threshold)
-- Execute the proposal...
pure ()
```
## Related Topics
Practical checklist for migrating from Ethereum.
Start writing Daml smart contracts.
# Network Architecture Comparison
Source: https://docs.canton.network/appdev/modules/m2-network-architecture
Understanding how Canton's network architecture differs from Ethereum
Canton's network architecture is fundamentally different from Ethereum. This page compares the architectures to help you understand the implications for your applications.
## High-Level Architecture
### Ethereum Architecture
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph Ethereum[Ethereum Network]
N1[Node 1
Full state]
N2[Node 2
Full state]
N3[Node 3
Full state]
NN[Node N
Full state]
N1 <--> N2
N2 <--> N3
N3 <--> NN
N1 <--> N3
end
User1[User A] --> N1
User2[User B] --> N2
User3[User C] --> N3
```
**Key characteristics:**
* All nodes store all state
* Any node can answer any query
* Consensus requires all validators
* Horizontal scaling limited
### Canton Architecture
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph Canton[Canton Network]
SYNC[Synchronizer
Coordination only]
V1[Validator A
Alice's data]
V2[Validator B
Bob's data]
V3[Validator C
Charlie's data]
V1 <--> SYNC
V2 <--> SYNC
V3 <--> SYNC
end
UserA[Alice] --> V1
UserB[Bob] --> V2
UserC[Charlie] --> V3
```
**Key characteristics:**
* Validators store only their parties' data
* Synchronizer coordinates without storing
* Consensus involves only affected parties
* A party can be hosted on multiple validators (multihosting)
## Canton Components
### Validators
Validators store only their parties' data and answer queries only for hosted parties. In consensus, they validate only transactions affecting their parties.
### Synchronizers
Synchronizers coordinate transaction ordering without storing state. They ensure all affected parties see consistent ordering.
### Key Differences from Traditional Blockchains
In Canton:
* There is no global state—each party has their own view
* State visibility is private to stakeholders
* Only hosting validators can answer queries for their parties
* Finality is deterministic after confirmation
## Data Flow Comparison
### Ethereum Transaction Flow
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
sequenceDiagram
participant User
participant Mempool
participant Validators
participant Network
User->>Mempool: Submit transaction
Note over Mempool: Visible to all
Validators->>Mempool: Select transactions
Validators->>Validators: Execute transaction
Validators->>Network: Broadcast block
Network->>Network: All nodes update state
```
### Canton Transaction Flow
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
sequenceDiagram
participant User
participant ValidatorA as Validator A (Alice)
participant Sync as Synchronizer
participant ValidatorB as Validator B (Bob)
User->>ValidatorA: Submit command
ValidatorA->>ValidatorA: Execute Daml, create views
ValidatorA->>Sync: Submit encrypted
Sync->>Sync: Order, distribute
Sync->>ValidatorA: Alice's view
Sync->>ValidatorB: Bob's view
ValidatorA->>Sync: Confirm
ValidatorB->>Sync: Confirm
Sync->>ValidatorA: Commit
Sync->>ValidatorB: Commit
```
## Query Architecture
### Ethereum: Global Queries
```javascript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Any node can answer any query
const totalSupply = await token.totalSupply();
const aliceBalance = await token.balanceOf(aliceAddress);
const bobBalance = await token.balanceOf(bobAddress);
const allTransfers = await getTransferEvents();
```
### Canton: Party-Scoped Queries
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Query only your party's data via your validator
const myContracts = await ledgerApi.getActiveContracts({
party: myParty,
templateId: "Token"
});
// Can't query other parties' balances
// Must be added as observer to see their data
```
| Query Type | Ethereum | Canton |
| ----------------------- | -------------- | -------------------------- |
| **My balance** | Query any node | Query my validator |
| **Other's balance** | Query any node | Must be observer |
| **Total supply** | Query any node | Only if designed to expose |
| **Transaction history** | Query any node | Only my transactions |
## Implications for Application Design
### Data Availability
| Consideration | Ethereum | Canton |
| ----------------- | -------------------- | -------------------------- |
| **Read replicas** | Any node | Only your validator |
| **Caching** | Cache any data | Cache only entitled data |
| **Analytics** | On-chain data public | Need explicit data sharing |
### User Experience
| Aspect | Ethereum | Canton |
| ----------------------- | --------------------- | ------------------------- |
| **Wallet connection** | Any RPC endpoint | Your validator's API |
| **Balance display** | Query public state | Query your contracts |
| **Transaction history** | Public block explorer | Personal transaction view |
### Scaling
Canton's architecture naturally distributes load because validators only process transactions affecting their hosted parties. Additional capacity can be achieved by distributing parties across more validators.
## Network Topology
### Ethereum: Flat P2P
All nodes are peers, all store the same data.
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
N1((Node 1)) <--> N2((Node 2))
N2 <--> N3((Node 3))
N3 <--> N4((Node 4))
N4 <--> N1
N1 <--> N3
N2 <--> N4
```
### Canton: Hierarchical
Synchronizer coordinates, validators serve parties.
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph SyncLayer[Coordination Layer]
SV1[Super Validator 1]
SV2[Super Validator 2]
SV3[Super Validator 3]
end
subgraph ValidatorLayer[Validator Layer]
V1[Validator A]
V2[Validator B]
V3[Validator C]
end
subgraph UserLayer[Application Layer]
App1[App 1]
App2[App 2]
end
SV1 <--> SV2
SV2 <--> SV3
SV3 <--> SV1
V1 <--> SyncLayer
V2 <--> SyncLayer
V3 <--> SyncLayer
App1 --> V1
App2 --> V2
```
## Trust Model Comparison
| Entity | Ethereum Trust | Canton Trust |
| --------------------- | ------------------------------ | ------------------------------------ |
| **Validators** | See all, validate all | See only relevant, validate relevant |
| **Network operators** | Block producers see everything | Synchronizer sees nothing |
| **Your node** | You trust your node | You trust your validator |
| **Other users** | Compete for block space | Independent transactions |
## Migration Considerations
When migrating from Ethereum to Canton:
| Aspect | Action Required |
| ---------------------------- | -------------------------------------- |
| **State queries** | Redesign for party-scoped queries |
| **Analytics** | Build explicit data sharing if needed |
| **Node infrastructure** | Set up validator, not full node |
| **User onboarding** | Connect users to your validator |
| **Third-party integrations** | Access Ledger API via gRPC or JSON API |
## Next Steps
Detailed Canton architecture documentation.
Begin writing Daml smart contracts.
# Privacy Model Differences
Source: https://docs.canton.network/appdev/modules/m2-privacy-differences
Understanding how Canton's privacy model differs fundamentally from Ethereum
Privacy is the most significant architectural difference between Canton and Ethereum. This page explains the differences in depth and provides guidance for adapting your thinking.
## The Fundamental Difference
| Aspect | Ethereum | Canton |
| ---------------------- | ----------------------------- | ----------------------------------------- |
| **Default visibility** | Public | Private |
| **Approach** | Add privacy on top | Build privacy in |
| **Data location** | All nodes store all data | Nodes store only relevant data |
| **Opt-in model** | Opt-in to privacy (difficult) | Opt-in to visibility (explicit observers) |
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph Ethereum[Ethereum Default]
TX1[Transaction] --> ALL[All Validators]
ALL --> PUBLIC[Public State
Everyone sees everything]
end
subgraph Canton[Canton Default]
TX2[Transaction] --> VIEWS[Split by stakeholder]
VIEWS --> A[Party A's view
A sees only their data]
VIEWS --> B[Party B's view
B sees only their data]
end
```
## Default Visibility Comparison
### Ethereum: Public by Default
On Ethereum, when you deploy a contract or send a transaction:
1. **Transaction data** is visible in the mempool before execution
2. **State changes** are recorded on every node
3. **Historical data** is queryable by anyone forever
4. **Contract code** is readable by anyone
5. **All interactions** are visible to all observers
Even "private" variables in Solidity are not private—they're just not exposed via the ABI. Anyone can read storage slots directly.
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Solidity: "private" is misleading
contract NotReallyPrivate {
uint256 private secretValue; // Anyone can read this from storage
}
```
### Canton: Private by Default
On Canton, data is visible only to entitled parties:
1. **Transaction content** is encrypted during submission
2. **Views** are delivered only to relevant validators
3. **State** is stored only on validators hosting stakeholders
4. **Contract data** is visible only to signatories and observers
5. **Third parties** see nothing unless explicitly declared
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Daml: Visibility is explicit
template TrulyPrivate
with
owner : Party
secretValue : Decimal
where
signatory owner
-- No observers: only owner sees this contract
```
## Sub-Transaction Privacy
Canton's key innovation is **sub-transaction privacy**: different parts of the same transaction are visible to different parties.
### Example: Atomic Three-Party Transfer
Consider: Alice pays Bob with Canton Coin, and Bob simultaneously pays Charlie.
**On Ethereum:**
* Everyone sees: Alice paid Bob, Bob paid Charlie
* Everyone knows: amounts, parties, timing
* Carol (unrelated) can see all details
**On Canton:**
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
subgraph TX[Single Atomic Transaction]
L1[Leg 1: Alice → Bob]
L2[Leg 2: Bob → Charlie]
end
subgraph Views[Views by Party]
VA["Alice sees:
- Leg 1 (her payment)
- Not Leg 2 or Charlie"]
VB["Bob sees:
- Leg 1
- Leg 2
- (both, he's involved)"]
VC["Charlie sees:
- Leg 2 (his receipt)
- Not Leg 1 or Alice"]
VD["Carol sees:
Nothing"]
end
TX --> Views
```
### Why This Matters
| Use Case | Ethereum Problem | Canton Solution |
| ---------------- | ------------------------------------ | -------------------------------- |
| **Trading** | Competitors see your trades | Only counterparty sees trade |
| **Lending** | Loan terms are public | Only borrower/lender see terms |
| **Supply chain** | All parties see all prices | Each party sees their portion |
| **Compliance** | Data visible to unauthorized parties | Regulator added as observer only |
## Comparison to Ethereum Privacy Solutions
### Layer 2 Solutions
**Approach:** Move transactions off MainNet; settle on-chain.
| Aspect | L2 Solutions | Canton |
| ----------------------- | --------------------------------- | ------------------------- |
| **Operator visibility** | L2 operator sees all transactions | Synchronizer sees nothing |
| **Data availability** | Must be available somewhere | Only to entitled parties |
| **Settlement** | Requires L1 confirmation | Immediate finality |
| **Trust model** | Trust L2 operator | Cryptographic privacy |
### Zero-Knowledge Solutions
**Approach:** Prove validity without revealing data.
Canton does not use zero-knowledge proofs. Instead, privacy is achieved through selective data distribution—parties only receive the transaction views they're entitled to see. This provides a simpler programming model without ZK circuit complexity or proof generation overhead.
### Private Channels (Hyperledger Fabric)
**Approach:** Separate channels for private transactions.
| Aspect | Private Channels | Canton |
| ----------------- | ------------------------------ | -------------------------- |
| **Granularity** | Channel-level | Sub-transaction level |
| **Cross-channel** | Complex to implement | Native atomic transactions |
| **Membership** | Must manage channel membership | Declared in contracts |
| **Flexibility** | Rigid channel structure | Dynamic per-transaction |
### Encryption at Rest
**Approach:** Encrypt data stored on-chain.
| Aspect | Encryption | Canton |
| ------------------ | ------------------------------- | ------------------------- |
| **Key management** | Complex key distribution | No shared keys needed |
| **Metadata** | Transaction patterns visible | Encrypted entirely |
| **Future risk** | May be decrypted later | Not stored to decrypt |
| **Computation** | Can't compute on encrypted data | Full computation on views |
## Private Ledger View
In Canton, each party has their own **private ledger view**—the collection of all contracts where they're a stakeholder.
### What This Means
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart TB
subgraph Network[Canton Network State]
C1[Contract 1
Signatories: A, B]
C2[Contract 2
Signatories: B, C]
C3[Contract 3
Signatories: A, C]
C4[Contract 4
Signatories: C, D]
end
subgraph PartyViews[Each Party's Ledger View]
VA["Alice sees:
Contract 1, Contract 3"]
VB["Bob sees:
Contract 1, Contract 2"]
VC["Charlie sees:
Contract 2, Contract 3, Contract 4"]
end
Network --> PartyViews
```
**Key implications:**
| On Ethereum | On Canton |
| ---------------------------------- | --------------------------------------------------------------- |
| Query all tokens: `getAllTokens()` | Query *your* tokens: `getMyContracts()` |
| Global state exists | No global state concept |
| Total supply is known | Total supply visible only if explicitly designed for visibility |
| Any node answers any query | Only your validator answers your queries |
### Querying Data
**Ethereum approach:**
```javascript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// Anyone can query any data
const totalSupply = await token.totalSupply();
const aliceBalance = await token.balanceOf(alice);
const bobBalance = await token.balanceOf(bob);
```
**Canton approach:**
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
// You can only query your own data
const myContracts = await ledgerApi.getActiveContracts({
templateId: "Token",
party: myParty // Only your party
});
// Can't query Bob's balance unless you're an observer
```
## What the Synchronizer Can and Cannot See
The synchronizer (sequencer + mediator) coordinates transactions but never sees content.
### Synchronizer Can See
* Encrypted message blobs (must deliver them)
* Message sizes (network routing)
* Timestamps (ordering)
* Confirmation results, such as committed or rejected (finality)
* Validator identifiers (routing)
* Party identifiers (for routing, but not end-user identity)
### Synchronizer Cannot See
* Transaction content (encrypted)
* Contract data (never transmitted to synchronizer)
* Choice being exercised (encrypted)
* Amounts, prices, terms (encrypted)
### Trust Implications
Canton's trust model differs fundamentally from traditional blockchains:
**What you trust the synchronizer for:**
* Ordering transactions fairly (not reordering to favor certain parties)
* Delivering messages to all entitled participants
* Availability (being online when you need to transact)
**What you don't need to trust the synchronizer for:**
* Your data privacy—it only sees encrypted blobs
* Transaction validation—your validator does this
* Correct execution—the protocol enforces this cryptographically
**What you trust your validator for:**
* Storing your contract data securely
* Executing transactions correctly on your behalf
* Not revealing your data to unauthorized parties
Your hosting validator sees all your party's data. Choose your validator carefully—this is a trust relationship similar to choosing a bank or custodian.
| Ethereum | Canton |
| -------------------------------------------- | ---------------------------------------------------- |
| Trust all validators not to censor | Trust synchronizer not to censor |
| All validators see everything | Synchronizer sees only encrypted views |
| MEV is possible (validators see pending txs) | MEV is significantly harder (transactions encrypted) |
| Front-running is structural | Front-running is much harder |
| Trust any node for queries | Trust only your validator for your data |
## Design Implications for Developers
### Rethinking State Queries
**Ethereum mindset:** "How do I query global state?"
**Canton mindset:** "What can my party see, and who else needs visibility?"
### Observer Pattern
When designing contracts, explicitly consider:
1. **Who are signatories?** They must authorize and always see.
2. **Who are observers?** They can see but not act.
3. **Who can exercise choices?** Controllers for each choice.
4. **What gets divulged?** Fetching contracts in transactions reveals them.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template DesignedForPrivacy
with
primaryParty : Party
counterparty : Party
auditor : Party -- Only if audit is required
details : SensitiveData
where
signatory primaryParty
observer counterparty -- Needs to see, but not auditor
-- Add auditor only when they need to see
choice AddAuditObserver : ContractId DesignedForPrivacy
with newAuditor : Party
controller primaryParty
do create this with auditor = newAuditor
```
### Privacy Checklist
When designing Canton applications:
| Question | Design Impact |
| ---------------------------------------- | ---------------------------------- |
| Who needs to see this contract? | Signatory + observer declarations |
| Who can act on this contract? | Controller declarations per choice |
| What happens when contracts are fetched? | Divulgence considerations |
| Do third parties need visibility? | Add observers explicitly |
| Should validators see everything? | Consider party hosting choices |
## Next Steps
Understand Daml's immutable contract model vs. Solidity.
Technical details of sub-transaction privacy.
# Smart Contract Paradigm Shift
Source: https://docs.canton.network/appdev/modules/m2-smart-contract-paradigm
Understanding the fundamental differences between Solidity and Daml programming models
Moving from Solidity to Daml requires a significant mental shift. This page explains the paradigm differences and how to adapt your thinking.
## Programming Model Comparison
| Aspect | Solidity | Daml |
| ---------------- | --------------------------- | ----------------------- |
| **Paradigm** | Imperative, object-oriented | Functional, declarative |
| **State** | Mutable storage | Immutable contracts |
| **Execution** | Sequential operations | Transaction trees |
| **Types** | Static with dynamic calls | Strongly typed, ADTs |
| **Side effects** | Unlimited | Controlled via monads |
## State Model: Mutable vs. Immutable
### Solidity: Mutable State
In Solidity, contracts have mutable storage that you modify directly:
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
contract Token {
mapping(address => uint256) public balances;
function transfer(address to, uint256 amount) public {
require(balances[msg.sender] >= amount, "Insufficient balance");
// Mutate state in place
balances[msg.sender] -= amount;
balances[to] += amount;
emit Transfer(msg.sender, to, amount);
}
}
```
**Mental model:** The contract is a persistent object with state you modify.
### Daml: Immutable Contracts
In Daml, contracts are immutable data. State changes create new contracts or archive existing contracts:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Token
with
owner : Party
issuer : Party
amount : Decimal
where
signatory issuer
observer owner
choice Transfer : ContractId Token
with
newOwner : Party
transferAmount : Decimal
controller owner
do
-- This contract will be archived
-- Create new contracts for the split
create Token with owner = newOwner, issuer, amount = transferAmount
create this with amount = amount - transferAmount
```
**Mental model:** Contracts are facts. Exercise archives the fact and creates new facts.
## UTXO vs. Account Model
### Ethereum: Account Model
* State is a global mapping of accounts to balances
* Transfers modify account entries
* Easy to query total balance
* Contention on popular accounts
### Canton: Extended UTXO Model
* State is a set of contracts (like unspent outputs)
* Transfers archive existing contracts, create new ones
* Balance is sum of owned contracts
* Better parallelism, explicit data flow
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Canton: Holdings are individual contracts
-- Alice's total balance = sum of all Token contracts where owner = Alice
-- Query: Find all my tokens
myTokens <- queryContractKey @Token myParty
totalBalance <- pure $ sum [amount | Token{amount} <- myTokens]
```
## Language Comparison
### Type System
| Feature | Solidity | Daml |
| ----------------- | ------------------ | ---------------------------- |
| **Type safety** | Moderate | Strong |
| **Null handling** | Implicit (0/empty) | Explicit (Optional) |
| **Custom types** | Structs, enums | ADTs, records |
| **Generics** | Limited | Full parametric polymorphism |
### Solidity Types
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
struct Asset {
address owner;
uint256 value;
bool isLocked;
}
enum State { Pending, Active, Completed }
```
### Daml Types
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Record type (like struct)
data Asset = Asset with
owner : Party
value : Decimal
isLocked : Bool
-- Sum type (algebraic data type)
data AssetState
= Pending
| Active with activatedAt : Time
| Completed with result : Text
-- Optional (explicit null handling)
data MaybeApprover = Some Party | None
```
### Solidity Control Flow
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
function process(uint256[] memory items) public {
for (uint i = 0; i < items.length; i++) {
if (items[i] > threshold) {
revert("Over threshold");
}
results[i] = items[i] * 2;
}
}
```
### Daml Control Flow
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
process : [Decimal] -> Update [Decimal]
process items = do
forA items \item -> do
assertMsg "Over threshold" (item <= threshold)
pure (item * 2.0)
```
## Authorization Model
### Solidity: Runtime Authorization
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
contract Ownable {
address public owner;
modifier onlyOwner() {
require(msg.sender == owner, "Not owner");
_;
}
function sensitiveAction() public onlyOwner {
// Only owner can call
}
}
```
**Issues:**
* Authorization checked at runtime
* Easy to forget checks
* Anyone can attempt the call
* Authorization mixed with logic
### Daml: Declarative Authorization
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template OwnedAsset
with
owner : Party
data : Text
where
signatory owner -- owner must authorize creation
choice SensitiveAction : ()
controller owner -- only owner can exercise
do
-- Protocol enforces: only owner can reach here
pure ()
```
**Benefits:**
* Authorization declared, not coded
* Impossible to forget (compiler enforces)
* Only authorized parties can attempt
* Clear separation of concerns
## Multi-Party Coordination
### Solidity: Manual Multi-Sig
```solidity theme={"theme":{"light":"github-light","dark":"github-dark"}}
contract MultiSig {
mapping(address => bool) public approved;
uint256 public approvalCount;
uint256 public requiredApprovals;
function approve() public {
require(!approved[msg.sender], "Already approved");
approved[msg.sender] = true;
approvalCount++;
}
function execute() public {
require(approvalCount >= requiredApprovals, "Not enough approvals");
// Execute action
}
}
```
### Daml: Native Multi-Party
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Agreement
with
partyA : Party
partyB : Party
terms : Text
where
signatory partyA, partyB -- Both must sign to create
-- Proposal pattern for gathering signatures
template AgreementProposal
with
proposer : Party
counterparty : Party
terms : Text
where
signatory proposer
observer counterparty
choice Accept : ContractId Agreement
controller counterparty
do create Agreement with
partyA = proposer
partyB = counterparty
terms
```
## Common Patterns Translated
| Solidity Pattern | Daml Equivalent |
| --------------------- | ----------------------------------------------------------------------------------------------------- |
| **Ownable** | Signatory declaration |
| **Pausable** | Contract archival + recreation |
| **ERC-20** | Token Standard ([CIP-0056](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md)) |
| **Proxy/Upgradeable** | Smart Contract Upgrade (SCU) |
| **Pull payment** | Propose/accept pattern |
| **Factory** | Template + create |
| **Registry** | Contract keys (when available) |
## What to Unlearn
| Solidity Habit | Daml Reality |
| -------------------------------- | ---------------------------------------------------- |
| Mutate state in place | Archive + create new contracts |
| Runtime `msg.sender` checks | Compile-time controller declarations |
| Public functions anyone can call | Only controllers can exercise |
| Global contract address | Contract IDs change on every update |
| Loops for iteration | Use `forA`, `mapA`, fold patterns |
| Try/catch everywhere | Assertions for validation; exceptions are deprecated |
## Next Steps
Compare network architecture and topology.
Start writing Daml smart contracts.
# Authorization Model
Source: https://docs.canton.network/appdev/modules/m3-authorization
Understand Daml's authorization model - signatories, observers, and controllers
# Parties and authority
Daml is designed for distributed applications involving mutually distrusting parties. In a well-constructed contract model, all parties have strong guarantees that nobody cheats or circumvents the rules laid out by templates and choices.
In this section you will learn about Daml's authorization rules and how to develop contract models that give all parties the required guarantees. In particular, you'll learn how to:
* Pass authority from one contract to another
* Write advanced choices
* Reason through Daml's Authorization model
Remember that you can load all the code for this section into a folder called `intro-parties` by running `dpm new intro-parties --template daml-intro-parties`
## Prevent IOU revocation
The `SimpleIou` contract from `choices` and `constraints` has one major problem: The contract is only signed by the `issuer`. The signatories are the parties with the power to create and archive contracts. If Alice gave Bob a `SimpleIou` for \$100 in exchange for some goods, she could just archive it after receiving the goods. Bob would have a record of such actions, but would have to resort to off-ledger means to get his money back:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template SimpleIou
with
issuer : Party
owner : Party
cash : Cash
where
signatory issuer
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
simple_iou_test = do
alice <- allocateParty "Alice"
bob <- allocateParty "Bob"
-- Alice and Bob enter into a trade.
-- Alice transfers the payment as a SimpleIou.
iou <- submit alice do
createCmd SimpleIou with
issuer = alice
owner = bob
cash = Cash with
amount = 100.0
currency = "USD"
passTime (days 1)
-- Bob delivers the goods.
passTime (minutes 10)
-- Alice just deletes the payment.
submit alice do
archiveCmd iou
```
For a party to have any guarantees that only those transformations specified in the choices are actually followed, they either need to be a signatory themselves, or trust one of the signatories to not agree to transactions that archive and re-create contracts in unexpected ways. To make the `SimpleIou` safe for Bob, you need to add him as a signatory:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Iou
with
issuer : Party
owner : Party
cash : Cash
where
signatory issuer, owner
choice Transfer
: ContractId Iou
with
newOwner : Party
controller owner
do
assertMsg "newOwner cannot be equal to owner." (owner /= newOwner)
create this with
owner = newOwner
```
There's a new problem here: There is no way for Alice to issue or transfer this `Iou` to Bob. To get an `Iou` with Bob's signature as `owner` onto the ledger, his authority is needed:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
iou_test = do
alice <- allocateParty "Alice"
bob <- allocateParty "Bob"
-- Alice and Bob enter into a trade.
-- Alice wants to give Bob an Iou, but she can't without Bob's authority.
submitMustFail alice do
createCmd Iou with
issuer = alice
owner = bob
cash = Cash with
amount = 100.0
currency = "USD"
-- She can issue herself an Iou.
iou <- submit alice do
createCmd Iou with
issuer = alice
owner = alice
cash = Cash with
amount = 100.0
currency = "USD"
-- However, she can't transfer it to Bob.
submitMustFail alice do
exerciseCmd iou Transfer with
newOwner = bob
```
This may seem awkward, but notice that the `ensure` clause is gone from the `Iou` again. The above `Iou` can contain negative values so Bob should be glad that `Alice` cannot put his signature on any `Iou`.
You'll now learn a couple of common ways of building issuance and transfer workflows for the above `Iou`, before diving into the authorization model in full.
## Use Propose-Accept workflow for one-off authorization
If there is no standing relationship between Alice and Bob, Alice can propose the issuance of an Iou to Bob, giving him the choice to accept. You can do so by introducing a proposal contract `IouProposal`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template IouProposal
with
iou : Iou
where
signatory iou.issuer
observer iou.owner
choice IouProposal_Accept
: ContractId Iou
controller iou.owner
do
create iou
```
Note how we have used the fact that templates are records here to store the `Iou` in a single field:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
iouProposal <- submit alice do
createCmd IouProposal with
iou = Iou with
issuer = alice
owner = bob
cash = Cash with
amount = 100.0
currency = "USD"
submit bob do
exerciseCmd iouProposal IouProposal_Accept
```
The `IouProposal` contract carries the authority of `iou.issuer` by virtue of them being a signatory. By exercising the `IouProposal_Accept` choice, Bob adds his authority to that of Alice, which is why an `Iou` with both signatories can be created in the context of that choice.
The choice is called `IouProposal_Accept`, not `Accept`, because propose-accept patterns are very common. In fact, you'll see another one just below. As each choice defines a record type, you cannot have two choices of the same name in scope. It's a good idea to qualify choice names to ensure uniqueness.
The above solves issuance, but not transfers. You can solve transfers exactly the same way, though, by creating a `TransferProposal`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template IouTransferProposal
with
iou : Iou
newOwner : Party
where
signatory (signatory iou)
observer (observer iou), newOwner
choice IouTransferProposal_Cancel
: ContractId Iou
controller iou.owner
do
create iou
choice IouTransferProposal_Reject
: ContractId Iou
controller newOwner
do
create iou
choice IouTransferProposal_Accept
: ContractId Iou
controller newOwner
do
create iou with
owner = newOwner
```
In addition to defining the signatories of a contract, `signatory` can also be used to extract the signatories from another contract. Instead of writing `signatory (signatory iou)`, you could write `signatory iou.issuer, iou.owner`.
The `IouProposal` had a single signatory so it could be cancelled easily by archiving it. Without a `Cancel` choice, the `newOwner` could abuse an open TransferProposal as an option. The triple `Accept`, `Reject`, `Cancel` is common to most proposal templates.
To allow an `iou.owner` to create such a proposal, you need to give them the choice to propose a transfer on the `Iou` contract. The choice looks just like the above `Transfer` choice, except that a `IouTransferProposal` is created instead of an `Iou`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice ProposeTransfer
: ContractId IouTransferProposal
with
newOwner : Party
controller owner
do
assertMsg "newOwner cannot be equal to owner." (owner /= newOwner)
create IouTransferProposal with
iou = this
newOwner
```
Bob can now transfer his `Iou`. The transfer workflow can even be used for issuance:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
charlie <- allocateParty "Charlie"
-- Alice issues an Iou using a transfer proposal.
tpab <- submit alice do
createCmd IouTransferProposal with
newOwner = bob
iou = Iou with
issuer = alice
owner = alice
cash = Cash with
amount = 100.0
currency = "USD"
-- Bob accepts the transfer from Alice.
iou2 <- submit bob do
exerciseCmd tpab IouTransferProposal_Accept
-- Bob offers Charlie a transfer.
tpbc <- submit bob do
exerciseCmd iou2 ProposeTransfer with
newOwner = charlie
-- Charlie accepts the transfer from Bob.
submit charlie do
exerciseCmd tpbc IouTransferProposal_Accept
```
## Use role contracts for ongoing authorization
Many actions, like the issuance of assets or their transfer, can be pre-agreed. You can represent this succinctly in Daml through relationship or role contracts.
Jointly, an `owner` and `newOwner` can transfer an asset, as demonstrated in the script above. In `compose`, you will see how to compose the `ProposeTransfer` and `IouTransferProposal_Accept` choices into a single new choice, but for now, here is a different way. You can give them the joint right to transfer an IOU:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Mutual_Transfer
: ContractId Iou
with
newOwner : Party
controller owner, newOwner
do
create this with
owner = newOwner
```
Up to now, the controllers of choices were known from the current contract. Here, the `newOwner` variable is part of the choice arguments, not the `Iou`.
This is also the first time we have shown a choice with more than one controller. If multiple controllers are specified, the authority of *all* the controllers is needed. Here, neither `owner`, nor `newOwner` can execute a transfer unilaterally, hence the name `Mutual_Transfer`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template IouSender
with
sender : Party
receiver : Party
where
signatory receiver
observer sender
nonconsuming choice Send_Iou
: ContractId Iou
with
iouCid : ContractId Iou
controller sender
do
iou <- fetch iouCid
assert (iou.cash.amount > 0.0)
assert (sender == iou.owner)
exercise iouCid Mutual_Transfer with
newOwner = receiver
```
The above `IouSender` contract now gives one party, the `sender` the right to send `Iou` contracts with positive amounts to a `receiver`. The `nonconsuming` keyword on the choice `Send_Iou` changes the behaviour of the choice so that the contract it's exercised on does not get archived when the choice is exercised. That way the `sender` can use the contract to send multiple Ious.
Here it is in action:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Bob allows Alice to send him Ious.
sab <- submit bob do
createCmd IouSender with
sender = alice
receiver = bob
-- Charlie allows Bob to send him Ious.
sbc <- submit charlie do
createCmd IouSender with
sender = bob
receiver = charlie
-- Alice can now send the Iou she issued herself earlier.
iou4 <- submit alice do
exerciseCmd sab Send_Iou with
iouCid = iou
-- Bob sends it on to Charlie.
submit bob do
exerciseCmd sbc Send_Iou with
iouCid = iou4
```
## Daml's authorization model
Hopefully, the above will have given you a good intuition for how authority is passed around in Daml. In this section you'll learn about the formal authorization model to allow you to reason through your contract models. This will allow you to construct them in such a way that you don't run into authorization errors at runtime, or, worse still, allow malicious transactions.
In `choices` you learned that a transaction is, equivalently, a tree of transactions, or a forest of actions, where each transaction is a list of actions, and each action has a child-transaction called its consequences.
Each action has a set of *required authorizers* -- the parties that must authorize that action -- and each transaction has a set of *authorizers* -- the parties that did actually authorize the transaction.
The authorization rule is that the required authorizers of every action are a subset of the authorizers of the parent transaction.
The required authorizers of actions are:
* The required authorizers of an **exercise action** are the controllers on the corresponding choice. Remember that `Archive` and `archive` are just an implicit choice with the signatories as controllers.
* The required authorizers of a **create action** are the signatories of the contract.
* The required authorizers of a **fetch action** (which also includes `fetchByKey`) are somewhat dynamic and covered later.
The authorizers of transactions are:
* The root transaction of a commit is authorized by the submitting party.
* The consequences of an exercise action are authorized by the actors of that action plus the signatories of the contract on which the action was taken.
### An authorization example
Consider the transaction from the script above where Bob sends an `Iou` to Charlie using a `Send_Iou` contract. It is authorized as follows, ignoring fetches:
* Bob submits the transaction so he's the authorizer on the root transaction.
* The root transaction has a single action, which is to exercise `Send_Iou` on a `IouSender` contract with Bob as `sender` and Charlie as `receiver`. Since the controller of that choice is the `sender`, Bob is the required authorizer.
* The consequences of the `Send_Iou` action are authorized by its actors, Bob, as well as signatories of the contract on which the action was taken. That's Charlie in this case, so the consequences are authorized by both Bob and Charlie.
* The consequences contain a single action, which is a `Mutual_Transfer` with Charlie as `newOwner` on an `Iou` with `issuer` Alice and `owner` Bob. The required authorizers of the action are the `owner`, Bob, and the `newOwner`, Charlie, which matches the parent's authorizers.
* The consequences of `Mutual_Transfer` are authorized by the actors (Bob and Charlie), as well as the signatories on the Iou (Alice and Bob).
* The single action on the consequences, the creation of an Iou with `issuer` Alice and `owner` Charlie has required authorizers Alice and Charlie, which is a proper subset of the parent's authorizers.
You can see the graph of this transaction in the transaction view of the IDE:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
TX 12 1970-01-01T00:00:00Z (Parties:276:3)
#12:0
│ disclosed to (since): 'Bob' (12), 'Charlie' (12)
└─> 'Bob' exercises Send_Iou on #10:0 (Parties:IouSender)
with
iouCid = #11:3
children:
#12:1
│ disclosed to (since): 'Bob' (12), 'Charlie' (12), 'Alice' (12)
└─> 'Alice' and 'Bob' fetch #11:3 (Parties:Iou)
#12:2
│ disclosed to (since): 'Bob' (12), 'Charlie' (12), 'Alice' (12)
└─> 'Bob' and 'Charlie' exercise Mutual_Transfer on #11:3 (Parties:Iou)
with
newOwner = 'Charlie'
children:
#12:3
│ disclosed to (since): 'Bob' (12), 'Charlie' (12), 'Alice' (12)
└─> 'Alice' and 'Charlie' create Parties:Iou
with
issuer = 'Alice';
owner = 'Charlie';
cash =
(Parties:Cash with
currency = "USD"; amount = 100.0000000000)
```
Note that authority is not automatically transferred transitively.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template NonTransitive
with
partyA : Party
partyB : Party
where
signatory partyA
observer partyB
choice TryA
: ContractId NonTransitive
controller partyA
do
create NonTransitive with
partyA = partyB
partyB = partyA
choice TryB
: ContractId NonTransitive
with
other : ContractId NonTransitive
controller partyB
do
exercise other TryA
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
nt1 <- submit alice do
createCmd NonTransitive with
partyA = alice
partyB = bob
nt2 <- submit alice do
createCmd NonTransitive with
partyA = alice
partyB = bob
submitMustFail bob do
exerciseCmd nt1 TryB with
other = nt2
```
The consequences of `TryB` are authorized by both Alice and Bob, but the action `TryA` only has Alice as an actor and Alice is the only signatory on the contract.
Therefore, the consequences of `TryA` are only authorized by Alice. Bob's authority is now missing to create the flipped `NonTransitive` so the transaction fails.
## Next up
In `compose` you will put everything you have learned together to build a simple asset holding and trading model akin to that in the `/app-dev/bindings-java/quickstart`. In that context you'll learn a bit more about the `Update` action and how to use it to compose transactions, as well as about privacy on Daml ledgers.
# Building and Packaging
Source: https://docs.canton.network/appdev/modules/m3-building-packaging
Build Daml projects, manage dependencies, and package DAR files for deployment
# Work with dependencies
The application from `compose` is a complete and secure model for atomic swaps of assets, but there is plenty of room for improvement. However, one can't implement all features before going live with an application so it's important to understand how to change already running code. There are fundamentally two types of change one may want to make:
1. Upgrades, which change existing logic. For example, one might want the `Asset` template to have multiple signatories.
2. Extensions, which merely add new functionality through additional templates.
Upgrades are covered in their own section outside this introduction to Daml: [smart contract upgrades](/appdev/deep-dives/smart-contract-upgrade),
so in this section we will extend the `compose` model with a simple second workflow: a multi-leg trade. In doing so, you'll learn about:
* The software architecture of the Daml stack
* Dependencies and data dependencies
* Identifiers
Since we are extending `compose`, the setup for this chapter is slightly more complex:
1. In a base directory, load the `compose` project using `dpm new intro-compose --template daml-intro-compose`. The directory `intro7` here is important as it'll be referenced by the other project we are creating.
2. In the same directory, load this chapter's project using `dpm new intro-9 --template daml-intro-9`.
`Dependencies` contains a new module `Intro.Asset.MultiTrade` and a corresponding test module `Test.Intro.Asset.MultiTrade`.
## DAR files, Daml-LF, and the engine
In `compose` you already learnt a little about projects, Daml-LF, DAR files, and dependencies. In this chapter we will actually need to have dependencies from the current project to the `compose` project so it's time to learn a little more about all this.
Let's have a look inside the DAR file of `compose`. DAR files, like Java JAR files, are just ZIP archives, but the SDK also has a utility to inspect DARs out of the box:
1. Navigate into the `intro7` directory.
2. Build using `dpm build -o assets.dar`
3. Run `dpm damlc inspect-dar assets.dar`
You'll get a whole lot of output. Under the header "DAR archive contains the following files:" you'll see that the DAR contains:
1. `*.dalf` files for the project and all its dependencies
2. The original Daml source code
3. `*.hi` and `*.hie` files for each `*.daml` file
4. Some meta-inf and config files
The first file is something like `intro7-1.0.0-887056cbb313b94ab9a6caf34f7fe4fbfe19cb0c861e50d1594c665567ab7625.dalf` which is the actual compiled package for the project. `*.dalf` files contain Daml-LF, which is Daml's intermediate language. The file contents are a binary encoded protobuf message defined by the [Daml-LF tooling in the `digital-asset/daml` repository](https://github.com/digital-asset/daml/tree/main/sdk/daml-lf). Daml-LF is evaluated on the ledger by the Daml Engine, which is a JVM component that is part of tools like the IDE's script runner, the Sandbox, or proper production ledgers. If Daml-LF is to Daml what Java Bytecode is to Java, the Daml Engine is to Daml what the JVM is to Java.
## Hashes and identifiers
Under the heading "DAR archive contains the following packages:" you get a similar looking list of package names, paired with only the long random string repeated. That hexadecimal string, `887056cbb313b94ab9a6caf34f7fe4fbfe19cb0c861e50d1594c665567ab7625` in this case, is the package hash and the primary and only identifier for a package that's guaranteed to be available and preserved. Meta information like name ("intro7") and version ("1.0.0") help make it human readable but should not be relied upon. You may not always get DAR files from your compiler, but be loading them from a running ledger, or get them from an artifact repository.
We can see this in action. When a DAR file gets deployed to a ledger, not all meta information is preserved.
1. Note down your main package hash from running `inspect-dar` above
2. Start the project by running a ledger and uploading the `assets.dar` DAR:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
cat < config.conf
dpm sandbox -c config.conf
```
3. Open another terminal and use the gRPC Ledger API to download the dar, making sure to replace the hash with the appropriate one
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
grpcurl localhost:6866 com.digitalasset.canton.admin.participant.v30.PackageService.GetDar \
-d '{"mainPackageId": "887056cbb313b94ab9a6caf34f7fe4fbfe19cb0c861e50d1594c665567ab7625"}' \
-plaintext | jq -r '.payload' | base64 --decode > assets_ledger.dar
```
4. Run `dpm damlc inspect-dar assets_ledger.dar`
You'll notice two things. Firstly, a lot of the dependencies have lost their names, they are now only identifiable by hash. We could of course also create a second project `intro7-1.0.0` with completely different contents so even when name and version are available, package hash is the only safe identifier.
That's why over the Ledger API, all types, like templates and records are identified by the triple `(entity name, module name, package hash)`. Your client application should know the package hashes it wants to interact with. To aid that, `inspect-dar` also provides a machine-readable format for the information it emits: `dpm damlc inspect-dar --json assets_ledger.dar`. The `main_package_id` field in the resulting JSON payload is the package hash of our project.
Secondly, you'll notice that all the `*.daml`, `*.hi` and `*.hie` files are gone. This leads us to data dependencies.
## Dependencies and data dependencies
Dependencies under the `daml.yaml` `dependencies` group rely on the `*.hi` files. The information in these files is crucial for dependencies like the Daml standard library, which provide functions, types, and typeclasses.
However, as you can see above, this information isn't preserved. Furthermore, preserving this information may not even be desirable. Imagine we had built `intro7` with SDK 1.100.0, and are building `intro9` with SDK 1.101.0. All the typeclasses and instances on the inbuilt types may have changed and are now present twice -- once from the current SDK and once from the dependency. This gets messy fast, which is why the SDK does not support `dependencies` across SDK versions. For dependencies on contract models that were fetched from a ledger, or come from an older SDK version, there is a simpler kind of dependency called `data-dependencies`. The syntax for `data-dependencies` is the same, but they only rely on the "binary" `*.dalf` files. The name tries to confer that the main purpose of such dependencies is to handle data: Records, Choices, Templates. The stuff one needs to use contract composability across projects.
For an extension model like this one,`data-dependencies` are appropriate, so the current project includes `compose` that way:
You'll notice a module `Test.Intro.Asset.TradeSetup`, which is almost a carbon copy of the `compose` trade setup Scripts. `data-dependencies` is designed to use existing contracts and data types. Daml Script is not imported. In practice, we also shouldn't expect that the DAR file we download from the ledger using the Ledger API to contain test scripts. For larger projects it's good practice to keep them separate and only deploy templates to the ledger.
## About project structures
As you've seen here, identifiers depend on the package as a whole and packages always bring all their dependencies with them. Thus changing anything in a complex dependency graph can have significant repercussions. It is therefore advisable to keep dependency graphs simple, and to separate concerns which are likely to change at different rates into separate packages.
For example, in all our projects in this intro, including this chapter, our scripts are in the same project as our templates. In practice, that means changing a test changes all identifiers, which is not desirable. It's better for maintainability to separate tests from main templates. If we had done that in [Design Patterns](/appdev/modules/m3-design-patterns), that would also have saved us from copying those modules.
Similarly, we included `Trade` in the same project as `Asset` in [Design Patterns](/appdev/modules/m3-design-patterns), even though `Trade` is a pure extension to the core `Asset` model. If we expect `Trade` to need more frequent changes, it may be a good idea to split it out into a separate project from the start.
## Next up
The `MultiTrade` model has more complex control flow and data handling than previous models. In [Language Fundamentals](/appdev/modules/m3-language-fundamentals) you'll learn how to write more advanced logic: control flow, folds, common typeclasses, custom functions, and the Daml standard library. We'll be using the same projects so don't delete your folders just yet.
## Building with dpm
When working on Canton Network projects, use the [`dpm`](/sdks-tools/cli-tools/dpm) tool for all build operations:
* `dpm build` — Compile your Daml project and produce a DAR file
* `dpm build --all` — Build all packages in a multi-package project
* `dpm test` — Run all Daml Script tests in the current package
* `dpm codegen-java` — Generate Java bindings from your Daml model
* `dpm codegen-js` — Generate TypeScript/JavaScript bindings from your Daml model
* `dpm sandbox` — Start a local Canton sandbox for integration testing
The `dpm` tool wraps the Daml compiler and build system with Canton Network defaults, handling SDK versioning and dependency resolution automatically.
# How to build Daml Archive (.dar) files
This guide shows you how to organize the source code defining your Daml workflows and how to build and package that code as Daml Archive (.dar) files, which you can deploy to the ledger or use to develop applications against. The guide is organized into the following smaller how-tos:
* How to define and build one or more Daml packages
* How to manage dependencies on third-party Daml packages
* How to decide what Daml code to put into what package
If you would like to learn more about the exact relationship between Daml package and Daml Archive (.dar) files, see Daml packages and Daml Archive (.dar) files. However, you do not need to know this in detail to use this guide. At a high-level you can just think of a Daml archive file to be the result of building a specific Daml package.
## How to define Daml packages
### Single package
All Daml packages require a daml.yaml file. Create this file at the root of your package directory. You will need the following information to populate this file:
* SDK Version: call `dpm version` to determine the installed SDK versions
* Package name: lower-skewer-case name that is unique to your package and company.
Add the following to your `daml.yaml`, replacing the `` as appropriate.
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
sdk-version:
name:
version: 1.0.0
source: daml
dependencies:
- daml-prim
- daml-stdlib
```
The source code (`.daml` files) for the package are placed in the directory specified by the source field above. Create a daml folder at the root of your package. Write your .daml files in this directory, the file name must match the module header in the file, treating dots as directories, as shown:
* `MyModule.daml` contains `module MyModule where`
* `Path/To/My/Module.daml` contains `module Path.To.My.Module where`
Directory and .daml file names must be in UpperCamel casing.
The dpm new command provides pre-made templates for various package structures and tutorials, see this DPM page for more information.
### Multiple packages
Your Daml project will usually need at least two packages, for workflows and testing. Daml provides support for building and developing these packages via the `multi-package.yaml` file. In a directory above your package(s) directories, create a `multi-package.yaml` file. List the relative paths to your packages in this file using the following structure:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
packages:
- ./my-package1
- ./my-package2
```
These are paths to the directories containing the `daml.yaml`, not to the `daml.yaml` itself.
### Environment variables in configuration files
When your project has more than one package, consider using environment variables to avoid duplication of information like the `sdk-version`. Replace the `sdk-version` field with `sdk-version: $SDK_VERSION` (or any other valid environment variable name), and ensure this variable is set before building. `SDK_VERSION=3.4.9 dpm build --all`
Variables can also be placed inline, and are supported on all string fields in the daml.yaml, as the following example shows:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
sdk-version: $SDK_VERSION
name: my-package-$PACKAGE_SUFFIX
version: 1.0.$MAIN_PATCH
source: daml
dependencies:
- daml-prim
- daml-stdlib
```
See Environment Variable Interpolation for more information.
## How to build Daml packages
To build a single package, navigate to its root directory and run `dpm build`. To build all packages in a multi-package project, navigate to the directory containing the `multi-package.yaml` and run `dpm build --all`. By default these will create a Daml Archive (.dar) file for each package built in `/.daml/dist/-.dar`. .dar files are used both for uploading to the Canton Ledger, and for package dependencies. The location where the .dar is created can be overridden using the `--output` flag for dpm build, which can also be provided in the `daml.yaml` file under the `build-options` field:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
build-options:
- --output=./output-bin/my-package.dar
```
See Daml Build Options for a full list of dpm build options, or run `dpm build --help`, which includes options for changing the LF version and configuring warnings. All of these options can also be provided via `build-options` above. Consider reading Recommended Build Options for our recommended set of warning flags.
If you face issues when changing configuration options like the `sdk-version`, or the LF version, cleaning the package(s) may help. To clean a single package, run `daml clean` from the package directory. To clean all packages in a project, run `daml clean --all` from the directory containing the `multi-package.yaml`
## How to depend on Daml packages
Dependencies in Daml are specified by their Daml Archive (.dar) file. To add a dependency to your package, add the paths to your dependency .dar files to your `daml.yaml` as follows:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
...
data-dependencies:
- ./path/to/your/dep.dar
- ./path/to/a/package/.daml/dist/my-package-1.0.0.dar
```
Note the use of `data-dependencies` instead of the previously covered `dependencies` field, the latter is reserved for `daml-prim`, `daml-stdlib`, and the optional testing library daml-script. Once added to the `daml.yaml`, modules from the dependency .dar can be imported from the modules of this package. In the event of collision between module names, either with this package or other dependencies, see module-prefixes.
In addition to using DARs already on your local file system, you can also depend on DARs available from a remote OCI repository while building your Daml project -- see the [Remote Dars](/appdev/modules/m5-environment-c#onfiguration#remote-dars) section for further guidance and examples.
When depending on .dar files from packages listed in the `multi-package.yaml`, calling `dpm build` and `dpm build --all` will build the relevant packages in the correct order for you.
## How to manage dependencies on third-party Daml packages
To build composed transactions, you will need to depend on the .dar files of third-party applications. At the time of writing there is no dedicated package repository for Daml Archives. However .dar files are reasonably small and change infrequently. You thus best check them into your repository, in a dars/vendored directory. If you instead retrieve the .dar files as part of a build step, check the hashes of these dars as part of this step.
If you intend to distribute your .dar files for others to build on, include the retrieval process in your documentation.
### Depending on daml-script test libraries
The `daml-script` library is not cross compatible with other releases from different Daml SDK versions. Therefore, when using Daml script test code shared by third-party apps, we recommend you to vendor in that Daml script code. For example, by checking it into a daml/vendored/ directory in your repository. A good example is the Canton Network Token Standard test harness provided by splice here: `https://github.com/DACH-NY/canton-network-node/tree/main/token-standard/splice-token-standard-test`. Adding these packages to your `multi-package.yaml` will ensure they are built as needed.
## How to decide what Daml code to put into what package
Use the following criteria to organize your project into separate packages:
**Separate workflow definitions from their tests**
Place tests for workflow definitions in a separate package to the workflows, to avoid distributing and uploading said tests to the ledger. Specifically avoid uploading the daml-script package to any production ledger.
**Separate public APIs from implementations**
If your application includes public APIs, intended to be used by other applications, define these APIs using Daml interfaces and place these interfaces in a different package to their implementation. See for example the interfaces defined in the Canton Network Token Standard here: [https://github.com/DACH-NY/canton-network-node/blob/da5dbe251b17f9c4c5d3e96840f486d14dc8e43e/token-standard/splice-api-token-holding-v1/daml/Splice/Api/Token/HoldingV1.daml](https://github.com/DACH-NY/canton-network-node/blob/da5dbe251b17f9c4c5d3e96840f486d14dc8e43e/token-standard/splice-api-token-holding-v1/daml/Splice/Api/Token/HoldingV1.daml)
**Separate by business domains**
Consider splitting workflows from different business domains into separate packages, so that stakeholders from one domain do not need to audit and vet the workflows from others domains that they do not directly interact with.
# Choices
Source: https://docs.canton.network/appdev/modules/m3-choices
Learn how to add behavior to Daml contracts using choices - the methods that transform contract state
## Introduction
In the previous section, you saw how to change data on a contract by archiving it and re-creating it with updated data. But what if you wanted to allow another party to make specific changes? Or if you wanted to provide a convenient way to transform contract data?
In this section you will learn about how to define simple data transformations using *choices* and how to delegate the right to *exercise* these choices to other parties.
You can load the code for this section by running `dpm new intro-choices --template daml-intro-choices`
## Choices as methods
If you think of templates as classes and contracts as objects, where are the methods?
Take as an example a `Contact` contract on which the contact owner wants to be able to change the telephone number. Rather than requiring them to manually look up the contract, archive the old one, and create a new one as you saw in [Contract Templates](/appdev/modules/m3-contract-templates), you can provide them a convenience method on `Contact`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Contact
with
owner : Party
party : Party
address : Text
telephone : Text
where
signatory owner
observer party
choice UpdateTelephone
: ContractId Contact
with
newTelephone : Text
controller owner
do
create this with
telephone = newTelephone
```
The above defines a *choice* called `UpdateTelephone`. Choices are part of a contract template. They're permissioned functions that result in an `Update`. Using choices, authority can be passed around, allowing the construction of complex transactions.
Let's unpack the code snippet above:
* The first line, `choice UpdateTelephone` indicates a choice definition, `UpdateTelephone` is the name of the choice. It starts a new block in which that choice is defined.
* `: ContractId Contact` is the return type of the choice.
This particular choice archives the current `Contact`, and creates a new one. What it returns is a reference to the new contract, in the form of a `ContractId Contact`
* The following `with` block is that of a record. Just like with templates, in the background, a new record type is declared: `data UpdateTelephone = UpdateTelephone with`
* The line `controller owner` says that this choice is *controlled* by `owner`, meaning `owner` is the only party that is allowed to *exercise* them.
* The `do` starts a block defining the action the choice should perform when exercised. In this case a new `Contact` is created.
* The new `Contact` is created using `this with`. `this` is a special value available within the `where` block of templates and takes the value of the current contract's arguments.
There is nothing here explicitly saying that the current `Contact` should be archived. That's because choices are *consuming* by default. That means when the above choice is exercised on a contract, that contract is archived.
As mentioned in `data`, within a choice we use `create` instead of `createCmd`. Whereas `createCmd` builds up a list of commands to be sent to the ledger, `create` builds up a more flexible `Update` that is executed directly by the ledger. You might have noticed that `create` returns an `Update (ContractId Contact)`, not a `ContractId Contact`. As a `do` block always returns the value of the last statement within it, the whole `do` block returns an `Update`, but the return type on the choice is just a `ContractId Contact`. This is a convenience. Choices *always* return an `Update` so for readability it's omitted on the type declaration of a choice.
Now to exercise the new choice in a script:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice_test = do
owner <- allocateParty "Alice"
party <- allocateParty "Bob"
contactCid <- submit owner do
createCmd Contact with
owner
party
address = "1 Bobstreet"
telephone = "012 345 6789"
-- Bob can't change his own telephone number as Alice controls
-- that choice.
submitMustFail party do
exerciseCmd contactCid UpdateTelephone with
newTelephone = "098 7654 321"
newContactCid <- submit owner do
exerciseCmd contactCid UpdateTelephone with
newTelephone = "098 7654 321"
```
You exercise choices using the `exercise` function, which takes a `ContractId a`, and a value of type `c`, where `c` is a choice on template `a`. Since `c` is just a record, you can also just fill in the choice parameters using the `with` syntax you are already familiar with.
`exerciseCmd` returns a `Commands r` where `r` is the return type specified on the choice, allowing the new `ContractId Contact` to be stored in the variable `newContactCid`. Just like for `createCmd` and `create`, there is also `exerciseCmd` and `exercise`. The versions with the `cmd` suffix is always used on the client side to build up the list of commands on the ledger. The versions without the suffix are used within choices and are executed directly on the server.
There is also `createAndExerciseCmd` and `createAndExercise` which we have seen in the previous section. This allows you to create a new contract with the given arguments and immediately exercise a choice on it. For a consuming choice, this archives the contract so the contract is created and archived within the same transaction.
## Choices as delegation
Up to this point all the contracts only involved one party. `party` may have been stored as `Party` field in the above, which suggests they are actors on the ledger, but they couldn't see the contracts, nor change them in any way. It would be reasonable for the party for which a `Contact` is stored to be able to update their own address and telephone number. In other words, the `owner` of a `Contact` should be able to *delegate* the right to perform a certain kind of data transformation to `party`.
The below demonstrates this using an `UpdateAddress` choice and corresponding extension of the script:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice UpdateAddress
: ContractId Contact
with
newAddress : Text
controller party
do
create this with
address = newAddress
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
newContactCid <- submit party do
exerciseCmd newContactCid UpdateAddress with
newAddress = "1-10 Bobstreet"
Some newContact <- queryContractId owner newContactCid
assert (newContact.address == "1-10 Bobstreet")
```
If you open the script view in the IDE, you will notice that Bob sees the `Contact`. This is because `party` is specified as an `observer` in the template, and in this case Bob is the `party`. More on *observers* later, but in short, they get to see any changes to the contract.
## Choices in the ledger model
In [Contract Templates](/appdev/modules/m3-contract-templates#daml-ledger-basics) you learned about the high-level structure of a Daml ledger. With choices and the `exercise` function, you have the next important ingredient to understand the structure of the ledger and transactions.
A *transaction* is a list of *actions*, and there are three kinds of action: `create`, `exercise` and `fetch`.
* A `create` action creates a new contract with the given arguments and sets its status to *active*.
* A `fetch` action checks the existence and activeness of a contract.
* An `exercise` action exercises a choice on a contract resulting in a transaction (list of sub-actions) called the *consequences*. Exercises come in two kinds called `consuming` and `nonconsuming`. `consuming` is the default kind and changes the contract's status from *active* to *archived*.
Each action can be visualized as a tree, where the action is the root node, and its children are its consequences. Every consequence may have further consequences. As `fetch`, `create` has no consequences, it is always a leaf node. You can see the actions and their consequences in the transaction view of the above script:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
Transactions:
TX 0 1970-01-01T00:00:00Z (Contact:46:17)
#0:0
│ consumed by: #2:0
│ referenced by #2:0
│ disclosed to (since): 'Alice' (0), 'Bob' (0)
└─> 'Alice' creates Contact:Contact
with
owner = 'Alice'; party = 'Bob'; address = "1 Bobstreet"; telephone = "012 345 6789"
TX 1 1970-01-01T00:00:00Z
mustFailAt actAs: {'Bob'} readAs: {} (Contact:55:3)
TX 2 1970-01-01T00:00:00Z (Contact:59:20)
#2:0
│ disclosed to (since): 'Alice' (2), 'Bob' (2)
└─> 'Alice' exercises UpdateTelephone on #0:0 (Contact:Contact)
with
newTelephone = "098 7654 321"
children:
#2:1
│ consumed by: #3:0
│ referenced by #3:0
│ disclosed to (since): 'Alice' (2), 'Bob' (2)
└─> 'Alice' creates Contact:Contact
with
owner = 'Alice'; party = 'Bob'; address = "1 Bobstreet"; telephone = "098 7654 321"
TX 3 1970-01-01T00:00:00Z (Contact:69:20)
#3:0
│ disclosed to (since): 'Alice' (3), 'Bob' (3)
└─> 'Bob' exercises UpdateAddress on #2:1 (Contact:Contact)
with
newAddress = "1-10 Bobstreet"
children:
#3:1
│ disclosed to (since): 'Alice' (3), 'Bob' (3)
└─> 'Alice' creates Contact:Contact
with
owner = 'Alice';
party = 'Bob';
address = "1-10 Bobstreet";
telephone = "098 7654 321"
Active contracts: #3:1
Return value: {}
```
There are four commits corresponding to the four `submit` statements in the script. Within each commit, we see that it's actually actions that have IDs of the form `#commit_number:action_number`. Contract IDs are just the ID of their `create` action.
So commits `#2` and `#3` contain `exercise` actions with IDs `#2:0` and `#3:0`. The `create` actions of the updated `Contact` contracts, `#2:1` and `#3:1`, are indented and found below a line reading `children:`, making the tree structure apparent.
### The archive choice
You may have noticed that there is no archive action. That's because `archive cid` is just shorthand for `exercise cid Archive`, where `Archive` is a choice implicitly added to every template, with the signatories as controllers.
## A simple cash model
With the power of choices, you can build your first interesting model: issuance of cash IOUs (I owe you). The model presented here is simpler than the one in [Language Fundamentals](/appdev/modules/m3-language-fundamentals) as it's not concerned with the location of the physical cash, but merely with liabilities:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Cash = Cash with
currency : Text
amount : Decimal
deriving (Eq, Show)
template SimpleIou
with
issuer : Party
owner : Party
cash : Cash
where
signatory issuer
observer owner
choice Transfer
: ContractId SimpleIou
with
newOwner : Party
controller owner
do
create this with owner = newOwner
```
The above model is fine as long as everyone trusts Dora. Dora could revoke the `SimpleIou` at any point by archiving it. However, the provenance of all transactions would be on the ledger so the owner could *prove* that Dora was dishonest and cancelled her debt.
## Next up
You can now store and transform data on the ledger, even giving other parties specific write access through choices.
In [Authorization Model](/appdev/modules/m3-authorization), you will learn more about the authorization rules that govern who can create, exercise, and archive contracts.
# Contract Keys
Source: https://docs.canton.network/appdev/modules/m3-contract-keys
Use contract keys for stable references to contracts and key-based lookups
# Reference: Contract Keys
Contract keys are an optional addition to templates. They let you specify a way of identifying contracts using the parameters to the template — similar to a key in a database.
Contract keys do not change and can be used to refer to a contract even when the contract ID changes. As a contract is updated via archive and create operations, the currently active contract(s) can easily be referenced via the contract key.
In Canton 3.x, it is possible for multiple active contracts of the same template to share the same key.
The general usage is that key uniqueness is guaranteed outside of the Daml engine — for example, in the Daml business logic or at the backend client level (e.g., a unique account number or invoice number). Because of this, the primary key-based API (`lookupByKey`, `fetchByKey`, `exerciseByKey`) are optimized for the common case where there is at most a key references one contract.
Here's an example of setting up a contract key for a bank account, to act as a bank account ID:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
type AccountKey = (Party, Text)
template Account with
bank : Party
number : Text
owner : Party
balance : Decimal
observers : [Party]
where
signatory [bank, owner]
observer observers
key (bank, number) : AccountKey
maintainer key._1
```
## What Can Be a Contract Key
The key can be an arbitrary serializable expression that does **not** contain contract IDs. However, it **must** include every party that you want to use as a `maintainer` (see [Specify Maintainers](#specify-maintainers) below).
It's best to use simple types for your keys like `Text` or `Int`, rather than a list or more complex type.
## Specify Maintainers
If you specify a contract key for a template, you must also specify a `maintainer` or maintainers, in a similar way to specifying signatories or observers. The maintainers "own" the key in the same way the signatories "own" a contract. Just like signatories of contracts prevent double spends or use of false contract data, maintainers of keys ensure consistent key lookups. Since the key is part of the contract, the maintainers **must** be signatories of the contract. However, maintainers are computed from the `key` instead of the template arguments. In the example above, the `bank` is ultimately the maintainer of the key.
Since multiple templates may use the same key type, some key-related functions must be annotated using the `@ContractType` as shown in the examples below.
When you are writing Daml models, the maintainers matter since they affect authorization -- much like signatories and observers. You don't need to do anything to "maintain" the keys. Validators hosting the maintainers of a key involved in a transaction verify that contracts are retrieved in a consistent order for that key within the transaction.
## Contract Lookups
The primary purpose of contract keys is to provide a stable, and possibly meaningful, identifier that can be used in Daml to fetch contracts. The main functions for key-based lookups are `fetchByKey`, `lookupByKey`, and `exerciseByKey`, all available by default.
When multiple contracts share the same key, these functions return the **first** contract according to the following lookup order:
1. Contracts created within the current transaction, starting with the most recent.
2. Explicitly disclosed contracts, in the order provided in the command.
3. Contracts known to the participant, in any order. (The current implementation returns them in recency order, but this is not guaranteed and should not be relied on.)
For use cases where multiple contracts per key are expected, the `DA.ContractKeys` module provides `lookupNByKey` and `lookupAllByKey` (see [Multi-Contract Key Lookups](#multi-contract-key-lookups) below).
Because disclosed contracts are prioritized over known contracts, you can use disclosures to ensure a specific retrieval order during command submission.
### fetchByKey
`(fetchedContractId, fetchedContract) <- fetchByKey @ContractType contractKey`
Use `fetchByKey` to fetch the ID and data of the first contract with the specified key (according to the lookup order above). It is an alternative to `fetch` and behaves the same in most ways.
It returns a tuple of the ID and the contract object (containing all its data).
Like `fetch`, `fetchByKey` needs to be authorized by at least one stakeholder.
`fetchByKey` fails and aborts the transaction with a [`CONTRACT_KEY_NOT_FOUND`](/appdev/reference/error-codes#contracts-and-transactions) error if no contract with the given key is visible to the submitting party.
### lookupByKey
`optionalContractId <- lookupByKey @ContractType contractKey`
Use `lookupByKey` to check whether a contract with the specified key exists. If it does exist, `lookupByKey` returns `Some contractId`, where `contractId` is the ID of the first contract matching the key (according to the lookup order above); otherwise, it returns `None`.
`lookupByKey` requires authorization from **all** maintainers of the key. This is necessary so that confirming participants hosting the maintainers can verify that contracts are retrieved in a consistent order for the given key within the transaction.
More precisely:
* `lookupByKey` returns `Some contractId` if a contract with the given key exists and the submitter is a stakeholder on that contract, and authorization from all maintainers is present.
* `lookupByKey` returns `None` if no contract with the given key exists (or none is visible to the submitter), and authorization from all maintainers is present.
* `lookupByKey` aborts the transaction if authorization from any maintainer is missing.
## exerciseByKey
`exerciseByKey @ContractType contractKey`
Use `exerciseByKey` to exercise a choice on the first contract with the given key (according to the lookup order above). Just like `exercise`, running `exerciseByKey` requires visibility of the contract and authorization from the controllers of the choice.
## Multi-Contract Key Lookups
For use cases where multiple contracts may share the same key, the `DA.ContractKeys` module provides:
* **`lookupNByKey @ContractType n key`** — looks up up to `n` contracts with the given key, returned in the lookup order described above.
* **`lookupAllByKey @ContractType key`** — looks up all contracts with the given key.
These functions are not imported by default. To use them, add `import DA.ContractKeys` to your module.
Performance is optimized for the common case of zero or one contract per key. The multi-contract functions (`lookupNByKey`, `lookupAllByKey`) may be less performant when many contracts share a key.
## Daml Script Functions
In addition to the Daml language primitives above (which run inside transactions), there are Daml Script functions for querying keys outside of transactions:
* **`queryByKey @ContractType party key`** — looks up a contract by key and returns its ID and data. Runs as a top-level `Script` action.
* **`queryNByKey @ContractType party n key`** — looks up up to `n` contracts by key and returns their IDs and data. Runs as a top-level `Script` action.
* **`exerciseByKeyCmd @ContractType key choiceArg`** — exercises a choice on the first contract with the given key. This is a `Commands` action and must be used inside a `submit` block, where it can be combined with other commands.
## Example
An example demonstrating key-based lookups and exercises:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Copyright (c) 2026 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved.
-- SPDX-License-Identifier: Apache-2.0
module Test where
import Daml.Script
import DA.Assert
import DA.ContractKeys
import DA.Optional
template WithKey
with
p : Party
payload : Text
where
signatory p
key p : Party
maintainer key
nonconsuming choice GetText : Text
with
party : Party
controller party
do pure payload
template Helper
with
p : Party
where
signatory p
choice PerformLookupByKey : Optional (ContractId WithKey)
controller p
do lookupByKey @WithKey p
choice PerformFetchByKey : (ContractId WithKey, WithKey)
controller p
do fetchByKey @WithKey p
choice PerformExerciseByKey : Text
controller p
do exerciseByKey @WithKey p GetText with party = p
choice PerformLookupNByKey : [(ContractId WithKey, WithKey)]
with n : Int
controller p
do lookupNByKey @WithKey n p
choice CreateThenPerformLookupNByKey : [(ContractId WithKey, WithKey)]
with
payload : Text
n : Int
controller p
do create (WithKey p payload)
lookupNByKey @WithKey n p
-- Demonstrates basic key operations: lookupByKey, fetchByKey, exerciseByKey
useKeyOperations : Script ()
useKeyOperations = script do
alice <- allocateParty "alice"
-- Create a contract with a key
cid <- alice `submit` createCmd (WithKey alice "hello")
-- lookupByKey returns Some if a contract with the key exists
mcid <- alice `submit`
createAndExerciseCmd (Helper alice) PerformLookupByKey
assert (isSome mcid)
-- fetchByKey returns the contract ID and data
(kcid, contract) <- alice `submit`
createAndExerciseCmd (Helper alice) PerformFetchByKey
kcid === cid
contract.payload === "hello"
-- exerciseByKey exercises a choice on the contract with the key
result <- alice `submit`
createAndExerciseCmd (Helper alice) PerformExerciseByKey
result === "hello"
-- Demonstrates non-unique keys: multiple contracts can share a key.
-- lookupByKey and fetchByKey return the first contract per lookup order;
-- lookupNByKey (from DA.ContractKeys) returns up to n contracts.
multipleContractsPerKey : Script ()
multipleContractsPerKey = script do
alice <- allocateParty "alice"
-- Create multiple contracts with the same key
cid1 <- alice `submit` createCmd (WithKey alice "first")
cid2 <- alice `submit` createCmd (WithKey alice "second")
cid3 <- alice `submit` createCmd (WithKey alice "third")
let contracts = [(cid1, "first"), (cid2, "second"), (cid3, "third")]
-- fetchByKey returns one of the created contracts
(kcid, contract) <- alice `submit`
createAndExerciseCmd (Helper alice) PerformFetchByKey
assert ((kcid, contract.payload) `elem` contracts)
-- lookupNByKey returns up to n contracts (any 2 of 3, in any order)
result <- alice `submit`
createAndExerciseCmd (Helper alice) PerformLookupNByKey with n = 2
let results = [(cid, c.payload) | (cid, c) <- result]
length results === 2
let [r1, r2] = results
assert (r1 `elem` contracts)
assert (r2 `elem` contracts)
r1 =/= r2
-- lookupNByKey returns:
-- - local contracts in recency order
-- - then disclosures in command-specified order
-- - then contracts known to the participant in any order
Some disclosure <- alice `queryDisclosure` cid1
result <- submit (actAs alice <> disclose disclosure) do
createAndExerciseCmd (Helper alice) CreateThenPerformLookupNByKey with
payload = "fourth"
n = 3
let results = [(cid, c.payload) | (cid, c) <- result]
length results === 3
let [r1, r2, r3] = results
r1._2 === "fourth"
r2 === (cid1, "first")
r3._1 =/= cid1
r3._2 =/= "fourth"
assert (r3 `elem` contracts)
-- Demonstrates exerciseByKeyCmd in a submit block
exerciseByKeyCmdExample : Script ()
exerciseByKeyCmdExample = script do
alice <- allocateParty "alice"
alice `submit` createCmd (WithKey alice "payload")
-- exerciseByKeyCmd is a Commands action used inside submit
result <- alice `submit` do
exerciseByKeyCmd @WithKey alice GetText with party = alice
result === "payload"
```
# Contract Templates
Source: https://docs.canton.network/appdev/modules/m3-contract-templates
Learn how to define Daml contract templates - the core building blocks of Canton smart contracts
## Introduction
To begin with, you're going to write a very small Daml template, which represents a self-issued, non-transferable token. Because it's a minimal template, it isn't actually useful on its own - you'll make it more useful later - but it's enough that it can show you the most basic concepts:
* Transactions
* Daml modules and files
* Templates
* Contracts
* Signatories
You can load the code for this section by running `dpm new intro-contracts --template daml-intro-contracts`
## Daml ledger basics
Like most structures called ledgers, a Daml Ledger is just a list of *commits*. When we say *commit*, we mean the final result of when a *party* successfully *submits* a *transaction* to the ledger.
*Transaction* is a concept we'll cover in more detail through this introduction. The most basic examples are the creation and archival of a *contract*.
A contract is *active* from the point where there is a committed transaction that creates it, up to the point where there is a committed transaction that *archives* it.
Individual contracts are *immutable* in the sense that an active contract can not be changed. You can only change the *active contract set* by creating a new contract, or archiving an old one.
Daml specifies what transactions are legal on a Daml Ledger. The rules the Daml code specifies are collectively called a *Daml model* or *contract model*.
## Daml modules and files
Each `.daml` file defines a *Daml Module* at the top:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Token where
```
Code comments in Daml are introduced with `--`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- A Daml file defines a module.
module Token where
```
## Templates
A `template` defines a type of contract that can be created, and who has the right to do so. *Contracts* are instances of *templates*.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Token
with
owner : Party
where
signatory owner
```
You declare a template starting with the `template` keyword, which takes a name as an argument.
Daml is whitespace-aware and uses layout to structure *blocks*. Everything that's below the first line is indented, and thus part of the template's body.
*Contracts* contain data, referred to as the *create arguments* or simply *arguments*. The `with` block defines the data type of the create arguments by listing field names and their types. The single colon `:` means "of type", so you can read this as "template `Token` with a field `owner` of type `Party`".
`Token` contracts have a single field `owner` of type `Party`. The fields declared in a template's `with` block are in scope in the rest of the template body, which is contained in a `where` block.
## Signatories
The `signatory` keyword specifies the *signatories* of a contract. These are the parties whose *authority* is required to create the contract or archive it -- just like a real contract. Every contract must have at least one signatory.
Furthermore, Daml ledgers *guarantee* that parties see all transactions where their authority is used. This means that signatories of a contract are guaranteed to see the creation and archival of that contract.
## Next up
In [Choices](/appdev/modules/m3-choices), you'll learn how to add behavior to your contracts using choices.
# Composition and Design Patterns
Source: https://docs.canton.network/appdev/modules/m3-design-patterns
Compose multi-step transactions, understand Daml's execution model and privacy, and apply common multi-party workflow patterns
# Compose choices
It's time to put everything you've learned so far together into a complete and secure Daml model for asset issuance, management, transfer, and trading. This application will have capabilities similar to the one in the [CN Quickstart](/sdks-tools/reference-projects/cn-quickstart). In the process you will learn about a few more concepts:
* Daml projects, packages, and modules
* Composition of transactions
* Observers and stakeholders
* Daml's execution model
* Privacy
The model in this section is not a single Daml file, but a Daml project consisting of several files that depend on each other.
Remember that you can load all the code for this section into a folder called `intro-compose` by running `dpm new intro-compose --template daml-intro-compose`
## Daml projects
Daml is organized in projects, packages, and modules. A Daml project is specified using a single `daml.yaml` file, and compiles into a package in Daml's intermediate language, or bytecode equivalent, Daml-LF. Each Daml file within a project becomes a Daml module, which is a bit like a namespace. Each Daml project has a source root specified in the `source` parameter in the project's `daml.yaml` file. The package will include all modules specified in `*.daml` files beneath that source directory.
You can start a new project with a skeleton structure using `dpm new project-name` in the terminal. A minimal project would contain just a `daml.yaml` file and an empty directory of source files.
> Take a look at the `daml.yaml` for the this chapter's project:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
sdk-version: __VERSION__
name: __PROJECT_NAME__
source: daml
version: 1.0.0
dependencies:
- daml-prim
- daml-stdlib
- daml-script
```
You can generally set `name` and `version` freely to describe your project. `dependencies` does what the name suggests: it includes dependencies. You should always include `daml-prim` and `daml-stdlib`. The former contains internals of the compiler and the Daml Runtime, the latter gives access to the Daml standard library. `daml-script` contains the types and functions for Daml Script.
You compile a Daml project by running `dpm build` from the project root directory. This creates a DAR file in `.daml/dist/dist/${project_name}-${project_version}.dar`. A DAR file is Daml's equivalent of a JAR file in Java: it's the artifact that gets deployed to a ledger to load the package and its dependencies. `dar` files are fully self-contained in that they contain all dependencies of the main package. More on all of this in [Building and Packaging](/appdev/modules/m3-building-packaging).
## Project structure
This project contains an asset holding model for transferable, fungible assets and a separate trade workflow. The templates are structured in three modules: `Intro.Asset`, `Intro.Asset.Role`, and `Intro.Asset.Trade`.
In addition, there are tests in modules `Test.Intro.Asset`, `Test.Intro.Asset.Role`, and `Test.Intro.Asset.Trade`.
All but the last `.`-separated segment in module names correspond to paths relative to the project source directory, and the last one to a file name. The folder structure therefore looks like this:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
.
├── daml
│ ├── Intro
│ │ ├── Asset
│ │ │ ├── Role.daml
│ │ │ └── Trade.daml
│ │ └── Asset.daml
│ └── Test
│ └── Intro
│ ├── Asset
│ │ ├── Role.daml
│ │ └── Trade.daml
│ └── Asset.daml
└── daml.yaml
```
Each file contains a module header. For example, `daml/Intro/Asset/Role.daml`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Intro.Asset.Role where
```
You can import one module into another using the `import` keyword. The `LibraryModules` module imports all six modules:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
import Intro.Asset
```
Imports always have to appear just below the module declaration. You can optionally add a list of names after the import to import only the selected names:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
import DA.List (sortOn, groupOn)
```
If your module contains any Daml Scripts, you need to import the corresponding functionality:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
import Daml.Script
```
## Project overview
The project both changes and adds to the `Iou` model presented in [Authorization](/appdev/modules/m3-authorization):
* Assets are fungible in the sense that they have `Merge` and `Split` choices that allow the `owner` to manage their holdings.
* Transfer proposals now need the authorities of both `issuer` and `newOwner` to accept. This makes `Asset` safer than `Iou` from the issuer's point of view.
With the `Iou` model, an `issuer` could end up owing cash to anyone as transfers were authorized by just `owner` and `newOwner`. In this project, only parties having an `AssetHolder` contract can end up owning assets. This allows the `issuer` to determine which parties may own their assets.
* The `Trade` template adds a swap of two assets to the model.
## Composed choices and scripts
This project showcases how you can put the `Update` and `Script` actions you learned about in [Authorization](/appdev/modules/m3-authorization) to good use. For example, the `Merge` and `Split` choices each perform several actions in their consequences.
* Two create actions in case of `Split`
* One create and one archive action in case of `Merge`
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Split
: SplitResult
with
splitQuantity : Decimal
controller owner
do
splitAsset <- create this with
quantity = splitQuantity
remainder <- create this with
quantity = quantity - splitQuantity
return SplitResult with
splitAsset
remainder
choice Merge
: ContractId Asset
with
otherCid : ContractId Asset
controller owner
do
other <- fetch otherCid
assertMsg
"Merge failed: issuer does not match"
(issuer == other.issuer)
assertMsg
"Merge failed: owner does not match"
(owner == other.owner)
assertMsg
"Merge failed: symbol does not match"
(symbol == other.symbol)
archive otherCid
create this with
quantity = quantity + other.quantity
```
The `return` function used in `Split` is available in any `Action` context. The result of `return x` is a no-op containing the value `x`. It has an alias `pure`, indicating that it's a pure value, as opposed to a value with side-effects. The `return` name makes sense when it's used as the last statement in a `do` block as its argument is indeed the "return"-value of the `do` block in that case.
Taking transaction composition a step further, the `Trade_Settle` choice on `Trade` composes two `exercise` actions:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Trade_Settle
: (ContractId Asset, ContractId Asset)
with
quoteAssetCid : ContractId Asset
baseApprovalCid : ContractId TransferApproval
controller quoteAsset.owner
do
fetchedBaseAsset <- fetch baseAssetCid
assertMsg
"Base asset mismatch"
(baseAsset == fetchedBaseAsset with
observers = baseAsset.observers)
fetchedQuoteAsset <- fetch quoteAssetCid
assertMsg
"Quote asset mismatch"
(quoteAsset == fetchedQuoteAsset with
observers = quoteAsset.observers)
transferredBaseCid <- exercise
baseApprovalCid TransferApproval_Transfer with
assetCid = baseAssetCid
transferredQuoteCid <- exercise
quoteApprovalCid TransferApproval_Transfer with
assetCid = quoteAssetCid
return (transferredBaseCid, transferredQuoteCid)
```
The resulting transaction, with its two nested levels of consequences, can be seen in the `test_trade` script in `Test.Intro.Asset.Trade`:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
TX 14 1970-01-01T00:00:00Z (Test.Intro.Asset.Trade:79:23)
#14:0
│ disclosed to (since): 'Alice' (14), 'Bob' (14)
└─> 'Bob' exercises Trade_Settle on #12:0 (Intro.Asset.Trade:Trade)
with
quoteAssetCid = #9:1; baseApprovalCid = #13:1
children:
#14:1
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Alice' and 'USD_Bank' fetch #10:1 (Intro.Asset:Asset)
#14:2
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Bob' and 'EUR_Bank' fetch #9:1 (Intro.Asset:Asset)
#14:3
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Alice' and 'Bob' exercise TransferApproval_Transfer on #13:1 (Intro.Asset:TransferApproval)
with
assetCid = #10:1
children:
#14:4
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Alice' and 'USD_Bank' fetch #10:1 (Intro.Asset:Asset)
#14:5
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Alice' and 'USD_Bank' exercise Archive on #10:1 (Intro.Asset:Asset)
#14:6
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Bob' and 'USD_Bank' create Intro.Asset:Asset
with
issuer = 'USD_Bank';
owner = 'Bob';
symbol = "USD";
quantity = 100.0000000000;
observers = []
#14:7
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Alice',
'Bob' exercises TransferApproval_Transfer on #11:1 (Intro.Asset:TransferApproval)
with
assetCid = #9:1
children:
#14:8
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Bob' and 'EUR_Bank' fetch #9:1 (Intro.Asset:Asset)
#14:9
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Bob' and 'EUR_Bank' exercise Archive on #9:1 (Intro.Asset:Asset)
#14:10
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Alice' and 'EUR_Bank' create Intro.Asset:Asset
with
issuer = 'EUR_Bank';
owner = 'Alice';
symbol = "EUR";
quantity = 90.0000000000;
observers = []
```
Similar to choices, you can see how the scripts in this project are built up from each other:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
test_issuance = do
setupResult@(alice, bob, bank, aha, ahb) <- setupRoles
assetCid <- submit bank do
exerciseCmd aha Issue_Asset
with
symbol = "USD"
quantity = 100.0
Some asset <- queryContractId bank assetCid
assert (asset == Asset with
issuer = bank
owner = alice
symbol = "USD"
quantity = 100.0
observers = []
)
return (setupResult, assetCid)
```
In the above, the `test_issuance` script in `Test.Intro.Asset.Role` uses the output of the `setupRoles` script in the same module.
The same line shows a new kind of pattern matching. Rather than writing `setupResult <- setupRoles` and then accessing the components of `setupResult` using `_1`, `_2`, etc., you can give them names. It's equivalent to writing:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
setupResult <- setupRoles
case setupResult of
(alice, bob, bank, aha, ahb) -> ...
```
Just writing `(alice, bob, bank, aha, ahb) <- setupRoles` would also be legal, but `setupResult` is used in the return value of `test_issuance` so it makes sense to give it a name, too. The notation with `@` allows you to give both the whole value as well as its constituents names in one go.
## Daml's execution model
Daml's execution model is fairly easy to understand, but has some important consequences. You can imagine the life of a transaction as follows:
Command submission
A user submits a list of commands via the Ledger API of a participant node, acting as a `Party` hosted on that node. That party is called the requester.
Interpretation
Each command corresponds to one or more actions. During this step, the `Update` corresponding to each action is evaluated in the context of the ledger to calculate all consequences, including transitive ones (consequences of consequences, etc.). The result of this is a complete transaction. Together with its requestor, this is also known as a commit.
Blinding
On ledgers with strong privacy, projections (see [Privacy Model](/overview/learn/privacy-model)) for all involved parties are created. This is also called *projecting*.
Transaction submission
The transaction/commit is submitted to the network.
Validation
The transaction/commit is validated by the network. Who exactly validates can differ from implementation to implementation. Validation also involves scheduling and collision detection, ensuring that the transaction has a well-defined place in the (partial) ordering of commits, and no double spends occur.
Commitment
The commit is actually committed according to the commit or consensus protocol of the ledger.
Confirmation
The network sends confirmations of the commitment back to all involved participant nodes.
Completion
The user gets back a confirmation through the Ledger API of the submitting participant node.
The first important consequence of the above is that all transactions are committed atomically. Either a transaction is committed as a whole and for all participants, or it fails.
That's important in the context of the `Trade_Settle` choice shown above. The choice transfers a `baseAsset` one way and a `quoteAsset` the other way. Thanks to transaction atomicity, there is no chance that either party is left out of pocket.
The second consequence is that the requester of a transaction knows all consequences of their submitted transaction -- there are no surprises in Daml. However, it also means that the requester must have all the information to interpret the transaction. We also refer to this as Principle 2 a bit later on this page.
That's also important in the context of `Trade`. In order to allow Bob to interpret a transaction that transfers Alice's cash to Bob, Bob needs to know both about Alice's `Asset` contract, as well as about some way for `Alice` to accept a transfer -- remember, accepting a transfer needs the authority of `issuer` in this example.
## Observers
*Observers* are Daml's mechanism to disclose contracts to other parties. They are declared just like signatories, but using the `observer` keyword, as shown in the `Asset` template:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Asset
with
issuer : Party
owner : Party
symbol : Text
quantity : Decimal
observers : [Party]
where
signatory issuer, owner
ensure quantity > 0.0
observer observers
```
The `Asset` template also gives the `owner` a choice to set the observers, and you can see how Alice uses it to show her `Asset` to Bob just before proposing the trade. You can try out what happens if she didn't do that by removing that transaction:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
usdCid <- submit alice do
exerciseCmd usdCid SetObservers with
newObservers = [bob]
```
Observers have guarantees in Daml. In particular, they are guaranteed to see actions that create and archive the contract on which they are an observer.
Since observers are calculated from the arguments of the contract, they always know about each other. That's why, rather than adding Bob as an observer on Alice's `AssetHolder` contract, and using that to authorize the transfer in `Trade_Settle`, Alice creates a one-time authorization in the form of a `TransferAuthorization`. If Alice had lots of counterparties, she would otherwise end up leaking them to each other.
Choice controllers are not automatically made observers, as they can only be calculated at the point in time when the choice arguments are known.
## Privacy
Daml's privacy model is based on two principles:
Principle 1. Parties see those actions that they have a stake in. Principle 2. Every party that sees an action sees its (transitive) consequences.
Principle 2 is necessary to ensure that every party can independently verify the validity of every transaction they see.
A party has a stake in an action if
* they are a required authorizer of it
* they are a signatory of the contract on which the action is performed
* they are an observer on the contract, and the action creates or archives it
What does that mean for the `exercise tradeCid Trade_Settle` action from `test_trade`?
Alice is the signatory of `tradeCid` and Bob a required authorizer of the `Trade_Settled` action, so both of them see it. According to principle 2 above, that means they get to see everything in the transaction.
The consequences contain, next to some `fetch` actions, two `exercise` actions of the choice `TransferApproval_Transfer`.
Each of the two involved `TransferApproval` contracts is signed by a different `issuer`, which see the action on "their" contract. So the EUR\_Bank sees the `TransferApproval_Transfer` action for the EUR `Asset` and the USD\_Bank sees the `TransferApproval_Transfer` action for the USD `Asset`.
Some Daml ledgers, like the script runner and the Sandbox, work on the principle of "data minimization", meaning nothing more than the above information is distributed. That is, the "projection" of the overall transaction that gets distributed to EUR\_Bank in step 4 of `execution_model` would consist only of the `TransferApproval_Transfer` and its consequences.
Other implementations, in particular those on public blockchains, may have weaker privacy constraints.
### Divulgence
Note that principle 2 of the privacy model means that sometimes parties see contracts that they are not signatories or observers on. If you look at the final ledger state of the `test_trade` script, for example, you may notice that both Alice and Bob now see both assets, as indicated by the Xs in their respective columns:
| Alice | Bob | EUR\_Bank | USD\_Bank | id | status | issuer | owner | symbol | quantity |
| ----- | --- | --------- | --------- | ------ | ------ | --------- | ----- | ------ | -------- |
| X | X | - | X | #15:6 | active | USD\_Bank | Bob | USD | 100.0 |
| X | X | X | - | #15:10 | active | EUR\_Bank | Alice | EUR | 90.0 |
This is because the `create` action of these contracts are in the transitive consequences of the `Trade_Settle` action both of them have a stake in. This kind of disclosure is often called "divulgence" and needs to be considered when designing Daml models for privacy sensitive applications.
## Common Daml design patterns
Beyond the composition patterns above, this section covers common multi-party workflow patterns used in Daml. All examples below use a `Coin` asset model to illustrate each pattern.
### Propose-Accept
The most common way to get multiple parties to agree on a shared contract. One party creates a proposal contract that the other party can accept, reject, or let expire. The `IouProposal` [in the authorization module](/appdev/modules/m3-authorization#use-propose-accept-workflow-for-one-off-authorization) is another example of this pattern.
The issuer creates a `CoinMaster` contract, then uses it to invite an owner. The invitation is a proposal contract with the issuer as signatory and the owner as observer:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template CoinMaster
with
issuer: Party
where
signatory issuer
nonconsuming choice Invite : ContractId CoinIssueProposal
with owner: Party
controller issuer
do create CoinIssueProposal
with coinAgreement = CoinIssueAgreement with issuer; owner
```
The proposal gives the owner a choice to accept. In a complete model, it would also include `Reject` and `Counter` choices:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template CoinIssueProposal
with
coinAgreement: CoinIssueAgreement
where
signatory coinAgreement.issuer
observer coinAgreement.owner
choice AcceptCoinProposal
: ContractId CoinIssueAgreement
controller coinAgreement.owner
do create coinAgreement
```
When the owner accepts, the result contract has both parties as signatories — neither can be forced into the agreement without consent:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template CoinIssueAgreement
with
issuer: Party
owner: Party
where
signatory issuer, owner
nonconsuming choice Issue : ContractId Coin
with amount: Decimal
controller issuer
do create Coin with issuer; owner; amount; delegates = []
```
This pattern can be verbose when more than two signatures are needed — see Multiple Party Agreement below for that case.
### Delegation
Gives one party the right to exercise a choice on behalf of another. The principal creates a delegation contract that authorizes an agent to act for them, without the principal committing each action. This models real-world custodian relationships where a bank holds securities and settles transactions on a client's behalf.
The delegation contract (`CoinPoA` — Power of Attorney) has the principal as signatory. The attorney controls a `TransferCoin` choice that exercises `Transfer` on the principal's coin:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template CoinPoA
with
attorney: Party
principal: Party
where
signatory principal
observer attorney
choice WithdrawPoA
: ()
controller principal
do return ()
-- Attorney has the delegated right to Transfer
nonconsuming choice TransferCoin
: ContractId TransferProposal
with
coinId: ContractId Coin
newOwner: Party
controller attorney
do
exercise coinId Transfer with newOwner
```
The coin must be disclosed to the attorney before they can exercise the delegated choice. This is done by adding them as an observer via a `Disclose` choice on `Coin`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Disclose : ContractId Coin
with p : Party
controller owner
do create this with delegates = p :: delegates
```
### Authorization
Verifies that a controlling party has the right permissions before they take certain actions. An authorization contract serves as proof — the choice body checks for its existence and validity before proceeding.
For example, an issuer wants to ensure that only accredited parties can receive coin transfers. The issuer creates an authorization token for approved owners:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template CoinOwnerAuthorization
with
owner: Party
issuer: Party
where
signatory issuer
observer owner
choice WithdrawAuthorization
: ()
controller issuer
do return ()
```
The `AcceptTransfer` choice on `TransferProposal` requires the new owner to supply their authorization token. The asserts verify the token matches the issuer and the new owner:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice AcceptTransfer
: ContractId Coin
with token: ContractId CoinOwnerAuthorization
controller newOwner
do
t <- fetch token
assert (coin.issuer == t.issuer)
assert (newOwner == t.owner)
create coin with owner = newOwner
```
If the issuer withdraws the authorization before the transfer is accepted, the transfer fails.
### Locking
Prevents choices from being exercised on a contract while it is in a locked state. Useful for scenarios like securities settlement where assets must be frozen during clearing.
One approach is **locking by state change** — the contract carries a `locker` field. When `owner == locker`, the coin is unlocked and can be transferred. When they differ, a third-party locker controls the unlock:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template LockableCoin
with
owner: Party
issuer: Party
amount: Decimal
locker: Party
where
signatory issuer, owner
observer locker
ensure amount > 0.0
-- Transfer can only happen if not locked
choice Transfer : ContractId TransferProposal
with newOwner: Party
controller owner
do
assert (locker == owner)
create TransferProposal with coin=this; newOwner
-- Lock by bringing a locker on board
choice Lock : ContractId LockableCoin
with newLocker: Party
controller owner
do
assert (newLocker /= owner)
create this with locker = newLocker
-- Unlock restores owner control
choice Unlock
: ContractId LockableCoin
controller locker
do
assert (locker /= owner)
create this with locker = owner
```
Two other approaches exist: **locking by archiving** (archive the original contract and create a `LockedCoin` wrapper with `Unlock` and `Clawback` choices) and **locking by safekeeping** (transfer custody to a trusted third party who controls the unlock).
### Multiple party agreement
Collects signatures from more than two parties. A `Pending` contract wraps the final `Agreement` and tracks who has signed. Each party signs by exercising a `Sign` choice, and once all parties have signed, any of them can `Finalize` to create the agreement.
The final agreement contract has multiple signatories:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Agreement
with
signatories: [Party]
where
signatory signatories
ensure unique signatories
```
The `Pending` contract collects signatures one by one. It is observable by all required signatories, so each can see when it is their turn to sign:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
toSign : Pending -> [Party]
toSign Pending { alreadySigned, finalContract } =
filter (`notElem` alreadySigned) finalContract.signatories
template Pending
with
finalContract: Agreement
alreadySigned: [Party]
where
signatory alreadySigned
observer finalContract.signatories
ensure unique alreadySigned
choice Sign : ContractId Pending with
signer : Party
controller signer
do
assert (signer `elem` toSign this)
create this with alreadySigned = signer :: alreadySigned
choice Finalize : ContractId Agreement with
signer : Party
controller signer
do
assert (sort alreadySigned == sort finalContract.signatories)
create finalContract
```
One party kicks off the workflow by creating a `Pending` contract listing only themselves as signed. The others sign in any order, and once complete, any signatory can finalize:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Any party can kick off by creating a Pending listing only themselves
pending <- person1 `submit` do
createCmd Pending with finalContract; alreadySigned = [person1]
-- Each party signs
pending <- person2 `submit` do exerciseCmd pending Sign with signer = person2
pending <- person3 `submit` do exerciseCmd pending Sign with signer = person3
pending <- person4 `submit` do exerciseCmd pending Sign with signer = person4
-- Once all have signed, any signatory can finalize
person1 `submit` do exerciseCmd pending Finalize with signer = person1
```
# Compose choices
It's time to put everything you've learned so far together into a complete and secure Daml model for asset issuance, management, transfer, and trading. This application will have capabilities similar to the one in the [CN Quickstart](/sdks-tools/reference-projects/cn-quickstart). In the process you will learn about a few more concepts:
* Daml projects, packages, and modules
* Composition of transactions
* Observers and stakeholders
* Daml's execution model
* Privacy
The model in this section is not a single Daml file, but a Daml project consisting of several files that depend on each other.
Remember that you can load all the code for this section into a folder called `intro-compose` by running `dpm new intro-compose --template daml-intro-compose`
## Daml projects
Daml is organized in projects, packages, and modules. A Daml project is specified using a single `daml.yaml` file, and compiles into a package in Daml's intermediate language, or bytecode equivalent, Daml-LF. Each Daml file within a project becomes a Daml module, which is a bit like a namespace. Each Daml project has a source root specified in the `source` parameter in the project's `daml.yaml` file. The package will include all modules specified in `*.daml` files beneath that source directory.
You can start a new project with a skeleton structure using `dpm new project-name` in the terminal. A minimal project would contain just a `daml.yaml` file and an empty directory of source files.
> Take a look at the `daml.yaml` for the this chapter's project:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
sdk-version: __VERSION__
name: __PROJECT_NAME__
source: daml
version: 1.0.0
dependencies:
- daml-prim
- daml-stdlib
- daml-script
```
You can generally set `name` and `version` freely to describe your project. `dependencies` does what the name suggests: it includes dependencies. You should always include `daml-prim` and `daml-stdlib`. The former contains internals of the compiler and the Daml Runtime, the latter gives access to the Daml standard library. `daml-script` contains the types and functions for Daml Script.
You compile a Daml project by running `dpm build` from the project root directory. This creates a DAR file in `.daml/dist/dist/${project_name}-${project_version}.dar`. A DAR file is Daml's equivalent of a JAR file in Java: it's the artifact that gets deployed to a ledger to load the package and its dependencies. `dar` files are fully self-contained in that they contain all dependencies of the main package. More on all of this in `dependencies`.
## Project structure
This project contains an asset holding model for transferable, fungible assets and a separate trade workflow. The templates are structured in three modules: `Intro.Asset`, `Intro.Asset.Role`, and `Intro.Asset.Trade`.
In addition, there are tests in modules `Test.Intro.Asset`, `Test.Intro.Asset.Role`, and `Test.Intro.Asset.Trade`.
All but the last `.`-separated segment in module names correspond to paths relative to the project source directory, and the last one to a file name. The folder structure therefore looks like this:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
.
├── daml
│ ├── Intro
│ │ ├── Asset
│ │ │ ├── Role.daml
│ │ │ └── Trade.daml
│ │ └── Asset.daml
│ └── Test
│ └── Intro
│ ├── Asset
│ │ ├── Role.daml
│ │ └── Trade.daml
│ └── Asset.daml
└── daml.yaml
```
Each file contains a module header. For example, `daml/Intro/Asset/Role.daml`:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Intro.Asset.Role where
```
You can import one module into another using the `import` keyword. The `LibraryModules` module imports all six modules:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
import Intro.Asset
```
Imports always have to appear just below the module declaration. You can optionally add a list of names after the import to import only the selected names:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
import DA.List (sortOn, groupOn)
```
If your module contains any Daml Scripts, you need to import the corresponding functionality:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
import Daml.Script
```
## Project overview
The project both changes and adds to the `Iou` model presented in `parties`:
* Assets are fungible in the sense that they have `Merge` and `Split` choices that allow the `owner` to manage their holdings.
* Transfer proposals now need the authorities of both `issuer` and `newOwner` to accept. This makes `Asset` safer than `Iou` from the issuer's point of view.
With the `Iou` model, an `issuer` could end up owing cash to anyone as transfers were authorized by just `owner` and `newOwner`. In this project, only parties having an `AssetHolder` contract can end up owning assets. This allows the `issuer` to determine which parties may own their assets.
* The `Trade` template adds a swap of two assets to the model.
## Composed choices and scripts
This project showcases how you can put the `Update` and `Script` actions you learned about in `parties` to good use. For example, the `Merge` and `Split` choices each perform several actions in their consequences.
* Two create actions in case of `Split`
* One create and one archive action in case of `Merge`
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Split
: SplitResult
with
splitQuantity : Decimal
controller owner
do
splitAsset <- create this with
quantity = splitQuantity
remainder <- create this with
quantity = quantity - splitQuantity
return SplitResult with
splitAsset
remainder
choice Merge
: ContractId Asset
with
otherCid : ContractId Asset
controller owner
do
other <- fetch otherCid
assertMsg
"Merge failed: issuer does not match"
(issuer == other.issuer)
assertMsg
"Merge failed: owner does not match"
(owner == other.owner)
assertMsg
"Merge failed: symbol does not match"
(symbol == other.symbol)
archive otherCid
create this with
quantity = quantity + other.quantity
```
The `return` function used in `Split` is available in any `Action` context. The result of `return x` is a no-op containing the value `x`. It has an alias `pure`, indicating that it's a pure value, as opposed to a value with side-effects. The `return` name makes sense when it's used as the last statement in a `do` block as its argument is indeed the "return"-value of the `do` block in that case.
Taking transaction composition a step further, the `Trade_Settle` choice on `Trade` composes two `exercise` actions:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Trade_Settle
: (ContractId Asset, ContractId Asset)
with
quoteAssetCid : ContractId Asset
baseApprovalCid : ContractId TransferApproval
controller quoteAsset.owner
do
fetchedBaseAsset <- fetch baseAssetCid
assertMsg
"Base asset mismatch"
(baseAsset == fetchedBaseAsset with
observers = baseAsset.observers)
fetchedQuoteAsset <- fetch quoteAssetCid
assertMsg
"Quote asset mismatch"
(quoteAsset == fetchedQuoteAsset with
observers = quoteAsset.observers)
transferredBaseCid <- exercise
baseApprovalCid TransferApproval_Transfer with
assetCid = baseAssetCid
transferredQuoteCid <- exercise
quoteApprovalCid TransferApproval_Transfer with
assetCid = quoteAssetCid
return (transferredBaseCid, transferredQuoteCid)
```
The resulting transaction, with its two nested levels of consequences, can be seen in the `test_trade` script in `Test.Intro.Asset.Trade`:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
TX 14 1970-01-01T00:00:00Z (Test.Intro.Asset.Trade:79:23)
#14:0
│ disclosed to (since): 'Alice' (14), 'Bob' (14)
└─> 'Bob' exercises Trade_Settle on #12:0 (Intro.Asset.Trade:Trade)
with
quoteAssetCid = #9:1; baseApprovalCid = #13:1
children:
#14:1
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Alice' and 'USD_Bank' fetch #10:1 (Intro.Asset:Asset)
#14:2
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Bob' and 'EUR_Bank' fetch #9:1 (Intro.Asset:Asset)
#14:3
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Alice' and 'Bob' exercise TransferApproval_Transfer on #13:1 (Intro.Asset:TransferApproval)
with
assetCid = #10:1
children:
#14:4
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Alice' and 'USD_Bank' fetch #10:1 (Intro.Asset:Asset)
#14:5
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Alice' and 'USD_Bank' exercise Archive on #10:1 (Intro.Asset:Asset)
#14:6
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'USD_Bank' (14)
└─> 'Bob' and 'USD_Bank' create Intro.Asset:Asset
with
issuer = 'USD_Bank';
owner = 'Bob';
symbol = "USD";
quantity = 100.0000000000;
observers = []
#14:7
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Alice',
'Bob' exercises TransferApproval_Transfer on #11:1 (Intro.Asset:TransferApproval)
with
assetCid = #9:1
children:
#14:8
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Bob' and 'EUR_Bank' fetch #9:1 (Intro.Asset:Asset)
#14:9
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Bob' and 'EUR_Bank' exercise Archive on #9:1 (Intro.Asset:Asset)
#14:10
│ disclosed to (since): 'Alice' (14), 'Bob' (14), 'EUR_Bank' (14)
└─> 'Alice' and 'EUR_Bank' create Intro.Asset:Asset
with
issuer = 'EUR_Bank';
owner = 'Alice';
symbol = "EUR";
quantity = 90.0000000000;
observers = []
```
Similar to choices, you can see how the scripts in this project are built up from each other:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
test_issuance = do
setupResult@(alice, bob, bank, aha, ahb) <- setupRoles
assetCid <- submit bank do
exerciseCmd aha Issue_Asset
with
symbol = "USD"
quantity = 100.0
Some asset <- queryContractId bank assetCid
assert (asset == Asset with
issuer = bank
owner = alice
symbol = "USD"
quantity = 100.0
observers = []
)
return (setupResult, assetCid)
```
In the above, the `test_issuance` script in `Test.Intro.Asset.Role` uses the output of the `setupRoles` script in the same module.
The same line shows a new kind of pattern matching. Rather than writing `setupResult <- setupRoles` and then accessing the components of `setupResult` using `_1`, `_2`, etc., you can give them names. It's equivalent to writing:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
setupResult <- setupRoles
case setupResult of
(alice, bob, bank, aha, ahb) -> ...
```
Just writing `(alice, bob, bank, aha, ahb) <- setupRoles` would also be legal, but `setupResult` is used in the return value of `test_issuance` so it makes sense to give it a name, too. The notation with `@` allows you to give both the whole value as well as its constituents names in one go.
## Daml's execution model
Daml's execution model is fairly easy to understand, but has some important consequences. You can imagine the life of a transaction as follows:
Command submission
A user submits a list of commands via the Ledger API of a participant node, acting as a `Party` hosted on that node. That party is called the requester.
Interpretation
Each command corresponds to one or more actions. During this step, the `Update` corresponding to each action is evaluated in the context of the ledger to calculate all consequences, including transitive ones (consequences of consequences, etc.). The result of this is a complete transaction. Together with its requestor, this is also known as a commit.
Blinding
On ledgers with strong privacy, projections (see `privacy`) for all involved parties are created. This is also called *projecting*.
Transaction submission
The transaction/commit is submitted to the network.
Validation
The transaction/commit is validated by the network. Who exactly validates can differ from implementation to implementation. Validation also involves scheduling and collision detection, ensuring that the transaction has a well-defined place in the (partial) ordering of commits, and no double spends occur.
Commitment
The commit is actually committed according to the commit or consensus protocol of the ledger.
Confirmation
The network sends confirmations of the commitment back to all involved participant nodes.
Completion
The user gets back a confirmation through the Ledger API of the submitting participant node.
The first important consequence of the above is that all transactions are committed atomically. Either a transaction is committed as a whole and for all participants, or it fails.
That's important in the context of the `Trade_Settle` choice shown above. The choice transfers a `baseAsset` one way and a `quoteAsset` the other way. Thanks to transaction atomicity, there is no chance that either party is left out of pocket.
The second consequence is that the requester of a transaction knows all consequences of their submitted transaction -- there are no surprises in Daml. However, it also means that the requester must have all the information to interpret the transaction. We also refer to this as Principle 2 a bit later on this page.
That's also important in the context of `Trade`. In order to allow Bob to interpret a transaction that transfers Alice's cash to Bob, Bob needs to know both about Alice's `Asset` contract, as well as about some way for `Alice` to accept a transfer -- remember, accepting a transfer needs the authority of `issuer` in this example.
## Observers
*Observers* are Daml's mechanism to disclose contracts to other parties. They are declared just like signatories, but using the `observer` keyword, as shown in the `Asset` template:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Asset
with
issuer : Party
owner : Party
symbol : Text
quantity : Decimal
observers : [Party]
where
signatory issuer, owner
ensure quantity > 0.0
observer observers
```
The `Asset` template also gives the `owner` a choice to set the observers, and you can see how Alice uses it to show her `Asset` to Bob just before proposing the trade. You can try out what happens if she didn't do that by removing that transaction:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
usdCid <- submit alice do
exerciseCmd usdCid SetObservers with
newObservers = [bob]
```
Observers have guarantees in Daml. In particular, they are guaranteed to see actions that create and archive the contract on which they are an observer.
Since observers are calculated from the arguments of the contract, they always know about each other. That's why, rather than adding Bob as an observer on Alice's `AssetHolder` contract, and using that to authorize the transfer in `Trade_Settle`, Alice creates a one-time authorization in the form of a `TransferAuthorization`. If Alice had lots of counterparties, she would otherwise end up leaking them to each other.
Choice controllers are not automatically made observers, as they can only be calculated at the point in time when the choice arguments are known.
## Privacy
Daml's privacy model is based on two principles:
Principle 1. Parties see those actions that they have a stake in. Principle 2. Every party that sees an action sees its (transitive) consequences.
Principle 2 is necessary to ensure that every party can independently verify the validity of every transaction they see.
A party has a stake in an action if
* they are a required authorizer of it
* they are a signatory of the contract on which the action is performed
* they are an observer on the contract, and the action creates or archives it
What does that mean for the `exercise tradeCid Trade_Settle` action from `test_trade`?
Alice is the signatory of `tradeCid` and Bob a required authorizer of the `Trade_Settled` action, so both of them see it. According to principle 2 above, that means they get to see everything in the transaction.
The consequences contain, next to some `fetch` actions, two `exercise` actions of the choice `TransferApproval_Transfer`.
Each of the two involved `TransferApproval` contracts is signed by a different `issuer`, which see the action on "their" contract. So the EUR\_Bank sees the `TransferApproval_Transfer` action for the EUR `Asset` and the USD\_Bank sees the `TransferApproval_Transfer` action for the USD `Asset`.
Some Daml ledgers, like the script runner and the Sandbox, work on the principle of "data minimization", meaning nothing more than the above information is distributed. That is, the "projection" of the overall transaction that gets distributed to EUR\_Bank in step 4 of `execution_model` would consist only of the `TransferApproval_Transfer` and its consequences.
Other implementations, in particular those on public blockchains, may have weaker privacy constraints.
### Divulgence
Note that principle 2 of the privacy model means that sometimes parties see contracts that they are not signatories or observers on. If you look at the final ledger state of the `test_trade` script, for example, you may notice that both Alice and Bob now see both assets, as indicated by the Xs in their respective columns:
![images/compose/divulgence.png]()
This is because the `create` action of these contracts are in the transitive consequences of the `Trade_Settle` action both of them have a stake in. This kind of disclosure is often called "divulgence" and needs to be considered when designing Daml models for privacy sensitive applications.
## Next up
In `exceptions`, we will learn about how errors in your model can be handled in Daml.
# Development Environment Setup
Source: https://docs.canton.network/appdev/modules/m3-dev-environment
Set up your development environment for writing Daml smart contracts
## Introduction
Daml is a smart contract language designed to build composable applications on Canton ledgers.
In this module, you will learn about the structure of a Daml ledger and how to write Daml applications that run on the Canton Network by building an asset-holding and trading application. You will gain an overview of the most important language features and how to use Daml's developer tools to write, test, compile, package and ship your application.
## Prerequisites
* You have installed [dpm](/sdks-tools/cli-tools/dpm)
## Loading Example Code
Each section in this module presents a new self-contained application with more functionality than the previous section. You can load the code for each section using `dpm`. For example:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Load the contract templates example
dpm new intro-contracts --template daml-intro-contracts
# Load the choices example
dpm new intro-choices --template daml-intro-choices
```
## Next Steps
Continue to [Contract Templates](/appdev/modules/m3-contract-templates) to start building Daml smart contracts.
If you're new to functional programming or want a refresher on Daml's syntax (types, pattern matching, records, type classes), see [Language Fundamentals](/appdev/modules/m3-language-fundamentals) first. If you have experience with Haskell or other ML-family languages, you can skip it and refer back as needed.
# Interfaces
Source: https://docs.canton.network/appdev/modules/m3-interfaces
Define and implement Daml interfaces for polymorphic contract behavior and cross-template interoperability
# Reference: Interfaces
In Daml, an interface defines an abstract type together with a behavior specified by its view type, method signatures, and choices. For a template to conform to this interface, there must be a corresponding `interface instance` definition where all the methods of the interface (including the special `view` method) are implemented. This allows decoupling such behavior from its implementation, so other developers can write applications in terms of the interface instead of the concrete template.
## Configuration
To use this feature your Daml project must target Daml-LF version `1.15` or higher, which is the current default.
If using Canton, the protocol version of the sync domain should be `4` or higher, see Canton protocol version for more details.
## Interface Declaration
An interface declaration is somewhat similar to a template declaration.
### Interface Name
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
interface MyInterface where
```
* This is the name of the interface.
* It's preceded by the keyword `interface` and followed by the keyword `where`.
* It must begin with a capital letter, like any other type name.
### Implicit abstract type
* Whenever an interface is defined, an abstract type is defined with the same name. "Abstract" here means:
* Values of this type cannot be created using a data constructor. Instead, they are constructed by applying the function `toInterface` to a template value.
* Values of this type cannot be inspected directly via case analysis. Instead, use functions such as `fromInterface`.
* See `daml-ref-interface-functions` for more information on these and other functions for interacting with interface values.
* An interface value carries inside it the type and parameters of the template value from which it was constructed.
* As for templates, the existence of a local binding `b` of type `I`, where `I` is an interface does not necessarily imply the existence on the ledger of a contract with the template type and parameters used to construct `b`. This can only be assumed if `b` the result of a fetch from the ledger within the same transaction.
### Interface Methods
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
method1 : Party
method2 : Int
method3 : Bool -> Int -> Int -> Int
```
* An interface may define any number of methods.
* A method definition consists of the method name and the method type, separated by a single colon `:`. The name of the method must be a valid identifier beginning with a lowercase letter or an underscore.
* A method definition introduces a top level function of the same name:
* If the interface is called `I`, the method is called `m`, and the method type is `M` (which might be a function type), this introduces the function `m : I -> M`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
func1 : MyInterface -> Party
func1 = method1
func2 : MyInterface -> Int
func2 = method2
func3 : MyInterface -> Bool -> Int -> Int -> Int
func3 = method3
```
* The first argument's type `I` means that the function can only be applied to values of the interface type `I` itself. Methods cannot be applied to template values, even if there exists an `interface instance` of `I` for that template. To use an interface method on a template value, first convert it using the `toInterface` function.
* Applying the function to such argument results in a value of type `M`, corresponding to the implementation of `m` in the interface instance of `I` for the underlying template type `t` (the type of the template value from which the interface value was constructed).
* One special method, `view`, must be defined for the viewtype. (see **Viewtype** below).
### Interface View Type
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data MyInterfaceViewType =
MyInterfaceViewType { name : Text, value : Int }
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
viewtype MyInterfaceViewType
```
* All interface instances must implement a special `view` method which returns a value of the type declared by `viewtype`.
* The type must be a record.
* This type is returned by subscriptions on interfaces.
### Interface Choices
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice MyChoice : (ContractId MyInterface, Int)
with
argument1 : Bool
argument2 : Int
controller method1 this
do
let n0 = method2 this
let n1 = method3 this argument1 argument2 n0
pure (self, n1)
nonconsuming choice MyNonConsumingChoice : Int
controller method1 this
do
pure $ method2 this
```
* Interface choices work in a very similar way to template choices. Any contract of a template type for which an interface instance exists will grant the choice to the controlling party.
* Interface choices can only be exercised on values of the corresponding interface type. To exercise an interface choice on a template value, first convert it using the `toInterface` function.
* Interface methods can be used to define the controller of a choice (e.g. `method1`) as well as the actions that run when the choice is *exercised* (e.g. `method2` and `method3`).
* As for template choices, the `choice` keyword can be optionally prefixed with the `nonconsuming` keyword to specify that the contract will not be consumed when the choice is exercised. If not specified, the choice will be `consuming`. Note that the `preconsuming` and `postconsuming` qualifiers are not supported on interface choices.
* See `choices` for full reference information, but note that controller-first syntax is not supported for interface choices.
### Empty Interfaces
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data EmptyInterfaceView = EmptyInterfaceView {}
interface YourInterface where
viewtype EmptyInterfaceView
```
* It is possible (though not necessarily useful) to define an interface without methods, precondition or choices. However, a view type must always be defined, though it can be set to unit.
## Interface Instances
For context, a simple template definition:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template NameOfTemplate
with
field1 : Party
field2 : Int
where
signatory field1
```
### `interface instance` clause
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
interface instance MyInterface for NameOfTemplate where
view = MyInterfaceViewType "NameOfTemplate" 100
method1 = field1
method2 = field2
method3 False _ _ = 0
method3 True x y
| x > 0 = x + y
| otherwise = y
```
* To make a template an instance of an existing interface, an `interface instance` clause must be defined in the template declaration.
* The template of the clause must match the enclosing declaration. In other words, a template `T` declaration can only contain `interface instance` clauses where the template is `T`.
* The clause must start with the keywords `interface instance`, followed by the name of the interface, then the keyword `for` and the name of the template, and finally the keyword `where`, which introduces a block where **all** the methods of the interface must be implemented.
* Within the clause, there's an implicit local binding `this` referring to the contract on which the method is applied, which has the type of the template's data record. The template parameters of this contract are also in scope.
* Method implementations can be defined using the same syntax as for top level functions, including pattern matches and guards (e.g. `method3`).
* Each method implementation must return the same type as specified for that method in the interface declaration.
* The implementation of the special `view` method must return the type specified as the `viewtype` in the interface declaration.
### Empty `interface instance` clause
* If the interface has no methods, the interface instance only needs to implement the `view` method:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
interface instance YourInterface for NameOfTemplate where
view = EmptyInterfaceView
```
## Interface Functions
### `interfaceTypeRep`
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------- |
| Type | `HasInterfaceTypeRep i =>`
`i -> TemplateTypeRep` |
| Instantiated Type | `MyInterface -> TemplateTypeRep` |
| Notes | The value of the resulting `TemplateTypeRep` indicates what template was used to construct the interface value. |
### `toInterface`
| | |
| ----------------- | -------------------------------------------------------- |
| Type | `forall i t.`
`HasToInterface t i =>`
`t -> i` |
| Instantiated Type | `MyTemplate -> MyInterface` |
| Notes | Converts a template value into an interface value. |
### `fromInterface`
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Type | `HasFromInterface t i =>`
`i -> Optional t` |
| Instantiated Type | `MyInterface -> Optional MyTemplate` |
| Notes | Attempts to convert an interface value back into a template value. The result is `None` if the expected template type doesn't match the underlying template type used to construct the contract. |
### `toInterfaceContractId`
| | |
| ----------------- | ------------------------------------------------------------------------------ |
| Type | `forall i t.`
`HasToInterface t i =>`
`ContractId t -> ContractId i` |
| Instantiated Type | `ContractId MyTemplate -> ContractId MyInterface` |
| Notes | Converts a template Contract ID into an Interface Contract ID. |
### `fromInterfaceContractId`
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Type | `forall t i.`
`HasFromInterface t i =>`
`ContractId i -> ContractId t` |
| Instantiated Type | `ContractId MyInterface -> ContractId MyTemplate` |
| Notes | Converts an interface contract id into a template contract id. This function does not verify that the given contract id actually points to a contract of the resulting type; if that is not the case, a subsequent `fetch`, `exercise` or `archive` will fail. Therefore, this should only be used when the underlying contract is known to be of the resulting type, or when the result is immediately used by a `fetch`, `exercise` or `archive` action and a transaction failure is the desired behavior in case of mismatch. In all other cases, consider using `fetchFromInterface` instead. |
### `coerceInterfaceContractId`
| | |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Type | `forall j i.`
`(HasInterfaceTypeRep i, HasInterfaceTypeRep j) =>`
`ContractId i -> ContractId j` |
| Instantiated Type | `ContractId SourceInterface -> ContractId TargetInterface` |
| Notes | Converts an interface contract id into a contract id of a different interface. This function does not verify that the given contract id actually points to a contract of the resulting type; if that is not the case, a subsequent `fetch`, `exercise` or `archive` will fail. Therefore, this should only be used when the underlying contract is known to be of the resulting type, or when the result is immediately used by a `fetch`, `exercise` or `archive` action and a transaction failure is the desired behavior in case of mismatch. |
### `fetchFromInterface`
| | |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Type | `forall t i.`
`(HasFromInterface t i, HasFetch i) =>`
`ContractId i -> Update (Optional (ContractId t, t))` |
| Instantiated Type | `ContractId MyInterface ->`
`Update (Optional (ContractId MyTemplate, MyTemplate))` |
| Notes | Attempts to fetch and convert an interface contract id into a template, returning both the converted contract and its contract id if the conversion is successful, or `None` otherwise. |
## Required Interfaces
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
interface OurInterface requires MyInterface, YourInterface where
viewtype EmptyInterfaceView
```
* An interface can depend on other interfaces. These are specified with the `requires` keyword after the interface name but before the `where` keyword, separated by commas.
* For an interface declaration to be valid, its list of required interfaces must be transitively closed. In other words, an interface `I` cannot require an interface `J` without also explicitly requiring all the interfaces required by `J`. The order, however, is irrelevant.
For example, given
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
interface Shape where
viewtype EmptyInterfaceView
interface Rectangle requires Shape where
viewtype EmptyInterfaceView
```
This declaration for interface `Square` would cause a compiler error
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Compiler error! "Interface Square is missing requirement [Shape]"
interface Square requires Rectangle where
viewtype EmptyInterfaceView
```
Explicitly adding `Shape` to the required interfaces fixes the error
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
interface Square requires Rectangle, Shape where
viewtype EmptyInterfaceView
```
* For a template `T` to be a valid `interface instance` of an interface `I`, `T` must also be an `interface instance` of each of the interfaces required by `I`.
### Interface Functions
| Function | Notes |
| ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `toInterface` | Can also be used to convert an interface value to one of its required interfaces. |
| `fromInterface` | Can also be used to convert a value of an interface type to one of its requiring interfaces. |
| `toInterfaceContractId` | Can also be used to convert an interface contract id into a contract id of one of its required interfaces. |
| `fromInterfaceContractId` | Can also be used to convert an interface contract id into a contract id of one of its requiring interfaces. |
| `fetchFromInterface` | Can also be used to fetch and convert an interface contract id into a contract and contract id of one of its requiring interfaces. |
## Interfaces on Canton Network
Interfaces are central to interoperability across the Canton Network. Two Canton Improvement Proposals (CIPs) define standard interfaces that app developers should be aware of:
* [CIP-0056 — Token Standard](https://github.com/canton-foundation/cips/blob/main/cip-0056/cip-0056.md): Defines a standard interface for fungible tokens on Canton Network. If your application issues or transfers tokens, implementing this interface ensures compatibility with wallets and other applications in the ecosystem.
* [CIP-0103 — dApp Standard](https://github.com/mjuchli-da/cips/blob/cip-dapp-standard/cip-0103/cip-0103.md): Defines standard interfaces for decentralized applications. This CIP is particularly relevant for developers coming from Ethereum, as it maps familiar ERC-style patterns to Daml interfaces.
When building applications on Canton Network, implementing these standard interfaces lets your contracts participate in the broader ecosystem without requiring custom integration with each counterparty.
# Language Fundamentals
Source: https://docs.canton.network/appdev/modules/m3-language-fundamentals
Learn the fundamentals of Daml - a functional programming language designed for smart contracts
## Introduction
In this chapter, you will learn more about expressing complex logic in a functional language like Daml. Specifically, you'll learn about:
* Function signatures and functions
* Advanced control flow (`if...else`, folds, recursion, `when`)
There is a project template `daml-intro-functional-101` for this chapter, containing the code snippets from this section.
## The Haskell connection
The previous chapters of this introduction to Daml have mostly covered the structure of templates, and their connection to the Daml Ledger Model. The logic of what happens within the `do` blocks of choices has been kept relatively simple. In this chapter, we will dive deeper into Daml's expression language, the part that allows you to write logic inside those `do` blocks. But we can only scratch the surface here. Daml borrows a lot of its language from [Haskell](https://www.haskell.org). If you want to dive deeper, or learn about specific aspects of the language you can refer to standard literature on Haskell. Some recommendations:
* [Finding Success and Failure in Haskell (Julie Moronuki, Chris Martin)](https://joyofhaskell.com/)
* [Haskell Programming from first principles (Christopher Allen, Julie Moronuki)](http://haskellbook.com/)
* [Learn You a Haskell for Great Good! (Miran Lipovača)](http://learnyouahaskell.com/)
* [Programming in Haskell (Graham Hutton)](http://www.cs.nott.ac.uk/~pszgmh/pih.html)
* [Real World Haskell (Bryan O'Sullivan, Don Stewart, John Goerzen)](http://book.realworldhaskell.org/)
When comparing Daml to Haskell it's worth noting:
* Haskell is a lazy language, which allows you to write things like `head [1..]`, meaning "take the first element of an infinite list". Daml by contrast is strict. Expressions are fully evaluated, which means it is not possible to work with infinite data structures.
* Daml has a `with` syntax for records and a dot syntax for record field access, neither of which is present in Haskell. However, Daml supports Haskell's curly brace record notation.
* Daml has a number of Haskell compiler extensions active by default.
* Daml doesn't support all features of Haskell's type system. For example, there are no existential types or GADTs.
* Actions are called Monads in Haskell.
## Functions
In `data` you learnt about one half of Daml's type system: Data types. It's now time to learn about the other, which are Function types. Function types in Daml can be spotted by looking for `->` which can be read as "maps to".
For example, the function signature `Int -> Int` maps an integer to another integer. There are many such functions, but one would be:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
increment : Int -> Int
increment n = n + 1
```
You can see here that the function declaration and the function definitions are separate. The declaration can be omitted in cases where the type can be inferred by the compiler, but for top-level functions (ie ones at the same level as templates, directly under a module), it's often a good idea to include them for readability.
In the case of `increment` it could have been omitted. Similarly, we could define a function `add` without a declaration:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
add n m = n + m
```
If you do this, and wonder what type the compiler has inferred, you can hover over the function name in the IDE:
```
add
: Additive a
=> a -> a -> a
Defined at /tmp/daml-intro-9/daml/Main.daml:20:1
add n m = n + m
```
What you see here is a slightly more complex signature:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
add : Additive a => a -> a -> a
```
There are two interesting things going on here:
1. We have more than one `->`.
2. We have a type parameter `a` with a constraint `Additive a`.
### Function application
Let's start by looking at the right hand part `a -> a -> a`. The `->` is right associative, meaning `a -> a -> a` is equivalent to `a -> (a -> a)`. Using the "maps to" way of reading `->`, we get "`a` maps to a function that maps `a` to `a`".
And this is indeed what happens. We can define a different version of `increment` by *partially applying* `add`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
increment2 = add 1
```
If you try this out in your IDE, you'll see that the compiler infers type `Int -> Int` again. It can do so because of the literal `1 : Int`.
So if we have a function `f : a -> b -> c -> d` and a value `valA : a`, we get `f valA : b -> c -> d`, i.e. we can apply the function argument by argument. If we also had `valB : b`, we would have `f valA valB : c -> d`. What this tells you is that function *application* is left associative: `f valA valB == (f valA) valB`.
### Infix functions
Now `add` is clearly just an alias for `+`, but what is `+`? `+` is just a function. It's only special because it starts with a symbol. Functions that start with a symbol are *infix* by default which means they can be written between two arguments. That's why we can write `1 + 2` rather than `+ 1 2`. The rules for converting between normal and infix functions are simple. Wrap an infix function in parentheses to use it as a normal function (i.e. *prefix*), and wrap a normal function in backticks to make it infix:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
three = 1 `add` 2
```
With that knowledge, we could have defined `add` more succinctly as the alias that it is:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
add2 : Additive a => a -> a -> a
add2 = (+)
```
If we want to partially apply an infix operation we can also do that as follows:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
increment3 = (1 +)
double = (* 2)
```
#### Associativity and Precedence
When dealing with multiple infix operators, precedence determines how the Daml compiler should parse an expression. For example, for the expression `x + y * z`, because `\*` has a higher precedence than `+`, the expression is parsed as `x + (y * z)` instead of `(x + y) * z`. When dealing with infix operators with the same precedence, associativity determines how the Daml compiler should parse an expression. For example, because `+` and `-` are left-associative, the expression `x + y - z` is parsed as `(x + y) - z` instead of `x + (y - z)`. For built-in operators this has been predefined, for user-defined operators, it must be user-defined. See the [Daml reference on Fixity, Associativity and Precedence](/appdev/reference/daml-language-reference#fixity-associativity-and-precedence) for details.
### Type constraints
The `Additive a =>` part of the signature of `add` is a type constraint on the type parameter `a`. `Additive` here is a typeclass. You already met typeclasses like `Eq` and `Show` earlier. The `Additive` typeclass says that you can add a thing, i.e. there is a function `(+) : a -> a -> a`. Now the way to read the full signature of `add` is "Given that `a` has an instance for the `Additive` typeclass, `a` maps to a function which maps `a` to `a`".
Typeclasses in Daml are a bit like interfaces in other languages. To be able to add two things using the `+` function, those things need to "expose" (have an instance for) the `Additive` interface (typeclass).
Unlike interfaces, typeclasses can have multiple type parameters. A good example, which also demonstrates the use of multiple constraints at the same time, is the signature of the `exercise` function:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
exercise : (Template t, Choice t c r) => ContractId t -> c -> Update r
```
Let's turn this into prose: Given that `t` is the type of a template, and that `t` has a choice `c` with return type `r`, the `exercise` function maps a `ContractId` for a contract of type `t` to a function that takes the choice arguments of type `c` and returns an `Update` resulting in type `r`.
That's quite a mouthful, and does require one to know what *meaning* the typeclass `Choice` gives to parameters `t` `c` and `r`, but in many cases, that's obvious from the context or names of typeclasses and variables.
Using single letters, while common, is not mandatory. The above may be made a little bit clearer by expanding the type parameter names, at the cost of making the code a bit longer:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
exercise : (Template template, Choice template choice result) =>
ContractId template -> choice -> Update result
```
### Pattern matching in arguments
You met pattern matching earlier, using `case` expressions which is one way of pattern matching. However, it can also be convenient to do the pattern matching at the level of function arguments. Think about implementing the function `uncurry`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
uncurry : (a -> b -> c) -> (a, b) -> c
```
`uncurry` takes a function with two arguments (or more, since `c` could be a function), and turns it into a function from a 2-tuple to `c`. Here are three ways of implementing it, using tuple accessors, `case` pattern matching, and function pattern matching:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
uncurry1 f t = f t._1 t._2
uncurry2 f t = case t of
(x, y) -> f x y
uncurry f (x, y) = f x y
```
Any pattern matching you can do in `case` you can also do at the function level, and the compiler helpfully warns you if you did not cover all cases, which is called "non-exhaustive".
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
fromSome : Optional a -> a
fromSome (Some x) = x
```
The above will give you a warning:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
warning:
Pattern match(es) are non-exhaustive
In an equation for ‘fromSome’: Patterns not matched: None
```
A function that does not cover all its cases, like `fromSome` here, is called a *partial* function. `fromSome None` will cause a runtime error.
We can use function level pattern matching together with a feature called *Record Wildcards* to write the function `issueAsset`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
issueAsset : Asset -> Script (ContractId Asset)
issueAsset asset@(Asset with ..) = do
assetHolders <- queryFilter @AssetHolder issuer
(\ah -> (ah.issuer == issuer) && (ah.owner == owner))
case assetHolders of
(ahCid, _)::_ -> submit asset.issuer do
exerciseCmd ahCid Issue_Asset with ..
[] -> abort ("No AssetHolder found for " <> show asset)
```
The `..` in the pattern match here means bind all fields from the given record to local variables, so we have local variables `issuer`, `owner`, etc.
The `..` in the second to last line means fill all fields of the new record using local variables of the matching names, in this case (per the definition of `Issue_Asset`), `symbol` and `quantity`, taken from the `asset` argument to the function. In other words, this is equivalent to:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
exerciseCmd ahCid Issue_Asset with symbol = asset.symbol, quantity = asset.quantity
```
because the notation `asset@(Asset with ..)` binds `asset` to the entire record, while also binding all of the fields of `asset` to local variables.
### Functions everywhere
You have probably already guessed it: Anywhere you can put a value in Daml you can also put a function. Even inside data types:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Predicate a = Predicate with
test : a -> Bool
```
More often it makes sense to define functions locally, inside a `let` clause or similar. Good examples of this are the `validate` and `transfer` functions defined locally in the `Trade_Settle` choice of the model from `dependencies`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
let
validate (asset, assetCid) = do
fetchedAsset <- fetch assetCid
assertMsg
"Asset mismatch"
(asset == fetchedAsset with
observers = asset.observers)
mapA_ validate (zip baseAssets baseAssetCids)
mapA_ validate (zip quoteAssets quoteAssetCids)
let
transfer (assetCid, approvalCid) = do
exercise approvalCid TransferApproval_Transfer with assetCid
transferredBaseCids <- mapA transfer (zip baseAssetCids baseApprovalCids)
transferredQuoteCids <- mapA transfer (zip quoteAssetCids quoteApprovalCids)
```
You can see that the function signature is inferred from the context here. If you look closely (or hover over the function in the IDE), you'll see that it has signature
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
validate : (HasFetch r, Eq r, HasField "observers" r a) => (r, ContractId r) -> Update ()
```
Bear in mind that functions are not serializable, so you can't use them inside template arguments, as choice inputs, or as choice outputs. They also don't have instances of the `Eq` or `Show` typeclasses which one would commonly want on data types.
The `mapA` and `mapA_` functions loop through the lists of assets and approvals, and apply the functions `validate` and `transfer` to each element of those lists, performing the resulting `Update` action in the process. We'll look at that more closely under `loops` below.
### Lambdas
Daml supports inline functions, called "lambda"s. They are defined using the `(\x y z -> ...)` syntax. For example, a lambda version of `increment` would be `(\n -> n + 1)`.
## Control flow
In this section, we will cover branches and loops, and look at a few common patterns of how to translate procedural code into functional code.
### Branches
Until `compose` the only real kind of control flow introduced has been `case`, which is a powerful tool for branching.
#### If-else expression
`constraints` also showed a seemingly self-explanatory `if ... else` expression, but didn't explain it further. Let's implement the function `boolToInt : Bool -> Int` which in typical fashion maps `True` to `1` and `False` to `0`. Here is an implementation using `case`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
boolToInt b = case b of
True -> 1
False -> 0
```
If you write this function in the IDE, you'll get a warning from the linter:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
Suggestion: Use if
Found:
case b of
True -> 1
False -> 0
Perhaps:
if b then 1 else 0
```
The linter knows the equivalence and suggests a better implementation:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
boolToInt2 b = if b
then 1
else 0
```
In short: `if ... else` expressions are equivalent to `case` expressions, but can be easier to read.
#### Control flow as expressions
`case` and `if ... else` expressions really are control flow in the sense that they short-circuit:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
doError t = case t of
"True" -> True
"False" -> False
_ -> error ("Not a Bool: " <> t)
```
This function behaves as you would expect: the error only gets evaluated if an invalid text is passed in.
This is different from functions, where all arguments are evaluated immediately:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
ifelse b t e = if b then t else e
boom = ifelse True 1 (error "Boom")
```
In the above, `boom` is an error.
While providing proper control flow, `case` and `if ... else` expressions do result in a value when evaluated. You can actually see that in the function definitions above. Since each of the functions is defined just as a `case` or `if ... else` expression, the value of the evaluated function is just the value of the `case` or `if ... else` expression. Values have a type: the `if ... else` expression in `boolToInt2` has type `Int` as that is what the function returns; similarly, the `case` expression in `doError` has type `Bool`. To be able to give such expressions an unambiguous type, each branch needs to have the same type. The below function does not compile as one branch tries to return an `Int` and the other a `Text`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
typeError b = if b
then 1
else "a"
```
If we need functions that can return two (or more) types of things we need to encode that in the return type. For two possibilities, it's common to use the `Either` type:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
intOrText : Bool -> Either Int Text
intOrText b = if b
then Left 1
else Right "a"
```
When you have more than two possible types (and sometimes even just for two types), it can be clearer to define your own variant type to wrap all possibilities.
#### Branches in actions
The most common case where this becomes important is inside `do` blocks. Say we want to create a contract of one type in one case, and of another type in another case. Let's say we have two template types and want to write a function that creates an `S` if a condition is met, and a `T` otherwise.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template T
with
p : Party
where
signatory p
template S
with
p : Party
where
signatory p
```
It would be tempting to write a simple `if ... else`, but it won't typecheck if each branch returns a different type:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
typeError b p = if b
then create T with p
else create S with p
```
We have two options:
1. Use the `Either` trick from above.
2. Get rid of the return types.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
ifThenSElseT1 b p = if b
then do
cid <- create S with p
return (Left cid)
else do
cid <- create T with p
return (Right cid)
ifThenSElseT2 b p = if b
then do
create S with p
return ()
else do
create T with p
return ()
```
The latter is so common that there is a utility function in `DA.Action` to get rid of the return type: `void : Functor f => f a -> f ()`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
ifThenSElseT3 b p = if b
then void (create S with p)
else void (create T with p)
```
`void` also helps express control flow of the type "Create a `T` only if a condition is met.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
conditionalS b p = if b
then void (create S with p)
else return ()
```
Note that we still need the `else` clause of the same type `()`. This pattern is so common, it's encapsulated in the standard library function `DA.Action.when : (Applicative f) => Bool -> f () -> f ()`.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
conditionalS2 b p = when b (void (create S with p))
```
Despite `when` looking like a simple function, the compiler does some magic so that it short-circuits evaluation just like `if ... else` and `case`. The following `noop` function is a no-op (i.e. "does nothing"), not an error as one might otherwise expect:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
noop : Update () = when False (error "Foo")
```
With `case`, `if ... else`, `void` and `when`, you can express all branching. However, one additional feature you may want to learn is guards. They are not covered here, but can help avoid deeply nested `if ... else` blocks. Here's just one example. The Haskell sources at the beginning of the chapter cover this topic in more depth.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
tellSize : Int -> Text
tellSize d
| d < 0 = "Negative"
| d == 0 = "Zero"
| d == 1 = "Non-Zero"
| d < 10 = "Small"
| d < 100 = "Big"
| d < 1000 = "Huge"
| otherwise = "Enormous"
```
### Loops
Other than branching, the most common form of control flow is looping. Looping is usually used to iteratively modify some state. We'll use JavaScript in this section to illustrate the procedural way of doing things.
```JavaScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
function sum(intArr) {
var result = 0;
intArr.forEach (i => {
result += i;
});
return result;
}
```
A more general loop looks like this:
```JavaScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
function whileF(init, cont, step, finalize) {
var state = init();
while (cont(state)) {
state = step(state);
}
return finalize(state);
}
```
In both cases, state is being mutated: `result` in the former, `state` in the latter. Values in Daml are immutable, so it needs to work differently. In Daml we will do this with folds and recursion.
#### Folds
Folds correspond to looping with an explicit iterator: `for` and `forEach` loops in procedural languages. The most common iterator is a list, as is the case in the `sum` function above. For such cases, Daml has the `foldl` function. The `l` stands for "left" and means the list is processed from the left. There is also a corresponding `foldr` which processes from the right.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
foldl : (b -> a -> b) -> b -> [a] -> b
```
Let's give the type parameters semantic names. `b` is the state, `a` is an item. `foldl`s first argument is a function which takes a state and an item, and returns a new state. That's the equivalent of the inner block of the `forEach`. It then takes a state, which is the initial state, and a list of items, which is the iterator. The result is again a state. The `sum` function above can be translated to Daml almost instantly with those correspondences in mind:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
sum ints = foldl (+) 0 ints
```
If we wanted to be more verbose, we could replace `(+)` with a lambda `(\result i -> result + i)` which makes the correspondence to `result += i` from the JavaScript clearer.
Almost all loops with explicit iterators can be translated to folds, though we have to take a bit of care with performance when it comes to translating `for` loops:
```JavaScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
function sumArrs(arr1, arr2) {
var l = min (arr1.length, arr2.length);
var result = new int[l];
for(var i = 0; i < l; i++) {
result[i] = arr1[i] + arr2[i];
}
return result;
}
```
Translating the `for` into a `forEach` is easy if you can get your hands on an array containing values `[0..(l-1)]`. And that's how you do it in Daml, using *ranges*. `[0..(l-1)]` is shorthand for `enumFromTo 0 (l-1)`, which returns the list you'd expect.
Daml also has an operator `(!!) : [a] -> Int -> a` which returns an element in a list. You may now be tempted to write `sumArrs` like this:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
sumArrs : [Int] -> [Int] -> [Int]
sumArrs arr1 arr2 =
let l = min (length arr1) (length arr2)
sumAtI i = (arr1 !! i) + (arr2 !! i)
in foldl (\state i -> (sumAtI i) :: state) [] [1..(l-1)]
```
Unfortunately, that's not a very good approach. Lists in Daml are linked lists, which makes access using `(!!)` too slow for this kind of iteration. A better approach in Daml is to get rid of the `i` altogether and instead merge the lists first using the `zip` function, and then iterate over the "zipped" up lists:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
sumArrs2 arr1 arr2 = foldl (\state (x, y) -> (x + y) :: state) [] (zip arr1 arr2)
```
`zip : [a] -> [b] -> [(a, b)]` takes two lists, and merges them into a single list where the first element is the 2-tuple containing the first element of the two input lists, and so on. It drops any left-over elements of the longer list, thus making the `min` logic unnecessary.
#### Maps
In effect, the lambda passed to `foldl` only "wants" to act on a single element of the (zipped-up) input list, but still has to manage the concatenation of the whole state. Acting on each element separately is a common-enough pattern that there is a specialized function for it: `map : (a -> b) -> [a] -> [b]`. Using it, we can rewrite `sumArr` to:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
sumArrs3 arr1 arr2 = map (\(x, y) -> (x + y)) (zip arr1 arr2)
```
As a rule, use `map` if the result has the same shape as the input and you don't need to carry state from one iteration to the next. Use folds if you need to accumulate state in any way.
#### Recursion
If there is no explicit iterator, you can use recursion. Let's try to write a function that reverses a list, for example. We want to avoid `(!!)` so there is no sensible iterator here. Instead, we use recursion:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
reverseWorker rev rem = case rem of
[] -> rev
x::xs -> reverseWorker (x::rev) xs
reverse xs = reverseWorker [] xs
```
You may be tempted to make `reverseWorker` a local definition inside `reverse`, but Daml only supports recursion for top-level functions so the recursive part `recurseWorker` has to be its own top-level function.
#### Folds and maps in action contexts
The folds and `map` function above are pure in the sense introduced earlier: The functions used to map or process items have no side effects. If you have looked at the multi-package models, you'll have noticed `mapA`, `mapA_`, and `forA`, which seem to serve a similar role but within `Action`s . A good example is the `mapA` call in the `testMultiTrade` script:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
let rels =
[ Relationship chfbank alice
, Relationship chfbank bob
, Relationship gbpbank alice
, Relationship gbpbank bob
]
[chfha, chfhb, gbpha, gbphb] <- mapA setupRelationship rels
```
Here we have a list of relationships (type `[Relationship]`) and a function `setupRelationship : Relationship -> Script (ContractId AssetHolder)`. We want the `AssetHolder` contracts for those relationships, i.e. something of type `[ContractId AssetHolder]`. Using the map function almost gets us there, but `map setupRelationship rels` would have type `[Update (ContractId AssetHolder)]`. This is a list of `Update` actions, each resulting in a `ContractId AssetHolder`. What we need is an `Update` action resulting in a `[ContractId AssetHolder]`. The list and `Update` are nested the wrong way around for our purposes.
Intuitively, it's clear how to fix this: we want the compound action consisting of performing each of the actions in the list in turn. There's a function for that: `sequence : : Applicative m => [m a] -> m [a]`. It implements that intuition and allows us to "take the `Update` out of the list", so to speak. So we could write `sequence (map setupRelationship rels)`. This is so common that it's encapsulated in the `mapA` function, a possible implementation of which is
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
mapA f xs = sequence (map f xs)
```
The `A` in `mapA` stands for "Action", and you'll find that many functions that have something to do with "looping" have an `A` equivalent. The most fundamental of all of these is `foldlA : Action m => (b -> a -> m b) -> b -> [a] -> m b`, a left fold with side effects. Here the inner function has a side-effect indicated by the `m` so the end result `m b` also has a side effect: the sum of all the side effects of the inner function.
To improve your familiarity with these concepts, try implementing `foldlA` in terms of `foldl`, as well as `sequence` and `mapA` in terms of `foldlA`. Here is one set of possible implementations:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
foldlA2 fn init xs =
let
work accA x = do
acc <- accA
fn acc x
in foldl work (pure init) xs
mapA2 fn [] = pure []
mapA2 fn (x :: xs) = do
y <- fn x
ys <- mapA2 fn xs
return (y :: ys)
sequence2 [] = pure []
sequence2 (x :: xs) = do
y <- x
ys <- sequence2 xs
return (y :: ys)
```
`forA` is just `mapA` with its arguments reversed. This is useful for readability if the list of items is already in a variable, but the function is a lengthy lambda.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
[usdCid, chfCid] <- forA [usdCid, chfCid] (\cid -> submit alice do
exerciseCmd cid SetObservers with
newObservers = [bob]
)
```
Lastly, you'll have noticed that in some cases we used `mapA_`, not `mapA`. The underscore indicates that the result is not used, so `mapA_ fn xs fn == void (mapA fn xs)`. The Daml Linter will alert you if you could use `mapA_` instead of `mapA`, and similarly for `forA_`.
## Next up
You now know the basics of functions and control flow, both in pure and Action contexts. The multi-package example shows just how much can be done with just the tools you have encountered here, but there are many more tools at your disposal in the Daml standard library, which provides functions and typeclasses for many common circumstances.
# Data types
In `contracts`, you learnt about contract templates, which specify the types of contracts that can be created on the ledger, and what data those contracts hold in their arguments.
In `daml-scripts`, you learnt about the script view in Daml Studio, which displays the current ledger state. It shows one table per template, with one row per contract of that type and one column per field in the arguments.
This actually provides a useful way of thinking about templates: like tables in databases. Templates specify a data schema for the ledger:
* each template corresponds to a table
* each field in the `with` block of a template corresponds to a column in that table
* each contract of that type corresponds to a table row
In this section, you'll learn how to create rich data schemas for your ledger. Specifically you'll learn about:
* Daml's built-in and native data types
* Record types
* Derivation of standard properties
* Variants
* Data manipulation
* Contract manipulation
After this section, you should be able to use a Daml ledger as a simple database where individual parties can write, read, and delete complex data.
Remember that you can load all the code for this section into a folder called `intro-data` by running `dpm new intro-data --template daml-intro-data`
## Native types
You have already encountered a few native Daml types: `Party` in `contracts`, and `Text` and `ContractId` in `daml-scripts`. Here are those native types and more:
* `Party` Stores the identity of an entity that is able to act on the ledger, in the sense that they can sign contracts and submit transactions. In general, `Party` is opaque.
* `Text` Stores a unicode character string like `"Alice"`.
* `ContractId a` Stores a reference to a contract of type `a`.
* `Int` Stores signed 64-bit integers. For example, `-123`.
* `Decimal` Stores fixed-point number with 28 digits before and 10 digits after the decimal point. For example, `0.0000000001` or `-9999999999999999999999999999.9999999999`.
* `Bool` Stores `True` or `False`.
* `Date` Stores a date.
* `Time` Stores absolute UTC time.
* `RelTime` Stores a difference in time.
The below script instantiates each one of these types, manipulates it where appropriate, and tests the result:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
import Daml.Script
import DA.Time
import DA.Date
native_test = script do
alice <- allocateParty "Alice"
bob <- allocateParty "Bob"
let
my_int = -123
my_dec = 0.001 : Decimal
my_text = "Alice"
my_bool = False
my_date = date 2020 Jan 01
my_time = time my_date 00 00 00
my_rel_time = hours 24
assert (alice /= bob)
assert (-my_int == 123)
assert (1000.0 * my_dec == 1.0)
assert (my_text == "Alice")
assert (not my_bool)
assert (addDays my_date 1 == date 2020 Jan 02)
assert (addRelTime my_time my_rel_time == time (addDays my_date 1) 00 00 00)
```
Despite its simplicity, there are quite a few things to note in this script:
* The `import` statements at the top import two packages from the Daml standard library, which contain all the date and time related functions we use here as well as the functions used in Daml Scripts. More on packages, imports and the standard library later.
* Most of the variables are declared inside a `let` block.
That's because the `script do` block expects script actions like `submit` or `allocateParty`. An integer like `123` is not an action, it's a pure expression, something we can evaluate without any ledger. You can think of the `let` as turning variable declaration into an action.
* Most variables do not have annotations to say what type they are.
That's because Daml is very good at *inferring* types. The compiler knows that `123` is an `Int`, so if you declare `my_int = 123`, it can infer that `my_int` is also an `Int`. This means you don't have to write the type annotation `my_int : Int = 123`.
However, if the type is ambiguous so that the compiler can't infer it, you do have to add a type annotation. This is the case for `0.001` which could be any `Numeric n`. Here we specify `0.001 : Decimal` which is a synonym for `Numeric 10`. You can always choose to add type annotations to aid readability.
* The `assert` function is an action that takes a boolean value and succeeds with `True` and fails with `False`.
Try putting `assert False` somewhere in a script and see what happens to the script result.
With templates and these native types, it's already possible to write a schema akin to a table in a relational database. Below, `Token` is extended into a simple `CashBalance`, administered by a party in the role of an accountant:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
template CashBalance
with
accountant : Party
currency : Text
amount : Decimal
owner : Party
account_number : Text
bank : Party
bank_address : Text
bank_telephone : Text
where
signatory accountant
cash_balance_test = script do
accountant <- allocateParty "Bob"
alice <- allocateParty "Alice"
bob <- allocateParty "Bank of Bob"
submit accountant do
createCmd CashBalance with
accountant
currency = "USD"
amount = 100.0
owner = alice
account_number = "ABC123"
bank = bob
bank_address = "High Street"
bank_telephone = "012 3456 789"
```
## Assemble types
There's quite a lot of information on the `CashBalance` above and it would be nice to be able to give that data more structure. Fortunately, Daml's type system has a number of ways to assemble these native types into much more expressive structures.
### Tuples
A common task is to group values in a generic way. Take, for example, a key-value pair with a `Text` key and an `Int` value. In Daml, you could use a two-tuple of type `(Text, Int)` to do so. If you wanted to express a coordinate in three dimensions, you could group three `Decimal` values using a three-tuple `(Decimal, Decimal, Decimal)`:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
import DA.Tuple
import Daml.Script
tuple_test = script do
let
my_key_value = ("Key", 1)
my_coordinate = (1.0 : Decimal, 2.0 : Decimal, 3.0 : Decimal)
assert (fst my_key_value == "Key")
assert (snd my_key_value == 1)
assert (my_key_value._1 == "Key")
assert (my_key_value._2 == 1)
assert (my_coordinate == (fst3 my_coordinate, snd3 my_coordinate, thd3 my_coordinate))
assert (my_coordinate == (my_coordinate._1, my_coordinate._2, my_coordinate._3))
```
You can access the data in the tuples using:
* functions `fst`, `snd`, `fst3`, `snd3`, `thd3`
* a dot-syntax with field names `_1`, `_2`, `_3`, etc.
Daml supports tuples with up to 20 elements, but accessor functions like `fst` are only included for 2- and 3-tuples.
### Lists
Lists in Daml take a single type parameter defining the type of thing in the list. So you can have a list of integers `[Int]` or a list of strings `[Text]`, but not a list mixing integers and strings.
That's because Daml is statically and strongly typed. When you get an element out of a list, the compiler needs to know what type that element has.
The below script instantiates a few lists of integers and demonstrates the most important list functions.
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
import DA.List
import Daml.Script
list_test = script do
let
empty : [Int] = []
one = [1]
two = [2]
many = [3, 4, 5]
-- `head` gets the first element of a list
assert (head one == 1)
assert (head many == 3)
-- `tail` gets the remainder after head
assert (tail one == empty)
assert (tail many == [4, 5])
-- `++` concatenates lists
assert (one ++ two ++ many == [1, 2, 3, 4, 5])
assert (empty ++ many ++ empty == many)
-- `::` adds an element to the beginning of a list.
assert (1 :: 2 :: 3 :: 4 :: 5 :: empty == 1 :: 2 :: many)
```
Note the type annotation on `empty : [Int] = []`. It's necessary because `[]` is ambiguous. It could be a list of integers or of strings, but the compiler needs to know which it is.
### Records
You can think of records as named tuples with named fields. Declare them using the `data` keyword: `data T = C with`, where `T` is the type name and `C` is the data constructor. In practice, it's a good idea to always use the same name for type and data constructor:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
data MyRecord = MyRecord with
my_txt : Text
my_int : Int
my_dec : Decimal
my_list : [Text]
-- Fields of same type can be declared in one line
data Coordinate = Coordinate with
x, y, z : Decimal
-- Custom data types can also have variables
data KeyValue k v = KeyValue with
my_key : k
my_val : v
data Nested = Nested with
my_coord : Coordinate
my_record : MyRecord
my_kv : KeyValue Text Int
record_test = script do
let
my_record = MyRecord with
my_txt = "Text"
my_int = 2
my_dec = 2.5
my_list = ["One", "Two", "Three"]
my_coord = Coordinate with
x = 1.0
y = 2.0
z = 3.0
-- `my_text_int` has type `KeyValue Text Int`
my_text_int = KeyValue with
my_key = "Key"
my_val = 1
-- `my_int_decimal` has type `KeyValue Int Decimal`
my_int_decimal = KeyValue with
my_key = 2
my_val = 2.0 : Decimal
-- If variables are in scope that match field names, we can pick them up
-- implicitly, writing just `my_coord` instead of `my_coord = my_coord`.
my_nested = Nested with
my_coord
my_record
my_kv = my_text_int
-- Fields can be accessed with dot syntax
assert (my_coord.x == 1.0)
assert (my_text_int.my_key == "Key")
assert (my_nested.my_record.my_dec == 2.5)
```
You'll notice that the syntax to declare records is very similar to the syntax used to declare templates. That's no accident because a template is really just a special record. When you write `template Token with`, one of the things that happens in the background is that this becomes a `data Token = Token with`.
In the `assert` statements above, we always compared values of in-built types. If you wrote `assert (my_record == my_record)` in the script, you may be surprised to get an error message `No instance for (Eq MyRecord) arising from a use of ‘==’`. Equality in Daml is always value equality and we haven't written a function to check value equality for `MyRecord` values. But don't worry, you don't have to implement this rather obvious function yourself. The compiler is smart enough to do it for you, if you use `deriving (Eq)`:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
data EqRecord = EqRecord with
my_txt : Text
my_int : Int
my_dec : Decimal
my_list : [Text]
deriving (Eq)
data MyContainer a = MyContainer with
contents : a
deriving (Eq)
eq_test = script do
let
eq_record = EqRecord with
my_txt = "Text"
my_int = 2
my_dec = 2.5
my_list = ["One", "Two", "Three"]
my_container = MyContainer with
contents = eq_record
other_container = MyContainer with
contents = eq_record
assert(my_container.contents == eq_record)
assert(my_container == other_container)
```
`Eq` is what is called a *typeclass*. You can think of a typeclass as being like an interface in other languages: it is the mechanism by which you can define a set of functions (for example, `==` and `/=` in the case of `Eq`) to work on multiple types, with a specific implementation for each type they can apply to.
There are some other typeclasses that the compiler can derive automatically. Most prominently, `Show` to get access to the function `show` (equivalent to `toString` in many languages) and `Ord`, which gives access to comparison operators `<`, `>`, `<=`, `>=`.
It's a good idea to always derive `Eq` and `Show` using `deriving (Eq, Show)`. The record types created using `template T with` do this automatically, and the native types have appropriate typeclass instances. For example, `Int` derives `Eq`, `Show`, and `Ord`. As another example, `ContractId a` derives `Eq` and `Show`.
Records can give the data on `CashBalance` a bit more structure:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Bank = Bank with
party : Party
address: Text
telephone : Text
deriving (Eq, Show)
data Account = Account with
owner : Party
number : Text
bank : Bank
deriving (Eq, Show)
data Cash = Cash with
currency : Text
amount : Decimal
deriving (Eq, Show)
template CashBalance
with
accountant : Party
cash : Cash
account : Account
where
signatory accountant
cash_balance_test = script do
accountant <- allocateParty "Bob"
owner <- allocateParty "Alice"
bank_party <- allocateParty "Bank"
let
bank = Bank with
party = bank_party
address = "High Street"
telephone = "012 3456 789"
account = Account with
owner
bank
number = "ABC123"
cash = Cash with
currency = "USD"
amount = 100.0
submit accountant do
createCmd CashBalance with
accountant
cash
account
pure ()
```
If you look at the resulting script view, you'll see that this still gives rise to one table. The records are expanded out into columns using dot notation.
### Variants and pattern matching
Suppose now that you also wanted to keep track of cash in hand. Cash in hand doesn't have a bank, but you can't just leave `bank` empty. Daml doesn't have an equivalent to `null`. Variants can express that cash can either be in hand or at a bank:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Bank = Bank with
party : Party
address: Text
telephone : Text
deriving (Eq, Show)
data Account = Account with
number : Text
bank : Bank
deriving (Eq, Show)
data Cash = Cash with
currency : Text
amount : Decimal
deriving (Eq, Show)
data Location
= InHand
| InAccount Account
deriving (Eq, Show)
template CashBalance
with
accountant : Party
owner : Party
cash : Cash
location : Location
where
signatory accountant
cash_balance_test = do
accountant <- allocateParty "Bob"
owner <- allocateParty "Alice"
bank_party <- allocateParty "Bank"
let
bank = Bank with
party = bank_party
address = "High Street"
telephone = "012 3456 789"
account = Account with
bank
number = "ABC123"
cash = Cash with
currency = "USD"
amount = 100.0
submit accountant do
createCmd CashBalance with
accountant
owner
cash
location = InHand
submit accountant do
createCmd CashBalance with
accountant
owner
cash
location = InAccount account
```
The way to read the declaration of `Location` is "*A Location either has value* `InHand` *OR has a value* `InAccount a` *where* `a` *is of type Account*". This is quite an explicit way to say that there may or may not be an `Account` associated with a `CashBalance` and gives both cases suggestive names.
Another option is to use the built-in `Optional` type. The `None` value of type `Optional a` is the closest Daml has to a `null` value:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Optional a
= None
| Some a
deriving (Eq, Show)
```
Variant types where none of the data constructors take a parameter are called enums:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
data DayOfWeek
= Monday
| Tuesday
| Wednesday
| Thursday
| Friday
| Saturday
| Sunday
deriving (Eq, Show)
```
To access the data in variants, you need to distinguish the different possible cases. For example, you can no longer access the account number of a `Location` directly, because if it is `InHand`, there may be no account number.
To do this, you can use *pattern matching* and either throw errors or return compatible types for all cases:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
{-
-- Commented out as `Either` is defined in the standard library.
data Either a b
= Left a
| Right b
-}
variant_access_test = script do
let
l : Either Int Text = Left 1
r : Either Int Text = Right "r"
-- If we know that `l` is a `Left`, we can error on the `Right` case.
l_value = case l of
Left i -> i
Right i -> error "Expecting Left"
-- Comment out at your own peril
{-
r_value = case r of
Left i -> i
Right i -> error "Expecting Left"
-}
-- If we are unsure, we can return an `Optional` in both cases
ol_value = case l of
Left i -> Some i
Right i -> None
or_value = case r of
Left i -> Some i
Right i -> None
-- If we don't care about values or even constructors, we can use wildcards
l_value2 = case l of
Left i -> i
Right _ -> error "Expecting Left"
l_value3 = case l of
Left i -> i
_ -> error "Expecting Left"
day = Sunday
weekend = case day of
Saturday -> True
Sunday -> True
_ -> False
assert (l_value == 1)
assert (l_value2 == 1)
assert (l_value3 == 1)
assert (ol_value == Some 1)
assert (or_value == None)
assert weekend
```
## Manipulate data
You've got all the ingredients to build rich types expressing the data you want to be able to write to the ledger, and you have seen how to create new values and read fields from values. But how do you manipulate values once created?
All data in Daml is immutable, meaning once a value is created, it will never change. Rather than changing values, you create new values based on old ones with some changes applied:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
manipulation_demo = script do
let
eq_record = EqRecord with
my_txt = "Text"
my_int = 2
my_dec = 2.5
my_list = ["One", "Two", "Three"]
-- A verbose way to change `eq_record`
changed_record = EqRecord with
my_txt = eq_record.my_txt
my_int = 3
my_dec = eq_record.my_dec
my_list = eq_record.my_list
-- A better way
better_changed_record = eq_record with
my_int = 3
record_with_changed_list = eq_record with
my_list = "Zero" :: eq_record.my_list
assert (eq_record.my_int == 2)
assert (changed_record == better_changed_record)
-- The list on `eq_record` can't be changed.
assert (eq_record.my_list == ["One", "Two", "Three"])
-- The list on `record_with_changed_list` is a new one.
assert (record_with_changed_list.my_list == ["Zero", "One", "Two", "Three"])
```
`changed_record` and `better_changed_record` are each a copy of `eq_record` with the field `my_int` changed. `better_changed_record` shows the recommended way to change fields on a record. The syntax is almost the same as for a new record, but the record name is replaced with the old value: `eq_record with` instead of `EqRecord with`. The `with` block no longer needs to give values to all fields of `EqRecord`. Any missing fields are taken from `eq_record`.
Throughout the script, `eq_record` never changes. The expression `"Zero" :: eq_record.my_list` doesn't change the list in-place, but creates a new list, which is `eq_record.my_list` with an extra element in the beginning.
## Manipulate contracts
Daml's type system allows you to store structured data within Daml contracts. Like records and other data structures, contracts are immutable. They can only be created and archived. If you need to change a contract, you must archive the original contract and create a new one with the updated data.
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Bank = Bank with
party : Party
address: Text
telephone : Text
deriving (Eq, Show)
template Account
with
accountant : Party
owner : Party
number : Text
bank : Bank
where
signatory accountant
contract_manipulation_test = do
accountant <- allocateParty "Bob"
owner <- allocateParty "Alice"
bank_party <- allocateParty "Bank"
let
bank = Bank with
party = bank_party
address = "High Street"
telephone = "012 3456 789"
accountCid <- submit accountant do
createCmd Account with
accountant
owner
bank
number = "ABC123"
-- Now the accountant updates the telephone number for the bank on the account
Some account <- queryContractId accountant accountCid
newAccountCid <- submit accountant do
archiveCmd accountCid
cid <- createCmd account with
bank = account.bank with
telephone = "098 7654 321"
pure cid
newAccountCid =/= accountCid
```
The script above uses `archiveCmd` and `createCmd` to modify a contract of type `Account` in the same transaction. This process creates a new contract, and a new, distinct contract ID, as shown by `newAccountCid =/= accountCid`
When you modify a contract, by archiving it and creating a new one, you generate a new contract ID. That makes contract IDs unstable, and it can cause stale references.
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Cash = Cash with
currency : Text
amount : Decimal
deriving (Eq, Show)
template CashBalance
with
accountant : Party
cash : Cash
account : ContractId Account
where
signatory accountant
stale_contract_id_test = do
accountant <- allocateParty "Bob"
owner <- allocateParty "Alice"
bank_party <- allocateParty "Bank"
let
bank = Bank with
party = bank_party
address = "High Street"
telephone = "012 3456 789"
cash = Cash with
currency = "USD"
amount = 100.0
accountCid <- submit accountant do
createCmd Account with
accountant
owner
bank
number = "ABC123"
balanceCid <- submit accountant do
createCmd CashBalance with
accountant
cash
account = accountCid
-- Now the accountant updates the telephone number for the bank on the account
Some account <- queryContractId accountant accountCid
_ <- submit accountant do
archiveCmd accountCid
cid <- createCmd account with
bank = account.bank with
telephone = "098 7654 321"
pure cid
-- The `account` field on the balance now refers to the archived
-- contract, so this returns None.
Some balance <- queryContractId accountant balanceCid
optAccount <- queryContractId accountant balance.account
optAccount === None
```
In the script above, the `ContractId` in `balance.account` still refers to the archived contract. Consequently, querying the `balance.account` fails and returns `None`.
## Next up
You can now define data schemas for the ledger, read, write, and delete data from the ledger.
In `choices` you'll learn how to define data transformations and give other parties the right to manipulate data in restricted ways.
## Standard library
For an overview of the Prelude, important types from the standard library, and how to search the library, see [The Daml Standard Library](/appdev/modules/m3-standard-library).
# The Daml Standard Library
Source: https://docs.canton.network/appdev/modules/m3-standard-library
Tour of the Daml standard library: the Prelude, important types and typeclasses, and how to browse and search the library.
In `data` and `functional-101` you learned how to define your own data types and functions. However, you don't have to implement everything from scratch. Daml comes with the Daml standard library, which contains types, functions, and typeclasses that cover a large range of use cases.
In this chapter, you'll get an overview of the essentials and learn how to browse and search the library to find functions. Being proficient with the standard library will make you considerably more efficient writing Daml code. Specifically, this chapter covers:
* The Prelude
* Important types from the standard library, and associated functions and typeclasses
* Typeclasses
* Important typeclasses like `Functor`, `Foldable`, and `Traversable`
* How to search the standard library
To go in depth on some of these topics, the literature referenced in `haskell-connection` covers them in much greater detail. The standard library typeclasses like `Applicative`, `Foldable`, `Traversable`, `Action` (called `Monad` in Haskell), and many more, are the bread and butter of Haskell programmers.
There is a project template `daml-intro-11` for this chapter, but it only contains a single source file with the code snippets embedded in this section.
## The Prelude
You've already used a lot of functions, types, and typeclasses without importing anything. Functions like `create`, `exercise`, and `(==)`, types like `[]`, `(,)`, `Optional`, and typeclasses like `Eq`, `Show`, and `Ord`. These all come from the Prelude. The Prelude is module that gets implicitly imported into every other Daml module and contains both Daml specific machinery as well as the essentials needed to work with the inbuilt types and typeclasses.
## Important types from the Prelude
In addition to the `native-types`, the Prelude defines a number of common types:
### Lists
You've already met lists. Lists have two constructors `[]` and `x :: xs`, the latter of which is "prepend" in the sense that `1 :: [2] == [1, 2]`. In fact `[1,2]` is just syntactical sugar for `1 :: 2 :: []`.
### Tuples
In addition to the 2-tuple you have already seen, the Prelude contains definitions for tuples of size up to 15. Tuples allow you to store mixed data in an ad-hoc fashion. Common use-cases are return values from functions consisting of several pieces or passing around data in folds, as you saw in `folds`. An example of a relatively wide Tuple can be found in the test modules of the `exceptions` project. `Test.Intro.Asset.TradeSetup.tradeSetup` returns the allocated parties and active contracts in a long tuple. `Test.Intro.Asset.MultiTrade.testMultiTrade` puts them back into scope using pattern matching:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
return (alice, bob, usdbank, eurbank, usdha, usdhb, eurha, eurhb, usdCid, eurCid)
```
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
(alice, bob, usdbank, eurbank, usdha, usdhb, eurha, eurhb, usdCid, eurCid) <- tradeSetup
```
Tuples, like lists have some syntactic magic. Both the types as well as the constructors for tuples are `(,,,)` where the number of commas determines the arity of the tuple. Type and data constructor can be applied with values inside the brackets, or outside, and partial application is possible:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
t1 : (Int, Text) = (1, "a")
t2 : (,) Int Text = (1, "a")
t3 : (Int, Text) = (1,) "a"
t4 : a -> (a, Text) = (,"a")
```
While tuples of great lengths are available, it is often advisable to define custom records with named fields for complex structures or long-lived values. Overuse of tuples can harm code readability.
### Optional
The `Optional` type represents a value that may be missing. It's the closest thing Daml has to a "nullable" value. `Optional` has two constructors: `Some`, which takes a value, and `None`, which doesn't take a value. In many languages one would write code like this:
```JavaScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
lookupResult = lookupByKey(k);
if( lookupResult == null) {
// Do something
} else {
// Do something else
}
```
In Daml the same thing would be expressed as:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
lookupResult <- lookupByKey @T k
case lookupResult of
None -> do -- Do Something
return ()
Some cid -> do -- Do Something
return ()
```
### Either
`Either` is used in cases where a value should store one of two types. It has two constructors, `Left` and `Right`, each of which take a value of one or the other of the two types. One typical use-case of `Either` is as an extended `Optional` where `Right` takes the role of `Some` and `Left` the role of `None`, but with the ability to store an error value. `Either Text`, for example behaves just like `Optional`, except that values with constructor `Left` have a text associated to them.
As with tuples, it's easy to overuse `Either` and harm readability. Consider writing your own more explicit type instead. For example if you were returning `South a` vs `North b` using your own type over `Either` would make your code clearer.
## Typeclasses
You've seen typeclasses in use all the way from `data`. It's now time to look under the hood.
Typeclasses are declared using the `class` keyword:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
class HasQuantity a q where
getQuantity : a -> q
setQuantity : q -> a -> a
```
This is akin to an interface declaration of an interface with a getter and setter for a quantity. To *implement* this interface, you need to define instances of this typeclass:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
data Foo = Foo with
amount : Decimal
instance HasQuantity Foo Decimal where
getQuantity foo = foo.amount
setQuantity amount foo = foo with amount
```
Typeclasses can have constraints like functions. For example: `class Eq a => Ord a` means "everything that is orderable can also be compared for equality". And that's almost all there's to it.
## Important typeclasses from the Prelude
### Eq
The `Eq` typeclass allows values of a type to be compared for (in)-equality. It makes available two function: `==` and `/=`. Most data types from the standard library have an instance of `Eq`. As you already learned in `data`, you can let the compiler automatically derive instances of `Eq` for you using the `deriving` keyword.
Templates always have an `Eq` instance, and all types stored on a template need to have one.
### Ord
The `Ord` typeclass allows values of a type to be compared for order. It makes available functions: `<`, `>`, `<=`, and `>=`. Most of the inbuilt data types have an instance of `Ord`. Furthermore, types like `List` and `Optional` get an instance of `Ord` if the type they contain has one. You can let the compiler automatically derive instances of `Ord` for you using the `deriving` keyword.
### Show
`Show` indicates that a type can be serialized to `Text`, ie "shown" in a shell. Its key function is `show`, which takes a value and converts it to `Text`. All inbuilt data types have an instance for `Show` and types like `List` and `Optional` get an instance if the type they contain has one. It also supports the `deriving` keyword.
### Functor
Functors are the closest thing to "containers" that Daml has. Whenever you see a type with a single type parameter, you are probably looking at a `Functor`: `[a]`, `Optional a`, `Either Text a`, `Update a`. Functors are things that can be mapped over and as such, the key function of `Functor` is `fmap`, which does generically what the `map` function does for lists.
Other classic examples of Functors are Sets, Maps, Trees, etc.
### Applicative Functor
Applicative Functors are a bit like Actions, which you met in `constraints`, except that you can't use the result of one action as the input to another action. The only important Applicative Functor that isn't an action in Daml is the `Commands` type submitted in a `submit` block in Daml Script. That's why in order to use `do` notation in Daml Script, you have to enable the `ApplicativeDo` language extension.
### Action
Actions were already covered in `constraints`. One way to think of them is as "recipes" for a value, which need to be "executed to get at that value. Actions are always Functors (and Applicative Functors). The intuition for that is simply that `fmap f x` is the recipe in `x` with the extra instruction to apply the pure function `f` to the result.
The really important Actions in Daml are `Update` and `Script`, but there are many others, like `[]`, `Optional`, and `Either a`.
### Semigroup and Monoid
Semigroups and monoids are about binary operations, but in practice, their important use is for `Text` and `[]`, where they allow concatenation using the `{<>}` operator.
### Additive and Multiplicative
Additive and Multiplicative abstract out arithmetic operations, so that `(+)`, `(-)`, `(*)`, and some other functions can be used uniformly between `Decimal` and `Int`.
## Important modules in the standard library
For almost all the types and typeclasses presented above, the standard library contains a module:
* [DA.List](/appdev/reference/daml-standard-library/da-list) for Lists
* [DA.Optional](/appdev/reference/daml-standard-library/da-optional) for `Optional`
* [DA.Tuple](/appdev/reference/daml-standard-library/da-tuple) for Tuples
* [DA.Either](/appdev/reference/daml-standard-library/da-either) for `Either`
* [DA.Functor](/appdev/reference/daml-standard-library/da-functor) for Functors
* [DA.Action](/appdev/reference/daml-standard-library/da-action) for Actions
* [DA.Functor](/appdev/reference/daml-standard-library/da-functor) and [DA.Semigroup](/appdev/reference/daml-standard-library/da-semigroup) for Monoids and Semigroups
* [DA.Text](/appdev/reference/daml-standard-library/da-text) for working with `Text`
* [DA.Time](/appdev/reference/daml-standard-library/da-time) for working with `Time`
* [DA.Date](/appdev/reference/daml-standard-library/da-date) for working with `Date`
You get the idea, the names are fairly descriptive.
Other than the typeclasses defined in Prelude, there are two modules generalizing concepts you've already learned, which are worth knowing about: `Foldable` and `Traversable`. The examples earlier in this module were based on lists, but there are many other possible iterators. This is expressed in two additional typeclasses: [DA.Traversable](/appdev/reference/daml-standard-library/da-traversable) and [DA.Foldable](/appdev/reference/daml-standard-library/da-foldable). For more detail on these concepts, see [Foldable and Traversable on the Haskell wiki](https://wiki.haskell.org/Foldable_and_Traversable).
## Search the standard library
Being able to browse the standard library starting from the [Daml Standard Library reference](/appdev/reference/daml-standard-library/index) is a start, and the module naming helps, but it's not an efficient process for finding out what a function you've encountered does, even less so for finding a function that does a thing you need to do.
Daml has its own version of the [Hoogle](https://hoogle.haskell.org/) search engine: [Daml Hoogle](https://hoogle.daml.com). It offers standard library search both by name and by signature.
### Search for functions by name
Say you come across some functions you haven't seen before, like the ones in the `ensure` clause of the `MultiTrade`.
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
ensure (length baseAssetCids == length baseAssets) &&
(length quoteApprovalCids == length quoteAssets) &&
not (null baseAssets) &&
not (null quoteAssets)
```
You may be able to guess what `not` and `null` do, but try searching those names in the documentation search. Search results from the standard library will show on top. `not`, for example, gives
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
not
: Bool -> Bool
Boolean “not”
```
Signature (including type constraints) and description usually give a pretty clear picture of what a function does.
### Search for functions by signature
The other very common use case for the search is that you have some values that you want to do something with, but don't know the standard library function you need. On the `MultiTrade` template we have a list `baseAssets`, and thanks to your ensure clause we know it's non-empty. In the original `Trade` we used `baseAsset.owner` as the signatory. How do you get the first element of this list to extract the `owner` without going through the motions of a complete pattern match using `case`?
The trick is to think about the signature of the function that's needed, and then to search for that signature. In this case, we want a single distinguished element from a list so the signature should be `[a] -> a`. If you search for that, you'll get a whole range of results, but again, standard library results are shown at the top.
Scanning the descriptions, `head` is the obvious choice, as used in the `let` of the `MultiTrade` template.
You may notice that in the search results you also get some hits that don't mention `[]` explicitly. For example:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
fold
: (Foldable t, Monoid m) => t m -> m
Combine the elements of a structure using a monoid.
```
The reason is that there is an instance for `Foldable [a]`.
Let's try another search. Suppose you didn't want the first element, but the one at index `n`. Remember that `(!!)` operator from `functional-101`? There are now two possible signatures we could search for: `[a] -> Int -> a` and `Int -> [a] -> a`. Try searching for both. You'll see that the search returns `(!!)` in both cases. You don't have to worry about the order of arguments.
## Next up
In the following section, you'll find options for testing and interacting with Daml code. We also talk about the operational semantics of some keywords and their commonly associated failures, and a little bit about how coverage reports work in Daml testing.
# Testing Daml Contracts
Source: https://docs.canton.network/appdev/modules/m3-testing
Write and run tests for Daml smart contracts using Daml Script
# Test Daml contracts
This section is about testing and debugging the Daml contracts you've built using the tools from earlier chapters. You've already met Daml Script as a way of testing your code inside the IDE. In this chapter you'll learn about more ways to run Daml Scripts, as well as other tools you can use for testing and debugging. Specifically we will cover:
* Running Daml Scripts to test and debug templates
* The `trace` and `debug` functions
* The choice coverage
Note that this section only covers testing your Daml contracts.
Remember that you can load all the code for this section into a folder called `intro-test` by running `dpm new intro-test --template daml-intro-test`
## Multi-package project structure
In the project structures section you learned that it is important to keep the scripts and templates in separate packages. The project for this section is a multi-package build, containing all the code from `compose` and `dependencies`. The folder structure looks like this:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
.
├── asset
├── asset-tests
├── multi-trade
├── multi-trade-tests
└── multi-package.yaml
```
`asset`, `asset-tests`, `multi-trade` and `multi-trade-tests` are packages. They each contain there own `daml.yaml` file. `asset` contains the code for the `Asset` and `Trade` templates. The scripts for testing `Asset` and `Trade` are in the `asset-tests` package, which depends on `asset`. Contrary to `asset-tests`, `asset` is meant to be uploaded to a Canton ledger.
Similarly `multi-trade` contains the `MultiTrade` template, and `multi-trade-tests` contains the corresponding scripts.
`multi-package.yaml` is the multi-package configuration. It allows `dpm` to orchestrate the build of each sub-package. It contains the list of sub-packages:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
packages:
- ./asset
- ./asset-tests
- ./multi-trade
- ./multi-trade-tests
```
Run `dpm build --all` at the root of the multi-package project, to build all sub-packages. Or you can run `dpm build` in any of the sub-package. It detects the dependencies, and build them in the correct order. For instance, running `dpm build` in `multi-trade-tests` triggers the build of `asset`, `multi-trade` and `multi-trade-tests` in that order.
## Test templates with Daml Scripts
Daml Script is the main tool to test Daml contracts. In a script, you can submit commands and queries from multiple parties on a fresh, initially empty, ledger.
There are two main ways to run a Daml Script: - Click the `Script results` lens in VS Code: it provides the table and transaction views, which are useful for debugging. - Run the `dpm test` command, which is useful for regression checks and coverage.
### Run a script in VS Code
In the earlier testing with scripts section, you learned how to write and run a Daml Script in VS Code.
As a refresher, find a script in the `asset-tests` or `multi-trade-tests` and click the `Script results` lens, that appears on top of the script name. VS Code should open the table view in a side pane. The table view describes the final state of the ledger at the end of the script. It shows the list of active contracts, their data, and for each party, if it can see the contract or not. Click the `Show archived` toggle to see the list of archived contracts.
In the side pane, click the `Show transaction view` button to switch to the transaction view. It shows you all the transactions, and sub-transactions, executed by the script. It also contain the failure message whenever a script fail. Try to change the submitting party of an `exerciseCmd` and see if the script is still succeeding.
You can use the table and transaction views in VS Code to better understand the visibility of each contract, and authority of each party.
### Run all scripts with `dpm`
`dpm` can run all the scripts inside a package. This is useful for quick regression check, and their automation in the CI.
Open a terminal in the `multi-trade-tests` folder and run `dpm test`. It should succeed and print the following test summary:
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
Test Summary
daml/Intro/Asset/MultiTradeTests.daml:testMultiTrade: ok, 12 active contracts, 28 transactions.
daml/Intro/Asset/TradeSetup.daml:setupRoles: ok, 2 active contracts, 4 transactions.
daml/Intro/Asset/TradeSetup.daml:test_issuance: ok, 3 active contracts, 5 transactions.
daml/Intro/Asset/TradeSetup.daml:tradeSetup: ok, 6 active contracts, 10 transactions.
Modules internal to this package:
- Internal templates
0 defined
0 (100.0%) created
- Internal template choices
0 defined
0 (100.0%) exercised
- Internal interface implementations
0 defined
0 internal interfaces
0 external interfaces
- Internal interface choices
0 defined
0 (100.0%) exercised
Modules external to this package:
- External templates
7 defined
5 ( 71.4%) created in any tests
5 ( 71.4%) created in internal tests
0 ( 0.0%) created in external tests
- External template choices
27 defined
7 ( 25.9%) exercised in any tests
7 ( 25.9%) exercised in internal tests
0 ( 0.0%) exercised in external tests
- External interface implementations
0 defined
- External interface choices
0 defined
0 (100.0%) exercised in any tests
0 (100.0%) exercised in internal tests
0 (100.0%) exercised in external tests
```
The first part of the summary is a list of each executed script. For each script, you can see the total number of transactions, and active contracts at the end of the script. For instance the `testMultiTrade` executed 28 transactions, to create 12 active contracts.
The second part of the summary is the coverage report. It shows you how many templates and choices are tested by the complete set of scripts in the package, in proportion of the total number of templates and choices.
## Debug, trace, and stacktraces
The above demonstrates nicely how to test the happy path, but what if a function doesn't behave as you expected? Daml has two functions that allow you to do fine-grained printf debugging: `debug` and `trace`. Both allow you to print something to StdOut if the code is reached. The difference between `debug` and `trace` is similar to the relationship between `abort` and `error`:
* `debug : Text -> m ()` maps a text to an Action that has the side-effect of printing to StdOut.
* `trace : Text -> a -> a` prints to StdOut when the expression is evaluated.
```none theme={"theme":{"light":"github-light","dark":"github-dark"}}
daml> let a : Script () = debug "foo"
daml> let b : Script () = trace "bar" (debug "baz")
[Daml.Script:378]: "bar"
daml> a
[DA.Internal.Prelude:532]: "foo"
daml> b
[DA.Internal.Prelude:532]: "baz"
daml>
```
If in doubt, use `debug`. It's the easier of the two to interpret the results of.
The thing in the square brackets is the last location. It'll tell you the Daml file and line number that triggered the printing, but often no more than that because full stacktraces could violate subtransaction privacy quite easily. If you want to enable stacktraces for some purely functional code in your modules, you can use the machinery in the `DA.Stack` module to do so, but we won't cover that any further here.
## Local testing with dpm sandbox
For integration testing beyond Daml Script, you can run a local Canton sandbox using `dpm sandbox`. This starts a local participant node with an in-memory ledger, letting you test your contracts in a more realistic environment before deploying to DevNet or TestNet.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
dpm sandbox
```
Once the sandbox is running, you can interact with it through the Ledger API (gRPC or JSON), submit commands from codegen-based Java or TypeScript clients, and test multi-party workflows with real API authentication.
### When to use sandbox vs Daml Script
**Use Daml Script** for testing contract logic in isolation: authorization rules, choice behavior, multi-party workflows, and edge cases. Scripts run in-process without a ledger, making them fast and deterministic. They are the right tool for unit-testing your Daml model.
**Use the sandbox** when you need to test how your application integrates with the ledger. Situations where the sandbox is the better choice:
* Testing your Java or TypeScript backend against the Ledger API
* Verifying that codegen-generated types serialize and deserialize correctly
* Testing JSON API interactions from a frontend
* Running multi-participant configurations
* Testing DAR upload, party allocation, and user management through the Admin API
* Validating authorization token handling in your application
### When to use a local Canton Network
The [CN Quickstart](/sdks-tools/reference-projects/cn-quickstart) project includes a LocalNet configuration that runs a full Canton Network locally (participant, sequencer, mediator, and Splice applications). Use LocalNet instead of the sandbox when you need to test:
* Canton Coin transfers and traffic purchases
* Wallet SDK integration
* Splice API interactions (Scan, Validator APIs)
* End-to-end application flows that involve the Global Synchronizer
See the [QuickStart walkthrough](/appdev/quickstart/running-the-demo) for instructions on starting LocalNet.
# Working with Time
Source: https://docs.canton.network/appdev/modules/m3-working-with-time
Implement time-based logic in Daml contracts including expiration, deadlines, and time constraints
# How To Implement Time Constraints
Contract time constraints may be implemented using either:
* ledger time primitives (i.e. `isLedgerTimeLT`, `isLedgerTimeLE`, `isLedgerTimeGT` and `isLedgerTimeGE`) or assertions (i.e. `assertWithinDeadline` and `assertDeadlineExceeded`)
> * the use of ledger time primitives and assertions do not constrain the time bound between transaction preparation and submission - e.g. they are suitable for workflows using external parties to sign transactions
* or, by calling `getTime`
> * calls to `getTime` constrain transaction preparation and submission workflows to be (by default) within 1 minute.
> * the 1 minute value is the default value for the ledger time record time tolerance parameter (a dynamic synchronizer parameter).
The next subsections demonstrate how the following Coin and TransferProposal contracts can be modified to use different types of ledger time constraints to control when parties are allowed to perform ledger writes.
Coin contract
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Coin
with
owner: Party
issuer: Party
amount: Decimal
where
signatory issuer
signatory owner
ensure amount > 0.0
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Transfer : ContractId TransferProposal
with
newOwner: Party
controller owner
do create TransferProposal
with coin=this; newOwner
```
TransferProposal contract
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template TransferProposal
with
coin: Coin
newOwner: Party
where
signatory coin.owner
signatory coin.issuer
observer newOwner
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice AcceptTransfer : ContractId Coin
controller newOwner
do
create coin with owner = newOwner
```
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice WithdrawTransfer : ContractId Coin
controller coin.owner
do
create coin
```
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
subgraph Coin["Coin (active)"]
C1["owner: Alice
issuer: Bank
newOwner: Bob"]
end
Coin -->|"Transfer"| TP
subgraph TP["TransferProposal (active)"]
T1["newOwner: Bob
coin: Coin"]
end
TP -->|"AcceptTransfer"| Coin2
subgraph Coin2["Coin (new owner)"]
C2["owner: Bob
issuer: Bank"]
end
```
*Simple coin transfer with consent withdrawal*
## How to check that a deadline is valid
This design pattern demonstrates how to limit choices so that they must occur by a given deadline.
### Motivation
When parties need to perform ledger writes by a given deadline.
### Implementation
Transfer proposals can be accepted at any point in time. To restrict this behaviour so that acceptance must occur by a fixed time, a guard for AcceptTransfer choice execution can be added.
TransferProposal contract
In the TransferProposal contract, the body of the AcceptTransfer choice is modified to assert that the contract deadline is valid.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice AcceptTransfer : ContractId Coin
controller newOwner
do
assertWithinDeadline "time-limited-transfer" timeLimit
create coin with owner = newOwner
```
As transfer proposals are created when a Transfer choice is executed, the time by which an AcceptTransfer can be executed needs to be passed in as a choice parameter.
Coin contract
In the Coin contract, the Transfer choice has an additional deadline argument, so that TransferProposal contracts can be given a fixed lifetime.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Transfer : ContractId TransferProposal
with
newOwner: Party
timeLimit: Time
controller owner
do create TransferProposal
with coin=this; newOwner; timeLimit
```
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
subgraph Coin["Coin (active)"]
C1["owner: Alice
issuer: Bank"]
end
Coin -->|"Transfer(deadline)"| TP
subgraph TP["TransferProposal"]
T1["newOwner: Bob
deadline: timestamp"]
end
TP -->|"AcceptTransfer
assert now ≤ deadline"| Coin2
subgraph Coin2["Coin (new owner)"]
C2["owner: Bob
issuer: Bank"]
end
```
*Time-limited coin ownership transfer*
## How to check that a deadline has passed
This design pattern demonstrates how to ensure choices only occur after a given deadline.
### Motivation
When parties need to perform ledger writes after a fixed time delay.
### Implementation
Transfer proposals can be accepted at any point in time. To restrict this behaviour so that acceptance can only occur after a fixed delay, a guard for AcceptTransfer choice execution can be added.
TransferProposal contract
In the TransferProposal contract, the body of the AcceptTransfer choice is modified to assert that the contract deadline has been exceeded or passed.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice AcceptTransfer : ContractId Coin
controller newOwner
do
assertDeadlineExceeded "delayed-transfer" delay
create coin with owner = newOwner
```
As transfer proposals are created when a Transfer choice is executed, the delay time after which an AcceptTransfer can be executed needs to be passed in as a choice parameter.
Coin contract
In the Coin contract, the Transfer choice has an additional deadline argument, so that TransferProposal contracts can be given a delay.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Transfer : ContractId TransferProposal
with
newOwner: Party
delay: Time
controller owner
do create TransferProposal
with coin=this; newOwner; delay
```
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
subgraph Coin["Coin (active)"]
C1["owner: Alice
issuer: Bank"]
end
Coin -->|"Transfer(delay)"| TP
subgraph TP["TransferProposal"]
T1["newOwner: Bob
delay: duration"]
end
TP -->|"AcceptTransfer
assert now > delay"| Coin2
subgraph Coin2["Coin (new owner)"]
C2["owner: Bob
issuer: Bank"]
end
```
*Delayed coin ownership transfer*
## Grant time-limited writes to parties
This design pattern demonstrates how to grant time-limited writes to parties.
### Motivation
When parties need to be able to perform ledger writes, but writes need to only be granted for a specific time window.
### Implementation
Transfer proposals can be accepted at any point in time. To restrict this behaviour so that acceptance can only occur within a given time window, a guard for AcceptTransfer choice execution can be added.
TransferProposal contract
In the TransferProposal contract, the body of the AcceptTransfer choice is modified to assert that the contract deadline has been exceeded or passed.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice AcceptTransfer : ContractId Coin
controller newOwner
do
withinWindow <- isLedgerTimeGE startTime && isLedgerTimeLT endTime
_ <- unless withinWindow $ failWithStatus $
FailureStatus
"transfer-outside-time-window"
InvalidGivenCurrentSystemStateOther
("Ledger time is outside permitted transfer time window [" <> show startTime <> ", " <> show endTime <> ")")
(TextMap.fromList [("startTime", show startTime), ("endTime", show endTime)])
create coin with owner = newOwner
```
As transfer proposals are created when a Transfer choice is executed, the interval start and end times, during which an AcceptTransfer can be executed need to be passed in as choice parameters.
Coin contract
In the Coin contract, the Transfer choice has an additional deadline argument, so that TransferProposal contracts can be given a delay.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Transfer : ContractId TransferProposal
with
newOwner: Party
startTime: Time
duration: RelTime
controller owner
do create TransferProposal
with coin=this; newOwner; startTime; addRelTime startTime duration
```
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
subgraph Coin["Coin (active)"]
C1["owner: Alice
issuer: Bank"]
end
Coin -->|"Transfer(start, end)"| TP
subgraph TP["TransferProposal"]
T1["newOwner: Bob
startTime: timestamp
endTime: timestamp"]
end
TP -->|"AcceptTransfer
assert start ≤ now ≤ end"| Coin2
subgraph Coin2["Coin (new owner)"]
C2["owner: Bob
issuer: Bank"]
end
```
*Coin ownership transfer within a time window*
## Where to use getTime
For workflows that prepare and submit transactions, care needs to be taken when using calls to getTime. This is because calls to getTime cause transactions to be bound to the ledger time, and in turn constrain how sequencers may re-order transactions. Global Synchronizers are configured such that the transaction prepare and submit time window is one minute, so any workflow using getTime must prepare and submit transactions within that one-minute time window.
For workflows where this constraint can not be met (e.g. workflows that sign transactions using external parties), it is recommended that workflows are designed to use the ledger time primitives and assertions.
### Motivation
When parties need to perform ledger writes by a given deadline, but are able to prepare and submit a transaction within 1 minute.
### Implementation
Transfer proposals can be accepted at any point in time. To require acceptance by a fixed time, you can add a guard for AcceptTransfer choice execution. Here you determine the current ledger time by calling getTime.
TransferProposal contract
In the TransferProposal contract, the body of the AcceptTransfer choice is modified to assert that the contract deadline is valid relative to the ledger time returned by calling getTime.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice AcceptTransfer : ContractId Coin
controller newOwner
do
t <- getTime
_ <- unless (t < timeLimit) $ failWithStatus $
FailureStatus
"deadline-exceeded"
InvalidGivenCurrentSystemStateOther
("Ledger time is after deadline " <> show timeLimit)
(TextMap.fromList [("timeLimit", show timeLimit)])
create coin with owner = newOwner
```
As transfer proposals are created when a Transfer choice is executed, the time by which an AcceptTransfer can be executed needs to be passed in as a choice parameter.
Coin contract
In the Coin contract, the Transfer choice has an additional deadline argument, so that TransferProposal contracts can be given a fixed lifetime.
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Transfer : ContractId TransferProposal
with
newOwner: Party
timeLimit: Time
controller owner
do create TransferProposal
with coin=this; newOwner; timeLimit
```
# Add constraints to a contract
You will often want to constrain the data stored or the allowed data transformations in your contract models. In this section, you will learn about the two main mechanisms provided in Daml:
* The `ensure` keyword.
* The `assert`, `abort` and, `error` keywords.
To make sense of the latter, you'll also learn more about the `Update` and `Script` types, and `do` blocks, which will be good preparation for `compose`, where you will use `do` blocks to compose choices into complex transactions.
Lastly, you will learn about time on the ledger and in Daml Script.
Remember that you can load all the code for this section into a folder called `intro-constraints` by running `dpm new intro-constraints --template daml-intro-constraints`
## Template preconditions
The first kind of restriction you may want to put on the contract model are called *template pre-conditions*. These are simply restrictions on the data that can be stored on a contract from that template.
Suppose, for example, that the `SimpleIou` contract from `simple_iou` should only be able to store positive amounts. You can enforce this using the `ensure` keyword:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
template SimpleIou
with
issuer : Party
owner : Party
cash : Cash
where
signatory issuer
observer owner
ensure cash.amount > 0.0
```
The `ensure` keyword takes a single expression of type `Bool`. If you want to add more restrictions, use logical operators `&&`, `||`, and `not` to build up expressions. The below shows the additional restriction that currencies are three capital letters:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
&& T.length cash.currency == 3
&& T.isUpper cash.currency
```
The `T` here stands for the `DA.Text` which has been imported from the Daml standard library using `import DA.Text as T`:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
test_restrictions = do
alice <- allocateParty "Alice"
bob <- allocateParty "Bob"
dora <- allocateParty "Dora"
-- Dora can't issue negative Ious.
submitMustFail dora do
createCmd SimpleIou with
issuer = dora
owner = alice
cash = Cash with
amount = -100.0
currency = "USD"
-- Or even zero Ious.
submitMustFail dora do
createCmd SimpleIou with
issuer = dora
owner = alice
cash = Cash with
amount = 0.0
currency = "USD"
-- Nor positive Ious with invalid currencies.
submitMustFail dora do
createCmd SimpleIou with
issuer = dora
owner = alice
cash = Cash with
amount = 100.0
currency = "Swiss Francs"
-- But positive Ious still work, of course.
iou <- submit dora do
createCmd SimpleIou with
issuer = dora
owner = alice
cash = Cash with
amount = 100.0
currency = "USD"
```
## Assertions
A second common kind of restriction is one on data transformations.
For example, the simple Iou in `simple_iou` allowed the no-op where the `owner` transfers to themselves. You can prevent that using an `assert` statement, which you have already encountered in the context of scripts.
`assert` does not return an informative error so often it's better to use the function `assertMsg`, which takes a custom error message:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Transfer
: ContractId SimpleIou
with
newOwner : Party
controller owner
do
assertMsg "newOwner cannot be equal to owner." (owner /= newOwner)
create this with owner = newOwner
```
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- Alice can't transfer to herself...
submitMustFail alice do
exerciseCmd iou Transfer with
newOwner = alice
-- ... but can transfer to Bob.
iou2 <- submit alice do
exerciseCmd iou Transfer with
newOwner = bob
```
Similarly, you can write a `Redeem` choice, which allows the `owner` to redeem an `Iou` *during business hours on weekdays*. The `Redeem` choice implementation below confirms that `getTime` returns a value that is during business hours on weekdays. If all those checks pass, the choice does not do anything other than archive the `SimpleIou`. (This assumes that actual cash changes hands off-ledger:)
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
choice Redeem
: ()
controller owner
do
now <- getTime
let
today = toDateUTC now
dow = dayOfWeek today
timeofday = now `subTime` time today 0 0 0
hrs = convertRelTimeToMicroseconds timeofday / 3600000000
if (hrs < 8 || hrs > 18) then
abort $ "Cannot redeem outside business hours. Current time: " <> show timeofday
else case dow of
Saturday -> abort "Cannot redeem on a Saturday."
Sunday -> abort "Cannot redeem on a Sunday."
_ -> return ()
```
In the above example, the time is taken apart into day of week and hour of day using standard library functions from `DA.Date` and `DA.Time`. The hour of the day is checked to be in the range from 8 to 18. The day of week is checked to not be Saturday or Sunday.
The following example shows how the `Redeem` choice is exercised in a script:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
-- June 1st 2019 is a Saturday.
setTime (time (date 2019 Jun 1) 0 0 0)
-- Bob cannot redeem on a Saturday.
submitMustFail bob do
exerciseCmd iou2 Redeem
-- Not even at mid-day.
passTime (hours 12)
-- Bob cannot redeem on a Saturday.
submitMustFail bob do
exerciseCmd iou2 Redeem
-- Bob also cannot redeem at 6am on a Monday.
passTime (hours 42)
submitMustFail bob do
exerciseCmd iou2 Redeem
-- Bob can redeem at 8am on Monday.
passTime (hours 2)
submit bob do
exerciseCmd iou2 Redeem
```
For the purposes of testing the `Redeem` choice, the above code sets and advances the ledger time with the `setTime` and `passTime` functions respectively. Exercising the choice should fail or should not fail depending on the day of week and the time of day. While that is straightforward, the issue of time on a Daml ledger is worthy of more discussion.
## Time on Daml ledgers
Each transaction on a Daml ledger has two timestamps: the *ledger time (LT)* and the *record time (RT)*.
**Ledger time (LT)** is the time associated with a transaction in the ledger model, as determined by the participant. It is the time of a transaction from a business and application perspective. When you call \`getTime, it is the LT that is returned. The LT is used when reasoning about related transactions and commits. The LT can be compared with other LTs to guarantee model consistency. For example, LTs are used to enforce that no transaction depends on a contract that does not exist. This is the requirement known as "causal monotonicity."
**Record time (RT)** is the time assigned by the persistence layer. It represents the time that the transaction is “physically” recorded. For example, “The backing database ledger has assigned the timestamp of such-and-such time to this transaction.” The only purpose of the RT is to ensure that transactions are being recorded in a timely manner.
Each Daml ledger has a policy on the allowed difference between LT and RT called the *skew*. A consistent zero-skew is not feasible because this is a distributed system. If it is too far off, the transaction will be rejected. This is the requirement known as “bounded skew.” The RT is not relevant beyond this determination of skew.
Returning to the theme of *business hours*, consider the following example: Suppose that the ledger had a skew of 10 seconds. At 17:59:55, just before the end of business hours, Alice submits a transaction to redeem an Iou. One second later, the transaction is assigned an LT of 17:59:56. However, there still may be a few seconds before the transaction is persisted to the underlying storage. For example, the transaction might be written in the underlying backing store at 18:00:06, *after business hours*. Because LT is within business hours and LT - RT .
### Time in test scripts
For tests, you can set time using the following functions:
* `setTime`, which sets the ledger time to the given time.
* `passTime`, which takes a `RelTime` (a relative time) and moves the ledger by that much.
On a distributed Daml ledger, there are no guarantees that LT or RT are strictly increasing. The only guarantee is that ledger time is increasing *with causality*. That is, if a transaction `TX2` depends on a transaction `TX1`, then the ledger enforces that the LT of `TX2` is greater than or equal to that of `TX1`.
The following script illustrates that idea by moving the logical time back by three days and then trying to exercise a choice on a contract *that hasn't been created yet*. That fails, as you would hope.
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
iou3 <- submit dora do
createCmd SimpleIou with
issuer = dora
owner = alice
cash = Cash with
amount = 100.0
currency = "USD"
passTime (days (-3))
submitMustFail alice do
exerciseCmd iou3 Redeem
```
## Actions and `do` blocks
You have come across `do` blocks and `<-` notations in two contexts by now: `Script` and `Update`. Both of these are examples of an `Action`, also called a *Monad* in functional programming. You can construct `Actions` conveniently using `do` notation.
Understanding `Actions` and `do` blocks is therefore crucial to being able to construct correct contract models and test them, so this section will explain them in some detail.
### Pure expressions compared to actions
Expressions in Daml are pure in the sense that they have no side-effects: they neither read nor modify any external state. If you know the value of all variables in scope and write an expression, you can work out the value of that expression on pen and paper.
However, the expressions you've seen that used the `<-` notation are not like that. For example, take `getTime`, which is an `Action`. Here's the example we used earlier:
\`\`daml
now syntax for functions later.
`Coin` and `play` are deliberately left obscure in the above. All you have is an action `getCoin` to get your hands on a `Coin` in a `Script` context and an action `flipCoin` which represents the simplest possible game: a single coin flip resulting in a `Face`.
You can't play any `CoinGame` game on pen and paper as you don't have a coin, but you can write down a script or recipe for a game:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
coin_test = do
-- The coin is pseudo-random on LT so change the parameter to change the game.
setTime (time (date 2019 Jun 1) 0 0 0)
passTime (seconds 2)
coin <- getCoin
let
game = do
f1r <- flipCoin
f2r <- flipCoin
f3r <- flipCoin
if all (== Heads) [f1r, f2r, f3r]
then return "Win"
else return "Loss"
(newCoin, result) = game.play coin
assert (result == "Win")
```
The `game` expression is a `CoinGame` in which a coin is flipped three times. If all three tosses return `Heads`, the result is `"Win"`, or else `"Loss"`.
In a `Script` context you can get a `Coin` using the `getCoin` action, which uses the LT to calculate a seed, and play the game.
*Somehow* the `Coin` is threaded through the various actions. If you want to look through the looking glass and understand in-depth what's going on, you can look at the source file to see how the `CoinGame` action is implemented, though be warned that the implementation uses a lot of Daml features we haven't introduced yet in this introduction.
More generally, if you want to learn more about Actions (aka Monads), we recommend a general course on functional programming, and Haskell in particular. See `haskell-connection` for some suggestions.
## Errors
Above, you've learnt about `assertMsg` and `abort`, which represent (potentially) failing actions. Actions only have an effect when they are performed, so the following script succeeds or fails depending on the value of `abortScript`:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
nonPerformedAbort = do
let abortScript = False
let failingAction : Script () = abort "Foo"
let successfulAction : Script () = return ()
if abortScript then failingAction else successfulAction
```
However, what about errors in contexts other than actions? Suppose we wanted to implement a function `pow` that takes an integer to the power of another positive integer. How do we handle that the second parameter has to be positive?
One option is to make the function explicitly partial by returning an `Optional`:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
optPow : Int -> Int -> Optional Int
optPow base exponent
| exponent == 0 = Some 1
| exponent > 0 =
let Some result = optPow base (exponent - 1)
in Some (base * result)
| otherwise = None
```
This is a useful pattern if we need to be able to handle the error case, but it also forces us to always handle it as we need to extract the result from an `Optional`. We can see the impact on convenience in the definition of the above function. In cases, like division by zero or the above function, it can therefore be preferable to fail catastrophically instead:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
errPow : Int -> Int -> Int
errPow base exponent
| exponent == 0 = 1
| exponent > 0 = base * errPow base (exponent - 1)
| otherwise = error "Negative exponent not supported"
```
The big downside to this is that even unused errors cause failures. The following script will fail, because `failingComputation` is evaluated:
```daml theme={"theme":{"light":"github-light","dark":"github-dark"}}
nonPerformedError = script do
let causeError = False
let failingComputation = errPow 1 (-1)
let successfulComputation = errPow 1 1
return if causeError then failingComputation else successfulComputation
```
`error` should therefore only be used in cases where the error case is unlikely to be encountered, and where explicit partiality would unduly impact usability of the function.
## Next up
You can now specify a precise data and data-transformation model for Daml ledgers. In `parties`, you will learn how to properly involve multiple parties in contracts, how authority works in Daml, and how to build contract models with strong guarantees in contexts with mutually distrusting entities.
# Application Architecture
Source: https://docs.canton.network/appdev/modules/m4-app-architecture
Understand the roles, layers, and component interactions in a Canton Network application
Canton applications follow a layered architecture where Daml smart contracts define the shared business logic, a backend mediates access to the ledger, and a frontend presents the user interface. This page describes the roles involved and how the layers connect.
## Roles
Three roles appear in most Canton applications.
**App Provider** -- The organization that builds, deploys, and operates the application. The App Provider typically runs its own validator, hosts the backend services, and serves the frontend. Because the App Provider's validator hosts the provider's party, it stores the provider-side contract data and exposes the Ledger API that the backend connects to.
**App User** -- A party that interacts with the application's contracts. An App User may connect through the App Provider's validator (if the provider hosts the user's party) or through the user's own validator. In either case, the user's party needs to be allocated on a validator that participates in the relevant synchronizer.
**End User** -- A person who interacts with the application through a browser, mobile app, or wallet UI. End Users typically do not manage validators or think about ledger mechanics. They authenticate through the frontend, and the backend handles ledger operations on their behalf.
## Architecture Layers
A Canton application has three main layers.
### Daml Model
The Daml model defines the contracts, choices, and authorization rules that make up the shared business logic. Once compiled into a DAR file with `dpm build`, this model is deployed to the validators that host the relevant parties. The Daml model is the single source of truth for what data exists on the ledger and what actions are possible.
### Backend
The backend is a service (Java or TypeScript) that connects to the validator's Ledger API. It submits commands (creating contracts, exercising choices) and reads the transaction stream or queries PQS for contract state. The backend also handles authentication, business logic that doesn't belong on-ledger, and any integration with external systems.
In the [cn-quickstart](https://github.com/digital-asset/cn-quickstart) reference application, the backend is a Spring Boot Java service.
### Frontend
The frontend is a web application (in this case it is React) that communicates with the backend over HTTP. It does not connect to the Ledger API directly. The backend exposes a REST API defined by an OpenAPI schema, and the frontend consumes it with generated TypeScript clients.
This separation keeps ledger concerns out of the browser and centralizes authentication and access control in the backend.
## Component Diagram
The following diagram shows how the layers connect at runtime.
```mermaid theme={"theme":{"light":"github-light","dark":"github-dark"}}
flowchart LR
FE[Frontend
React / Web UI]
BE[Backend
Java / TypeScript]
PN[Validator
Participant Node
Ledger API gRPC + JSON]
PQS[(PQS
PostgreSQL)]
SYNC[Synchronizer]
FE -- "HTTP/REST" --> BE
BE -- "Commands &
Transactions" --> PN
PN <-- "Consensus" --> SYNC
PN -- "Projections" --> PQS
BE -- "SQL Queries" --> PQS
```
* The **frontend** sends HTTP requests to the backend.
* The **backend** submits commands and reads transactions through the Ledger API (gRPC).
* The **validator** (participant node) processes commands submitted via the LAPI, stores contract data for its hosted parties, and synchronizes with other validators through the synchronizer.
* **PQS** maintains a PostgreSQL projection of ledger state that the backend can query with SQL.
## Where PQS Fits In
The Ledger API is optimized for command submission and streaming transaction updates. For read-heavy workloads -- dashboards, reports, filtered queries, aggregations -- PQS is a better fit. PQS subscribes to the validator's transaction stream and writes contract data into PostgreSQL tables. Your backend queries these tables with standard SQL. PQS can be used to create new projections of data by combining data using standard SQL joins. PQS also stores historical data which can be used for audits.
Using PQS means your read path does not compete with your write path for Ledger API resources, and you can use the full power of PostgreSQL (joins, indexes, full-text search) on your contract data.
## Choosing an Architecture Style
The cn-quickstart project uses a **fully mediated** architecture: the backend handles all ledger interactions, and the frontend only talks to the backend. This is the simplest model for most applications.
An alternative is a **CQRS** architecture where the frontend submits commands directly to the Ledger API (using TypeScript bindings from `dpm codegen-js`) while the backend handles queries. This gives the frontend tighter integration with the ledger but requires it to understand Canton concepts like party IDs and contract IDs.
Pick the fully mediated approach unless you have a specific reason to expose ledger concepts to the frontend.
## Next Steps
* [SDKs and APIs](/appdev/modules/m4-sdks-apis) -- The tools and interfaces available for each layer
* [Backend Development](/appdev/modules/m4-backend-dev) -- Patterns for connecting to the Ledger API and PQS
* [Frontend Development](/appdev/modules/m4-frontend-dev) -- Building a React UI against the backend API
# Backend Development
Source: https://docs.canton.network/appdev/modules/m4-backend-dev
Build a backend service that connects to the Canton Ledger API and PQS, with code examples from cn-quickstart
The backend is the layer between your frontend and the Canton ledger. It submits commands, reads transactions, and queries contract state.
This page uses [cn-quickstart](https://github.com/digital-asset/cn-quickstart) as a running example. cn-quickstart is a full-stack reference application that implements a software licensing workflow on Canton Network. It includes a Spring Boot backend, a React frontend, Daml smart contracts, and all the configuration needed to run locally or deploy to DevNet. The patterns shown here — connecting to the Ledger API, submitting commands, querying PQS, handling errors — apply to any Canton backend, but the code samples are drawn directly from the [cn-quickstart backend](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/backend) so you can see them in a working context.
## Backend Language
cn-quickstart uses a Java backend built on Spring Boot. Java has first-class code generation support (`dpm codegen-java`).
TypeScript is also supported through `dpm codegen-js`. A TypeScript backend uses the same Ledger API (via gRPC-js or the JSON API) and can be a good choice if your team prefers a single language across frontend and backend.
## Connecting to the Ledger API
The Ledger API is a service exposed by the validator's participant node. Your backend connects to it as an authenticated client, acting on behalf of one or more parties.
In cn-quickstart, [`LedgerApi.java`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/ledger/LedgerApi.java) manages this connection. The constructor builds a gRPC channel with an authentication interceptor that attaches a bearer token to every call:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
ManagedChannelBuilder builder = ManagedChannelBuilder
.forAddress(ledgerConfig.getHost(), ledgerConfig.getPort())
.usePlaintext();
builder.intercept(new Interceptor(tokenProvider));
ManagedChannel channel = builder.build();
submission = CommandSubmissionServiceGrpc.newFutureStub(channel);
commands = CommandServiceGrpc.newFutureStub(channel);
```
The `Interceptor` class attaches the bearer token to gRPC metadata on every call:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
@Override
public ClientCall interceptCall(
MethodDescriptor method, CallOptions callOptions, Channel next) {
ClientCall clientCall = next.newCall(method, callOptions);
return new ForwardingClientCall.SimpleForwardingClientCall<>(clientCall) {
@Override
public void start(Listener responseListener, Metadata headers) {
headers.put(AUTHORIZATION_HEADER, "Bearer " + tokenProvider.getToken());
super.start(responseListener, headers);
}
};
}
```
On LocalNet, the token comes from Keycloak. In production, it comes from your OAuth2 provider.
## Command Submission
Commands are how you write to the ledger. There are two primary operations: creating a contract and exercising a choice.
### Creating a contract
To create a contract, build a `Create` command with the template identifier and payload, then submit it. In cn-quickstart, the `LedgerApi.create()` method handles this:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
public CompletableFuture create(T entity, String commandId) {
CommandsOuterClass.Command.Builder command = CommandsOuterClass.Command.newBuilder();
ValueOuterClass.Value payload = dto2Proto.template(entity.templateId()).convert(entity);
command.getCreateBuilder()
.setTemplateId(toIdentifier(entity.templateId()))
.setCreateArguments(payload.getRecord());
return submitCommands(List.of(command.build()), commandId)
.thenApply(submitResponse -> null);
}
```
The `dto2Proto` converter translates generated Java classes into Protobuf values. The `commandId` is a UUID that the Ledger API uses for deduplication.
### Exercising a choice
Exercising a choice requires the contract ID and the choice arguments. The `exerciseAndGetResult()` method builds an `Exercise` command and submits it through the `CommandService`, which waits for the transaction result:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
CommandsOuterClass.Command.Builder cmdBuilder = CommandsOuterClass.Command.newBuilder();
ValueOuterClass.Value payload =
dto2Proto.choiceArgument(choice.templateId(), choice.choiceName()).convert(choice);
cmdBuilder.getExerciseBuilder()
.setTemplateId(toIdentifier(choice.templateId()))
.setContractId(contractId.getContractId)
.setChoice(choice.choiceName())
.setChoiceArgument(payload);
```
A REST endpoint that exercises a choice ties these pieces together. Here's how [`LicenseApiImpl.java`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/service/LicenseApiImpl.java) expires a license:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
@Override
public CompletableFuture> expireLicense(
String contractId, String commandId, LicenseExpireRequest request) {
return auth.asAuthenticatedParty(party -> {
return damlRepository.findLicenseById(contractId).thenCompose(optContract -> {
var license = ensurePresent(optContract,
"License not found for contract %s", contractId);
License_Expire choice = new License_Expire(
new Party(auth.getAppProviderPartyId()),
toTokenStandardMetadata(request.getMeta().getData()));
return ledger.exerciseAndGetResult(license.contractId, choice, commandId)
.thenApply(result -> ResponseEntity.ok("License expired successfully"));
});
});
}
```
The pattern is: look up the contract (from PQS), build the choice object (from generated code), and submit it through the Ledger API.
### Handling Contract IDs
Every active contract has a unique contract ID. To exercise a choice, you need the target contract's ID. Your backend obtains contract IDs by querying PQS or reading them from the transaction stream. The cn-quickstart REST API passes contract IDs in URL paths (e.g., `POST /api/licenses/{contractId}/expire`).
Contract IDs are opaque strings. Do not parse or construct them manually.
## Querying PQS
For read operations, the cn-quickstart backend queries PQS rather than the Ledger API. PQS maintains a PostgreSQL database that mirrors the ledger state visible to the validator's hosted parties.
### The Pqs adapter
[`Pqs.java`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/pqs/Pqs.java) wraps Spring's `JdbcTemplate` and uses PQS's `active()` table-valued function to query active contracts:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
public CompletableFuture>> active(Class clazz) {
Identifier identifier = Utils.getTemplateIdByClass(clazz);
String sql = "select contract_id, payload from active(?)";
return runAndTraceAsync(ctx, () ->
jdbcTemplate.query(sql, new PqsContractRowMapper<>(identifier),
identifier.qualifiedName())
);
}
```
The `active()` function takes a qualified template name (e.g., `quickstart_licensing:Licensing.License:License`) and returns all active contracts of that type. Each row contains a `contract_id` and a JSON `payload`.
For filtered queries, `activeWhere()` appends a `WHERE` clause:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
public CompletableFuture>> activeWhere(
Class clazz, String whereClause, Object... params) {
Identifier identifier = Utils.getTemplateIdByClass(clazz);
String sql = "select contract_id, payload from active(?) where " + whereClause;
return runAndTraceAsync(ctx, () ->
jdbcTemplate.query(sql, new PqsContractRowMapper<>(identifier),
combineParams(identifier.qualifiedName(), params))
);
}
```
### Domain-specific queries
[`DamlRepository.java`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/repository/DamlRepository.java) builds higher-level queries. For example, `findActiveLicenses()` joins licenses with their renewal requests and allocation contracts in a single SQL query:
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
SELECT license.contract_id AS license_contract_id,
license.payload AS license_payload,
renewal.contract_id AS renewal_contract_id,
renewal.payload AS renewal_payload,
allocation.contract_id AS allocation_contract_id
FROM active(?) license
LEFT JOIN active(?) renewal ON
license.payload->>'licenseNum' = renewal.payload->>'licenseNum'
AND license.payload->>'user' = renewal.payload->>'user'
LEFT JOIN active(?) allocation ON
renewal.payload->>'requestId' =
allocation.payload->'allocation'->'settlement'->'settlementRef'->>'id'
WHERE license.payload->>'user' = ? OR license.payload->>'provider' = ?
ORDER BY license.contract_id
```
PQS stores contract payloads as JSONB, so you use PostgreSQL's `->` and `->>` operators to filter and join on contract fields. You can create additional PostgreSQL indexes on frequently queried JSON paths for performance.
## Reading Transactions
The Ledger API also exposes a transaction stream that your backend can subscribe to. Each transaction includes the created and archived contracts visible to your party. This is useful when you need real-time event processing rather than polling PQS.
A transaction stream subscription uses gRPC server streaming. Here's the pattern from the [docs-website quickstart](https://github.com/DACH-NY/docs-website):
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
UpdateServiceGrpc.UpdateServiceStub updateServiceStub = UpdateServiceGrpc.newStub(channel);
updateServiceStub.getUpdates(getUpdatesRequest.toProto(), new StreamObserver<>() {
public void onNext(UpdateServiceOuterClass.GetUpdatesResponse r) {
GetUpdatesResponse response = GetUpdatesResponse.fromProto(r);
response.getTransaction().ifPresent(transaction -> {
for (Event event : transaction.getEvents()) {
if (event instanceof CreatedEvent createdEvent) {
Iou.Contract contract = Iou.Contract.fromCreatedEvent(createdEvent);
// update local cache or trigger side effects
} else if (event instanceof ArchivedEvent archivedEvent) {
// remove from local cache
}
}
});
}
public void onError(Throwable throwable) { /* handle error */ }
public void onCompleted() { /* stream ended */ }
});
```
In cn-quickstart, PQS handles read-side projection, so the backend does not maintain its own event-sourced state. If you are not using PQS, streaming transactions and maintaining your own projections is the alternative.
## Error Handling
Canton uses structured error codes. When a command fails, the Ledger API returns a gRPC `StatusRuntimeException` with a Canton-specific error code. Common categories:
* **NOT\_FOUND** -- The contract ID does not exist or is not visible to the submitting party
* **ALREADY\_EXISTS** -- A duplicate command was submitted (commands are deduplicated by command ID)
* **INVALID\_ARGUMENT** -- The command payload does not match the template or choice signature
* **FAILED\_PRECONDITION** -- A contract was archived between the time you read it and the time you exercised a choice on it (contention)
In Java, catch `StatusRuntimeException` and inspect the status code:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
import io.grpc.StatusRuntimeException;
import io.grpc.Status;
try {
ledger.exerciseAndGetResult(contractId, choice, commandId).join();
} catch (CompletionException e) {
if (e.getCause() instanceof StatusRuntimeException sre) {
if (sre.getStatus().getCode() == Status.Code.NOT_FOUND) {
// contract was archived — re-query PQS and retry
}
}
}
```
For contention errors, a retry strategy often resolves the issue: re-read the current contract ID from PQS, then resubmit the command.
Set a unique command ID on each new submission. The Ledger API deduplicates commands by this ID, which prevents double-submission if your backend retries a command that actually succeeded but whose response was lost in transit.
## Backend Architecture in cn-quickstart
The cn-quickstart backend follows this module structure (under [`backend/src/main/java/com/digitalasset/quickstart/`](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/backend/src/main/java/com/digitalasset/quickstart)):
* [`service/`](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/service) -- REST endpoint implementations. Each endpoint combines a PQS query or a Ledger API command.
* [`ledger/`](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/ledger) -- The gRPC Ledger API client. Submits commands to the validator.
* [`repository/`](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/repository) -- Domain-specific PQS query methods.
* [`pqs/`](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/pqs) -- Low-level SQL generation and PostgreSQL access.
* [`utility/`](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/utility) -- JSON configuration, tracing, and helper methods.
* [`security/`](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/security) -- OAuth2 bearer token validation and party authentication.
* [`config/`](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/config) -- Spring Boot configuration properties.
This structure cleanly separates read concerns (PQS) from write concerns (Ledger API) while keeping the REST layer thin.
## Exercise: Add License Comments
This exercise walks you through adding a comment feature to cn-quickstart. Users will be able to post comments on licenses and view all comments for a given license. The feature touches every layer of the backend: Daml model, code generation, OpenAPI spec, PQS queries, and REST endpoints.
By the end, you'll have a working `/licenses/{contractId}/comments` API that creates `LicenseComment` contracts on the ledger and reads them back through PQS.
### Step 1: Define the Daml Template
Create a new file `quickstart/daml/licensing/daml/Licensing/LicenseComment.daml`:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
module Licensing.LicenseComment where
template LicenseComment
with
provider : Party
user : Party
licenseNum : Int
commenter : Party
body : Text
createdAt : Time
where
signatory commenter
observer provider, user
```
The `commenter` is the signatory — only the person writing the comment can create it. Both `provider` and `user` are observers so they can see comments on their licenses. The `licenseNum` field links the comment to a specific license without requiring a contract ID reference (which would break if the license is renewed or archived).
Compare this with the `License` template in [`Licensing/License.daml`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/daml/licensing/daml/Licensing/License.daml), which uses dual signatories (`signatory provider, user`). A comment only needs the commenter's authority, making it simpler to create — no multi-party workflow required.
### Step 2: Build and Generate Java Bindings
Compile the new Daml module and regenerate the Java bindings. If you're working outside cn-quickstart, use `dpm` directly:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
dpm build
dpm codegen-java -o
```
In cn-quickstart, the Gradle build handles both steps. Run the Daml build task, which compiles the `.daml` files into a DAR and then runs the transcode codegen plugin to produce Java classes:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
./gradlew :daml:build
```
After this step, you'll have a generated `LicenseComment` Java class under `backend/build/generated-daml-bindings/` in the `quickstart_licensing.licensing.licensecomment` package. The generated class uses public final fields (`getProvider`, `getUser`, `getLicenseNum`, etc.) rather than getter methods — this is the transcode codegen style used throughout cn-quickstart.
### Step 3: Add the OpenAPI Spec
Add the comment schema and endpoints to `quickstart/common/openapi.yaml`.
First, define the schemas in the `components/schemas` section:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
LicenseComment:
type: object
required:
- contractId
- provider
- user
- licenseNum
- commenter
- body
- createdAt
properties:
contractId:
type: string
provider:
type: string
user:
type: string
licenseNum:
type: integer
commenter:
type: string
body:
type: string
createdAt:
type: string
format: date-time
AddCommentRequest:
type: object
required:
- body
properties:
body:
type: string
```
Then add two endpoints under `paths`:
```yaml theme={"theme":{"light":"github-light","dark":"github-dark"}}
/licenses/{contractId}/comments:
get:
tags: [Licenses]
summary: List comments for a license
operationId: listLicenseComments
parameters:
- $ref: '#/components/parameters/ContractId'
responses:
'200':
description: A list of comments
content:
application/json:
schema:
type: array
items:
$ref: '#/components/schemas/LicenseComment'
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
'500':
$ref: '#/components/responses/InternalError'
post:
tags: [Licenses]
summary: Add a comment to a license
operationId: addLicenseComment
parameters:
- $ref: '#/components/parameters/ContractId'
- $ref: '#/components/parameters/CommandId'
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/AddCommentRequest'
responses:
'201':
description: Comment created
'401':
$ref: '#/components/responses/Unauthorized'
'404':
$ref: '#/components/responses/NotFound'
'500':
$ref: '#/components/responses/InternalError'
```
The GET endpoint takes only the contract ID (to look up the license's `licenseNum`). The POST endpoint also takes a `commandId` for Ledger API deduplication, following the same convention as `renewLicense` and `expireLicense`.
Use `tags: [Licenses]` so these endpoints are grouped with the existing license operations. The cn-quickstart OpenAPI generator produces a `LicensesApi` Java interface from all endpoints whose paths start with `/licenses/`, so your new methods will appear in the same interface that `LicenseApiImpl` already implements.
After updating the spec, regenerate the Spring server stubs:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
./gradlew openApiGenerate
```
This reads `common/openapi.yaml` and writes the generated interfaces and model classes to `backend/build/generated-spring/`. Your new `listLicenseComments` and `addLicenseComment` methods will appear in the `LicensesApi` interface, and the `LicenseComment` and `AddCommentRequest` model classes will be generated in `org.openapitools.model`.
### Step 4: Add the PQS Query
Add a method to [`DamlRepository.java`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/repository/DamlRepository.java) that finds comments by license number. This follows the same `activeWhere()` pattern used for filtering licenses:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
public CompletableFuture>> findCommentsByLicenseNum(int licenseNum) {
return pqs.activeWhere(
LicenseComment.class,
"payload->>'licenseNum' = ?",
String.valueOf(licenseNum)
);
}
```
PQS stores all contract payloads as JSON, so `payload->>'licenseNum'` extracts the field as text. The `activeWhere()` method in [`Pqs.java`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/pqs/Pqs.java) appends this `WHERE` clause to the `active()` table-valued function, returning only `LicenseComment` contracts whose `licenseNum` matches.
### Step 5: Add the REST Endpoints
Add two methods to [`LicenseApiImpl.java`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/service/LicenseApiImpl.java). After the Gradle build regenerates `LicensesApi` from the updated OpenAPI spec, `LicenseApiImpl` won't compile until you implement the two new interface methods. Both follow the existing pattern: authenticate, look up data, execute, and return a response.
**List comments** — authenticate the caller, look up the license to get its `licenseNum`, query PQS for matching comments:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
@Override
@WithSpan
public CompletableFuture>> listLicenseComments(
String contractId) {
var ctx = tracingCtx(logger, "listLicenseComments", "contractId", contractId);
return auth.asAuthenticatedParty(party -> traceServiceCallAsync(ctx, () ->
damlRepository.findLicenseById(contractId).thenCompose(optLicense -> {
var license = ensurePresent(optLicense,
"License not found for contract %s", contractId);
return damlRepository.findCommentsByLicenseNum(
license.payload.getLicenseNum.intValue())
.thenApply(comments -> {
var result = comments.stream()
.map(LicenseApiImpl::toLicenseCommentApi)
.toList();
return ResponseEntity.ok(result);
});
})
));
}
```
**Add a comment** — authenticate the caller, look up the license, create a `LicenseComment` contract on the ledger:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
@Override
@WithSpan
public CompletableFuture> addLicenseComment(
String contractId, String commandId, AddCommentRequest request) {
var ctx = tracingCtx(logger, "addLicenseComment",
"contractId", contractId, "commandId", commandId);
return auth.asAuthenticatedParty(party -> traceServiceCallAsync(ctx, () ->
damlRepository.findLicenseById(contractId).thenCompose(optLicense -> {
var license = ensurePresent(optLicense,
"License not found for contract %s", contractId);
var now = Instant.now();
var comment = new quickstart_licensing.licensing
.licensecomment.LicenseComment(
license.payload.getProvider,
license.payload.getUser,
license.payload.getLicenseNum,
new Party(party),
request.getBody(),
now
);
return ledger.create(comment, commandId)
.thenApply(v -> ResponseEntity.status(HttpStatus.CREATED)
.build());
})
));
}
```
Note the fully-qualified `quickstart_licensing.licensing.licensecomment.LicenseComment` type name — this is the Daml-generated class, distinct from the OpenAPI-generated `org.openapitools.model.LicenseComment` that the REST response uses. The field accessors (`license.payload.getProvider`, `license.payload.getLicenseNum`) are public final fields on the transcode-generated classes, not getter methods.
You'll also need a converter method to map between the two `LicenseComment` types — from the Daml contract (read from PQS) to the API model (returned as JSON):
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
private static LicenseComment toLicenseCommentApi(
Contract contract) {
var p = contract.payload;
var api = new LicenseComment();
api.setContractId(contract.contractId.getContractId);
api.setProvider(p.getProvider.getParty);
api.setUser(p.getUser.getParty);
api.setLicenseNum(p.getLicenseNum.intValue());
api.setCommenter(p.getCommenter.getParty);
api.setBody(p.getBody);
api.setCreatedAt(toOffsetDateTime(p.getCreatedAt));
return api;
}
```
This follows the same pattern as `toLicenseApi()` already in the file. The write path uses `ledger.create()` — the same method shown in the [Command Submission](#creating-a-contract) section above. The read path queries PQS through `DamlRepository`, keeping reads and writes separated.
### Step 6: Test It
Since you changed the Daml model, you need a clean environment — the new DAR won't upload over an existing one. Stop and reset LocalNet, then rebuild and start fresh:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
make clean-docker
make build
make start
```
Wait for all services to come up. The onboarding containers will automatically register the app-user tenant.
**Get an auth token.** The backend uses OAuth2 via Keycloak. Obtain a token using the client credentials grant with the backend service account:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
TOKEN=$(curl -s -X POST \
"http://localhost:8082/realms/AppProvider/protocol/openid-connect/token" \
-H "Content-Type: application/x-www-form-urlencoded" \
-d "client_id=app-provider-backend" \
-d "client_secret=05dmL9DAUmDnIlfoZ5EQ7pKskWmhBlNz" \
-d "grant_type=client_credentials" \
-d "scope=openid" | jq -r .access_token)
```
These credentials are from `docker/backend-service/onboarding/env/oauth2.env` and are only valid for the local development environment.
**Create a license to test with.** The quickstart doesn't create licenses automatically — they're created by the app provider through the `AppInstall` workflow. First, create an `AppInstallRequest` from the app-user participant:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
make create-app-install-request
```
Then accept the request and create a license using the backend API:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Get the AppInstallRequest contract ID
REQUEST_CID=$(curl -s -H "Authorization: Bearer $TOKEN" \
http://localhost:8080/app-install-requests | jq -r '.[0].contractId')
# Accept the install request
curl -s -X POST \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"installMeta":{"data":{}},"meta":{"data":{}}}' \
"http://localhost:8080/app-install-requests/${REQUEST_CID}:accept?commandId=accept-$(date +%s)"
# Wait a moment for PQS to index, then get the AppInstall contract ID
sleep 3
INSTALL_CID=$(curl -s -H "Authorization: Bearer $TOKEN" \
http://localhost:8080/app-installs | jq -r '.[0].contractId')
# Create a license from the AppInstall
curl -s -X POST \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d '{"params":{"meta":{"data":{}}}}' \
"http://localhost:8080/app-installs/${INSTALL_CID}:create-license?commandId=license-$(date +%s)"
```
**Test the comment endpoints.** List the active licenses and pick a contract ID:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
LICENSE_CID=$(curl -s -H "Authorization: Bearer $TOKEN" \
http://localhost:8080/licenses | jq -r '.[0].contractId')
```
Now test the comment endpoints:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Add a comment
curl -X POST "http://localhost:8080/licenses/${LICENSE_CID}/comments?commandId=$(uuidgen)" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{"body": "Renewed for another quarter"}'
# Wait for PQS to index, then list comments
sleep 3
curl -s "http://localhost:8080/licenses/${LICENSE_CID}/comments" \
-H "Authorization: Bearer $TOKEN" | jq .
```
## Next Steps
* [Frontend Development](/appdev/modules/m4-frontend-dev) -- Build a React UI that consumes this backend's REST API, including a frontend exercise that adds comment UI on top of these backend endpoints
* [Canton Coin and Traffic](/appdev/modules/m4-canton-coin) -- Understand traffic costs and wallet integration for payments
* [cn-quickstart repository](https://github.com/digital-asset/cn-quickstart) -- Full working backend implementation
## Advanced Topics
* [Command Deduplication](/appdev/deep-dives/command-deduplication) — Designing application command flows so an intended ledger change is executed exactly once, even under retries, crashes, and lost network messages.
* [Explicit Contract Disclosure](/appdev/deep-dives/explicit-contract-disclosure) — Submitting commands that read a contract you do not stakeholder by passing it as a disclosed contract on the Ledger API.
# Building Applications
Source: https://docs.canton.network/appdev/modules/m4-building-apps-intro
Go from Daml smart contracts to full-stack Canton Network applications
Module 4 bridges the gap between writing Daml contracts and shipping a complete application. You will learn how Canton applications are structured, which SDKs and APIs are available, and how to build both backend and frontend components that interact with the ledger.
## Prerequisites
Before starting this module, you should have completed [Module 3: Daml Smart Contracts](/appdev/modules/m3-dev-environment). You need a working understanding of templates, choices, and authorization in Daml. Familiarity with Java or TypeScript is helpful but not required.
## What You Will Learn
* How Canton application architecture maps roles (App Provider, App User, End User) to infrastructure
* Which SDKs, APIs, and code generation tools are available for building against the ledger
* How to build a backend service that submits commands and reads transactions
* How to build a frontend that displays contract data and integrates with wallets
* How Canton Coin and traffic work from an application developer's perspective
## Module Pages
Roles, layers, and how the pieces of a Canton application fit together.
Code generation, Ledger API, JSON API, PQS, and the Wallet SDK.
Connect to the ledger, submit commands, read transactions, and query PQS.
Build a React UI with generated TypeScript bindings and wallet integration.
Understand how CC buys traffic and how to manage transaction costs.
# Canton Coin and Traffic
Source: https://docs.canton.network/appdev/modules/m4-canton-coin
How Canton Coin buys traffic credits and what application developers need to know about transaction costs
Every transaction on the Global Synchronizer costs traffic. Canton Coin (CC) is the native currency you use to purchase traffic credits. As an application developer, you need to understand this relationship so your users' transactions go through without interruption.
## Canton Coin
Canton Coin is the native utility token of the Global Synchronizer. It is implemented through [Splice](https://github.com/canton-network/splice), the open-source infrastructure layer for decentralized Canton synchronizers.
CC has several primary purposes on the network:
* **Buying traffic** -- Convert CC into traffic credits so your validator can submit transactions
* **Validator rewards** -- Validators earn CC for operating infrastructure and processing transactions
* **Application rewards** -- Application providers earn CC based on the use of their application (adding value) to the Canton Network
* **Governance** -- Super Validators stake CC to participate in synchronizer governance
CC balances and transaction history are publicly visible through the network's scan service. Private application contract data on Canton is visible only to entitled parties, but CC does not have this privacy property.
## Traffic: How Transaction Fees Work
Traffic is Canton's term for transaction fees. You do not pay CC directly when submitting a transaction. Instead, your validator maintains a **traffic budget** measured in traffic credits. Each transaction deducts credits from this budget based on the transaction's size and complexity.
The two-step process:
1. **Top up** -- Convert CC into traffic credits, adding them to your validator's traffic budget
2. **Consume** -- When your validator submits a transaction, traffic credits are deducted from the budget
Traffic credits are non-transferable. Once CC is converted to traffic, those credits can only be consumed by transaction fees on that validator.
### What Affects Traffic Cost
Traffic cost for a given transaction depends on:
* **Transaction size** -- Larger payloads (more contract data, more parties) cost more credits
* **Network conditions** -- Costs may vary with network load
## The Auto-Top-Up Feature
Validators can enable **auto-top-up** to automatically purchase traffic credits when the budget drops below a configured threshold. When auto-top-up is active, the validator converts CC to traffic credits without manual intervention.
This is the recommended approach for App Providers. Without auto-top-up, you would need to monitor your traffic budget and manually trigger top-ups before it runs out. If the budget is exhausted, your validator's transactions will fail until more credits are added.
Auto-top-up requires that the validator's wallet holds sufficient CC. If the wallet balance is too low to purchase traffic, auto-top-up cannot proceed and transactions will start failing.
## Managing Traffic as an App Provider
As an App Provider, your validator submits transactions on behalf of your application's parties. You are responsible for ensuring your validator has enough traffic credits to handle the transaction volume your application generates.
Practical considerations:
* **Estimate traffic usage** -- Understand how many transactions your application generates per day and how large they are. Use DevNet or TestNet to measure actual traffic consumption. When a transaction is prepared for an external party, an estimate of the traffic used for that transaction can be provided.
* **Enable auto-top-up** -- Configure your validator to automatically replenish traffic credits. Set the threshold high enough to absorb traffic spikes.
* **Monitor your wallet balance** -- Auto-top-up draws from your wallet's CC balance. Keep it funded.
* **Handle insufficient-traffic errors** -- If a transaction fails because of insufficient traffic, your backend should surface a clear error rather than silently retrying. The fix is to top up the traffic budget, not to retry the command.
## Obtaining Canton Coin
How you get CC depends on the environment:
* **LocalNet** -- Test CC is available automatically. No action needed.
* **DevNet** -- Use the faucet (tap) to receive free test CC. Rate-limited, no real value.
* **TestNet** -- Same faucet mechanism as DevNet. Test CC only.
* **MainNet** -- Purchase CC from supported exchanges, earn it through validator operations, or receive it via direct transfer from another party.
## Wallet Integration for Applications
If your application involves CC payments between parties (e.g., a user paying for a license), you integrate with the Splice wallet system. In cn-quickstart, the license renewal flow demonstrates this:
1. The [`License_Renew`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/daml/licensing/daml/Licensing/License.daml) choice creates a `LicenseRenewalRequest` contract that implements the Splice `AllocationRequest` interface
2. The Splice wallet detects the allocation request, creates an `AppPaymentRequest`, and transfers CC from the user to the provider
3. Once payment settles, the provider exercises [`LicenseRenewalRequest_CompleteRenewal`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/daml/licensing/daml/Licensing/License.daml) to create the renewed license
See [`LicenseApiImpl.java`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/backend/src/main/java/com/digitalasset/quickstart/service/LicenseApiImpl.java) for the backend implementation of this flow.
The wallet handles CC transfers, balance lookups, and payment confirmations. Your Daml model defines payment request contracts that the wallet system understands.
## Traffic vs. CC: A Summary
* **Canton Coin (CC)** is the currency. You hold it in a wallet, transfer it between parties, and use it to buy traffic.
* **Traffic credits** are what your validator spends when submitting transactions. They live in the validator's traffic budget, not in a wallet.
* You convert CC to traffic credits via top-up (manually or automatically). The reverse is not possible -- traffic credits cannot be converted back to CC.
## Further Reading
* [Canton Coin overview](/overview/understand/canton-coin) -- Deeper background on CC tokenomics, validator rewards, and governance
* [Backend Development](/appdev/modules/m4-backend-dev) -- Handling transaction errors, including insufficient-traffic failures
* [cn-quickstart](https://github.com/digital-asset/cn-quickstart) -- Working example of wallet integration in a Canton application
# Featured App Activity Markers
Source: https://docs.canton.network/appdev/modules/m4-featured-app-activity-marker
How to create FeaturedAppActivityMarker contracts to record app activity and earn rewards
When your application runs on Canton Network, economically meaningful events can be recorded as **app activity**. The preferred mechanism for this is the `FeaturedAppActivityMarker` contract. SV (Super Validator) automation converts these markers into `AppRewardCoupon` contracts, which translate to Canton Coin (CC) rewards for your application.
## What is a FeaturedAppActivityMarker?
A `FeaturedAppActivityMarker` records a specific activity event in your application. It signals to the network that something economically relevant happened -- for example, a real-world asset was locked, a token was minted, or a license was renewed.
After [CIP-0078](https://github.com/canton-foundation/cips/blob/main/cip-0078/cip-0078.md), only **featured** apps receive rewards. Unfeatured apps can still create markers, but they will not generate reward coupons.
## When to Create Markers
Beyond the basics above, other examples of appropriate markers include settling a payment or trade, and renewing or issuing a license. Do not create markers for routine housekeeping operations (cache refreshes, health checks, internal bookkeeping) that do not represent user-facing economic value.
## How to Create Markers
The `FeaturedAppActivityMarker` contract is specified in [CIP-0047](https://github.com/canton-foundation/cips/blob/main/cip-0047/cip-0047.md). Your Daml code creates the marker as part of the transaction that performs the economically meaningful action.
The marker contract includes fields that identify your application, the type of activity, and metadata about the event. Refer to [CIP-0047](https://github.com/canton-foundation/cips/blob/main/cip-0047/cip-0047.md) for the exact contract interface and required fields.
The *featured application activity marker* is specified in [CIP 47](https://github.com/canton-foundation/cips/blob/main/cip-0047/cip-0047.md). A summary follows.
Featured application activity markers (FeaturedAppActivityMarker) can be created for a transaction that adds value but does not involve a CC Transfer (e.g., a stable coin transfer or the settlement of a trade). [CIP 47](https://github.com/canton-foundation/cips/blob/main/cip-0047/cip-0047.md) says:
> Featured application providers are expected to create featured application activity markers only for transactions that correspond to a transfer of an asset, or an equivalent transaction, which was enabled by the application provider. The detailed fair usage policy and enforcement thereof is left up to the Tokenomics Committee of the Canton Foundation (CF).
Generally the guidance is to create a `FeaturedAppActivityMarker` for any economically important event, such as:
* Lock or unlock a Real World Asset (RWA).
* Transfer a RWA.
* Mint or burn tokens.
However, it should not be created for any intermediate steps or a propose step.
A `FeaturedAppActivityMarker` is immediately converted into an AppRewardCoupon by the automation run by the Super Validators. The AppRewardCoupon is created with the DSO as the `signatory` and the `provider` field as an observer. By default, the `provider` can mint CC in the minting step. The `featured` field is set to `true` to indicate eligibility to receive featured application rewards, based on a FeaturedAppRight contract.
There can be several `FeaturedAppActivityMarkers` per transaction tree which increases the total reward. However, this is only allowed for composed transactions (e.g. a settlement transaction) where trading venue and all the registries of the transferred assets would get featured app rewards. It is also possible for a single Canton transaction tree to include ValidatorRewardCoupon, an AppRewardCoupon and a FeaturedAppActivityMarker(s) if there are sub-transcations that create each separately.
A non-featured app cannot accrue a `FeaturedAppActivityMarker`.
### Creating a Featured Application Activity Marker
There are two prerequisites for an application to create a `FeaturedAppActivityMarker`. The first is to become an approved featured application which was described in the [types\_of\_activity\_records](/overview/reference/canton-coin-tokenomics) section. The second is to update the application code:
> 1\. Find the fully qualified package-id of the interface definition for the FeaturedAppRight interface which is `7804375fe5e4c6d5afe067bd314c42fe0b7d005a1300019c73154dd939da4dda:Splice.Api.FeaturedAppRightV1:FeaturedAppRight` for `Splice.Api.FeaturedAppRightV1`. The command `daml damlc inspect-dar` can be used to find this.
>
> 2\. Query the ledger using this ID to retrieve contracts from the Daml ledger that implement the FeaturedAppRight interface. The `curl` example below illustrates this approach.
>
> ````bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
> curl "http://$lapiParticipant/v2/state/active-contracts" \
> "$jwtToken" "application/json" \
> --data-raw '{
> "filter": {
> "filtersByParty": {
> "'$holderPartyId'": {
> "cumulative":
> [
> {
> "identifierFilter": {
> "InterfaceFilter": {
> "value": {
> "interfaceId": "'7804375fe5e4c6d5afe067bd314c42fe0b7d005a1300019c73154dd939da4dda:Splice.Api.FeaturedAppRightV1:FeaturedAppRight'",
> "includeInterfaceView": true,
> "includeCreatedEventBlob": false
> }
> }
> }
> }
> ]}
> }
> },
> "verbose": false,
> "activeAtOffset":"'$latestOffset'"
> }'
> ```text
>
> 3\. The application's Daml code will have to depend on the `splice-api-featured-app-v1.dar` and take an argument of type `ContractId FeaturedAppRight` on the choice whose execution should be featured, which allows that choice's body to call the `FeaturedAppRight_CreateActivityMarker` in the next step.
>
> 3\. In the application's Daml code, using the `FeaturedAppRight` interface, exercise the `FeaturedAppRight_CreateActivityMarker` choice. Set the `templateId` to the fully qualified interface ID above.
>
> 4\. For testing examples, please review the example DamlScript test [here](https://github.com/canton-network/splice/blob/a32995a0df2d447b9e76d81b770a06c296295ab5/daml/splice-dso-governance-test/daml/Splice/Scripts/TestFeaturedAppActivityMarkers.daml#L4).
> ````
Consider a single, simple transaction of a RWA which creates a single `FeaturedAppActivityMarker` activity record for one `provider` and the `beneficiary` is the `provider`:
> 1\. A FeaturedAppActivityMarker contract is created in the business transaction. The `provider` is set to the featured application provider's party. The `beneficiary` must be set (unlike an AppRewardCoupon) to the party that should be eligible to mint the CC for that activity. The `provider` field of the FeaturedAppActivityMarker is set by calling the interface choice FeaturedAppRight\_CreateActivityMarker.
>
> 2. No `ValidatorRewardCoupon` is created.
It is possible to share the attribution of activity for the `FeaturedAppActivityMarker`. The `FeaturedAppRight_CreateActivityMarker` choice accepts a list of AppRewardBeneficiary contracts. Then a `FeaturedAppActivityMarker` is created for each `beneficiary` with the `weight` field set appropriately.
### Markers in cn-quickstart
In [cn-quickstart](https://github.com/digital-asset/cn-quickstart), the license renewal flow creates a `FeaturedAppActivityMarker` when a license is renewed. The marker is created inside the Daml choice that performs the renewal, so it is part of the same transaction and recorded atomically.
## Getting Your App Featured
To receive rewards from your markers, your application must be **featured** on the network.
### On DevNet
You can self-feature your application on DevNet for testing purposes. This lets you verify that your markers are created correctly and that the SV automation converts them into reward coupons, without going through the formal review process.
### On TestNet and MainNet
For TestNet and MainNet, the process is:
1. **Submit your application** via the CF (Canton Foundation) form. You provide details about your application, the types of activities you record, and why they represent genuine economic value.
2. **Tokenomics committee review** -- The CF tokenomics committee evaluates your submission. They assess whether the recorded activities reflect real economic value and whether the marker creation is appropriate.
3. **Approval and featuring** -- If approved, your application is added to the featured apps list. From that point, your markers generate reward coupons.
## Reward Mechanics
The reward calculation depends on the number and type of activity markers your application creates, relative to other featured apps on the network. The exact formula is governed by the tokenomics rules defined by the CF.
Key points:
* Rewards are paid in CC to the app provider party
* More activity markers (representing genuine economic events) generally means more rewards
* The reward pool is shared across all featured apps, so your share depends on your relative activity
* Gaming the system (creating markers for non-economic events) risks losing featured status
## Testing Markers Locally
On LocalNet (via cn-quickstart), the SV automation that converts markers to coupons is part of the local Splice infrastructure. You can:
1. Trigger an action in your application that creates a marker
2. Query PQS for active `FeaturedAppActivityMarker` contracts to verify they were created
3. Wait for the SV automation cycle, then query for `AppRewardCoupon` contracts to confirm conversion
This local testing loop lets you validate your marker creation logic before deploying to DevNet.
## Further Reading
* [Canton Coin and Traffic](/appdev/modules/m4-canton-coin) -- How CC rewards relate to traffic and transaction costs
* [CIP-0047](https://github.com/canton-foundation/cips/blob/main/cip-0047/cip-0047.md) -- The specification for FeaturedAppActivityMarker
* [CIP-0078](https://github.com/canton-foundation/cips/blob/main/cip-0078/cip-0078.md) -- The change restricting rewards to featured apps
* [Deployment Progression](/appdev/modules/m5-deployment-progression) -- Moving from LocalNet to DevNet to MainNet
# Frontend Development
Source: https://docs.canton.network/appdev/modules/m4-frontend-dev
Build a React frontend for Canton applications with TypeScript bindings and wallet integration
The frontend is the user-facing layer of your Canton application. This page uses [cn-quickstart](https://github.com/digital-asset/cn-quickstart) as a running example. cn-quickstart is a full-stack reference application that implements a software licensing workflow on Canton Network. The [cn-quickstart frontend](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/frontend) is a React application built with TypeScript and Vite. It communicates with the backend over HTTP using types generated from a shared OpenAPI schema. The patterns shown here — connecting to the backend, displaying contract data, handling authentication — apply to any Canton frontend, but the code samples are drawn directly from cn-quickstart so you can see them in a working context.
## Connecting to the Backend
In cn-quickstart, the frontend does not talk to the Ledger API directly — all ledger interactions go through the backend's REST endpoints. Canton does provide a [JSON API](/sdks-tools/api-reference/json-api) that frontends can use for direct ledger access, but the cn-quickstart architecture routes everything through the backend for separation of concerns.
The API client is configured in [`api.ts`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/src/api.ts) using the `openapi-client-axios` library, which reads the OpenAPI schema and produces a typed HTTP client:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAPIClientAxios from 'openapi-client-axios';
import openApi from '../../common/openapi.yaml'
const api: OpenAPIClientAxios = new OpenAPIClientAxios({
definition: openApi as any,
withServer: { url: '/api' },
});
api.init();
export default api;
```
The shared [`common/openapi.yaml`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/common/openapi.yaml) defines every endpoint, request body, and response shape. The `openapi-client-axios` library generates a typed `Client` interface from this spec at build time, so every API call in the frontend is type-checked against the backend's contract:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import type { Client, License } from '../openapi.d.ts';
const client: Client = await api.getClient();
const response = await client.listLicenses(); // typed as License[]
```
If the backend API changes, the frontend build breaks rather than failing silently at runtime.
## TypeScript Code Generation
The cn-quickstart frontend generates its TypeScript types from the OpenAPI spec using the `gen:openapi` script in [`package.json`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/package.json):
```json theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
"scripts": {
"gen:openapi": "npx --yes openapicmd typegen --client ../common/openapi.yaml >| src/openapi.d.ts",
"build": "npm run gen:openapi && tsc -b --noEmit && vite build"
}
}
```
The `build` script runs type generation before compilation, so the TypeScript types always match the OpenAPI schema.
Separately, `dpm codegen-js` generates TypeScript types from your compiled DAR file. These types mirror your Daml templates, choices, and data types:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
dpm codegen-js -o
```
Whether you use DAR-generated types depends on your architecture:
* **Fully mediated** (cn-quickstart default) -- The frontend uses OpenAPI-generated types from the backend's REST schema. The Daml-generated TypeScript types are not needed in the frontend because the backend translates between ledger concepts and REST DTOs.
* **Direct ledger access via JSON API** -- The frontend submits commands through the [JSON API](/sdks-tools/api-reference/json-api) using the Daml-generated TypeScript bindings. This gives tighter integration with the ledger but requires the frontend to handle party IDs, contract IDs, and command submission directly.
For most applications, the fully mediated approach is simpler. The JSON API approach makes sense when you want a thin or no backend layer.
## Application Structure
The cn-quickstart frontend uses React Context providers for state management. [`App.tsx`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/src/App.tsx) composes them at the top level:
```tsx theme={"theme":{"light":"github-light","dark":"github-dark"}}
const App: React.FC = () => {
const AppProviders = composeProviders(
ToastProvider,
UserProvider,
TenantRegistrationProvider,
AppInstallProvider,
LicenseProvider
);
return (
} />
} />
} />
} />
} />
);
};
```
Each domain has its own store under [`stores/`](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/frontend/src/stores) that wraps a React Context with the API calls and state for that domain. The stores use the typed `Client` from the OpenAPI schema for all backend communication.
## Displaying Contract Data
The frontend fetches contract data from the backend's GET endpoints and renders it in React components. The [`licenseStore`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/src/stores/licenseStore.tsx) manages license state and API calls:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
export const LicenseProvider = ({ children }: { children: React.ReactNode }) => {
const [licenses, setLicenses] = useState([]);
const toast = useToast();
const fetchLicenses = useCallback(
withErrorHandling(`Fetching Licenses`)(async () => {
const client: Client = await api.getClient();
const response = await client.listLicenses();
setLicenses(response.data);
}), [withErrorHandling, setLicenses, toast]);
// ... other operations (renew, expire, complete renewal)
return (
{children}
);
};
```
[`LicensesView.tsx`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/src/views/LicensesView.tsx) consumes this store and renders the data with periodic polling to keep the UI current:
```tsx theme={"theme":{"light":"github-light","dark":"github-dark"}}
const LicensesView: React.FC = () => {
const { licenses, fetchLicenses, initiateLicenseRenewal,
initiateLicenseExpiration, completeLicenseRenewal } = useLicenseStore();
const { user } = useUserStore();
useEffect(() => {
fetchLicenses();
const intervalId = setInterval(() => {
fetchLicenses();
}, 5000);
return () => clearInterval(intervalId);
}, [fetchLicenses]);
return (
Licenses
| License Contract ID |
Expires At |
License # |
Status |
Actions |
{licenses.map((license) => (
| {license.contractId} |
{formatDateTime(license.expiresAt)} |
{license.licenseNum} |
{license.isExpired ? 'EXPIRED' : 'ACTIVE'} |
{/* Renew, Archive buttons */}
|
))}
);
};
```
Because the backend handles all ledger translation, the frontend works with plain JSON objects. Fields like `contractId`, `expiresAt`, and `licenseNum` appear as simple strings and numbers in the `License` type — not ledger-specific types.
## Exercising Choices via the Backend
When the user takes an action (renew a license, expire a license), the frontend posts to the backend's REST API. Each request includes a unique command ID that the backend passes to the Ledger API for deduplication. From the license store:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
const expireLicense = useCallback(
withErrorHandling(`Archiving License`)(async (contractId: string, meta: Metadata) => {
const client: Client = await api.getClient();
const commandId = generateCommandId();
await client.expireLicense({ contractId, commandId }, { meta });
await fetchLicenses();
toast.displaySuccess('License archived successfully');
}),
[withErrorHandling, fetchLicenses, toast]
);
```
The [`generateCommandId()`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/src/utils/commandId.ts) utility produces a UUID using the browser's `crypto.randomUUID()` API. The backend forwards this ID to the Ledger API, which uses it to prevent duplicate command submission if the user retries an action.
## Authentication
Canton applications typically use OAuth2 / OpenID Connect (OIDC) for user authentication. On LocalNet, cn-quickstart uses Keycloak as the identity provider. The backend handles the OAuth2 flow, and the frontend manages session state through the [`userStore`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/src/stores/userStore.tsx):
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
export const UserProvider = ({ children }: { children: React.ReactNode }) => {
const [user, setUser] = useState(null);
const [loading, setLoading] = useState(true);
const fetchUser = useCallback(async () => {
setLoading(true);
try {
const client: Client = await api.getClient();
const response = await client.getAuthenticatedUser();
setUser(response.data);
} catch (error) {
if ((error as any)?.response?.status === 401) {
setUser(null);
} else {
toast.displayError('Error fetching user');
}
} finally {
setLoading(false);
}
}, [setUser, setLoading, toast]);
const logout = useCallback(async () => {
const response = await fetch('/api/logout', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'X-XSRF-TOKEN': getCsrfToken(),
},
});
if (response.ok) {
clearUser();
navigate('/');
}
}, [clearUser, toast, navigate]);
return (
{children}
);
};
```
The `AuthenticatedUser` type (from the OpenAPI spec) includes the user's name, whether they're an admin, and their wallet URL. Components use this to conditionally render admin-only features and to link to the Splice wallet. The backend handles all authorization decisions — the frontend is responsible only for checking whether the user is logged in and displaying the appropriate UI.
## Wallet Integration
Applications that involve Canton Coin payments integrate with a wallet component in the frontend. In cn-quickstart, the license renewal flow triggers a payment through the Splice wallet system:
1. The user clicks "Renew License" in the UI
2. The frontend calls `client.renewLicense()`, which posts to the backend
3. The backend exercises the `License_Renew` choice, creating a `LicenseRenewalRequest` on the ledger that implements the Splice `AllocationRequest` interface
4. The Splice wallet detects the allocation request and creates an `AppPaymentRequest` for the user to approve
5. Once payment is confirmed, the provider calls `completeLicenseRenewal()` to create the renewed license
The `LicensesView` tracks renewal state by polling. Each license object includes its `renewalRequests` array, and the UI shows the count of pending and accepted renewals. The user's wallet URL comes from the `AuthenticatedUser` object and is used to link to the Splice wallet for payment approval.
## Development Workflow
When developing the frontend iteratively:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
make build-frontend # Rebuild after source changes
make restart # Restart services to pick up changes
```
For faster iteration, run the Vite dev server directly against a running LocalNet backend:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
make start-vite-dev
```
The Vite dev server proxies API requests to the backend. The proxy configuration in [`vite.config.ts`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/vite.config.ts) routes `/api`, `/login`, and `/oauth2` paths to the backend:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
export default defineConfig(({ mode }: ConfigEnv) => {
const env = loadEnv(mode, '../');
const backendPort = env.VITE_BACKEND_PORT || 8080;
return {
plugins: [react(), ViteYaml()],
server: {
host: 'localhost',
strictPort: true,
allowedHosts: ['app-provider.localhost'],
proxy: {
'/api': {
target: `http://localhost:${backendPort}/`,
changeOrigin: false,
rewrite: path => path.replace(/^\/api/, ''),
},
'/login': {
target: `http://localhost:${backendPort}/`,
changeOrigin: false,
},
'/oauth2': {
target: `http://localhost:${backendPort}/`,
changeOrigin: false,
},
},
},
}
});
```
The `ViteYaml` plugin allows importing the OpenAPI YAML file directly as a JavaScript module, which is how `api.ts` loads the schema at build time.
## Exercise: Add License Comments UI
This exercise builds on the backend exercise in [Backend Development](/appdev/modules/m4-backend-dev#exercise-add-license-comments). Complete that first — you need the `LicenseComment` Daml template, the OpenAPI endpoints, and the backend implementation before the frontend can use them.
You'll add a comment list and comment form to the licenses view, following the same store/view patterns that cn-quickstart uses for licenses.
### Step 1: Regenerate TypeScript Types
The backend exercise added `LicenseComment`, `AddCommentRequest`, and two new endpoints to `openapi.yaml`. Regenerate the frontend types so the typed client picks them up:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
npm run gen:openapi
```
This runs `openapicmd typegen` against the shared `openapi.yaml` and overwrites `src/openapi.d.ts`. After this, your `Client` type will include `listLicenseComments()` and `addLicenseComment()` methods with the correct parameter and return types.
### Step 2: Create a Comment Store
Create a new store at `quickstart/frontend/src/stores/commentStore.tsx`. Follow the same Context + Provider pattern used by [`licenseStore.tsx`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/src/stores/licenseStore.tsx):
```tsx theme={"theme":{"light":"github-light","dark":"github-dark"}}
import React, { createContext, useContext, useState, useCallback } from 'react';
import { useToast } from './toastStore';
import api from '../api';
import { generateCommandId } from '../utils/commandId';
import type { Client, LicenseComment } from '../openapi.d.ts';
import { withErrorHandling } from '../utils/error';
interface CommentContextType {
comments: LicenseComment[];
fetchComments: (contractId: string) => Promise;
addComment: (contractId: string, body: string) => Promise;
}
const CommentContext = createContext(undefined);
export const CommentProvider = ({ children }: { children: React.ReactNode }) => {
const [comments, setComments] = useState([]);
const toast = useToast();
const fetchComments = useCallback(
withErrorHandling('Fetching Comments')(async (contractId: string) => {
const client: Client = await api.getClient();
const response = await client.listLicenseComments({ contractId });
setComments(response.data);
}),
[withErrorHandling, setComments, toast]
);
const addComment = useCallback(
withErrorHandling('Adding Comment')(async (contractId: string, body: string) => {
const client: Client = await api.getClient();
const commandId = generateCommandId();
await client.addLicenseComment({ contractId, commandId }, { body });
await fetchComments(contractId);
toast.displaySuccess('Comment added successfully');
}),
[withErrorHandling, fetchComments, toast]
);
return (
{children}
);
};
export const useCommentStore = () => {
const context = useContext(CommentContext);
if (context === undefined) {
throw new Error('useCommentStore must be used within a CommentProvider');
}
return context;
};
```
The structure mirrors `licenseStore.tsx`: a React Context holds the state, each API call is wrapped with `withErrorHandling` (which displays toast messages on HTTP errors) and `useCallback`, and the typed `Client` provides compile-time checking against the OpenAPI spec. The `addComment` function generates a command ID for deduplication — the same pattern `expireLicense` and `renewLicense` use.
Register the provider in `App.tsx` alongside the existing providers:
```tsx theme={"theme":{"light":"github-light","dark":"github-dark"}}
const AppProviders = composeProviders(
ToastProvider,
UserProvider,
TenantRegistrationProvider,
AppInstallProvider,
LicenseProvider,
CommentProvider // add this
);
```
### Step 3: Add the Comment UI
Add a comment section to [`LicensesView.tsx`](https://github.com/digital-asset/cn-quickstart/blob/main/quickstart/frontend/src/views/LicensesView.tsx). You'll need to import the comment store and add some local state:
```tsx theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { useCommentStore } from '../stores/commentStore';
// inside the component:
const { comments, fetchComments, addComment } = useCommentStore();
const [showComments, setShowComments] = useState(null);
const [commentBody, setCommentBody] = useState('');
```
Add an effect that polls for comments when a license's comment panel is open. This follows the same 5-second `setInterval` pattern `LicensesView` already uses for the license list:
```tsx theme={"theme":{"light":"github-light","dark":"github-dark"}}
useEffect(() => {
if (showComments) {
fetchComments(showComments);
const intervalId = setInterval(() => fetchComments(showComments), 5000);
return () => clearInterval(intervalId);
}
}, [showComments, fetchComments]);
```
Add a "Comments" toggle button in each license row's actions column, and render the comment panel below the license table. The panel shows existing comments and a form for new ones:
```tsx theme={"theme":{"light":"github-light","dark":"github-dark"}}
{showComments && (
Comments for License #
{licenses.find(l => l.contractId === showComments)?.licenseNum}
{comments.length === 0 ? (
No comments yet.
) : (
{comments.map((c) => (
-
{c.commenter}
{formatDateTime(c.createdAt)}
{c.body}
))}
)}
)}
```
The `showComments` state tracks which license's contract ID is currently expanded. The `formatDateTime` utility is already imported in `LicensesView` for the expiration column. The `handleAddComment` handler calls `addComment` from the store, then clears the input.
### Step 4: Build and Test
Rebuild both the backend (to pick up the OpenAPI changes and new Java code) and the frontend:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
make build-frontend # runs npm run gen:openapi && tsc && vite build
make build-backend # runs ./gradlew :backend:build
make restart # restart all services
```
Or for faster frontend iteration with hot reload:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
make start-vite-dev
```
Open the app, navigate to the Licenses page, click the "Comments" button on a license, and try posting a comment. The comment should appear in the list within 5 seconds (or immediately after the POST completes, since `addComment` calls `fetchComments` after success).
## Next Steps
* [Canton Coin and Traffic](/appdev/modules/m4-canton-coin) -- How CC and traffic affect your application
* [Backend Development](/appdev/modules/m4-backend-dev) -- The backend that the frontend communicates with
* [cn-quickstart frontend source](https://github.com/digital-asset/cn-quickstart/tree/main/quickstart/frontend) -- Full working frontend implementation
# Canton and the JSON Ledger API
Source: https://docs.canton.network/appdev/modules/m4-json-api-tutorial
Get started with Canton and the JSON Ledger API, first with curl and then with TypeScript.
## Get started with Canton and the JSON Ledger API
This tutorial demonstrates how to interact with the Canton Ledger using the JSON Ledger API from the command line. It uses `curl` and optionally `websocat` to communicate with the Ledger.
### Overview
This example shows how to:
* enable the JSON Ledger API in the Canton configuration
* create a contract using the JSON Ledger API and basic Bash tools (`curl`)
* list active contracts using `curl`
* basic error handling and troubleshooting
### Prerequisites
#### Tools
Before running the project, ensure you have the following installed:
* A Bash-compatible terminal (e.g., macOS Terminal, Git Bash, etc.)
* **Dpm** — Install following these instructions
* `canton` — Canton release with installation and examples, check Canton demo for details
* `curl` — command-line HTTP client: [https://github.com/curl/curl](https://github.com/curl/curl) (installation: [https://curl.se/download.html](https://curl.se/download.html))
* (Optional) `jq` — command-line JSON processor: [https://github.com/jqlang/jq](https://github.com/jqlang/jq) (installation: [https://jqlang.org/download/](https://jqlang.org/download/))
#### Daml Model
To demonstrate the JSON Ledger API, you need an example Daml model.
Run the following command in the console:
```
dpm new json-tests
```
A folder named `json-tests` should be created. Check its contents and inspect the code in `daml/Main.daml`.
It should contain a Daml model with a template named `Asset`:
```
template Asset
with
issuer : Party
owner : Party
name : Text
where
ensure name /= ""
signatory issuer
observer owner
choice Give : AssetId
with
newOwner : Party
controller owner
do create this with
owner = newOwner
```
Before proceeding, compile the Daml model by running the following command inside the folder:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
dpm build
```
A file named `.daml/dist/json-tests-0.0.1.dar` should be created.
### Starting Canton with JSON Ledger API
First, ensure you can run the Canton sandbox with the JSON Ledger API enabled.
Start the Canton sandbox, providing a path to the created `dar` file:
```
dpm sandbox --json-api-port 7575 --dar /.daml/dist/json-tests-0.0.1.dar
```
To simplify this tutorial, no security-related configuration is used. You will be able to send commands as any user.
Open a new Bash terminal, but keep the Canton sandbox running.
#### Verification - download OpenAPI
To verify that the JSON Ledger API is working, check if the documentation endpoint is available.
In a new Bash terminal, run:
```
curl localhost:7575/docs/openapi
```
You should receive a long YAML document starting with:
```
openapi: 3.0.3
info:
title: JSON Ledger API HTTP endpoints
version: 3.3.0
paths:
/v2/commands/submit-and-wait:
post:
```
This `openapi.yaml` document provides an overview of the available endpoints and can be pasted into a tool like [https://editor-next.swagger.io](https://editor-next.swagger.io).
It can also be used to generate client stubs in languages like `Java` or `TypeScript`.
You can use the `http://localhost:757/livez` endpoint to check whether the server is running. It might be used as a health check in production environments.
#### Create a party
Run this command in the terminal:
```
curl -d '{"partyIdHint":"Alice", "identityProviderId": ""}' -H "Content-Type: application/json" -X POST localhost:7575/v2/parties
```
This should return a response similar to:
```
{"partyDetails":{"party":"Alice::122084768362d0ce21f1ffec870e55e365a292cdf8f54c5c38ad7775b9bdd462e141","isLocal":true,"localMetadata":{"resourceVersion":"0","annotations":{}},"identityProviderId":""}}
```
Take note of the party ID, which will be used in the following steps. In this example, it is:
`Alice::122084768362d0ce21f1ffec870e55e365a292cdf8f54c5c38ad7775b9bdd462e141` — but this will differ in your case.
### Create a contract
Create a file called `create.json` with the following content, replacing `\` with the actual party ID shown earlier:
```
{
"commands": [
{
"CreateCommand": {
"createArguments": {
"issuer": "",
"owner": "",
"name": "Example Asset Name"
},
"templateId": "#json-tests:Main:Asset"
}
}
],
"userId": "ledger-api-user",
"commandId": "example-app-create-1234",
"actAs": [
""
],
"readAs": [
""
]
}
```
Now submit the request:
```
curl localhost:7575/v2/commands/submit-and-wait -H "Content-Type: application/json" -d@create.json
```
A successful response should look like:
```
{"updateId":"...","completionOffset":20}
```
This confirms that a contract was created on the ledger.
### Query the ledger
To query the ledger for active contracts, first create a file named \`acs.json\`:
```
{
"eventFormat": {
"filtersByParty": {},
"filtersForAnyParty": {
"cumulative": [
{
"identifierFilter": {
"WildcardFilter": {
"value": {
"includeCreatedEventBlob": true
}
}
}
}
]
},
"verbose": false
},
"verbose": false,
"activeAtOffset":
}
```
Replace `\` with the `completionOffset` you received earlier (e.g., `20`).
Run the query:
```
curl localhost:7575/v2/state/active-contracts -H "Content-Type: application/json" -d@acs.json
```
You should receive a response containing contract information.
To improve readability, pipe the output to \`jq\`:
```
curl localhost:7575/v2/state/active-contracts -H "Content-Type: application/json" -d@acs.json | jq
```
Look for the `createdEvent` section, which contains contract details like:
```
"createdEvent": {
"offset": 20,
"nodeId": 0,
"contractId": "00572c50513ced94f9cddaf1e6d2d3f050ae35d7fea0affe06f65f4238e84136edca1112202d01e45b5cfafb61e3942e4610547689dbebfebd3ea7d10d57944401fc17e81b",
"templateId": "6fd1d46124d5ab0c958ce35e9bb370bb2835b2672a0d6fa039a3855c11b8801d:Main:Asset",
"contractKey": null,
"createArgument": {
"issuer": "Alice::122084768362d0ce21f1ffec870e55e365a292cdf8f54c5c38ad7775b9bdd462e141",
"owner": "Alice::122084768362d0ce21f1ffec870e55e365a292cdf8f54c5c38ad7775b9bdd462e141",
"name": "Example Asset Name"
},
...
}
```
### Troubleshooting
If you encounter issues while calling the using curl - you should enable `-v` (verbose) mode to see the request and response details. For instance:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -v -d '{"partyIdHint":"Alice", "identityProviderId": ""}' -H "Content-Type: application/json" -X POST localhost:7575/v2/parties
```
Http response different than 200 (e.g., 400, 404, etc.) indicates an error. The response body will contain details about the error.
If it does not help, read logs available in the canton sandbox terminal or in the file `\/logs/canton.log`.
If nothing is returned when you query `localhost:7575/v2/state/active-contracts` ensure that the offset provided is correct and corresponds to the `completionOffset` from the `localhost:7575/v2/commands/submit-and-wait` command. You can also check current offset by running:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl localhost:7575/v2/state/ledger-end
```
### Next steps
#### Canton examples
For a more advanced scenario involving two parties, explore the examples provided with Canton installation: `\/examples/json-ledger-api\>`
A `TypeScript` version is also available that demonstrates how to create a JSON Ledger API client for use in a web browser.
#### OpenAPI and AsyncAPI
Read more about the OpenAPI and AsyncAPI specifications for the Canton JSON Ledger API: `references_json-api`.
#### Authentication and security
Read how to configure and use jwt token to access JSON Ledger API `json-api-access-tokens`.
#### Error codes
For more information about error codes returned by the JSON Ledger API, see `json-error-codes`.
## Get Started with Canton, the JSON Ledger API, and TypeScript
This tutorial shows you how to interact with a Canton Ledger using the JSON Ledger API and TypeScript.
### Overview
You will use and modify an existing example project provided with the Canton distribution.
### Prerequisites
You should be familiar with the basics of TypeScript and tools such as `npm` and `node.js`.
#### Tools
Before starting, ensure you have the following installed:
* **Node.js and npm** — Download from [https://nodejs.org/en/download/](https://nodejs.org/en/download/). Recommended version: `18.20.x` or later.
* **Dpm** — Install following these instructions
* **Canton** — Includes pre-built examples. See the Canton demo for details.
#### Example TypeScript Project
Open a terminal and navigate to the JSON Ledger API example folder:
```
cd /examples/09-json-ledger-api
```
Start Canton:
```
./run.sh
```
Once the Canton console is ready, open a new terminal and navigate to the TypeScript example folder:
```
cd /examples/09-json-ledger-api/typescript
```
### Running the Example
Install the project dependencies:
```
npm install
```
You may see some warnings, which can be ignored for now.
The JSON Ledger API provides an OpenAPI specification. You can use this to generate TypeScript client classes (stubs).
To generate the TypeScript client:
```
npm run generate_api
```
Check the generated code in the `generated/api` folder. It should include the TypeScript classes for the API.
```
less generated/api/ledger-api.d.ts
```
This file contains definitions for services and models, such as `/v2/commands/submit-and-wait`, `CreateCommand`, and many others.
Next, generate TypeScript types from the Daml model:
```
npm run generate_daml_bindings
```
This creates bindings in the `./generated` folder. Each Daml module has its own subfolder, for example: `generated/model-tests-1.0.0/lib/Iou`.
Now compile the TypeScript code:
```
npm run build
```
Once the build succeeds, run the example:
```
npm run scenario
```
You should see output similar to:
```
Alice creates contract
Ledger offset: 23
...
Bob accepts transfer
...
End of scenario
```
### Code Highlights
The JSON Ledger API client is configured in \`src/client.ts\`:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import type { paths } from "../generated/api/ledger-api";
export const client = createClient({ baseUrl: "http://localhost:7575" });
```
The `openapi-fetch` library is used to create the API client.
#### Allocating a Party
In `src/user.ts`, a party is allocated as follows:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
const resp = await client.POST("/v2/parties", {
body: {
partyIdHint: user,
identityProviderId: "",
}
});
```
`openapi-fetch` uses the Indexed Access Types TypeScript feature to provide type safety. The example above looks like untyped JavaScript, but it is in fact type-safe.
#### Creating a Contract
In `src/commands.ts`, a contract is created using:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
export async function createContract(userParty: string): Promise {
const iou: Iou.Iou = {
issuer: userParty,
owner: userParty,
currency: "USD",
amount: "100",
observers: []
};
const command: components["schemas"]["CreateCommand"] = {
createArguments: iou,
templateId: Iou.Iou.templateId
};
const jsCommands = makeCommands(userParty, [{ CreateCommand: command }]);
const resp = await client.POST("/v2/commands/submit-and-wait", {
body: jsCommands
});
return valueOrError(resp);
}
```
#### Querying Contracts
In `src/index.ts`, contracts are queried like this:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
const { data, error } = await client.POST("/v2/state/active-contracts", {
body: filter
});
if (data === undefined)
return Promise.reject(error);
else {
const contracts: components["schemas"]["CreatedEvent"][] = data
.map((res) => res.contractEntry)
.filter((res) => "JsActiveContract" in res)
.map((res) => res.JsActiveContract.createdEvent);
return Promise.resolve(contracts);
}
```
### Extending the Example
In this step, you extend the `Iou` template with a new field and update the TypeScript code accordingly.
1. Open the `Io.daml` file in `canton/examples/09-json-ledger-api/model`.
2. Modify the `Iou` template to include a new `comment` field:
```haskell theme={"theme":{"light":"github-light","dark":"github-dark"}}
template Iou
with
issuer : Party
owner : Party
currency : Text
amount : Decimal
observers : [Party]
comment : Optional Text -- added field
where
-- leave the rest of the template unchanged
```
\> The `comment` field is optional, making this a backwards-compatible change.
3. Stop the Canton server (`Ctrl+C`) and restart it:
```
./run.sh
```
4. Rebuild the TypeScript bindings:
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
npm run generate_daml_bindings
```
5. Update the `createContract` function in `src/commands.ts` to include the new field:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
export async function createContract(userParty: string): Promise {
const iou: Iou.Iou = {
issuer: userParty,
owner: userParty,
currency: "USD",
amount: "100",
observers: [],
comment: "This is a test comment" // new field
};
// leave the rest of the function unchanged
```
6. Rebuild the TypeScript code:
```
npm run build
```
7. Run the example:
```
npm run scenario
```
You won’t see the new `comment` field in the output yet. To display it, modify the `showAcs` function call in `src/index.ts` to include the new field:
```typescript theme={"theme":{"light":"github-light","dark":"github-dark"}}
showAcs(contracts, ["owner", "amount", "currency", "comment"], c => [c.owner, c.amount, c.currency, c.comment]);
```
The, execute the `npm run scenario` once again.
## Related
A TypeScript version is also available that demonstrates how to create a JSON Ledger API client for use in a web browser.
# Observability
Source: https://docs.canton.network/appdev/modules/m4-observability
Setting up logging, metrics, and dashboards for Canton applications
Running a Canton application in production means you need visibility into what your backend and the underlying Canton nodes are doing. This page covers logging, metrics, and dashboards from an application developer's perspective.
## Logging
### Structured Logging with Logback
Canton nodes and cn-quickstart both use [Logback](https://logback.qos.ch/) for logging. The default configuration writes human-readable logs to the console, but for production you should switch to structured JSON output. JSON logs are easier to ingest into log aggregation systems like the ELK stack or Grafana Loki.
A typical `logback.xml` configuration for JSON output:
```xml theme={"theme":{"light":"github-light","dark":"github-dark"}}
command-id
party
```
Including the `command-id` in your MDC (Mapped Diagnostic Context) lets you trace a single Ledger API command through your backend logs and correlate it with the Canton node logs that processed it.
### What to Log
Focus your application logs on:
* **Command submissions** -- Log the command ID, template or choice name, and submitting party before each Ledger API call
* **Command completions** -- Log success or failure, including the error code on failure
* **PQS query performance** -- Log slow queries (above a threshold you define) with the SQL and elapsed time
* **Authentication events** -- Log token validation failures and party resolution
Avoid logging contract payloads in production. They may contain private data that should not leave the validator boundary.
## Metrics
### Prometheus Endpoints
Canton nodes expose Prometheus-compatible metrics endpoints. When running cn-quickstart locally, the validator's metrics are available at `http://localhost:10013/metrics` by default.
Key metrics to monitor from your application's perspective:
* `daml_commands_submissions_total` -- Total commands submitted, broken down by status (success, failure)
* `daml_commands_completions_total` -- Completed commands with their result status
* `daml_commands_delayed_submissions` -- Commands that were delayed due to backpressure
* `daml_execution_total` -- Daml interpretation count and duration
* `canton_sequencer_client_submissions_sequencing_time` -- Time from submission to sequencing, which reflects synchronizer latency
### Application-Level Metrics
cn-quickstart includes OpenTelemetry tracing via the `@WithSpan` annotation on service methods. You can export these spans to any OpenTelemetry-compatible backend (Jaeger, Zipkin, Grafana Tempo).
For custom metrics in your backend, use Micrometer (bundled with Spring Boot) or the OpenTelemetry metrics API:
```java theme={"theme":{"light":"github-light","dark":"github-dark"}}
@WithSpan
public CompletableFuture>> listLicenses() {
// The @WithSpan annotation creates a trace span automatically
return damlRepository.findActiveLicenses()
.thenApply(licenses -> ResponseEntity.ok(licenses));
}
```
## Dashboards
### Grafana with cn-quickstart
cn-quickstart ships with pre-configured Grafana dashboards in the `ops/` directory. When you run `make start`, Grafana is available at `http://localhost:3000`.
The bundled dashboards cover:
* **Ledger API overview** -- Command submission rates, latencies, and error rates
* **Canton node health** -- JVM memory, gRPC connection states, sequencer connectivity
* **PQS indexing** -- Lag between the ledger head and the PQS projection offset
### Building Your Own Dashboard
If you add application-specific metrics, create a Grafana dashboard that combines Canton node metrics with your backend metrics. A practical starting layout:
* **Top row** -- Command submission rate and error rate (from the Canton node Prometheus endpoint)
* **Middle row** -- Your application's REST endpoint latencies (from Spring Boot Actuator or OpenTelemetry)
* **Bottom row** -- PQS query latencies and active contract counts for your key templates
### Alerting
Set up alerts for conditions that affect your application's reliability:
* Command error rate exceeding a threshold (e.g., more than 5% of submissions failing)
* PQS indexing lag above a few seconds (queries return stale data)
* Traffic budget dropping below your auto-top-up threshold (transactions will start failing)
* JVM heap usage consistently above 80% on the validator
## Further Reading
* [Backend Development](/appdev/modules/m4-backend-dev) -- Error handling patterns that pair with observability
* [Canton Coin and Traffic](/appdev/modules/m4-canton-coin) -- Monitoring your traffic budget
* [cn-quickstart repository](https://github.com/digital-asset/cn-quickstart) -- Pre-configured observability stack
# Query Contracts and Transactions with PQS
Source: https://docs.canton.network/appdev/modules/m4-query-with-pqs
Use the Participant Query Store (PQS) to query Daml contracts and transactions via SQL.
# How to query contracts and transactions using SQL
## Query
### How to query contracts that are in Active Contract Set (ACS)
To fetch contracts in the ACS use the `active()` table function[^1] which is part of Participant Query Store’s (PQS) schema and can be used as any other PostgreSQL function.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select * from active();
```
```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
----------------------+----------------------------------------------------------------------------------------------
template_fqn | register:DA.Register:IssuerApproval
payload_type | template
create_event_pk | 124095
create_event_id | #12202a593e23a993b63ac897d5e87b4a1ee087bb62cc32d6f5b32ea5001f3d2d40a1:5
created_at_ix | 37524
created_at_offset | 000000000000009343
archive_event_pk |
archive_event_id |
archived_at_ix |
archived_at_offset |
life_ix | [37524,)
contract_id | 0074486cb0469f59b0b5bda4b5794a68cd9350a677ad5e4363d0e4e69b0b925d77ca0212202e62fe6fd912dc18...
payload | {"issue": {...}}
contract_key |
metadata |
created_effective_at | 2025-05-19 03:34:10.929+00
archived_effective_at |
redaction_id |
package_name | register
package_version | 0.0.0
package_id | c9238339098524de2923b702aaf1ea3d832250f05b5930d2fa66c1308590505a
signatories | {issuer-12::12209223396a4c57103512bcc3ff188549d14ecebe084b21f6508989c9caf403998b}
observers | {}
witnesses | {issuer-12::12209223396a4c57103512bcc3ff188549d14ecebe084b21f6508989c9caf403998b}
divulged_only | false
creation_package_id | c9238339098524de2923b702aaf1ea3d832250f05b5930d2fa66c1308590505a
contract_key_hash |
```
### How to query contracts that are in ACS by type
The previous example is very resource-consuming and unadvisable to use except on tiny datasets. A more typical (and efficient at the same time) use case is to limit the result set to a specific Daml template type. The reason for improved efficiency is PostgreSQL’s ability to prune unused partitions when planning the query execution. As long as there is no ambiguity in resolving the type, the following forms are equivalent.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select * from active('register:DA.Register:IssuerApproval');
select * from active('DA.Register:IssuerApproval');
select * from active('IssuerApproval');
```
### How to query contracts that are in ACS by information in payload
To query contracts by business information in the payload, use JSONB operators[^2] in the `WHERE` clause.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select *
from active('register:DA.Register:IssuerApproval')
where payload->'issue'->>'issuer' = 'foo';
```
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select payload->'token'->'wallet'->>'label'
from active('TokenOpen')
where payload->'token'->'issue'->>'issuer' = 'foo'
and (payload->'token'->>'quantity')::decimal > 42;
```
See also `pqs-how-to-add-an-index-on-expression-over-jsonb-payload`.
Querying by payload contents may require additional development and administrative work in order to achieve desired performance outcomes. Please refer to [Performance Optimization](/appdev/deep-dives/performance-optimization) for ideas on how to get started.
### How to join contracts that are in ACS
To join contracts from multiple Daml template types use standard `JOIN` syntax.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select ia.contract_id, ip.contract_id
from active('IssuerApproval') ia
inner join active('IssuerProposal') ip
on ia.payload->>'transferId' = ip.payload->>'transferId';
```
### How to aggregate data from contract values
Since PQS is backed by PostgreSQL database, one can use aggregate functions available as part of Structured Query Language (SQL) syntax.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select payload->'token'->'issue'->>'issuer' as issuer,
sum((payload->'token'->>'quantity')::decimal) as total_quantity
from active('TokenOpen')
group by payload->'token'->'issue'->>'issuer';
```
### How to query state in the past
PQS uses Event Sourcing[^3] architecture and therefore users can peek into the state at an arbitrary point in ledger history by providing an offset explicitly.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select ia.contract_id, ip.contract_id
from active('IssuerApproval', 'some_offset') ia
inner join active('IssuerProposal', 'some_offset') ip
on ia.payload->>'transferId' = ip.payload->>'transferId';
```
### How to fetch a contract by its contract ID
Contracts may be fetched by their contract ID (regardless of their activeness state). Keep in mind, that interface views and contracts share the same contract ID so the function will return interface views as well.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select * from lookup_contract('contract_id');
```
### How to get a list of contracts created in an offset range
To get a sorted list of "contract created" events within a specific offset range.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select *
from creates(from_offset := 'from', to_offset := 'to')
order by created_at_offset, create_event_pk;
```
This query is potentially resource-heavy in the presence of multiple template types and is a poor substitute for Ledger API transaction stream.
### How to get a list of contracts created in an open offset range
By default, offset edges in `creates()` and `archives()` SQL functions represent closed ranges. To make either or both open, use `WHERE` clauses with desired refinements.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select *
from creates(from_offset := 'from', to_offset := 'to')
where created_at_offset > 'from';
```
### How to emulate a stream of "contract created" events with SQL
Although PQS is a poor substitute for Ledger API stream of events, it is sometimes justifiable to get a stream of "contract created" events directly from a PQS database.
To emulate such a stream, one needs to implement client-side busy polling of the database using a similar query.
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select *
from creates(from_offset := 'last_memoized_offset')
where created_at_offset > 'last_memoized_offset'
order by created_at_offset;
```
The client driver should execute this query in a loop, memoizing[^4] the latest observed offset and passing it into the next iteration. It might be a good idea to add sleeping intervals in the loop when no results are returned to minimize resource wasting.
However, if there is a hard requirement on latency and scalability, the best approach is to source ledger events directly from the Ledger API. PQS is designed to be used best as a complementary source to Ledger API’s transaction stream to query ledger *states* at particular offsets, rather than provide event streams.
## Optimize
PQS is backed by PostgreSQL database. Any query optimization is essentially a database/SQL tuning concern. The usual SQL development best practices apply here:
* Apply indexes for frequently used clauses
* Limit data fetching (remove unnecessary columns in final projection, avoid using `SELECT *`)
* Design Daml models to be read-friendly
* Avoid paginating with the help of `OFFSET` SQL clause
* Avoid using queries similar to `SELECT COUNT(*)` if possible
For additional information concerning performance optimization, refer to [Performance Optimization](/appdev/deep-dives/performance-optimization).
### How to add an index on expression over JSONB payload
To optimize read performance of queries that use `WHERE` clauses with information from payload add indexes on expression[^5].
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
call create_index_for_contract(
'token_wallet_holder_idx',
'register:DA.Register:Token',
'(payload->''wallet''->>''holder'')',
'hash'
);
```
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
call create_index_for_contract(
'token_open_wallet_qty_idx',
'register:DA.Register:TokenOpen',
'((payload->''wallet''->>''quantity'')::decimal)',
'btree'
);
```
Provide full expressions used in `WHERE` clauses (including operators and casts).
### How to limit data fetching
List only columns with data required and avoid using `*`.
Most Read API functions return an abundance of data in the result set, including contract’s liveness, divulgence and disclosure metadata. Fetching this data involves joining several internal tables. However, if metadata is not required, the query will cause unnecessary burden on PostgreSQL resources. On the other hand, the query planner in PostgreSQL is smart to prune automatically unnecessary joins[^6].
Bad:
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select * from active('Token');
```
Good:
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select payload->'wallet'->>'holder' as holder,
(payload->'wallet'->>'quantity')::decimal as quantity
from active('Token');
```
### How to paginate efficiently
Make sure your paginated access[^7] to data is efficient and does not cause performance degradation.
Bad:
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select *
from the_source
order by the_key
limit page_size
offset (page_num * page_size);
```
Good:
```sql theme={"theme":{"light":"github-light","dark":"github-dark"}}
select *
from the_source
where the_key > prev_page_last_key
order by the_key
limit page_size;
```
[^1]: [https://www.postgresql.org/docs/current/xfunc-sql.html#XFUNC-SQL-TABLE-FUNCTIONS](https://www.postgresql.org/docs/current/xfunc-sql.html#XFUNC-SQL-TABLE-FUNCTIONS)
[^2]: [https://www.postgresql.org/docs/current/functions-json.html](https://www.postgresql.org/docs/current/functions-json.html)
[^3]: [https://learn.microsoft.com/en-us/azure/architecture/patterns/event-sourcing](https://learn.microsoft.com/en-us/azure/architecture/patterns/event-sourcing)
[^4]: [https://en.wikipedia.org/wiki/Memoization](https://en.wikipedia.org/wiki/Memoization)
[^5]: [https://www.postgresql.org/docs/current/indexes-expressional.html](https://www.postgresql.org/docs/current/indexes-expressional.html)
[^6]: [https://www.oreilly.com/library/view/mastering-postgresql-12/9781838988821/736b4431-b85e-4879-9a93-e5133b42db1f.xhtml](https://www.oreilly.com/library/view/mastering-postgresql-12/9781838988821/736b4431-b85e-4879-9a93-e5133b42db1f.xhtml)
[^7]: [https://www.postgresql.org/docs/current/queries-limit.html](https://www.postgresql.org/docs/current/queries-limit.html)
# SDKs and APIs
Source: https://docs.canton.network/appdev/modules/m4-sdks-apis
Overview of the Canton SDK, code generation, Ledger API, JSON API, PQS, and Wallet SDK
Canton provides a set of tools and APIs for building applications against the ledger. This page covers what each component does and when to use it.
## Daml SDK
The Daml SDK bundles the compiler, code generators, sandbox, and supporting tools you need to develop Canton applications. The main entry point is the `dpm` command-line tool.
Key capabilities of `dpm`:
* `dpm build` -- Compile Daml source code into a DAR file (Daml Archive)
* `dpm codegen-java` -- Generate Java bindings from a compiled DAR
* `dpm codegen-js` -- Generate TypeScript/JavaScript bindings from a compiled DAR
* `dpm sandbox` -- Start a local Canton sandbox node for testing
* `dpm test` -- Run Daml Script tests
* `dpm new` -- Scaffold a new project from a template
`dpm` has additional functionality which can be explored by adding the `--help` option, such as `dpm inspect-dar --help` or `dpm test --help`
The SDK also includes the Daml Studio VS Code extension for syntax highlighting, type checking, and error diagnostics.
## Code Generation
Code generation produces type-safe client bindings from your Daml model. Instead of constructing raw gRPC messages or JSON payloads by hand, you work with generated classes that mirror your templates and choices.
### Java Bindings
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
dpm codegen-java .daml/dist/.dar -o generated-java
```
The generated Java classes integrate with the gRPC Ledger API client. Each Daml template becomes a Java class with methods for creating contracts and exercising choices. The cn-quickstart backend uses these bindings through the codegen capabilities.
### TypeScript Bindings
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
dpm codegen-js .daml/dist/.dar -o generated-ts
```
The generated TypeScript types can be used in a Node.js backend or in a frontend that submits commands directly to the Ledger API. In a fully mediated architecture (like cn-quickstart), the frontend instead uses the backend's REST API and doesn't need these bindings directly.
### Community Bindings
In addition to the officially maintained bindings, see [Community Bindings](/sdks-tools/language-bindings/community) for community-maintained Python, Rust, Go, C#, and other Ledger API clients.
## Ledger API (gRPC)
The Ledger API is the primary interface for submitting commands and reading transactions. It is accessible via the gRPC or JSON API transport exposed by the validator's participant node.
The Ledger API provides:
* **Command submission** -- Create contracts and exercise choices. Commands are submitted as a party acting on the ledger, and the validator processes them through the synchronizer.
* **Transaction stream** -- Subscribe to a stream of committed transactions for one or more parties. Your backend processes these events to keep its state up to date.
* **Active contract set** -- Query the currently active contracts visible to a party.
* **Package management** -- Upload DAR files to a validator.
* **User and party management** -- Create and admin parties on the validator.
The Ledger API uses TLS and token-based authentication in production environments. On LocalNet, authentication can be configured through Keycloak or disabled entirely for faster experimentation.
## JSON API
In Canton 3.x, the JSON API is integrated into the validator. It provides an HTTP/JSON interface to the Ledger API. Architecturally, the JSON API LAPI endpoint translates all of its calls to flow through the gRPC LAPI endpoint.
Use the JSON API when:
* You want simpler HTTP integration without a gRPC client
* You are prototyping and want to test commands with `curl`
* Your language or platform has limited gRPC support
The JSON API supports the same operations as the Ledger API (command submission, active contracts, transaction stream) but through HTTP endpoints and JSON payloads. In cn-quickstart, the JSON API is available on port `x975` for each validator (e.g., `2975` for the App User validator).
For production backends that need very high throughput, the gRPC Ledger API is the better choice because it avoids the serialization overhead of the JSON translation layer.
## Participant Query Store (PQS)
PQS is a service that subscribes to a validator's transaction stream and projects contract data into a PostgreSQL database. Your backend queries this database with standard SQL.
PQS is the right choice when you need:
* Filtered queries across many contracts (e.g., "all licenses expiring this month")
* Aggregations and reporting
* Full-text search or complex joins
* A read path that doesn't load the Ledger API
PQS keeps its PostgreSQL tables synchronized with the ledger. As contracts are created and archived on the ledger, PQS updates the corresponding rows. Your SQL queries always reflect the current ledger state (with a small propagation delay).
In the cn-quickstart project, the backend's `repository/` and `pqs/` modules demonstrate how to query PQS tables for contract data.
## Wallet SDK
The Wallet SDK provides integration with Canton Coin (CC) wallets for applications that need to handle payments or traffic management. Through the wallet integration, your application can:
* Initiate CC transfers between parties
* Create payment requests tied to application workflows (e.g., license renewals)
* Check wallet balances
In cn-quickstart, the licensing application uses wallet integration for license renewal payments. The `LicenseRenewalRequest` contract implements the Splice `AllocationRequest` interface, which the wallet system detects and processes as a payment.
For more on how CC and traffic work, see [Canton Coin and Traffic](/appdev/modules/m4-canton-coin).
## Working Examples
The [cn-quickstart](https://github.com/digital-asset/cn-quickstart) repository demonstrates all of these components working together:
* Daml contracts in `daml/` compiled with `dpm build`
* Java backend in `backend/` using generated bindings and PQS
* React frontend in `frontend/` consuming the backend's REST API
* LocalNet with validators, PQS, JSON API, and wallet services running via Docker Compose
Clone the repository and run `cd quickstart && make setup && make build && make start` to see the full stack in action.
## Next Steps
* [Backend Development](/appdev/modules/m4-backend-dev) -- Patterns for using the Ledger API and PQS in a Java backend
* [Frontend Development](/appdev/modules/m4-frontend-dev) -- Building a React frontend against the backend API
# CI/CD Integration
Source: https://docs.canton.network/appdev/modules/m5-ci-cd-integration
Building CI/CD pipelines for Canton applications with dpm build, test, and deployment automation
A CI/CD pipeline for Canton applications follows the same structure as any other software project: build, package, test, deploy. The Canton-specific parts are the `dpm` commands for compiling Daml, running Script tests, and managing DAR artifacts.
## Pipeline Structure
A typical pipeline has four stages:
1. **Build** — Compile Daml code and generate client bindings
2. **Package** — Produce deployable artifacts (DARs, container images)
3. **Test** — Run Daml Script unit tests and backend integration tests
4. **Deploy** — Upload DARs and deploy off-ledger services to the target environment
## Build Stage
The build stage compiles your Daml packages and generates code bindings for your backend and frontend.
```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Install the SDK version from daml.yaml
`dpm install` <>
# Compile Daml to DAR
dpm build
# Generate Java bindings
dpm codegen-java .daml/dist/<> -o <>
# Generate TypeScript bindings (if needed)
dpm codegen-js .daml/dist/<> -o <