This section was copied from existing reviewed documentation. Source: docs/replicated/daml/3.4/sdk/sdlc-howtos/smart-contracts/upgrade/smart-contract-upgrades.rst Reviewers: Skip this section. Remove markers after final approval.

Smart Contract Upgrade

Overview

What is Smart Contract Upgrade (SCU)?

Smart Contract Upgrade (SCU) allows Daml models (packages in DAR files) to be updated on Canton transparently, provided some guidelines in making the changes are followed. For example, you can fix an application bug by uploading the DAR of the fixed package. Smart Contract Upgrade (SCU) is a feature for Daml packages which enable authors to publish new versions of their templates while maintaining compatibility with prior versions, without any downtime for package users and existing contracts. This feature is well-suited for developing and rolling out incremental template changes. There are guidelines to ensure upgrade compatibility between DAR files. The compatibility is checked at compile time, DAR upload time, and runtime. This is to ensure data backwards upgrade compatibility and forwards compatibility (subject to the guidelines being followed) so that DARs can be safely upgraded to new versions. It also prevents unexpected data loss at clients if a runtime downgrade occurs (e.g., a client is using template version 1.0.0 while the participant node has the newer version 1.1.0). A general guideline for SCU is that additive application changes are allowed but items cannot be removed. A summary of the allowed changes in templates are:

A template can add new Optional fields at the end of the list of fields;
A record datatype can add new optional fields at the end of the list of fields, and a variant or enum datatype can add new constructors at the end;
The ensure predicate can be changed and it is reevaluated at interpretation;
A choice signature can be changed by adding Optional parameters at the end;
The controller of a choice can be changed;
The observer of a choice can be changed
The body of a choice can be changed;
A new choice can be added to a template;
The implementation of an interface instance can be changed;

The following table summarizes the changes supported by SCU. Consult the sections below for additional information.

Daml Model Change	Scope	Covered by SCU?	Comments
Rename of Module	Template Definition	No
Rename of Template	Template Definition	No
Change of Field Type	Template Definition	No
Change of Field Ordering	Template Definition	No
Change of Field Name	Template Definition	No
Change of Signatories	Template Definition	No	A new template version needs to return the same signatory result for a persisted contract, otherwise the contracts cannot be used by an upgraded package (this fails at runtime). One can change the function that computes the signatories (for example to take into account new fields) when creating contracts.
Change of	Template	No	The key’s type cannot be changed, like adding a new field in the record it contains. One can change the function that computes the key in the same way the function computing the signatories can be changed. A persisted contract can be used by an upgrade only if the result of the function does not change.
Change of Observers	Template Definition	No	Same as template signatories
Change of Key Maintainers	Template Definition	No	Same as template signatories
Change of Ensure Predicate	Template Definition	Yes	Note that the Ensure is recalculated just before usage so any changes need to ensure that a V1 contract is still valid as a V2 contract.
Addition of New Definition Fields, Enum, and Variant Constructors	Template Definition	Yes, but they they must be optional
Deletion of Fields from Template	Template Definition	No
Change of Choice Implementation	Template Definition	Yes
Change in Choice Controller	Choice	Yes
Addition of Choice Parameters	Choice	Yes, but they must be optional
Deletion of Choice Parameters	Choice	No
Change of Consuming Choice to Non-Consuming Choice, or Vice-versa	Choice	No
Change of Choice Body	Choice	Yes
Change of Choice Return Type	Choice	Yes, except for Tuple or scalar types	You can change a user data type (Record, Variant, Enum) contained in the return type. In this case you can add a field to the record or a new case to the Variant/Enum. In particular: you cannot change a collection, Optional, List, or Map; you can change Tuple, (to a triple) if you define your own type.
Change of Interface Definition	Interface Definition	No	Interfaces are static and cannot be changed. Separate interface definition and keep the implementation in the template code.
Adding Interfaces to Templates	Template Definition	Yes
Removing Interfaces from Templates	Template Definition	No	Though the implementation can be changed.
Change or Deletion of Exception	Exception Definition	No	Keep the exception definition separate. Exceptions cannot be upgraded.

In this way, package authors can publish new package versions that improve contract functionality without requiring any error-prone migrations, without downtime, without requiring any additional network traffic, and without any extensive communication with downstream users. With SCU, packages may now be referenced by either:

the package ID
the package name.

Package names are used to group versions of a package. Any two packages with the same package name must have distinct package versions. Note:

When defining a Daml package, the name field of the package’s daml.yaml is now used to specify the SCU package name.

The Java and TypeScript codegen allow the use of package name and package ID (if needed). PQS supports using package-name (specified in daml.yaml) instead of package-id. When specifying a template/interface/choice name, simply substitute any package-id with the package-name (eg. now register:DA.Register:Token) instead of the old deadbeefpackageidhex:DA.Register:Token format. This applies to template filters and SQL queries (eg. via the active() function). These functions will always return all versions of a given identifier. Qualified name can be:

fully qualified, e.g. <package-name>:<module-path>:<template-name>
partially qualified, e.g. <module-path>:<template-name>

Qualified names cannot be ambiguous. The PQS Read API now returns the package-name, package-id, and package-version for each contract instance, making it easy for users to determine and inspect different versions over time. To reconstruct the old experience (should you need to) of querying one specific version, use a filter predicate in the SQL.

SELECT *
  FROM active('mypackage:My.App:MyTemplate')
  WHERE package_id = 'deadbeefpackageidhex'

Smart Contract Upgrade Basics

To upgrade a package the package author modifies their existing package to add new functionality, such as new fields and choices. When the new package is uploaded to a participant with the old version, the participant ensures that every modification to the model in the new version is a valid upgrade of the previous version. To be able to automatically upgrade a contract or datatype, SCU restricts the kinds of changes that a new package version can introduce over its prior version. For example, the simplest kind of data transformation that SCU supports is adding a field to a template. Given the following first version of a template:

template IOU
  with
    issuer: Party
    owner: Party
    value: Int
  where
    signatory issuer
    observer owner

You can add a new field for currency:

template IOU
  with
    issuer: Party
    owner: Party
    value: Int
    -- New field:
    currency: Optional String
  where
    signatory issuer
    observer owner

With SCU, any new template fields must be optional - templates from the old version are automatically upgraded to new versions by setting the new field to None. This optional field requirement extends to all records in your package. Conversely, newer contracts with this field set to None can be automatically downgraded to previous versions of the template in workflows that have not yet been updated.

Automatic Data Upgrades and Downgrades

When extending data in a Daml model, SCU requires the old model to be representable in the new model. For extending a record, we can only add nullable (Optional) fields, so that old data can be represented by setting these fields to None. Similar constraints hold for Variants and Enums, which only allow adding constructors, with some other restrictions covered in Continuing to Write Your Upgrades. This approach is inspired by Protobuf and Typescript’s ability to ignore excess fields via as. Automatic data upgrades occur in the following places: Submissions to the Ledger API When you submit a command, and provide only a package-name instead of a package-id, Canton will automatically upgrade (or downgrade) the payloads you give to the most recent version of the package that is uploaded on the participant. It will also use the most recent implementation of any choices you exercise directly through the Ledger API, by automatically upgrading/downgrading the choice argument. Choice result upgrading/downgrading is handled by Consuming Clients, as discussed later in this section. This behavior can be influenced by the package preference. Updates in a choice body When Fetching a contract, the contract payload will be automatically upgraded/downgraded to match the version expected by the calling choice body, as compiled into the DAR. When Exercising a choice on a contract, the contract payload will be upgraded/downgraded to match the version of the choice expected by the calling choice body. This means that in a choice body, an exercised choice argument or return type is never upgraded/downgraded. Consuming Clients (such as Daml Script, TypeScript/Java codegen) When clients query the Ledger API for contracts, the returned event payload format matches the template originally used for generating the event (creating a contract/exercising a choice). It is the responsibility of these clients to upgrade/downgrade the payloads they receive to match what is expected downstream. The same applies to choice results. Daml Script, as well as TypeScript/Java codegen, does this for you to match the Ledger API response to the package versions they were run/built from.

Upgrading Across the Stack

These are all the components that interact with SCU, and how you as a user should be aware that they interacts.

Canton

When considering the Canton ledger nodes, only the Canton participant node is aware of smart contract upgrading. Below, we provide a brief overview of the interactions with the participant node that have been adapted for supporting the smart contract upgrading feature:

DAR upload requests go through an additional validation stage to check the contained new packages for upgrade-compatibility with other packages previously uploaded on the participant.
Ledger API command submissions can be automatically or explicitly up/downgraded if multiple packages exist for the same package-name.
Ledger API streaming queries are adapted to support fetching events more generically, by package-name.

Code Generation

The Java and TypeScript CodeGen have been updated to perform upgrades on retrieved contracts, and now use package-names over package-ids for commands to the participant.

JSON Ledger API

To match the changes to the gRPC Ledger API, the JSON Ledger API similarly supports package-name queries and command submission.

PQS & Daml Shell

PQS only supports querying contracts via package-name, dropping support for direct package-id queries. See the PQS section for more information and a work-around. Daml Shell builds on top of PQS, so inherits this behavior.

Daml Script

Support for SCU is available in the opt-in LTS version of Daml Script. This version acts as a drop-in replacement for the previous Daml Script, and enables support for upgrades on all queries and command submissions. We also expose functions for more advanced interactions with upgrades, as well as to revert to the previous submission behavior.

Daml Compiler

The Daml compiler supports the upgrades: configuration field - every time dpm build is invoked, it validates the current package for upgrade compatibility against the package specified in the upgrades: field. Validation emits custom error messages for common upgrading mistakes, and warns the package author when upgrading a package in a potentially unsafe way. Note however that this validation cannot be complete, as upgrade validity depends on a participant’s package store. The participant’s DAR upload checks have the final say on upgrade validity.

Limitations

To allow SCU to minimize downtime, and multiple versions of a package to be active at once, we limit the types of transformations that can be performed on live data. Following are some data transformations that cannot be made using SCU upgrades:

Renaming, removal, or rearrangement of fields in a template
Conversion of records to variants and vice versa
Moving templates/datatypes to other modules
Upgrading interface and exception definitions
Removing an interface instance from a template

These restrictions are required to give a simple model of runtime upgrades, avoiding ambiguity and non-obvious side effects. If you require any of these types of changes, you may need to consider a redeployment with downtime, either using the approach suggested in Rolling out backwards-incompatible changes. In this version of SCU, the following functionality has not yet been implemented, but may be implemented in future releases.

Daml Script does not support SCU or LF1.17, you must use Daml Script LTS.

There are further limitations with respect to managing the packages on a running ledger:

Once a version of a package is uploaded to the ledger, it cannot be replaced or removed. If a package is uploaded by mistake it can only be overriden by uploading an even newer version.
As a consequence of the above, if a record is extended by mistake with an optional field, that field must be part of all future versions.
Package versions cannot be deleted, they can only be unvetted. As a good rule of thumb the highest version for each package name should be vetted.
If, for whatever reason, a lower version of a package is vetted and preferred, that package version needs to be explicitly mentioned in the command submissions in the specific package_id_selection_preference field of the grpc Commands message.
Participants that confirm and observe a certain transaction must have vetted the package version that is preferred by the submitting participant. If that is not the case, the transaction is rejected.
The contract create events sent as part of the active contracts and transaction streams are always represented according to the template definition in the package version used at time of their creation.

The Programming Model by Example

Writing Your First Smart Contract Upgrade

Setup

We continue with the example introduced in Smart Contract Upgrade Basics. Begin by defining the first (old) version of our package:

> mkdir -p v1/my-pkg
> cd v1/my-pkg
> dpm init
> dpm version
SDK versions:
  3.3.0  (project SDK version from daml.yaml)

Running dpm version should print a line showing that 3.3.0 or higher is the “project SDK version from daml.yaml”. Add daml-script to the list of dependencies in v1/my-pkg/daml.yaml, as well as --target=2.1 to the build-options:

...
name: my-pkg
...
dependencies:
- daml-prim
- daml-stdlib
- daml-script
build-options:
- --target=2.1

Note: Throughout this tutorial we ignore some best practices in favor of simplicity. In particular, we recommend a package undergoing SCU should contain a version identifier in its name as well, but we omit this - consult the section on package naming best practices to learn more. We also recommend that you do not depend on daml-script in any package that is uploaded to the ledger - more on this here. Then create v1/my-pkg/daml/Main.daml:

module Main where

import Daml.Script

template IOU
  with
    issuer: Party
    owner: Party
    value: Int
  where
    signatory issuer
    observer owner
    key issuer : Party
    maintainer key

Running dpm build should successfully produce a DAR in v1/my-pkg/.daml/dist/my-pkg-1.0.0.dar:

> dpm build
Running single package build of my-pkg as no multi-package.yaml was found.
...
Compiling my-pkg to a DAR.
...
Created .daml/dist/my-pkg-1.0.0.dar

Now you can create the second (new) version of this package, which upgrades the first version. Navigate back to the root directory and copy the v1 package into a v2 directory.

> cd ../..
> cp -r v1 v2
> cd v2/my-pkg

Edit the daml.yaml to update the package version, and add the upgrades: field pointing to v1:

version: 1.1.0
...
dependencies:
- daml-prim
- daml-stdlib
- daml-script
upgrades: ../../v1/my-pkg/.daml/dist/my-pkg-1.0.0.dar
build-options:
- --target=2.1

Any changes you make to v2 are now validated as correct upgrades over v1.

Adding a New Field

Begin by adding a new currency field to v2/my-pkg/daml/Main.daml:

...
template IOU
  with
    issuer: Party
    owner: Party
    value: Int
    currency: Text -- New field
  where
...

Run dpm build. An error is emitted:

> dpm build
...
error type checking template Main.IOU :
  The upgraded template IOU has added new fields, but those fields are not Optional.
ERROR: Creation of DAR file failed.

Any new fields added to a template must be optional - old contracts from the previous version are automatically upgraded by setting new fields to None. Fix the currency field to be optional, and re-run dpm build:

...
    currency: Optional Text -- New field
...

> dpm build
...
Created .daml/dist/my-pkg-1.0.0.dar

The build may produce warnings about expression changes - this is covered in the Continuing to Write Your Upgrades section.

Seeing Upgraded Fields in Contracts

Using the Daml Sandbox, we can see our old contracts automatically upgrade. Add a script to make and get IOUs to v1/my-pkg/daml/Main.daml:

module Main where

import Daml.Script
...
mkIOU : Script Party
mkIOU = do
  alice <- allocateParty "alice"
  alice `submit` createCmd (IOU alice alice 1)
  pure alice

getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key

Open v2/my-pkg/daml/Main.daml and add scripts to make IOUs with and without a currency field, and a script to get any IOU:

module Main where

import Daml.Script
...
mkIOU : Script Party
mkIOU = do
  alice <- allocateParty "alice"
  alice `submit` createCmd (IOU alice alice 1 (Some "USD"))
  pure alice

mkIOUWithoutCurrency : Script Party
mkIOUWithoutCurrency = do
  alice <- allocateParty "alice"
  alice `submit` createCmd (IOU alice alice 1 None)
  pure alice

getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key

Start a new terminal, run dpm sandbox to start a simple ledger in which to test upgrades.

> dpm sandbox --json-api-port 7575
Starting Canton sandbox.
Listening at port 6865
Canton sandbox is ready.

Start another terminal, separately from the terminal in which the sandbox is running. From inside v1/my-pkg, upload and run the mkIOU script and place the resulting party for Alice into an output file alice-v1:

> cd v1/my-pkg
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> dpm script \
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.0.0.dar \
    --script-name Main:mkIOU \
    --output-file alice-v1
...

From inside v2/my-pkg, upload and run the getIOU script, passing in the alice-v1 file as the script’s input:

> cd ../../v2/my-pkg
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> dpm script \
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.1.0.dar \
    --script-name Main:getIOU \
    --output-file /dev/stdout \
    --input-file ../../v1/my-pkg/alice-v1
...
{
  "_1": "...",
  "_2": {
  "issuer": "party-...",
  "owner": "party-...",
  "value": 1,
  "currency": null
  }
}
...

The returned contract has a field currency which is set to null. When running the getIOU script from v1, this field does not appear.

> cd ../../v1/my-pkg
> dpm script \
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.0.0.dar \
    --script-name Main:getIOU \
    --output-file /dev/stdout \
    --input-file alice-v1
...
{
  "_1": "...",
  "_2": {
  "issuer": "party-...",
  "owner": "party-...",
  "value": 1
  }
}
...

Downgrading Contracts

New contracts cannot be downgraded if they have a value in their Optional fields. Create a new v2 IOU contract from the v2/my-pkg directory, with USD as currency:

> # Create a new v2 IOU contract, with USD as currency
> cd ../../v2/my-pkg
> dpm script \
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.1.0.dar \
    --script-name Main:mkIOU \
    --output-file alice-v2
...

Query it from a v1 script in the v1/my-pkg directory:

> # Query from v1 package
> cd ../../v1/my-pkg
> dpm script \
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.0.0.dar \
    --script-name Main:getIOU \
    --output-file /dev/stdout \
    --input-file ../../v2/my-pkg/alice-v2
...
Exception in thread "main" com.daml.lf.engine.script.Script$FailedCmd: Command QueryContractKey failed: Failed to translate create argument:
...
An optional contract field with a value of Some may not be dropped during downgrading.

The error states that the optional field may not be dropped during downgrading. Contracts created in a workflow from a v2 package may be used if the optional, upgraded fields are set to None. For example, create an IOU with the currency field set to None using mkIOUWithoutCurrency:

> # Create a new v2 IOU contract, without USD as currency
> cd ../../v2/my-pkg
> dpm script \
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.1.0.dar \
    --script-name Main:mkIOUWithoutCurrency \
    --output-file alice-v2
...

And then query it from v1:

> # Query from v1 package
> cd ../../v1/my-pkg
> dpm script \
  --ledger-host localhost --ledger-port 6865 \
  --dar .daml/dist/my-pkg-1.0.0.dar \
  --script-name Main:getIOU \
  --output-file /dev/stdout \
  --input-file ../../v2/my-pkg/alice-v2
...
  "issuer": "party-...",
  "owner": "party-...",
  "value": 1
...

In this case, the query from v1 succeeded because all upgraded fields are set to None.

Adding a Choice

SCU also allows package authors to add new choices - add the example choice Double to v2/my-pkg/daml/Main.daml, which archives the current contract and produces a new one with twice the value.

...
    maintainer key
    choice Double : ContractId IOU
        controller issuer
        do create this with value = value * 2
...

Along with a script to call it.

import DA.Optional (fromSome)

...

doubleIOU : Party -> Script IOU
doubleIOU alice = do
  oIOU <- getIOU alice
  case oIOU of
    Some (cid, _) -> do
      newCid <- alice `submit` exerciseCmd cid Double
      fromSome <$> queryContractId alice newCid
    None -> fail "Failed to find IOU"

Compiled changes are checked against the previous version and pass:

> dpm build
...
Compiling my-pkg to a DAR.
...
Created .daml/dist/my-pkg-1.1.0.dar
...

Restart the sandbox and re-upload both v1 and v2:

> cd v1/my-deps
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> # Make a new IOU
> dpm script \
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.0.0.dar \
    --script-name Main:mkIOU \
    --output-file alice-v1
...
> cd ../../v2/my-deps
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
...
> dpm script \
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.1.0.dar \
    --script-name Main:doubleIOU \
    --output-file /dev/stdout \
    --input-file ../../v1/my-pkg/alice-v1
...
  "issuer": "party-...",
  "owner": "party-...",
  "value": 2,
  "currency": null
...

Contracts made in v1 can now be exercised with choices introduced in v2. Exercising a v1 choice on a v2 contract is also possible if upgraded fields are set to None, but this requires a different script function -replace the use of exerciseCmd with exerciseExactCmd in the body of doubleIOU in v1, and restart your sandbox.

> # Replace exerciseCmd with exerciseExactCmd in v1
> # Do it using your editor, or use `sed`
> sed -i -E 's/exerciseCmd/exerciseExactCmd/g' \
    v1/my-pkg/daml/Main.daml

Once you’ve restarted your sandbox, create an IOU without a currency in V2 via mkIOUWithoutCurrency, then run doubleIOU on it from V1:

> # Create a new v2 IOU contract, without USD as currency
> cd v2/my-pkg
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> dpm script \
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.1.0.dar \
    --script-name Main:mkIOUWithoutCurrency \
    --output-file alice-v2
> cd ../../v1/my-pkg
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> dpm script \
  --ledger-host localhost --ledger-port 6865 \
  --dar .daml/dist/my-pkg-1.0.0.dar \
  --script-name Main:doubleIOU \
  --output-file /dev/stdout \
  --input-file ../../v2/my-pkg/alice-v2
...
  "issuer": "party-...",
  "owner": "party-...",
  "value": 2
...

Existing choices can also be upgraded, as covered in Continuing to Write Your Upgrades.

Deploying Your First Upgrade

Configuring Canton to Support Smart Upgrading

When using the feature one must be using Protocol Version 7.

Using Smart Contract Upgrading Enabled Packages

Once you have finished development of your smart contract app, use the mentioned upgrade-enabled options in daml.yaml to compile and generate the related DAR. This can be uploaded using the existing gRPC endpoints without modifications and is immediately available for use.

Once a DAR is successfully uploaded, it cannot be safely removed from the participant node. Participant operators must then ensure that uploaded functionality does not break the intended upgrade lineage as newly uploaded DARs affect the upgrading line (i.e. all subsequent uploads must be compatible with this one as well).

Packages stored on the participant must lead to unique package-id -> (package-name, package-version) relationships since runtime package-name -> package-id resolution must be deterministic (see Ledger API). For this reason, once a DAR has been uploaded with its main package having a specific package-name/package-version, this relationship cannot be overridden. Hence, uploading a DAR with different content for the same name/version as an existing DAR on the participant leads to a rejection with error code KNOWN_DAR_VERSION.

Validate the DAR Against a Running Participant Node

You can validate your DAR against the current participant node state without uploading it to the participant via the PackageManagementService.validateDar Ledger API endpoint. This allows participant node operators to first check the DAR before uploading it. This operation is also available via the Canton Admin API and Console:

participant.dars.validate("dars/CantonExamples.dar")

Upgrading and Package Vetting

Upgradable packages are also subject to package vetting restrictions: in to be able to use a package in Daml transactions with smart contract upgrading, it must be vetted by all participants informed about the transaction. This applies to both the packages used for creating the contracts and the target packages. Note: Package vetting is enabled by default on DAR upload operations.

Continuing to Write Your Upgrades

SCU allows package authors to change many more aspects of their packages - fields can be extended in templates, choices, and data type definitions. Choice bodies can be changed, and other expressions such as key definitions and signatory lists can be changed with caveats.

Setup

Continue the package defined in the Writing Your First Upgrade section above, but overwrite the v1 and v2 IOU modules. The v1 IOU module should be overwritten as follows:

module Main where

import Daml.Script

template IOU
  with
    issuer: Party
    owner: Party
    value: Int
  where
    signatory issuer
    observer owner
    key issuer : Party
    maintainer key

mkIOU : Script Party
mkIOU = do
  alice <- allocateParty "alice"
  alice `submit` createCmd (IOU alice alice 1)
  pure alice

getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key

The v2 IOU module should be overwritten to look like the following:

module Main where

import Daml.Script
import DA.Optional (fromOptional)

template IOU
  with
    issuer: Party
    owner: Party
    value: Int
    currency: Optional Text
  where
    signatory issuer
    observer owner
    key issuer : Party
    maintainer key

mkIOU : Script Party
mkIOU = do
  alice <- allocateParty "alice"
  alice `submit` createCmd (IOU alice alice 1 (Some "USD"))
  pure alice

mkIOUWithoutCurrency : Script Party
mkIOUWithoutCurrency = do
  alice <- allocateParty "alice"
  alice `submit` createCmd (IOU alice alice 1 None)
  pure alice

getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key

All other files should remain the same.

Changing Choices

Add the following choice, Duplicate, to both v1 and v2 versions of IOU:

data DuplicateResult = DuplicateResult with
  newCid : ContractId IOU

choice Duplicate : DuplicateResult
  controller issuer
  do
    cid <- create this with value = value * 2
    return DuplicateResult with newCid = cid

Running dpm build should succeed without errors.

> cd v1/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.0.0.dar
> cd ../../v2/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.1.0.dar

Next, upgrade the Duplicate choice by adding an optional field amount, and changing the behavior of the choice to default to a multiple of 3. Also upgrade the DuplicateResult data type to include the old value. Replace the definitions of the DuplicateResult data type and of the Duplicate choice in v2 only:

...
-- Add import to top of module
import DA.Optional (fromOptional)
...
-- Replace DuplicateResult definition
data DuplicateResult = DuplicateResult with
  newCid : ContractId IOU
  oldValue : Optional Int -- New optional oldValue field
...
   -- Replace Duplicate choice implementation
   choice Duplicate : DuplicateResult
     with
       amount : Optional Int -- New optional amount
     controller issuer
     do
       let amt = fromOptional 3 amount
       cid <- create this with value = value * amt
       return DuplicateResult with
         newCid = cid
         oldValue = Some value
...

Add a script called duplicateIOU in V1:

...
duplicateIOU : Party -> Script (Optional (ContractId IOU, IOU))
duplicateIOU key = do
  mbIOU <- getIOU key
  case mbIOU of
    None -> pure None
    Some (iouCid, _) -> do
      res <- key `submit` exerciseExactCmd iouCid Duplicate
      mbNewIOU <- queryContractId key res.newCid
      case mbNewIOU of
        Some newIOU -> pure (Some (newCid, newIOU))
        None -> pure None

A similar script called duplicateIOU should be added in V2, supplying an amount field:

...
duplicateIOU : Party -> Script (Optional (ContractId IOU, Int, IOU))
duplicateIOU key = do
  mbIOU <- getIOU key
  case mbIOU of
    None -> pure None
    Some (iouCid, _) -> do
      res <- key `submit` exerciseExactCmd iouCid Duplicate { amount = Some 4 }
      case res.oldValue of
        None -> pure None -- This should never happen
        Some oldValue ->
          mbNewIOU <- queryContractId key res.newCid
          case mbNewIOU of
            Some newIOU -> pure (Some (newCid, oldValue, newIOU))
            None -> pure None

Running the v1 duplicateIOU script with exerciseExactCmd always runs the v1 implementation for the Duplicate choice, and likewise for v2.

Modifying Signatory Definitions

Other definitions can be changed, but warnings are emitted to remind the developer that the changes can be unsafe and need to be made with care to preserve necessary invariants. Signatories and observers are one expression that can be changed. Importantly, SCU assumes that the new definition does not alter the computed values of the signatories. The computed value of the observers is allowed to change in one specific way: observers that are also signatories can be removed. Any other change to the computed value of the observers (losing a non-signatory observer, adding an observer) is not allowed. For example, add a new field of “outside observers” to the v2 IOU template, and add them to the observer definition.

...
    -- Add a new outsideObservers field
    outsideObservers: Optional [Party]
  where
    signatory issuer
    -- Add outsideObservers to the observer definition
    observer owner, fromOptional [] outsideObservers
...

The new observer definition allows v2 contracts and beyond to add new observers via the outsideObservers field. However, any existing contracts default to None for the outsideObservers field, so all existing contracts have the same observer list as before: the single owner. In the case where a contract’s signatories or observers change in during an upgrade/downgrade in a way that doesn’t meet the constraints above, the upgrade, and thus full transaction, fails at runtime.

Upgrading Enums

Variants and enums can be extended using SCU, either by adding fields to an existing constructor, or by adding a new constructor to the end of the list. Redefine the IOU package, overwriting the v1 and v2 sections similarly to the previous section. Overwrite the IOU package in both V1 and V2 with the following:

module Main where

import Daml.Script

template IOU
  with
    issuer: Party
    owner: Party
    value: Int
    currency: Currency
  where
    signatory issuer
    observer owner
    key issuer : Party
    maintainer key

data Currency
  = USD
  | GBP
  deriving (Show, Eq, Ord)

mkIOU : Script Party
mkIOU = do
  alice <- allocateParty "alice"
  alice `submit` createCmd (IOU alice alice 1 USD)
  pure alice

getIOU : Party -> Script (Optional (ContractId IOU, IOU))
getIOU key = queryContractKey @IOU key key

Instead of using Text for the currency field, here we use an enum data-type Currency with two constructors: USD and GBP. Running dpm build should succeed with no errors:

> cd v1/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.0.0.dar
> cd ../../v2/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.1.0.dar

When you want to extend our contract to support new currencies, you can add new entries to the end of our Currency enum.

...
data Currency
  = USD
  | GBP
  | CHF -- Add a new currency type
  deriving (Show, Eq, Ord)
...

Upgrades of extended enums from an old version to a new version always succeed. In the case of IOUs, a v1 IOU can always be interpreted as a v2 IOU because the constructors for its currency field are a subset of those in a v2 contract. For example, create an IOU with USD via v1’s mkIOU script, and query it via v2’s getIOU script:

> cd v1/my-pkg
> dpm script
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.0.0.dar \
    --script-name Main:mkIOU \
    --output-file alice-v1
...
> cd ../../v2/my-pkg
> dpm script
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.1.0.dar \
    --script-name Main:getIOU \
    --output-file /dev/stdout \
    --input-file ../../v1/my-pkg/alice-v1
...
    "issuer": "party-...",
    "owner": "party-...",
    "value": 1,
    "currency": "USD"
...

Only constructors that are defined in both v1 and v2 can be downgraded from v2 to v1. Any constructor that does not exist in the v1 package fails to downgrade with a runtime error. In the case of our IOU, any CHF fails to downgrade, so any v2 contracts with a CHF currency cannot be used in v1 workflows. For example, create a contract with CHF as its currency field via v2’s mkIOU script. Attempting to query it via v1’s getIOU script fails with a lookup error for the CHF variant.

> cd v2/my-pkg
> dpm script
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.1.0.dar \
    --script-name Main:mkIOU \
    --output-file alice-v2
...
> cd ../../v1/my-pkg
> dpm script
    --ledger-host localhost --ledger-port 6865 \
    --dar .daml/dist/my-pkg-1.0.0.dar \
    --script-name Main:getIOU \
    --output-file /dev/stdout \
    --input-file ../../v2/my-pkg/alice-v2
...
Failed to translate create argument: Lookup(NotFound(DataVariantConstructor(c1543a5c2b7ff03650162e68e03e469d1ecf9f546565d3809cdec2e0255ed902:Main:Currency,CHF),DataEnumConstructor(c1543a5c2b7ff03650162e68e03e469d1ecf9f546565d3809cdec2e0255ed902:Main:Currency,CHF)))
...

Upgrading Variants

Variants, also known as algebraic data types, are very similar to enums except that they also contain structured data. For example, the following variant has two constructors, each with unique fields. Overwrite both v1 and v2 modules with the following source:

module Main where

data Shape
  = Circle
  | Polygon { sides : Int }

You can extend this variant in two ways. You can add a new constructor, similarly to enums. Modify the v2 module to add a new Line constructor with a len field:

module Main where

data Shape
  = Circle
  | Polygon { sides : Int }
  | Line { len : Numeric 10 } -- New line constructor

As before, building should succeed.

> cd v1/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.0.0.dar
> cd ../../v2/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.1.0.dar

You can also add a new field to a constructor, similarly to templates -for example, add a sideLen field to the Polygon constructor, to specify the lengths of the sides of the polygon.

data Shape
  = Circle
  | Polygon
      { sides : Int
      , sideLen : Numeric 10 -- New field
      }
  | Line { len : Numeric 10 }

Building this fails because the new sideLen field is non-optional.

> cd v2/my-pkg
> dpm build
...
error type checking data type Main.Shape:
  The upgraded variant constructor Polygon from variant Shape has added a field.
ERROR: Creation of DAR file failed.

Making the new sideLen field optional fixes the error:

...
      , sideLen : Optional (Numeric 10) -- New field
...

> cd v2/my-pkg
> dpm build
...
Created .daml/dist/my-pkg-1.1.0.dar

Limitations in Upgrading Variants

Upgrading variants has some limitations - because the Circle constructor has been defined without a field in curly braces, it cannot be upgraded with new fields.

...
  -- Add a field where no fields existed
  = Circle { radius : Optional (Numeric 10) }
...

> cd v2/my-pkg
> dpm build
...
error type checking data type Main.Shape:
  The upgraded data type Shape has changed the type of a variant.
ERROR: Creation of DAR file failed.

The same applies to variants with unnamed fields. If the v1 definition of the Line constructor were as follows, it would also not be able to upgrade:

...
  | Line (Numeric 10)

In general, in order to enable future upgrades, it is strongly recommended that all constructors use named fields, and that all constructors have at least one field. If a constructor has no fields in an initial v1 package, one can assign a dummy field. For example, the correct way to write the v1 Circle constructor would be as follows:

...
  = Circle { dummy : () }
...

The subsequent v2 upgrade would succeed:

...
  = Circle { dummy : (), radius : Optional (Numeric 10) }
...

Nested Datatypes

If a data type, choice, or template has a field which refers to another data type, the larger data type can be upgraded if the field’s data type is upgradeable. For example, given the data type A with a field referring to data type B,

data A = A { b : B }
data B = B { field : Text }

If modifications made to B are valid for SCU, then A is also valid.

Dependencies

Package authors may upgrade the dependencies of a package as well as the package itself. A new version of a package may add new dependencies, and must have all the (non-utility-package) dependencies of the old version. If these dependencies are used in ways that are checked for upgrades, each existing dependency must be either unchanged from the old DAR or an upgrade of its previous version. For example, given a dependencies folder containing v1, v2, and v3 of a dependency package dep:

> ls ./dependencies
dep-1.0.0.dar
dep-2.0.0.dar
dep-3.0.0.dar

Then assume a version 1.0.0 of a package main that depends on a datatype from version 2.0.0 of dep:

module Main where

import qualified Dep

data MyData = MyData
  { depData : Dep.AdditionalData
  }

...
dependencies:
- daml-prim
- daml-stdlib
- daml3-script
data-dependencies:
- dependencies/dep-2.0.0.dar
...

Because a package with a newer version may upgrade any dependency to a newer version (or keep the version the same), version 2.0.0 of the main package may keep its dependencies the same, or it may upgrade dep to 3.0.0:

...
dependencies:
- daml-prim
- daml-stdlib
- daml3-script
data-dependencies:
- dependencies/dep-3.0.0.dar # Can upgrade dep-2.0.0 to dep-3.0.0, or leave it unchanged
...

You cannot downgrade a dependency when using that dependency’s datatypes. For example, main may not downgrade dep to version 1.0.0. The following daml.yaml would be invalid:

...
dependencies:
- daml-prim
- daml-stdlib
- daml3-script
data-dependencies:
- dependencies/dep-1.0.0.dar # Cannot downgrade dep-2.0.0 to dep-1.0.0
...

If you try to build this package, the typechecker returns an error on a package ID mismatch for the Dep:AdditionalData field, because the Dep:AdditionalData reference in this case has changed to a package that is not a legitimate upgrade of the original.

> dpm build
...
error type checking data type Main.MyData:
The upgraded data type MyData has changed the types of some of its original fields:
  Field 'depData' changed type from <dep-2.0.0 package ID>:Dep:AdditionalData to <dep-1.0.0 package ID>:Dep:AdditionalData

Upgrading Interface Instances

SCU also supports changing Interface instances. First, create a new package directory my-iface, with my-iface/daml.yaml and module my-iface/daml/MyIface.daml:

sdk-version: 3.3.0
name: my-iface
version: 1.0.0
source: daml/MyIface.daml
parties:
- Alice
- Bob
dependencies:
- daml-prim
- daml-stdlib
build-options:
- --target=2.1

module MyIface where

data HasValueView = HasValueView { hasValueView : Int }

interface HasValue where
  viewtype HasValueView
  getValue : Int

And build the module:

> cd my-iface
> dpm build
...
Created .daml/dist/my-iface-1.0.0.dar

Add references to the newly created DAR in both v1/my-pkg/daml.yaml and v2/my-pkg/daml.yaml:

...
dependencies:
- daml-prim
- daml-stdlib
- daml3-script
- ../../my-iface/.daml/dist/my-iface-1.0.0.dar
...

Overwrite both v1/my-pkg/daml/Main.daml and v2/my-pkg/daml/Main.daml with the following:

module Main where

import Daml.Script
import MyIface

template IOU
  with
    issuer: Party
    owner: Party
    value: Int
  where
    signatory issuer
    observer owner
    key issuer : Party
    maintainer key

    interface instance HasValue for IOU where
      view = HasValueView value
      getValue = value

Interface instances can be changed by an upgrade. For example, v2 can change the definition of getValue in the HasValue instance. Add a quantity field to the v2 IOU package, and amend the definition of getValue to use it:

...
import DA.Optional (fromOptional)

template IOU
  with
    issuer: Party
    owner: Party
    value: Int
    quantity: Optional Int -- new quantity field
  where
...
    interface instance HasValue for IOU where
      view = HasValueView (value * fromOptional 1 quantity)
      -- Use quantity field to calculate value
      getValue = value * fromOptional 1 quantity

Shut down and relaunch the Daml sandbox, then build and upload the two DARs. They should both succeed again:

> cd v1/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> cd ../../v2/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages

Packages with new versions cannot remove an instance that is already there. For example, the v2 IOU template cannot remove its instance of HasValue. Comment out the interface instance for HasValue from v2/my-pkg/daml/Main.daml completely, then restart the sandbox and try to reupload the two versions:

> cd v1/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> cd ../../v2/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
...
Implementation of interface ...:MyIface:HasValue by template IOU appears in package that is being upgraded, but does not appear in this package.

As another example, restore the instance deleted in the previous step and remove the HasValue interface from v2/my-pkg/daml/Main.daml instead. Then restart the sandbox and try to reupload the two versions.

> cd v1/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages
> cd ../../v2/my-pkg
> dpm build
> curl -XPOST -T .daml/dist/my-pkg-1.0.0.dar http://localhost:7575/v2/packages

Similarly to choices, scripts may invoke interface implementations from their own version using exerciseExactCmd.

Upgrading Interfaces

Interface instances may be upgraded, but interface definitions cannot be upgraded. If an interface definition is present in v1 of a package, it must be removed from all subsequent versions of that package. Because interfaces definitions may not be defined in subsequent versions, any package that uses an interface definition from a dependency package can never upgrade that dependency to a new version. For this reason, it is strongly recommended that interfaces always be defined in their own packages separately from templates.

Developer Workflow

This section contains suggestions on how to set up your development environment for SCU to help you iterate more quickly on your projects.

Multi-Package Builds

Following the best practices outlined below and the testing recommendations leads to a proliferation of packages in your project. Use Multi-Package builds to reliably rebuild these packages as you iterate on your project. Multi-package builds also enable cross-package navigation in Daml Studio. The Multi-package builds for upgrades section goes into more detail on how to set up multi-package builds for SCU and how it can help with testing.

Working with a Running Canton Instance

With SCU, it is no longer possible to iterate on a package by uploading it to a running participant after each rebuild. A participant rejects two packages with the same name and version whose content differs. There are two ways to work around this:

restart the participant after each rebuild
change the version name of the package before each rebuild

Environment variable interpolation in DPM can help with the latter. In the daml.yaml file of each of your packages, append an environment variable to the name of the package:

name: my-package-v${UNIQUE_BUILD_ID}

Make sure to also append the variable to the name of the DAR file produced by my-package in the daml.yaml files that depend on it:

dependencies:
- daml-prim
- daml-stdlib
data-dependencies:
- ../my-package/.daml/dist/my-package-${UNIQUE_BUILD_ID}.dar

Then, before invoking dpm build --all, increment the UNIQUE_BUILD_ID environment variable. This ensures that the build produces unique DAR files that can be uploaded to the participant without conflict.

Working with the Daml Sandbox

For the same reason, the Daml sandbox does not support hot-swapping packages. There are two ways to work around this:

restart the sandbox after each rebuild
change the version name of your packages after each rebuild, as outlined in the previous section

The Upgrade Model in Depth - Reference

You can find the in-depth upgrading model, which can be used as a reference for valid upgrades, here.

Package Selection in the Ledger API

Until the introduction of SCU, template IDs in requests to the Ledger API were all of the form <package-id>:<module-name>:<template-name>. For disambiguation, going forward, we refer to this format as by-package-id template IDs. With SCU, we introduce a more generic template reference of the format #<package-name>:<module-name>:<template-name> (by-package-name template ID) that scopes templates by package-name, allowing version-agnostic addressing of templates on the Ledger API. It serves as a reference to all template IDs with the qualified name <module-name>:<template-name> from all packages with the name package-name known to the Ledger API server. By-package-name template ID addressing is supported for all packages and is possible on both the write path (command submission) and read path (Ledger API queries). The by-package-name template ID is an API-level concept of the Ledger API. Internally, Canton uses by-package-id template IDs for all operations. Therefore, when the new format is used in requests to the Ledger API, a dynamic resolution is performed as described in the following sections.

Package Preference

On processing a command submission, there are scenarios where the Ledger API server needs to resolve by-package-name template IDs to by-package-id template IDs (see dynamic-package-resolution-in-command-submission). For this purpose, the package preference concept is introduced as the mapping from package-name to package ID for all known package-names on the participant. The package preference is needed at each command submission time and is assembled from two sources:

The default package preference of the Ledger API server to which the command is submitted. By default, for each package name the Ledger API picks a package version known by all participants involved in the transaction, if it exists.
The package-id selection preference list specified in the submitted command’s package_id_selection_preference in a command submission. This is package-id resolution list explicitly provided by the client to override the default package preference mentioned above.
- See here for how to provide this in Daml-Script
- Note: The package_id_selection_preference must not lead to ambiguous resolutions for package-names, meaning that it must not contain two package-ids pointing to packages with the same package-name, as otherwise the submission will fail with an INVALID_ARGUMENT error.

Dynamic Package Resolution in Command Submission

Dynamic package resolution can happen in two cases during command submission:

For command submissions that use a by-package-name template ID in the command’s templateId field (e.g. in a create command here)
For command submissions whose Daml interpretation requires the execution of interface choices or fetch-by-interface actions.

In these situations, the package preference is used for selecting the target implementation package for these interface actions.

Dynamic Package Resolution in Ledger API Queries

When subscribing for transaction or active contract streams, users can now use the by-package-name template ID format in the template-id request filter field. to specify that they’re interested in fetching events for all templates pertaining to the specified package-name. This template selection set is dynamic and it widens with each uploaded template/package.

Example

Given the following packages existing on the participant node:

Package AppV1
- package-name: app1
- package-version: 1.0.0
- template-ids: pkgId1:mod:T
Package AppV2
- package-name: app1
- package-version: 1.1.0
- template-ids: pkgId2:mod:T

If a transaction query is created with a templateId specified as #app1:mod:T, then the events stream will include events from both template-ids: pkgId1:mod:T and pkgId2:mod:T

Best Practices

To ensure that future upgrades and DAR lifecycling go smoothly, we recommend the following practices:

Separate Interfaces/Exceptions from Templates

Interface and exception definitions are not upgradable. As such, if you attempt to redefine an interface or exception in version 2 of a package, even if it is unchanged, the package does not type check. Removing an interface from the version 2 package also causes issues, especially if the interface has choices. This means that template definitions that exist in the same package as interfaces and exception definitions are not upgradeable. To avoid this issue, move interface and exception definitions into a separate package such that subsequent versions of your template package all depend on the same version of the package with interfaces/exceptions. For example, a single package main defined as follows would not be able to upgrade, leaving the template T non-upgradeable.

module Main where

interface I where
  ...

template T with
  ...

sdk-version: 3.3.0
name: main
version: 1.0.0
source: Main.daml
dependencies:
- daml-prim
- daml-stdlib
build-options:
- --target=2.1

The SCU type checker emits an error and refuses to compile this module:

error type checking <none>:
  This package defines both interfaces and templates. This may make this package and its dependents not upgradeable.

  It is recommended that interfaces are defined in their own package separate from their implementations.
  Downgrade this error to a warning with -Wupgrade-interfaces
  Disable this error entirely with -Wno-upgrade-interfaces

Note: It is very strongly recommended that you do not compile interfaces or exceptions in a package alongside templates. However, you can downgrade this error to a warning by passing the -Wupgrade-interfaces flag, or ignore this error entirely with the -Wno-upgrade-interfaces flag. The recommended way to fix this is to split the main package by redefining it as two packages, helper and main:

module Helper where

interface I where
  ...

sdk-version: 3.3.0
name: helper
version: 1.0.0
source: Helper.daml
dependencies:
- daml-prim
- daml-stdlib
build-options:
- --target=2.1

module Main where

import qualified Helper

template T with
  ...

sdk-version: 3.3.0
name: main
version: 1.0.0
source: Main.daml
dependencies:
- daml-prim
- daml-stdlib
data-dependencies:
- <path to helper DAR>
build-options:
- --target=2.1

To manage the complexity of working with multiple packages at once, we recommend using multi-package builds.

Avoid Contract Metadata Changes

The signatories, stakeholders, contract key, and ensure clauses of a contract should be fixed at runtime for a given contract. Changing their definitions in your Daml code is discouraged and triggers a warning from the SCU typechecker. We strongly recommend against altering the type of a key. If changing the type of a key cannot be avoided, consider using an off-ledger migration instead of SCU.

Breaking Changes via Explicit Package Version

To make a breaking change to your package that is not upgrade compatible, you can change the name of your package to indicate a breaking version bump. To enable this, we recommend that your package name contains a version marker for when a breaking change occurs. For example, for your first iteration of a package, you would name it main-v1, starting with package version 1.0.0. In this case, the v1 is part of the package name, not the package version. You could publish upgrade-compatible versions by changing the version: field from 1.0.0 to 2.0.0 to 3.0.0. These versions would all be upgrade-compatible with one another:

main-v1-1.0.0
main-v1-2.0.0
main-v1-3.0.0

Note how the v1 in all three packages remains stable - this means the package name has not changed, and ensures that these three packages and their datatypes are considered by the runtime and the type checker to be upgradeable. To make a breaking change, publish a new package version with package name main-v2. Because this package has a different package name from those with main-v1, it is not typechecked against those packages and its datatypes are not automatically converted. You would need to manually migrate values from main-v1 packages to main-v2.

Do not depend on Daml Script Packages

We recommend only depending on Daml Script packages such as daml-script in dedicated packages for running tests written in Daml Script. These packages should not be part of your model and should not be uploaded to the ledger. Daml Script packages are not guaranteed to be upgradeable across SDK versions. If you depend on a Daml Script datatype in a serializable position (e.g. the field of a template), your package may rely on a Daml Script package in a way that can neither be removed nor upgraded to the next SDK version. Your package and any of its SCU upgrades would be stuck on that SDK version.

Operational Design Guideline for Upgrading Daml Apps

When considering upgrading, we regard each Daml application as composed of:

On-ledger components: The Daml code running the on-ledger logic (i.e. the DARs uploaded to all participant nodes interacting with the app)
Off-ledger components interacting with the ledger via the Ledger API or other supported Canton Ledger API clients (JSON Ledger API or PQS). These are typically Java or TypeScript services implementing off-ledger business logic.

Zero-Downtime Upgrades

Upgrading an application without operational downtime can be achieved by designing the above-mentioned components to allow:

Asynchronous rollout: Operators deploy updated software at their own pace, similar to established CI/CD practices in micro-service environments.
Synchronous switch-over: All components switch to using updated Daml code at the same time.

Designing for asynchronous upgrade roll-outs To allow for asynchronous roll-outs, off-ledger components should:

use package names in Ledger API requests: App components interacting with the Ledger API should use by-package-name template IDs instead of by-package-id template IDs in all their command submissions and queries. This allows:
- Ledger API server-side version selection of the package preference for command submissions.
- reading all versions of the templates in queries, even newer versions than the one the component was developed against.
handle missing optional fields: App components reading from the Ledger API or Ledger API clients must be prepared to handle missing optional fields in records, including those in the initial package. The TypeScript and Java codegens for reading Daml values do so by default.
use exhaustive package preference: on each command submission, the package_id_selection_preference is set ensuring that:
- The package ID of every package used in the command and of every package used by all possible interface instances is included in the package preference.
- Within an application, all submissions use the same package preference.

Following these recommendations decouples the application’s behavior from the DAR uploads (see the note regarding Package Preference), as package resolution is fully determined by the explicit and exhaustive package preference provided in command submissions.

Operational steps for upgrading Once a new Daml package version of the application is ready, define an operational window for allowing the asynchronous rollout of the updated components. During this window, Canton node and off-ledger app operators should:

Upload the upgraded DARs to the participant nodes
Roll-out the updated off-ledger components

After the operational window passes, the application can switch over to the upgraded logic as such:

A switch-over time is decided and communicated to all app clients in advance, along with the updated package preference pertaining to the application’s upgraded DARs. For example, you may encode the switch-over time in a config value set on all components; or publish it at a well-known HTTP endpoint from which all components read it.
After the switch-over time, all Ledger API clients update their package preference and use it for subsequent command submissions.

Ledger API clients or participant nodes that do not finish the components rollout before the update switch-over time may not be able to participate in the upgraded workflow.

Rolling out backwards-incompatible changes

Some changes to a Daml workflow cannot be made backwards-compatible, such as changing the definition of a template in a breaking way (e.g., extending the observer set). To handle such changes, you can replace the existing contract with an upgraded one of the target template as follows:

Introduce the target template as a new template following the Breaking changes via Explicit Package Version
Add a consuming OriginalTemplate_UpgradeToNewTemplate choice to the existing template by rolling out a new version in a backwards-compatible fashion.
Where required, provide reference data for the upgrade choice via additional choice arguments.
Implement backend automation to migrate all old contracts to the new ones by exercising the upgrade choice on the existing contracts.

Rolling out backwards-incompatible changes as described in this section incurs downtime on affected workflows until their contracts have been converted by the automation. In order to not disrupt business, such rollouts should be executed during maintenance windows.

Note that this kind of upgrade requires O(number-of-active-contracts) of transactions to roll out. Depending on the size of your ACS this can take a long time and consume significant compute and storage resources. In contrast, the backwards-compatible upgrades can be rolled out with constant cost, independent of the size of your ACS.

Testing

Standalone Upgradeability Checks

We recommend using the upgrade-check tool to perform a standalone check that all of the DARs typecheck against one another correctly as further validation of your upgraded packages.

This tool takes a list of DARs and runs Canton’s participant-side upgrade typechecking without spinning up an instance of Canton. You should pass the tool the list of DARs constituting your previous model and the list of DARs for your new model. For example, assume you have a helper package helper that does not change, and two packages main and dep.

main-1.0.0.dar
dep-1.0.0.dar
helper-1.0.0.dar

After upgrading your model, you would publish a new DAR main-2.0.0.dar for main and a new DAR dep-2.0.0.dar for dep. We would then recommend running the upgrade-check tool as follows:

> dpm upgrade-check --participant helper-1.0.0.dar dep-1.0.0.dar main-1.0.0.dar dep-2.0.0.dar main-2.0.0.dar
...

This runs the same upload validation over these DARs that would be run in the event of an upload to the ledger, and prints out the same messages and errors. Because it does not require a ledger to be spun up, the command runs much more quickly. We can also check that all of the DARs pass compiler-side checks, but this is much less likely to indicate an issue because the DARs are typechecked during compilation.

> dpm upgrade-check --compiler helper-1.0.0.dar dep-1.0.0.dar main-1.0.0.dar dep-2.0.0.dar main-2.0.0.dar
...

Dry Run Uploading to a Test Environment

If you have a test environment with DARs that are not available to you, you may not be able to supply a complete list of DARs for your previous model to the standalone dpm upgrade-check tool. For workflows involving multiple DARs, we recommend more robust testing by running a Canton sandbox with the same version and environment as your in-production participant and uploading all the old and new packages that constitute your Daml app. If you want to test the validity of your package against a running ledger, but without uploading the dar, you can hit the validateDar gRPC endpoint

> grpcurl -plaintext -d @ \
  localhost:6865 \
  com.daml.ledger.api.v2.admin.PackageManagementService.ValidateDarFile < helper-1.0.0.dar
...

Daml Script Testing

Daml Script has been used for demonstrative purposes in this document, however usually the complexities of live upgrades comes with your workflows, not the data transformations themselves. You can use Daml Script (with Canton) to test some isolated simple cases, but for thorough tests of you system using SCU, you should prefer full Workflow Testing. We recommend placing your Daml Script tests in a separate package which depends on all versions of your business logic when testing your upgrades with Daml Script. This testing package should not be uploaded to the ledger if possible, as it depends on the daml-script package. This package emits a warning on the participant when uploaded, as it serves no purpose on a participant, cannot be fully removed (as with any package), and may not be uploadable to the ledger in future versions (Daml 3). More information about this limitation here. Depending on multiple versions of the same package does however face ambiguity issues with imports. You can resolve these issues using module prefixes:

name: my-testing-package
...
data-dependencies:
  - my/v1/main-package.dar
  - my/v2/main-package.dar
module-prefixes:
  main-1.0.0.dar: V11
  main-1.1.0.dar: V12

It is important to verify old workflows are still functional under new data and new implementation when writing your tests. You also need to verify that any new workflows intended to be backward compatible can consume old data. You should build your testing structure to cover this how you see fit, but we give the following recommendation as a starting point: If your new version only includes choice body or interface instance changes (that is, it is a patch release)

Run your existing test suite for V1 but updated to call V2 choices. This can be done with a rewrite, or by passing down a package preference and calling the test with both the V1 and V2 package ID.

If your new version includes data changes, be that to contract payloads or choices (that is, it is a minor release)

Assuming your data change affects a template payload, write separate setup code for V1 and V2, populating new fields setupV1 : Script V1TestData setupV2 : Script V2TestData These new data types should mostly just hold Contract IDs
Update your existing test suite from V1 to take a package preference, allowing the V2 implementation without additional fields to choices to be called. testV1 : [PackageId] -> V1TestData -> Script ()
Run the above test suite against V1 data, passing a V1 preference, then a V2 preference. This ensures your changes haven’t broken any existing workflows.
Next write tests for your new/modified workflows, using the V2 choice implementations. This does not need a package preference. testV2 : V2TestData -> Script ()
Run these tests against both the V1 setup and the V2 setup, to ensure your new workflows support existing/old templates. In order to do this, you’ll need some way to upcast your V1TestData, i.e. upcastTestData : V1TestData -> V2TestData This function should mostly just call coerceContractId on any contract IDs, and fill in any None values if needed.
Finally, you can cover any workflows that require the contract data to already be upgraded (new fields populated), these are written entirely in V2 without any special considerations.

Multi-package builds for upgrades

When you are developing upgrades, you may have multiple DARs in scope that need to be built together. Tracking these DARs and building them in the right order can be complicated, especially as you develop live and as the project grows.

Multi-package builds help with projects containing multiple DARs, for example, a project using upgrades. To understand how multi-package builds simplify the development of a project using upgrades, begin by creating a new Daml project with the upgrades-example template.

> dpm new upgraded-iou --template upgrades-example
> cd upgraded-iou
> tree
.
├── multi-package.yaml
├── run-test.sh
├── interfaces
│   ├── daml/...
│   └── daml.yaml
├── main-v1
│   ├── daml/...
│   └── daml.yaml
├── main-v2
│   ├── daml/...
│   └── daml.yaml
└── test
    ├── daml/...
    └── daml.yaml

The example template contains:

A package upgraded-iou-interfaces, which defines an interface Asset and a viewtype Asset.View.
The first version (1.0.0) of a package upgraded-iou-main, which defines a template IOU with instance of upgraded-iou-interfaces:Main.Asset.
The second version (2.0.0) of upgraded-iou-main which upgrades the first. It adds a new description field to IOU, and uses it (when the field is defined) in an upgraded implementation of Asset.
A testing package upgraded-iou-test, which depends on both upgraded-iou-main-1.0.0 and upgraded-iou-main-2.0.0. It defines a script which exercises v1.0.0 and v2.0.0 IOUs via their Asset interface.
A script run-test.sh, which runs the main test in upgraded-iou-test.
A multi-package.yaml file which lists our four packages.

Without multi-package builds you would test your program like this:

> # Run sandbox in the background, wait until the three lines below are shown
> dpm sandbox &
Starting Canton sandbox.
Listening at port 6865
Canton sandbox is ready.
> # Build all, run test
> cd interfaces/; dpm build --enable-multi-package=no
> cd ../main-v1/; dpm build --enable-multi-package=no
> cd ../main-v2/; dpm build --enable-multi-package=no
> cd ../test/; dpm build --enable-multi-package=no
> cd ..
> ./run-test.sh
> # Modify v2, run test
> cd main-v2/
> ... modify main-v2 ...
> dpm build --enable-multi-package=no
> cd ../test/; dpm build --enable-multi-package=no
> cd ..
> # Modify test, run test
> cd test/
> dpm build --enable-multi-package=no
> cd ..
> ./run-test.sh
...
Test output:
...
> kill %1 # Kill backgrounded sandbox process

Forgetting to rebuild packages after changing their source would not cause a failure - for example, if you modified the source from main-v2 in an incompatible way but did not recompile it, the test package would still compile successfully against the previous DAR for main-v2.

> # Modify main-v2 in an incompatible way
> cd main-v2/
> ... add a non-optional field `currency: Text` to template T in main-v2 ...
> cd ../test/
> dpm build --enable-multi-package=no
...
Created .daml/dist/upgraded-iou-upgrades-template-test-1.0.0.dar
> # Compiling `test` succeeded even though main-v2 was changed incorrectly

With Daml multi-package builds, all builds automatically rebuild dependencies if their source has changed:

> cd test/
> dpm build # --enable-multi-package is set to true by default
...
Building /home/dylanthinnes/ex-upgrades-template/main-v2
...
Severity: DsError
Message:
error type checking template Main.IOU :
  The upgraded template IOU has added new fields, but the following new fields are not Optional:
    Field 'currency' with type Text
...
> # Compiling `test` failed as expected because main-v2 was changed incorrectly

The ./run-test.sh script automatically rebuilds all DARs in the package that need to be rebuilt:

> dpm sandbox & # Start sandbox in background
Starting Canton sandbox.
Listening at port 6865
Canton sandbox is ready.
> ... Fix main-v2 by dropping non-optional `currency` field ...
> # Re-run test
> ./run-test.sh
...
Building /home/dylanthinnes/ex-upgrades-template/main-v2
...
> # Modify test, run test
> ... modify test ...
> dpm build --all
> ./run-test.sh

Multi-package builds invoked by dpm build --all always recompile stale dependencies and DARs in order. This ensures a fully up-to-date package environment before running ./run-test.sh.

Workflow Testing

While testing your workflows is application-specific, we still recommend at least one test for your core workflows that follows this pattern:

Start your app using version 2.0 of your DAR, but only upload version 1.0.
Initialize the app and start one instance of every core workflow.
Upload version 2.0 of your DAR.
Switch your backends to start using version 2.0, ideally this should be a flag.
Validate that the core workflows are in the same state and advance them to check that they are not stuck.

SCU Support in Daml Tooling

Codegen

Generated code uses package names in place of package IDs in template IDs. Retrieved data from the ledger is subject to the upgrade transformations described in previous sections. Concretely, this is implemented as follows:

Java

The classes that are generated for each template and interface contain a TEMPLATE_ID field, which, for upgradable packages, now use a package name rather than a package ID. To help you determine the package ID of these packages, we have added a new PACKAGE_ID field to all such classes. Upgradable packages also have PACKAGE_NAME and PACKAGE_VERSION fields. If you need to identify a template by the specific package ID of the DAR from which the code was generated, you can use the TEMPLATE_ID_WITH_PACKAGE_ID field, which is on all generated classes and their companion objects. When submitting commands you may also use the packageIdSelectionPreference to explicitly specify which package ID(s) you want the ledger to use to interpret the commands. Note that using these options prevents your application from seamlessly supporting upgrades.

TypeScript

The templateId field on generated template classes has been updated to use the package name as the package qualifier for upgrade-compatible packages. This is used for command submission and queries. However, note that queries return the package qualifier with the package ID rather than the package name. Generated modules now also give the package “reference”, which is the package name for upgrade-compatible packages; for other packages it is the package ID. To perform package ID-qualified commands/queries in an upgrade compatible package, create a copy of the template object using the following:

const MyTemplateWithPackageId = {
  ...pkg.Mod.MyTemplate,
  templateId: `${pkg.packageId}:Mod:MyTemplate`,
}

JSON Ledger API

Packages can be addressed both by package ID and by package name in the interaction with the JSON Ledger API. Note: template IDs in query results always use a package ID. This allows you to distinguish the source of a particular contract. This means that if you use a template with a package name in the request, you can no longer expect the template IDs in the result to exactly match the input template ID. Package ID selection preference: preferences apply to JSON Ledger API where you can specify your preferred selection of package versions.

PQS

To match the package-name changes to the Ledger API, PQS has changed how packages are selected for queries. All queries that take a Daml identity in the form <package-id>:<module-name>:<template-name> now take a package name in place of package ID. Note that this differs from the Ledger API in that the \# prefix is not required for PQS, as PQS has dropped direct package ID queries. Queries for package names return all versions of a given contract, alongside the package version and package ID for each contract.

If you still need to perform a query with an explicit package ID, you can either use a previous version of PQS or add the following filter predicate to your query: SELECT \* FROM active('my_package:My.App:MyTemplate') WHERE package_id = 'my_package_id'

Given that PQS uses a document-oriented model for ledger content (JSONB), extensions to contract payloads are handled simply by returning the additional data in the blob.

Daml Shell

Daml Shell builds on PQS by providing a shell interface to inspect the ledger using package name to create an integrated view of all versions of contracts.

Daml-Script

All commands and queries in this version of Daml Script now use upgrades/downgrades automatically to ensure that they exercise the correct versions of choices and return correct payloads. The following additional functionality is available for more advanced uses of SCU. Exact commands Each of the four submission commands now has an “exact” variant: createExactCmd, exerciseExactCmd, exerciseByKeyExactCmd, and createAndExerciseExactCmd. These commands force the participant to use the exact version of the package that your script uses, so you can be certain of the choice code you are calling. Note that exact and non-exact commands can be mixed in the same submission.

Daml Script Package Preference

A submission can specify a package preference as a list of package IDs:

(actAs alice <> packagePreference [myPackageId]) `submitWithOptions` createCmd ...

Note the use of submitWithOptions : SubmitOptions -> Commands a -> Script a. You can build SubmitOptions by combining the actAs and packagePreference functions with <>. The full list of builders for SubmitOptions is as follows:

-- `p` can be `Party`, `[Party]`, etc.
actAs : IsParties p => p -> SubmiOptions
readAs : IsParties p => p -> SubmitOptions

disclose : Disclosure -> SubmitOptions
discloseMany : [Disclosure] -> SubmitOptions

newtype PackageId = PackageId Text
packagePreference : [PackageId] -> SubmitOptions

A PackageId can be hard-coded in your script, in which case it must be updated whenever the package changes. Otherwise, it can be provided using the --input-file flag of the dpm script command line tool. The following example demonstrates reading the package ID from a DAR and passing it to a script:

# Path to the dar you want to pass as package preference.
PACKAGE_DAR=path/to/main/dar.dar
# Path to the dar containing the Daml script for which you want to pass the package-id
SCRIPT_DAR=path/to/script/dar.dar
# Extract the package-id of PACKAGE_DAR's main package.
dpm damlc inspect-dar ${PACKAGE_DAR} --json | jq '.main_package_id' > ./package-id-script-input.json
# replace --ide-ledger with --ledger-host and --ledger-port for deployed Canton
dpm script --dar ${SCRIPT_DAR} --script-name Main:main --ide-ledger --input-file ./package-id-script-input.json

Following this, your script would look like:

module Main where

import Daml.Script

main : Text -> Script ()
main rawPkgId = do
  let pkgId = PackageId rawPkgId
  alice <- allocateParty "alice"
  -- Commands omitted for brevity
  let submitOptions = actAs alice <> packagePreference [pkgId]
  submitOptions `submitWithOptions` createCmd ...

Daml Studio support

Daml Studio runs a reference model of Canton called the IDE Ledger. This ledger has been updated to support the relevant parts of the Smart Contract Upgrades feature.

Get Started

Module 1: Understanding Canton

Module 2: Canton for Ethereum Developers

Module 3: Daml Smart Contracts

Module 4: Full-Stack Application Development

Module 5: Testing & Deployment

Module 6: Smart Contract Upgrades

Module 7: Production Best Practices

Deep Dives

Tooling

App Rewards

Troubleshooting

Reference

Documentation Index

​Smart Contract Upgrade

​Overview

​What is Smart Contract Upgrade (SCU)?

​Smart Contract Upgrade Basics

​Automatic Data Upgrades and Downgrades

​Upgrading Across the Stack

​Canton

​Code Generation

​JSON Ledger API

​PQS & Daml Shell

​Daml Script

​Daml Compiler

​Limitations

​The Programming Model by Example

​Writing Your First Smart Contract Upgrade

​Setup

​Adding a New Field

​Seeing Upgraded Fields in Contracts

​Downgrading Contracts

​Adding a Choice

​Deploying Your First Upgrade

​Configuring Canton to Support Smart Upgrading

​Using Smart Contract Upgrading Enabled Packages

Validate the DAR Against a Running Participant Node

Upgrading and Package Vetting

​Continuing to Write Your Upgrades

​Setup

​Changing Choices

​Modifying Signatory Definitions

​Upgrading Enums

​Upgrading Variants

​Limitations in Upgrading Variants

​Nested Datatypes

​Dependencies

​Upgrading Interface Instances

​Upgrading Interfaces

​Developer Workflow

​Multi-Package Builds

​Working with a Running Canton Instance

​Working with the Daml Sandbox

​The Upgrade Model in Depth - Reference

​Package Selection in the Ledger API

​Package Preference

​Dynamic Package Resolution in Command Submission

​Dynamic Package Resolution in Ledger API Queries

​Example

​Best Practices

​Separate Interfaces/Exceptions from Templates

​Avoid Contract Metadata Changes

​Breaking Changes via Explicit Package Version

​Do not depend on Daml Script Packages

​Operational Design Guideline for Upgrading Daml Apps

​Zero-Downtime Upgrades

​Rolling out backwards-incompatible changes

​Testing

​Standalone Upgradeability Checks

​Dry Run Uploading to a Test Environment

​Daml Script Testing

​Multi-package builds for upgrades

​Workflow Testing

​SCU Support in Daml Tooling

​Codegen

​Java

​TypeScript

​JSON Ledger API

​PQS

Smart Contract Upgrade

Overview

What is Smart Contract Upgrade (SCU)?

Smart Contract Upgrade Basics

Automatic Data Upgrades and Downgrades

Upgrading Across the Stack

Canton

Code Generation

JSON Ledger API

PQS & Daml Shell

Daml Script

Daml Compiler

Limitations

The Programming Model by Example

Writing Your First Smart Contract Upgrade

Setup

Adding a New Field

Seeing Upgraded Fields in Contracts

Downgrading Contracts

Adding a Choice

Deploying Your First Upgrade

Configuring Canton to Support Smart Upgrading

Using Smart Contract Upgrading Enabled Packages

Continuing to Write Your Upgrades

Setup

Changing Choices

Modifying Signatory Definitions

Upgrading Enums

Upgrading Variants

Limitations in Upgrading Variants

Nested Datatypes

Dependencies

Upgrading Interface Instances

Upgrading Interfaces

Developer Workflow

Multi-Package Builds

Working with a Running Canton Instance

Working with the Daml Sandbox

The Upgrade Model in Depth - Reference

Package Selection in the Ledger API

Package Preference

Dynamic Package Resolution in Command Submission

Dynamic Package Resolution in Ledger API Queries

Example

Best Practices

Separate Interfaces/Exceptions from Templates

Avoid Contract Metadata Changes

Breaking Changes via Explicit Package Version

Do not depend on Daml Script Packages

Operational Design Guideline for Upgrading Daml Apps

Zero-Downtime Upgrades

Rolling out backwards-incompatible changes

Testing

Standalone Upgradeability Checks

Dry Run Uploading to a Test Environment

Daml Script Testing

Multi-package builds for upgrades

Workflow Testing

SCU Support in Daml Tooling

Codegen

Java

TypeScript

JSON Ledger API

PQS

Daml Shell

Daml-Script

Daml Studio support