Ledger API overview
What is the Ledger API
The Ledger API is an API that’s exposed by any Participant Node. Users access and manipulate the ledger state through the Ledger API. There are two protocols available for the Ledger API: gRPC and JSON. The core services are implemented using gRPC and Protobuf. There is a translation layer that additionally exposes all the Ledger API services as JSON Ledger API. For a high level introduction to the services seeledger-api-services.
You may also want to read the protobuf documentation of the API, which explains how each service is defined through its protobuf messages.
How to Access the Ledger API
You can access the gRPC Ledger API via the Java bindings. If you don’t use a language that targets the JVM, you can use gRPC to generate the code to access the Ledger API in several programming languages that support theproto standard.
If you don’t want to use the gRPC API, you can also use the JSON Ledger API. This API is formally described using an openapi and asyncapi descriptions. You can use any language that supports OpenAPI standard to generate the clients using the OpenAPI Generators.
Daml-LF
When you compile Daml source into a .dar file, the underlying format is Daml-LF. Daml-LF is similar to Daml, but is stripped down to a core set of features. The relationship between the surface Daml syntax and Daml-LF is loosely similar to that between Java and JVM bytecode. As a user, you don’t need to interact with Daml-LF directly. But internally, it’s used for:- Executing Daml code on the Ledger
- Sending and receiving values via the Ledger API
- Generating code in other languages for interacting with Daml models (often called “codegen”)
When You Need to Know About Daml-LF
Daml-LF is only really relevant when you’re dealing with the objects you send to or receive from the ledger. If you use any of the provided code generators, you don’t need to know about Daml-LF at all. Otherwise, it can be helpful to know what the types in your Daml code look like at the Daml-LF level, so you know what to expect from the Ledger API. For example, if you are writing an application that creates some Daml contracts, you need to construct values to pass as parameters to the contract. These values are determined by the Daml-LF types in that contract template. This means you need an idea of how the Daml-LF types correspond to the types in the original Daml model. For the most part the translation of types from Daml to Daml-LF should not be surprising. This page goes through all the cases in detail.Daml-LF JSON encoding
Daml-LF values are represented as JSON values in the JSON Ledger API. This representation is used whenever theJSON Ledger API expects a Daml record or value, such as in the createArguments field of the CreateCommand component. The conversion from Daml values to JSON also occurs when a value is returned from the JSON Ledger API. For more information about Daml types, see daml-ref-built-in-types.
Strings are preferred for
Int and Decimal values because many languages, most notably JavaScript, use IEEE doubles for JSON numbers and cannot precisely represent every Daml-LF value. For convenience, parsing accepts both JSON numbers and strings.Timestamp parsing is flexible and accepts
yyyy-mm-ddThh:mm:ss(.s+)?Z. Sub-second precision beyond microseconds is dropped. The UTC timezone designator must be included to reduce the risk of accidentally passing local times.How Daml Types are translated to Daml-LF
This page shows how types in Daml are translated into Daml-LF. It should help you understand and predict the generated client interfaces, which is useful when you’re building a Daml-based application that uses the Ledger API or client bindings in other languages.Primitive types
Built-in data types in Daml have straightforward mappings to Daml-LF. This section only covers the serializable types, as these are what client applications can interact with via the generated Daml-LF. (Serializable types are ones whose values can exist on the ledger. Function types,Update and Scenario types and any types built up from these are excluded, and there are several other restrictions.)
Most built-in types have the same name in Daml-LF as in Daml. These are the exact mappings:
Be aware that only the Daml primitive types exported by the Prelude module map to the Daml-LF primitive types above. That means that, if you define your own type named
Party, it will not translate to the Daml-LF primitive Party.
Tuple types
Daml tuple type constructors take typesT1, T2, …, TN to the type (T1, T2, …, TN). These are exposed in the Daml surface language through the Prelude module.
The equivalent Daml-LF type constructors are daml-prim:DA.Types:TupleN, for each particular N (where 2 <= N <= 20). This qualified name refers to the package name (ghc-prim) and the module name (GHC.Tuple).
For example: the Daml pair type (Int, Text) is translated to daml-prim:DA.Types:Tuple2 Int64 Text.
Data Types
Daml-LF has three kinds of data declarations:- Record types, which define a collection of data
- Variant or sum types, which define a number of alternatives
- Enum, which defines simplified sum types without type parameters nor argument.
data keyword) are translated to record, variant or enum types. It’s sometimes not obvious what they will be translated to, so this section lists many examples of data types in Daml and their translations in Daml-LF.
Record declarations
This section uses the syntax for Daml records with curly braces.Variant declarations
Enum declarations
Banned declarations
There are two gotchas to be aware of: things you might expect to be able to do in Daml that you can’t because of Daml-LF. The first: a single constructor data type must be made unambiguous as to whether it is a record or a variant type. Concretely, the data type declarationdata Foo = Foo causes a compile-time error, because it is unclear whether it is declaring a record or a variant type.
To fix this, you must make the distinction explicitly. Write data Foo = Foo {} to declare a record type with no fields, or data Foo = Foo () for a variant with a single constructor taking unit argument.
The second gotcha is that a constructor in a data type declaration can have at most one unlabelled argument type. This restriction is so that we can provide a straight-forward encoding of Daml-LF types in a variety of client languages.
Restrictions for upgrades
The flavour of a datatype’s Daml-LF representation restricts the ways in which it can be upgraded via smart contract upgrades: only records can add fields, and only variants and enums can add new constructors. It is not possible to change the flavour of a datatype once it has been chosen. Therefore, the ideal choice of the flavour of a datatype strongly depends on what upgrade behaviours are planned for it. For example, the following datatype:bar3 and a constructor Bat:
Type synonyms
Type synonyms (starting with thetype keyword) are eliminated during conversion to Daml-LF. The body of the type synonym is inlined for all occurrences of the type synonym name.
For example, consider the following Daml type declarations.
Username type is eliminated in the Daml-LF translation, as follows:
Template Types
A template declaration in Daml results in one or more data type declarations behind the scenes. These data types, detailed in this section, are not written explicitly in the Daml program but are created by the compiler. They are translated to Daml-LF using the same rules as for record declarations above. These declarations are all at the top level of the module in which the template is defined.Template Data Types
Every contract template defines a record type for the parameters of the contract. For example, the template declaration:Choice Data Types
Every choice within a contract template results in a record type for the parameters of that choice. For example, let’s suppose the earlierIou template has the following choices:
Names with Special Characters
All names in Daml—of types, templates, choices, fields, and variant data constructors—are translated to the more restrictive rules of Daml-LF. ASCII letters, digits, and_ underscore are unchanged in Daml-LF; all other characters must be mangled in some way, as follows:
$changes to$$,- Unicode codepoints less than 65536 translate to
$uABCD, whereABCDare exactly four (zero-padded) hexadecimal digits of the codepoint in question, using only lowercasea-f, and - Unicode codepoints greater translate to
$UABCD1234, whereABCD1234are exactly eight (zero-padded) hexadecimal digits of the codepoint in question, with the samea-frule.
How Daml Types are Translated to Protobuf
This page gives an overview and reference on how Daml types and contracts are represented by the gRPC Ledger API as protobuf messages, most notably:- in the stream of transactions from the
com.daml.ledger.api.v1.transactionservice - as payload for
com.daml.ledger.api.v1.createcommandandcom.daml.ledger.api.v1.exercisecommandsent tocom.daml.ledger.api.v1.commandsubmissionserviceandcom.daml.ledger.api.v1.commandservice.
Notation
The notation used on this page for the protobuf messages is the same as you get if you invokeprotoc --decode=Foo < some_payload.bin. To illustrate the notation, here is a simple definition of the messages Foo and Bar:
Foo is then represented by the Ledger API in this way:
Records and Primitive Types
Records or product types are translated tocom.daml.ledger.api.v1.record. Here’s an example Daml record type that contains a field for each primitive type:
MyProductType. See Contract templates for the translation of Daml contracts to the representation by the Ledger API.
Variants
Variants or sum types are types with multiple constructors. This example defines a simple variant type with two constructors:MyConstructor1 takes a single parameter of type Integer, whereas the constructor MyConstructor2 takes a tuple with two fields as parameter. The snippet below shows how you can create values with either of the constructors.
mySum1 and mySum2 respectively as they would be transmitted on the Ledger API within a contract.
Contract Templates
Contract templates are represented as records with the same identifier as the template. This first example template below contains only the signatory party and a simple choice to exercise:Create a Contract
Creating contracts is done by sending acom.daml.ledger.api.v1.createcommand to the com.daml.ledger.api.v1.commandsubmissionservice or the com.daml.ledger.api.v1.commandservice. The message to create a MySimpleTemplate contract with Alice being the owner is shown below:
Receive a Contract
Contracts are received from thecom.daml.ledger.api.v1.transactionservice in the form of a com.daml.ledger.api.v1.createdevent. The data contained in the event corresponds to the data that was used to create the contract.
Exercise a Choice
A choice is exercised by sending ancom.daml.ledger.api.v1.exercisecommand. Taking the same contract template again, exercising the choice MyChoice would result in a command similar to the following:
com.daml.ledger.api.v1.exercisebykeycommand can be used. It works in a similar way as com.daml.ledger.api.v1.exercisecommand, but instead of specifying the contract identifier you have to provide its key. The example above could be rewritten as follows:
Daml packages and archive (.dar) files
When a Daml package is compiled, it is packed into a final artifact called a DAR (.dar) file. The purpose of this DAR file is to contain all of the necessary code and logic to run the package’s templates, without requiring any other files.
For example, assume a simple package mypkg with a single dependency dep:
dpm build compiles it and reports the path of the resulting DAR as the last line:
Structure of an archive file
A DAR is actually a zip file which contains many different files, all of which work together to provide a ledger everything it needs to know in order to run the code it was compiled from..dalf). Each .dalf file contains the entire compiled code for a specific package.
One of the DALF files will be the “main” or “primary” package that the DAR was compiled from - this DALF will contain the definitions of templates, interfaces, datatypes, and functions that were originally described in the Daml code that the DAR was compiled from. In this case, that is the mypkg-1.0.0-<mypkg-package-id>.dalf file listed above.
All of the other DALF files will be for dependency packages of that “main” package, which are required to run the package. This includes the dep-1.0.0-<dep-package-id>.dalf file, as well as many DALF files for the daml-prim and daml-stdlib libraries.
Aside from these files, there will be:
- A
MANIFEST.MFfile, which contains metadata about the rest of the artifacts in the DAR.- the name of the “main” package that was compiled into the DAR
- a list of all of the dependencies of the main package
- some more metadata about the package
- The source code (
.daml) for the primary package. This can be used by consumers of the DAR to verify that the DALF they’re running corresponds to the code inside of it. It is also used by Daml Studio for code intelligence such as jump-to-definition when the DAR is included as a dependency of another project. - Interface files (
.hi,.hie,.conf) for the primary package. This is also used by Daml Studio to provide jump-to-definition.
Difference between DALF files and Daml files
A common question is why DAR files contain DALF files - why don’t they just contain all of the source code for all of the packages directly? To understand why, it is important to understand the distinction between Daml and DALF files:- Daml files contain the Daml source code that Daml developers write. Daml source code is human-writable and human-readable, and is readable as a general purpose programming language.
- DALF files, on the other hand contain a compact, binary-encoded representation of Daml-LF. Daml-LF is very restricted, comparatively simple computer-executable programming language. Daml-LF is not intended to be human-readable nor human-writable, it is intended to be deterministic, fast to execute, and secure.
DARs as dependencies
When a new project needs to depend on a different package, the DAR that the package was compiled to is supplied as a data-dependency in the new project’sdaml.yaml.
For example, suppose a new package next-project that uses the mypkg package as a dependency:
mypkg-1.0.0 DAR, finds its primary package, and exposes that as a dependency to code inside next-project. When next-project is compiled, it retains all of the DALF files inside the mypkg DAR, including the mypkg package’s dependencies.
In general, any time a DAR is compiled for a package that has further DAR dependencies, those DAR dependencies are unpacked and all of their DALF files are copied into the new output DAR. However, while DALF files are copied over, the dependency DARs’ manifest files are not copied over, and neither are the source code and interface files. Only the source code and interface files for the primary package of a DAR can show up in a DAR.
For more information on how to open up and inspect the DAR files and DALF files, refer to the documentation on how to parse Daml archive files.
How to parse Daml archive files
When a Daml project is compiled, it produces a DAR (extension.dar), short for Daml Archive. The Daml compiler exposes commands for inspecting this archive.
Inspecting a DAR file
You can rundpm damlc inspect-dar /path/to/your.dar to get a human-readable listing of the files inside it and a list of packages and their package ids. This is often useful to find the package id of the project you just built.
For example, consider a package mypkg which depends on a package dep:
mypkg-1.0.0 is compiled to a DAR, we can inspect that DAR to ensure that it contains both the mypkg package and its dependency dep:
Inspecting a DAR file as JSON
In addition to the human-readable output, you can also get the output as JSON. This is easier to consume programmatically and it is more robust to changes across SDK versions:name and version will be null for packages in Daml-LF < 1.8.
Inspecting the main package of a DAR file
If you’d like to inspect the code inside the main package of a DAR, the Daml compiler provides theinspect tool; running dpm damlc inspect <path-to-dar-file> prints all of the code in the main package of that DAR file in a human-readable format.
For example, run the inspect tool on the DAR produced in the previous section:
Inspecting a DALF file
Theinspect tool also accepts DALF files; running dpm damlc inspect <path-to-dalf-file> on a DALF file prints all of the code in that DALF file.
We can unzip a DAR to access its dalfs and inspect them, for example with the DAR from the previous section:
inspect directly on the DAR file would require fewer steps.
Parsing DAR and DALF files
To parse a DAR or DALF file from within Scala code, thecom-daml:daml-lf-archive-reader library on Maven provides a Scala package object com.digitalasset.daml.lf.archive with several decoders. Below are the common types of inputs and outputs a decoder can have, and which decoders to use depending on the input and output that is desired. For more details on inputs, outputs, and decoders, please refer to Maven to find the source code for the associated libraries.
Output types
When decoding a package, a decoder can have one of several possible outputs, depending on what is needed.-
When the full code of the package is needed, pick a decoder returning tuples
(PackageId, Package). In this case,PackageIdis a string-like type that comes fromcom.digitalasset.daml.lf.language.Refin thecom.daml:daml-lf-datalibrary on Maven.Packagerepresents the full structure of a package, and comes fromcom.digitalasset.daml.lf.language.Ast, in thecom.daml:daml-lf-languagelibrary on Maven. Because fully decoding the package takes more processing time than the next two examples, only use it when the full package code is needed. For example, thecom.digitalasset.daml.lf.typesig.reader.SignatureReaderclass from thecom-daml:daml-lf-api-type-signaturelibrary on Maven takes a(PackageId, Package)pair to produce acom.digitalasset.daml.lf.typesig.PackageSignature(also from theapi-type-signature) package, which specifies all of the templates, datatypes, and interfaces in a package. -
When only the simplest representation of the protobuf of the package is needed, pick a decoder returning a
com.digitalasset.daml.lf.ArchivePayload(from thecom-daml:daml-lf-archivelibrary on Maven). This should only be needed when working with internal protobuf representations of a package. -
When only the package’s byte representation and hash is needed, use a decoder that returns
Archive(from thecom-daml:daml-lf-archive-protolibrary on Maven). When using this, the decoder will not spend time decoding any of the package’s actual content, such as its metadata or its code.
Input types
A decoder can either accept DALF files, DAR files, or it can accept both.- If a decoder accepts DALF files, it will parse the single package in that DALF file to its output type (one of the three specified above).
- If a decoder accepts DAR files, it will parse multiple packages from a DAR file to a struct
Dar[X], which is a case class that encodes a DAR as two public fields,main: Xanddependencies: List[X]. - If a decoder accepts both, it will always produce a
Dar[X]. When given a DAR, the decoder will run as a normal DAR decoder would. When given a DALF, the decoder will decode the DALF as a single package and return aDar[X]with amainpackage and an empty list of dependencies.
Decoders
Decoders for reading DALFs are instances ofGenReader[X], which provides the method readArchiveFromFile(file: java.io.File): Either[Error, X].
-
val ArchiveReader: GenReader[ArchivePayload]RunArchiveReader.readArchiveFromFile(new java.io.File("<path-to-dalf>"))to parse out theArchivePayloadof a dalf file. -
val ArchiveDecoder: GenReader[(PackageId, Ast.Package)]RunArchiveDecoder.readArchiveFromFile(new java.io.File("<path-to-dalf>"))to parse out the(Ref.PackageId, Ast.Package)of a dalf file. -
val ArchiveParser: GenReader[DamlLf.Archive]RunArchiveParser.readArchiveFromFile(new java.io.File("<path-to-dalf>"))to parse out theDamlLf.Archiveof a dalf file.
GenDarReader, which provides the method readArchiveFromFile(file: java.io.File): Either[Error, Dar[X]].
-
val DarReader: GenDarReader[ArchivePayload]RunDarReader.readArchiveFromFile(new java.io.File("<path-to-dar>"))to parse out theDar[ArchivePayload]of a dar file. -
val DarDecoder: GenDarReader[(PackageId, Ast.Package)]RunDarDecoder.readArchiveFromFile(new java.io.File("<path-to-dar>"))to parse out theDar[(Ref.PackageId, Ast.Package)]of a dar file. -
val DarParser: GenDarReader[DamlLf.Archive]RunDarParser.readArchiveFromFile(new java.io.File("<path-to-dar>"))to parse out theDar[DamlLf.Archive]of a dar file.
GenUniversalArchiveReader, which provides the method readFile(file: java.io.File): Either[Error, Dar[X]].
-
val UniversalArchiveReader: GenUniversalArchiveReader[ArchivePayload]RunUniversalArchiveReader.readFile(new java.io.File("<path-to-dar-or-dalf>"))to parse out theDar[ArchivePayload]of a dar file. -
val UniversalArchiveDecoder: GenUniversalArchiveReader[(PackageId, Ast.Package)]RunUniversalArchiveDecoder.readFile(new java.io.File("<path-to-dar-or-dalf>"))to parse out theDar[(Ref.PackageId, Ast.Package)]of a dar file.
Example
We can load up a Scala REPL with thedaml-lf-archive-reader library to interactively parse our mypkg DAR:
.all which returns the main package and dependencies as a single list. Mapping _1 over this gets all of the package IDs in the DAR:
.metadata.name field in the Ast.Package datatype: