This is a detailed log of the development activities that were required for the Substrate client customization. We’d like this log to serve as guidance for new developers wishing to understand the process with the goal of implementing their own versions.
⬇️ Download the minimal template.
This version of the node includes the most basic features of a Substrate node.
Check out Polkadot's official installation instructions to install Rust and the additional dependencies needed for your system.
Usually, it will be necessary to add the wasm32v1-none target, and the rust-src component, both of which can be installed, for example in Linux by executing the following commands:
$ rustup target add wasm32v1-none --toolchain stable-x86_64-unknown-linux-gnu
$ rustup component add rust-src --toolchain stable-x86_64-unknown-linux-gnuThe runtime represents the state transition function for a blockchain. In Polkadot SDK, the runtime is stored as a Wasm binary in the chain state. The Runtime is stored under a unique state key and can be modified during the execution of the state transition function.
The node, also known as the client, is the core component responsible for executing the Wasm runtime and orchestrating various essential blockchain components. It ensures the correct execution of the state transition function and manages multiple critical subsystems, including:
- Wasm execution: Runs the blockchain runtime, which defines the state transition rules.
- Database management: Stores blockchain data.
- Networking: Facilitates peer-to-peer communication, block propagation, and transaction gossiping.
- Transaction pool (Mempool): Manages pending transactions before they are included in a block.
- Consensus mechanism: Ensures agreement on the blockchain state across nodes.
- RPC services: Provides external interfaces for applications and users to interact with the node.
As part of the Substrate customizations that can be done, we can modify the ledger model and the storage structure. This is where Griffin comes in. Griffin is a Substrate-based clone of a Cardano node. It incorporates a simplified eUTxO ledger and hosts a virtual machine capable of executing Plutus scripts for app-logic. This setup provides the essential primitives and logic required for our appchain nodes.
Another modification to the minimal template is the choice of a consensus mechanism. For this example, we have chosen GRANDPA for the finality protocol. Block finality is obtained through consecutive rounds of voting by validator nodes.
In the Changes made to the template section, we'll explain the modifications made to the template to add Griffin and Grandpa, but for your own choice of ledger and consensus mechanism the modifications can be made similarly.
A common Substrate node uses an account model ledger and its runtime is built with FRAME pallets, which are runtime "building blocks" or modules. In contrast, Griffin is designed for the UTxO model, making it incompatible with FRAME, as pallets inherently assume an account-oriented underlying model. This imposes a restriction on the design and implementation, opposed to usual Substrate app-chains. For developers coming from the Cardano ecosystem though, Griffin provides a familiar environment. We can think about decentralized applications in terms of validators and UTxOs, with the advantage of having a completely modifiable starting point as well.
Another key difference between Griffin and other Polkadot-based projects is the chain specification and execution. In general, the chain specification holds the entire code for the node, which is the genesis, among other information that is used at boot time, and it is passed to the node executable as an argument. In Griffin, the node executable takes as argument a simple file that describes the initial set of UTxOs, and some other details.
The original Griffin is a bit outdated, so to use it we updated some dependencies and made some minor compatibility changes, which will be detailed below. We encourage the use of the ad-hoc version provided in this repository at [griffin-core].
As mentioned, the version of Griffin that we use for this project has some modifications compared to the original. Most of these changes are dependency upgrades, but below we'll go over other more interesting modifications:
- [Authorities set function]: we re-implemented the authorities setting function to utilize the EUTxO model. The new function reads the authorities list from a UTxO that is set in the Genesis. A more detailed explanation on how it works and how to use it can be found in the respective readme.
- [Griffin-RPC]: We extended the native node RPC with some queries to obtain UTxOs information through an output reference, an address, or an asset class. More over, we also added a method to submit a transaction in CBOR format. More information and usage examples can be found in the Griffin RPC [readme].
- [Wallet]: The wallet was also improved on through the addition of new functionalities like the queries by address and asset. The
build-txcommand was also modified to take as input a whole json file, instead of many arguments for each component of the transaction.
Griffin is based on Cardano and the eUTxO model so it bears a lot of similarities, but there are some key differences that we will clarify here.
Griffin addresses are ed25519 keys. Public keys are prefixed by 61 and script addresses are prefixed by 70. For simplicity, addresses don't have a staking part.
Transactions don't pay fees. Staking, governance and delegation fields are included in the transaction body, but their functionalities are not supported.
Griffin provides a CLI wallet to interact with the node. This wallet has helpful commands:
show-all-outputs: Displays every UTxO in the chain with brief details about the owner, the coins and value.show-outputs-at: Displays UTxOs owned by the provided address.show-outputs-with-asset: Displays UTxOs that contain a certain token in its value.insert-key: Inserts a key into the wallet's keystore.generate-key: Creates a new key and inserts it into the keystore. Also displays the details.show-balance: summarizes Value amounts for each address.build-tx: Transaction builder command, which takes as input a complete Griffin transaction in json. This transaction must be balanced manually.
More information can be found in the wallet [readme] and also there are some usage examples in the [examples folder].
Polkadot-SDK: From this point on, we steer away from using
polkadot-sdkas one big dependency. Instead, we pick and choose what we need from the Polkadot-SDK and set each as their own dependency. This might look more complicated in terms of mantaining but we pull each dependency from the regsitry and as long as we pull the same stable version for each package there should not be any conflicts. At the time of writing this we use the releasepolkadot-stable2506-2. Having clarified this, it is necessary to add the dependencies in allCargo.tomlfiles, and also to modify the imports where used. To reduce clutter we won't be mentioning these steps while talking about the modifications.
Firstly, copy the source code for griffin-core, griffin-rpc, gpc-wallet and demo. Then we have to add these packages as workspace members to the project manifest: add the paths to the packages under the [members] section.
partnerchain-reference-implementation/Cargo.toml
Lines 11 to 17 in 67c4953
And add the packages as workspace dependencies, so that they can be used by other modules:
partnerchain-reference-implementation/Cargo.toml
Lines 56 to 59 in 67c4953
To integrate Griffin into the node these are the necessary modifications:
We can change the name of the runtime here:
Add dependencies to the Cargo.toml:
partnerchain-reference-implementation/runtime/Cargo.toml
Lines 11 to 13 in 67c4953
Add a new genesis file that includes the information for the initial set of UTxOs and a get_genesis_config function to build the genesis in the runtime.
partnerchain-reference-implementation/runtime/src/genesis.rs
Lines 57 to 67 in 67c4953
In the runtime library:
- Import Griffin types for
Address,AssetName,Datum,InputandPolicyId. These types will be used to implement the runtime apis necessary for Griffin RPC. - Import
TransparentUTxOSetfrom Griffin. - Import
MILLI_SECS_PER_SLOTfrom Griffin, which will be used to define the slot duration of the chain. - Import
GenesisConfigfrom Griffin's config builder.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 14 to 18 in 67c4953
- Import Authorities from
demo, which holds customaura_authoritiesandgrandpa_authoritiesimplementations.
- Define
SessionKeysstruct withinimpl_opaque_keysmacro, with fields forAuraandGrandpapublic keys.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 44 to 75 in 67c4953
- Remove
genesis_config_presetsmod definitions, since we are using our custom genesis. - Define
Transaction,Block,ExecutiveandOutputusing the imported types from Griffin.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 99 to 102 in 67c4953
- Declare
Runtimestruct without FRAME pallets.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 105 to 106 in 67c4953
- Remove all FRAME trait implementations for Runtime.
Runtime APIs are traits that are implemented in the runtime and provide both a runtime-side implementation and a client-side API for the node to interact with. To utilize Griffin we provide new implementations for the required traits.
- Core: use Griffin’s
Executive::execute_blockandExecutive::open_blockinexecute_blockandinitialize_blockmethods implementations.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 124 to 136 in 67c4953
-
Metadata: use trivial implementation.
-
BlockBuilder: call Griffin’s
Executive::apply_extrinsicandExecutive::close_blockinapply_extrinsicandfinalize_blockmethods, and provide trivial implementations ofinherent_extrinsicsandcheck_inherentsmethods.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 152 to 163 in 67c4953
- TaggedTransactionQueue: use Griffin’s
Executive::validate_transaction.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 173 to 181 in 67c4953
- SessionKeys: use the
generateanddecode_into_raw_public_keysmethods of our definedSessionKeystype ingenerate_session_keysanddecode_session_keysmethods implementations.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 183 to 193 in 67c4953
- GenesisBuilder: use Griffin’s
GriffinGenesisConfigBuilder::buildandget_genesis_configfunctions to implementbuild_stateandget_presetmethods. Give trivial implementation ofpreset_names.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 232 to 249 in 67c4953
- Include
sp_consensus_aura::AuraApi<Block, AuraId>. Use customaura_authoritiesimplementation forauthoritiesmethod. UseSlotDuration::from_millisfromsp_consensus_aurawith previously imported MILLI_SECS_PER_SLOT to defineslot_duration.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 195 to 203 in 67c4953
- Include
sp_consensus_grandpa::GrandpaApi<Block>. Use customgrandpa_authoritiesimplementation for the homonymous function from the api. Give a trivial implementation forcurrent_set_id,submit_report_equivocation_unsigned_extrinsicandgenerate_key_ownership_proof.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 205 to 222 in 67c4953
- Include
griffin_core::utxo_set::TransparentUtxoSetApi<Block>. Usepeek_utxo,peek_utxo_from_addressandpeek_utxo_with_assetfromTransparentUtxoSetto implementpeek_utxo,peek_utxo_by_addressandpeek_utxo_with_assetfrom the api, respectively.
partnerchain-reference-implementation/runtime/src/lib.rs
Lines 109 to 121 in 67c4953
- Remove
OffchainWorkerApi,AccountNonceApiandTransactionPaymentApitrait implementations.
Add dependencies to the Cargo.toml:
partnerchain-reference-implementation/node/Cargo.toml
Lines 20 to 22 in 67c4953
In chain_spec, we redefine the functions that build the chain from the specification:
- Import
get_genesis_configfrom runtime genesis, andWASM_BINARYfrom runtime.
partnerchain-reference-implementation/node/src/chain_spec.rs
Lines 1 to 2 in 67c4953
- Modify
development_chain_spec()to take a String as an argument and add the logic that uses it. The name was changed to reflect the purpose of the function more accurately.
partnerchain-reference-implementation/node/src/chain_spec.rs
Lines 8 to 18 in 67c4953
- Add a new function for the configuration of a local test chain.
partnerchain-reference-implementation/node/src/chain_spec.rs
Lines 20 to 30 in 67c4953
In cli:
- Add new
ExportChainSpeccommand and add deprecated warning toBuildSpeccommand.
partnerchain-reference-implementation/node/src/cli.rs
Lines 45 to 51 in 67c4953
In command:
- Modify chain name
partnerchain-reference-implementation/node/src/command.rs
Lines 10 to 12 in 67c4953
- Modify
load_specfunction to use the new config functions defined inchain_spec.rs.
partnerchain-reference-implementation/node/src/command.rs
Lines 34 to 44 in 67c4953
- Add new
ExportChainSpeccommand and a deprecated warning toBuildSpec.
partnerchain-reference-implementation/node/src/command.rs
Lines 70 to 73 in 67c4953
partnerchain-reference-implementation/node/src/command.rs
Lines 53 to 57 in 67c4953
- Provide Griffin's
OpaqueBlocktype inNetworkWorker.
partnerchain-reference-implementation/node/src/command.rs
Lines 132 to 137 in 67c4953
In service:
- Import
GriffinGenesisBlockBuilderandOpaqueBlockasBlockfrom Griffin. - Import
selfandRuntimeApifrom our runtime (necessary if the runtime name changed).
partnerchain-reference-implementation/node/src/service.rs
Lines 4 to 5 in 67c4953
-
Within
new_partial:- Define a new backend using
sc_service::new_db_backend.
- Define
genesis_block_builderfromGriffinGenesisBlockBuilder.
partnerchain-reference-implementation/node/src/service.rs
Lines 52 to 57 in 67c4953
- Modify the creation of the initial parts of the node to use our custom genesis block builder.
partnerchain-reference-implementation/node/src/service.rs
Lines 59 to 67 in 67c4953
- Delete
offchain_workerdefinition, as Griffin’s executive module doesn’t implement it.
- Define a new backend using
-
Within
new_full:- Define
chain_specand its new way of parsing the json.
partnerchain-reference-implementation/node/src/service.rs
Lines 168 to 170 in 67c4953
- Define
zero_timefor the ledger.
partnerchain-reference-implementation/node/src/service.rs
Lines 171 to 173 in 67c4953
- Sleep until reaching zero time for the genesis of the chain.
partnerchain-reference-implementation/node/src/service.rs
Lines 204 to 212 in 67c4953
- Define
In rpc:
- Import
CardanoRpcandCardanoRpcApiServerfrom Cardano RPC within Griffin RPC. - Import
TransparentUtxoSetRpcandTransparentUtxoSetRpcApiServerfrom RPC within Griffin RPC.
partnerchain-reference-implementation/node/src/rpc.rs
Lines 8 to 9 in 67c4953
- Add TransparentUtxoSetApi dependency to
create_fullfunction.
partnerchain-reference-implementation/node/src/rpc.rs
Lines 41 to 43 in 67c4953
- Add the new RPC modules in the
create_fullfunction.
partnerchain-reference-implementation/node/src/rpc.rs
Lines 46 to 50 in 67c4953
These are some common errors that can happen when developing on Substrate:
Errors like:
Double lang item in crate <crate> (whichstd/serdedepends on):...Attempted to define built-in macro more than once
happen commonly when using std crates in a non-std environment, like Substrate's runtime. Std crates can't be used because we compile to WASM. If you run into an error like this and the crate you are using is no-std, make sure you are setting them up correctly. For example, make sure that the dependency is imported with default-features = false or that the std feature is set correctly in the respective Cargo.toml. If you are writing a new module, make sure that it is premised by ´#![cfg_attr(not(feature = "std"), no_std)]´.
When trying to use alloc features like vec, you might run into the trouble that the compiler can't find the alloc crate. This feature can be imported from various dependencies like serde and serde_json. To use it make sure to add extern crate alloc; at the top of your file.