Skip to content

Update transaction-lifecycle.mdx #175

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
subbuac1 wants to merge 1 commit into
base: dev
Choose a base branch
from
Open

Update transaction-lifecycle.mdx #175

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
107 changes: 65 additions & 42 deletions content/docs/developer/architecture/transaction-lifecycle.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Transaction Lifecycle
description: Learn Shardeum's transaction lifecycle
title: Transaction Lifecycle on Shardeum
description: Learn how transactions flow through the network from submission to finality.
---

import { Callout } from 'fumadocs-ui/components/callout';
Expand All @@ -9,57 +9,80 @@ import { Callout } from 'fumadocs-ui/components/callout';
<strong>This page is getting updated</strong>
</Callout>

### 1. Overview
## Overview

The transaction lifecycle in the network encompasses both protocol-level transactions, which maintain network cycles and handle consensus, and user transactions, which include coin transfers and smart contract interactions. User transactions are particularly significant for measuring transactions per second (TPS), as these are the interactions that most users care about. Unlike some other networks that count internal transactions towards TPS, this network only counts user transactions to provide an honest metric.
This section describes the end-to-end lifecycle of a transaction on the Shardeum network, from submission to finalization at a high level. The goal is to guide developers understand confirmation behavior, execution guarantees, and how applications should reason about state changes.

### 2. Transaction Initiation
### 1. Transaction Creation

* **User Intent**: The lifecycle begins when a user expresses intent to perform a transaction, such as a coin transfer or interacting with a smart contract, via a wallet like MetaMask.
* **RPC Server Interaction**: The user's transaction request is sent to an RPC (Remote Procedure Call) server, which acts as the Ethereum standard interface to submit transactions to the network and handle various network queries. The RPC server connects the user's wallet to the network.
A transaction is created by a user or application using a standard Ethereum-compatible tool such as a wallet, SDK, or backend service.

### 3. Transaction Submission
At this stage, the transaction includes:

* **Nonce and Gas Estimation**: The RPC server fetches the nonce (a sequential number to keep transactions in order) and estimates the gas required for the transaction. The gas estimate involves simulating the transaction on a local data RPC server with all account data to predict the computational resources needed. For smart contracts, they are pre-run to tally up all the opcodes and their gas, which gives a detailed gas estimate. A feature called "nonce mode" ensures smoother transaction handling. If a user sends a transaction with a higher nonce than expected, instead of instant rejection, the network keeps these transactions until the nonces catch up. For example, if the account nonce is five, and the user sends a transaction with nonce ten, it is kept in the queue until nonces six to nine are processed. This ensures that once nonce nine is processed, nonce ten is automatically injected by the validator network, greatly improving user experience. More about the technical side of things can be seen at [`addTransactionToNonceQueue()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionQueue.ts#L982).
* **Signing and Sending**: Once the nonce and gas details are confirmed, the user signs the transaction with their private key in MetaMask. The signed transaction is then submitted from MetaMask to the RPC server, which injects the transaction into the validator network.
- sender and recipient addresses,
- transaction data (value transfer or contract call),
- gas limit and gas price parameters, and
- a cryptographic signature from the sender.

![Transaction Submission](/img/new/i4.jpg)
Shardeum accepts standard EVM transaction formats, allowing existing Ethereum tooling to work without modification.

### 4. Transaction Processing
### 2. Transaction Submission

* **Transaction Lifecycle States**: A transaction goes through a lifecycle resembling a state machine. It starts in a waiting state after being injected into the network. This waiting state lasts about six seconds, allowing the transaction to mature. During this time, it is queued by multiple validators who need to process it. The six-second wait ensures that transactions are ordered correctly based on timestamps. Once the six seconds have passed, the transaction can be processed.
* **Validator Network**: The transaction is initially sent to one active validator. Then, this transaction is distributed to validators in the transaction group using the gossip protocol. Each validator in the group holds a part of the transaction data necessary for processing. For example, if a transaction requires access to account data for accounts A, B, and C, then each of these accounts will have its own consensus group. Let's say there is a consensus group for account A, a consensus group for account B, and a consensus group for account C. Together, these three consensus groups form a transaction group. Each validator within these consensus groups collaborates to process the transaction. These consensus groups can be perfectly or partially overlapped, ensuring flexibility and efficiency in transaction processing. The forwarding of a transaction is executed by the gossip protocol with the [`spread_tx_to_group`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionQueue.ts#L2238) message.
* **Execution Group Selection**: Before placing the transaction in a queue, an execution group is selected. The developers have determined that 2/3 of the shard size is enough to determine the value of a transaction. However, the shard executing the transaction may not have all necessary information for accounts B and C. The function that enqueues and choses the execution group is [`routeAndQueueAcceptedTransaction()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionQueue.ts#L1810).
The signed transaction is submitted to the network through a JSON-RPC endpoint, typically via:

![Transaction Submission](/img/new/iframe3540.jpg)
- a Web3 wallet,
- a frontend dApp, or
- a backend service.

* **Transaction Group Dynamics**: Validators use a "corresponding tell" process to efficiently share data needed for transaction processing. For instance, if Account B's data is required by an execution group, nodes in Account B's consensus group will forward this data to the nodes in the execution group. This process is designed to be efficient and secure, preventing any single node from unduly influencing the transaction by ensuring data is only forwarded to a specific node as defined by sharding math. The sending of this message is done in [`tellCorrespondingNodes()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionQueue.ts#L3930).
* **Consensus Formation**: Validators execute the transaction, apply it to their local state, and form a consensus on the transaction’s result. The process begins with execution group validators waiting to receive all necessary data for a given transaction, sometimes from non-execution group nodes. Note that no data can be sent or loaded while there are upstream transactions that could modify the state.\
\
Once execution group nodes have the required state, they apply the transaction to an in-memory copy of the data. The result of this transaction application includes outcomes such as applied/not-applied, along with hashes of the updated account values, forming the basis of the proposal. The hash is created by [`calculateVoteHash()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionConsensus.ts#L2785).\
\
Validators create a vote by hashing the proposal, signing it, and sharing this signed hash with other nodes. Each validator's vote is based on the proposal hash it created and signed. If a majority of the nodes agree and have the same hash, the transaction is confirmed. This hash is cryptographically signed, creating a provable receipt. The creation of this receipt is handled in the [`tryProduceReceipt()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionConsensus.ts#L1348) function.\
If a validator observes that a majority of other nodes have the same hash, it acknowledges the consensus and confirms the transaction receipt. The creation of a vote occurs in the [`createAndShareVote()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionConsensus.ts#L2531) function, and appending it to the list of votes happens in [`tryAppendMessage()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionConsensus.ts#L2812).\
\
The structure that contains the proposal, the proposal hash, and the array of signatures along with the IDs of the signers constitutes the signed receipt.\
Once a supermajority of votes is achieved, a valid signed receipt can be produced. After a node has this receipt, it can commit (save) any data it owns related to the transaction.
* **Commit Phase**: Once the receipt is created and the transaction is confirmed, validators proceed to commit the receipt. This involves reversing the data sharing process, where nodes that received account data for execution send updated data back to the original nodes. This ensures that all nodes in the transaction group have the correct and updated account states. This phase is expressed as code in [`commitConsensedTransaction()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionQueue.ts#L1392).
* **Gossip Protocol**: The network employs a gossip protocol to ensure all nodes in the transaction group receive the transaction efficiently. This protocol optimizes communication by sending messages to a few nodes, which then propagate the message further, reducing the load on any single node. The function that sends the gossip message is [`sendGossip()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/p2p/Comms.ts#L826).\
\
There are two variations of gossip algorithms used in the network: "Burst" and "Linear." These algorithms differ in their deterministic calculations of how many messages are sent out at each level of gossip. "Burst" sends more messages and is typically used for gossiping to large groups, such as all nodes. In most cases, "Linear" gossip is preferred, note that while it also includes a burst aspect, it is less pronounced compared to the "Burst" algorithm.
* The main function the processes transactions is [`processTransactions()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionQueue.ts#L4623).
Once submitted, the transaction enters the network’s transaction pool, where it awaits validation and inclusion by validators.

![Validator Stages](/img/new/iframe3541.jpg)
### 3. Validation and Ordering

### 5. Data Distribution and State Updates
Validators perform initial checks to ensure the transaction is valid, including:

* **Transaction Receipt:** The transaction receipt is sent to an archiver, a specialized node responsible for long-term storage and data integrity. The archiver ensures data integrity before storing it. The sending to the archiver is done in [`addReceiptToForward()`](https://github.com/shardeum/shardus-core/blob/e8c14ce4ba19785145646e840082acb57ec8ce3b/src/state-manager/TransactionQueue.ts#L6630).
* **Archiver Setup**: Once the archiver approves the transaction receipt, it writes the receipt to a data log for inter-process communication and also stores it in a database. This database is not highly optimized for queries but is efficient for storage purposes. It is crucial that valid receipts make it to an Archive Server, as failing to do so can cause serious issues.
* **Data Forwarding Mechanism**: The archiver forwards the receipt to a distributor, which disseminates the transaction data to a collector. Distributors use WebSockets to send data efficiently to collectors, ensuring real-time updates across the network. The collector then sends the data back to the local data RPC server, which notifies MetaMask, which is polling, that the transaction has been completed. Once MetaMask receives the confirmation, it updates the user on the transaction status.
* **Data Syncing**: The archiver also forwards transaction data to an explorer collector, which updates the blockchain explorer. This allows users and developers to independently verify transaction status through the explorer interface.
* **State Updates**: Validators maintain only the latest state to optimize performance, while archivers store historical state data. This separation ensures efficient access to current data while preserving the ability to audit and review past transactions.
* **Distributor and Collector Mechanism**: The distributor, a microservice, reads the data log and continuously pulls in data. Collectors connected to distributors receive a constant stream of transactions and their updated states.
* **Flexible Topologies**: The system supports various topologies, allowing for operational flexibility. For instance, one archiver can connect to multiple collectors, or a collector can link to several distributors. This flexibility ensures that the network can scale effectively, addressing potential overloads in the main net.
* **Efficiency of State Updates**: Validators only store the latest state to ensure quick access and high performance, relying on archivers for historical data retrieval when necessary.
- signature verification,
- nonce correctness,
- sufficient balance to cover fees, and
- compliance with protocol rules.

Valid transactions are ordered and considered for inclusion in the next proposed block.

### 4. Block Proposal and Consensus

A validator proposes a block containing a set of valid transactions. Other validators participate in consensus to verify the block’s correctness.

During this phase:

- transactions are executed deterministically,
- state transitions are computed, and
- validators agree on the resulting state.

Shardeum’s consensus mechanism provides deterministic finality, meaning that once a block is finalized, it cannot be reverted.

### 5. Block Finalization

After consensus is reached:

- the block is committed to the canonical chain,
- all transactions in the block are finalized, and
- the updated state becomes immediately authoritative.

At this point, applications can safely rely on the transaction outcome without waiting for additional confirmations.

### 6. State Update and Receipts

The finalized transaction produces:

- updated account and contract state, and
- a transaction receipt containing execution results, gas usage, and logs.

These receipts can be queried through standard RPC methods and are commonly used by dApps to update UI state or trigger follow-up actions.

## What Developers Should Take Away

- Transactions follow a single, deterministic path from submission to finality.
- Finalized transactions are irreversible under normal network operation.
- Applications can safely act on transaction results immediately after finalization.

This predictable lifecycle simplifies application logic, especially for financial, gaming, and real-time use cases.

![Network Stages](/img/new/i7.jpg)