|
| 1 | +--- |
| 2 | +title: Token Transfer Processor |
| 3 | +sidebar_position: 0 |
| 4 | +--- |
| 5 | + |
| 6 | +## Overview |
| 7 | + |
| 8 | +The Token Transfer Processor (TTP) is a [Go package](https://github.com/stellar/go/tree/ttp-v1.0.0/ingest/processors/token_transfer) which uses the [ingest-sdk](../../ingest-sdk/README.mdx) to parse Stellar network transaction data and derive token transfer events. Before TTP, developers had to manually parse complex ledger data, operation results, and ledger entry changes to understand when and how value moved between accounts, contracts, and other entities on the network. |
| 9 | + |
| 10 | +Prior to [CAP-67 Unified Events][cap67], tracking token transfers required significant custom logic to handle different operation types, interpret ledger changes, and reconstruct the flow of assets. CAP-67 introduced a standardized event format that simplifies this process by providing a unified way to represent all token transfer activities. |
| 11 | + |
| 12 | +TTP serves as a facade to CAP-67, automatically generating these standardized events from Stellar ledger data. It can operate in two modes: |
| 13 | + |
| 14 | +- **Standalone mode**: TTP analyzes operations, operation results, and ledger entry changes to derive transfer events |
| 15 | +- **Unified events mode**: TTP reads directly from CAP-67 compliant unified events when available in the ledger data |
| 16 | + |
| 17 | +For more details on operational modes, see the [Modes of Operation](#modes-of-operation) section. |
| 18 | + |
| 19 | +## Key Features |
| 20 | + |
| 21 | +- Captures token movements resulting from: |
| 22 | + |
| 23 | + - Simple payments |
| 24 | + - Path payments |
| 25 | + - DEX operations |
| 26 | + - Account merges |
| 27 | + - Trustline revocations |
| 28 | + - Claimable balance operations |
| 29 | + - Liquidity pool operations |
| 30 | + - Clawback operations |
| 31 | + - Stellar Asset Contract events |
| 32 | + - [SEP-41][sep41] compliant token events |
| 33 | + |
| 34 | +- Generates CAP-67 standardized token events: |
| 35 | + |
| 36 | + - Transfer: Movement of tokens between accounts |
| 37 | + - Mint: Creation of new tokens |
| 38 | + - Burn: Destruction of tokens |
| 39 | + - Clawback: Asset issuer reclaiming tokens |
| 40 | + - Fee: Network fees paid |
| 41 | + |
| 42 | +- Handles muxed account information in compliance with CAP-67 multiplexing support |
| 43 | + |
| 44 | +- Reconciliation for older protocol versions to ensure consistency between operation changes and generated events |
| 45 | + |
| 46 | +## Events |
| 47 | + |
| 48 | +TTP generates events in Go bindings based on [protobuf definitions](https://github.com/stellar/go/blob/ttp-v1.0.0/protos/ingest/processors/token_transfer/token_transfer_event.proto). The definitions codify an IDL for the standardized token transfer event models put forth in CAP-67. Each event contains metadata and type-specific information structured as follows: |
| 49 | + |
| 50 | +| Event Type | Description | Key Fields | When Generated | |
| 51 | +| --- | --- | --- | --- | |
| 52 | +| **Transfer** | Asset movement between two entities | `from`, `to`, `amount`, `asset`, `toMuxedInfo` | When assets move between accounts, contracts, or other entities | |
| 53 | +| **Mint** | Asset creation by the issuer | `to`, `amount`, `asset`, `toMuxedInfo` | When an issuer creates new tokens or when assets are sent from the issuer | |
| 54 | +| **Burn** | Asset destruction to the issuer | `from`, `amount`, `asset` | When assets are returned to the issuer for destruction | |
| 55 | +| **Clawback** | Forced asset recovery by issuer | `from`, `amount`, `asset` | When an issuer uses clawback operations to recover assets | |
| 56 | +| **Fee** | Network fee payment or refund | `account`, `amount` | For all transaction fees and Soroban fee refunds | |
| 57 | + |
| 58 | +### Fee Events |
| 59 | + |
| 60 | +TTP generates fee events to track network fees associated with transaction processing. Understanding the different types of fee events is important for accurate accounting: |
| 61 | + |
| 62 | +Fee events are generated for all transactions, whether they succeed or fail. These represent the network fees that accounts pay to submit transactions to the Stellar network. |
| 63 | + |
| 64 | +- **Present for**: Every transaction |
| 65 | +- **Amount representation**: Positive values indicating fees paid |
| 66 | +- **Asset**: Always XLM (Stellar's native asset) |
| 67 | + |
| 68 | +Fee refund events are generated only for Soroban (smart contract) transactions and only when there are unused resources that qualify for a refund. |
| 69 | + |
| 70 | +- **Present for**: Soroban transactions with unused resource fees |
| 71 | +- **Amount representation**: Negative values to indicate money being returned |
| 72 | +- **Asset**: Always XLM |
| 73 | +- **Event type**: Uses the same `Fee` event type, distinguished by the negative amount |
| 74 | + |
| 75 | +:::note |
| 76 | + |
| 77 | +TTP uses negative amounts in fee events to represent refunds rather than creating a separate refund event type. This approach maintains consistency with the CAP-67 specification while clearly indicating the direction of the fee transaction. |
| 78 | + |
| 79 | +::: |
| 80 | + |
| 81 | +### Event Metadata |
| 82 | + |
| 83 | +Every token transfer event includes comprehensive metadata to provide context about when and where the event occurred: |
| 84 | + |
| 85 | +| Field | Type | Description | |
| 86 | +| --- | --- | --- | |
| 87 | +| `ledgerSequence` | `uint32` | The ledger number where this event occurred. This provides chronological ordering across the entire network. | |
| 88 | +| `txHash` | `string` | The transaction hash that generated this event. This allows you to trace events back to their originating transaction. | |
| 89 | +| `operationIndex` | `uint32*` | The one-based index of the operation within the transaction that caused this event as defined by [SEP-35][sep35]. This field is `nil` for transaction-level events like fees. | |
| 90 | +| `contractAddress` | `string` | The contract address associated with the asset/token being moved. For classic operations or Stellar Asset Contract Events, this field will be the contractId of the underlying classic asset. This enables integration with Stellar's smart contract ecosystem. | |
| 91 | + |
| 92 | +:::note |
| 93 | + |
| 94 | +The `toMuxedInfo` field is included in Transfer and Mint events when the destination uses a muxed account (M-address) and/or when transaction-level memo is set (in the case of non-smart contract transactions), providing additional routing information. |
| 95 | + |
| 96 | +::: |
| 97 | + |
| 98 | +Please refer to [this](../../../../../learn/encyclopedia/transactions-specialized/pooled-accounts-muxed-accounts-memos.mdx) section for more information on muxed account/memo usage. |
| 99 | + |
| 100 | +Please refer to [this](https://stellar.org/protocol/cap-67#prohibit-the-transaction-memo-and-muxed-source-accounts-from-being-set-on-soroban-transactions) section in CAP-67 to learn more on what to expect in the `toMuxedInfo` field. |
| 101 | + |
| 102 | +:::note |
| 103 | + |
| 104 | +The `contractAddress` field is particularly important for DeFi applications as it provides the bridge between classic Stellar assets and their smart contract representations. |
| 105 | + |
| 106 | +For events from classic transactions and SAC events from smart contract transactions, the `contractAddress` field reflects the SAC address for the asset. |
| 107 | + |
| 108 | +::: |
| 109 | + |
| 110 | +## Go API Overview |
| 111 | + |
| 112 | +TTP provides three distinct functions which derive events from different levels of granularity from the underlying Stellar network data. |
| 113 | + |
| 114 | +### EventsFromLedger |
| 115 | + |
| 116 | +```go |
| 117 | +func (p *EventsProcessor) EventsFromLedger(lcm xdr.LedgerCloseMeta) ([]*TokenTransferEvent, error) |
| 118 | +``` |
| 119 | + |
| 120 | +This function processes an entire ledger and returns a flattened list of `TokenTransferEvent` objects. The order of events in the returned slice represents the chronological ordering of debits, credits, and fees as they were applied to accounts, trustlines, and contracts during ledger processing. |
| 121 | + |
| 122 | +The chronological ordering is critical for applications that need to maintain accurate balance tracking or audit trails. For detailed information about how events are ordered, see the [Event Ordering](#event-ordering) section. |
| 123 | + |
| 124 | +### EventsFromTransaction |
| 125 | + |
| 126 | +```go |
| 127 | +func (p *EventsProcessor) EventsFromTransaction(tx ingest.LedgerTransaction) (TransactionEvents, error) |
| 128 | +``` |
| 129 | + |
| 130 | +This function processes a single transaction and returns a `TransactionEvents` structure that separates fee-related events from operation-related events: |
| 131 | + |
| 132 | +- `FeeEvents`: Contains fee charges and refunds associated with the transaction |
| 133 | +- `OperationEvents`: Contains all events generated by the transaction's operations |
| 134 | + |
| 135 | +This separation is useful when you need to handle fees differently from operational transfers, such as for accounting or analytics purposes. |
| 136 | + |
| 137 | +### EventsFromOperation |
| 138 | + |
| 139 | +```go |
| 140 | +func (p *EventsProcessor) EventsFromOperation( |
| 141 | + tx ingest.LedgerTransaction, |
| 142 | + opIndex uint32, |
| 143 | + op xdr.Operation, |
| 144 | + opResult xdr.OperationResult |
| 145 | + ) ([]*TokenTransferEvent, error) |
| 146 | +``` |
| 147 | + |
| 148 | +This function processes a single operation within a transaction and returns a list of events generated by that specific operation. This granular approach is useful for applications that need to analyze or react to specific types of operations. |
| 149 | + |
| 150 | +## Modes of Operation |
| 151 | + |
| 152 | +TTP can operate in two distinct modes depending on how the ledger data was generated and what information is available. |
| 153 | + |
| 154 | +### Default Mode (Recommended) |
| 155 | + |
| 156 | +In default mode, TTP analyzes three sources of information to derive token transfer events: |
| 157 | + |
| 158 | +- **Operations**: The operations submitted in transactions |
| 159 | +- **Operation Results**: The success/failure results of each operation |
| 160 | +- **Ledger Entry Changes**: The changes made to the ledger state |
| 161 | + |
| 162 | +This mode works with all Stellar ledgers regardless of how they were generated or which stellar-core version produced them. It is the safest and most compatible option. |
| 163 | + |
| 164 | +```go |
| 165 | +// Default mode - works with all ledgers |
| 166 | +processor := token_transfer.NewEventsProcessor(networkPassphrase) |
| 167 | +``` |
| 168 | + |
| 169 | +### Unified Events Stream Mode |
| 170 | + |
| 171 | +In unified events stream mode, TTP reads token transfer events directly from the unified events stream embedded in the ledger data. This mode is more efficient but requires ledgers that were generated with specific stellar-core configuration flags. |
| 172 | + |
| 173 | +```go |
| 174 | +// Unified events mode - only for specially configured ledgers |
| 175 | +processor := token_transfer.NewEventsProcessorForUnifiedEvents(networkPassphrase) |
| 176 | +``` |
| 177 | + |
| 178 | +Only use unified events stream mode if you are certain that your ledgers contain unified events. These ledgers must be generated by stellar-core with both `EMIT_CLASSIC_EVENTS=true` and `BACKFILL_STELLAR_ASSET_EVENTS=true` configuration flags enabled. TTP cannot dynamically determine whether a ledger contains unified events or not. |
| 179 | + |
| 180 | +:::caution |
| 181 | + |
| 182 | +If you configure TTP for unified events mode and then provide it ledgers without unified events, TTP will silently produce no events. |
| 183 | + |
| 184 | +::: |
| 185 | + |
| 186 | +**When in doubt, always use the default mode**, as it works reliably with all ledger types. |
| 187 | + |
| 188 | +## Event Ordering |
| 189 | + |
| 190 | +The order of events returned by TTP depends on the Stellar protocol version that was active when the ledger was created. This ordering is crucial for maintaining accurate chronological records of asset movements. |
| 191 | + |
| 192 | +### Pre-Protocol 23 Ordering |
| 193 | + |
| 194 | +Before Protocol 23, events follow this chronological pattern: |
| 195 | + |
| 196 | +``` |
| 197 | +All Fee Events (from all transactions) |
| 198 | +↓ |
| 199 | +For each transaction in ledger: |
| 200 | + - Operation Events (from all operations in the transaction) |
| 201 | + - Fee Refund Event (if applicable, immediately after operation events) |
| 202 | +``` |
| 203 | + |
| 204 | +In this ordering, fee refunds appear immediately after the operation events for each individual transaction. |
| 205 | + |
| 206 | +### Protocol 23+ Ordering |
| 207 | + |
| 208 | +Starting with Protocol 23, events follow this chronological pattern: |
| 209 | + |
| 210 | +``` |
| 211 | +All Fee Events (from all transactions) |
| 212 | +↓ |
| 213 | +All Operation Events (from all transactions, maintaining transaction and operation order) |
| 214 | +↓ |
| 215 | +All Fee Refund Events (from all transactions) |
| 216 | +``` |
| 217 | + |
| 218 | +In this newer ordering, all fee refunds are grouped together at the end, after all transactions have been processed. |
| 219 | + |
| 220 | +:::note |
| 221 | + |
| 222 | +TTP automatically detects the protocol version and applies the correct ordering rules. You don't need to configure this manually, but understanding the ordering differences is important for applications that depend on event sequence. |
| 223 | + |
| 224 | +::: |
| 225 | + |
| 226 | +The chronological ordering ensures that when you process events in the order returned by TTP, you're following the exact sequence in which debits and credits were applied to accounts during ledger processing. This is essential for maintaining accurate balance calculations and audit trails. |
| 227 | + |
| 228 | +## References |
| 229 | + |
| 230 | +- [CAP-67: Unified Events][cap67] |
| 231 | +- [SEP-41: Asset Token Contract Specification][sep41] |
| 232 | +- [CAP-38: Automated Market Makers](https://stellar.org/protocol/cap-38) |
| 233 | +- [SEP-35: ID Scheme for Stellar Operations][sep35] |
| 234 | + |
| 235 | +[sep41]: https://stellar.org/protocol/sep-41 |
| 236 | +[cap67]: https://stellar.org/protocol/cap-67 |
| 237 | +[sep35]: https://stellar.org/protocol/sep-35 |
0 commit comments