tx binary with block time

README.md

@@ -0,0 +1,249 @@
+# pump-parser
+
+Solana transaction parser focused on swap, liquidity, migration, platform, MEV, compute budget, and compact binary persistence workflows.
+
+The package works with a normalized `RawTx` representation, parses it into `Tx`, and emits one or more `Swap` records when a supported protocol action is found.
+
+## Features
+
+- Parse Solana RPC / Yellowstone transactions into local `RawTx`.
+- Extract swap and liquidity events into `Tx.Swaps`.
+- Preserve transaction metadata such as slot, block time, signer, fee, CU limit, CU consumed, token balances, platform fees, and MEV agent hints.
+- Encode/decode parsed transactions with the `PTXB` / `PTXS` binary formats.
+- Encode/decode raw transactions with the `PRTX` / `PRTS` / `PRBS` binary formats.
+- Stream decode large `PTXS` / `PRTS` payloads without loading every transaction into memory.
+- Merge `PTXS` batches while remapping address tables.
+
+## Supported Parsers
+
+Default parser initialization enables the hot-path Pump parsers only:
+
+- Pump
+- Pump AMM
+
+`EnableAllParsers()` additionally enables:
+
+- Meteora DLMM
+- Meteora Pools
+- Meteora DAMM v2
+- Meteora Bonding Curve
+- Orca Whirlpool
+- Raydium AMM v4
+- Raydium CLMM
+- Raydium CPMM
+- Raydium LaunchLab
+
+Use `InitParser(WithMeteoraDlmm())` when only Meteora DLMM should be added to the default parser set.
+
+## Installation
+
+```bash
+go get github.com/thloyi/pump-parser
+```
+
+This module currently declares `go 1.25.1` in `go.mod`.
+
+## Basic Usage
+
+```go
+package main
+
+import (
+	"fmt"
+	"log"
+
+	pump_parser "github.com/thloyi/pump-parser"
+)
+
+func main() {
+	// Enable all known parser programs. Omit this call to keep the default
+	// Pump + Pump AMM parser set.
+	pump_parser.EnableAllParsers()
+
+	var rawTx *pump_parser.RawTx
+	// Fill rawTx from RPC, Yellowstone, JSON, or RawTx binary decoding.
+
+	tx, err := pump_parser.ParseRawTx(rawTx)
+	if err != nil {
+		log.Fatal(err)
+	}
+
+	fmt.Println("tx:", tx.GetTxHash())
+	for _, swap := range tx.Swaps {
+		fmt.Printf("%s %s base=%s quote=%s pool=%s user=%s\n",
+			swap.Program,
+			swap.Event,
+			swap.BaseAmount,
+			swap.QuoteAmount,
+			swap.Pool,
+			swap.User,
+		)
+	}
+}
+```
+
+## Swap Result
+
+`ParseRawTx` returns a `Tx`. A transaction can contain zero, one, or multiple parsed swap records in `Tx.Swaps`.
+
+Each `Swap` describes one protocol-level swap, liquidity, migration, or pool operation that the parser can normalize. The most important fields are:
+
+| Field | Meaning |
+| --- | --- |
+| `Program` | Normalized protocol name, such as `Pump`, `PumpAMM`, `MeteoraDLMM`, `RaydiumV4`, or `OrcaWhirPool`. |
+| `Event` | Normalized action, such as `buy`, `sell`, `add_liquidity`, `remove_liquidity`, `create`, `complete`, or `migrate`. |
+| `TxIndex` | Approximate execution order across outer and inner instructions. |
+| `InstrIdx` / `InnerIdx` | Outer instruction index and inner instruction index where the swap was found. |
+| `Pool` | Pool, pair, bonding curve, or market account for the parsed action. |
+| `BaseMint` / `QuoteMint` | Base and quote mint accounts after parser normalization. |
+| `BaseTokenProgram` / `QuoteTokenProgram` | Token program for each side, useful when Token-2022 is involved. |
+| `BaseMintDecimals` / `QuoteMintDecimals` | Decimals used to interpret raw token amounts. |
+| `User` | User or effective owner account for the action. If the parsed user is not on-curve, the parser may fall back to the transaction signer. |
+| `BaseAmount` / `QuoteAmount` | Actual parsed base-side and quote-side amounts, stored as `decimal.Decimal`. |
+| `BaseReserve` / `QuoteReserve` | Pool reserves when the protocol event or accounts expose them. |
+| `UserBaseBalance` / `UserQuoteBalance` | User token balances after the transaction when available from token balance metadata. |
+| `AfterSOLBalance` | User or signer SOL balance after the transaction. |
+| `EntryContract` | Known router / entry contract account when detected. |
+| `Mayhem` / `Cashback` | Protocol or platform-specific labels detected from the transaction path. |
+
+Swap amount and slippage fields normalize instruction intent:
+
+| Field | Meaning |
+| --- | --- |
+| `SwapMode` | `exact_in`, `exact_out`, or empty when unknown. |
+| `FixedAmount` / `FixedAmountSide` / `FixedMint` | The user-specified fixed side of the swap. For `exact_in`, this is the input amount. For `exact_out`, this is the target output amount. |
+| `LimitAmountType` | `min_out` for `exact_in`, `max_in` for `exact_out`, or empty when unknown. |
+| `LimitAmount` / `LimitAmountSide` / `LimitMint` | User-specified limit on the opposite side. |
+| `ActualLimitAmount` / `ActualLimitAmountSide` | Actual executed amount on the limited side. |
+| `SlippageBps` | Remaining headroom to the user's limit in basis points. See `SLIPPAGE_MAPPING.md` for protocol-specific derivation rules. |
+
+Liquidity, migration, and DLMM-specific fields are populated only for protocols that expose them:
+
+| Field | Meaning |
+| --- | --- |
+| `Creator` | Creator account when exposed by pool or launch events. |
+| `MigrateToPool` / `MigrateTopProgram` | Destination pool and program for migration events. |
+| `LpMint` | LP token mint for liquidity operations when available. |
+| `ActiveBinId` / `StartBinId` / `EndBinId` | Meteora DLMM bin identifiers. |
+| `RemoveBp` | DLMM remove-liquidity basis points. |
+| `PositionAccount` | DLMM position account. |
+| `FeeAmount` / `LpFeeAmount` | Parsed fee amounts when the protocol exposes fee breakdowns. |
+| `FeeSide` / `FeeMint` / `FeeTokenProgram` / `FeeMintDecimals` | Fee side and mint metadata. |
+| `ConsumeUnit` | Per-swap compute unit value when available. |
+
+Amounts are normalized into decimals in parser output, but the exact scale depends on the source event and parser path. For persisted binary output, `tx_binary.go` defines the conversion rules and schema version.
+
+## Convert Transactions
+
+RPC transactions can be converted with `FromRpcTransactionWithMeta`:
+
+```go
+rawTx, err := pump_parser.FromRpcTransactionWithMeta(
+	txWithMeta,
+	blockTime,
+	slot,
+	indexWithinBlock,
+)
+```
+
+Yellowstone transactions can be converted with `ConvertYellowstoneGrpcTransactionToSolanaTransaction`:
+
+```go
+rawTx, err := pump_parser.ConvertYellowstoneGrpcTransactionToSolanaTransaction(
+	update,
+	createdUnix,
+)
+```
+
+Both conversion paths accept optional `RawTxConvertOptions`:
+
+```go
+rawTx, err := pump_parser.FromRpcTransactionWithMeta(
+	txWithMeta,
+	blockTime,
+	slot,
+	indexWithinBlock,
+	pump_parser.RawTxConvertOptions{
+		ParseLogEvents: true,
+	},
+)
+```
+
+`ParseLogEvents` attaches decoded program log events to matching instructions. `IgnoreLogMessages` skips log-message retention.
+
+## Binary Formats
+
+Parsed transaction binary:
+
+- `EncodeTxBinary` / `DecodeTxBinary` for one parsed `Tx`.
+- `EncodeTxsBinary` / `DecodeTxsBinary` for a batch of parsed `Tx`.
+- `DecodeTxsBinaryReader` for streaming `PTXS` reads.
+- `MergeTxsBinaryBytes` and `MergeTxsBinarySourcesToWriter` for merging `PTXS` batches.
+
+Raw transaction binary:
+
+- `EncodeRawTxBinary` / `DecodeRawTxBinary` for one `RawTx`.
+- `EncodeRawTxsBinary` / `DecodeRawTxsBinary` for a batch of `RawTx`.
+- `EncodeRawTxBlocksBinary` / `DecodeRawTxBlocksBinary` for grouped block data.
+- `DecodeRawTxsBinaryReader` for streaming `PRTS` reads.
+
+Format magic values:
+
+- `PTXB`: one parsed `Tx`.
+- `PTXS`: parsed `Tx` batch.
+- `PRTX`: one raw `RawTx`.
+- `PRTS`: raw `RawTx` batch.
+- `PRBS`: raw block-grouped `RawTx` batch.
+
+When adding or renaming transaction-facing enum values, update `tx_binary.go` enum tables by appending new values only. Reordering existing values changes persisted numeric IDs.
+
+## Commands
+
+Parse a live transaction by signature:
+
+```bash
+TX_HASH=<signature> go run ./cmd/rpc_parse
+```
+
+Collect Yellowstone transactions into a `.prbs` raw-block binary file:
+
+```bash
+YELLOWSTONE_X_TOKEN=<token> go run ./cmd/collect_yellowstone_rawtx_binary \
+  -endpoint ams.rpc.orbitflare.com:10000 \
+  -duration 5m \
+  -output testdata/rawtx-binary/sample.prbs
+```
+
+Measure parsed `Tx` binary size from a `getBlock` JSON payload:
+
+```bash
+go run ./cmd/measure_tx_binary_block \
+  -file /path/to/block.json \
+  -slot 413539056 \
+  -swaps-only
+```
+
+Analyze raw transaction binary size distribution:
+
+```bash
+go run ./cmd/analyze_rawtx_binary_size \
+  -file testdata/rawtx-binary/sample.prbs
+```
+
+## Development
+
+Run the full test suite:
+
+```bash
+go test ./...
+```
+
+Useful focused checks:
+
+```bash
+go test -run 'TestTxBinary|TestRawTxBinary' .
+go test -run 'TestMetaoraPoolSwapEventFromLogsUsesMatchingInvocation|TestMetaoraPoolSwapInstructionOccurrenceIncludesInnerInstructions|TestAttachLogEventsToInstructions' .
+TX_HASH=<signature> go run ./cmd/rpc_parse
+```
+
+For parser regressions, reproduce with the exact transaction signature first, then add a targeted unit test or fixture around the corrected parser path.

chainlink.go

@@ -25,6 +25,9 @@ func chainLinkParser(tx *Tx, instruction Instruction, inners InnerInstructions,
 	}
 
 	decode := instruction.Data
+	if len(decode) < 4 {
+		return increaseOffset(offset), nil
+	}
 	discriminator := binary.LittleEndian.Uint32(decode[0:4])
 
 	switch discriminator {
@@ -55,6 +58,9 @@ func chainLinkSubmitParser(instruction Instruction, inners InnerInstructions, of
 	if storeInstruction.Accounts[0] >= len(tx.rawTx.accountList) || tx.rawTx.accountList[storeInstruction.Accounts[0]] != chainlinkSOLUSDFeedAccount {
 		return increaseOffset(offset), InstructionIgnoredError
 	}
+	if len(storeInstruction.Data) < 8 {
+		return increaseOffset(offset), InstructionIgnoredError
+	}
 	if !bytes.Equal(storeInstruction.Data[0:8], chainlinkSubmitDiscriminator[:]) {
 		return increaseOffset(offset), InstructionIgnoredError
 	}

metaorapool.go

@@ -741,6 +741,9 @@ func metaoraPoolRemoveLiquidity(tx *Tx, instruction Instruction, innerInstructio
 }
 
 func metaoraPoolSwap(tx *Tx, instruction Instruction, innerInstructions InnerInstructions, offset [2]uint) ([]Swap, [2]uint, error) {
+	if len(instruction.Accounts) < 13 {
+		return nil, increaseOffset(offset), fmt.Errorf("not enough accounts for swap instruction")
+	}
 	swapOffset := offset
 	var args metaoraPoolSwapArgs
 	if err := agbinary.NewBorshDecoder(instruction.Data[8:]).Decode(&args); err != nil {

meteoradamm.go

@@ -83,7 +83,13 @@ type MetaoraDammInitializePoolEvent struct {
 }
 
 func meteoraDammV2InitializePoolParser(tx *Tx, instruction Instruction, innerInstructions InnerInstructions, offset [2]uint) ([]Swap, [2]uint, error) {
-	if len(instruction.Accounts) < 12 {
+	requiredAccounts := 12
+	if bytes.Equal(instruction.Data[:8], meteoraDammV2InitializePoolWithDynamicConfig[:]) {
+		requiredAccounts = 13
+	} else if bytes.Equal(instruction.Data[:8], meteoraDammV2InitializeCustomizablePoolDiscriminator[:]) {
+		requiredAccounts = 11
+	}
+	if len(instruction.Accounts) < requiredAccounts {
 		return nil, increaseOffset(offset), fmt.Errorf("invalid instruction accounts length")
 	}
 	var entryContract = tx.rawTx.accountList[tx.rawTx.Transaction.Message.Instructions[offset[0]].ProgramIDIndex]
@@ -439,7 +445,7 @@ func meteoraDammV2AddLiquidityParser(tx *Tx, instruction Instruction, innerInstr
 }
 
 func meteoraDammV2RemoveLiquidityParser(tx *Tx, instruction Instruction, innerInstructions InnerInstructions, offset [2]uint) ([]Swap, [2]uint, error) {
-	if len(instruction.Accounts) < 8 {
+	if len(instruction.Accounts) < 9 {
 		return nil, increaseOffset(offset), fmt.Errorf("invalid instruction accounts length")
 	}
 	tokenAMint := tx.rawTx.accountList[instruction.Accounts[7]]

pump.go

@@ -173,9 +173,17 @@ func CreateParser(tx *Tx, instr Instruction, innerInstructions InnerInstructions
 	}
 	userIndex := 0
 	if bytes.HasPrefix(instr.Data, pumpCreateV2Discriminator[:]) {
+		if len(instr.Accounts) < 6 {
+			return nil, increaseOffset(offset), InstructionIgnoredError
+		}
 		userIndex = instr.Accounts[5]
 	} else if bytes.HasPrefix(instr.Data, pumpCreateDiscriminator[:]) {
+		if len(instr.Accounts) < 8 {
+			return nil, increaseOffset(offset), InstructionIgnoredError
+		}
 		userIndex = instr.Accounts[7]
+	} else {
+		return nil, increaseOffset(offset), InstructionIgnoredError
 	}
 	userBase := getAccountBalanceAfterTx(result, userIndex)
 	userQuote, _ := GetSolAfterTx(result, userIndex)

pumpamm.go

@@ -182,6 +182,9 @@ func pumpAmmParser(tx *Tx, instruction Instruction, innerInstructions InnerInstr
 }
 
 func ammCreatePoolParser(tx *Tx, instruction Instruction, innerInstructions InnerInstructions, offset [2]uint) ([]Swap, [2]uint, error) {
+	if len(instruction.Accounts) < 15 {
+		return nil, increaseOffset(offset), InstructionIgnoredError
+	}
 	result := tx.rawTx
 	var entryContract = result.accountList[result.Transaction.Message.Instructions[offset[0]].ProgramIDIndex]
 	var err error
@@ -275,6 +278,9 @@ func pumpAmmSwapAmountInfoFromArgs(args PumpSwapArgs) (swapMode SwapMode, fixedA
 }
 
 func failedTxAmmBuyParser(tx *Tx, instruction Instruction, innerInstructions InnerInstructions, offset [2]uint) ([]Swap, [2]uint, error) {
+	if len(instruction.Accounts) < 13 {
+		return nil, increaseOffset(offset), InstructionIgnoredError
+	}
 	if tx.Err == nil || tx.Err.UnKnown != "" {
 		return nil, increaseOffset(offset), fmt.Errorf("tx pump amm sell failed but error is nil, offset, %d, %d", offset[0], offset[1])
 	}
@@ -401,6 +407,9 @@ func failedTxAmmBuyParser(tx *Tx, instruction Instruction, innerInstructions Inn
 }
 
 func failedTxAmmSellParser(tx *Tx, instruction Instruction, innerInstructions InnerInstructions, offset [2]uint) ([]Swap, [2]uint, error) {
+	if len(instruction.Accounts) < 13 {
+		return nil, increaseOffset(offset), InstructionIgnoredError
+	}
 	if tx.Err == nil || tx.Err.UnKnown != "" {
 		return nil, increaseOffset(offset), fmt.Errorf("tx pump amm sell failed but error is nil, offset, %d, %d", offset[0], offset[1])
 	}
@@ -525,6 +534,9 @@ func ammBuyParser(tx *Tx, instruction Instruction, innerInstructions InnerInstru
 	var entryContract = result.accountList[result.Transaction.Message.Instructions[offset[0]].ProgramIDIndex]
 	var err error
 	var prefixLen = offset[1]
+	if len(instruction.Accounts) < 13 {
+		return nil, increaseOffset(offset), InstructionIgnoredError
+	}
 	inners, err := getInnerInstructions(innerInstructions, prefixLen)
 	if err != nil {
 		return nil, increaseOffset(offset), fmt.Errorf("pumpamm create get inner instructions error: %v, offset: %d,  %d", err, offset[0], prefixLen)
@@ -662,6 +674,9 @@ func ammSellParser(tx *Tx, instruction Instruction, innerInstructions InnerInstr
 	result := tx.rawTx
 	var entryContract = result.accountList[result.Transaction.Message.Instructions[offset[0]].ProgramIDIndex]
 	var err error
+	if len(instruction.Accounts) < 13 {
+		return nil, increaseOffset(offset), InstructionIgnoredError
+	}
 	var prefixLen = offset[1]
 	inners, err := getInnerInstructions(innerInstructions, prefixLen)
 	if err != nil {
@@ -789,6 +804,9 @@ func depositParse(tx *Tx, instruction Instruction, innerInstructions InnerInstru
 	result := tx.rawTx
 	var entryContract = result.accountList[result.Transaction.Message.Instructions[offset[0]].ProgramIDIndex]
 	var err error
+	if len(instruction.Accounts) < 11 {
+		return nil, increaseOffset(offset), InstructionIgnoredError
+	}
 	var prefixLen = offset[1]
 	inners, err := getInnerInstructions(innerInstructions, prefixLen)
 	if err != nil {
@@ -887,6 +905,9 @@ func withdrawParse(tx *Tx, instruction Instruction, innerInstructions InnerInstr
 	result := tx.rawTx
 	var entryContract = result.accountList[result.Transaction.Message.Instructions[offset[0]].ProgramIDIndex]
 	var err error
+	if len(instruction.Accounts) < 11 {
+		return nil, increaseOffset(offset), InstructionIgnoredError
+	}
 	var prefixLen = offset[1]
 	inners, err := getInnerInstructions(innerInstructions, prefixLen)
 	if err != nil {

raydiumclmm.go

@@ -108,38 +108,14 @@ func raydiumClmmAddLiquidityParser(tx *Tx, instruction Instruction, innerInstruc
 	switch discriminator {
 	case raydiumClmmIncreaseLiquidityDiscriminator:
 		accountMin = 12
-		market = tx.rawTx.accountList[instruction.Accounts[2]]
-		vault0 = instruction.Accounts[9]
-		vault1 = instruction.Accounts[10]
 	case raydiumClmmIncreaseLiquidityV2Discriminator:
 		accountMin = 15
-		market = tx.rawTx.accountList[instruction.Accounts[2]]
-		vault0 = instruction.Accounts[9]
-		vault1 = instruction.Accounts[10]
-		//token0 = tx.rawTx.accountList[instruction.Accounts[13]]
-		//token1 = tx.rawTx.accountList[instruction.Accounts[14]]
 	case raydiumClmmOpenPositionDiscriminator:
 		accountMin = 19
-		market = tx.rawTx.accountList[instruction.Accounts[5]]
-		vault0 = instruction.Accounts[12]
-		vault1 = instruction.Accounts[13]
-		lpToken = tx.rawTx.accountList[instruction.Accounts[2]]
 	case raydiumClmmOpenPositionV2Discriminator:
 		accountMin = 22
-		market = tx.rawTx.accountList[instruction.Accounts[5]]
-		vault0 = instruction.Accounts[12]
-		vault1 = instruction.Accounts[13]
-		lpToken = tx.rawTx.accountList[instruction.Accounts[2]]
-		//token0 = tx.rawTx.accountList[instruction.Accounts[20]]
-		//token1 = tx.rawTx.accountList[instruction.Accounts[21]]
 	case raydiumClmmOpenPositionWithToken22NftDiscriminator:
 		accountMin = 20
-		market = tx.rawTx.accountList[instruction.Accounts[4]]
-		vault0 = instruction.Accounts[11]
-		vault1 = instruction.Accounts[12]
-		lpToken = tx.rawTx.accountList[instruction.Accounts[2]]
-		//token0 = tx.rawTx.accountList[instruction.Accounts[18]]
-		//token1 = tx.rawTx.accountList[instruction.Accounts[19]]
 	default:
 		return nil, increaseOffset(offset), fmt.Errorf("invalid discriminator")
 	}
@@ -148,6 +124,23 @@ func raydiumClmmAddLiquidityParser(tx *Tx, instruction Instruction, innerInstruc
 		return nil, increaseOffset(offset), fmt.Errorf("not enough accounts for raydiumClmm add liquidity instruction, offset, %d, %d", offset[0], offset[1])
 	}
 
+	switch discriminator {
+	case raydiumClmmIncreaseLiquidityDiscriminator, raydiumClmmIncreaseLiquidityV2Discriminator:
+		market = tx.rawTx.accountList[instruction.Accounts[2]]
+		vault0 = instruction.Accounts[9]
+		vault1 = instruction.Accounts[10]
+	case raydiumClmmOpenPositionDiscriminator, raydiumClmmOpenPositionV2Discriminator:
+		market = tx.rawTx.accountList[instruction.Accounts[5]]
+		vault0 = instruction.Accounts[12]
+		vault1 = instruction.Accounts[13]
+		lpToken = tx.rawTx.accountList[instruction.Accounts[2]]
+	case raydiumClmmOpenPositionWithToken22NftDiscriminator:
+		market = tx.rawTx.accountList[instruction.Accounts[4]]
+		vault0 = instruction.Accounts[11]
+		vault1 = instruction.Accounts[12]
+		lpToken = tx.rawTx.accountList[instruction.Accounts[2]]
+	}
+
 	baseTokenBalance, err := getTokenBalanceAfterTx(tx.rawTx, vault0)
 	if err != nil {
 		return nil, increaseOffset(offset), fmt.Errorf("failed to get token0 vault balance after tx: %v", err)
@@ -197,6 +190,8 @@ func raydiumClmmDecreaseLiquidityParser(tx *Tx, instruction Instruction, innerIn
 		accountMin = 14
 	} else if discriminator == raydiumClmmDecreaseLiquidityV2Discriminator {
 		accountMin = 16
+	} else {
+		return nil, increaseOffset(offset), fmt.Errorf("invalid discriminator")
 	}
 	if len(instruction.Accounts) < accountMin {
 		return nil, increaseOffset(offset), fmt.Errorf("not enough accounts for decrease liquidity instruction")
@@ -299,23 +294,21 @@ func raydiumClmmSwapParser(tx *Tx, instruction Instruction, innerInstructions In
 	}
 	if discriminator == raydiumClmmSwapDiscriminator {
 		accountMin = 9
-		pool = tx.rawTx.accountList[instruction.Accounts[2]]
-		userTokenInAccount = instruction.Accounts[3]
-		userTokenOutAccount = instruction.Accounts[4]
-		tokenInVault = instruction.Accounts[5]
-		tokenOutVault = instruction.Accounts[6]
 	} else if discriminator == raydiumClmmSwapV2Discriminator {
 		accountMin = 13
-		pool = tx.rawTx.accountList[instruction.Accounts[2]]
+	} else {
-		userTokenInAccount = instruction.Accounts[3]
+		return nil, increaseOffset(offset), fmt.Errorf("invalid discriminator")
-		userTokenOutAccount = instruction.Accounts[4]
-		tokenInVault = instruction.Accounts[5]
-		tokenOutVault = instruction.Accounts[6]
 	}
 	if len(instruction.Accounts) < accountMin {
 		return nil, increaseOffset(offset), fmt.Errorf("not enough accounts for swap instruction")
 	}
 
+	pool = tx.rawTx.accountList[instruction.Accounts[2]]
+	userTokenInAccount = instruction.Accounts[3]
+	userTokenOutAccount = instruction.Accounts[4]
+	tokenInVault = instruction.Accounts[5]
+	tokenOutVault = instruction.Accounts[6]
+
 	baseTokenBalance, err := getTokenBalanceAfterTx(tx.rawTx, tokenInVault)
 	if err != nil {
 		return nil, increaseOffset(offset), fmt.Errorf("failed to get tokenIn vault balance after tx: %w", err)

raydiumlaunchlab.go

@@ -373,6 +373,9 @@ type raydiumLaunchLabSwapArgs struct {
 }
 
 func raydiumLaunchLabSwapParser(tx *Tx, instruction Instruction, innerInstructions InnerInstructions, offset [2]uint) ([]Swap, [2]uint, error) {
+	if len(instruction.Accounts) < 13 {
+		return nil, increaseOffset(offset), InstructionIgnoredError
+	}
 	platformConfig := tx.rawTx.accountList[instruction.Accounts[3]]
 	var programName string
 	if platformConfig.Equals(bonkPlatformConfig) {

system.go

@@ -15,6 +15,9 @@ func systemParser(tx *Tx, instruction Instruction, _ InnerInstructions, offset [
 	}
 
 	decode := instruction.Data
+	if len(decode) < 4 {
+		return increaseOffset(offset), nil
+	}
 	discriminator := binary.LittleEndian.Uint32(decode[0:4])
 
 	switch discriminator {
@@ -29,6 +32,9 @@ func TransferParser(result *RawTx, instruction Instruction, offset [2]uint, tx *
 	if len(decodeData) < 8 {
 		return increaseOffset(offset), nil
 	}
+	if len(instruction.Accounts) < 2 || len(result.Transaction.Message.Instructions[offset[0]].Accounts) < 1 {
+		return increaseOffset(offset), InstructionIgnoredError
+	}
 	var lamports uint64 = binary.LittleEndian.Uint64(decodeData)
 
 	from := result.accountList[result.Transaction.Message.Instructions[offset[0]].Accounts[0]]

tx_binary.go

@@ -17,7 +17,8 @@ import (
 )
 
 const (
-	txBinarySchemaVersionCurrent uint16 = 3
+	txBinarySchemaVersionV3      uint16 = 3
+	txBinarySchemaVersionCurrent uint16 = 4
 	txBinaryEnumVersionV1        uint16 = 1
 
 	txBinarySOLScale     int32 = 9
@@ -27,6 +28,10 @@ const (
 var txBinaryMagic = [4]byte{'P', 'T', 'X', 'B'}
 var txsBinaryMagic = [4]byte{'P', 'T', 'X', 'S'}
 
+func txBinarySchemaVersionSupported(version uint16) bool {
+	return version >= txBinarySchemaVersionV3 && version <= txBinarySchemaVersionCurrent
+}
+
 type TxBinary struct {
 	SchemaVersion        uint16
 	EnumVersion          uint16
@@ -34,6 +39,7 @@ type TxBinary struct {
 	Signer               uint32
 	Block                uint64
 	BlockIndex           uint64
+	BlockAt              int64
 	TxHash               *[64]byte
 	CuFee                uint64
 	Swaps                []SwapBinary
@@ -204,6 +210,7 @@ func newTxBinaryWithAddressTable(tx *Tx, addressTable []solana.PublicKey, addres
 		EnumVersion:          txBinaryEnumVersionV1,
 		Block:                tx.Block,
 		BlockIndex:           tx.BlockIndex,
+		BlockAt:              tx.BlockAt,
 		CuLimit:              tx.CuLimit,
 		ComputeUnitsConsumed: tx.ComputeUnitsConsumed,
 	}
@@ -411,6 +418,8 @@ func MergeTxsBinarySourcesToWriterWithOptions(sources []TxsBinaryReaderSource, w
 					return fmt.Errorf("source[%d].batch[%d].tx[%d]: %w", sourceIndex, batchIndex, txIndex, err)
 				}
 
+				tx.SchemaVersion = plan.schemaVersion
+				tx.EnumVersion = plan.enumVersion
 				bodyBytes, err := txBinaryMarshalTxBody(&tx, plan.enumTable)
 				if err != nil {
 					reader.Close()
@@ -432,7 +441,7 @@ func (tx *TxBinary) MarshalBinary() ([]byte, error) {
 	if tx == nil {
 		return nil, fmt.Errorf("tx binary is nil")
 	}
-	if tx.SchemaVersion != txBinarySchemaVersionCurrent {
+	if !txBinarySchemaVersionSupported(tx.SchemaVersion) {
 		return nil, fmt.Errorf("unsupported tx binary schema version: %d", tx.SchemaVersion)
 	}
 
@@ -460,7 +469,7 @@ func (txs *TxsBinary) MarshalBinary() ([]byte, error) {
 	if txs == nil {
 		return nil, fmt.Errorf("txs binary is nil")
 	}
-	if txs.SchemaVersion != txBinarySchemaVersionCurrent {
+	if !txBinarySchemaVersionSupported(txs.SchemaVersion) {
 		return nil, fmt.Errorf("unsupported tx binary schema version: %d", txs.SchemaVersion)
 	}
 
@@ -478,8 +487,11 @@ func (txs *TxsBinary) MarshalBinary() ([]byte, error) {
 	}
 	enc.writeUint32(uint32(len(txs.Txs)))
 	for i := range txs.Txs {
-		if err := enc.writeTxBinaryBody(&txs.Txs[i], enumTable); err != nil {
+		tx := txs.Txs[i]
-			return nil, fmt.Errorf("tx[%d], %s: %w", i, base58.Encode(txs.Txs[i].TxHash[:]), err)
+		tx.SchemaVersion = txs.SchemaVersion
+		tx.EnumVersion = txs.EnumVersion
+		if err := enc.writeTxBinaryBody(&tx, enumTable); err != nil {
+			return nil, fmt.Errorf("tx[%d], %s: %w", i, base58.Encode(tx.TxHash[:]), err)
 		}
 	}
 	return enc.bytes(), nil
@@ -520,7 +532,7 @@ func (tx *TxBinary) UnmarshalBinary(data []byte) error {
 	if err != nil {
 		return err
 	}
-	if tx.SchemaVersion != txBinarySchemaVersionCurrent {
+	if !txBinarySchemaVersionSupported(tx.SchemaVersion) {
 		return fmt.Errorf("unsupported tx binary schema version: %d", tx.SchemaVersion)
 	}
 
@@ -560,7 +572,7 @@ func (txs *TxsBinary) UnmarshalBinary(data []byte) error {
 	if err != nil {
 		return err
 	}
-	if txs.SchemaVersion != txBinarySchemaVersionCurrent {
+	if !txBinarySchemaVersionSupported(txs.SchemaVersion) {
 		return fmt.Errorf("unsupported tx binary schema version: %d", txs.SchemaVersion)
 	}
 
@@ -613,6 +625,7 @@ func (tx *TxBinary) ToTx() (*Tx, error) {
 		Signer:               signer,
 		Block:                tx.Block,
 		BlockIndex:           tx.BlockIndex,
+		BlockAt:              tx.BlockAt,
 		CuFee:                decimal.NewFromUint64(tx.CuFee),
 		CUPrice:              decimal.NewFromUint64(tx.CUPrice).Shift(-txBinaryCUPriceScale),
 		BeforeSolBalance:     txBinaryFloat64ToDecimal(tx.BeforeSolBalance, txBinarySOLScale),
@@ -1166,6 +1179,9 @@ func (enc *txBinaryEncoder) writeTxBinaryBody(tx *TxBinary, enumTable *txBinaryE
 	enc.writeUint32(tx.Signer)
 	enc.writeUint64(tx.Block)
 	enc.writeUint64(tx.BlockIndex)
+	if tx.SchemaVersion >= txBinarySchemaVersionCurrent {
+		enc.writeUint64(uint64(tx.BlockAt))
+	}
 	enc.writeBool(tx.TxHash != nil)
 	if tx.TxHash != nil {
 		enc.writeBytes(tx.TxHash[:])
@@ -1474,7 +1490,7 @@ func (dec *txBinaryStreamDecoder) readTxsBinaryHeader() (*txsBinaryHeader, error
 	if err != nil {
 		return nil, err
 	}
-	if schemaVersion != txBinarySchemaVersionCurrent {
+	if !txBinarySchemaVersionSupported(schemaVersion) {
 		return nil, fmt.Errorf("unsupported tx binary schema version: %d", schemaVersion)
 	}
 
@@ -1531,7 +1547,7 @@ func (dec *txBinaryStreamDecoder) readTxsBinaryHeaderOrEOF() (*txsBinaryHeader,
 	if err != nil {
 		return nil, err
 	}
-	if schemaVersion != txBinarySchemaVersionCurrent {
+	if !txBinarySchemaVersionSupported(schemaVersion) {
 		return nil, fmt.Errorf("unsupported tx binary schema version: %d", schemaVersion)
 	}
 
@@ -1799,6 +1815,13 @@ func txBinaryReadTxBody(dec txBinaryBodyReader, tx *TxBinary, enumTable *txBinar
 	if tx.BlockIndex, err = dec.readUint64(); err != nil {
 		return err
 	}
+	if tx.SchemaVersion >= txBinarySchemaVersionCurrent {
+		blockAt, err := dec.readUint64()
+		if err != nil {
+			return err
+		}
+		tx.BlockAt = int64(blockAt)
+	}
 
 	hasTxHash, err := dec.readBool()
 	if err != nil {
@@ -1854,7 +1877,9 @@ func txBinaryBuildMergePlan(sources []TxsBinaryReaderSource, opts TxsBinaryMerge
 	builder := txBinaryAddressTableBuilder{
 		index: make(map[solana.PublicKey]struct{}),
 	}
-	plan := &txsBinaryMergePlan{}
+	plan := &txsBinaryMergePlan{
+		schemaVersion: txBinarySchemaVersionCurrent,
+	}
 	hasBatch := false
 
 	for sourceIndex, source := range sources {
@@ -1896,15 +1921,10 @@ func txBinaryBuildMergePlan(sources []TxsBinaryReaderSource, opts TxsBinaryMerge
 			}
 
 			if !hasBatch {
-				plan.schemaVersion = header.schemaVersion
 				plan.enumVersion = header.enumVersion
 				plan.enumTable = header.enumTable
 				hasBatch = true
 			} else {
-				if header.schemaVersion != plan.schemaVersion {
-					reader.Close()
-					return nil, fmt.Errorf("source[%d].batch[%d]: schema version mismatch: got %d want %d", sourceIndex, batchIndex, header.schemaVersion, plan.schemaVersion)
-				}
 				if header.enumVersion != plan.enumVersion {
 					reader.Close()
 					return nil, fmt.Errorf("source[%d].batch[%d]: enum version mismatch: got %d want %d", sourceIndex, batchIndex, header.enumVersion, plan.enumVersion)

tx_binary_test.go

@@ -19,6 +19,7 @@ func TestTxBinaryRoundTrip(t *testing.T) {
 		Signer:               mustPubKey("So11111111111111111111111111111111111111112"),
 		Block:                123456789,
 		BlockIndex:           42,
+		BlockAt:              1710000000,
 		TxHash:               &txHash,
 		CuFee:                decimal.NewFromInt(5000),
 		CUPrice:              decimal.RequireFromString("0.123456"),
@@ -118,6 +119,9 @@ func TestTxBinaryRoundTrip(t *testing.T) {
 	if decoded.BlockIndex != original.BlockIndex {
 		t.Fatalf("BlockIndex = %d, want %d", decoded.BlockIndex, original.BlockIndex)
 	}
+	if decoded.BlockAt != original.BlockAt {
+		t.Fatalf("BlockAt = %d, want %d", decoded.BlockAt, original.BlockAt)
+	}
 	if decoded.TxHash == nil {
 		t.Fatal("TxHash = nil, want non-nil")
 	}
@@ -510,6 +514,7 @@ func TestTxsBinaryRoundTripWithSharedAddressTable(t *testing.T) {
 		Signer:               mustPubKey("So11111111111111111111111111111111111111112"),
 		Block:                1,
 		BlockIndex:           1,
+		BlockAt:              1710000001,
 		CuFee:                decimal.NewFromInt(1000),
 		CUPrice:              decimal.RequireFromString("0.123456"),
 		BeforeSolBalance:     decimal.RequireFromString("1.000000000"),
@@ -560,6 +565,7 @@ func TestTxsBinaryRoundTripWithSharedAddressTable(t *testing.T) {
 	tx2 := tx1
 	tx2.Block = 2
 	tx2.BlockIndex = 2
+	tx2.BlockAt = 1710000002
 	tx2.CuFee = decimal.NewFromInt(2000)
 	tx2.AfterSOLBalance = decimal.RequireFromString("0.700000000")
 	tx2.Swaps = []Swap{tx1.Swaps[0]}
@@ -581,6 +587,9 @@ func TestTxsBinaryRoundTripWithSharedAddressTable(t *testing.T) {
 	if decoded[0].Signer != tx1.Signer || decoded[1].Signer != tx2.Signer {
 		t.Fatalf("decoded signer mismatch")
 	}
+	if decoded[0].BlockAt != tx1.BlockAt || decoded[1].BlockAt != tx2.BlockAt {
+		t.Fatalf("decoded block_at mismatch")
+	}
 	if decoded[0].Swaps[0].Pool != tx1.Swaps[0].Pool || decoded[1].Swaps[0].Pool != tx2.Swaps[0].Pool {
 		t.Fatalf("decoded shared address mismatch")
 	}
@@ -603,6 +612,7 @@ func TestDecodeTxsBinaryReader(t *testing.T) {
 		Signer:               mustPubKey("So11111111111111111111111111111111111111112"),
 		Block:                100,
 		BlockIndex:           7,
+		BlockAt:              1710000100,
 		CuFee:                decimal.NewFromInt(111),
 		CUPrice:              decimal.RequireFromString("0.123456"),
 		BeforeSolBalance:     decimal.RequireFromString("1.000000000"),
@@ -649,6 +659,7 @@ func TestDecodeTxsBinaryReader(t *testing.T) {
 	tx2 := tx1
 	tx2.Block = 101
 	tx2.BlockIndex = 8
+	tx2.BlockAt = 1710000101
 	tx2.CuFee = decimal.NewFromInt(222)
 	tx2.AfterSOLBalance = decimal.RequireFromString("0.300000000")
 	tx2.Swaps = []Swap{tx1.Swaps[0]}
@@ -677,6 +688,9 @@ func TestDecodeTxsBinaryReader(t *testing.T) {
 	if decoded[0].Block != tx1.Block || decoded[1].Block != tx2.Block {
 		t.Fatalf("decoded block mismatch")
 	}
+	if decoded[0].BlockAt != tx1.BlockAt || decoded[1].BlockAt != tx2.BlockAt {
+		t.Fatalf("decoded block_at mismatch")
+	}
 	if decoded[0].Swaps[0].BaseAmount.Cmp(tx1.Swaps[0].BaseAmount) != 0 {
 		t.Fatalf("decoded tx1 swap base amount = %s, want %s", decoded[0].Swaps[0].BaseAmount, tx1.Swaps[0].BaseAmount)
 	}
@@ -724,6 +738,7 @@ func TestMergeTxsBinaryBytes(t *testing.T) {
 		Signer:               mustPubKey("So11111111111111111111111111111111111111112"),
 		Block:                11,
 		BlockIndex:           1,
+		BlockAt:              1710000011,
 		CuFee:                decimal.NewFromInt(10),
 		CUPrice:              decimal.RequireFromString("0.000123"),
 		BeforeSolBalance:     decimal.RequireFromString("1.100000000"),
@@ -755,6 +770,7 @@ func TestMergeTxsBinaryBytes(t *testing.T) {
 		Signer:               mustPubKey("SysvarRent111111111111111111111111111111111"),
 		Block:                12,
 		BlockIndex:           2,
+		BlockAt:              1710000012,
 		CuFee:                decimal.NewFromInt(20),
 		CUPrice:              decimal.RequireFromString("0.000456"),
 		BeforeSolBalance:     decimal.RequireFromString("2.200000000"),
@@ -818,6 +834,9 @@ func TestMergeTxsBinaryBytes(t *testing.T) {
 	if decoded[0].Block != tx1.Block || decoded[1].Block != tx2.Block {
 		t.Fatalf("decoded block mismatch")
 	}
+	if decoded[0].BlockAt != tx1.BlockAt || decoded[1].BlockAt != tx2.BlockAt {
+		t.Fatalf("decoded block_at mismatch")
+	}
 }
 
 func TestMergeTxsBinarySourcesToWriterWithConcatenatedBatches(t *testing.T) {
@@ -825,6 +844,7 @@ func TestMergeTxsBinarySourcesToWriterWithConcatenatedBatches(t *testing.T) {
 		Signer:               mustPubKey("So11111111111111111111111111111111111111112"),
 		Block:                21,
 		BlockIndex:           1,
+		BlockAt:              1710000021,
 		CuFee:                decimal.NewFromInt(1),
 		CUPrice:              decimal.RequireFromString("0.000001"),
 		BeforeSolBalance:     decimal.RequireFromString("1.000000000"),
@@ -835,9 +855,11 @@ func TestMergeTxsBinarySourcesToWriterWithConcatenatedBatches(t *testing.T) {
 	tx2 := tx1
 	tx2.Block = 22
 	tx2.BlockIndex = 2
+	tx2.BlockAt = 1710000022
 	tx2.Signer = mustPubKey("SysvarRent111111111111111111111111111111111")
 	tx3 := tx1
 	tx3.Block = 23
+	tx3.BlockAt = 1710000023
 	tx3.BlockIndex = 3
 	tx3.Signer = mustPubKey("ComputeBudget111111111111111111111111111111")
 
@@ -880,6 +902,9 @@ func TestMergeTxsBinarySourcesToWriterWithConcatenatedBatches(t *testing.T) {
 	if decoded[0].Block != tx1.Block || decoded[1].Block != tx2.Block || decoded[2].Block != tx3.Block {
 		t.Fatalf("decoded block order mismatch")
 	}
+	if decoded[0].BlockAt != tx1.BlockAt || decoded[1].BlockAt != tx2.BlockAt || decoded[2].BlockAt != tx3.BlockAt {
+		t.Fatalf("decoded block_at order mismatch")
+	}
 }
 
 func TestMergeTxsBinarySourcesToWriterWithBatchHeaderFuncSkip(t *testing.T) {
@@ -887,6 +912,7 @@ func TestMergeTxsBinarySourcesToWriterWithBatchHeaderFuncSkip(t *testing.T) {
 		Signer:               mustPubKey("So11111111111111111111111111111111111111112"),
 		Block:                31,
 		BlockIndex:           1,
+		BlockAt:              1710000031,
 		CuFee:                decimal.NewFromInt(1),
 		CUPrice:              decimal.RequireFromString("0.000001"),
 		BeforeSolBalance:     decimal.RequireFromString("1.000000000"),
@@ -897,10 +923,12 @@ func TestMergeTxsBinarySourcesToWriterWithBatchHeaderFuncSkip(t *testing.T) {
 	tx2 := tx1
 	tx2.Block = 32
 	tx2.BlockIndex = 2
+	tx2.BlockAt = 1710000032
 	tx2.Signer = mustPubKey("SysvarRent111111111111111111111111111111111")
 	tx3 := tx1
 	tx3.Block = 33
 	tx3.BlockIndex = 3
+	tx3.BlockAt = 1710000033
 	tx3.Signer = mustPubKey("ComputeBudget111111111111111111111111111111")
 
 	batch1, err := EncodeTxsBinary([]Tx{tx1})
@@ -960,15 +988,127 @@ func TestMergeTxsBinarySourcesToWriterWithBatchHeaderFuncSkip(t *testing.T) {
 	if decoded[0].Block != tx1.Block || decoded[1].Block != tx3.Block {
 		t.Fatalf("decoded block order mismatch after skip")
 	}
+	if decoded[0].BlockAt != tx1.BlockAt || decoded[1].BlockAt != tx3.BlockAt {
+		t.Fatalf("decoded block_at order mismatch after skip")
+	}
 	if source.opens != 2 {
 		t.Fatalf("source.opens = %d, want 2", source.opens)
 	}
 }
 
+func TestTxBinaryDecodeSchemaV3LeavesBlockAtZero(t *testing.T) {
+	original := &Tx{
+		Signer:     mustPubKey("So11111111111111111111111111111111111111112"),
+		Block:      41,
+		BlockIndex: 1,
+		BlockAt:    1710000041,
+	}
+
+	encoded := mustEncodeTxBinaryV3(t, original)
+	decoded, err := DecodeTxBinary(encoded)
+	if err != nil {
+		t.Fatalf("DecodeTxBinary(v3) error = %v", err)
+	}
+	if decoded.Block != original.Block || decoded.BlockIndex != original.BlockIndex {
+		t.Fatalf("decoded block mismatch: got (%d,%d), want (%d,%d)", decoded.Block, decoded.BlockIndex, original.Block, original.BlockIndex)
+	}
+	if decoded.BlockAt != 0 {
+		t.Fatalf("BlockAt = %d, want 0 for legacy v3", decoded.BlockAt)
+	}
+}
+
+func TestMergeTxsBinaryBytesUpgradesSchemaV3AndPreservesV4BlockAt(t *testing.T) {
+	legacyTx := Tx{
+		Signer:     mustPubKey("So11111111111111111111111111111111111111112"),
+		Block:      51,
+		BlockIndex: 1,
+		BlockAt:    1710000051,
+	}
+	currentTx := Tx{
+		Signer:     mustPubKey("SysvarRent111111111111111111111111111111111"),
+		Block:      52,
+		BlockIndex: 2,
+		BlockAt:    1710000052,
+	}
+
+	merged, err := MergeTxsBinaryBytes([][]byte{
+		mustEncodeTxsBinaryV3(t, []Tx{legacyTx}),
+		mustEncodeTxsBinary(t, []Tx{currentTx}),
+	})
+	if err != nil {
+		t.Fatalf("MergeTxsBinaryBytes(v3,v4) error = %v", err)
+	}
+
+	var mergedBinary TxsBinary
+	if err := mergedBinary.UnmarshalBinary(merged); err != nil {
+		t.Fatalf("UnmarshalBinary(merged) error = %v", err)
+	}
+	if mergedBinary.SchemaVersion != txBinarySchemaVersionCurrent {
+		t.Fatalf("merged schema version = %d, want %d", mergedBinary.SchemaVersion, txBinarySchemaVersionCurrent)
+	}
+
+	decoded, err := DecodeTxsBinary(merged)
+	if err != nil {
+		t.Fatalf("DecodeTxsBinary(merged) error = %v", err)
+	}
+	if len(decoded) != 2 {
+		t.Fatalf("decoded len = %d, want 2", len(decoded))
+	}
+	if decoded[0].BlockAt != 0 {
+		t.Fatalf("legacy BlockAt = %d, want 0", decoded[0].BlockAt)
+	}
+	if decoded[1].BlockAt != currentTx.BlockAt {
+		t.Fatalf("current BlockAt = %d, want %d", decoded[1].BlockAt, currentTx.BlockAt)
+	}
+}
+
 func mustPubKey(value string) solana.PublicKey {
 	return solana.MustPublicKeyFromBase58(value)
 }
 
+func mustEncodeTxBinaryV3(t *testing.T, tx *Tx) []byte {
+	t.Helper()
+
+	binaryTx, err := NewTxBinary(tx)
+	if err != nil {
+		t.Fatalf("NewTxBinary() error = %v", err)
+	}
+	binaryTx.SchemaVersion = txBinarySchemaVersionV3
+	encoded, err := binaryTx.MarshalBinary()
+	if err != nil {
+		t.Fatalf("MarshalBinary(v3) error = %v", err)
+	}
+	return encoded
+}
+
+func mustEncodeTxsBinary(t *testing.T, txs []Tx) []byte {
+	t.Helper()
+
+	encoded, err := EncodeTxsBinary(txs)
+	if err != nil {
+		t.Fatalf("EncodeTxsBinary() error = %v", err)
+	}
+	return encoded
+}
+
+func mustEncodeTxsBinaryV3(t *testing.T, txs []Tx) []byte {
+	t.Helper()
+
+	binaryTxs, err := NewTxsBinary(txs)
+	if err != nil {
+		t.Fatalf("NewTxsBinary() error = %v", err)
+	}
+	binaryTxs.SchemaVersion = txBinarySchemaVersionV3
+	for i := range binaryTxs.Txs {
+		binaryTxs.Txs[i].SchemaVersion = txBinarySchemaVersionV3
+	}
+	encoded, err := binaryTxs.MarshalBinary()
+	if err != nil {
+		t.Fatalf("MarshalBinary(v3) error = %v", err)
+	}
+	return encoded
+}
+
 func mustTxBinary(t *testing.T, data []byte) *TxsBinary {
 	t.Helper()