pump-parser/README.md

# pump-parser

Solana transaction parser focused on swap, liquidity, migration, platform, MEV, compute budget, and compact binary persistence workflows.

The package works with a normalized `RawTx` representation, parses it into `Tx`, and emits one or more `Swap` records when a supported protocol action is found.

## Features

- Parse Solana RPC / Yellowstone transactions into local `RawTx`.
- Extract swap and liquidity events into `Tx.Swaps`.
- Preserve transaction metadata such as slot, block time, signer, fee, CU limit, CU consumed, token balances, platform fees, and MEV agent hints.
- Encode/decode parsed transactions with the `PTXB` / `PTXS` binary formats.
- Encode/decode raw transactions with the `PRTX` / `PRTS` / `PRBS` binary formats.
- Stream decode large `PTXS` / `PRTS` payloads without loading every transaction into memory.
- Merge `PTXS` batches while remapping address tables.

## Supported Parsers

Default parser initialization enables the hot-path Pump parsers only:

- Pump
- Pump AMM

`EnableAllParsers()` additionally enables:

- Meteora DLMM
- Meteora Pools
- Meteora DAMM v2
- Meteora Bonding Curve
- Orca Whirlpool
- Raydium AMM v4
- Raydium CLMM
- Raydium CPMM
- Raydium LaunchLab

Use `InitParser(WithMeteoraDlmm())` when only Meteora DLMM should be added to the default parser set.

## Installation

```bash
go get github.com/thloyi/pump-parser
```

This module currently declares `go 1.25.1` in `go.mod`.

## Basic Usage

```go
package main

import (
	"fmt"
	"log"

	pump_parser "github.com/thloyi/pump-parser"
)

func main() {
	// Enable all known parser programs. Omit this call to keep the default
	// Pump + Pump AMM parser set.
	pump_parser.EnableAllParsers()

	var rawTx *pump_parser.RawTx
	// Fill rawTx from RPC, Yellowstone, JSON, or RawTx binary decoding.

	tx, err := pump_parser.ParseRawTx(rawTx)
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println("tx:", tx.GetTxHash())
	for _, swap := range tx.Swaps {
		fmt.Printf("%s %s base=%s quote=%s pool=%s user=%s\n",
			swap.Program,
			swap.Event,
			swap.BaseAmount,
			swap.QuoteAmount,
			swap.Pool,
			swap.User,
		)
	}
}
```

## Swap Result

`ParseRawTx` returns a `Tx`. A transaction can contain zero, one, or multiple parsed swap records in `Tx.Swaps`.

Each `Swap` describes one protocol-level swap, liquidity, migration, or pool operation that the parser can normalize. The most important fields are:

| Field | Meaning |
| --- | --- |
| `Program` | Normalized protocol name, such as `Pump`, `PumpAMM`, `MeteoraDLMM`, `RaydiumV4`, or `OrcaWhirPool`. |
| `Event` | Normalized action, such as `buy`, `sell`, `add_liquidity`, `remove_liquidity`, `create`, `complete`, or `migrate`. |
| `TxIndex` | Approximate execution order across outer and inner instructions. |
| `InstrIdx` / `InnerIdx` | Outer instruction index and inner instruction index where the swap was found. |
| `Pool` | Pool, pair, bonding curve, or market account for the parsed action. |
| `BaseMint` / `QuoteMint` | Base and quote mint accounts after parser normalization. |
| `BaseTokenProgram` / `QuoteTokenProgram` | Token program for each side, useful when Token-2022 is involved. |
| `BaseMintDecimals` / `QuoteMintDecimals` | Decimals used to interpret raw token amounts. |
| `User` | User or effective owner account for the action. If the parsed user is not on-curve, the parser may fall back to the transaction signer. |
| `BaseAmount` / `QuoteAmount` | Actual parsed base-side and quote-side amounts, stored as `decimal.Decimal`. |
| `BaseReserve` / `QuoteReserve` | Pool reserves when the protocol event or accounts expose them. For Pump AMM buy/sell events, `QuoteReserve` is the post-trade effective quote reserve. |
| `RealQuoteReserve` / `VirtualQuoteReserve` | Quote-reserve components. Pump AMM buy/sell events expose the post-trade real reserve and the signed virtual reserve separately. |
| `UserBaseBalance` / `UserQuoteBalance` | User token balances after the transaction when available from token balance metadata. |
| `AfterSOLBalance` | User or signer SOL balance after the transaction. |
| `EntryContract` | Known router / entry contract account when detected. |
| `Mayhem` / `Cashback` | Protocol or platform-specific labels detected from the transaction path. |

Swap amount and slippage fields normalize instruction intent:

| Field | Meaning |
| --- | --- |
| `SwapMode` | `exact_in`, `exact_out`, or empty when unknown. |
| `FixedAmount` / `FixedAmountSide` / `FixedMint` | The user-specified fixed side of the swap. For `exact_in`, this is the input amount. For `exact_out`, this is the target output amount. |
| `LimitAmountType` | `min_out` for `exact_in`, `max_in` for `exact_out`, or empty when unknown. |
| `LimitAmount` / `LimitAmountSide` / `LimitMint` | User-specified limit on the opposite side. |
| `ActualLimitAmount` / `ActualLimitAmountSide` | Actual executed amount on the limited side. |
| `SlippageBps` | Remaining headroom to the user's limit in basis points. See `SLIPPAGE_MAPPING.md` for protocol-specific derivation rules. |

Liquidity, migration, and DLMM-specific fields are populated only for protocols that expose them:

| Field | Meaning |
| --- | --- |
| `Creator` | Creator account when exposed by pool or launch events. |
| `MigrateToPool` / `MigrateTopProgram` | Destination pool and program for migration events. |
| `LpMint` | LP token mint for liquidity operations when available. |
| `ActiveBinId` / `StartBinId` / `EndBinId` | Meteora DLMM bin identifiers. |
| `RemoveBp` | DLMM remove-liquidity basis points. |
| `PositionAccount` | DLMM position account. |
| `FeeAmount` / `LpFeeAmount` | Parsed fee amounts when the protocol exposes fee breakdowns. |
| `FeeSide` / `FeeMint` / `FeeTokenProgram` / `FeeMintDecimals` | Fee side and mint metadata. |
| `ConsumeUnit` | Per-swap compute unit value when available. |

Amounts are normalized into decimals in parser output, but the exact scale depends on the source event and parser path. For persisted binary output, `tx_binary.go` defines the conversion rules and schema version.

## Convert Transactions

RPC transactions can be converted with `FromRpcTransactionWithMeta`:

```go
rawTx, err := pump_parser.FromRpcTransactionWithMeta(
	txWithMeta,
	blockTime,
	slot,
	indexWithinBlock,
)
```

Yellowstone transactions can be converted with `ConvertYellowstoneGrpcTransactionToSolanaTransaction`:

```go
rawTx, err := pump_parser.ConvertYellowstoneGrpcTransactionToSolanaTransaction(
	update,
	createdUnix,
)
```

Both conversion paths accept optional `RawTxConvertOptions`:

```go
rawTx, err := pump_parser.FromRpcTransactionWithMeta(
	txWithMeta,
	blockTime,
	slot,
	indexWithinBlock,
	pump_parser.RawTxConvertOptions{
		ParseLogEvents: true,
	},
)
```

`ParseLogEvents` attaches decoded program log events to matching instructions. `IgnoreLogMessages` skips log-message retention.

## Binary Formats

Parsed transaction binary:

- `EncodeTxBinary` / `DecodeTxBinary` for one parsed `Tx`.
- `EncodeTxsBinary` / `DecodeTxsBinary` for a batch of parsed `Tx`.
- `DecodeTxsBinaryReader` for streaming `PTXS` reads.
- `MergeTxsBinaryBytes` and `MergeTxsBinarySourcesToWriter` for merging `PTXS` batches.
- Schema v5 persists `RealQuoteReserve` and signed `VirtualQuoteReserve`; v3/v4 inputs remain readable and default these components to legacy `QuoteReserve` and zero.

Raw transaction binary:

- `EncodeRawTxBinary` / `DecodeRawTxBinary` for one `RawTx`.
- `EncodeRawTxsBinary` / `DecodeRawTxsBinary` for a batch of `RawTx`.
- `EncodeRawTxBlocksBinary` / `DecodeRawTxBlocksBinary` for grouped block data.
- `DecodeRawTxsBinaryReader` for streaming `PRTS` reads.

Format magic values:

- `PTXB`: one parsed `Tx`.
- `PTXS`: parsed `Tx` batch.
- `PRTX`: one raw `RawTx`.
- `PRTS`: raw `RawTx` batch.
- `PRBS`: raw block-grouped `RawTx` batch.

When adding or renaming transaction-facing enum values, update `tx_binary.go` enum tables by appending new values only. Reordering existing values changes persisted numeric IDs.

## Commands

Parse a live transaction by signature:

```bash
TX_HASH=<signature> go run ./cmd/rpc_parse
```

Collect Yellowstone transactions into a `.prbs` raw-block binary file:

```bash
YELLOWSTONE_X_TOKEN=<token> go run ./cmd/collect_yellowstone_rawtx_binary \
  -endpoint ams.rpc.orbitflare.com:10000 \
  -duration 5m \
  -output testdata/rawtx-binary/sample.prbs
```

Measure parsed `Tx` binary size from a `getBlock` JSON payload:

```bash
go run ./cmd/measure_tx_binary_block \
  -file /path/to/block.json \
  -slot 413539056 \
  -swaps-only
```

Analyze raw transaction binary size distribution:

```bash
go run ./cmd/analyze_rawtx_binary_size \
  -file testdata/rawtx-binary/sample.prbs
```

## Development

Run the full test suite:

```bash
go test ./...
```

Useful focused checks:

```bash
go test -run 'TestTxBinary|TestRawTxBinary' .
go test -run 'TestMetaoraPoolSwapEventFromLogsUsesMatchingInvocation|TestMetaoraPoolSwapInstructionOccurrenceIncludesInnerInstructions|TestAttachLogEventsToInstructions' .
TX_HASH=<signature> go run ./cmd/rpc_parse
```

For parser regressions, reproduce with the exact transaction signature first, then add a targeted unit test or fixture around the corrected parser path.