yoloyolo/.planning/phases/04-power-user-features/04-01-PLAN.md

---
phase: 04-power-user-features
plan: 01
type: execute
wave: 1
depends_on: []
files_modified:
  - classify/types.go
  - capture/capture.go
  - capture/bpf.go
  - capture/pcap_reader.go
  - aggregate/window.go
  - capture/capture_test.go
  - capture/bpf_test.go
  - capture/pcap_reader_test.go
  - aggregate/window_test.go
autonomous: true
requirements: [CAPT-05, CAPT-06]

must_haves:
  truths:
    - "BPF filter expressions can be validated without a live socket"
    - "A valid pcap file can be read and packets emitted on a channel"
    - "Invalid/missing pcap files produce clear error messages"
    - "Pcap packets are aggregated into time windows using packet timestamps, not wall clock"
    - "Gap windows (no packets) produce empty snapshots preserving silence"
    - "BPF filter can be applied to live capture handles"
    - "Software BPF filtering works for pcap file packets"
  artifacts:
    - path: "capture/bpf.go"
      provides: "BPF validation and software filter compilation"
      exports: ["ValidateBPFFilter", "CompileSoftwareBPF"]
    - path: "capture/pcap_reader.go"
      provides: "Pcap file reading into packet channel"
      exports: ["ReadPcapFile"]
    - path: "aggregate/window.go"
      provides: "Timestamp-based aggregation for pcap mode"
      exports: ["AggregatePcap"]
    - path: "classify/types.go"
      provides: "Timestamp field on ClassifiedPacket"
      contains: "Timestamp time.Time"
  key_links:
    - from: "capture/pcap_reader.go"
      to: "gopacket/pcapgo"
      via: "pcapgo.NewReader"
      pattern: "pcapgo\\.NewReader"
    - from: "capture/bpf.go"
      to: "packetcap/go-pcap/filter"
      via: "filter.NewExpression"
      pattern: "filter\\.NewExpression"
    - from: "aggregate/window.go"
      to: "classify/types.go"
      via: "ClassifiedPacket.Timestamp for window assignment"
      pattern: "ev\\.Timestamp"
---

<objective>
Add core library functions for BPF filtering and pcap file reading.

Purpose: Build the foundation that Plan 02 wires into the CLI. BPF validation, pcap reading, software BPF matching, and timestamp-based aggregation are all independently testable library functions.
Output: New files capture/bpf.go, capture/pcap_reader.go; updated classify/types.go (Timestamp field), updated capture/capture.go (filter param on OpenCapture), updated aggregate/window.go (AggregatePcap); all with tests.
</objective>

<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>

<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/04-power-user-features/04-CONTEXT.md
@.planning/phases/04-power-user-features/04-RESEARCH.md

@capture/capture.go
@capture/capture_test.go
@classify/types.go
@classify/classifier.go
@aggregate/window.go
@aggregate/window_test.go

<interfaces>
<!-- Key types and contracts the executor needs. -->

From classify/types.go:
```go
type TrafficClass string
type ClassifiedPacket struct {
    Class    TrafficClass
    SrcPort  uint16
    DstPort  uint16
    Protocol string
    Length   int
}
type WindowSnapshot struct {
    Counts       map[TrafficClass]int64
    TotalPackets int64
    WindowIndex  int
}
```

From capture/capture.go:
```go
func OpenCapture(ctx context.Context, iface string) (*pcap.Handle, error)
func StartCapture(ctx context.Context, iface string) (<-chan gopacket.Packet, *int64, error)
```

From aggregate/window.go:
```go
const DefaultWindowMs = 500
func Aggregate(done <-chan struct{}, events <-chan classify.ClassifiedPacket, windowMs int, onSnapshot func(classify.WindowSnapshot)) <-chan classify.WindowSnapshot
```

From go-pcap (verified in module cache):
```go
// pcap.Handle method:
func (h *Handle) SetBPFFilter(expr string) error

// filter package:
func NewExpression(expr string) *Expression
func (e *Expression) Compile() *Filter
func (f *Filter) Compile() ([]bpf.Instruction, error)
```

From gopacket/pcapgo (verified in module cache):
```go
func NewReader(r io.Reader) (*Reader, error)
func (r *Reader) LinkType() layers.LinkType
// Reader implements gopacket.PacketDataSource
```

From golang.org/x/net/bpf:
```go
func NewVM(filter []bpf.Instruction) (*VM, error)
func (vm *VM) Run(in []byte) (int, error)  // >0 means packet passes
```
</interfaces>
</context>

<tasks>

<task type="auto" tdd="true">
  <name>Task 1: Add Timestamp to ClassifiedPacket, BPF validation, pcap reading, and software BPF filter</name>
  <files>classify/types.go, capture/bpf.go, capture/pcap_reader.go, capture/capture.go, capture/bpf_test.go, capture/pcap_reader_test.go, capture/capture_test.go</files>
  <read_first>classify/types.go, capture/capture.go, capture/capture_test.go, classify/classifier.go</read_first>
  <behavior>
    - TestValidateBPFFilter: ValidateBPFFilter("port 53") returns nil; ValidateBPFFilter("invalid garbage xyz") returns non-nil error containing "invalid BPF filter"
    - TestValidateBPFFilterEmpty: ValidateBPFFilter("") returns nil (empty = no filter)
    - TestCompileSoftwareBPF: CompileSoftwareBPF("tcp") returns a non-nil *bpf.VM and nil error
    - TestCompileSoftwareBPFInvalid: CompileSoftwareBPF("invalid garbage") returns nil VM and non-nil error
    - TestReadPcapFile: ReadPcapFile with a programmatically-generated pcap (using pcapgo.NewWriter to write 3 test packets) returns a channel that yields exactly 3 packets, then closes
    - TestReadPcapFileNotFound: ReadPcapFile("/nonexistent/file.pcap", "") returns error containing "cannot open"
    - TestReadPcapFileInvalid: ReadPcapFile on a non-pcap file (e.g., a temp file with garbage bytes) returns error containing "invalid pcap file" or "not a valid pcap"
    - TestReadPcapFileWithFilter: ReadPcapFile with filter="tcp" on a pcap containing both TCP and UDP packets returns only TCP packets
    - TestOpenCaptureWithFilter: OpenCapture now accepts a filter string parameter (signature change verified by compilation)
    - TestClassifiedPacketTimestamp: ClassifiedPacket struct has a Timestamp field of type time.Time
  </behavior>
  <action>
    **1. Update classify/types.go** — Add `Timestamp time.Time` field to `ClassifiedPacket`:
    ```go
    import "time"

    type ClassifiedPacket struct {
        Class     TrafficClass
        SrcPort   uint16
        DstPort   uint16
        Protocol  string
        Length    int
        Timestamp time.Time  // Set from pkt.Metadata().CaptureInfo.Timestamp; zero in live mode
    }
    ```
    This is a backward-compatible addition. Live mode leaves it zero-valued (no behavior change). Per D-01, pcap mode uses it for window assignment.

    **2. Create capture/bpf.go** — Two exported functions:

    ```go
    package capture

    import (
        "fmt"
        "strings"

        "golang.org/x/net/bpf"
        gpcapfilter "github.com/packetcap/go-pcap/filter"
    )

    // ValidateBPFFilter checks if a BPF expression string is valid without needing a live socket.
    // Returns nil for empty strings (empty = no filter). Per CAPT-05.
    func ValidateBPFFilter(expr string) error {
        if strings.TrimSpace(expr) == "" {
            return nil
        }
        e := gpcapfilter.NewExpression(expr)
        if e == nil {
            return fmt.Errorf("invalid BPF filter expression: %q", expr)
        }
        compiled := e.Compile()
        if _, err := compiled.Compile(); err != nil {
            return fmt.Errorf("invalid BPF filter %q: %v", expr, err)
        }
        return nil
    }

    // CompileSoftwareBPF compiles a BPF expression into a VM for user-space packet matching.
    // Used for --read mode where kernel BPF is unavailable. Per D-04 (filter works with --read).
    func CompileSoftwareBPF(expr string) (*bpf.VM, error) {
        e := gpcapfilter.NewExpression(expr)
        if e == nil {
            return nil, fmt.Errorf("invalid BPF filter expression: %q", expr)
        }
        instructions, err := e.Compile().Compile()
        if err != nil {
            return nil, fmt.Errorf("BPF compile error for %q: %v", expr, err)
        }
        return bpf.NewVM(instructions)
    }
    ```

    **3. Create capture/pcap_reader.go** — ReadPcapFile function:

    ```go
    package capture

    import (
        "fmt"
        "os"

        "github.com/gopacket/gopacket"
        "github.com/gopacket/gopacket/layers"
        "github.com/gopacket/gopacket/pcapgo"
        "golang.org/x/net/bpf"
    )

    // ReadPcapFile opens a pcap file and returns a channel of packets.
    // If filter is non-empty, applies software BPF filtering (kernel BPF unavailable for files).
    // Per D-03: replaces -i as packet source. Per D-04: --filter works with --read.
    func ReadPcapFile(path string, filter string) (<-chan gopacket.Packet, error) {
        f, err := os.Open(path)
        if err != nil {
            return nil, fmt.Errorf("cannot open pcap file %q: %w", path, err)
        }
        r, err := pcapgo.NewReader(f)
        if err != nil {
            f.Close()
            return nil, fmt.Errorf("not a valid pcap file %q: %w", path, err)
        }

        // Compile software BPF filter if provided
        var vm *bpf.VM
        if filter != "" {
            vm, err = CompileSoftwareBPF(filter)
            if err != nil {
                f.Close()
                return nil, err
            }
        }

        lt := layers.LinkType(r.LinkType())
        packetSource := gopacket.NewPacketSource(r, lt)
        packetSource.NoCopy = false  // CRITICAL: do NOT use NoCopy=true with pcapgo (Pitfall 3)

        packets := make(chan gopacket.Packet, 512)
        go func() {
            defer close(packets)
            defer f.Close()
            for pkt := range packetSource.Packets() {
                // Apply software BPF filter if active
                if vm != nil {
                    result, err := vm.Run(pkt.Data())
                    if err != nil || result == 0 {
                        continue // packet does not match filter
                    }
                }
                packets <- pkt
            }
        }()
        return packets, nil
    }
    ```

    **4. Update capture/capture.go** — Add `filter string` parameter to `OpenCapture` and `StartCapture`. In `OpenCapture`, after successfully opening the handle, call `handle.SetBPFFilter(filter)` if filter is non-empty. If SetBPFFilter fails, close the handle and return error wrapped with `fmt.Errorf("invalid BPF filter %q: %w", filter, err)`. Update `StartCapture` signature to `StartCapture(ctx context.Context, iface string, filter string)` and pass filter through to `OpenCapture`.

    **5. Create test files:**
    - `capture/bpf_test.go` — Tests for ValidateBPFFilter and CompileSoftwareBPF per behavior list above.
    - `capture/pcap_reader_test.go` — Uses `pcapgo.NewWriter` to create temporary pcap files programmatically (no binary fixtures). Write a helper `createTestPcap(t *testing.T, packets [][]byte) string` that creates a temp file with pcapgo.NewWriter, writes Ethernet+IP+TCP/UDP raw packets, and returns the path. Test ReadPcapFile with valid pcap, nonexistent file, invalid file (garbage bytes), and filter.
    - `capture/capture_test.go` — Add TestOpenCaptureSignature that verifies the new signature compiles (the existing tests don't call OpenCapture directly due to privilege requirements; just ensure the file compiles with the new signature).

    **6. Update cmd/netsynth/main.go call site** — The call `capture.StartCapture(ctx, ifaceName)` must become `capture.StartCapture(ctx, ifaceName, "")` to match the updated signature. This is a minimal change to keep existing code compiling. (The actual --filter flag wiring happens in Plan 02.)
  </action>
  <verify>
    <automated>cd /home/dev/workspace/yoloyolo && export PATH="/home/dev/tools/go-install/go/bin:$PATH" && go test ./capture/... -run "TestValidateBPF|TestCompileSoftware|TestReadPcap|TestClassifiedPacket" -v -count=1 && go build ./...</automated>
  </verify>
  <acceptance_criteria>
    - classify/types.go contains `Timestamp time.Time` inside ClassifiedPacket struct
    - capture/bpf.go contains `func ValidateBPFFilter(expr string) error`
    - capture/bpf.go contains `func CompileSoftwareBPF(expr string) (*bpf.VM, error)`
    - capture/pcap_reader.go contains `func ReadPcapFile(path string, filter string) (<-chan gopacket.Packet, error)`
    - capture/pcap_reader.go contains `packetSource.NoCopy = false`
    - capture/capture.go contains `func OpenCapture(ctx context.Context, iface string, filter string)`
    - capture/capture.go contains `handle.SetBPFFilter(filter)`
    - capture/capture.go contains `func StartCapture(ctx context.Context, iface string, filter string)`
    - cmd/netsynth/main.go contains `capture.StartCapture(ctx, ifaceName, "")`
    - `go test ./capture/... -v` passes all new tests
    - `go build ./...` succeeds (all packages compile)
  </acceptance_criteria>
  <done>BPF validation rejects invalid expressions and accepts valid ones. Pcap reader opens valid files and returns packet channels, rejects invalid/missing files. Software BPF filters pcap packets. OpenCapture/StartCapture accept filter parameter. ClassifiedPacket has Timestamp field. All tests pass, all packages compile.</done>
</task>

<task type="auto" tdd="true">
  <name>Task 2: Timestamp-based pcap aggregation with gap-filling</name>
  <files>aggregate/window.go, aggregate/window_test.go</files>
  <read_first>aggregate/window.go, aggregate/window_test.go, classify/types.go</read_first>
  <behavior>
    - TestAggregatePcapBasic: AggregatePcap with 3 packets at times T+0ms, T+100ms, T+200ms (all in window 0 for 500ms windows) returns 1 snapshot with TotalPackets=3
    - TestAggregatePcapMultipleWindows: AggregatePcap with packets at T+0ms, T+600ms, T+1200ms returns 3 snapshots (window 0, 1, 2) each with TotalPackets=1
    - TestAggregatePcapGaps: AggregatePcap with packets at T+0ms and T+1500ms (skip window 1 and 2) returns 4 snapshots: window 0 with 1 packet, windows 1-2 with TotalPackets=0, window 3 with 1 packet (per D-02: gaps are silent)
    - TestAggregatePcapEmpty: AggregatePcap with empty channel returns empty slice (no snapshots)
    - TestAggregatePcapWindowIndex: Each returned snapshot has correct sequential WindowIndex 0,1,2,...
    - TestAggregatePcapClassCounts: Packets of different classes in same window have correct per-class counts in snapshot.Counts
    - TestAggregatePcapOnSnapshot: If onSnapshot callback is provided, it is called for each emitted snapshot (for --verbose support per D-07)
  </behavior>
  <action>
    Add `AggregatePcap` function to `aggregate/window.go`. This function reads from a `<-chan classify.ClassifiedPacket` (same channel type as live mode), collects all events into a slice, then assigns each to a time window using `pkt.Timestamp.Sub(firstTimestamp).Milliseconds() / windowMs`.

    **Function signature:**
    ```go
    // AggregatePcap reads all ClassifiedPackets (with Timestamp set), assigns to time windows
    // using packet timestamps per D-01, fills gap windows with empty snapshots per D-02,
    // and returns all WindowSnapshots. Calls onSnapshot for each if non-nil (D-07: --verbose).
    // Returns synchronously after channel closes (pcap processing is finite).
    func AggregatePcap(events <-chan classify.ClassifiedPacket, windowMs int, onSnapshot func(classify.WindowSnapshot)) []classify.WindowSnapshot
    ```

    **Implementation details:**
    1. Drain the `events` channel into a `[]classify.ClassifiedPacket` slice.
    2. If slice is empty, return `nil` (empty slice).
    3. Find `minTimestamp` by scanning all events (handles non-monotonic pcaps per Pitfall 5).
    4. Compute `maxWindowIdx` = `int(maxTimestamp.Sub(minTimestamp).Milliseconds()) / windowMs`.
    5. Create `snapshots := make([]classify.WindowSnapshot, maxWindowIdx+1)`. Initialize each with `Counts: make(map[classify.TrafficClass]int64)` and `WindowIndex: i`.
    6. For each event, compute `idx := int(ev.Timestamp.Sub(minTimestamp).Milliseconds()) / windowMs`. Clamp to `[0, maxWindowIdx]` for safety. Increment `snapshots[idx].Counts[ev.Class]++` and `snapshots[idx].TotalPackets++`.
    7. If `onSnapshot` is non-nil, call it for each snapshot in order (supports --verbose per D-07).
    8. Return `snapshots`.

    **Tests in aggregate/window_test.go** — Add test functions per behavior list. Use helper that creates `classify.ClassifiedPacket` values with specific `Timestamp` values relative to a base time `time.Date(2024, 1, 1, 0, 0, 0, 0, time.UTC)`. Send packets into a buffered channel, close it, then call `AggregatePcap`.
  </action>
  <verify>
    <automated>cd /home/dev/workspace/yoloyolo && export PATH="/home/dev/tools/go-install/go/bin:$PATH" && go test ./aggregate/... -run "TestAggregatePcap" -v -count=1</automated>
  </verify>
  <acceptance_criteria>
    - aggregate/window.go contains `func AggregatePcap(events <-chan classify.ClassifiedPacket, windowMs int, onSnapshot func(classify.WindowSnapshot)) []classify.WindowSnapshot`
    - aggregate/window_test.go contains `TestAggregatePcapBasic`
    - aggregate/window_test.go contains `TestAggregatePcapGaps`
    - aggregate/window_test.go contains `TestAggregatePcapEmpty`
    - `go test ./aggregate/... -run TestAggregatePcap -v` shows all tests PASS
    - Gap test verifies that windows with no packets have TotalPackets=0
  </acceptance_criteria>
  <done>AggregatePcap assigns packets to correct time windows using timestamps (D-01). Gap windows produce empty snapshots with TotalPackets=0 (D-02). onSnapshot callback fires for each window (D-07). All aggregation tests pass.</done>
</task>

</tasks>

<verification>
Full package compilation and test suite:
```bash
cd /home/dev/workspace/yoloyolo && export PATH="/home/dev/tools/go-install/go/bin:$PATH" && go test ./... -count=1 && go build ./...
```
All existing tests must continue to pass. All new tests must pass. `go build ./...` must succeed.
</verification>

<success_criteria>
1. `ValidateBPFFilter("port 53")` returns nil; `ValidateBPFFilter("invalid xyz")` returns error
2. `ReadPcapFile` reads programmatically-generated pcap files and emits correct packet count
3. `ReadPcapFile` with filter only passes matching packets
4. `AggregatePcap` assigns packets to correct windows and fills gaps with empty snapshots
5. `OpenCapture` and `StartCapture` accept filter parameter
6. `ClassifiedPacket` has `Timestamp time.Time` field
7. `go test ./...` all green, `go build ./...` succeeds
</success_criteria>

<output>
After completion, create `.planning/phases/04-power-user-features/04-01-SUMMARY.md`
</output>