gurix
0f15d9fca4
- Create 04-01-SUMMARY.md for BPF validation, pcap reading, timestamp aggregation - Update STATE.md: advance plan, record metrics, add decisions, update session - Update ROADMAP.md: phase 4 progress to 1/2 plans complete - Mark CAPT-05, CAPT-06 requirements complete in REQUIREMENTS.md
95 lines
4.5 KiB
Markdown
95 lines
4.5 KiB
Markdown
---
|
|
phase: 04-power-user-features
|
|
plan: "01"
|
|
subsystem: capture/aggregate/classify
|
|
tags: [bpf, pcap-reader, timestamp, aggregation, gap-filling]
|
|
dependency_graph:
|
|
requires: [03-02]
|
|
provides: [ValidateBPFFilter, CompileSoftwareBPF, ReadPcapFile, AggregatePcap, ClassifiedPacket.Timestamp]
|
|
affects: [capture, aggregate, classify, cmd/netsynth]
|
|
tech_stack:
|
|
added: []
|
|
patterns:
|
|
- software BPF VM (golang.org/x/net/bpf) for user-space packet filtering
|
|
- pcapgo.NewReader for pcap file parsing without live socket
|
|
- timestamp-based window assignment replacing wall-clock ticker
|
|
key_files:
|
|
created:
|
|
- capture/bpf.go
|
|
- capture/bpf_test.go
|
|
- capture/pcap_reader.go
|
|
- capture/pcap_reader_test.go
|
|
modified:
|
|
- classify/types.go
|
|
- capture/capture.go
|
|
- aggregate/window.go
|
|
- aggregate/window_test.go
|
|
- cmd/netsynth/main.go
|
|
decisions:
|
|
- OpenCapture/StartCapture accept filter string; empty string = no filter (backward compatible)
|
|
- Software BPF via golang.org/x/net/bpf.VM for pcap file filtering (kernel BPF unavailable for files)
|
|
- AggregatePcap drains entire channel synchronously then assigns to windows (vs streaming)
|
|
- Gap windows initialized with make(map) at creation time to avoid nil map panics
|
|
metrics:
|
|
duration: "3 minutes"
|
|
completed_date: "2026-03-26"
|
|
tasks_completed: 2
|
|
files_changed: 9
|
|
---
|
|
|
|
# Phase 04 Plan 01: BPF validation, pcap reading, and timestamp-based aggregation
|
|
|
|
**One-liner:** Core library functions for BPF filter validation, pcap file reading with software BPF, Timestamp field on ClassifiedPacket, and offline timestamp-based windowing with gap-filling silence.
|
|
|
|
## What Was Built
|
|
|
|
### classify/types.go
|
|
Added `Timestamp time.Time` field to `ClassifiedPacket`. This is a backward-compatible addition — live mode leaves it zero-valued (no behavior change). Pcap mode sets it from packet metadata for window assignment.
|
|
|
|
### capture/bpf.go
|
|
Two exported functions:
|
|
- `ValidateBPFFilter(expr string) error` — validates BPF expressions without a live socket using go-pcap's filter package; empty strings return nil (no filter)
|
|
- `CompileSoftwareBPF(expr string) (*bpf.VM, error)` — compiles BPF to user-space VM for pcap file filtering where kernel BPF is unavailable
|
|
|
|
### capture/pcap_reader.go
|
|
`ReadPcapFile(path, filter string) (<-chan gopacket.Packet, error)` — opens a pcap file via `pcapgo.NewReader`, optionally applies a software BPF VM filter, emits packets on a buffered channel, closes channel when done. Sets `NoCopy = false` per Pitfall 3 (pcapgo requires owned buffers).
|
|
|
|
### capture/capture.go
|
|
Updated `OpenCapture` and `StartCapture` to accept a `filter string` parameter. `OpenCapture` calls `handle.SetBPFFilter(filter)` if filter is non-empty, returning a wrapped error on failure.
|
|
|
|
### aggregate/window.go
|
|
Added `AggregatePcap` — synchronously drains the events channel, assigns packets to time windows using `Timestamp.Sub(minTimestamp).Milliseconds()/windowMs`, fills gaps with empty snapshots (TotalPackets=0), fires `onSnapshot` callback per window for `--verbose` support. Returns `[]WindowSnapshot`.
|
|
|
|
## Tests Added
|
|
|
|
| File | Tests |
|
|
|------|-------|
|
|
| capture/bpf_test.go | TestValidateBPFFilter, TestValidateBPFFilterEmpty, TestValidateBPFFilterWhitespace, TestValidateBPFFilterInvalid, TestCompileSoftwareBPF, TestCompileSoftwareBPFInvalid |
|
|
| capture/pcap_reader_test.go | TestReadPcapFile, TestReadPcapFileNotFound, TestReadPcapFileInvalid, TestReadPcapFileWithFilter, TestClassifiedPacketTimestamp |
|
|
| aggregate/window_test.go | TestAggregatePcapBasic, TestAggregatePcapMultipleWindows, TestAggregatePcapGaps, TestAggregatePcapEmpty, TestAggregatePcapWindowIndex, TestAggregatePcapClassCounts, TestAggregatePcapOnSnapshot |
|
|
|
|
## Commits
|
|
|
|
| Task | Commit | Description |
|
|
|------|--------|-------------|
|
|
| Task 1 | 52c6010 | feat(04-01): BPF validation, pcap reading, Timestamp field, filter support |
|
|
| Task 2 | d13844f | feat(04-01): AggregatePcap with timestamp-based windowing and gap-filling |
|
|
|
|
## Deviations from Plan
|
|
|
|
None — plan executed exactly as written.
|
|
|
|
## Known Stubs
|
|
|
|
None. All functions are fully wired and functional. Plan 02 will wire them into the CLI (--filter and --read flags).
|
|
|
|
## Self-Check: PASSED
|
|
|
|
Files exist:
|
|
- /home/dev/workspace/yoloyolo/capture/bpf.go: FOUND
|
|
- /home/dev/workspace/yoloyolo/capture/pcap_reader.go: FOUND
|
|
- /home/dev/workspace/yoloyolo/aggregate/window.go: FOUND (AggregatePcap added)
|
|
- /home/dev/workspace/yoloyolo/classify/types.go: FOUND (Timestamp field added)
|
|
|
|
Commits verified: 52c6010, d13844f in git log.
|