Files
yoloyolo/.planning/research/PITFALLS.md
T
2026-03-24 22:36:28 +01:00

23 KiB

Pitfalls Research

Domain: Network-traffic-to-audio synthesis CLI tool (Go) Researched: 2026-03-24 Confidence: HIGH (packet capture / CGo pitfalls verified against official issues and docs; audio synthesis pitfalls cross-referenced against encoder project post-mortems and DSP literature)


Critical Pitfalls

Pitfall 1: Using google/gopacket Instead of the Active Community Fork

What goes wrong: The original github.com/google/gopacket repository is unmaintained. Bugs go unpatched, open PRs accumulate, and compatibility with newer Go versions degrades. Projects that import it are pinned to a stale library.

Why it happens: google/gopacket has enormous search mindshare and most tutorials still reference it. Developers reach for the first result without checking maintenance status.

How to avoid: Import github.com/gopacket/gopacket (the community fork, v1.5.0 released November 2025, requires Go 1.24+). Major projects including Cilium have already migrated. Treat google/gopacket as deprecated.

Warning signs:

  • go.mod referencing github.com/google/gopacket
  • Build errors on Go 1.21+ not fixed upstream

Phase to address: Phase 1 (packet capture scaffolding) — set the correct import path from day one; migrating later is a find-and-replace across the whole codebase.


Pitfall 2: CGo Destroys the "Single Binary" Promise

What goes wrong: gopacket/pcap requires libpcap via CGo. By default Go produces a dynamically linked binary. On a target machine without libpcap.so installed, the binary silently or loudly fails with error while loading shared libraries: libpcap.so.0.8. The "just copy the binary" distribution story breaks completely.

Why it happens: CGo is enabled by default and Go gives no compile-time warning that the resulting binary has a runtime C dependency. The binary runs perfectly on the build machine (which has libpcap-dev installed) and fails on clean machines.

How to avoid: Choose one of these strategies before writing a line of capture code:

  1. Fully static build: CGO_ENABLED=1 go build -ldflags "-linkmode 'external' -extldflags '-static'" with libpcap.a present. Requires musl-gcc or equivalent on Alpine/musl.
  2. pcapgo (pure Go): gopacket/pcapgo provides an EthernetHandle that avoids CGo entirely — lower performance but zero C dependency. Sufficient for ambient audio capture at non-Gbps rates.
  3. Document the dependency explicitly: If CGo/dynamic linking is accepted, README must state "requires libpcap (apt install libpcap-dev / brew install libpcap)".

Warning signs:

  • CGO_ENABLED not explicitly set in your build script
  • ldd ./netsynth shows libpcap.so as a dependency
  • No CI test on a minimal (Alpine, scratch Docker) container

Phase to address: Phase 1 — this is a foundational architecture decision. Changing from dynamic to static after the fact is painful and causes build pipeline rewrites.


Pitfall 3: CAP_NET_RAW + Binary Location = Silent Failure on Linux

What goes wrong: On Ubuntu and many Linux distributions, setcap cap_net_raw+eip ./netsynth appears to succeed but the binary fails at runtime if it lives in /home/user/bin, /tmp, or any filesystem mounted nosuid. The kernel silently ignores the capability. AppArmor compounds this by enforcing path-based restrictions.

Why it happens: Developers test from their build directory (~/projects/netsynth/) — a path frequently on a nosuid filesystem. The tool appears broken with no clear error message beyond "permission denied" or "you must be root."

How to avoid:

  • Install to /usr/local/bin or /usr/bin for capability-based operation
  • Document two run modes: sudo ./netsynth (always works) vs. setcap (requires standard path)
  • In the CLI, detect permission failure and emit a clear message: "Packet capture requires root or CAP_NET_RAW. Run as root or: sudo setcap cap_net_raw+eip $(which netsynth)"
  • Test capability-mode explicitly from a non-home path in CI

Warning signs:

  • Testing only via sudo go run .
  • No test of the installed-binary path in README instructions
  • macOS-only development (macOS uses a different privilege model; Linux nosuid behavior won't surface)

Phase to address: Phase 1 (capture scaffolding) and the CLI UX phase — the error message is user-facing and needs to be explicit.


Pitfall 4: Packet Buffer Overflow Under Moderate Traffic Load

What goes wrong: At high packet rates (busy LAN, server NIC), gopacket's kernel ring buffer fills faster than the processing goroutine consumes it. The OS drops packets silently. The tool appears to work, but 50-98% of packets never reach the classifier. The audio output misrepresents actual traffic.

Why it happens: The default pcap buffer is 1-2 MB. Each packet triggers a cgo call (with pcap backend), creating per-packet overhead that compounds at speed. Developers test on quiet home networks and never observe drops.

How to avoid:

  • Set a large capture buffer explicitly: handle.SetBufferSize(32 * 1024 * 1024) (32 MB)
  • Use a non-blocking channel between capture and classification goroutines with a buffer of at least 1000 packets; drop metrics count drops so they are visible
  • For high-throughput scenarios, prefer afpacket backend over pcapafpacket eliminates per-packet CGo calls and dramatically improves throughput (benchmark: 1.27 MB/s → 21.17 MB/s)
  • NetSynth's ambient audio goal tolerates lossy capture — document this explicitly so users understand the tool provides a statistical fingerprint, not a perfect census

Warning signs:

  • Capture and classification in a single goroutine
  • No SetBufferSize call
  • Testing only on loopback (lo) which has near-zero real packet rates

Phase to address: Phase 1/2 (capture pipeline) — the goroutine architecture must be designed for async processing from the start. Retrofitting is a significant rewrite.


Pitfall 5: ZeroCopy Packet Data Use-After-Free

What goes wrong: ZeroCopyReadPacketData() returns a slice pointing into a buffer owned by the pcap handle. The next call to ZeroCopyReadPacketData() invalidates the previous slice's backing memory. If any goroutine holds a reference to old packet bytes and reads them after the next call, it reads corrupted or incorrect data. This produces silent data corruption — wrong protocol classifications, no crash.

Why it happens: The zero-copy API looks identical to the copying API. Developers reach for it for performance without reading the "each call invalidates previous data" contract.

How to avoid: Use ReadPacketData() (copies data) unless you have profiling evidence that allocation is a bottleneck. If ZeroCopyReadPacketData() is used, never pass the slice to another goroutine without first copying it: data := append([]byte(nil), raw...).

Warning signs:

  • ZeroCopyReadPacketData in a goroutine-per-packet pattern
  • Intermittent wrong protocol classifications that are not reproducible
  • Using gopacket.Lazy decode with concurrent goroutines (the gopacket docs explicitly warn against this combination)

Phase to address: Phase 1 (capture/decode) — establish the correct API choice at the read loop level.


Pitfall 6: MP3 Output Is Corrupt or Unplayable Due to LAME Initialization Errors

What goes wrong: MP3 encoding via CGo LAME bindings requires calling InitParams() after setting all encoder parameters. Skipping or reordering this call produces a file with a valid .mp3 extension that most players refuse to open or that plays as noise. The encoder returns no error from the encode calls themselves.

Why it happens: The LAME C API is stateful and order-dependent. Go wrappers vary in how much they enforce initialization order. Many tutorial examples show minimal code that happens to work for 44100 Hz stereo but silently breaks for other configurations.

How to avoid:

  • Always call InitParams() before writing any frames
  • Restrict to known-safe parameters: sample rate 44100 or 48000, stereo or mono (LAME does not support dual-channel mode)
  • Write a single integration test that encodes 1 second of silence and confirms the output file is valid (use mp3val or ffprobe in CI)
  • Consider shine-mp3 (pure Go port) as an alternative that eliminates CGo entirely; output files are larger but the library has no C initialization state

Warning signs:

  • No test that validates the output MP3 with an external tool
  • Sample rate set to anything other than 44100 or 48000
  • Encoder parameters set after InitParams() has been called

Phase to address: Audio synthesis / encoding phase — establish the encode pipeline with an end-to-end smoke test (silence → valid MP3) before wiring up synthesis.


Pitfall 7: PCM Sample Overflow Produces Wrap-Around Distortion

What goes wrong: Synthesizing audio as int16 samples and summing multiple sine layers without clamping causes integer overflow. The value wraps around (e.g., 32767 + 100 = -32667 in int16), producing a sharp click or a buzzing distortion that corrupts the ambient soundscape. This is not clipping — it is a distinctly worse artifact.

Why it happens: Developers model audio math in their head as real-valued floats, implement it in int16 for "efficiency," and forget that Go integer overflow is undefined-behavior-free but still wraps. With 6-8 drone layers simultaneously active, summing them easily exceeds ±32767.

How to avoid: Synthesize internally in float64 in the range [-1.0, 1.0]. Apply a normalisation/soft-limiter pass before converting to int16 for encoding. Clamp before cast: sample := int16(math.Max(-1.0, math.Min(1.0, floatSample)) * 32767). Never do mixed-type audio math that passes through int16 as an intermediate.

Warning signs:

  • Audio synthesis structs storing amplitude as int16 or int32
  • Adding layer outputs with += without a final normalisation step
  • Distorted output that correlates with traffic spikes (more active layers = more overflow)

Phase to address: Audio synthesis phase — establish the internal sample representation as float64 from the start.


Pitfall 8: Tone-per-Protocol Mapping Produces Perceptual Chaos

What goes wrong: Assigning arbitrary frequencies to protocols (e.g., DNS=440 Hz, HTTPS=880 Hz, ICMP=1320 Hz, SSH=1760 Hz, 6 auto-clusters=random) creates a soundscape where all tones are in the same frequency range, fighting each other. At moderate traffic the result is an undifferentiated buzz rather than distinct recognizable layers.

Why it happens: Developers choose frequencies programmatically (e.g., multiples of a base frequency) without considering auditory scene analysis — the human perceptual process by which listeners separate simultaneous sounds into distinct streams. Sounds too close in frequency mask each other.

How to avoid: Space protocol tones across register bands: low drones (80-200 Hz) for high-volume background traffic (HTTPS bulk), mid tones (300-600 Hz) for control traffic (DNS, NTP), high tones (800-1600 Hz) for interactive protocols (SSH, ICMP). Use harmonic or musical intervals (octaves, fifths) rather than arithmetic spacing. Keep auto-cluster frequencies in the 200-500 Hz mid-range so they don't obscure the "signature" tones. Limit simultaneous active layers to avoid masking.

Warning signs:

  • Frequency assignments as an arithmetic sequence: baseFreq + n*200
  • No perceptual test — only waveform-level correctness checks
  • Auto-cluster frequencies chosen randomly from the full audible range

Phase to address: Audio mapping / synthesis phase — the frequency mapping table should be designed up front with the perceptual goals in mind, not patched after "it sounds like noise" feedback.


Pitfall 9: Time Window Too Short — Unstable, Jittery Audio

What goes wrong: Aggregating traffic into windows shorter than ~500ms causes rapid amplitude oscillation in the synthesized drones. A single ICMP ping becomes a brief tone burst; a DNS query causes a momentary volume spike. The output sounds jittery and event-driven rather than ambient.

Why it happens: Developers choose a "natural" update interval (100ms or 200ms matches CPU scheduling intuition) without considering audio envelope times. Human perception of tonal stability requires note durations of at least 200-500ms; drones need even longer.

How to avoid:

  • Use a minimum window of 500ms for amplitude updates; 1-2s for tonal shifts
  • Apply amplitude smoothing (exponential moving average with a decay of ~2-5s) so a single-packet burst doesn't cause an immediate amplitude jump
  • Separate the "data collection" window (can be shorter) from the "audio parameter update" window (should be longer)

Warning signs:

  • time.Tick(100 * time.Millisecond) driving audio parameter updates
  • No smoothing/interpolation between amplitude values
  • Testing with ping floods (bursty) rather than continuous traffic

Phase to address: Traffic aggregation / audio mapping phase — establish the window and smoothing strategy before wiring traffic data to audio parameters.


Technical Debt Patterns

Shortcut Immediate Benefit Long-term Cost When Acceptable
google/gopacket instead of gopacket/gopacket Familiar, more tutorials Unmaintained; Go compat breaks Never
sudo ./netsynth only, no setcap docs Simpler setup instructions Users won't run as root in practice; tool appears broken MVP only — document the limitation
Dynamic libpcap linking (no static build) Faster to compile Binary doesn't work on target machines without libpcap installed Only acceptable if distributing via package manager that declares the dep
ReadPacketData (copying) instead of ZeroCopy Safe, simple ~20% memory overhead at high packet rates Always acceptable; optimize only if profiling proves allocation bottleneck
Sine-wave-only synthesis (no ADSR, no envelope) Much simpler code Tonal changes are abrupt, not perceptually smooth Acceptable for v1 ambient/drone if EMA smoothing is applied to amplitude
Hard-coded frequency table (no config) No CLI complexity Can't tune without recompiling Acceptable for v1 per PROJECT.md out-of-scope decision

Integration Gotchas

Integration Common Mistake Correct Approach
gopacket/pcap handle Not calling handle.Close() on signal — leaks capture resources Use defer handle.Close() and ensure the goroutine exits before process termination
LAME CGo encoder Not flushing the encoder before closing — truncated final MP3 frame Call encoder.Flush() / lame.EncodeFlush() after the sample loop ends
OS signal handling (SIGINT) Goroutine receives SIGINT but the capture loop is blocked on ReadPacketData Use handle.SetReadDeadline(time.Now()) or close the handle to unblock
MP3 encoder sample format Passing float64 samples directly to LAME (expects int16 or float32 depending on binding) Explicitly convert and clamp to the binding's expected type; check each binding's API
pcapgo.EthernetHandle Only captures Ethernet frames — fails on WiFi (802.11), loopback, or tunnel interfaces For non-Ethernet interfaces, use the pcap backend or check link type at startup

Performance Traps

Trap Symptoms Prevention When It Breaks
Single goroutine: capture + classify + synthesize CPU-bound synthesis blocks packet reads; drops spike under any real traffic Three-stage pipeline: capture goroutine → classify channel → synthesis goroutine Breaks on any network with > ~1000 pps
One goroutine per packet Goroutine creation overhead exceeds packet processing time; OOM on busy networks Channel-based batching: one reader, N classifiers from a worker pool Breaks above ~10k pps
Recomputing sine wave sample-by-sample in inner loop using math.Sin CPU pegged at 100% during synthesis; output can't keep pace Precompute wavetable per frequency; iterate with phase accumulator Breaks with > 4-5 simultaneous drone layers at 44100 Hz
Blocking channel between capture and synthesis with no buffer Any synthesis stall causes packet drops Buffered channel of 1000+ packets; separate goroutines Breaks immediately on any CPU scheduling hiccup

Security Mistakes

Mistake Risk Prevention
Requesting full root and keeping it throughout capture Privilege escalation if a parsing bug in gopacket can be exploited via crafted packets Drop privileges after opening the capture handle: syscall.Setuid(originalUID)
Promiscuous mode on by default without user opt-in Captures all LAN traffic, not just traffic to/from the host — legal and privacy risk on shared networks Default to non-promiscuous; add --promiscuous flag with a warning message
No limit on capture duration or file size Unbounded run produces an arbitrarily large MP3 or consumes all memory in the aggregator maps Add --max-duration flag (default: warn at 10min, hard limit at 1hr); prune old flow state periodically
Logging decoded packet payloads in debug mode Inadvertently logs credentials or private data Never log packet payload bytes; log only headers and metadata

UX Pitfalls

Pitfall User Impact Better Approach
Silent failure when interface doesn't exist User specifies -i eth1 on a machine with only ens3; tool exits with cryptic libpcap error List available interfaces at startup with pcap.FindAllDevs() and suggest correct name
No progress feedback during capture User has no idea if the tool is working; assumes it hung Print periodic status line: "Capturing... 1,234 packets classified (HTTPS:45% DNS:30% ICMP:8% other:17%)"
Output MP3 path collision without warning Re-running overwrites previous output Warn if output file exists; suggest timestamped default filename
Ctrl+C produces empty or invalid MP3 User interrupts too quickly before any traffic is captured Detect zero-packet case and emit an error instead of an empty file
No indication of which interface is being captured Confusing when multiple interfaces exist Print "Capturing on: eth0 (192.168.1.5)" at startup

"Looks Done But Isn't" Checklist

  • Packet capture: Binary runs as non-root user with setcap — verify from a non-/home path, not just from the build directory
  • MP3 output: File validates with ffprobe or mp3val — not just "has .mp3 extension and non-zero size"
  • Static binary: ldd ./netsynth shows "not a dynamic executable" (or explicitly "requires libpcap" if dynamic is accepted)
  • Signal handling: Ctrl+C during capture produces a valid (playable) MP3, not a truncated file
  • High-traffic: Drop counter is zero (or documented/acceptable) when tested against a network with > 1000 pps
  • Audio layers: Output with 6+ simultaneous traffic types does not distort — no wrap-around clipping audible
  • Empty capture: Graceful error message when zero packets were captured, not a silent empty file
  • Interface not found: Helpful error with available interface list, not a libpcap raw error string

Recovery Strategies

Pitfall Recovery Cost Recovery Steps
Wrong gopacket fork LOW go mod edit -replace github.com/google/gopacket=github.com/gopacket/gopacket@v1.5.0; update import paths
Dynamic binary on clean machine MEDIUM Add static build Makefile target; update CI; update README
PCM overflow / distortion LOW Refactor synthesis to float64 internal representation; add clamp before int16 cast
Corrupt MP3 (missing flush) LOW Add Flush() call in the shutdown path
Perceptual chaos (tone mapping) MEDIUM Redesign frequency table (no code change to synthesis engine); requires subjective listening tests
Time window jitter LOW Add EMA smoothing and increase window; no architectural change needed
ZeroCopy data corruption MEDIUM Replace ZeroCopyReadPacketData with ReadPacketData; audit all goroutine handoffs

Pitfall-to-Phase Mapping

Pitfall Prevention Phase Verification
Wrong gopacket fork Phase 1: Packet Capture go.mod references gopacket/gopacket; go list -m github.com/gopacket/gopacket
CGo / single binary contract Phase 1: Packet Capture ldd output on CI; test on clean Alpine container
CAP_NET_RAW binary location Phase 1 + CLI UX phase Test setcap from /usr/local/bin; verify helpful error message from non-root
Packet buffer overflow Phase 1/2: Capture Pipeline SetBufferSize call present; goroutine architecture is async (channel-separated)
ZeroCopy use-after-free Phase 1: Capture/Decode Code review: no ZeroCopy passed to goroutines without copy; or use ReadPacketData
LAME init errors / corrupt MP3 Audio synthesis phase CI smoke test: 1s silence → ffprobe validates output file
PCM overflow wrap-around Audio synthesis phase Unit test: 8 simultaneous max-amplitude layers produce no distortion
Perceptual tone chaos Audio mapping phase Subjective listen test with mixed traffic capture; frequency table reviewed against auditory masking
Time window jitter Traffic aggregation / mapping phase Capture test with bursty traffic; verify EMA smoothing produces stable amplitude

Sources


Pitfalls research for: network-traffic-to-audio synthesis CLI (Go) — NetSynth Researched: 2026-03-24