yoloyolo/.planning/research/ARCHITECTURE.md

# Architecture Research

**Domain:** Network traffic sonification CLI (Go)
**Researched:** 2026-03-24
**Confidence:** MEDIUM — Go audio synthesis patterns verified via official docs and real libraries; sonification architecture inferred from academic literature (SoNSTAR) and Go concurrency canon.

## Standard Architecture

### System Overview

```
┌─────────────────────────────────────────────────────────────┐
│                     CLI Entry Point                          │
│          (flags: interface, output path, duration)           │
└───────────────────────────┬─────────────────────────────────┘
                            │
                            ▼
┌─────────────────────────────────────────────────────────────┐
│                   Capture Layer                              │
│  ┌─────────────────────────────────────────────────────┐    │
│  │  PacketSource (gopacket/pcap)                        │    │
│  │  Produces: chan Packet                               │    │
│  └──────────────────────┬──────────────────────────────┘    │
└─────────────────────────┼───────────────────────────────────┘
                          │  raw packet stream
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                 Classification Layer                          │
│  ┌─────────────────────────┐  ┌─────────────────────────┐   │
│  │  Protocol Classifier    │  │  Unknown Traffic         │   │
│  │  (ICMP, DNS, HTTPS,     │  │  Clusterer               │   │
│  │   SSH, TCP-other, UDP)  │  │  (feature-based bucketer)│   │
│  └───────────┬─────────────┘  └────────────┬────────────┘   │
│              └──────────────┬──────────────┘                 │
└─────────────────────────────┼───────────────────────────────┘
                              │  classified packet events
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                  Aggregation Layer                            │
│  ┌─────────────────────────────────────────────────────┐    │
│  │  Time-Window Accumulator                             │    │
│  │  - fixed window (e.g. 500ms)                         │    │
│  │  - counts + byte-volume per traffic class            │    │
│  │  Produces: chan WindowSnapshot                       │    │
│  └──────────────────────┬──────────────────────────────┘    │
└─────────────────────────┼───────────────────────────────────┘
                          │  window snapshots
                          ▼
┌─────────────────────────────────────────────────────────────┐
│                   Synthesis Layer                             │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐    │
│  │ Layer 0  │  │ Layer 1  │  │ Layer N  │  │ Layer X  │    │
│  │ (ICMP)   │  │ (DNS)    │  │ (HTTPS)  │  │ (auto)   │    │
│  │ Osc+Amp  │  │ Osc+Amp  │  │ Osc+Amp  │  │ Osc+Amp  │    │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────┬─────┘    │
│       └──────────────┴────────────┴──────────────┘          │
│                          │                                   │
│                    ┌─────▼──────┐                            │
│                    │   Mixer    │                            │
│                    │ (sum+clip) │                            │
│                    └─────┬──────┘                            │
└──────────────────────────┼──────────────────────────────────┘
                           │  PCM sample stream (float32[])
                           ▼
┌─────────────────────────────────────────────────────────────┐
│                    Encoding Layer                             │
│  ┌─────────────────────────────────────────────────────┐    │
│  │  PCM Buffer Accumulator → LAME MP3 Encoder           │    │
│  │  (go-lame / CGo libmp3lame)                          │    │
│  └──────────────────────┬──────────────────────────────┘    │
└─────────────────────────┼───────────────────────────────────┘
                          │  .mp3 file
                          ▼
                    Output File
```

### Component Responsibilities

| Component | Responsibility | Typical Implementation |
|-----------|----------------|------------------------|
| CLI Entry | Parse flags, wire all components, handle Ctrl+C via `os.Signal` | `main.go`, `cobra` or `flag` package |
| PacketSource | Open interface via pcap/AF_PACKET, emit packets into channel | `gopacket.PacketSource.Packets()` → `<-chan gopacket.Packet` |
| Protocol Classifier | Inspect decoded layers (IP, TCP, UDP, ICMP, DNS); assign class label | Pure Go switch on `packet.Layer()` type assertions |
| Unknown Traffic Clusterer | Hash or bucket unclassified flows by port range / packet size signature; assign stable label ID | Simple feature-hash bucketer; no heavy ML needed for v1 |
| Time-Window Accumulator | Batch packets into N-ms windows; emit packet-count and byte-volume per class | `ticker`-driven goroutine, map accumulation |
| Sound Layer (per class) | Maintain a sine oscillator at a fixed root frequency; update amplitude from window snapshot | Oscillator struct with phase accumulator; amplitude lerp |
| Mixer | Sum all layer outputs sample-by-sample; clamp/normalize to [-1, 1] | Simple additive sum with soft clip |
| MP3 Encoder | Accept PCM float32 frames; encode to MP3 on flush/stop | go-lame (CGo) or pure-Go fallback |
| Output File | Write encoded bytes to disk path from CLI flag | `os.File` + buffered writer |

## Recommended Project Structure

```
netsynth/
├── main.go                  # CLI wiring, signal handling, top-level orchestration
├── capture/
│   └── capture.go           # PacketSource wrapper, interface open/close, chan Packet
├── classify/
│   ├── classifier.go        # Protocol dispatch, class label assignment
│   └── cluster.go           # Unknown traffic bucketer (feature hash)
├── aggregate/
│   └── window.go            # Time-window accumulator, WindowSnapshot type
├── synth/
│   ├── oscillator.go        # Phase-accumulator sine oscillator
│   ├── layer.go             # Per-traffic-class sound layer (osc + amp target)
│   └── mixer.go             # Sum layers → float32 PCM frames
├── encode/
│   └── mp3.go               # PCM → MP3 via go-lame; file flush on close
└── config/
    └── mapping.go           # Protocol → frequency/harmonic assignment table
```

### Structure Rationale

- **capture/:** Isolates pcap/root-privilege boundary. Everything above it operates on typed Go channels with no pcap dependency.
- **classify/:** Cleanly separates rule-based (known protocol) from heuristic (unknown cluster) logic. Each can be tested with synthetic packet fixtures independently.
- **aggregate/:** The only stateful time-domain component. Isolating it makes window size configurable without touching synthesis.
- **synth/:** Pure PCM math — no I/O, no pcap. Fully unit-testable with deterministic inputs. The mixer owns the sample rate constant.
- **encode/:** CGo boundary lives here and nowhere else. If LAME is replaced (e.g., pure Go encoder), only this package changes.
- **config/:** Static frequency-to-protocol table. Separating it avoids magic numbers scattered across synth/.

## Architectural Patterns

### Pattern 1: Channel-Connected Pipeline Stages

**What:** Each component is a goroutine that reads from an inbound channel and writes to an outbound channel. The `done` channel (closed on Ctrl+C) signals all stages to drain and exit cleanly.

**When to use:** Always — this is the idiomatic Go pipeline pattern described in the Go Blog.

**Trade-offs:** Slightly more setup than direct function calls; pays off immediately with clean shutdown and testability of individual stages.

**Example:**
```go
// Each stage signature follows this pattern
func Classify(done <-chan struct{}, packets <-chan gopacket.Packet) <-chan ClassifiedPacket {
    out := make(chan ClassifiedPacket, 256)
    go func() {
        defer close(out)
        for {
            select {
            case <-done:
                return
            case pkt, ok := <-packets:
                if !ok { return }
                out <- classify(pkt)
            }
        }
    }()
    return out
}
```

### Pattern 2: Ticker-Driven Window Flush

**What:** The aggregation goroutine owns a `time.Ticker`. On each tick it snapshots accumulated counters and sends a `WindowSnapshot` downstream, then resets counters.

**When to use:** Anywhere time-based batching converts a high-frequency stream into low-frequency control signals.

**Trade-offs:** Fixed window size (e.g. 500ms) is simple but loses sub-window dynamics. Sliding windows add complexity with marginal benefit for ambient synthesis.

**Example:**
```go
func Aggregate(done <-chan struct{}, events <-chan ClassifiedPacket, windowMs int) <-chan WindowSnapshot {
    out := make(chan WindowSnapshot, 8)
    ticker := time.NewTicker(time.Duration(windowMs) * time.Millisecond)
    go func() {
        defer close(out)
        counts := map[TrafficClass]int{}
        for {
            select {
            case <-done:
                return
            case <-ticker.C:
                out <- snapshot(counts)
                counts = map[TrafficClass]int{}
            case ev, ok := <-events:
                if !ok { return }
                counts[ev.Class]++
            }
        }
    }()
    return out
}
```

### Pattern 3: Per-Layer Amplitude Lerp

**What:** Each sound layer holds a current amplitude and a target amplitude. On each audio frame the current value moves toward the target by a smoothing coefficient. The layer's oscillator always runs; silence is achieved by targeting amplitude = 0.

**When to use:** Whenever window snapshots drive synthesis — avoids clicks/pops from abrupt amplitude changes.

**Trade-offs:** Adds minimal CPU overhead (one multiply per frame per layer); necessary for perceptually smooth audio.

## Data Flow

### Primary Flow: Packets to PCM

```
Network Interface
    │
    ▼ (gopacket pcap handle)
PacketSource.Packets() chan
    │
    ▼ (classify goroutine)
ClassifiedPacket chan
    │
    ▼ (aggregate goroutine, ticker)
WindowSnapshot chan  ─────────────────────────────────┐
                                                      │
                                             (synth goroutine,
                                              per window snap:
                                              update amplitude targets)
                                                      │
                                             PCM frame generator loop
                                             (renders N frames per window,
                                              one frame = sum of all layers)
                                                      │
                                                      ▼
                                             PCM []float32 blocks
                                                      │
                                                      ▼
                                             LAME encoder (streaming)
                                                      │
                                                      ▼
                                             MP3 bytes → output file
```

### Shutdown Flow

```
Ctrl+C → os.Signal → close(done) channel
    │
    ├── capture goroutine: drain + close packet chan
    ├── classify goroutine: drain + close event chan
    ├── aggregate goroutine: drain + close snapshot chan
    └── synth goroutine: flush remaining PCM → encoder.Flush() → file.Close()
```

### Key Data Types

1. **`gopacket.Packet`** → raw decoded packet from pcap; carries layer stack.
2. **`ClassifiedPacket{Packet, Class TrafficClass, Bytes int}`** → labeled event.
3. **`WindowSnapshot{ClassCounts map[TrafficClass]int, ClassBytes map[TrafficClass]int}`** → per-window aggregate; drives amplitude targets.
4. **`[]float32` PCM block** → mixer output at 44100 Hz, mono; flows into LAME.

## Build Order (Phase Implications)

Build in dependency order — each layer is independently testable before the next is added:

```
1. capture/        → can test: "does it open an interface and emit packets?"
2. classify/       → can test: "does ICMP get labeled ICMP?" (synthetic packets)
3. aggregate/      → can test: "does a 500ms window count correctly?"
4. synth/          → can test: "does mixer output expected amplitude?" (no pcap needed)
5. encode/         → can test: "does PCM produce valid MP3 bytes?"
6. main.go wiring  → integration: full end-to-end pipeline
```

This ordering means:
- **Phase 1** can deliver a working capture + classify pipeline writing JSON/text summaries — validating the hardest privilege/pcap risk early.
- **Phase 2** delivers the synthesis engine in isolation — testable with synthetic `WindowSnapshot` inputs before any real traffic.
- **Phase 3** wires them together with the MP3 encoder.

## Anti-Patterns

### Anti-Pattern 1: Synchronous Per-Packet Audio Rendering

**What people do:** Generate one audio sample or tone event per packet — a 10 Gbps link produces 14M packets/sec, making synchronous render impossible.

**Why it's wrong:** Breaks at any real traffic volume; produces click-heavy output, not smooth drone.

**Do this instead:** Batch packets into time windows (500ms–1s) and drive amplitude targets from the batch, not individual packets.

### Anti-Pattern 2: Blocking Channel Sends in the Capture Path

**What people do:** Use unbuffered channels between PacketSource and classifier; slow classifier stalls the pcap ring buffer and causes kernel drops.

**Why it's wrong:** libpcap's kernel buffer is fixed-size; if userspace can't drain it fast enough, packets are silently dropped. For audio purposes this introduces silent gaps.

**Do this instead:** Use buffered channels (capacity 256–1024) between capture and classify. Drop packets on full buffer with a counter — acceptable for sonification, fatal to log completeness tools.

### Anti-Pattern 3: CGo MP3 Encoding in the Hot Audio Loop

**What people do:** Call `lame.Encode()` synchronously inside the frame-render loop, stalling synthesis.

**Why it's wrong:** CGo calls carry overhead; libmp3lame may block on I/O; this disrupts the synthesis clock.

**Do this instead:** The synth goroutine pushes PCM blocks onto a buffered channel; a separate encoder goroutine drains and encodes. On shutdown, close the PCM channel and drain completely before `lame.Close()`.

### Anti-Pattern 4: Global Mutable State for Class Frequency Mapping

**What people do:** Use a global `map[TrafficClass]float64` for frequency assignments modified at runtime.

**Why it's wrong:** Race conditions; hard to test; makes the mapping invisible to callers.

**Do this instead:** Pass the mapping table as an immutable struct at construction time. Auto-clustered classes append to a local slice protected by a mutex inside the clusterer — not a global.

## Integration Points

### External Services

| Dependency | Integration Pattern | Notes |
|------------|---------------------|-------|
| libpcap / pcap.h | CGo via gopacket/pcap — requires libpcap-dev at build time | Can substitute AF_PACKET (linux only) to avoid CGo in capture; still needs root |
| libmp3lame | CGo via go-lame — requires libmp3lame-dev at build time | Binary distribution requires static linking or Docker; pure-Go MP3 (e.g. oto + gmp3) is an option but quality/speed tradeoff |

### Internal Boundaries

| Boundary | Communication | Notes |
|----------|---------------|-------|
| capture ↔ classify | `chan gopacket.Packet` (buffered 512) | classify must never block capture |
| classify ↔ aggregate | `chan ClassifiedPacket` (buffered 1024) | aggregate is slower (ticker-driven); buffer absorbs bursts |
| aggregate ↔ synth | `chan WindowSnapshot` (buffered 4) | synth consumes synchronously per window; small buffer is fine |
| synth ↔ encode | `chan []float32` (buffered 8 blocks) | encoder runs in separate goroutine to decouple CGo latency |
| all stages ↔ main | `chan struct{}` done channel | closed on Ctrl+C; all stages select on it |

## Scaling Considerations

This is a single-binary CLI tool, not a distributed service. Scaling concerns are throughput-based:

| Traffic Rate | Architecture Adjustments |
|--------------|--------------------------|
| Home/office (< 10K pps) | Default design handles easily with no tuning |
| Datacenter (100K–1M pps) | Increase capture buffer size; consider AF_PACKET with TPACKET_V3 ring buffer instead of pcap; classify goroutine may need fan-out to 2–4 workers |
| Line-rate 10G (> 5M pps) | Out of scope for v1 ambient audio tool — synthesis granularity at 500ms windows means exact packet-level accuracy is not required |

### Scaling Priorities

1. **First bottleneck:** Kernel pcap buffer drops — mitigated by buffered channels and accepting lossy capture (fine for sonification).
2. **Second bottleneck:** CGo encoding latency coupling synthesis clock — mitigated by decoupled encoder goroutine.

## Sources

- Go Pipeline patterns: [Go Concurrency Patterns: Pipelines and cancellation](https://go.dev/blog/pipelines) — HIGH confidence, official Go blog
- SoNSTAR network sonification architecture: [Sonification of Network Traffic Flow for Monitoring and Situational Awareness, arXiv 1712.07029](https://arxiv.org/abs/1712.07029) — MEDIUM confidence (abstract only accessed)
- gopacket channel API: [gopacket pkg.go.dev](https://pkg.go.dev/github.com/google/gopacket) — HIGH confidence, official package docs
- bleep synthesizer architecture (Go): [GitHub bspaans/bleep](https://github.com/bspaans/bleep) — MEDIUM confidence (README inspection)
- Waveform synthesis PCM patterns in Go: [Audio From Scratch With Go — Dylan Meeus](https://dylanmeeus.github.io/posts/audio-from-scratch-pt8/) — MEDIUM confidence
- go-lame MP3 encoding: [go-lame pkg.go.dev](https://pkg.go.dev/github.com/sunicy/go-lame) — MEDIUM confidence
- Drone amplitude/frequency modulation patterns: [Drone auralization model, Acta Acustica 2024](https://acta-acustica.edpsciences.org/articles/aacus/full_html/2024/01/aacus240076/aacus240076.html) — MEDIUM confidence

---
*Architecture research for: NetSynth — network-traffic-to-audio synthesis CLI (Go)*
*Researched: 2026-03-24*