docs: complete project research
This commit is contained in:
+304
-272
@@ -1,344 +1,376 @@
|
||||
# Architecture Research
|
||||
# Architecture Patterns
|
||||
|
||||
**Domain:** Network traffic sonification CLI (Go)
|
||||
**Researched:** 2026-03-24
|
||||
**Confidence:** MEDIUM — Go audio synthesis patterns verified via official docs and real libraries; sonification architecture inferred from academic literature (SoNSTAR) and Go concurrency canon.
|
||||
**Domain:** Network traffic sonification CLI (Go) — v1.1 Custom Sound Mappings
|
||||
**Researched:** 2026-03-26
|
||||
**Confidence:** HIGH — based on direct code inspection of the existing v1.0 codebase
|
||||
|
||||
## Standard Architecture
|
||||
---
|
||||
|
||||
### System Overview
|
||||
## v1.1 Integration Overview
|
||||
|
||||
This document supersedes the pre-implementation v1.0 architecture research. It is grounded in the actual codebase (3,254 lines, 6 packages) and answers: what changes, what's new, and in what order.
|
||||
|
||||
---
|
||||
|
||||
## Existing Package Map (v1.0 Baseline)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ CLI Entry Point │
|
||||
│ (flags: interface, output path, duration) │
|
||||
└───────────────────────────┬─────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Capture Layer │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ PacketSource (gopacket/pcap) │ │
|
||||
│ │ Produces: chan Packet │ │
|
||||
│ └──────────────────────┬──────────────────────────────┘ │
|
||||
└─────────────────────────┼───────────────────────────────────┘
|
||||
│ raw packet stream
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Classification Layer │
|
||||
│ ┌─────────────────────────┐ ┌─────────────────────────┐ │
|
||||
│ │ Protocol Classifier │ │ Unknown Traffic │ │
|
||||
│ │ (ICMP, DNS, HTTPS, │ │ Clusterer │ │
|
||||
│ │ SSH, TCP-other, UDP) │ │ (feature-based bucketer)│ │
|
||||
│ └───────────┬─────────────┘ └────────────┬────────────┘ │
|
||||
│ └──────────────┬──────────────┘ │
|
||||
└─────────────────────────────┼───────────────────────────────┘
|
||||
│ classified packet events
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Aggregation Layer │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ Time-Window Accumulator │ │
|
||||
│ │ - fixed window (e.g. 500ms) │ │
|
||||
│ │ - counts + byte-volume per traffic class │ │
|
||||
│ │ Produces: chan WindowSnapshot │ │
|
||||
│ └──────────────────────┬──────────────────────────────┘ │
|
||||
└─────────────────────────┼───────────────────────────────────┘
|
||||
│ window snapshots
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Synthesis Layer │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │
|
||||
│ │ Layer 0 │ │ Layer 1 │ │ Layer N │ │ Layer X │ │
|
||||
│ │ (ICMP) │ │ (DNS) │ │ (HTTPS) │ │ (auto) │ │
|
||||
│ │ Osc+Amp │ │ Osc+Amp │ │ Osc+Amp │ │ Osc+Amp │ │
|
||||
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬─────┘ │
|
||||
│ └──────────────┴────────────┴──────────────┘ │
|
||||
│ │ │
|
||||
│ ┌─────▼──────┐ │
|
||||
│ │ Mixer │ │
|
||||
│ │ (sum+clip) │ │
|
||||
│ └─────┬──────┘ │
|
||||
└──────────────────────────┼──────────────────────────────────┘
|
||||
│ PCM sample stream (float32[])
|
||||
▼
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Encoding Layer │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ PCM Buffer Accumulator → LAME MP3 Encoder │ │
|
||||
│ │ (go-lame / CGo libmp3lame) │ │
|
||||
│ └──────────────────────┬──────────────────────────────┘ │
|
||||
└─────────────────────────┼───────────────────────────────────┘
|
||||
│ .mp3 file
|
||||
▼
|
||||
Output File
|
||||
cmd/netsynth/main.go CLI, pipeline wiring, Cobra flags
|
||||
capture/ go-pcap live capture + pcap file reader + BPF
|
||||
classify/
|
||||
types.go TrafficClass, ClassifiedPacket, WindowSnapshot
|
||||
classifier.go NewClassifier(rules []Rule) — first-match-wins
|
||||
rules.go DefaultRules []Rule (12 hardcoded rules)
|
||||
aggregate/
|
||||
window.go 500ms time-windowed snapshot accumulation
|
||||
synth/
|
||||
config.go ClassFreqConfigs — fixed map[TrafficClass]FreqConfig
|
||||
oscillator.go Phase-accumulator oscillator — sine only
|
||||
layer.go EMA amplitude smoothing per layer
|
||||
bank.go NewBank(tau) — one Layer per AllClasses()
|
||||
mixer.go PanGains, StereoFramesToInt16Bytes
|
||||
encode/
|
||||
mp3.go RunSynthesis(snapshots, path) — NewBank + EncodeMP3
|
||||
```
|
||||
|
||||
### Component Responsibilities
|
||||
---
|
||||
|
||||
| Component | Responsibility | Typical Implementation |
|
||||
|-----------|----------------|------------------------|
|
||||
| CLI Entry | Parse flags, wire all components, handle Ctrl+C via `os.Signal` | `main.go`, `cobra` or `flag` package |
|
||||
| PacketSource | Open interface via pcap/AF_PACKET, emit packets into channel | `gopacket.PacketSource.Packets()` → `<-chan gopacket.Packet` |
|
||||
| Protocol Classifier | Inspect decoded layers (IP, TCP, UDP, ICMP, DNS); assign class label | Pure Go switch on `packet.Layer()` type assertions |
|
||||
| Unknown Traffic Clusterer | Hash or bucket unclassified flows by port range / packet size signature; assign stable label ID | Simple feature-hash bucketer; no heavy ML needed for v1 |
|
||||
| Time-Window Accumulator | Batch packets into N-ms windows; emit packet-count and byte-volume per class | `ticker`-driven goroutine, map accumulation |
|
||||
| Sound Layer (per class) | Maintain a sine oscillator at a fixed root frequency; update amplitude from window snapshot | Oscillator struct with phase accumulator; amplitude lerp |
|
||||
| Mixer | Sum all layer outputs sample-by-sample; clamp/normalize to [-1, 1] | Simple additive sum with soft clip |
|
||||
| MP3 Encoder | Accept PCM float32 frames; encode to MP3 on flush/stop | go-lame (CGo) or pure-Go fallback |
|
||||
| Output File | Write encoded bytes to disk path from CLI flag | `os.File` + buffered writer |
|
||||
## What v1.1 Adds
|
||||
|
||||
## Recommended Project Structure
|
||||
Three independent but related features:
|
||||
|
||||
1. **TOML config file** — override frequencies and waveforms per built-in class
|
||||
2. **Additional waveforms** — square, sawtooth, triangle alongside existing sine
|
||||
3. **User-defined classification rules** — TOML-defined rules prepended before DefaultRules
|
||||
|
||||
---
|
||||
|
||||
## Integration Point Analysis
|
||||
|
||||
### Feature 1: TOML Config File
|
||||
|
||||
**Where config is consumed today:** `synth/config.go` holds a package-level `var ClassFreqConfigs`. `synth/bank.go:NewBank()` reads it directly with `ClassFreqConfigs[class]`. No config is passed through `encode.RunSynthesis` or `main.go`.
|
||||
|
||||
**Required change:** `NewBank` must accept a config parameter instead of reading the global. `encode.RunSynthesis` must accept and forward a config. `main.go` must load config from disk and pass it in.
|
||||
|
||||
**New package: `config/`**
|
||||
|
||||
This package does not exist yet in the codebase (the pre-implementation research anticipated it but it was deferred). It should own:
|
||||
- TOML struct definitions
|
||||
- File discovery logic (auto-detect `./netsynth.toml`, then `~/.config/netsynth/config.toml`)
|
||||
- Merging: loaded config overlays defaults, does not replace them entirely
|
||||
|
||||
```
|
||||
netsynth/
|
||||
├── main.go # CLI wiring, signal handling, top-level orchestration
|
||||
├── capture/
|
||||
│ └── capture.go # PacketSource wrapper, interface open/close, chan Packet
|
||||
├── classify/
|
||||
│ ├── classifier.go # Protocol dispatch, class label assignment
|
||||
│ └── cluster.go # Unknown traffic bucketer (feature hash)
|
||||
├── aggregate/
|
||||
│ └── window.go # Time-window accumulator, WindowSnapshot type
|
||||
├── synth/
|
||||
│ ├── oscillator.go # Phase-accumulator sine oscillator
|
||||
│ ├── layer.go # Per-traffic-class sound layer (osc + amp target)
|
||||
│ └── mixer.go # Sum layers → float32 PCM frames
|
||||
├── encode/
|
||||
│ └── mp3.go # PCM → MP3 via go-lame; file flush on close
|
||||
└── config/
|
||||
└── mapping.go # Protocol → frequency/harmonic assignment table
|
||||
config/
|
||||
config.go Config struct, Load(path string) (*Config, error)
|
||||
defaults.go DefaultConfig() — wraps existing ClassFreqConfigs values
|
||||
```
|
||||
|
||||
### Structure Rationale
|
||||
**TOML struct shape:**
|
||||
|
||||
- **capture/:** Isolates pcap/root-privilege boundary. Everything above it operates on typed Go channels with no pcap dependency.
|
||||
- **classify/:** Cleanly separates rule-based (known protocol) from heuristic (unknown cluster) logic. Each can be tested with synthetic packet fixtures independently.
|
||||
- **aggregate/:** The only stateful time-domain component. Isolating it makes window size configurable without touching synthesis.
|
||||
- **synth/:** Pure PCM math — no I/O, no pcap. Fully unit-testable with deterministic inputs. The mixer owns the sample rate constant.
|
||||
- **encode/:** CGo boundary lives here and nowhere else. If LAME is replaced (e.g., pure Go encoder), only this package changes.
|
||||
- **config/:** Static frequency-to-protocol table. Separating it avoids magic numbers scattered across synth/.
|
||||
```toml
|
||||
[[class]]
|
||||
name = "HTTPS"
|
||||
frequency_hz = 200.0
|
||||
waveform = "sawtooth"
|
||||
|
||||
## Architectural Patterns
|
||||
[[class]]
|
||||
name = "myservice" # user-defined class (Feature 3)
|
||||
frequency_hz = 350.0
|
||||
waveform = "triangle"
|
||||
```
|
||||
|
||||
### Pattern 1: Channel-Connected Pipeline Stages
|
||||
The `Config` struct passed into `NewBank` should merge with `ClassFreqConfigs`:
|
||||
|
||||
**What:** Each component is a goroutine that reads from an inbound channel and writes to an outbound channel. The `done` channel (closed on Ctrl+C) signals all stages to drain and exit cleanly.
|
||||
|
||||
**When to use:** Always — this is the idiomatic Go pipeline pattern described in the Go Blog.
|
||||
|
||||
**Trade-offs:** Slightly more setup than direct function calls; pays off immediately with clean shutdown and testability of individual stages.
|
||||
|
||||
**Example:**
|
||||
```go
|
||||
// Each stage signature follows this pattern
|
||||
func Classify(done <-chan struct{}, packets <-chan gopacket.Packet) <-chan ClassifiedPacket {
|
||||
out := make(chan ClassifiedPacket, 256)
|
||||
go func() {
|
||||
defer close(out)
|
||||
for {
|
||||
select {
|
||||
case <-done:
|
||||
return
|
||||
case pkt, ok := <-packets:
|
||||
if !ok { return }
|
||||
out <- classify(pkt)
|
||||
}
|
||||
}
|
||||
}()
|
||||
return out
|
||||
// config/config.go
|
||||
type ClassConfig struct {
|
||||
Name string `toml:"name"`
|
||||
FrequencyHz float64 `toml:"frequency_hz"`
|
||||
Waveform string `toml:"waveform"` // "sine" | "square" | "sawtooth" | "triangle"
|
||||
}
|
||||
|
||||
type Config struct {
|
||||
Classes []ClassConfig `toml:"class"`
|
||||
Rules []RuleConfig `toml:"rule"` // Feature 3
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 2: Ticker-Driven Window Flush
|
||||
**TOML library:** Use `github.com/BurntSushi/toml`. It is the de-facto standard for TOML in Go (used by Hugo, dep, buf, etc.). Already a transitive dependency in many Go module graphs. Provides struct-tag-based decode, good error messages.
|
||||
|
||||
**What:** The aggregation goroutine owns a `time.Ticker`. On each tick it snapshots accumulated counters and sends a `WindowSnapshot` downstream, then resets counters.
|
||||
---
|
||||
|
||||
**When to use:** Anywhere time-based batching converts a high-frequency stream into low-frequency control signals.
|
||||
### Feature 2: Additional Waveforms
|
||||
|
||||
**Trade-offs:** Fixed window size (e.g. 500ms) is simple but loses sub-window dynamics. Sliding windows add complexity with marginal benefit for ambient synthesis.
|
||||
**Where waveform logic lives today:** `synth/oscillator.go:Advance()` — pure sine via `math.Sin`. The `HarmonicDef.Ratio` and `HarmonicDef.Amplitude` fields are stored in `FreqConfig.Harmonics` but the waveform function is hardcoded.
|
||||
|
||||
**Required change:** `Oscillator.Advance` must dispatch on a waveform type. Two clean approaches:
|
||||
|
||||
**Option A (recommended): Waveform enum on Oscillator**
|
||||
|
||||
Add a `waveform` field to `Oscillator`. `Advance` switches on it. `NewOscillator` gains a waveform parameter.
|
||||
|
||||
**Example:**
|
||||
```go
|
||||
func Aggregate(done <-chan struct{}, events <-chan ClassifiedPacket, windowMs int) <-chan WindowSnapshot {
|
||||
out := make(chan WindowSnapshot, 8)
|
||||
ticker := time.NewTicker(time.Duration(windowMs) * time.Millisecond)
|
||||
go func() {
|
||||
defer close(out)
|
||||
counts := map[TrafficClass]int{}
|
||||
for {
|
||||
select {
|
||||
case <-done:
|
||||
return
|
||||
case <-ticker.C:
|
||||
out <- snapshot(counts)
|
||||
counts = map[TrafficClass]int{}
|
||||
case ev, ok := <-events:
|
||||
if !ok { return }
|
||||
counts[ev.Class]++
|
||||
}
|
||||
}
|
||||
}()
|
||||
return out
|
||||
type Waveform int
|
||||
|
||||
const (
|
||||
WaveformSine Waveform = iota
|
||||
WaveformSquare
|
||||
WaveformSawtooth
|
||||
WaveformTriangle
|
||||
)
|
||||
|
||||
type Oscillator struct {
|
||||
phase float64
|
||||
freq float64
|
||||
sr float64
|
||||
waveform Waveform
|
||||
}
|
||||
|
||||
func (o *Oscillator) sampleAt(phase, ratio float64) float64 {
|
||||
p := phase * float64(ratio)
|
||||
p -= math.Floor(p) // wrap to [0, 1)
|
||||
switch o.waveform {
|
||||
case WaveformSquare:
|
||||
if p < 0.5 { return 1.0 }
|
||||
return -1.0
|
||||
case WaveformSawtooth:
|
||||
return 2.0*p - 1.0
|
||||
case WaveformTriangle:
|
||||
if p < 0.5 { return 4.0*p - 1.0 }
|
||||
return 3.0 - 4.0*p
|
||||
default: // WaveformSine
|
||||
return math.Sin(2 * math.Pi * p)
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 3: Per-Layer Amplitude Lerp
|
||||
**Option B: Function field on Oscillator**
|
||||
|
||||
**What:** Each sound layer holds a current amplitude and a target amplitude. On each audio frame the current value moves toward the target by a smoothing coefficient. The layer's oscillator always runs; silence is achieved by targeting amplitude = 0.
|
||||
Store `waveFn func(phase float64) float64`. More flexible but harder to serialize/configure.
|
||||
|
||||
**When to use:** Whenever window snapshots drive synthesis — avoids clicks/pops from abrupt amplitude changes.
|
||||
Option A is preferred because waveform type maps cleanly to the TOML `waveform` string field without reflection tricks.
|
||||
|
||||
**Trade-offs:** Adds minimal CPU overhead (one multiply per frame per layer); necessary for perceptually smooth audio.
|
||||
**`FreqConfig` change:** Add `Waveform` field:
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Primary Flow: Packets to PCM
|
||||
|
||||
```
|
||||
Network Interface
|
||||
│
|
||||
▼ (gopacket pcap handle)
|
||||
PacketSource.Packets() chan
|
||||
│
|
||||
▼ (classify goroutine)
|
||||
ClassifiedPacket chan
|
||||
│
|
||||
▼ (aggregate goroutine, ticker)
|
||||
WindowSnapshot chan ─────────────────────────────────┐
|
||||
│
|
||||
(synth goroutine,
|
||||
per window snap:
|
||||
update amplitude targets)
|
||||
│
|
||||
PCM frame generator loop
|
||||
(renders N frames per window,
|
||||
one frame = sum of all layers)
|
||||
│
|
||||
▼
|
||||
PCM []float32 blocks
|
||||
│
|
||||
▼
|
||||
LAME encoder (streaming)
|
||||
│
|
||||
▼
|
||||
MP3 bytes → output file
|
||||
```go
|
||||
type FreqConfig struct {
|
||||
BaseHz float64
|
||||
Harmonics []HarmonicDef
|
||||
Pan float64
|
||||
Waveform Waveform // NEW: defaults to WaveformSine
|
||||
}
|
||||
```
|
||||
|
||||
### Shutdown Flow
|
||||
`NewLayer` passes `cfg.Waveform` to `NewOscillator`. `NewOscillator` signature changes to accept the waveform.
|
||||
|
||||
```
|
||||
Ctrl+C → os.Signal → close(done) channel
|
||||
│
|
||||
├── capture goroutine: drain + close packet chan
|
||||
├── classify goroutine: drain + close event chan
|
||||
├── aggregate goroutine: drain + close snapshot chan
|
||||
└── synth goroutine: flush remaining PCM → encoder.Flush() → file.Close()
|
||||
**What does NOT change:** `HarmonicDef`, `EMAAlpha`, `Layer.UpdateTarget`, `Layer.AdvanceSample`, `OscillatorBank.RenderWindow`, `mixer.go`, `encode/mp3.go`. The waveform change is contained to `oscillator.go` and the `FreqConfig` struct.
|
||||
|
||||
---
|
||||
|
||||
### Feature 3: User-Defined Classification Rules
|
||||
|
||||
**Where rules are wired today:** `main.go` lines 111, 175 — both `runLiveMode` and `runPcapMode` call `classify.NewClassifier(classify.DefaultRules)` directly. No config is passed.
|
||||
|
||||
**Required change:** User rules from TOML prepend before `DefaultRules`. `Classifier` already supports arbitrary `[]Rule` — `NewClassifier(rules []Rule)` is the constructor. No change to `classifier.go` itself.
|
||||
|
||||
**`RuleConfig` TOML struct:**
|
||||
|
||||
```toml
|
||||
[[rule]]
|
||||
protocol = "tcp"
|
||||
dst_port = 8443
|
||||
class = "myservice"
|
||||
```
|
||||
|
||||
### Key Data Types
|
||||
|
||||
1. **`gopacket.Packet`** → raw decoded packet from pcap; carries layer stack.
|
||||
2. **`ClassifiedPacket{Packet, Class TrafficClass, Bytes int}`** → labeled event.
|
||||
3. **`WindowSnapshot{ClassCounts map[TrafficClass]int, ClassBytes map[TrafficClass]int}`** → per-window aggregate; drives amplitude targets.
|
||||
4. **`[]float32` PCM block** → mixer output at 44100 Hz, mono; flows into LAME.
|
||||
|
||||
## Build Order (Phase Implications)
|
||||
|
||||
Build in dependency order — each layer is independently testable before the next is added:
|
||||
|
||||
```
|
||||
1. capture/ → can test: "does it open an interface and emit packets?"
|
||||
2. classify/ → can test: "does ICMP get labeled ICMP?" (synthetic packets)
|
||||
3. aggregate/ → can test: "does a 500ms window count correctly?"
|
||||
4. synth/ → can test: "does mixer output expected amplitude?" (no pcap needed)
|
||||
5. encode/ → can test: "does PCM produce valid MP3 bytes?"
|
||||
6. main.go wiring → integration: full end-to-end pipeline
|
||||
```go
|
||||
// config/config.go
|
||||
type RuleConfig struct {
|
||||
Protocol string `toml:"protocol"`
|
||||
DstPort uint16 `toml:"dst_port"`
|
||||
Class string `toml:"class"` // must match a name in [[class]] or a builtin class name
|
||||
}
|
||||
```
|
||||
|
||||
This ordering means:
|
||||
- **Phase 1** can deliver a working capture + classify pipeline writing JSON/text summaries — validating the hardest privilege/pcap risk early.
|
||||
- **Phase 2** delivers the synthesis engine in isolation — testable with synthetic `WindowSnapshot` inputs before any real traffic.
|
||||
- **Phase 3** wires them together with the MP3 encoder.
|
||||
**Merging in main.go:**
|
||||
|
||||
## Anti-Patterns
|
||||
```go
|
||||
userRules := config.ToClassifyRules(cfg.Rules) // []classify.Rule
|
||||
allRules := append(userRules, classify.DefaultRules...)
|
||||
classifier := classify.NewClassifier(allRules)
|
||||
```
|
||||
|
||||
### Anti-Pattern 1: Synchronous Per-Packet Audio Rendering
|
||||
**New `TrafficClass` values:** User-defined classes in TOML produce new `TrafficClass` string values (e.g., `"myservice"`). `AllClasses()` in `classify/types.go` is currently a hardcoded slice. For user-defined classes, `AllClasses()` cannot be the source of truth for bank layer construction. `NewBank` must instead iterate over whatever classes have a `FreqConfig` entry.
|
||||
|
||||
**What people do:** Generate one audio sample or tone event per packet — a 10 Gbps link produces 14M packets/sec, making synchronous render impossible.
|
||||
This is a critical integration point: `bank.go:NewBank` currently ranges over `classify.AllClasses()`. If user classes can appear, `NewBank` must accept the full config map and range over that instead.
|
||||
|
||||
**Why it's wrong:** Breaks at any real traffic volume; produces click-heavy output, not smooth drone.
|
||||
---
|
||||
|
||||
**Do this instead:** Batch packets into time windows (500ms–1s) and drive amplitude targets from the batch, not individual packets.
|
||||
## New vs Modified Components
|
||||
|
||||
### Anti-Pattern 2: Blocking Channel Sends in the Capture Path
|
||||
### New
|
||||
|
||||
**What people do:** Use unbuffered channels between PacketSource and classifier; slow classifier stalls the pcap ring buffer and causes kernel drops.
|
||||
| Component | Location | Purpose |
|
||||
|-----------|----------|---------|
|
||||
| `config` package | `config/config.go` | TOML struct, `Load()`, file discovery, merge with defaults |
|
||||
| `config/defaults.go` | optional split | `DefaultConfig()` wrapping existing `ClassFreqConfigs` values |
|
||||
|
||||
**Why it's wrong:** libpcap's kernel buffer is fixed-size; if userspace can't drain it fast enough, packets are silently dropped. For audio purposes this introduces silent gaps.
|
||||
### Modified
|
||||
|
||||
**Do this instead:** Use buffered channels (capacity 256–1024) between capture and classify. Drop packets on full buffer with a counter — acceptable for sonification, fatal to log completeness tools.
|
||||
| Component | Change | Impact |
|
||||
|-----------|--------|--------|
|
||||
| `synth/oscillator.go` | Add `Waveform` type + `waveform` field; dispatch in `Advance` | Self-contained; no caller signature breaks except `NewOscillator` |
|
||||
| `synth/config.go` | Add `Waveform Waveform` field to `FreqConfig`; default to `WaveformSine` | Requires `NewLayer` to pass waveform to `NewOscillator` |
|
||||
| `synth/layer.go` | Pass `cfg.Waveform` to `NewOscillator` | One-line change |
|
||||
| `synth/bank.go` | Accept `map[classify.TrafficClass]FreqConfig` param instead of reading global; range over param keys not `AllClasses()` | Decouples bank from global; enables user classes |
|
||||
| `encode/mp3.go` | Accept `*config.Config` or merged `FreqConfig` map; pass to `NewBank` | Thin forwarding change |
|
||||
| `cmd/netsynth/main.go` | Add `--config` flag; load config; prepend user rules; pass config to `RunSynthesis` | Touches both `runLiveMode` and `runPcapMode` |
|
||||
| `classify/rules.go` | No change — `DefaultRules` stays as the fallback | Unchanged |
|
||||
| `classify/classifier.go` | No change — already accepts `[]Rule` | Unchanged |
|
||||
| `classify/types.go` | `AllClasses()` may need a note that it returns only builtins; bank no longer relies on it | Low risk; document only |
|
||||
|
||||
### Anti-Pattern 3: CGo MP3 Encoding in the Hot Audio Loop
|
||||
---
|
||||
|
||||
**What people do:** Call `lame.Encode()` synchronously inside the frame-render loop, stalling synthesis.
|
||||
## Data Flow Changes
|
||||
|
||||
**Why it's wrong:** CGo calls carry overhead; libmp3lame may block on I/O; this disrupts the synthesis clock.
|
||||
### v1.0 Flow (config hardcoded)
|
||||
|
||||
**Do this instead:** The synth goroutine pushes PCM blocks onto a buffered channel; a separate encoder goroutine drains and encodes. On shutdown, close the PCM channel and drain completely before `lame.Close()`.
|
||||
```
|
||||
main.go
|
||||
└─ classify.NewClassifier(classify.DefaultRules)
|
||||
└─ encode.RunSynthesis(snapshots, path)
|
||||
└─ synth.NewBank(1.0)
|
||||
└─ ClassFreqConfigs[class] ← global, hardcoded
|
||||
```
|
||||
|
||||
### Anti-Pattern 4: Global Mutable State for Class Frequency Mapping
|
||||
### v1.1 Flow (config injected)
|
||||
|
||||
**What people do:** Use a global `map[TrafficClass]float64` for frequency assignments modified at runtime.
|
||||
```
|
||||
main.go
|
||||
└─ config.Load(configPath) ← NEW: resolve path, parse TOML, merge defaults
|
||||
└─ cfg *config.Config
|
||||
└─ classify.NewClassifier(
|
||||
append(config.ToClassifyRules(cfg.Rules), classify.DefaultRules...)
|
||||
) ← user rules prepend built-ins
|
||||
└─ encode.RunSynthesis(snapshots, path, cfg.FreqConfigs())
|
||||
└─ synth.NewBank(1.0, freqConfigs) ← map passed in, not read from global
|
||||
└─ freqConfigs[class] ← merged: user overrides + defaults
|
||||
```
|
||||
|
||||
**Why it's wrong:** Race conditions; hard to test; makes the mapping invisible to callers.
|
||||
---
|
||||
|
||||
**Do this instead:** Pass the mapping table as an immutable struct at construction time. Auto-clustered classes append to a local slice protected by a mutex inside the clusterer — not a global.
|
||||
## Suggested Build Order
|
||||
|
||||
## Integration Points
|
||||
The following order minimizes integration risk. Each step is independently testable before the next begins.
|
||||
|
||||
### External Services
|
||||
### Step 1: Waveform types in `synth/oscillator.go`
|
||||
|
||||
| Dependency | Integration Pattern | Notes |
|
||||
|------------|---------------------|-------|
|
||||
| libpcap / pcap.h | CGo via gopacket/pcap — requires libpcap-dev at build time | Can substitute AF_PACKET (linux only) to avoid CGo in capture; still needs root |
|
||||
| libmp3lame | CGo via go-lame — requires libmp3lame-dev at build time | Binary distribution requires static linking or Docker; pure-Go MP3 (e.g. oto + gmp3) is an option but quality/speed tradeoff |
|
||||
No external dependencies. Pure math. Testable with golden-sample unit tests (square wave sample at phase 0.25 should be 1.0, etc.). Does not affect `Layer`, `Bank`, or `encode` yet.
|
||||
|
||||
### Internal Boundaries
|
||||
**Files changed:** `synth/oscillator.go` only.
|
||||
|
||||
| Boundary | Communication | Notes |
|
||||
|----------|---------------|-------|
|
||||
| capture ↔ classify | `chan gopacket.Packet` (buffered 512) | classify must never block capture |
|
||||
| classify ↔ aggregate | `chan ClassifiedPacket` (buffered 1024) | aggregate is slower (ticker-driven); buffer absorbs bursts |
|
||||
| aggregate ↔ synth | `chan WindowSnapshot` (buffered 4) | synth consumes synchronously per window; small buffer is fine |
|
||||
| synth ↔ encode | `chan []float32` (buffered 8 blocks) | encoder runs in separate goroutine to decouple CGo latency |
|
||||
| all stages ↔ main | `chan struct{}` done channel | closed on Ctrl+C; all stages select on it |
|
||||
### Step 2: Wire `Waveform` through `FreqConfig` and `Layer`
|
||||
|
||||
## Scaling Considerations
|
||||
Add `Waveform` to `FreqConfig`. Update `NewLayer` to pass it to `NewOscillator`. `ClassFreqConfigs` entries default to `WaveformSine` (zero value — valid if `WaveformSine = 0`).
|
||||
|
||||
This is a single-binary CLI tool, not a distributed service. Scaling concerns are throughput-based:
|
||||
Existing tests continue to pass without modification since all existing configs use the zero-value waveform.
|
||||
|
||||
| Traffic Rate | Architecture Adjustments |
|
||||
|--------------|--------------------------|
|
||||
| Home/office (< 10K pps) | Default design handles easily with no tuning |
|
||||
| Datacenter (100K–1M pps) | Increase capture buffer size; consider AF_PACKET with TPACKET_V3 ring buffer instead of pcap; classify goroutine may need fan-out to 2–4 workers |
|
||||
| Line-rate 10G (> 5M pps) | Out of scope for v1 ambient audio tool — synthesis granularity at 500ms windows means exact packet-level accuracy is not required |
|
||||
**Files changed:** `synth/config.go`, `synth/layer.go`.
|
||||
|
||||
### Scaling Priorities
|
||||
### Step 3: Decouple `NewBank` from the global
|
||||
|
||||
1. **First bottleneck:** Kernel pcap buffer drops — mitigated by buffered channels and accepting lossy capture (fine for sonification).
|
||||
2. **Second bottleneck:** CGo encoding latency coupling synthesis clock — mitigated by decoupled encoder goroutine.
|
||||
Change `NewBank(tau float64)` to `NewBank(tau float64, cfgs map[classify.TrafficClass]FreqConfig)`. Update `encode/mp3.go:RunSynthesis` to pass `synth.ClassFreqConfigs` as default.
|
||||
|
||||
At this point the system is functionally identical to v1.0 but `NewBank` no longer reads a global.
|
||||
|
||||
**Files changed:** `synth/bank.go`, `encode/mp3.go`.
|
||||
|
||||
### Step 4: `config` package — TOML structs and file discovery
|
||||
|
||||
Implement `config.Load()`, file discovery, and the `DefaultConfig()` function that wraps `synth.ClassFreqConfigs`. No TOML parsing yet — start with the struct definitions and the merge logic.
|
||||
|
||||
Add `github.com/BurntSushi/toml` dependency (`go get`).
|
||||
|
||||
**Files added:** `config/config.go`.
|
||||
|
||||
### Step 5: `--config` flag and user rule merging in `main.go`
|
||||
|
||||
Wire `config.Load()` into `run()`. Pass user rules to both `runLiveMode` and `runPcapMode`. Pass merged `FreqConfig` map to `RunSynthesis`.
|
||||
|
||||
At this point a minimal TOML config (empty file, or `[[rule]]` only) can be validated end-to-end.
|
||||
|
||||
**Files changed:** `cmd/netsynth/main.go`.
|
||||
|
||||
### Step 6: Custom frequency and waveform overrides in config
|
||||
|
||||
Implement the `[[class]]` TOML section parsing. Add `FreqConfigs()` method to `Config` that returns the merged map (user overrides applied over defaults). Write table-driven tests: "TOML sets HTTPS to 200 Hz sawtooth, bank layer for HTTPS uses 200 Hz sawtooth."
|
||||
|
||||
**Files changed:** `config/config.go`.
|
||||
|
||||
### Step 7: User-defined classes end-to-end
|
||||
|
||||
Support `[[class]]` entries with names not in `classify.AllClasses()`. These become new `TrafficClass` values. User `[[rule]]` entries pointing to these classes are prepended to `DefaultRules`. The bank creates layers for all classes in the merged `FreqConfig` map.
|
||||
|
||||
This step requires the most cross-package coordination but by this point each piece is already in place.
|
||||
|
||||
**Files changed:** `config/config.go`, `cmd/netsynth/main.go` (verification that unknown class names don't panic).
|
||||
|
||||
---
|
||||
|
||||
## Component Boundaries After v1.1
|
||||
|
||||
| Component | Responsibility | Communicates With |
|
||||
|-----------|---------------|-------------------|
|
||||
| `config` | TOML parsing, file discovery, merge logic, `DefaultConfig()` | `synth` (FreqConfig type), `classify` (Rule type) |
|
||||
| `synth/oscillator` | Phase-accumulator for sine/square/sawtooth/triangle | Used by `Layer` |
|
||||
| `synth/bank` | Accepts freq config map, constructs one `Layer` per entry | `encode` passes config map in |
|
||||
| `encode` | Receives config map from `main`, passes to `NewBank` | Thin pass-through |
|
||||
| `cmd/netsynth/main` | Loads config, merges rules, wires all stages | All packages |
|
||||
| `classify` | Rules engine (unchanged); `DefaultRules` stays as package-level var | `main` constructs with merged rules |
|
||||
|
||||
---
|
||||
|
||||
## Critical Integration Constraints
|
||||
|
||||
### `AllClasses()` Is Not the Source of Truth for Bank Construction
|
||||
|
||||
`bank.go` currently iterates `classify.AllClasses()` to construct layers. After v1.1, the bank must iterate the keys of the `FreqConfig` map passed to it. User-defined classes will not appear in `AllClasses()`. If this is not changed, user-defined class packets will be aggregated in `WindowSnapshot.Counts` but have no corresponding layer — they will produce silence and no error.
|
||||
|
||||
**Fix:** `NewBank` iterates `maps.Keys(cfgs)` (or equivalent range over the map), not `classify.AllClasses()`.
|
||||
|
||||
### Class Name Validation Must Happen at Config Load Time
|
||||
|
||||
If a `[[rule]]` references a class name that has no corresponding `[[class]]` entry and is not a builtin, the system will silently mis-classify packets into a layer that doesn't exist. Validate at `config.Load()` time: every class name in `[[rule]]` must resolve to either a builtin `TrafficClass` or a `[[class]]` entry in the same config.
|
||||
|
||||
### `encode.RunSynthesis` Signature Change Is a Breaking API Change
|
||||
|
||||
`encode.RunSynthesis(snapshots []classify.WindowSnapshot, outputPath string)` will need to accept the config. If any external code (tests, future callers) uses this signature, they will break. Keep the change to a single place and update all call sites in the same commit.
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
### Anti-Pattern: Reading Global `ClassFreqConfigs` from Multiple Places
|
||||
|
||||
If `NewBank`, `encode.RunSynthesis`, and config loading all reference the package-level `synth.ClassFreqConfigs`, the merge point becomes ambiguous. The fix (Step 3 above) centralizes config reading to one place: `config.DefaultConfig()` reads from `ClassFreqConfigs` once when building defaults; everything downstream receives the already-merged map.
|
||||
|
||||
### Anti-Pattern: Storing `Waveform` as a String Everywhere
|
||||
|
||||
Keeping `waveform` as a `string` from TOML all the way into `Oscillator` means every advance call parses or switches on a string. Parse the string to a `Waveform` int type at config-load time. The `Oscillator` field should be a typed `Waveform`, not `string`.
|
||||
|
||||
### Anti-Pattern: User Rules Appended After DefaultRules
|
||||
|
||||
User rules must **prepend** `DefaultRules`, not append. `DefaultRules` ends with catch-all rules (`DstPort: 0`) that match any TCP or UDP packet. Appending user rules after these catch-alls means they will never be reached.
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
- Go Pipeline patterns: [Go Concurrency Patterns: Pipelines and cancellation](https://go.dev/blog/pipelines) — HIGH confidence, official Go blog
|
||||
- SoNSTAR network sonification architecture: [Sonification of Network Traffic Flow for Monitoring and Situational Awareness, arXiv 1712.07029](https://arxiv.org/abs/1712.07029) — MEDIUM confidence (abstract only accessed)
|
||||
- gopacket channel API: [gopacket pkg.go.dev](https://pkg.go.dev/github.com/google/gopacket) — HIGH confidence, official package docs
|
||||
- bleep synthesizer architecture (Go): [GitHub bspaans/bleep](https://github.com/bspaans/bleep) — MEDIUM confidence (README inspection)
|
||||
- Waveform synthesis PCM patterns in Go: [Audio From Scratch With Go — Dylan Meeus](https://dylanmeeus.github.io/posts/audio-from-scratch-pt8/) — MEDIUM confidence
|
||||
- go-lame MP3 encoding: [go-lame pkg.go.dev](https://pkg.go.dev/github.com/sunicy/go-lame) — MEDIUM confidence
|
||||
- Drone amplitude/frequency modulation patterns: [Drone auralization model, Acta Acustica 2024](https://acta-acustica.edpsciences.org/articles/aacus/full_html/2024/01/aacus240076/aacus240076.html) — MEDIUM confidence
|
||||
- Direct code inspection: `synth/config.go`, `synth/oscillator.go`, `synth/bank.go`, `synth/layer.go`, `classify/classifier.go`, `classify/rules.go`, `classify/types.go`, `encode/mp3.go`, `cmd/netsynth/main.go` — HIGH confidence
|
||||
- BurntSushi/toml usage in production Go projects (Hugo, dep): MEDIUM confidence (well-known in Go ecosystem)
|
||||
- Phase-accumulator waveform synthesis formulas (square, sawtooth, triangle): HIGH confidence (standard DSP, textbook formulas)
|
||||
|
||||
---
|
||||
*Architecture research for: NetSynth — network-traffic-to-audio synthesis CLI (Go)*
|
||||
*Researched: 2026-03-24*
|
||||
|
||||
*Architecture research for: NetSynth v1.1 — custom sound mappings integration*
|
||||
*Researched: 2026-03-26*
|
||||
|
||||
Reference in New Issue
Block a user