280 lines
19 KiB
Markdown
280 lines
19 KiB
Markdown
# Feature Research
|
|
|
|
**Domain:** Network traffic sonification CLI tool (packet capture -> ambient MP3)
|
|
**Researched:** 2026-03-24 (v1.0), updated 2026-03-26 (v1.1 custom sound mappings)
|
|
**Confidence:** MEDIUM — niche domain; comparable tools are research prototypes or GUI applications, not CLI tools. Table stakes are derived from tcpdump/packet-capture CLI conventions and sonification research literature.
|
|
|
|
---
|
|
|
|
## v1.1 Feature Research: Custom Sound Mappings via TOML Config
|
|
|
|
This section addresses the milestone question: "How do custom sound mapping config files typically work in audio/network tools? What are expected behaviors for config file loading, merging with defaults, validation, and error reporting?"
|
|
|
|
### Config File Loading: Standard Behaviors Expected by CLI Users
|
|
|
|
Based on patterns from established CLI tools (git, golangci-lint, mise, hugo), users expect:
|
|
|
|
1. **Auto-discovery with a defined search order.** The tool looks in a conventional set of locations without requiring an explicit flag. Failing silently (no config found = run with defaults) is correct behavior.
|
|
|
|
2. **Explicit override via a flag.** `--config` (or `-c`) lets users point at a non-standard path. If `--config` is supplied and the file does not exist, that is an error — not silent fallback.
|
|
|
|
3. **Discovery search order (standard precedence):**
|
|
- `--config path/to/file.toml` (explicit flag, highest priority)
|
|
- `./netsynth.toml` (working directory, project-local)
|
|
- `$XDG_CONFIG_HOME/netsynth/config.toml` (defaults to `~/.config/netsynth/config.toml`)
|
|
- No config found → run with all defaults (not an error)
|
|
|
|
This is the pattern used by git (`.git/config` -> `~/.gitconfig` -> `/etc/gitconfig`), golangci-lint (`.golangci.yml` in working dir), and mise (`mise.toml` -> `~/.config/mise/config.toml`). **HIGH confidence** — XDG Base Directory Specification is the Linux/macOS standard.
|
|
|
|
4. **Partial overrides only — not a replacement config.** The config file expresses only what the user wants to change. Absent keys retain default values. This is universally expected: users do not want to replicate the full default table in order to change one frequency.
|
|
|
|
### Config File Merging: How Defaults and User Config Combine
|
|
|
|
The dominant pattern across well-designed CLI tools:
|
|
|
|
**Merge strategy: user values override defaults, defaults fill gaps.**
|
|
|
|
```
|
|
builtin defaults <-- loaded first (in-code, always present)
|
|
+
|
|
user config file <-- loaded second (overrides per-key)
|
|
=
|
|
effective config <-- what the program runs with
|
|
```
|
|
|
|
For NetSynth's classification rules specifically, there are two distinct semantics that must be clearly chosen:
|
|
|
|
- **Override by name:** User supplies a `[rule.DNS]` block that replaces the built-in DNS sound parameters. The predefined DNS rule's classification logic is kept; only its sound output changes.
|
|
- **Prepend user rules:** User-defined rules are inserted before the built-in rule list, allowing them to match first (first-match-wins). This enables the user to add entirely new protocol-to-sound mappings.
|
|
|
|
Both are needed. They serve different use cases:
|
|
- Sound overrides (change frequency/waveform for a known protocol) use the override-by-name pattern.
|
|
- Custom traffic rules (classify "tcp port 8443 as MyApp") use prepend semantics.
|
|
|
|
### Validation: What Users Expect When Config Has Errors
|
|
|
|
Based on patterns in go-toml v2's strict mode and golangci-lint error reporting:
|
|
|
|
**Expected validation behaviors (roughly in order of importance):**
|
|
|
|
| Behavior | Why Expected | Go Implementation Note |
|
|
|----------|--------------|----------------------|
|
|
| Unknown keys caught and reported | Prevents silent typos (user writes `frequncy`, expects it to work) | `go-toml/v2` `DisallowUnknownFields()` or BurntSushi's `Undecoded()` check |
|
|
| Line number in error message | Users need to know where the problem is | Both go-toml/v2 DecodeError and BurntSushi include position info |
|
|
| Human-readable field path | "invalid value for `rules[0].waveform`" not "decode error" | go-toml v2's `DecodeError` produces contextualized messages |
|
|
| Invalid enum values rejected | `waveform = "sqaure"` (typo) should list valid options | Post-decode validation loop with explicit error message listing valid values |
|
|
| Out-of-range numbers rejected | `frequency = -50` or `frequency = 25000` should fail with reason | Post-decode bounds check with message |
|
|
| Missing required fields in new rules | A user rule block missing `protocol` is ambiguous | Post-decode presence check |
|
|
| Config error prevents startup | Do not silently ignore errors and run with partial config | Error should exit with non-zero and print the problem before capturing any packets |
|
|
|
|
**Critical:** Validation errors must surface before capture begins. A user who runs the tool, captures for 30 minutes, then gets a corrupt MP3 because a config value was silently ignored would rightly be frustrated.
|
|
|
|
### Error Reporting: Standard UX Patterns
|
|
|
|
From studying tools in the same class (golangci-lint, hugo, suricata):
|
|
|
|
- Print config errors to **stderr** (not stdout).
|
|
- Prefix with the config file path: `netsynth.toml:12: unknown field "frequncy"`.
|
|
- List ALL errors found in one pass rather than stopping at the first error. Users prefer fixing 5 things in one edit over 5 sequential runs.
|
|
- Warn (not error) for non-fatal issues such as "config file found but empty" or "unknown field in a comment-like position" — but for NetSynth's scope, unknown keys should be hard errors to prevent silent misconfigurations.
|
|
- On `--config path` flag with missing file: hard error immediately.
|
|
- On auto-discovered config with missing file: silent success (no config = defaults).
|
|
|
|
---
|
|
|
|
## Table Stakes for v1.1
|
|
|
|
Features users expect in any CLI tool that introduces a config file. Missing these makes v1.1 feel incomplete.
|
|
|
|
| Feature | Why Expected | Complexity | Depends On |
|
|
|---------|--------------|------------|------------|
|
|
| TOML config file auto-discovery (`./netsynth.toml`, `~/.config/netsynth/config.toml`) | Standard CLI convention; users expect zero-flag discovery | LOW | New: config loader module |
|
|
| `--config` flag for explicit path | Required when multiple configs exist or working dir is wrong | LOW | New: config loader + cobra flag |
|
|
| Partial override semantics (absent keys retain defaults) | Users must not copy the entire default table to change one field | LOW | New: merge logic |
|
|
| Custom frequency per known traffic class | Core v1.1 ask; directly maps to `synth.FreqConfig.BaseHz` | LOW | Existing `synth.ClassFreqConfigs` |
|
|
| Custom waveform per known traffic class | Core v1.1 ask; maps to `synth.Oscillator.Advance()` harmonic shape | MEDIUM | Existing oscillator (needs waveform type support) |
|
|
| User-defined classification rules with custom sounds | Core v1.1 ask; prepend to `classify.DefaultRules` | MEDIUM | Existing `classify.Rule` struct (needs `Class` name generation) |
|
|
| Config validation with line-number errors | Users cannot fix config errors without location info | LOW | go-toml v2 DecodeError (built-in) |
|
|
| Unknown field detection | Prevents silent typos | LOW | go-toml v2 `DisallowUnknownFields()` |
|
|
| Startup-time validation (fail before capture) | No wasted captures with bad config | LOW | Load config in `cmd` root before starting capture |
|
|
| Clear error message listing valid enum values | `waveform` has exactly 4 valid values; list them on error | LOW | Post-decode validation |
|
|
|
|
## Differentiators for v1.1
|
|
|
|
Features that make the config experience polished beyond the minimum.
|
|
|
|
| Feature | Value Proposition | Complexity | Notes |
|
|
|---------|-------------------|------------|-------|
|
|
| `netsynth --print-config` command to dump effective config as TOML | Users want to see what defaults they're overriding; essential for creating a starting-point config file | LOW | Marshal `ClassFreqConfigs` + active rules to TOML; makes discoverability easy |
|
|
| Config documentation via inline comments in generated TOML | When `--print-config` outputs commented TOML, users get self-documenting starting point | LOW | Write comment strings alongside marshaled output |
|
|
| Named custom rules (user assigns a label) | User writes `name = "MyApp"` in a rule block; that name appears in the exit summary and `--verbose` output | LOW | Extend `classify.Rule` to carry optional display name |
|
|
| Waveform preview hint in config error message | "valid waveforms: sine, square, sawtooth, triangle" inline with the error | LOW | Hard-code the valid set in the validator |
|
|
| Harmonic override per class (not just base frequency) | Advanced users can tune the timbre, not just the pitch | MEDIUM | Requires exposing `HarmonicDef` slice in TOML schema; nesting adds parsing complexity |
|
|
|
|
## Anti-Features for v1.1
|
|
|
|
Features that seem natural but should be avoided.
|
|
|
|
| Anti-Feature | Why Avoid | What to Do Instead |
|
|
|--------------|-----------|-------------------|
|
|
| Config file hot-reload during capture | Appears useful but mid-capture parameter change would corrupt synthesis state and produce jarring audio discontinuities | Require restart to apply config changes; document this explicitly |
|
|
| Environment variable config overrides | Adds a third precedence layer (flags > env > file > defaults) that increases combinatorial test surface with low user demand for this tool | Stick to flags + file + defaults; NetSynth is not a server needing 12-factor config |
|
|
| Multiple config file includes / inheritance (`extends = "base.toml"`) | Sounds powerful, creates debugging nightmares when users do not understand the merge order | Single user config file merged with in-code defaults is sufficient; if a user needs multiple environments they can use `--config` |
|
|
| YAML or JSON config format as alternatives | "Why not YAML?" is a common request; supporting multiple formats multiplies parser dependency surface and doubles validation code paths | TOML only; document the choice (TOML is unambiguous, has clean table syntax, is the standard for Go tooling) |
|
|
| Silent partial load on validation error | Some tools load what they can and warn about the rest | Hard error on any invalid field; the user's intent for that field is unknown, so continuing is worse than stopping |
|
|
| Config wizard / interactive setup | Out of scope for a CLI tool with a non-interactive model | Provide `--print-config` with comments as a self-service starting point |
|
|
| Stereo pan position in config | Requested but explicitly deferred in PROJECT.md for this milestone | Out of scope for v1.1; document as v1.2 candidate |
|
|
|
|
---
|
|
|
|
## Feature Dependencies for v1.1
|
|
|
|
```
|
|
[Config file loader (TOML parse + merge)]
|
|
|
|
|
+--provides--> [Custom frequency overrides] (maps to synth.FreqConfig.BaseHz)
|
|
|
|
|
+--provides--> [Custom waveform per class] (requires oscillator waveform dispatch)
|
|
| |
|
|
| +--requires--> [Waveform type in oscillator] (new: sine/square/sawtooth/triangle)
|
|
|
|
|
+--provides--> [User-defined classification rules]
|
|
|
|
|
+--requires--> [Dynamic TrafficClass generation] (new: user rule class names)
|
|
+--prepended-to--> [classify.DefaultRules]
|
|
|
|
[--config flag] --overrides--> [Config file loader search path]
|
|
[--print-config] --reads--> [Effective config after merge] (new subcommand)
|
|
```
|
|
|
|
### Dependency Notes for v1.1
|
|
|
|
- **Waveform type is a new concept in the oscillator.** The v1.0 `Oscillator.Advance()` only does additive sine. To support square/sawtooth/triangle, the oscillator needs a `WaveformType` field and dispatch logic. This is an internal change, but it's required before waveform config can be wired up.
|
|
- **User-defined rules require dynamic `TrafficClass` values.** v1.0 `TrafficClass` is a string type with predefined constants. User rules name their own classes (e.g., `"MyApp"`). The classifier already uses `TrafficClass` as a string; the `synth` layer needs to handle classes not in `ClassFreqConfigs` by looking up user-supplied sound parameters.
|
|
- **Config loading must happen in `cmd` before the capture pipeline starts.** The cobra root command's `RunE` (or `PersistentPreRunE`) function loads and validates config, then passes effective config into the pipeline constructors. This is a structural change to `cmd/root.go`.
|
|
- **`--print-config` is independent** of capture and can be implemented as a separate cobra subcommand reading only the config loader output.
|
|
|
|
---
|
|
|
|
## Implementation Complexity Summary
|
|
|
|
| Feature | Complexity | Reason |
|
|
|---------|------------|--------|
|
|
| Config file loader (TOML parse + merge + validation) | LOW-MEDIUM | go-toml v2 handles parsing; merge logic is a loop; validation is a post-decode pass |
|
|
| Custom frequency per class | LOW | Direct map lookup override; one line per class |
|
|
| Custom waveform per class | MEDIUM | Oscillator needs waveform dispatch (new `WaveformType`); synthesis loop changes |
|
|
| User-defined classification rules | MEDIUM | Dynamic class names; synth layer must handle unknown class names via config lookup |
|
|
| `--config` flag + auto-discovery | LOW | Cobra flag + os.Stat checks on 2-3 paths |
|
|
| `--print-config` subcommand | LOW | Marshal effective config to TOML; add comments |
|
|
| Named custom rules in exit summary | LOW | `classify.Rule` struct gains optional `Name string` field |
|
|
|
|
**No new external dependencies required.** go-toml v2 is the only addition to `go.mod`.
|
|
|
|
---
|
|
|
|
## TOML Schema Sketch (Informational)
|
|
|
|
This is not a binding decision — it informs the roadmap's implementation phase. The schema should feel natural to a user who has seen other Go tool configs (golangci-lint, goreleaser).
|
|
|
|
```toml
|
|
# Override built-in protocol sounds
|
|
[classes.HTTPS]
|
|
frequency = 220.0
|
|
waveform = "square" # sine | square | sawtooth | triangle
|
|
|
|
[classes.DNS]
|
|
frequency = 90.0
|
|
|
|
# Add custom classification rules (prepended before built-in rules, first-match-wins)
|
|
[[rules]]
|
|
name = "Internal API"
|
|
protocol = "tcp"
|
|
port = 8443
|
|
frequency = 300.0
|
|
waveform = "sawtooth"
|
|
|
|
[[rules]]
|
|
name = "Game Traffic"
|
|
protocol = "udp"
|
|
port = 27015
|
|
frequency = 450.0
|
|
waveform = "triangle"
|
|
```
|
|
|
|
Key schema design choices:
|
|
- `[classes.X]` uses the same class name strings already used in `--verbose` output and exit summary (`HTTPS`, `DNS`, etc.) — no new naming system to learn.
|
|
- `[[rules]]` is a TOML array of tables, consistent with how goreleaser and other tools express lists of items.
|
|
- `protocol` and `port` map directly to the existing `classify.Rule` fields, minimizing translation.
|
|
- Waveform is an enum string, not an integer — readable and self-documenting in the config file.
|
|
|
|
---
|
|
|
|
## v1.0 Feature Landscape (Retained from Original Research)
|
|
|
|
### Table Stakes (v1.0)
|
|
|
|
| Feature | Why Expected | Complexity | Notes |
|
|
|---------|--------------|------------|-------|
|
|
| Network interface selection (`-i eth0`) | tcpdump/tshark convention | LOW | Implemented: v1.0 |
|
|
| Output file path flag (`-o output.mp3`) | Any file-producing CLI | LOW | Implemented: v1.0 |
|
|
| Graceful Ctrl+C with file save | Users expect clean finalize | MEDIUM | Implemented: v1.0 |
|
|
| Per-protocol sound distinction | Core value prop | MEDIUM | Implemented: v1.0, 12 rules |
|
|
| Packet count / traffic summary on exit | Every capture tool does this | LOW | Implemented: v1.0 |
|
|
| Privilege error message | Silent pcap failure is confusing | LOW | Implemented: v1.0 |
|
|
| List available interfaces (`--list-interfaces`) | Users don't know interface names | LOW | Implemented: v1.0 |
|
|
| Minimum viable duration guard | Zero-packet = no corrupt MP3 | LOW | Implemented: v1.0 |
|
|
|
|
### Differentiators (v1.0)
|
|
|
|
| Feature | Value Proposition | Complexity | Status |
|
|
|---------|-------------------|------------|--------|
|
|
| Auto-clustering of unrecognized traffic | Honest audio fingerprint | HIGH | Implemented: hash-bucket, 4 classes |
|
|
| Ambient/drone style (layered sine harmonics) | Distinct from event-ping tools | HIGH | Implemented: v1.0 |
|
|
| Time-windowed amplitude evolution | Mix evolves dynamically | MEDIUM | Implemented: 500ms windows + EMA |
|
|
| BPF capture filter (`--filter`) | Power users scope what's sonified | MEDIUM | Implemented: v1.0 |
|
|
| Offline pcap file input (`--read`) | Sonify historical captures | MEDIUM | Implemented: v1.0 |
|
|
| Verbose protocol activity log (`--verbose`) | Developers see classifications | LOW | Implemented: v1.0 |
|
|
|
|
### Anti-Features (v1.0)
|
|
|
|
| Feature | Why Avoided |
|
|
|---------|-------------|
|
|
| Real-time audio playback | Platform audio API complexity; file output is correct |
|
|
| GUI or web dashboard | Negates single-binary CLI value |
|
|
| Custom sound mapping (v1.0) | Deferred to v1.1 — now the current milestone |
|
|
| Rhythmic/percussive output | Ambient/drone is the deliberate differentiator |
|
|
| Deep-packet inspection | Massive complexity; header classification sufficient |
|
|
| Streaming MP3 output | MP3 finalization requires full buffer |
|
|
| Anomaly detection / alerting | Different user job |
|
|
|
|
---
|
|
|
|
## Competitor Feature Analysis
|
|
|
|
| Feature | SoNSTAR (Python) | Network-Sonification (C# GUI) | Peep (C, Unix) | NetSynth v1.0 | NetSynth v1.1 |
|
|
|---------|-----------------|-------------------------------|----------------|----------------|----------------|
|
|
| Custom sound config | No | No | Config file (fixed format) | No | Yes (TOML) |
|
|
| Config file discovery | n/a | n/a | Hardcoded path | n/a | XDG + working dir |
|
|
| Partial override semantics | n/a | n/a | Full replacement | n/a | Partial override |
|
|
| Waveform selection | Recorded samples | sine/square/triangle | Fixed | sine only | sine/square/sawtooth/triangle |
|
|
| Custom classification rules | No | No | No | No | Yes (user-defined port/proto rules) |
|
|
| Named custom classes | n/a | n/a | n/a | n/a | Yes (appears in summary output) |
|
|
|
|
---
|
|
|
|
## Sources
|
|
|
|
- [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir/latest/) — standard for `~/.config` discovery path
|
|
- [adrg/xdg — Go XDG implementation](https://github.com/adrg/xdg) — if explicit XDG library is needed (probably not for NetSynth's 2-path lookup)
|
|
- [pelletier/go-toml v2 — strict mode and DecodeError](https://pkg.go.dev/github.com/pelletier/go-toml/v2) — recommended TOML library; DisallowUnknownFields() and human-readable errors
|
|
- [BurntSushi/toml — Undecoded() for unknown key detection](https://github.com/BurntSushi/toml) — alternative; simpler API but less actively maintained
|
|
- [Building CLI Applications with Go: Cobra and Viper Guide (2026)](https://dasroot.net/posts/2026/03/building-cli-applications-go-cobra-viper/) — config loading patterns in Cobra CLI tools
|
|
- [A Guide to TOML in Golang — kelche.co](https://www.kelche.co/blog/go/toml/) — go-toml v2 vs BurntSushi comparison and practical examples
|
|
- [Configuration | mise-en-place](https://mise.jdx.dev/configuration.html) — example of working-dir + XDG config discovery
|
|
- [golangci-lint configuration](https://golangci-lint.run/docs/configuration/cli/) — real-world example of partial override config in a Go CLI tool
|
|
- [Online Tone Generator — waveform types](https://onlinetonegenerator.com/) — confirms sine/square/sawtooth/triangle as the standard 4 waveform set
|
|
|
|
---
|
|
*v1.0 research: 2026-03-24*
|
|
*v1.1 custom sound mappings research: 2026-03-26*
|