gurix/yoloyolo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: complete project research

Browse Source
This commit is contained in:
gurix
2026-03-27 08:25:02 +01:00
parent 6de2fcbb99
commit 6b26be825c
5 changed files with 1341 additions and 1107 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.planning/research/FEATURES.md
+292 -205
View File
@@ -1,279 +1,366 @@
# Feature Research
**Domain:** Network traffic sonification CLI tool (packet capture -> ambient MP3)
**Researched:** 2026-03-24 (v1.0), updated 2026-03-26 (v1.1 custom sound mappings)
**Researched:** 2026-03-24 (v1.0), updated 2026-03-26 (v1.1 custom sound mappings), updated 2026-03-27 (v1.2 extended protocol coverage)
**Confidence:** MEDIUM — niche domain; comparable tools are research prototypes or GUI applications, not CLI tools. Table stakes are derived from tcpdump/packet-capture CLI conventions and sonification research literature.
---
## v1.1 Feature Research: Custom Sound Mappings via TOML Config
## v1.2 Feature Research: Extended Protocol Coverage with Grouped Protocol Families
This section addresses the milestone question: "How do custom sound mapping config files typically work in audio/network tools? What are expected behaviors for config file loading, merging with defaults, validation, and error reporting?"
This section addresses the milestone question: "How should protocol grouping work for network sonification? What are the most common protocols seen on typical networks? Which protocols are worth adding? What's the right granularity — individual protocol vs group?"
### Config File Loading: Standard Behaviors Expected by CLI Users
### What Real Networks Actually See
Based on patterns from established CLI tools (git, golangci-lint, mise, hugo), users expect:
Based on IANA well-known ports, nDPI's traffic classification taxonomy (450+ protocols, 17 application categories), Wireshark's protocol dissector set, and widely-cited network security references, traffic on real networks breaks down into recognizable families.
1. **Auto-discovery with a defined search order.** The tool looks in a conventional set of locations without requiring an explicit flag. Failing silently (no config found = run with defaults) is correct behavior.
**Home network (residential broadband):**
The dominant traffic types are HTTPS (streaming, browsing, cloud sync), DNS (constant background noise, every connection starts here), NTP (infrequent but present on all devices), DHCP (device join/renew events), ICMP (ping, router probe), and mDNS/SSDP (device discovery on the local segment). IoT device traffic adds MQTT. Video calls add RTP/SRTP.
2. **Explicit override via a flag.** `--config` (or `-c`) lets users point at a non-standard path. If `--config` is supplied and the file does not exist, that is an error — not silent fallback.
**Office network (enterprise LAN):**
Same HTTPS/DNS/NTP/DHCP base, plus: LDAP/Kerberos (domain auth), SMB (file sharing), RDP (remote desktop), SMTP/IMAP/POP3 (mail), SNMP (monitoring), syslog (log aggregation), SSH (server access), FTP (legacy file transfer still common in many environments).
3. **Discovery search order (standard precedence):**
- `--config path/to/file.toml` (explicit flag, highest priority)
- `./netsynth.toml` (working directory, project-local)
- `$XDG_CONFIG_HOME/netsynth/config.toml` (defaults to `~/.config/netsynth/config.toml`)
- No config found → run with all defaults (not an error)
**Server host (Linux box exposed to internet):**
SSH (constant scan attempts), HTTPS (serving), DNS (resolver queries), ICMP (reachability probes), NTP (drift correction), syslog (local log collection), PostgreSQL/MySQL/Redis (local app traffic), SMTP (outbound mail relay).
This is the pattern used by git (`.git/config` -> `~/.gitconfig` -> `/etc/gitconfig`), golangci-lint (`.golangci.yml` in working dir), and mise (`mise.toml` -> `~/.config/mise/config.toml`). **HIGH confidence** — XDG Base Directory Specification is the Linux/macOS standard.
**Cloud workload:**
HTTPS dominates. DNS for service discovery. Redis/PostgreSQL for app data. Kafka/AMQP for message queues. gRPC (still port 443 via HTTP/2). ICMP suppressed. NTP locked down.
4. **Partial overrides only — not a replacement config.** The config file expresses only what the user wants to change. Absent keys retain default values. This is universally expected: users do not want to replicate the full default table in order to change one frequency.
### Why Protocol Grouping Matters for Sonification
### Config File Merging: How Defaults and User Config Combine
Without grouping, adding 20+ individual protocols creates:
1. **Spectrum crowding:** 20+ tones across the audible range become indistinguishable mud.
2. **No perceptual structure:** Listeners cannot form a mental model of what they are hearing.
3. **Frequency allocation complexity:** Designing 20+ non-conflicting frequency slots is difficult.
The dominant pattern across well-designed CLI tools:
The nDPI project (the leading open-source DPI library, used by ntopng) faced the same problem and solved it with 17 application categories that group hundreds of protocols. For sonification, the goal is different — not to identify every protocol, but to produce a soundscape where related activity sounds related.
**Merge strategy: user values override defaults, defaults fill gaps.**
**Recommended approach: Protocol families, not individual protocols.**
```
builtin defaults <-- loaded first (in-code, always present)
+
user config file <-- loaded second (overrides per-key)
=
effective config <-- what the program runs with
```
A "Mail" family chord — one tonal cluster covering SMTP + IMAP + POP3 — is more musically coherent and more perceptually useful than three separate isolated tones. A user can hear "mail activity is elevated" rather than "something is happening on port 143."
For NetSynth's classification rules specifically, there are two distinct semantics that must be clearly chosen:
### Protocol Grouping Taxonomy for NetSynth v1.2
- **Override by name:** User supplies a `[rule.DNS]` block that replaces the built-in DNS sound parameters. The predefined DNS rule's classification logic is kept; only its sound output changes.
- **Prepend user rules:** User-defined rules are inserted before the built-in rule list, allowing them to match first (first-match-wins). This enables the user to add entirely new protocol-to-sound mappings.
After surveying IANA port assignments, nDPI's category taxonomy, and the Wireshark protocol support matrix, the following family groupings are recommended. Each family occupies a frequency region (not individual tones), and protocols within a family use slight detuning or waveform variation to stay distinct.
Both are needed. They serve different use cases:
- Sound overrides (change frequency/waveform for a known protocol) use the override-by-name pattern.
- Custom traffic rules (classify "tcp port 8443 as MyApp") use prepend semantics.
**Family: Web (already covered)**
- HTTPS / TLS (port 443) — already built in
- HTTP (port 80) — already built in
- HTTP/3 / QUIC (port 443 UDP) — port-based detection is possible (UDP 443)
- Alt-HTTPS ports (8443, 8080) — users can add via custom rules
### Validation: What Users Expect When Config Has Errors
**Family: Mail**
- SMTP (port 25) — already built in (single class)
- SMTP submission (port 587, 465) — distinct submission path, worth adding
- IMAP (port 143, 993) — pull email; very common on office networks
- POP3 (port 110, 995) — legacy pull email; still common with older mail clients
Based on patterns in go-toml v2's strict mode and golangci-lint error reporting:
**Family: Remote Access**
- SSH (port 22) — already built in
- Telnet (port 23) — unencrypted, legacy; worth representing (security signal)
- RDP (port 3389) — Windows remote desktop; ubiquitous in enterprise
- VNC (port 5900) — remote frame buffer; common on servers/developer machines
**Expected validation behaviors (roughly in order of importance):**
**Family: Infrastructure (already partially covered)**
- DNS (port 53) — already built in
- DHCP (ports 67/68) — already built in
- NTP (port 123) — already built in
- mDNS (port 5353 UDP) — Bonjour/Avahi discovery; prominent on home/office LANs
- SSDP (port 1900 UDP) — UPnP device discovery; common with IoT devices
- LLMNR (port 5355) — Windows local name resolution; common on Windows networks
- SNMP (port 161/162 UDP) — network monitoring; common on office/server networks
- Syslog (port 514 UDP) — log forwarding; common on server and enterprise networks
| Behavior | Why Expected | Go Implementation Note |
|----------|--------------|----------------------|
| Unknown keys caught and reported | Prevents silent typos (user writes `frequncy`, expects it to work) | `go-toml/v2` `DisallowUnknownFields()` or BurntSushi's `Undecoded()` check |
| Line number in error message | Users need to know where the problem is | Both go-toml/v2 DecodeError and BurntSushi include position info |
| Human-readable field path | "invalid value for `rules[0].waveform`" not "decode error" | go-toml v2's `DecodeError` produces contextualized messages |
| Invalid enum values rejected | `waveform = "sqaure"` (typo) should list valid options | Post-decode validation loop with explicit error message listing valid values |
| Out-of-range numbers rejected | `frequency = -50` or `frequency = 25000` should fail with reason | Post-decode bounds check with message |
| Missing required fields in new rules | A user rule block missing `protocol` is ambiguous | Post-decode presence check |
| Config error prevents startup | Do not silently ignore errors and run with partial config | Error should exit with non-zero and print the problem before capturing any packets |
**Family: File Transfer**
- FTP (port 21) — still widely used in legacy environments, NAS devices
- TFTP (port 69 UDP) — boot/config transfer; common in network infrastructure (switches, PXE boot)
- SMB (port 445) — Windows file sharing; ubiquitous on any Windows or Samba network
**Critical:** Validation errors must surface before capture begins. A user who runs the tool, captures for 30 minutes, then gets a corrupt MP3 because a config value was silently ignored would rightly be frustrated.
**Family: Database**
- MySQL/MariaDB (port 3306) — most common SQL database port
- PostgreSQL (port 5432) — second most common SQL database
- Redis (port 6379) — in-memory cache; present on almost every modern app server
- MongoDB (port 27017) — document store; very common in web apps
### Error Reporting: Standard UX Patterns
**Family: VoIP / Real-Time**
- SIP (port 5060/5061) — VoIP signaling; present in any office with IP phones
- RTP (ports 16384-32767 UDP, dynamic) — voice/video payload; hard to detect by port alone
From studying tools in the same class (golangci-lint, hugo, suricata):
**Family: Directory / Authentication**
- LDAP (port 389, 636) — Active Directory / OpenLDAP; present on any enterprise network
- Kerberos (port 88) — Active Directory authentication; present on any Windows domain network
- Print config errors to **stderr** (not stdout).
- Prefix with the config file path: `netsynth.toml:12: unknown field "frequncy"`.
- List ALL errors found in one pass rather than stopping at the first error. Users prefer fixing 5 things in one edit over 5 sequential runs.
- Warn (not error) for non-fatal issues such as "config file found but empty" or "unknown field in a comment-like position" — but for NetSynth's scope, unknown keys should be hard errors to prevent silent misconfigurations.
- On `--config path` flag with missing file: hard error immediately.
- On auto-discovered config with missing file: silent success (no config = defaults).
Note: gopacket/layers does NOT natively decode SMTP, IMAP, POP3, FTP, TFTP, SMB, MySQL, PostgreSQL, Redis, SIP, RTP, LDAP, or Kerberos at the application layer. Classification must happen at the transport layer (port number matching), which is exactly how the existing Rule classifier works. No new dependencies required.
### Which Protocols to Include: Priority Tiers
**Tier 1 — Add immediately (high real-world frequency, low complexity)**
| Protocol | Ports | Network Type | Detection Method | Family |
|----------|-------|-------------|-----------------|--------|
| IMAP / IMAPS | TCP 143, 993 | Home, Office, Server | Port match | Mail |
| POP3 / POP3S | TCP 110, 995 | Home, Office | Port match | Mail |
| SMTP Submission | TCP 587, 465 | Home, Office, Server | Port match | Mail |
| FTP | TCP 20, 21 | Office, Server, NAS | Port match | File Transfer |
| SMB | TCP 445 | Office, Windows networks | Port match | File Transfer |
| RDP | TCP 3389 | Office, Enterprise | Port match | Remote Access |
| mDNS | UDP 5353 | Home, Office | Port match | Infrastructure |
| SSDP | UDP 1900 | Home, IoT | Port match | Infrastructure |
| SNMP | UDP 161, 162 | Office, Server | Port match | Infrastructure |
| MySQL | TCP 3306 | Server, Cloud | Port match | Database |
| PostgreSQL | TCP 5432 | Server, Cloud | Port match | Database |
| Redis | TCP 6379 | Server, Cloud | Port match | Database |
**Tier 2 — Include if groups are being formed (moderate frequency)**
| Protocol | Ports | Network Type | Detection Method | Family |
|----------|-------|-------------|-----------------|--------|
| Telnet | TCP 23 | Legacy, embedded | Port match | Remote Access |
| VNC | TCP 5900 | Office, Developer | Port match | Remote Access |
| TFTP | UDP 69 | Infrastructure, PXE | Port match | File Transfer |
| SIP | TCP/UDP 5060, 5061 | Office, VoIP | Port match | VoIP |
| LDAP / LDAPS | TCP 389, 636 | Enterprise | Port match | Directory/Auth |
| Kerberos | UDP/TCP 88 | Enterprise, Windows | Port match | Directory/Auth |
| Syslog | UDP 514 | Server, Enterprise | Port match | Infrastructure |
| MongoDB | TCP 27017 | Server, Cloud | Port match | Database |
| QUIC/HTTP3 | UDP 443 | Home, Office | Port match (UDP 443) | Web |
**Tier 3 — Defer or leave to user custom rules**
| Protocol | Reason to Defer |
|----------|----------------|
| MQTT (TCP 1883/8883) | IoT-specific; not present on most general networks |
| AMQP (TCP 5672) | Message queue; only relevant on specific server workloads |
| Kafka (TCP 9092) | Rarely seen outside distributed systems environments |
| BGP (TCP 179) | Routing protocol; not visible on end-host captures |
| OSPF | Link-state routing; IP protocol 89, not TCP/UDP; gopacket has native decoder but it is infrastructure traffic only |
| RADIUS (UDP 1812/1813) | Authentication forwarding; only on network infrastructure |
| gRPC | Uses HTTP/2 on port 443; indistinguishable from HTTPS at transport layer |
| XMPP (TCP 5222) | Near-obsolete for general messaging |
### What Granularity Is Right: Individual Protocol vs Group?
**Recommendation: Implement individual protocol classes, but group them for frequency allocation.**
This gives the user the most information value while maintaining musical coherence:
- Each protocol gets a distinct `TrafficClass` constant and rule (e.g., `ClassIMAP`, `ClassSMB`).
- Related protocols are assigned frequencies within a shared family frequency band (e.g., all Mail protocols live in 400-550 Hz).
- Within a family band, slight detuning (5-15 Hz apart) produces audible distinction without crowding.
- Family identity is perceivable because the tones are harmonically close.
This matches how nDPI handles the tension: individual protocol identity for precise classification, category grouping for user-facing display and policy. NetSynth's "display" is sonic — the grouping manifests as tonal proximity.
The alternative — collapsing IMAP+SMTP+POP3 into a single "Mail" class — loses information. A user cannot tell whether mail noise is inbound (IMAP) or outbound (SMTP). Individual classes preserve that distinction.
### Frequency Allocation for New Families
The current frequency map uses 65 Hz – 1047 Hz across 14 classes. Adding ~20 new protocols requires rethinking the allocation.
**Current allocation issues:**
- The range from 65-110 Hz (ICMP, DNS) is very low; adding families in this region creates muddiness.
- The unknown buckets at 862-1047 Hz occupy space that could be used for real protocols.
- The spread from 600-780 Hz (DHCP, OtherTCP, OtherUDP) is dense.
**Recommended approach for v1.2:**
- Assign families to distinct octave/register bands rather than a linear frequency sweep.
- Use musically meaningful intervals within each family (minor thirds, perfect fourths — intervals that sound related without clashing).
- Keep the existing 10 class frequencies backward-compatible; assign new protocols to new frequency slots.
- Push unknown buckets above 1200 Hz (the current auto-assign range already does this via FNV hash).
**Proposed family frequency bands:**
| Family | Band | Rationale |
|--------|------|-----------|
| Infrastructure (DNS, DHCP, NTP, mDNS, SNMP, Syslog, SSDP) | 80-200 Hz | Low, grounding tones; infrastructure is the "bass" of the network |
| Web (HTTP, HTTPS, QUIC) | 220-320 Hz | Mid-low; dominant traffic, warm register |
| Mail (SMTP, IMAP, POP3) | 350-480 Hz | Mid; distinct from web, harmonically separate |
| Remote Access (SSH, Telnet, RDP, VNC) | 500-620 Hz | Mid-high; noticeable — admin activity is important |
| File Transfer (FTP, TFTP, SMB) | 650-750 Hz | Upper-mid; distinct texture |
| Database (MySQL, PostgreSQL, Redis, MongoDB) | 800-950 Hz | Upper register; database chatter is server-side signal |
| Directory / Auth (LDAP, Kerberos) | 960-1050 Hz | High; auth traffic is sparse but significant |
| VoIP (SIP, RTP) | 1100-1200 Hz | High; real-time traffic stands out |
| Unknown buckets | 1300+ Hz | Highest; unclassified traffic is "noise above the signal" |
| ICMP | 65 Hz | Remains as fundamental ping pulse below all families |
---
## Table Stakes for v1.1
## Table Stakes for v1.2
Features users expect in any CLI tool that introduces a config file. Missing these makes v1.1 feel incomplete.
Features required to call v1.2 complete. Missing these means the milestone goal ("expanded protocol classification with grouped protocol families") is not delivered.
| Feature | Why Expected | Complexity | Depends On |
|---------|--------------|------------|------------|
| TOML config file auto-discovery (`./netsynth.toml`, `~/.config/netsynth/config.toml`) | Standard CLI convention; users expect zero-flag discovery | LOW | New: config loader module |
| `--config` flag for explicit path | Required when multiple configs exist or working dir is wrong | LOW | New: config loader + cobra flag |
| Partial override semantics (absent keys retain defaults) | Users must not copy the entire default table to change one field | LOW | New: merge logic |
| Custom frequency per known traffic class | Core v1.1 ask; directly maps to `synth.FreqConfig.BaseHz` | LOW | Existing `synth.ClassFreqConfigs` |
| Custom waveform per known traffic class | Core v1.1 ask; maps to `synth.Oscillator.Advance()` harmonic shape | MEDIUM | Existing oscillator (needs waveform type support) |
| User-defined classification rules with custom sounds | Core v1.1 ask; prepend to `classify.DefaultRules` | MEDIUM | Existing `classify.Rule` struct (needs `Class` name generation) |
| Config validation with line-number errors | Users cannot fix config errors without location info | LOW | go-toml v2 DecodeError (built-in) |
| Unknown field detection | Prevents silent typos | LOW | go-toml v2 `DisallowUnknownFields()` |
| Startup-time validation (fail before capture) | No wasted captures with bad config | LOW | Load config in `cmd` root before starting capture |
| Clear error message listing valid enum values | `waveform` has exactly 4 valid values; list them on error | LOW | Post-decode validation |
| Mail family (IMAP, IMAPS, POP3, POP3S, SMTP submission 587/465) | Requested directly in todo; mail traffic is high-frequency on any office network | LOW | Existing Rule/TrafficClass pattern; add constants + rules |
| Remote Access expansion (RDP, VNC, Telnet) | SSH is already present; the family is incomplete without RDP on enterprise captures | LOW | Same as above |
| File Transfer family (FTP, SMB, TFTP) | FTP/SMB appear on almost every office or NAS-connected home network | LOW | Same as above |
| Infrastructure expansion (mDNS, SSDP, SNMP, Syslog) | These are constant background noise on every LAN; without them they land in other-UDP | LOW | Same as above |
| Database family (MySQL, PostgreSQL, Redis, MongoDB) | Any developer machine has these; they currently all land in other-TCP | LOW | Same as above |
| Frequency rebalancing to accommodate new classes | Without rebalancing, the new classes crowd the existing spectrum | MEDIUM | requires touching synth.ClassFreqConfigs; backward-compatible if existing class constants keep their names |
| TrafficClass constants and AllClasses() updated | Config, synth, and print-config must know about new classes | LOW | classify/types.go extension |
| DefaultRules updated with new port rules | New classes only work if packets reach them via rules | LOW | classify/rules.go extension |
| --print-config reflects new classes | Users need to see and override the new classes | LOW | Falls out automatically once ClassFreqConfigs and AllClasses() are updated |
| Group concept exposed in --print-config | Grouped comments (# Mail family, # Database family) make the config readable | LOW | PrintConfig formatting only; no struct changes needed |
## Differentiators for v1.1
## Differentiators for v1.2
Features that make the config experience polished beyond the minimum.
Features that make the extended protocol coverage polished beyond the minimum.
| Feature | Value Proposition | Complexity | Notes |
|---------|-------------------|------------|-------|
| `netsynth --print-config` command to dump effective config as TOML | Users want to see what defaults they're overriding; essential for creating a starting-point config file | LOW | Marshal `ClassFreqConfigs` + active rules to TOML; makes discoverability easy |
| Config documentation via inline comments in generated TOML | When `--print-config` outputs commented TOML, users get self-documenting starting point | LOW | Write comment strings alongside marshaled output |
| Named custom rules (user assigns a label) | User writes `name = "MyApp"` in a rule block; that name appears in the exit summary and `--verbose` output | LOW | Extend `classify.Rule` to carry optional display name |
| Waveform preview hint in config error message | "valid waveforms: sine, square, sawtooth, triangle" inline with the error | LOW | Hard-code the valid set in the validator |
| Harmonic override per class (not just base frequency) | Advanced users can tune the timbre, not just the pitch | MEDIUM | Requires exposing `HarmonicDef` slice in TOML schema; nesting adds parsing complexity |
| Within-family tonal design: shared waveform within a family | Mail protocols all use the same waveform (e.g., sawtooth); listeners perceive the family by timbre as well as frequency | LOW | Assign waveform by family at design time; no new code needed |
| Within-family detuning: minor third intervals between protocols in a family | Protocols within a family are harmonically close; the family "chord" is identifiable | LOW | Frequency assignment arithmetic at design time |
| Directory/Auth family (LDAP, Kerberos) | Present on every enterprise network; their absence means enterprise traffic sounds like "other-TCP" noise | LOW | 2 more class constants + rules |
| VoIP family (SIP, SIP-TLS) | IP phone traffic is prominent in offices; SIP port 5060 is easily matched | LOW | 2 more class constants + rules |
| QUIC / HTTP3 class (UDP 443) | HTTP/3 now represents a significant fraction of web traffic; treating it identically to HTTPS when it arrives via UDP is perceptually meaningful | LOW | 1 rule: UDP port 443 maps to ClassQUIC or ClassHTTPS3 |
## Anti-Features for v1.1
## Anti-Features for v1.2
Features that seem natural but should be avoided.
Features that seem natural for this milestone but should be avoided.
| Anti-Feature | Why Avoid | What to Do Instead |
|--------------|-----------|-------------------|
| Config file hot-reload during capture | Appears useful but mid-capture parameter change would corrupt synthesis state and produce jarring audio discontinuities | Require restart to apply config changes; document this explicitly |
| Environment variable config overrides | Adds a third precedence layer (flags > env > file > defaults) that increases combinatorial test surface with low user demand for this tool | Stick to flags + file + defaults; NetSynth is not a server needing 12-factor config |
| Multiple config file includes / inheritance (`extends = "base.toml"`) | Sounds powerful, creates debugging nightmares when users do not understand the merge order | Single user config file merged with in-code defaults is sufficient; if a user needs multiple environments they can use `--config` |
| YAML or JSON config format as alternatives | "Why not YAML?" is a common request; supporting multiple formats multiplies parser dependency surface and doubles validation code paths | TOML only; document the choice (TOML is unambiguous, has clean table syntax, is the standard for Go tooling) |
| Silent partial load on validation error | Some tools load what they can and warn about the rest | Hard error on any invalid field; the user's intent for that field is unknown, so continuing is worse than stopping |
| Config wizard / interactive setup | Out of scope for a CLI tool with a non-interactive model | Provide `--print-config` with comments as a self-service starting point |
| Stereo pan position in config | Requested but explicitly deferred in PROJECT.md for this milestone | Out of scope for v1.1; document as v1.2 candidate |
| Application-layer (DPI) detection | gopacket does not decode SMTP, SMB, Redis, etc. at the application layer. Implementing DPI requires a full protocol parser per protocol — massive scope. | Port-number matching is sufficient for classification purposes; DPI adds complexity without enough sonification value |
| Single "Mail" or "Database" class (collapsing all protocols in family) | Loses per-protocol information. "Mail" does not tell you if it is inbound or outbound. "Database" does not distinguish Redis latency spikes from a slow Postgres query. | Keep individual TrafficClass per protocol; use frequency proximity for family grouping |
| Dynamic port detection (e.g., FTP data port 20 bidirectional, ephemeral RTP ports) | FTP uses negotiated dynamic ports for data transfer; RTP uses ports negotiated over SIP. Tracking these requires stateful flow tracking across packets — significant architectural change. | Classify on well-known control/server ports only; dynamic data flows land in other-TCP/UDP. Document this limitation. |
| Runtime group concept (Group struct with members) | Adding a Group abstraction to TrafficClass, Rule, or FreqConfig requires touching multiple packages and complicates the TOML schema. | Groups are a classification/display concept only, not a data structure. Implement them as naming conventions and config comment sections. |
| Backward-incompatible frequency changes to existing 10 classes | Users who have existing TOML configs relying on the current frequencies would have their carefully tuned soundscapes broken | Keep ICMP=65Hz, DNS=110Hz, HTTPS=175Hz, HTTP=220Hz, SSH=330Hz, SMTP=440Hz, NTP=520Hz, DHCP=600Hz, OtherTCP=700Hz, OtherUDP=780Hz; assign new protocols to unoccupied slots |
| Replacing other-TCP / other-UDP with something smarter | The catch-all classes serve an important role: unrecognized traffic is still represented. Removing them creates silent gaps. | Keep other-TCP and other-UDP as catch-alls; new specific classes reduce how much traffic lands there |
| SRC port matching rules | Some protocols run on ephemeral source ports; adding src-port rules would double rule count and create false matches. Current architecture matches dst-port only. | Stick to dst-port matching. This is how nmap, iptables, and most classifiers work by default. |
## Feature Dependencies for v1.2
```
[classify/types.go: add ~20 new TrafficClass constants]
|
+--enables--> [classify/rules.go: add new Rule entries per protocol]
| |
| +--feeds--> [classify.Classifier: matches packets to new classes]
|
+--enables--> [synth/config.go: add FreqConfig entries for new classes]
| |
| +--requires--> [Frequency rebalancing: shift new classes into family bands]
| | (backward-compatible: existing 10 classes unchanged)
| |
| +--feeds--> [synth.Bank: synthesizes new layers]
| (NumLayers constant must increase from 14 to cover new classes)
|
+--enables--> [classify.AllClasses(): include new classes in display order]
|
+--feeds--> [config.PrintConfig(): groups appear in --print-config output]
+--feeds--> [aggregate.Summary: new classes appear in exit summary]
```
### Dependency Notes for v1.2
- **NumLayers constant must increase.** `synth/config.go` has `NumLayers = 14` and derives `GainPerLayer = 1.0 / float64(NumLayers)` from it. Adding 20 protocols brings total classes to ~34. `NumLayers` must be updated, or the gain calculation must become dynamic. This is a straightforward arithmetic change but affects all layers' amplitude. Test the mix with more layers to confirm it still sounds balanced.
- **AllClasses() ordering determines --print-config output order.** Currently returns a flat slice. For v1.2, ordering by family group (all Mail classes together, all Database classes together) makes --print-config more readable. This is a display concern only — the order has no effect on classification.
- **No new external dependencies required.** All new protocols are detected via port number using the existing Rule struct. gopacket is not being asked to decode new application-layer protocols.
- **QUIC/HTTP3 requires a UDP 443 rule.** The current rules only match TCP 443 for HTTPS. Adding a separate rule for UDP 443 is one line. The question is naming: `ClassHTTPS3` or `ClassQUIC`. QUIC is the transport; HTTP/3 is the application. For sonification purposes, `ClassQUIC` is clearer because it describes the observable port behavior.
- **SIP detection covers control plane only.** SIP signals calls on port 5060/5061, but the actual voice/video payload travels via RTP on dynamically negotiated ports (typically in 16384-32767 range). The current classifier cannot detect RTP payloads without stateful flow tracking. Classify SIP only; document that RTP payload traffic lands in other-UDP.
---
## Feature Dependencies for v1.1
## Protocol List: Final Recommended Set
```
[Config file loader (TOML parse + merge)]
|
+--provides--> [Custom frequency overrides] (maps to synth.FreqConfig.BaseHz)
|
+--provides--> [Custom waveform per class] (requires oscillator waveform dispatch)
| |
| +--requires--> [Waveform type in oscillator] (new: sine/square/sawtooth/triangle)
|
+--provides--> [User-defined classification rules]
|
+--requires--> [Dynamic TrafficClass generation] (new: user rule class names)
+--prepended-to--> [classify.DefaultRules]
This is the complete recommended protocol class list for v1.2, including existing + new.
[--config flag] --overrides--> [Config file loader search path]
[--print-config] --reads--> [Effective config after merge] (new subcommand)
```
### Existing (unchanged, backward-compatible)
| Class | Protocol | Port | Transport |
|-------|----------|------|-----------|
| ICMP | ICMP/ICMPv6 | — | ICMP |
| DNS | Domain Name System | 53 | TCP+UDP |
| HTTPS | HTTP Secure / TLS | 443 | TCP |
| HTTP | HTTP | 80 | TCP |
| SSH | Secure Shell | 22 | TCP |
| SMTP | Mail Transfer (server-to-server) | 25 | TCP |
| NTP | Network Time Protocol | 123 | UDP |
| DHCP | Dynamic Host Config | 67, 68 | UDP |
| other-TCP | Unclassified TCP | — | TCP |
| other-UDP | Unclassified UDP | — | UDP |
| unknown-1..4 | Hash-bucketed unknowns | — | any |
### Dependency Notes for v1.1
### New: Tier 1 (high frequency, recommended for v1.2)
| Class | Protocol | Port | Transport | Family |
|-------|----------|------|-----------|--------|
| IMAP | IMAP / IMAPS | 143, 993 | TCP | Mail |
| POP3 | POP3 / POP3S | 110, 995 | TCP | Mail |
| SMTP-Submission | SMTP client submission | 587, 465 | TCP | Mail |
| FTP | File Transfer Protocol | 20, 21 | TCP | File Transfer |
| SMB | Server Message Block | 445 | TCP | File Transfer |
| RDP | Remote Desktop Protocol | 3389 | TCP | Remote Access |
| mDNS | Multicast DNS (Bonjour) | 5353 | UDP | Infrastructure |
| SSDP | Simple Service Discovery | 1900 | UDP | Infrastructure |
| SNMP | Simple Network Mgmt | 161, 162 | UDP | Infrastructure |
| MySQL | MySQL database | 3306 | TCP | Database |
| PostgreSQL | PostgreSQL database | 5432 | TCP | Database |
| Redis | Redis in-memory store | 6379 | TCP | Database |
- **Waveform type is a new concept in the oscillator.** The v1.0 `Oscillator.Advance()` only does additive sine. To support square/sawtooth/triangle, the oscillator needs a `WaveformType` field and dispatch logic. This is an internal change, but it's required before waveform config can be wired up.
- **User-defined rules require dynamic `TrafficClass` values.** v1.0 `TrafficClass` is a string type with predefined constants. User rules name their own classes (e.g., `"MyApp"`). The classifier already uses `TrafficClass` as a string; the `synth` layer needs to handle classes not in `ClassFreqConfigs` by looking up user-supplied sound parameters.
- **Config loading must happen in `cmd` before the capture pipeline starts.** The cobra root command's `RunE` (or `PersistentPreRunE`) function loads and validates config, then passes effective config into the pipeline constructors. This is a structural change to `cmd/root.go`.
- **`--print-config` is independent** of capture and can be implemented as a separate cobra subcommand reading only the config loader output.
### New: Tier 2 (moderate frequency, recommended for v1.2 completeness)
| Class | Protocol | Port | Transport | Family |
|-------|----------|------|-----------|--------|
| Telnet | Telnet (unencrypted shell) | 23 | TCP | Remote Access |
| VNC | VNC / Remote Frame Buffer | 5900 | TCP | Remote Access |
| TFTP | Trivial File Transfer | 69 | UDP | File Transfer |
| SIP | SIP VoIP signaling | 5060, 5061 | TCP+UDP | VoIP |
| LDAP | Directory Access Protocol | 389, 636 | TCP | Directory/Auth |
| Kerberos | Kerberos authentication | 88 | TCP+UDP | Directory/Auth |
| Syslog | System log forwarding | 514 | UDP | Infrastructure |
| MongoDB | MongoDB document store | 27017 | TCP | Database |
| QUIC | QUIC / HTTP/3 transport | 443 | UDP | Web |
**Total: 11 existing known + 21 new = 32 known protocol classes + 4 unknown buckets = 36 total.**
---
## Implementation Complexity Summary
| Feature | Complexity | Reason |
|---------|------------|--------|
| Config file loader (TOML parse + merge + validation) | LOW-MEDIUM | go-toml v2 handles parsing; merge logic is a loop; validation is a post-decode pass |
| Custom frequency per class | LOW | Direct map lookup override; one line per class |
| Custom waveform per class | MEDIUM | Oscillator needs waveform dispatch (new `WaveformType`); synthesis loop changes |
| User-defined classification rules | MEDIUM | Dynamic class names; synth layer must handle unknown class names via config lookup |
| `--config` flag + auto-discovery | LOW | Cobra flag + os.Stat checks on 2-3 paths |
| `--print-config` subcommand | LOW | Marshal effective config to TOML; add comments |
| Named custom rules in exit summary | LOW | `classify.Rule` struct gains optional `Name string` field |
| Area | Complexity | Reason |
|------|------------|--------|
| New TrafficClass constants (~21) | LOW | Add string constants; no logic change |
| New Rule entries in DefaultRules (~25 rules for 21 classes, some need 2 ports) | LOW | Add Rule structs; existing matcher handles them |
| New FreqConfig entries (~21) | LOW | Add map entries with chosen Hz values and waveform |
| Frequency rebalancing design | MEDIUM | Must assign ~21 new Hz values that (a) stay within audible range, (b) are musically coherent within families, (c) do not collide with existing 10 classes |
| NumLayers update | LOW | One constant change; test mix amplitude |
| AllClasses() family-ordered output | LOW | Reorder the returned slice by family |
| PrintConfig family section headers | LOW | Add comment lines between family groups in PrintConfig |
| Test updates | LOW | Add new classes to classifier tests; confirm no regressions |
**No new external dependencies required.** go-toml v2 is the only addition to `go.mod`.
**No new external dependencies required for v1.2.**
---
## TOML Schema Sketch (Informational)
## Competitor Feature Analysis (Updated for v1.2)
This is not a binding decision — it informs the roadmap's implementation phase. The schema should feel natural to a user who has seen other Go tool configs (golangci-lint, goreleaser).
```toml
# Override built-in protocol sounds
[classes.HTTPS]
frequency = 220.0
waveform = "square" # sine | square | sawtooth | triangle
[classes.DNS]
frequency = 90.0
# Add custom classification rules (prepended before built-in rules, first-match-wins)
[[rules]]
name = "Internal API"
protocol = "tcp"
port = 8443
frequency = 300.0
waveform = "sawtooth"
[[rules]]
name = "Game Traffic"
protocol = "udp"
port = 27015
frequency = 450.0
waveform = "triangle"
```
Key schema design choices:
- `[classes.X]` uses the same class name strings already used in `--verbose` output and exit summary (`HTTPS`, `DNS`, etc.) — no new naming system to learn.
- `[[rules]]` is a TOML array of tables, consistent with how goreleaser and other tools express lists of items.
- `protocol` and `port` map directly to the existing `classify.Rule` fields, minimizing translation.
- Waveform is an enum string, not an integer — readable and self-documenting in the config file.
---
## v1.0 Feature Landscape (Retained from Original Research)
### Table Stakes (v1.0)
| Feature | Why Expected | Complexity | Notes |
|---------|--------------|------------|-------|
| Network interface selection (`-i eth0`) | tcpdump/tshark convention | LOW | Implemented: v1.0 |
| Output file path flag (`-o output.mp3`) | Any file-producing CLI | LOW | Implemented: v1.0 |
| Graceful Ctrl+C with file save | Users expect clean finalize | MEDIUM | Implemented: v1.0 |
| Per-protocol sound distinction | Core value prop | MEDIUM | Implemented: v1.0, 12 rules |
| Packet count / traffic summary on exit | Every capture tool does this | LOW | Implemented: v1.0 |
| Privilege error message | Silent pcap failure is confusing | LOW | Implemented: v1.0 |
| List available interfaces (`--list-interfaces`) | Users don't know interface names | LOW | Implemented: v1.0 |
| Minimum viable duration guard | Zero-packet = no corrupt MP3 | LOW | Implemented: v1.0 |
### Differentiators (v1.0)
| Feature | Value Proposition | Complexity | Status |
|---------|-------------------|------------|--------|
| Auto-clustering of unrecognized traffic | Honest audio fingerprint | HIGH | Implemented: hash-bucket, 4 classes |
| Ambient/drone style (layered sine harmonics) | Distinct from event-ping tools | HIGH | Implemented: v1.0 |
| Time-windowed amplitude evolution | Mix evolves dynamically | MEDIUM | Implemented: 500ms windows + EMA |
| BPF capture filter (`--filter`) | Power users scope what's sonified | MEDIUM | Implemented: v1.0 |
| Offline pcap file input (`--read`) | Sonify historical captures | MEDIUM | Implemented: v1.0 |
| Verbose protocol activity log (`--verbose`) | Developers see classifications | LOW | Implemented: v1.0 |
### Anti-Features (v1.0)
| Feature | Why Avoided |
|---------|-------------|
| Real-time audio playback | Platform audio API complexity; file output is correct |
| GUI or web dashboard | Negates single-binary CLI value |
| Custom sound mapping (v1.0) | Deferred to v1.1 — now the current milestone |
| Rhythmic/percussive output | Ambient/drone is the deliberate differentiator |
| Deep-packet inspection | Massive complexity; header classification sufficient |
| Streaming MP3 output | MP3 finalization requires full buffer |
| Anomaly detection / alerting | Different user job |
---
## Competitor Feature Analysis
| Feature | SoNSTAR (Python) | Network-Sonification (C# GUI) | Peep (C, Unix) | NetSynth v1.0 | NetSynth v1.1 |
| Feature | SoNSTAR (Python) | Network-Sonification (C# GUI) | Peep (C, Unix) | NetSynth v1.1 | NetSynth v1.2 |
|---------|-----------------|-------------------------------|----------------|----------------|----------------|
| Custom sound config | No | No | Config file (fixed format) | No | Yes (TOML) |
| Config file discovery | n/a | n/a | Hardcoded path | n/a | XDG + working dir |
| Partial override semantics | n/a | n/a | Full replacement | n/a | Partial override |
| Waveform selection | Recorded samples | sine/square/triangle | Fixed | sine only | sine/square/sawtooth/triangle |
| Custom classification rules | No | No | No | No | Yes (user-defined port/proto rules) |
| Named custom classes | n/a | n/a | n/a | n/a | Yes (appears in summary output) |
| Protocol count | ~8 (TCP flow types) | ~10 | ~6 | 10 known + 4 unknown | ~32 known + 4 unknown |
| Family grouping | No | No | No | No | Yes (7 families) |
| Tonal family identity | No | No | No | No | Yes (freq proximity + shared waveform) |
| Database protocols | No | No | No | No | Yes (MySQL, PostgreSQL, Redis, MongoDB) |
| Mail family (IMAP/POP3) | No | No | No | SMTP only | Yes (SMTP + IMAP + POP3) |
| Enterprise protocols (RDP, LDAP, Kerberos, SMB) | No | No | No | No | Yes |
| Infrastructure expansion (mDNS, SNMP, Syslog) | No | No | No | No | Yes |
---
## Sources
- [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir/latest/) — standard for `~/.config` discovery path
- [adrg/xdg — Go XDG implementation](https://github.com/adrg/xdg) — if explicit XDG library is needed (probably not for NetSynth's 2-path lookup)
- [pelletier/go-toml v2 — strict mode and DecodeError](https://pkg.go.dev/github.com/pelletier/go-toml/v2) — recommended TOML library; DisallowUnknownFields() and human-readable errors
- [BurntSushi/toml — Undecoded() for unknown key detection](https://github.com/BurntSushi/toml) — alternative; simpler API but less actively maintained
- [Building CLI Applications with Go: Cobra and Viper Guide (2026)](https://dasroot.net/posts/2026/03/building-cli-applications-go-cobra-viper/) — config loading patterns in Cobra CLI tools
- [A Guide to TOML in Golang — kelche.co](https://www.kelche.co/blog/go/toml/) — go-toml v2 vs BurntSushi comparison and practical examples
- [Configuration | mise-en-place](https://mise.jdx.dev/configuration.html) — example of working-dir + XDG config discovery
- [golangci-lint configuration](https://golangci-lint.run/docs/configuration/cli/) — real-world example of partial override config in a Go CLI tool
- [Online Tone Generator — waveform types](https://onlinetonegenerator.com/) — confirms sine/square/sawtooth/triangle as the standard 4 waveform set
- [IANA Service Name and Transport Protocol Port Number Registry](https://www.iana.org/assignments/service-names-port-numbers) — authoritative port assignments
- [List of TCP and UDP port numbers — Wikipedia](https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers) — comprehensive reference for well-known ports
- [Common Ports Cheat Sheet — StationX](https://www.stationx.net/common-ports-cheat-sheet/) — grouped protocol reference used for family taxonomy
- [nDPI Protocols List — ntop](https://www.ntop.org/guides/nDPI/protocols.html) — nDPI's 450+ protocol list and 17-category taxonomy; source for family grouping inspiration
- [nDPI 5.0: Enhanced Traffic Fingerprinting — ntop blog](https://www.ntop.org/ndpi-5-0-enhanced-traffic-fingerprinting-and-fpc-many-new-protocols/) — confirms category-based grouping as the production approach for managing large protocol sets
- [SoNSTAR: Sonification of Network Traffic — Paul Vickers](https://paulvickers.github.io/SoNSTAR/) — academic network sonification tool; uses TCP/IP flow features rather than protocol families
- [Sonification of network traffic flow for monitoring and situational awareness — PLoS One 2018](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0195948) — research literature on what protocol-level groupings are perceptually useful in sonification
- [pkg.go.dev/github.com/gopacket/gopacket/layers](https://pkg.go.dev/github.com/gopacket/gopacket/layers) — confirmed that gopacket natively decodes DNS, DHCP, NTP, ICMP, OSPF, BGP, but NOT SMTP, IMAP, FTP, SMB, MySQL, Redis, SIP, RTP, LDAP at the application layer; port-based classification is the correct approach for v1.2
- [Realtime High-Speed Network Traffic Monitoring Using ntopng — LISA 2014](https://luca.ntop.org/Lisa2014.pdf) — confirms category-based protocol grouping as standard in production monitoring tools
- Internal codebase review: `/home/dev/workspace/yoloyolo/classify/rules.go`, `types.go`, `synth/config.go` — confirmed existing 10 known classes, Rule struct pattern, FreqConfig pattern, NumLayers=14 constant, and auto-assign frequency range (1200-2350 Hz)
---
*v1.0 research: 2026-03-24*
*v1.1 custom sound mappings research: 2026-03-26*
*v1.2 extended protocol coverage research: 2026-03-27*