gurix/yoloyolo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: complete project research

Browse Source
This commit is contained in:
gurix
2026-03-27 08:25:02 +01:00
parent 6de2fcbb99
commit 6b26be825c
5 changed files with 1341 additions and 1107 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.planning/research/STACK.md
+143 -147
View File
@@ -1,8 +1,8 @@
# Technology Stack
**Project:** NetSynth v1.1 — Custom Sound Mappings
**Project:** NetSynth v1.2 — Extended Protocol Coverage with Grouped Sound Families
**Researched:** 2026-03-26
**Scope:** Additions/changes only. Existing stack (gopacket, go-pcap, go-lame, cobra) is validated and unchanged.
**Scope:** Additions/changes only. Existing stack (gopacket, go-pcap, go-lame, cobra, BurntSushi/toml) is validated and unchanged.
---
@@ -10,160 +10,125 @@
| Technology | Version | Status |
|------------|---------|--------|
| `github.com/gopacket/gopacket` | v1.5.0 | Validated in v1.0, unchanged |
| `github.com/packetcap/go-pcap` | v0.0.0-20251215 | Validated in v1.0, unchanged |
| `github.com/sjzar/go-lame` | v0.0.9 | Validated in v1.0, unchanged |
| `github.com/spf13/cobra` | v1.10.2 | Validated in v1.0, unchanged |
| Hand-rolled sine oscillator + EMA | — | Validated in v1.0, extend in place |
| Ordered `[]Rule` classifier | — | Validated in v1.0, extend in place |
| `github.com/gopacket/gopacket` | v1.5.0 | Validated in v1.0/v1.1, unchanged |
| `github.com/packetcap/go-pcap` | v0.0.0-20251215 | Validated in v1.0/v1.1, unchanged |
| `github.com/sjzar/go-lame` | v0.0.9 | Validated in v1.0/v1.1, unchanged |
| `github.com/spf13/cobra` | v1.10.2 | Validated in v1.0/v1.1, unchanged |
| `github.com/BurntSushi/toml` | v1.6.0 | Validated in v1.1, unchanged |
| Hand-rolled additive synth + EMA | — | Validated, extend in place |
| Ordered `[]Rule` classifier | — | Validated, extend in place |
---
## New Dependencies for v1.1
## New Dependencies for v1.2
### TOML Config Parsing
**None required.**
| Technology | Version | Purpose | Why |
|------------|---------|---------|-----|
| `github.com/BurntSushi/toml` | v1.6.0 | Parse `netsynth.toml` config files | Single-function `toml.Decode()` into a struct. The `MetaData.Undecoded()` method catches unknown keys in user configs — surfacing typos like `frequncy` rather than silently ignoring them. This is the right behavior for a config file tool. v1.6.0 released December 2025, Go 1.18+ required. Zero indirect dependencies. |
**Version confirmed:** v1.6.0, December 18, 2025, via pkg.go.dev and GitHub releases page.
**Why not `pelletier/go-toml v2`:** go-toml v2.3.0 (March 2026) is faster but the performance difference is irrelevant — config is read once at startup. go-toml v2's `Strict` mode can detect unknown keys but requires more setup than BurntSushi's `MetaData.Undecoded()`. BurntSushi's API is simpler for this use case and has clearer error message patterns for user-facing config mistakes.
### Config Auto-Discovery
No new dependency. Use Go stdlib only:
```go
// Probe order: --config flag > ./netsynth.toml > ~/.config/netsynth/config.toml
func findConfigPath(flagValue string) (string, bool) {
if flagValue != "" {
return flagValue, true
}
if _, err := os.Stat("./netsynth.toml"); err == nil {
return "./netsynth.toml", true
}
if dir, err := os.UserConfigDir(); err == nil {
p := filepath.Join(dir, "netsynth", "config.toml")
if _, err := os.Stat(p); err == nil {
return p, true
}
}
return "", false
}
```
`os.UserConfigDir()` returns `$XDG_CONFIG_HOME` if set, else `$HOME/.config` on Linux/macOS — confirmed against Go stdlib docs. No third-party XDG library needed.
### Additional Waveform Types
No new dependency. Extend the existing `synth.Oscillator` in place.
Square, sawtooth, and triangle are pure math — each is ~3 lines. The existing oscillator uses a phase accumulator (0.0–1.0 range), which is the right representation for all four waveforms:
```go
// Waveform enum addition to synth package
type Waveform int
const (
WaveformSine Waveform = iota
WaveformSquare
WaveformSawtooth
WaveformTriangle
)
// Per-sample generation (replaces math.Sin call in Advance())
func sample(phase float64, w Waveform) float64 {
switch w {
case WaveformSquare:
if phase < 0.5 { return 1.0 }
return -1.0
case WaveformSawtooth:
return 2*phase - 1.0
case WaveformTriangle:
if phase < 0.5 { return 4*phase - 1.0 }
return 3.0 - 4*phase
default: // WaveformSine
return math.Sin(2 * math.Pi * phase)
}
}
```
The `Oscillator` struct gains a `Waveform` field; `Advance()` dispatches to `sample()`. Harmonics still work the same way — each harmonic's phase is `phase * ratio`, which maps correctly for all waveform types.
All features for extended protocol coverage and grouped sound families can be implemented by extending existing packages in place. No new external dependencies are needed.
---
## Installation Delta
## gopacket Protocol Decoder Coverage
```bash
# Add only this new dependency
go get github.com/BurntSushi/toml@v1.6.0
```
This is the critical research question for v1.2. The `layers` package in `gopacket/gopacket v1.5.0` is the authoritative source.
No changes to build flags. `CGO_ENABLED=1` still required for go-lame.
### Protocols with Native gopacket Layer Decoders
These protocols have a dedicated `LayerType` constant and `DecodeFromBytes` implementation in `github.com/gopacket/gopacket/layers`. They auto-register via UDP/TCP port dispatch — `pkt.Layer(layers.LayerTypeSIP)` just works after gopacket decodes the packet.
| Protocol | LayerType Constant | Port Auto-Registered | Notes |
|----------|-------------------|---------------------|-------|
| ICMP v4 | `LayerTypeICMPv4` | IP protocol 1 | Already used in v1.0 |
| ICMP v6 | `LayerTypeICMPv6` | IP protocol 58 | Already used in v1.0 |
| DNS | `LayerTypeDNS` | UDP/TCP 53 | Already used in v1.0 |
| DHCP v4 | `LayerTypeDHCPv4` | UDP 67, 68 | Already used in v1.0 |
| DHCP v6 | `LayerTypeDHCPv6` | UDP 546, 547 | NEW: can add DHCPv6 classification rule |
| NTP | `LayerTypeNTP` | UDP 123 | Already used in v1.0 |
| TLS | `LayerTypeTLS` | TCP 443, 636, 989-995, 5061, etc. | Can use to improve HTTPS/SMTPS/LDAPS detection |
| SIP | `LayerTypeSIP` | UDP/TCP/SCTP 5060, 5082, 5083 | NEW: native layer decoder available |
| RADIUS | `LayerTypeRADIUS` | UDP 1812 | Possible addition for network infra traffic |
| SCTP | `LayerTypeSCTP` | IP protocol 132 | Available if needed |
| GRE | `LayerTypeGRE` | IP protocol 47 | Tunnel protocol, probably skip |
| Modbus TCP | `LayerTypeModbusTCP` | TCP/UDP 502 | Industrial — niche |
Source: `github.com/gopacket/gopacket/blob/master/layers/layertypes.go` and `layers/ports.go` — confirmed via direct inspection.
### Protocols WITHOUT gopacket Layer Decoders (Port-Based Classification Only)
These protocols do NOT have a `LayerType` in gopacket. Classification must use the existing `Rule{Protocol, DstPort, Class}` mechanism — matching by transport protocol + destination port number. This is already how most of the v1.0 rules work (SSH, HTTP, HTTPS, SMTP are all port-based).
| Protocol | Standard Port(s) | Transport | Classification Approach |
|----------|-----------------|-----------|------------------------|
| FTP | 21 (control), 20 (data) | TCP | Port-based rule: `{tcp, 21, ClassFTP}` |
| IMAP | 143, 993 (TLS) | TCP | Port-based rules: `{tcp, 143}`, `{tcp, 993}` |
| POP3 | 110, 995 (TLS) | TCP | Port-based rules: `{tcp, 110}`, `{tcp, 995}` |
| SNMP | 161 (queries), 162 (traps) | UDP | Port-based rules: `{udp, 161}`, `{udp, 162}` |
| LDAP | 389, 636 (TLS) | TCP | Port-based rules: `{tcp, 389}`, `{tcp, 636}` (note: 636 already hits LayerTypeTLS) |
| RDP | 3389 | TCP | Port-based rule: `{tcp, 3389}` |
| SMB | 445 (direct), 139 (NetBIOS) | TCP | Port-based rules: `{tcp, 445}`, `{tcp, 139}` |
| mDNS | 5353 | UDP | Port-based rule: `{udp, 5353}` — gopacket uses LayerTypeDNS registered on 53, not 5353 |
| QUIC / HTTP3 | 443 | UDP | Port-based rule: `{udp, 443}` distinguishes from HTTPS/TLS on TCP 443 |
| Telnet | 23 | TCP | Port-based rule: `{tcp, 23}` |
| HTTP alt | 8080, 8443 | TCP | Can add as additional Web family rules |
**mDNS detail:** gopacket's DNS layer registers only on UDP port 53. mDNS on UDP 5353 will decode as raw UDP payload — the existing `hashBucket` fallback handles it. A `{udp, 5353, ClassMDNS}` rule is correct and sufficient for classification without needing any layer decoder.
**QUIC detail:** QUIC uses UDP port 443 (same port HTTPS uses on TCP). The existing `{tcp, 443, ClassHTTPS}` rule only fires on TCP. A `{udp, 443, ClassQUIC}` rule is unambiguous — UDP 443 is QUIC/HTTP3 traffic on modern networks. No deep packet inspection needed for classification purposes.
**SIP detail:** gopacket v1.5.0 has a native SIP decoder (`LayerTypeSIP`) registered on UDP/TCP 5060. This means `pkt.Layer(layers.LayerTypeSIP)` works after gopacket decodes the packet. However, since the existing classifier already dispatches by transport + port via the `Rule` struct, a simple `{udp, 5060, ClassSIP}` / `{tcp, 5060, ClassSIP}` rule pair is simpler and more consistent than adding a special Layer-based code path. Use port-based rules. The native SIP layer decoder is available if future features need SIP message parsing (call rates, request types), but v1.2 only needs classification.
---
## Integration Points
## In-Place Extensions Required
### Where Config Feeds Existing Code
### 1. classify package — New TrafficClass constants and DefaultRules
The TOML config needs to override two existing data structures:
Add new `TrafficClass` constants to `classify/types.go` for each new protocol. Extend `classify/rules.go` `DefaultRules` with new ordered entries.
1. **`synth.ClassFreqConfigs`** (map in `synth/config.go`) — user can override `BaseHz` and add a `Waveform` field per class
2. **`classify.DefaultRules`** (slice in `classify/rules.go`) — user can prepend custom rules before the defaults
The config loader should apply overrides at startup before any other initialization. The cleanest integration is:
**Proposed new classes by family:**
```
cmd/netsynth/main.go
-> config.Load(path) // returns *AppConfig
-> classify.MergeRules(cfg) // prepend user rules to DefaultRules
-> synth.ApplyOverrides(cfg) // patch ClassFreqConfigs entries
Mail family: ClassIMAP, ClassPOP3, ClassSMTPS (SMTP over TLS = 465/587)
Web family: ClassHTTP (existing), ClassHTTPS (existing), ClassHTTP8080, ClassQUIC
Remote family: ClassSSH (existing), ClassRDP, ClassTelnet
Discovery: ClassMDNS, ClassDHCP (existing), ClassDHCPv6
File Transfer: ClassFTP
Directory: ClassLDAP
Monitoring: ClassSNMP
Messaging: ClassSIP
Infra: ClassSMB
```
Both `classify.DefaultRules` and `synth.ClassFreqConfigs` are currently package-level vars — they can be replaced or cloned at startup without changing the downstream pipeline.
The exact set is a product decision (FEATURES.md), but every entry requires only a new `TrafficClass` string constant and a `Rule{Protocol, DstPort, Class}` entry in `DefaultRules`. No code path changes needed.
### TOML Struct Shape
**Insertion point in DefaultRules:** New rules must come before the existing catch-alls (`{tcp, 0, ClassOtherTCP}` and `{udp, 0, ClassOtherUDP}`). Ordering within the new rules does not matter since they are distinct ports.
The config schema maps naturally to the existing types:
### 2. synth package — Frequency map and group detuning
```toml
# netsynth.toml
[[rules]]
protocol = "tcp"
dst_port = 8443
class = "my-https-alt"
Extend `synth/config.go` `ClassFreqConfigs` with an entry for each new `TrafficClass`. No API change — it's a map addition.
[sounds.my-https-alt]
frequency = 195.0
waveform = "square"
**Group-based frequency allocation approach (no new code needed):**
[sounds.ICMP]
frequency = 80.0 # override built-in
waveform = "triangle"
```
Group related protocols into a frequency band, using slight detuning within the band for distinction. The existing `FreqConfig.BaseHz` + `FreqConfig.Harmonics` already supports this — give family members adjacent base frequencies (e.g., 5-15 Hz apart at low frequencies, 15-30 Hz at mid frequencies) with the same harmonic shape but different waveform types.
Example for Mail family:
```go
type AppConfig struct {
Rules []RuleConfig `toml:"rules"`
Sounds map[string]SoundConfig `toml:"sounds"`
}
type RuleConfig struct {
Protocol string `toml:"protocol"`
DstPort uint16 `toml:"dst_port"`
Class string `toml:"class"`
}
type SoundConfig struct {
Frequency float64 `toml:"frequency"`
Waveform string `toml:"waveform"` // "sine"|"square"|"sawtooth"|"triangle"
}
ClassSMTP: {BaseHz: 440.0, Harmonics: ...sawtooth..., Pan: -0.55} // existing
ClassIMAP: {BaseHz: 450.0, Harmonics: ...sawtooth..., Pan: 0.55} // same family, detuned +10 Hz
ClassPOP3: {BaseHz: 435.0, Harmonics: ...sawtooth..., Pan: -0.3} // same family, detuned -5 Hz
```
Use `toml.Decode()` and check `meta.Undecoded()` to warn on unknown keys.
The `WaveformType` field already encodes "same character within group." The existing bandlimited synthesis code handles all this correctly.
**NumLayers constant:** Currently hardcoded to 14 in `synth/config.go`. Must be updated to reflect the new total class count. Alternatively, compute it dynamically from `len(ClassFreqConfigs)`. The dynamic approach is more maintainable and requires touching only `synth/config.go`.
**GainPerLayer:** Computed as `1.0 / float64(NumLayers)`. With more layers active simultaneously, individual gain drops. This is the correct behavior — prevents clipping. Verify mix levels after adding classes.
### 3. config package — --print-config output
`--print-config` currently emits commented TOML grouped by class. With protocol families, adding a `Group` field to `FreqConfig` or a separate group-to-classes mapping in `synth/config.go` allows `--print-config` to emit sections with comment headers like `# Mail family`. This is cosmetic; no behavioral change needed.
No new dependency needed. Add a `GroupName string` field to `FreqConfig` (zero value = ungrouped) or a `var ClassGroups = map[string][]TrafficClass{...}` in `synth/config.go`.
---
@@ -171,20 +136,48 @@ Use `toml.Decode()` and check `meta.Undecoded()` to warn on unknown keys.
| Avoid | Why | What to Do Instead |
|-------|-----|-------------------|
| `adrg/xdg` or any XDG library | `os.UserConfigDir()` in stdlib already handles `$XDG_CONFIG_HOME` on Linux — confirmed | Use `os.UserConfigDir()` directly |
| `pelletier/go-toml v2` | No advantage over BurntSushi for a single-file startup read; `MetaData.Undecoded()` in BurntSushi is more ergonomic for typo detection | `github.com/BurntSushi/toml` |
| `spf13/viper` | Massive dependency (brings in 20+ transitive deps) for a use case that is one TOML file — Viper adds remote config, env var binding, hot reload, none of which are needed | `BurntSushi/toml` + manual flag override |
| Any waveform/audio library | Square/sawtooth/triangle are 3 lines of math each; no library adds value | Extend `synth.Oscillator` in place |
| `gopkg.in/yaml.v3` or JSON config | TOML is explicitly specified for this milestone and is the right format for user-editable config files (comments supported, less noisy than JSON) | TOML only |
| Any deep packet inspection library (gopacket TLS layer for HTTPS detection) | v1.2 goal is protocol family classification by port, not payload analysis. TLS handshake parsing adds complexity for no classification benefit since port is unambiguous. | Port-based `Rule{tcp, 443, ClassHTTPS}` — already working |
| `github.com/google/gopacket` (original) | Superseded by community fork; 270 open issues, not maintained | `gopacket/gopacket v1.5.0` (already in use) |
| Any SNMP library (e.g., `gosnmp`) | v1.2 only needs to detect SNMP traffic, not decode OIDs or walk MIBs | `{udp, 161, ClassSNMP}` port rule |
| Any SIP parsing library | v1.2 only needs to detect SIP presence for sonification, not parse SIP messages, headers, or call state | `{udp, 5060, ClassSIP}` + `{tcp, 5060, ClassSIP}` port rules |
| Separate "group" abstraction layer in classify | A `Group` field on `FreqConfig` (in synth) is sufficient for --print-config display. The classifier itself doesn't need to know about groups — families emerge from frequency proximity in the audio output. | `GroupName string` in `synth.FreqConfig` |
| Dynamic port range rules (e.g., "all TCP 1024-65535 → ClassOtherTCP") | Existing catch-alls (`DstPort: 0`) already cover this. Current Rule struct is optimized for exact-match dispatch. | Keep existing catch-all rules |
---
## Version Compatibility
## Frequency Rebalancing Scope
Current v1.1 spectrum allocation (for reference):
```
65 Hz — ICMP
110 Hz — DNS
175 Hz — HTTPS
220 Hz — HTTP
330 Hz — SSH
440 Hz — SMTP
520 Hz — NTP
600 Hz — DHCP
700 Hz — OtherTCP
780 Hz — OtherUDP
862 Hz — Unknown-1 (dissonant band)
920 Hz — Unknown-2
981 Hz — Unknown-3
1047 Hz — Unknown-4
```
Adding ~8-12 new protocol classes requires rebalancing. The 65-780 Hz "known protocol" band currently has 8 classes spread over ~715 Hz (average spacing ~90 Hz). Adding 8+ new entries will compress that to ~40-50 Hz average spacing — still audibly distinct with different waveforms.
The unknown-1-4 dissonant band (862-1047 Hz) should stay — it provides the "something unknown" sound character. The rebalancing task is purely a `synth/config.go` constant edit, not a code change.
---
## Version Compatibility (Unchanged)
| Package | Version | Compatible With | Notes |
|---------|---------|-----------------|-------|
| `BurntSushi/toml` | v1.6.0 | Go 1.18+ | No issues with Go 1.24 |
| `os.UserConfigDir()` | stdlib | Go 1.13+ | Returns `$XDG_CONFIG_HOME` or `$HOME/.config` on Linux |
| `gopacket/gopacket` | v1.5.0 | Go 1.24+ | New protocol rules use existing API — no compat concerns |
| All other existing packages | (unchanged) | (unchanged) | No updates needed |
---
@@ -192,24 +185,27 @@ Use `toml.Decode()` and check `meta.Undecoded()` to warn on unknown keys.
| Area | Confidence | Source |
|------|------------|--------|
| BurntSushi/toml v1.6.0 version | HIGH | pkg.go.dev confirmed, GitHub releases confirmed |
| `os.UserConfigDir()` XDG behavior | HIGH | Official Go stdlib docs at pkg.go.dev/os |
| Waveform math (no library needed) | HIGH | Trivial math, Dylan Meeus Go audio blog confirms the same approach |
| go-toml v2.3.0 version | HIGH | pkg.go.dev confirmed |
| Recommendation of BurntSushi over go-toml v2 | MEDIUM | Based on API ergonomics for the specific `Undecoded()` use case; both would work |
| gopacket LayerType SIP exists at v1.5.0 | HIGH | Direct inspection of `layers/layertypes.go` and `layers/sip.go` via GitHub |
| gopacket LayerType TLS exists at v1.5.0 | HIGH | Direct inspection of `layers/layertypes.go` and `layers/ports.go` via GitHub |
| gopacket port registrations (ports.go) | HIGH | Direct inspection of `layers/ports.go` via GitHub; explicit list of pre-registered UDP/TCP ports |
| mDNS NOT registered in gopacket layers | HIGH | Port 5353 absent from `layers/ports.go` pre-registration list; confirmed via GitHub |
| QUIC NOT registered in gopacket layers | HIGH | No `quic.go` in layers directory; no port 443 UDP registration in `layers/ports.go` |
| SNMP, LDAP, RDP, SMB, FTP, IMAP, POP3 NOT in gopacket layers | HIGH | No corresponding .go files found in layers directory |
| Port-based Rule classification sufficiency for all new protocols | HIGH | All protocols have well-known IANA port assignments; existing Rule struct handles them identically to SSH/HTTP/SMTP |
| No new external dependencies needed | HIGH | All new functionality is data additions (constants, map entries) to existing packages |
---
## Sources
- `pkg.go.dev/github.com/BurntSushi/toml` — v1.6.0 confirmed, December 18, 2025
- `github.com/BurntSushi/toml/releases` — v1.6.0 release notes, TOML 1.1 enabled by default
- `pkg.go.dev/github.com/pelletier/go-toml/v2` — v2.3.0 confirmed, March 24, 2026
- `pkg.go.dev/os#UserConfigDir` — XDG_CONFIG_HOME behavior on Linux confirmed via official Go docs
- `dylanmeeus.github.io/posts/audio-from-scratch-pt8/` — Go waveform synthesis from scratch, confirms no library needed
- `github.com/golang/go/issues/76320` — UserConfigDir XDG_CONFIG_HOME discussion (Nov 2025), confirms existing stdlib support on Linux
- `github.com/gopacket/gopacket/blob/master/layers/layertypes.go` — LayerTypeSIP (id 133), LayerTypeTLS (id 140) confirmed
- `github.com/gopacket/gopacket/blob/master/layers/sip.go` — SIP decoder implementation confirmed
- `github.com/gopacket/gopacket/blob/master/layers/ports.go` — UDP/TCP port pre-registration list; mDNS (5353), SNMP (161/162), QUIC (UDP 443) absent; SIP (5060, 5082, 5083) present
- `github.com/gopacket/gopacket/tree/master/layers` — directory listing; no mdns.go, quic.go, snmp.go, ldap.go, smb.go, rdp.go, ftp.go, imap.go, or pop3.go files
- `pkg.go.dev/github.com/gopacket/gopacket/layers` — package index confirming layer types
- IANA port assignments — standard reference for FTP/21, IMAP/143, POP3/110, SNMP/161, LDAP/389, RDP/3389, SMB/445, mDNS/5353, SIP/5060, QUIC/UDP-443
---
*Stack research for: NetSynth v1.1 — Custom Sound Mappings milestone*
*Stack research for: NetSynth v1.2 — Extended Protocol Coverage with Grouped Sound Families*
*Researched: 2026-03-26*