405 lines
16 KiB
Markdown
405 lines
16 KiB
Markdown
---
|
|
phase: 03-pipeline-integration-and-mvp
|
|
plan: 01
|
|
type: execute
|
|
wave: 1
|
|
depends_on: []
|
|
files_modified:
|
|
- classify/types.go
|
|
- classify/classifier.go
|
|
- classify/classifier_test.go
|
|
- synth/config.go
|
|
- synth/bank_test.go
|
|
- synth/config_test.go
|
|
autonomous: true
|
|
requirements: [CLAS-02]
|
|
|
|
must_haves:
|
|
truths:
|
|
- "AllClasses() returns exactly 14 classes with no ClassUnknown"
|
|
- "Unrecognized TCP/UDP packets are deterministically assigned to ClassUnknown1-4 via hash-bucketing"
|
|
- "Packets with no transport layer (ARP etc.) are assigned to an unknown bucket, not silently dropped"
|
|
- "Each unknown bucket has a distinct tone in the 850-1100 Hz dissonant range"
|
|
- "GainPerLayer is 1/14 so 14 layers mix without clipping"
|
|
artifacts:
|
|
- path: "classify/types.go"
|
|
provides: "ClassUnknown1-4 constants, 14-element AllClasses()"
|
|
contains: "ClassUnknown1"
|
|
- path: "classify/classifier.go"
|
|
provides: "hashBucket() function routing unrecognized traffic"
|
|
contains: "func hashBucket"
|
|
- path: "synth/config.go"
|
|
provides: "FreqConfig entries for all 14 classes, NumLayers=14"
|
|
contains: "NumLayers = 14"
|
|
- path: "synth/config_test.go"
|
|
provides: "Test ensuring every AllClasses() entry has a FreqConfig"
|
|
contains: "TestClassFreqConfigsComplete"
|
|
key_links:
|
|
- from: "classify/classifier.go"
|
|
to: "classify/types.go"
|
|
via: "hashBucket returns ClassUnknown1-4 constants"
|
|
pattern: "ClassUnknown[1-4]"
|
|
- from: "synth/config.go"
|
|
to: "classify/types.go"
|
|
via: "ClassFreqConfigs map keys include all 14 classes"
|
|
pattern: "classify\\.ClassUnknown[1-4]"
|
|
---
|
|
|
|
<objective>
|
|
Extend the traffic classification type system from 11 classes to 14 by replacing ClassUnknown with 4 hash-bucketed unknown classes, and add corresponding synthesis tone configurations.
|
|
|
|
Purpose: Implements CLAS-02 (auto-clustering) so unrecognized traffic produces distinct drone tones instead of collapsing into a single undifferentiated layer.
|
|
Output: Updated classify and synth packages with 14-class type system, hash-bucketing classifier, dissonant tone configs, and comprehensive tests.
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
|
@$HOME/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
@.planning/phases/03-pipeline-integration-and-mvp/03-CONTEXT.md
|
|
|
|
<interfaces>
|
|
<!-- Key types and contracts the executor needs. -->
|
|
|
|
From classify/types.go (CURRENT — will be modified):
|
|
```go
|
|
type TrafficClass string
|
|
const (
|
|
ClassICMP TrafficClass = "ICMP"
|
|
ClassDNS TrafficClass = "DNS"
|
|
ClassHTTPS TrafficClass = "HTTPS"
|
|
ClassHTTP TrafficClass = "HTTP"
|
|
ClassSSH TrafficClass = "SSH"
|
|
ClassSMTP TrafficClass = "SMTP"
|
|
ClassNTP TrafficClass = "NTP"
|
|
ClassDHCP TrafficClass = "DHCP"
|
|
ClassOtherTCP TrafficClass = "other-TCP"
|
|
ClassOtherUDP TrafficClass = "other-UDP"
|
|
ClassUnknown TrafficClass = "unknown" // TO BE REMOVED
|
|
)
|
|
func AllClasses() []TrafficClass // currently returns 11, will return 14
|
|
```
|
|
|
|
From classify/classifier.go (CURRENT — ClassUnknown return sites at lines 23, 37, 52, 67, 71):
|
|
```go
|
|
func (c *Classifier) Classify(pkt gopacket.Packet) ClassifiedPacket
|
|
// Falls through to ClassUnknown when no rule matches
|
|
```
|
|
|
|
From synth/config.go (CURRENT — will be modified):
|
|
```go
|
|
const NumLayers = 11
|
|
const GainPerLayer = 1.0 / float64(NumLayers) // ~0.0909
|
|
var ClassFreqConfigs = map[classify.TrafficClass]FreqConfig{...} // 11 entries
|
|
```
|
|
|
|
From encode/mp3.go (NOT modified — consumes AllClasses() via synth.NewBank):
|
|
```go
|
|
func RunSynthesis(snapshots []classify.WindowSnapshot, outputPath string) error
|
|
```
|
|
</interfaces>
|
|
</context>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 1: Extend TrafficClass type system and write Wave 0 tests</name>
|
|
<files>classify/types.go, classify/classifier_test.go, synth/config_test.go, synth/bank_test.go</files>
|
|
<read_first>classify/types.go, classify/classifier_test.go, synth/bank_test.go, synth/config.go</read_first>
|
|
<behavior>
|
|
- TestAllClassesCount: `len(AllClasses()) == 14`
|
|
- TestAllClassesNoPlainUnknown: no element in AllClasses() equals `"unknown"`
|
|
- TestAllClassesHasUnknownBuckets: AllClasses() contains `"unknown-1"`, `"unknown-2"`, `"unknown-3"`, `"unknown-4"`
|
|
- TestHashBucketDistribution: calling hashBucket with ports 0-999 and protocol "tcp" produces all 4 ClassUnknown1-4 values
|
|
- TestClassifyUnknown (UPDATE): buildUnknownPacket returns a class with `strings.HasPrefix(string(got.Class), "unknown-")`, not `ClassUnknown`
|
|
- TestNewBankHas11Layers (UPDATE): assert `len(b.layers) == 14` (rename to TestNewBankHas14Layers)
|
|
- TestClassFreqConfigsComplete (NEW in synth/config_test.go): every class from `AllClasses()` has a key in `ClassFreqConfigs`
|
|
</behavior>
|
|
<action>
|
|
**Step 1: Update classify/types.go (per D-04)**
|
|
|
|
Remove the `ClassUnknown TrafficClass = "unknown"` constant. Add four new constants:
|
|
```go
|
|
ClassUnknown1 TrafficClass = "unknown-1"
|
|
ClassUnknown2 TrafficClass = "unknown-2"
|
|
ClassUnknown3 TrafficClass = "unknown-3"
|
|
ClassUnknown4 TrafficClass = "unknown-4"
|
|
```
|
|
|
|
Update `AllClasses()` to return 14 elements:
|
|
```go
|
|
func AllClasses() []TrafficClass {
|
|
return []TrafficClass{
|
|
ClassICMP, ClassDNS, ClassHTTPS, ClassHTTP, ClassSSH,
|
|
ClassSMTP, ClassNTP, ClassDHCP, ClassOtherTCP, ClassOtherUDP,
|
|
ClassUnknown1, ClassUnknown2, ClassUnknown3, ClassUnknown4,
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Add tests to classify/classifier_test.go**
|
|
|
|
Add `TestAllClassesCount` subtest:
|
|
```go
|
|
func TestAllClassesCount(t *testing.T) {
|
|
classes := classify.AllClasses()
|
|
if len(classes) != 14 {
|
|
t.Errorf("AllClasses() returned %d classes, want 14", len(classes))
|
|
}
|
|
// No plain "unknown" should exist
|
|
for _, c := range classes {
|
|
if c == "unknown" {
|
|
t.Error("AllClasses() still contains plain \"unknown\" — should be removed per D-04")
|
|
}
|
|
}
|
|
// All 4 buckets must be present
|
|
buckets := map[classify.TrafficClass]bool{
|
|
classify.ClassUnknown1: false, classify.ClassUnknown2: false,
|
|
classify.ClassUnknown3: false, classify.ClassUnknown4: false,
|
|
}
|
|
for _, c := range classes {
|
|
if _, ok := buckets[c]; ok {
|
|
buckets[c] = true
|
|
}
|
|
}
|
|
for bucket, found := range buckets {
|
|
if !found {
|
|
t.Errorf("AllClasses() missing bucket %q", bucket)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Add `TestHashBucketDistribution` (will fail until Task 2 implements hashBucket — that's the TDD red phase, both tasks are in the same plan so this is fine):
|
|
```go
|
|
func TestHashBucketDistribution(t *testing.T) {
|
|
// hashBucket is unexported, so test through Classify with unmatched TCP ports
|
|
c := classify.NewClassifier(classify.DefaultRules)
|
|
seen := make(map[classify.TrafficClass]bool)
|
|
// Try a range of unmatched TCP ports to hit all 4 buckets
|
|
for port := uint16(10000); port < 11000; port++ {
|
|
pkt := buildTCPPacket(t, port)
|
|
got := c.Classify(pkt)
|
|
if strings.HasPrefix(string(got.Class), "unknown-") {
|
|
seen[got.Class] = true
|
|
}
|
|
}
|
|
for _, bucket := range []classify.TrafficClass{
|
|
classify.ClassUnknown1, classify.ClassUnknown2,
|
|
classify.ClassUnknown3, classify.ClassUnknown4,
|
|
} {
|
|
if !seen[bucket] {
|
|
t.Errorf("hashBucket never produced %q across ports 10000-10999", bucket)
|
|
}
|
|
}
|
|
}
|
|
```
|
|
|
|
Add `"strings"` to the import block in classifier_test.go.
|
|
|
|
Update `TestClassifyUnknown` subtest (line 220-226): replace the assertion with:
|
|
```go
|
|
t.Run("TestClassifyUnknown", func(t *testing.T) {
|
|
pkt := buildUnknownPacket(t)
|
|
got := c.Classify(pkt)
|
|
if !strings.HasPrefix(string(got.Class), "unknown-") {
|
|
t.Errorf("Unknown packet: got class %q, want unknown-N bucket", got.Class)
|
|
}
|
|
})
|
|
```
|
|
|
|
**Step 3: Update synth/bank_test.go**
|
|
|
|
In `TestNewBankHas11Layers` (line 10-21):
|
|
- Rename function to `TestNewBankHas14Layers`
|
|
- Change assertion from `!= 11` to `!= 14`
|
|
- Change error message from `want 11` to `want 14`
|
|
|
|
In `TestMixerNoClip` (line 76-102):
|
|
- Update comment from `All 11 classes` to `All 14 classes`
|
|
- Update `TotalPackets` from `11000` to `14000`
|
|
|
|
**Step 4: Create synth/config_test.go**
|
|
|
|
New file:
|
|
```go
|
|
package synth
|
|
|
|
import (
|
|
"testing"
|
|
"github.com/netsynth/netsynth/classify"
|
|
)
|
|
|
|
func TestClassFreqConfigsComplete(t *testing.T) {
|
|
for _, class := range classify.AllClasses() {
|
|
if _, ok := ClassFreqConfigs[class]; !ok {
|
|
t.Errorf("ClassFreqConfigs missing entry for class %q", class)
|
|
}
|
|
}
|
|
}
|
|
|
|
func TestNumLayersMatchesAllClasses(t *testing.T) {
|
|
if NumLayers != len(classify.AllClasses()) {
|
|
t.Errorf("NumLayers=%d but AllClasses() has %d entries", NumLayers, len(classify.AllClasses()))
|
|
}
|
|
}
|
|
```
|
|
</action>
|
|
<verify>
|
|
<automated>cd /home/dev/workspace/yoloyolo && go vet ./classify/... ./synth/... 2>&1</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- classify/types.go contains `ClassUnknown1 TrafficClass = "unknown-1"` through `ClassUnknown4 TrafficClass = "unknown-4"`
|
|
- classify/types.go does NOT contain `ClassUnknown TrafficClass = "unknown"` (plain unknown removed)
|
|
- classify/types.go AllClasses() returns slice with 14 elements
|
|
- classify/classifier_test.go contains `func TestAllClassesCount`
|
|
- classify/classifier_test.go contains `func TestHashBucketDistribution`
|
|
- classify/classifier_test.go TestClassifyUnknown uses `strings.HasPrefix(string(got.Class), "unknown-")`
|
|
- synth/bank_test.go contains `func TestNewBankHas14Layers` (not 11)
|
|
- synth/bank_test.go TestNewBankHas14Layers asserts `!= 14`
|
|
- synth/config_test.go contains `func TestClassFreqConfigsComplete`
|
|
- synth/config_test.go contains `func TestNumLayersMatchesAllClasses`
|
|
- `go vet ./classify/... ./synth/...` exits 0 (compiles cleanly)
|
|
</acceptance_criteria>
|
|
<done>Type system extended to 14 classes, all Wave 0 tests written, code compiles. Some tests will fail (RED phase) until Task 2 implements hashBucket and updates config.</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 2: Implement hash-bucketing and update synth config for 14 layers</name>
|
|
<files>classify/classifier.go, synth/config.go</files>
|
|
<read_first>classify/classifier.go, synth/config.go, classify/types.go, classify/classifier_test.go, synth/config_test.go, synth/bank_test.go</read_first>
|
|
<action>
|
|
**Step 1: Add hashBucket function to classify/classifier.go (per D-01/D-03)**
|
|
|
|
Add this unexported function at the bottom of classifier.go:
|
|
```go
|
|
// hashBucket maps an unrecognized packet to one of 4 unknown traffic classes.
|
|
// Deterministic: same (dstPort, protocol) always maps to the same bucket.
|
|
// ARP and other non-transport packets have dstPort=0, protocol="" -> bucket 0 (ClassUnknown1).
|
|
func hashBucket(dstPort uint16, protocol string) TrafficClass {
|
|
var protoNum uint16
|
|
switch protocol {
|
|
case "tcp":
|
|
protoNum = 6
|
|
case "udp":
|
|
protoNum = 17
|
|
case "icmp":
|
|
protoNum = 1
|
|
}
|
|
h := uint32(dstPort)*31 + uint32(protoNum)*7
|
|
switch h % 4 {
|
|
case 0:
|
|
return ClassUnknown1
|
|
case 1:
|
|
return ClassUnknown2
|
|
case 2:
|
|
return ClassUnknown3
|
|
default:
|
|
return ClassUnknown4
|
|
}
|
|
}
|
|
```
|
|
|
|
**Step 2: Replace all ClassUnknown return sites in Classify() method**
|
|
|
|
There are 4 sites where ClassUnknown is used in classifier.go:
|
|
|
|
1. Line 23 — initial default: change `ClassUnknown` to `ClassUnknown1` (temporary; this gets overwritten by the final fallthrough, but the initial value doesn't matter since every code path sets Class before returning. However, to avoid a stale reference, use `""` or just leave the field unset since the default zero value is fine).
|
|
|
|
Actually, the simplest approach: change the initial default at line 23 to empty string `""`, then ensure every return path sets the class. The 4 return paths are:
|
|
|
|
a. **ICMP no-rule-match (line 37):** `return result` after ICMP rule loop fails. Replace with:
|
|
```go
|
|
result.Class = hashBucket(result.DstPort, result.Protocol)
|
|
return result
|
|
```
|
|
|
|
b. **TCP no-rule-match (line 52):** `return result` after TCP rule loop. Replace with:
|
|
```go
|
|
result.Class = hashBucket(result.DstPort, result.Protocol)
|
|
return result
|
|
```
|
|
|
|
c. **UDP no-rule-match (line 67):** `return result` after UDP rule loop. Replace with:
|
|
```go
|
|
result.Class = hashBucket(result.DstPort, result.Protocol)
|
|
return result
|
|
```
|
|
|
|
d. **No transport layer (line 71):** `return result` at the very end. Replace with:
|
|
```go
|
|
result.Class = hashBucket(result.DstPort, result.Protocol)
|
|
return result
|
|
```
|
|
|
|
Remove the initial `Class: ClassUnknown` at line 23. Instead set `Class: ""` or omit the field (zero value for string is ""). The `Classify` doc comment referencing "Per D-03: returns ClassUnknown" should be updated to "Per D-03: returns ClassUnknown1-4 via hash-bucketing".
|
|
|
|
**Step 3: Update synth/config.go (per D-02/D-05/D-06)**
|
|
|
|
Replace `NumLayers = 11` with `NumLayers = 14`.
|
|
|
|
This automatically changes `GainPerLayer = 1.0 / float64(NumLayers)` to ~0.0714.
|
|
|
|
Remove the `classify.ClassUnknown` entry from `ClassFreqConfigs`. Add 4 new entries:
|
|
|
|
```go
|
|
// D-05: Unknown buckets in 850-1100 Hz dissonant range, detuned intervals
|
|
// D-06: Same dissonant harmonic character {1,1.0},{2,0.8},{3,0.4} for all 4
|
|
classify.ClassUnknown1: {862.0, []HarmonicDef{{1, 1.0}, {2, 0.8}, {3, 0.4}}, 0.6},
|
|
classify.ClassUnknown2: {920.0, []HarmonicDef{{1, 1.0}, {2, 0.8}, {3, 0.4}}, -0.6},
|
|
classify.ClassUnknown3: {981.0, []HarmonicDef{{1, 1.0}, {2, 0.8}, {3, 0.4}}, 0.9},
|
|
classify.ClassUnknown4: {1047.0, []HarmonicDef{{1, 1.0}, {2, 0.8}, {3, 0.4}}, -0.9},
|
|
```
|
|
|
|
Remove the old `classify.ClassUnknown` entry and its comment about 437 Hz / D-04 beating.
|
|
</action>
|
|
<verify>
|
|
<automated>cd /home/dev/workspace/yoloyolo && go test ./classify/... ./synth/... -count=1 -v 2>&1 | tail -40</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- classify/classifier.go contains `func hashBucket(dstPort uint16, protocol string) TrafficClass`
|
|
- classify/classifier.go does NOT contain the string `ClassUnknown` without a digit suffix (no bare ClassUnknown references)
|
|
- classify/classifier.go hashBucket uses `uint32(dstPort)*31 + uint32(protoNum)*7` and `h % 4`
|
|
- synth/config.go contains `NumLayers = 14`
|
|
- synth/config.go contains `classify.ClassUnknown1:` through `classify.ClassUnknown4:`
|
|
- synth/config.go ClassUnknown1 BaseHz is 862.0, ClassUnknown2 is 920.0, ClassUnknown3 is 981.0, ClassUnknown4 is 1047.0
|
|
- synth/config.go does NOT contain `classify.ClassUnknown:` (old entry removed)
|
|
- `go test ./classify/... -count=1` passes (all classifier tests green including TestHashBucketDistribution, TestAllClassesCount, TestClassifyUnknown)
|
|
- `go test ./synth/... -count=1` passes (TestNewBankHas14Layers, TestClassFreqConfigsComplete, TestNumLayersMatchesAllClasses, TestMixerNoClip all green)
|
|
</acceptance_criteria>
|
|
<done>Hash-bucketing classifier routes unrecognized traffic to 4 distinct unknown classes. Synth config has tone entries for all 14 classes. All classify and synth tests pass.</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<verification>
|
|
```bash
|
|
# All classify and synth tests pass
|
|
go test ./classify/... ./synth/... -count=1
|
|
|
|
# Full suite still passes (no regressions in encode, aggregate, capture)
|
|
go test ./... -count=1
|
|
|
|
# No bare ClassUnknown references remain (should only find ClassUnknown1-4)
|
|
grep -rn 'ClassUnknown[^1234]' classify/ synth/ encode/ cmd/ aggregate/ | grep -v '_test.go.*unknown-' | grep -v '// '
|
|
```
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- AllClasses() returns 14 classes; ClassUnknown is fully removed
|
|
- hashBucket deterministically maps (dstPort, protocol) to ClassUnknown1-4
|
|
- All 4 unknown buckets have FreqConfig entries in 850-1100 Hz range
|
|
- GainPerLayer = 1/14; TestMixerNoClip passes with 14 layers
|
|
- All tests in classify/ and synth/ pass
|
|
- No bare ClassUnknown references remain in production code
|
|
</success_criteria>
|
|
|
|
<output>
|
|
After completion, create `.planning/phases/03-pipeline-integration-and-mvp/03-01-SUMMARY.md`
|
|
</output>
|