muse-protocol.md
markdown
| 1 | # Muse Protocol --- Language Agnostic Specification |
| 2 | |
| 3 | Status: Canonical Source of Truth Version: v1.0 Audience: Any frontend, |
| 4 | backend, or tool implementing Muse |
| 5 | |
| 6 | ------------------------------------------------------------------------ |
| 7 | |
| 8 | # 0. PURPOSE |
| 9 | |
| 10 | Muse defines a **platform-neutral protocol** for proposing, reviewing, |
| 11 | auditioning, approving, and committing musical changes. |
| 12 | |
| 13 | Muse is NOT tied to: - Swift - Python - Stori - Maestro - Any DAW or UI |
| 14 | framework |
| 15 | |
| 16 | Muse defines WHAT must happen --- not HOW it is implemented. |
| 17 | |
| 18 | ------------------------------------------------------------------------ |
| 19 | |
| 20 | # 1. DESIGN PRINCIPLES |
| 21 | |
| 22 | 1. Non‑destructive by default |
| 23 | 2. Humans approve musical mutation |
| 24 | 3. Beat‑based time (never seconds) |
| 25 | 4. Deterministic streaming order |
| 26 | 5. Language and platform agnostic |
| 27 | 6. Canonical state MUST NOT change during review |
| 28 | |
| 29 | ------------------------------------------------------------------------ |
| 30 | |
| 31 | # 2. TERMINOLOGY (NORMATIVE) |
| 32 | |
| 33 | Term Meaning |
| 34 | ----------------- ----------------------------------------- |
| 35 | Muse System proposing musical ideas |
| 36 | Variation Structured change proposal |
| 37 | Phrase Independently reviewable musical region |
| 38 | NoteChange Atomic note delta |
| 39 | Canonical State Actual project state |
| 40 | Proposed State Ephemeral variation preview |
| 41 | |
| 42 | ------------------------------------------------------------------------ |
| 43 | |
| 44 | # 3. EXECUTION MODES |
| 45 | |
| 46 | Execution mode is decided by backend intent classification. |
| 47 | |
| 48 | Intent Mode Behavior |
| 49 | ----------- ----------- ----------------- |
| 50 | COMPOSING variation Requires review |
| 51 | EDITING apply Immediate |
| 52 | REASONING none Chat only |
| 53 | |
| 54 | Frontends MUST NOT override execution mode. |
| 55 | |
| 56 | ------------------------------------------------------------------------ |
| 57 | |
| 58 | # 4. LIFECYCLE |
| 59 | |
| 60 | 1. Variation Proposed |
| 61 | 2. Meta Event Streamed |
| 62 | 3. Phrase Events Streamed |
| 63 | 4. Review Mode Active |
| 64 | 5. Accept OR Discard |
| 65 | |
| 66 | Canonical mutation occurs ONLY after commit. |
| 67 | |
| 68 | ------------------------------------------------------------------------ |
| 69 | |
| 70 | # 5. IDENTIFIERS |
| 71 | |
| 72 | All requests MUST include: |
| 73 | |
| 74 | - `projectId` |
| 75 | - `variationId` |
| 76 | - `baseStateId` |
| 77 | - `requestId` (optional but recommended) |
| 78 | |
| 79 | `baseStateId` enables optimistic concurrency. |
| 80 | |
| 81 | ------------------------------------------------------------------------ |
| 82 | |
| 83 | # 6. KEY NAMING CONTRACT |
| 84 | |
| 85 | There is exactly one canonical key name per concept. No aliases, no fallbacks. |
| 86 | |
| 87 | **Casing rule:** All JSON on the wire uses **camelCase**. Python internals use snake_case. Swift internals use camelCase. MCP tool names use snake_case (MCP convention). |
| 88 | |
| 89 | ## Project Context (frontend → backend) |
| 90 | |
| 91 | Sent as the `project` field in compose requests. Entities use `"id"` as their self-identifier. All keys are **camelCase**. |
| 92 | |
| 93 | ``` json |
| 94 | { |
| 95 | "id": "uuid", |
| 96 | "name": "My Project", |
| 97 | "tempo": 120, |
| 98 | "key": "C", |
| 99 | "timeSignature": "4/4", |
| 100 | "tracks": [ |
| 101 | { |
| 102 | "id": "uuid", |
| 103 | "name": "Drums", |
| 104 | "drumKitId": "acoustic", |
| 105 | "gmProgram": null, |
| 106 | "regions": [ |
| 107 | { |
| 108 | "id": "uuid", |
| 109 | "name": "Pattern 1", |
| 110 | "startBeat": 0, |
| 111 | "durationBeats": 16, |
| 112 | "noteCount": 24 |
| 113 | } |
| 114 | ] |
| 115 | } |
| 116 | ], |
| 117 | "buses": [ |
| 118 | { "id": "uuid", "name": "Reverb" } |
| 119 | ] |
| 120 | } |
| 121 | ``` |
| 122 | |
| 123 | Rules: |
| 124 | |
| 125 | - Project's own ID: `"id"` (not `"projectId"` — that's for cross-references) |
| 126 | - Entity self-IDs: always `"id"` (never `"trackId"`, `"regionId"`, or `"busId"`) |
| 127 | - Regions: always `"regions"` (never `"midiRegions"`) |
| 128 | - Key: always `"key"` (never `"keySignature"`) |
| 129 | - Time signature: always `"timeSignature"` (never `"time_signature"`) |
| 130 | - Track instrument: `"drumKitId"` and `"gmProgram"` (camelCase) |
| 131 | - Region timing: `"startBeat"` and `"durationBeats"` (camelCase) |
| 132 | - Notes may be omitted (send `"noteCount"` instead); backend preserves prior note data |
| 133 | |
| 134 | ## Tool Call Events (backend → frontend) |
| 135 | |
| 136 | Tool names use **snake_case** (MCP convention). Parameters use **camelCase**. |
| 137 | |
| 138 | ``` json |
| 139 | { |
| 140 | "type": "toolCall", |
| 141 | "name": "stori_add_notes", |
| 142 | "params": { |
| 143 | "regionId": "uuid", |
| 144 | "notes": [...] |
| 145 | } |
| 146 | } |
| 147 | ``` |
| 148 | |
| 149 | ## SSE Events (backend → frontend) |
| 150 | |
| 151 | All SSE event data uses **camelCase** for both type values and payload keys. |
| 152 | |
| 153 | Event type values: `state`, `status`, `content`, `reasoning`, `plan`, `planStepUpdate`, `toolStart`, `toolCall`, `toolError`, `meta`, `phrase`, `done`, `complete`, `budgetUpdate`, `error`. |
| 154 | |
| 155 | ``` json |
| 156 | { |
| 157 | "type": "meta", |
| 158 | "variationId": "uuid", |
| 159 | "baseStateId": "string", |
| 160 | "aiExplanation": "string|null", |
| 161 | "affectedTracks": ["uuid"], |
| 162 | "affectedRegions": ["uuid"], |
| 163 | "noteCounts": { "added": 0, "removed": 0, "modified": 0 } |
| 164 | } |
| 165 | ``` |
| 166 | |
| 167 | ------------------------------------------------------------------------ |
| 168 | |
| 169 | # 7. EVENT ENVELOPE (STREAMING CONTRACT) |
| 170 | |
| 171 | All streaming messages MUST use camelCase keys: |
| 172 | |
| 173 | ``` json |
| 174 | { |
| 175 | "type": "meta|phrase|done|error|heartbeat", |
| 176 | "sequence": 1, |
| 177 | "variationId": "uuid", |
| 178 | "projectId": "uuid", |
| 179 | "baseStateId": "uuid", |
| 180 | "timestampMs": 0, |
| 181 | "payload": {} |
| 182 | } |
| 183 | ``` |
| 184 | |
| 185 | Rules: |
| 186 | |
| 187 | - sequence strictly increasing |
| 188 | - meta MUST be first |
| 189 | - done MUST be last |
| 190 | |
| 191 | ------------------------------------------------------------------------ |
| 192 | |
| 193 | # 8. DATA MODELS |
| 194 | |
| 195 | All wire-format models use **camelCase** keys. |
| 196 | |
| 197 | ## Variation (meta event) |
| 198 | |
| 199 | ``` json |
| 200 | { |
| 201 | "variationId": "uuid", |
| 202 | "intent": "string", |
| 203 | "aiExplanation": "string|null", |
| 204 | "noteCounts": { |
| 205 | "added": 0, |
| 206 | "removed": 0, |
| 207 | "modified": 0 |
| 208 | } |
| 209 | } |
| 210 | ``` |
| 211 | |
| 212 | ## Phrase |
| 213 | |
| 214 | ``` json |
| 215 | { |
| 216 | "phraseId": "uuid", |
| 217 | "trackId": "uuid", |
| 218 | "regionId": "uuid", |
| 219 | "startBeat": 0.0, |
| 220 | "endBeat": 4.0, |
| 221 | "label": "Bars 1-4", |
| 222 | "noteChanges": [], |
| 223 | "controllerChanges": [] |
| 224 | } |
| 225 | ``` |
| 226 | |
| 227 | ## NoteChange |
| 228 | |
| 229 | ``` json |
| 230 | { |
| 231 | "noteId": "uuid", |
| 232 | "changeType": "added|removed|modified", |
| 233 | "before": { "pitch": 60, "startBeat": 0, "durationBeats": 1, "velocity": 100, "channel": 0 }, |
| 234 | "after": { "pitch": 62, "startBeat": 0, "durationBeats": 1, "velocity": 100, "channel": 0 } |
| 235 | } |
| 236 | ``` |
| 237 | |
| 238 | Rules: |
| 239 | |
| 240 | added → before MUST be null\ |
| 241 | removed → after MUST be null\ |
| 242 | modified → both present |
| 243 | |
| 244 | ------------------------------------------------------------------------ |
| 245 | |
| 246 | # 9. BACKEND RESPONSIBILITIES |
| 247 | |
| 248 | Backend MUST: |
| 249 | |
| 250 | - classify intent |
| 251 | - construct proposed state |
| 252 | - compute phrases |
| 253 | - stream envelopes |
| 254 | - enforce ordering |
| 255 | - validate base_state_id on commit |
| 256 | - apply accepted phrases atomically |
| 257 | |
| 258 | Backend MUST NEVER mutate canonical state during proposal. |
| 259 | |
| 260 | ------------------------------------------------------------------------ |
| 261 | |
| 262 | # 10. FRONTEND RESPONSIBILITIES |
| 263 | |
| 264 | Frontend MUST: |
| 265 | |
| 266 | - render proposed vs canonical state |
| 267 | - provide audition modes: |
| 268 | - Original |
| 269 | - Variation |
| 270 | - Delta |
| 271 | - allow partial phrase acceptance |
| 272 | - send commit/discard requests |
| 273 | |
| 274 | ------------------------------------------------------------------------ |
| 275 | |
| 276 | # 11. AUDITION MODES (CONCEPTUAL) |
| 277 | |
| 278 | Original → canonical only\ |
| 279 | Variation → proposed state\ |
| 280 | Delta → changed notes only |
| 281 | |
| 282 | Implementation is language specific. |
| 283 | |
| 284 | ------------------------------------------------------------------------ |
| 285 | |
| 286 | # 12. FAILURE RULES |
| 287 | |
| 288 | If stream fails: - keep received phrases - allow discard |
| 289 | |
| 290 | If commit rejected: - frontend MUST regenerate variation |
| 291 | |
| 292 | ------------------------------------------------------------------------ |
| 293 | |
| 294 | # 13. SAFETY MODEL |
| 295 | |
| 296 | - review mode must be isolated |
| 297 | - destructive edits should be blocked |
| 298 | - commit is single undo boundary |
| 299 | |
| 300 | ------------------------------------------------------------------------ |
| 301 | |
| 302 | # 14. PHILOSOPHY |
| 303 | |
| 304 | Muse is a protocol for **human‑guided AI creativity**. |
| 305 | |
| 306 | AI proposes. Humans curate. |