docs/architecture/muse_vcs.md · cgcardona/muse

1

# Muse VCS — Musical Version Control System

2

3

> **Status:** Canonical Implementation Reference

4

> **E2E demo:** [`muse_e2e_demo.md`](muse_e2e_demo.md)

---

## What Muse Is

Muse is a persistent, Git-style version control system for musical compositions. It tracks every committed change as a variation in a DAG (directed acyclic graph), enabling:

11

12

- **Commit history** — every accepted variation is recorded with parent lineage

13

- **Branching** — multiple variations can diverge from the same parent

14

- **Three-way merge** — auto-merges non-conflicting changes, reports conflicts

15

- **Drift detection** — compares HEAD snapshot against the live DAW state (`git status`)

16

- **Checkout / time travel** — reconstruct any historical state via deterministic tool calls

17

- **Log graph** — serialize the full commit DAG as Swift-ready JSON

---

## Module Map

### CLI Entry Point

```

maestro/muse_cli/

├── __init__.py — Package marker

28

├── app.py — Typer application root (console script: `muse`)

29

├── errors.py — Exit-code enum (0 success / 1 user / 2 repo / 3 internal) + exceptions

30

├── _repo.py — Repository detection (.muse/ directory walker)

31

└── commands/

32

├── __init__.py

33

├── init.py — muse init ✅ fully implemented

34

├── status.py — muse status ✅ branch + commit state display

35

├── commit.py — muse commit ✅ fully implemented (issue #32)

36

├── log.py — muse log ✅ fully implemented (issue #33)

37

├── checkout.py — muse checkout ✅ fully implemented (issue #34)

38

├── snapshot.py — walk_workdir, hash_file, build_snapshot_manifest, compute IDs

39

├── models.py — MuseCliCommit, MuseCliSnapshot, MuseCliObject (SQLAlchemy)

40

├── db.py — open_session, upsert_object/snapshot/commit helpers

41

├── merge.py — muse merge (stub — issue #35)

42

├── remote.py — muse remote (stub — issue #38)

43

├── push.py — muse push (stub — issue #38)

44

└── pull.py — muse pull (stub — issue #38)

45

```

46

47

The CLI delegates to existing `maestro/services/muse_*.py` service modules. Stub subcommands print "not yet implemented" and exit 0.

---

## `muse log` Output Formats

52

53

### Default (`git log` style)

54

55

```

56

commit a1b2c3d4e5f6... (HEAD -> main)

57

Parent: f9e8d7c6

58

Date: 2026-02-27 17:30:00

boom bap demo take 1

commit f9e8d7c6...

Date: 2026-02-27 17:00:00

initial take

```

Commits are printed newest-first. The first commit (root) has no `Parent:` line.

### `--graph` mode

Reuses `maestro.services.muse_log_render.render_ascii_graph` by adapting `MuseCliCommit` rows to the `MuseLogGraph`/`MuseLogNode` dataclasses the renderer expects.

73

74

```

75

* a1b2c3d4 boom bap demo take 1 (HEAD)

76

* f9e8d7c6 initial take

77

```

78

79

Merge commits (two parents) require `muse merge` (issue #35) — `parent2_commit_id` is reserved for that iteration.

### Flags

| Flag | Default | Description |

84

|------|---------|-------------|

85

| `--limit N` / `-n N` | 1000 | Cap the walk at N commits |

86

| `--graph` | off | ASCII DAG mode |

---

## Branching Model

### `muse checkout` — branch creation and HEAD pointer management

93

94

Branches are tracked as files under `.muse/refs/heads/<branch-name>`, each containing the `commit_id` of the branch tip (the same convention as Git's packed-refs but in plain files).

95

96

`.muse/HEAD` holds the symbolic ref of the currently active branch:

```

refs/heads/main

```

### Switching branches

103

104

`muse checkout <branch>` rewrites `.muse/HEAD` to `refs/heads/<branch>`. Subsequent `muse commit` and `muse log` calls read this file to know which branch to operate on.

105

106

### Creating branches

107

108

`muse checkout -b <branch>` forks from the current HEAD commit:

109

110

1. Reads the current branch tip from `.muse/refs/heads/<current>`.

111

2. Writes that same `commit_id` to `.muse/refs/heads/<new-branch>`.

112

3. Rewrites `.muse/HEAD` to `refs/heads/<new-branch>`.

113

114

The new branch starts with the same history as its parent — divergence happens on the next `muse commit`.

115

116

### Dirty working-tree guard

117

118

Before switching branches, `muse checkout` compares the on-disk `snapshot_id` of `muse-work/` against the last committed snapshot on the **current** branch. If they differ the command exits `1` with a message. Use `--force` / `-f` to bypass the guard.

119

120

If the current branch has no commits yet (empty branch) the tree is never considered dirty.

### Flags

| Flag | Description |

125

|------|-------------|

126

| `-b` / `--create` | Create a new branch at current HEAD and switch to it |

127

| `--force` / `-f` | Ignore uncommitted changes in `muse-work/` |

128

129

### DB-level branch table

130

131

A `muse_cli_branches` Postgres table is deferred to the `muse merge` iteration (issue #35), when multi-branch DAG queries will require stable foreign-key references. Until then, branches live exclusively in `.muse/refs/heads/`.

---

## Commit Data Model

`muse commit` persists three content-addressed table types to Postgres:

138

139

### `muse_cli_objects` — File blobs (sha256-keyed)

140

141

| Column | Type | Description |

142

|--------|------|-------------|

143

| `object_id` | `String(64)` PK | `sha256(file_bytes)` hex digest |

144

| `size_bytes` | `Integer` | Raw file size |

145

| `created_at` | `DateTime(tz=True)` | Wall-clock insert time |

146

147

Objects are deduplicated across commits: the same file committed on two branches is stored exactly once.

148

149

### `muse_cli_snapshots` — Snapshot manifests

150

151

| Column | Type | Description |

152

|--------|------|-------------|

153

| `snapshot_id` | `String(64)` PK | `sha256(sorted("path:object_id" pairs))` |

154

| `manifest` | `JSON` | `{rel_path: object_id}` mapping |

155

| `created_at` | `DateTime(tz=True)` | Wall-clock insert time |

156

157

Two identical working trees always produce the same `snapshot_id`.

158

159

### `muse_cli_commits` — Commit history

160

161

| Column | Type | Description |

162

|--------|------|-------------|

163

| `commit_id` | `String(64)` PK | Deterministic sha256 (see below) |

164

| `repo_id` | `String(36)` | UUID from `.muse/repo.json` |

165

| `branch` | `String(255)` | Branch name at commit time |

166

| `parent_commit_id` | `String(64)` nullable | Previous HEAD commit on branch |

167

| `snapshot_id` | `String(64)` FK | Points to the snapshot row |

168

| `message` | `Text` | User-supplied commit message |

169

| `author` | `String(255)` | Reserved (empty for MVP) |

170

| `committed_at` | `DateTime(tz=True)` | Timestamp used in hash derivation |

171

| `created_at` | `DateTime(tz=True)` | Wall-clock DB insert time |

172

173

### ID Derivation (deterministic)

174

175

```

176

object_id = sha256(file_bytes)

177

snapshot_id = sha256("|".join(sorted(f"{path}:{oid}" for path, oid in manifest.items())))

178

commit_id = sha256(

179

"|".join(sorted(parent_ids))

180

+ "|" + snapshot_id

181

+ "|" + message

182

+ "|" + committed_at.isoformat()

)

```

Given the same working tree state, message, and timestamp two machines produce identical IDs. `sorted()` ensures insertion-order independence for both snapshot manifests and parent lists.

---

## Local Repository Structure (`.muse/`)

191

192

`muse init` creates the following layout in the current working directory:

```

.muse/

repo.json Repo identity: repo_id (UUID), schema_version, created_at

197

HEAD Current branch pointer, e.g. "refs/heads/main"

198

config.toml [user], [auth], [remotes] configuration

199

refs/

200

heads/

201

main Commit ID of branch HEAD (empty = no commits yet)

202

<branch> One file per branch

```

### File semantics

| File | Source of truth for | Notes |

208

|------|-------------------|-------|

209

| `repo.json` | Repo identity | `repo_id` persists across `--force` reinitialise |

210

| `HEAD` | Current branch name | Always `refs/heads/<branch>` |

211

| `refs/heads/<branch>` | Branch → commit pointer | Empty string = branch has no commits yet |

212

| `config.toml` | User identity, auth token, remotes | Not overwritten on `--force` |

213

214

### Repo-root detection

215

216

Every CLI command locates the active repo by walking up the directory tree until `.muse/` is found:

217

218

```python

219

# maestro/muse_cli/_repo.py

220

find_repo_root(start: Path | None = None) -> Path | None

221

```

222

223

- Returns the directory containing `.muse/`, or `None` if not found (never raises).

224

- Set `MUSE_REPO_ROOT=/path/to/repo` to override traversal (useful in tests and scripts).

225

- `require_repo()` wraps `find_repo_root()` for command callbacks: exits 2 with "Not a Muse repository. Run `muse init`." if root is `None`.

226

227

### `config.toml` example

```toml

[user]

name = "Gabriel"

email = "g@example.com"

233

234

[auth]

235

token = "eyJ..." # Muse Hub Bearer token — keep out of version control

[remotes]

[remotes.origin]

url = "https://story.audio/musehub/repos/abcd1234"

240

```

241

242

> **Security note:** `.muse/config.toml` contains the Hub auth token. Add `.muse/config.toml` to `.gitignore` (or `.museignore`) to prevent accidental exposure.

### VCS Services

```

app/services/

├── muse_repository.py — Persistence adapter (DB reads/writes)

249

├── muse_replay.py — History reconstruction (lineage walking)

250

├── muse_drift.py — Drift detection engine (HEAD vs working)

251

├── muse_checkout.py — Checkout plan builder (pure data → tool calls)

252

├── muse_checkout_executor.py — Checkout execution (applies plan to StateStore)

253

├── muse_merge_base.py — Merge base finder (LCA in the DAG)

254

├── muse_merge.py — Three-way merge engine

255

├── muse_history_controller.py— Orchestrates checkout + merge flows

256

├── muse_log_graph.py — DAG serializer (topological sort → JSON)

257

├── muse_log_render.py — ASCII graph + JSON + summary renderer

258

└── variation/

259

└── note_matching.py — Note + controller event matching/diffing

260

261

app/api/routes/

262

├── muse.py — Production HTTP routes (5 endpoints)

263

└── variation/ — Existing variation proposal routes

264

265

app/db/

266

└── muse_models.py — ORM: Variation, Phrase, NoteChange tables

267

268

tests/

269

├── test_muse_persistence.py — Repository + lineage tests

270

├── test_muse_drift.py — Drift detection tests

271

├── test_muse_drift_controllers.py — Controller drift tests

272

├── test_commit_drift_safety.py — 409 conflict enforcement

273

├── test_muse_checkout.py — Checkout plan tests

274

├── test_muse_checkout_execution.py — Checkout execution tests

275

├── test_muse_merge.py — Merge engine tests

276

├── test_muse_log_graph.py — Log graph serialization tests

277

└── e2e/

278

├── muse_fixtures.py — Deterministic IDs + snapshot builders

279

└── test_muse_e2e_harness.py — Full VCS lifecycle E2E test

```

---

## Data Model

### Variation (ORM: `app/db/muse_models.py`)

287

288

| Column | Type | Purpose |

289

|--------|------|---------|

290

| `variation_id` | PK | Unique ID |

291

| `project_id` | FK | Project this belongs to |

292

| `parent_variation_id` | FK (self) | Primary parent (lineage) |

293

| `parent2_variation_id` | FK (self) | Second parent (merge commits only) |

294

| `is_head` | bool | Whether this is the current HEAD |

295

| `commit_state_id` | str | State version at commit time |

296

| `intent` | text | User intent / description |

297

| `status` | str | `ready` / `committed` / `discarded` |

298

299

### HeadSnapshot (`app/services/muse_replay.py`)

300

301

Reconstructed from walking the variation lineage. Contains the cumulative state at any point in history:

302

303

| Field | Type | Contents |

304

|-------|------|----------|

305

| `notes` | `dict[region_id, list[note_dict]]` | All notes per region |

306

| `cc` | `dict[region_id, list[cc_event]]` | CC events per region |

307

| `pitch_bends` | `dict[region_id, list[pb_event]]` | Pitch bends per region |

308

| `aftertouch` | `dict[region_id, list[at_event]]` | Aftertouch per region |

309

| `track_regions` | `dict[region_id, track_id]` | Region-to-track mapping |

---

## HTTP API

All routes require JWT auth (`Authorization: Bearer <token>`).

316

Prefix: `/api/v1/muse/`

317

318

| Method | Path | Purpose |

319

|--------|------|---------|

320

| `POST` | `/muse/variations` | Save a variation directly into history |

321

| `POST` | `/muse/head` | Set HEAD pointer to a variation |

322

| `GET` | `/muse/log?project_id=X` | Get the full commit DAG as `MuseLogGraph` JSON |

323

| `POST` | `/muse/checkout` | Checkout to a variation (time travel) |

324

| `POST` | `/muse/merge` | Three-way merge of two variations |

### Response codes

| Code | Meaning |

|------|---------|

| 200 | Success |

| 404 | Variation not found (checkout) |

332

| 409 | Checkout blocked by drift / merge has conflicts |

---

## VCS Primitives

### Commit (save + set HEAD)

339

340

```

341

save_variation(session, variation, project_id, parent_variation_id, ...)

342

set_head(session, variation_id)

```

### Lineage

```

get_lineage(session, variation_id) → [root, ..., target]

349

get_head(session, project_id) → HistoryNode | None

350

get_children(session, variation_id) → [HistoryNode, ...]

```

### Drift Detection

```

compute_drift_report(head_snapshot, working_snapshot, ...) → DriftReport

357

```

358

359

Compares HEAD (from DB) against working state (from StateStore). Severity levels: `CLEAN`, `DIRTY`, `DIVERGED`.

360

361

### Replay / Reconstruction

362

363

```

364

reconstruct_head_snapshot(session, project_id) → HeadSnapshot

365

reconstruct_variation_snapshot(session, variation_id) → HeadSnapshot

366

build_replay_plan(session, project_id, target_id) → ReplayPlan

```

### Checkout

```

build_checkout_plan(target_notes, working_notes, ...) → CheckoutPlan

373

execute_checkout_plan(plan, store, trace) → CheckoutExecutionResult

374

checkout_to_variation(session, project_id, target_id, store, ...) → CheckoutSummary

```

### Merge

```

find_merge_base(session, a, b) → str | None

381

build_merge_result(base, left, right) → MergeResult

382

merge_variations(session, project_id, left, right, store, ...) → MergeSummary

```

### Log Graph

```

build_muse_log_graph(session, project_id) → MuseLogGraph

389

```

390

391

Topologically sorted (Kahn's algorithm), deterministic tie-breaking by `(timestamp, variation_id)`. Output is camelCase JSON for the Swift frontend.

---

## Architectural Boundaries

396

397

17 AST-enforced rules in `scripts/check_boundaries.py`. Key constraints:

398

399

| Module | Must NOT import |

400

|--------|----------------|

401

| `muse_repository` | StateStore, executor, VariationService |

402

| `muse_replay` | StateStore, executor, LLM handlers |

403

| `muse_drift` | StateStore, executor, LLM handlers |

404

| `muse_checkout` | StateStore, executor, handlers |

405

| `muse_checkout_executor` | LLM handlers, VariationService |

406

| `muse_merge`, `muse_merge_base` | StateStore, executor, MCP, handlers |

407

| `muse_log_graph` | StateStore, executor, handlers, engines |

408

| `note_matching` | handlers, StateStore |

409

410

The boundary philosophy: Muse VCS modules are **pure data** — they consume snapshots and produce plans/reports. StateStore mutation only happens in `muse_checkout_executor` (via duck-typed store parameter) and the history controller.

---

## E2E Demo

Run the full VCS lifecycle test:

417

418

```bash