op-log.md
markdown
| 1 | # Op Log: Append-Only Operation Log |
| 2 | |
| 3 | ## Purpose |
| 4 | |
| 5 | The op log is the staging area between real-time collaborative edits and |
| 6 | the immutable Muse commit DAG. During a live session, every operation is |
| 7 | appended to the log as it occurs. At commit time, the log is collapsed into |
| 8 | a `StructuredDelta` and stored with the commit record. |
| 9 | |
| 10 | ## Why not just commit frequently? |
| 11 | |
| 12 | The commit DAG is optimised for immutability and verifiability, not for |
| 13 | sub-second edit throughput. A live AI agent producing 100 note edits per |
| 14 | second cannot commit 100 times per second — the overhead of hashing, writing |
| 15 | to the object store, and updating refs would dominate. |
| 16 | |
| 17 | The op log is optimised for append throughput: it is a flat JSON-lines file |
| 18 | with no locking beyond OS-level file appends. A checkpoint converts the |
| 19 | accumulated ops into a single commit. |
| 20 | |
| 21 | ## Structure |
| 22 | |
| 23 | ``` |
| 24 | .muse/op_log/<session_id>/ |
| 25 | ops.jsonl — one JSON line per OpEntry (append-only) |
| 26 | checkpoint.json — most recent checkpoint record |
| 27 | ``` |
| 28 | |
| 29 | ## OpEntry fields |
| 30 | |
| 31 | | Field | Description | |
| 32 | |-------|-------------| |
| 33 | | `op_id` | UUID4 — stable identifier for this operation | |
| 34 | | `actor_id` | Agent or human identity | |
| 35 | | `lamport_ts` | Logical Lamport timestamp for causal ordering | |
| 36 | | `parent_op_ids` | Causal dependencies (empty = root entry) | |
| 37 | | `domain` | Domain tag (`"music"`, `"code"`, …) | |
| 38 | | `domain_op` | The typed domain operation | |
| 39 | | `created_at` | ISO 8601 wall-clock timestamp (informational) | |
| 40 | | `intent_id` | Coordination intent linkage (empty if none) | |
| 41 | | `reservation_id` | Coordination reservation linkage (empty if none) | |
| 42 | |
| 43 | ## Lamport timestamps |
| 44 | |
| 45 | Lamport timestamps provide total ordering across concurrent actors without |
| 46 | wall-clock coordination. Each actor maintains a counter; every new entry |
| 47 | increments it. When two actors merge their logs, the resulting Lamport |
| 48 | clock continues from `max(a.lamport, b.lamport) + 1`. |
| 49 | |
| 50 | The `OpLog` class initialises its counter from the highest value found in |
| 51 | the log file on first access, so that a reopened session continues correctly. |
| 52 | |
| 53 | ## Checkpoints |
| 54 | |
| 55 | A checkpoint marks the point where all ops up to a given Lamport timestamp |
| 56 | have been crystallised into a Muse commit: |
| 57 | |
| 58 | ```python |
| 59 | ckpt = log.checkpoint(snapshot_id="snap-abc123") |
| 60 | ``` |
| 61 | |
| 62 | After a checkpoint, `replay_since_checkpoint()` returns only ops that arrived |
| 63 | after the checkpoint — enabling incremental application without re-reading the |
| 64 | full log. |
| 65 | |
| 66 | The log file itself is never truncated. Compaction (deleting old log files) |
| 67 | is a separate archival operation outside the scope of this module. |
| 68 | |
| 69 | ## Lifecycle |
| 70 | |
| 71 | ``` |
| 72 | live edits → OpLog.append() → ops.jsonl |
| 73 | session end → OpLog.to_structured_delta() → StructuredDelta |
| 74 | commit → OpLog.checkpoint(snap) → checkpoint.json |
| 75 | → normal Muse commit DAG |
| 76 | ``` |
| 77 | |
| 78 | ## Domain neutrality |
| 79 | |
| 80 | The op log stores `DomainOp` values unchanged. The core engine has no |
| 81 | opinion about what those ops mean. Each domain plugin collapses its own |
| 82 | slice using `OpLog.to_structured_delta(domain)`. |
| 83 | |
| 84 | ## Related files |
| 85 | |
| 86 | | File | Role | |
| 87 | |------|------| |
| 88 | | `muse/core/op_log.py` | `OpEntry`, `OpLogCheckpoint`, `OpLog`, `list_sessions` | |
| 89 | | `muse/domain.py` | `DomainOp`, `StructuredDelta` | |
| 90 | | `tests/test_op_log.py` | Unit tests | |