docs/reference/remotes.md · cgcardona/muse

1

# Remotes — Muse Remote Sync Reference

2

3

Muse supports synchronizing repositories with a remote host (e.g. MuseHub)

4

using a small set of commands modelled on Git's porcelain/plumbing separation.

5

The CLI is a **pure client** — no server runs inside `muse`. The server lives on

MuseHub.

---

## Table of Contents

1. [Overview](#overview)

13

2. [Transport Architecture](#transport-architecture)

14

3. [MuseHub API Contract](#musehub-api-contract)

15

4. [PackBundle Wire Format](#packbundle-wire-format)

16

5. [Authentication](#authentication)

17

6. [Commands](#commands)

18

- [muse remote](#muse-remote)

19

- [muse clone](#muse-clone)

20

- [muse fetch](#muse-fetch)

21

- [muse pull](#muse-pull)

22

- [muse push](#muse-push)

23

- [muse plumbing ls-remote](#muse-plumbing-ls-remote)

24

7. [Tracking Branches](#tracking-branches)

25

8. [Token Lifecycle](#token-lifecycle)

---

## Overview

The remote sync workflow mirrors Git's model:

32

33

```

34

muse clone <url> # one-time: download a full copy of a remote repo

35

muse fetch [remote] # download new commits without merging

36

muse pull [remote] # fetch + three-way merge into current branch

37

muse push [remote] # upload local commits to the remote

38

muse remote add <n> <url> # manage named remotes

39

muse plumbing ls-remote [remote] # list remote branches (Tier 1 plumbing)

40

```

41

42

Remotes are named connections to a MuseHub repository URL. The default remote

43

name is `origin` (set automatically by `muse clone`).

---

## Transport Architecture

48

49

```

50

Muse CLI process MuseHub server

51

───────────────── ──────────────

52

HttpTransport REST API

53

└─ urllib.request (stdlib) ────HTTPS──► GET /refs

POST /fetch

POST /push

```

The `MuseTransport` Protocol in `muse/core/transport.py` is the seam between

59

CLI command code and the HTTP implementation. It is entirely synchronous — the

60

Muse CLI has no async code. The `HttpTransport` class uses Python's stdlib

61

`urllib.request` (HTTP/1.1 + TLS), requiring zero new dependencies.

62

63

**Why HTTP/1.1?**

64

Each `muse` CLI invocation is a short-lived OS process making 2–3 requests.

65

HTTP/2 multiplexing benefits arise within a single long-lived connection.

66

Agent scale comes from MuseHub handling millions of concurrent HTTP/1.1

67

connections via its load balancer — not from any individual CLI process.

68

When MuseHub is ready to upgrade, the `MuseTransport` Protocol seam means

69

only `HttpTransport` changes; all CLI command code stays the same.

---

## MuseHub API Contract

74

75

MuseHub must implement these three endpoints under each repository URL:

76

77

### `GET {repo_url}/refs`

78

79

Returns the current state of the repository.

**Response (JSON):**

```json

{

"repo_id": "<uuid>",

"domain": "midi",

"default_branch": "main",

88

"branch_heads": {

89

"main": "<commit_id>",

"dev": "<commit_id>"

}

}

```

### `POST {repo_url}/fetch`

96

97

Downloads commits, snapshots, and objects the client does not have.

98

99

**Request body (JSON):**

```json

{

"want": ["<commit_id>", ...],

104

"have": ["<commit_id>", ...]

}

```

`want` is the list of remote commit IDs the client wants to receive.

109

`have` is the list of commit IDs the client already has locally, allowing the

110

server to compute the minimal delta to send (analogous to Git's fetch

111

negotiation).

112

113

**Response:** JSON `PackBundle` (see below).

114

115

### `POST {repo_url}/push`

116

117

Receives commits, snapshots, and objects from the client.

118

119

**Request body (JSON):**

```json

{

"bundle": { ... PackBundle ... },

"branch": "main",

"force": false

}

```

When `force` is `false`, MuseHub must verify the push is a fast-forward (the

130

current remote HEAD is an ancestor of the pushed commit). Return HTTP 409 to

131

reject a non-fast-forward push.

**Response (JSON):**

```json

{

"ok": true,

"message": "ok",

"branch_heads": { "main": "<new_commit_id>" }

}

```

**HTTP error codes:**

| Code | Meaning |

|------|---------|

| 401 | Invalid or missing bearer token |

148

| 404 | Repository does not exist |

149

| 409 | Push rejected — non-fast-forward without `force: true` |

150

| 5xx | Server error |

---

## PackBundle Wire Format

155

156

A `PackBundle` is a self-contained JSON object carrying everything needed to

157

reconstruct a slice of commit history:

```json

{

"commits": [

{

"commit_id": "<sha256>",

164

"repo_id": "<uuid>",

165

"branch": "main",

166

"snapshot_id": "<sha256>",

167

"message": "Add verse",

168

"committed_at": "2026-03-18T12:00:00+00:00",

169

"parent_commit_id": "<sha256> | null",

"author": "gabriel",

...

}

],

"snapshots": [

{

"snapshot_id": "<sha256>",

177

"manifest": { "tracks/drums.mid": "<sha256>", ... },

178

"created_at": "2026-03-18T12:00:00+00:00"

}

],

"objects": [

{

"object_id": "<sha256>",

184

"content_b64": "<base64-encoded raw bytes>"

}

],

"branch_heads": {

"main": "<commit_id>"

}

}

```

Objects are the raw blob bytes stored in `.muse/objects/`, base64-encoded for

194

JSON transport. `apply_pack()` in `muse/core/pack.py` writes objects, then

195

snapshots, then commits — in dependency order.

---

## Authentication

All MuseHub API calls include an `Authorization: Bearer <token>` header when a

202

token is configured.

203

204

Store your token in `.muse/config.toml`:

```toml

[auth]

token = "your-hub-token-here"

209

```

210

211

The token is read by `muse.cli.config.get_auth_token()` and is **never**

212

written to any log line. Add `.muse/config.toml` to `.gitignore` to prevent

accidental commit.

---

## Commands

### muse remote

Manage named remote connections. Remote state is stored entirely in

222

`.muse/config.toml` and `.muse/remotes/<name>/<branch>`. No network calls.

#### Subcommands

```

muse remote add <name> <url>

228

```

229

Register a new named remote.

230

`<name>` — identifier (e.g. `origin`, `upstream`).

231

`<url>` — full MuseHub repository URL.

232

233

```

234

muse remote remove <name>

235

```

236

Remove a remote and all its tracking refs under `.muse/remotes/<name>/`.

237

238

```

239

muse remote rename <old> <new>

240

```

241

Rename a remote in config and move its tracking refs directory.

242

243

```

244

muse remote list [-v]

245

```

246

List configured remotes. With `-v` / `--verbose`, shows URL and upstream

247

tracking branch for each remote.

248

249

```

250

muse remote get-url <name>

251

```

252

Print the URL of a named remote.

253

254

```

255

muse remote set-url <name> <url>

256

```

257

Update the URL of an existing remote.

---

### muse clone

Clone a remote Muse repository into a new local directory.

264

265

```

266

muse clone <url> [<directory>]

```

**Options:**

| Flag | Description |

272

|------|-------------|

273

| `--branch ` | Check out branch `` instead of the remote default branch |

**What clone does:**

1. Calls `GET <url>/refs` to discover the remote's repo_id, domain, and branch heads.

278

2. Creates the target directory and initialises `.muse/` with the remote's `repo_id` and `domain`.

279

3. Calls `POST <url>/fetch` with `want=all, have=[]` to download the complete history.

280

4. Applies the `PackBundle` (objects → snapshots → commits).

281

5. Sets `origin` remote and upstream tracking.

282

6. Restores `muse-work/` from the default branch HEAD snapshot.

**Examples:**

```bash

muse clone https://hub.muse.io/repos/my-song

288

muse clone https://hub.muse.io/repos/my-song local-copy

289

muse clone https://hub.muse.io/repos/my-song --branch dev

```

---

### muse fetch

Download commits, snapshots, and objects from a remote without merging.

297

298

```

299

muse fetch [<remote>] [--branch ]

```

**Options:**

| Flag | Description |

305

|------|-------------|

306

| `<remote>` | Remote name (default: `origin`) |

307

| `--branch ` / `-b ` | Remote branch to fetch (default: tracked branch or current branch) |

308

309

After fetch, the remote tracking pointer `.muse/remotes/<remote>/<branch>` is

310

updated. The local branch HEAD is **not** changed. Use `muse merge` or

311

`muse pull` to integrate the fetched commits.

**Examples:**

```bash

muse fetch

muse fetch origin

muse fetch origin --branch dev

```

---

### muse pull

Fetch from a remote and merge into the current branch.

326

327

```

328

muse pull [<remote>] [options]

```

**Options:**

| Flag | Description |

334

|------|-------------|

335

| `<remote>` | Remote name (default: `origin`) |

336

| `--branch ` / `-b ` | Remote branch to pull (default: tracked branch or current branch) |

337

| `--no-merge` | Stop after fetch; do not merge |

338

| `-m <msg>` / `--message <msg>` | Override the merge commit message |

**Merge behaviour:**

- Fast-forward: if the remote HEAD is a direct descendant of local HEAD, the

343

local branch ref and `muse-work/` are advanced without a merge commit.

344

- Three-way merge: delegates to the active domain plugin's `merge()` /

345

`merge_ops()` — identical to `muse merge`.

346

- Conflicts: MERGE_STATE.json is written; the user fixes conflicts then runs

`muse commit`.

**Examples:**

```bash

muse pull

muse pull origin --branch dev

354

muse pull origin --no-merge # equivalent to muse fetch

```

---

### muse push

Upload local commits, snapshots, and objects to a remote.

362

363

```

364

muse push [<remote>] [options]

```

**Options:**

| Flag | Description |

370

|------|-------------|

371

| `<remote>` | Remote name (default: `origin`) |

372

| `--branch ` / `-b ` | Branch to push (default: tracked branch or current branch) |

373

| `-u` / `--set-upstream` | Record `<remote>/<branch>` as the upstream for this branch |

374

| `--force` | Force-push even if the remote has diverged |

375

376

**Fast-forward enforcement:** by default, MuseHub rejects a push if its current

377

HEAD is not an ancestor of the local HEAD (HTTP 409). Pass `--force` to

378

override. Use `muse pull` first to integrate remote changes before pushing.

379

380

**First push workflow:**

381

382

```bash

383

muse push origin -u # push + record origin/main as upstream

384

```

385

386

Subsequent pushes on the same branch:

387

388

```bash

389

muse push # infers remote and branch from upstream config

```

---

### muse plumbing ls-remote

395

396

List branch references on a remote repository. **Plumbing command** — no local

state is written.

```

muse plumbing ls-remote [<remote-or-url>] [--json]

```

**Options:**

| Flag | Description |

406

|------|-------------|

407

| `<remote-or-url>` | Named remote or a full HTTPS URL (default: `origin`) |

408

| `--json` | Emit structured JSON instead of tab-delimited text |

409

410

**Default output** (tab-delimited, `*` marks default branch):

411

412

```

413

abc123def456... main *

789012abc345... dev

```

**JSON output** (for agent consumption):

```json

{

"repo_id": "<uuid>",

"domain": "midi",

"default_branch": "main",

424

"branches": {

425

"main": "<commit_id>",

"dev": "<commit_id>"

}

}

```

**Examples:**

```bash

muse plumbing ls-remote

435

muse plumbing ls-remote upstream

436

muse plumbing ls-remote https://hub.muse.io/repos/r1

437

muse plumbing ls-remote --json origin

```

---

## Tracking Branches

Muse tracks the relationship between local branches and remote branches in two

445

places:

446

447

1. **Upstream config** — `.muse/config.toml`:

448

```toml

449

[remotes.origin]

450

url = "https://hub.muse.io/repos/my-song"

451

branch = "main" # local branch "main" tracks origin/main

452

```

453

Written by `muse push -u` or `muse clone`.

454

455

2. **Remote tracking heads** — `.muse/remotes/<name>/<branch>`:

456

Plain-text files containing the last-known commit ID for each remote branch.

457

Written by `muse fetch`, `muse pull`, and `muse push`.

458

459

When a tracking relationship is set, `muse push` and `muse pull` resolve the

460

remote and branch automatically without additional arguments.

---

## Authentication

Muse stores bearer tokens in `~/.muse/identity.toml` — a machine-scoped,

467

`0o600` credential file that is never part of any repository snapshot.

468

Credentials are **not** stored in `.muse/config.toml`.

469

470

See [`docs/reference/auth.md`](auth.md) for the complete reference:

471

472

- `muse auth login` — store a token

473

- `muse auth whoami` — inspect stored identity

474

- `muse auth logout` — remove a token

475

- File format, permissions model, and security properties

476

477

The bearer token is automatically picked up by `fetch`, `pull`, `push`, and

478

`ls-remote` via `muse.core.identity.resolve_token()`. The token value is

479

**never** written to any log line — only `"Bearer ***"` appears in debug logs.

---

*See also:*

- [`docs/reference/auth.md`](auth.md) — identity management (`muse auth`)

485

- [`docs/reference/hub.md`](hub.md) — hub fabric connection (`muse hub`)

486

- [`docs/reference/security.md`](security.md) — full security architecture

487

- [`docs/reference/museignore.md`](museignore.md) — domain-scoped ignore rules

488

- [`docs/reference/muse-attributes.md`](muse-attributes.md) — merge strategy overrides

489

- [`docs/architecture/muse-vcs.md`](../architecture/muse-vcs.md) — system architecture