v0.14.0: a removal you make sticks

Round 43 made the merge report honest, and in doing so made its one real
limitation impossible to miss: every merge was a union, so a song removed from a
playlist on one machine was handed straight back by the other on the next sync,
and a track deleted from the library came back with it. A deletion you cannot
make stick is not a deletion.

The union was there for a real reason — two copies with no common ancestor
cannot tell "A added this" from "B removed it" — so the missing evidence is
written down instead of inferred. New lintunes/tombstones.py: a playlist keeps
track_events {track id: [when, add|remove]}, the library keeps deleted_tracks
{track id: when} in library_metadata.json, and a merge applies the newest event
per track across both copies. A removal beats a copy that merely still had the
song; a deliberate re-add afterwards beats the removal; a track nobody touched
still merges as a union, which stays the safe behavior where there is no
evidence either way.

Deliberately not "the newer copy wins wholesale": that one-liner silently drops
a song the other machine added while you were removing one, which
test_an_unrelated_addition_is_not_lost pins.

Events are recorded in the funnels that already exist — _set_track_ids diffs
before/after so a reorder records nothing, _remove_tracks stamps the library,
_restore_tracks clears it so Ctrl+Z takes the tombstone back — and pruned after
30 days at the save boundary.

Three edges worth naming:

* Track ids are never reused. The next id came from max(library.tracks), so
  deleting the highest-numbered track freed its id, and the next import would be
  dropped on sight by the dead id's own tombstone on every machine.
* library_metadata.json is merged before library.json, since it carries the
  record the library merge is filtered against and rglob order is not a plan.
* A merge applying a deletion never touches a music file — it drops the library
  entry only, and test_a_merge_never_touches_a_music_file fails the run if
  send_to_trash is so much as called. Applied removals grade WARNING and name
  the song.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y9ZEFi4qNJ39FMiBtiAxy2
This commit is contained in:
2026-08-27 18:31:33 -04:00
co-authored by Claude Opus 5
parent c8de124543
commit 8c097faacc
12 changed files with 594 additions and 26 deletions
+21 -1
View File
@@ -57,7 +57,8 @@ persistence) → GUI (Qt widgets that read the manager and connect to its signal
`plays/<machine-id>.json` per machine. Writes are atomic (`*.json.tmp` `plays/<machine-id>.json` per machine. Writes are atomic (`*.json.tmp`
→ rename). **`storage/conflict_resolver.py`** merges Syncthing → rename). **`storage/conflict_resolver.py`** merges Syncthing
`*.sync-conflict-*` files on startup: play counts take the max, edited fields `*.sync-conflict-*` files on startup: play counts take the max, edited fields
take the newest, playlist membership takes the union. Since Round 39 that take the newest, playlist membership takes the union **except where a removal
was recorded** (Round 44, see `tombstones.py`). Since Round 39 that
union is **anchor-based** (`merge_track_order`): a track only one copy has is union is **anchor-based** (`merge_track_order`): a track only one copy has is
re-inserted after the nearest track both share, not appended at the tail, so re-inserted after the nearest track both share, not appended at the tail, so
a middle insert stays in the middle. Whose order wins is decided by a middle insert stays in the middle. Whose order wins is decided by
@@ -169,6 +170,25 @@ persistence) → GUI (Qt widgets that read the manager and connect to its signal
back to `Path("Music")`, which resolved against a working directory GNOME's back to `Path("Music")`, which resolved against a working directory GNOME's
dash doesn't set predictably. dash doesn't set predictably.
- **`lintunes/tombstones.py`** — why a deletion sticks. Two copies with no
common ancestor cannot tell "A added this" from "B removed it", which is why
every merge was a union — and why a song removed on one machine came back from
the other forever. So removals are *recorded*: `Playlist.track_events`
(`{track id: [when, "add"|"remove"]}`) and `Library.deleted_tracks`
(`{track id: when}`, in `library_metadata.json`). A merge takes the **newest
event per track** across both copies and applies it, so a removal beats a copy
that merely still had the song and a later re-add beats the removal; a track
with no event still merges as a union. Events are recorded in the same single
funnels as everything else — `_set_track_ids` for playlists, `_remove_tracks`
for the library — and pruned after `RETENTION_DAYS` (30) at the save boundary,
so a machine offline longer than that can resurrect something. Three rules:
**track ids are never reused** (`_highest_track_id` counts deletions, or a new
track would be dropped on sight by the dead id's own tombstone),
`library_metadata.json` is merged **before** `library.json` (it carries the
record the library merge is filtered against — `resolve_conflicts` sorts for
it), and a merge applying a deletion **never touches a music file**; it only
drops the library entry, and reports at WARNING.
- **`lintunes/sync_identity.py`** — turns a conflict filename's 7-char device - **`lintunes/sync_identity.py`** — turns a conflict filename's 7-char device
token into a device name, by reading Syncthing's `config.xml` and deriving token into a device name, by reading Syncthing's `config.xml` and deriving
*our own* device ID from `cert.pem` (base32 of the SHA-256 of the DER cert). *our own* device ID from `cert.pem` (base32 of the SHA-256 of the DER cert).
+1 -1
View File
@@ -1,3 +1,3 @@
"""LinTunes — iTunes-style music library manager and player for Linux.""" """LinTunes — iTunes-style music library manager and player for Linux."""
__version__ = "0.13.0" __version__ = "0.14.0"
+2 -1
View File
@@ -25,7 +25,8 @@ from lintunes.storage.conflict_resolver import (
INTRO = ("LinTunes found changes made on more than one machine and merged them " INTRO = ("LinTunes found changes made on more than one machine and merged them "
"(play counts kept highest, newest edits win, nothing removed). Both " "(play counts kept highest, newest edits win, and a removal made on "
"either machine is applied on both). Both "
"versions were backed up first — restore them if a merge isn't what " "versions were backed up first — restore them if a merge isn't what "
"you wanted. The backup folder holds what-changed.txt: the same merge " "you wanted. The backup folder holds what-changed.txt: the same merge "
"in full, every track named.") "in full, every track named.")
+34 -4
View File
@@ -8,7 +8,7 @@ from PyQt6.QtCore import QObject, QTimer, pyqtSignal
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
from lintunes import music_folder as music_folder_mod from lintunes import music_folder as music_folder_mod
from lintunes import smart, tagging, trash from lintunes import smart, tagging, tombstones, trash
from lintunes.importers.file_importer import organized_destination, unique_path from lintunes.importers.file_importer import organized_destination, unique_path
from lintunes.models import Library, Playlist, PlaylistType from lintunes.models import Library, Playlist, PlaylistType
from lintunes.paths import to_relative from lintunes.paths import to_relative
@@ -96,13 +96,19 @@ class LibraryManager(QObject):
self._own_sigs: dict[str, tuple | None] = {} self._own_sigs: dict[str, tuple | None] = {}
self.snapshot_sync_state() self.snapshot_sync_state()
self._max_track_id = max(self.library.tracks.keys(), default=0) self._max_track_id = self._highest_track_id()
# ---- id generation ---- # ---- id generation ----
def new_track_id(self) -> int: def new_track_id(self) -> int:
return self._max_track_id + 1 return self._max_track_id + 1
def _highest_track_id(self) -> int:
"""The highest id ever used, deletions included. Ids must never be
reused: a new track handed a deleted one's id would be dropped on sight
by that id's own tombstone (and on every other machine too)."""
return max([*self.library.tracks, *self.library.deleted_tracks], default=0)
def new_persistent_id(self) -> str: def new_persistent_id(self) -> str:
while True: while True:
pid = secrets.token_hex(8).upper() pid = secrets.token_hex(8).upper()
@@ -336,6 +342,7 @@ class LibraryManager(QObject):
playlist = self.library.playlists.get(pid) playlist = self.library.playlists.get(pid)
if playlist is None: if playlist is None:
return return
before = list(playlist.track_ids)
playlist.track_ids = list(ids) playlist.track_ids = list(ids)
# The single funnel for every content change (add / remove / reorder / # The single funnel for every content change (add / remove / reorder /
# undo). date_modified moves only where content genuinely changed: here, # undo). date_modified moves only where content genuinely changed: here,
@@ -347,6 +354,10 @@ class LibraryManager(QObject):
# conflict_resolver._playlist_newer. # conflict_resolver._playlist_newer.
if touch: if touch:
playlist.date_modified = _utc_now_iso() playlist.date_modified = _utc_now_iso()
# Record *what* changed, not just that something did. A merge has no
# common ancestor, so without this a removal is indistinguishable
# from the other copy's addition and the union hands the song back.
tombstones.record(playlist.track_events, before, playlist.track_ids)
self._mark_playlist(pid, content=True) self._mark_playlist(pid, content=True)
self.playlist_content_changed.emit(pid) self.playlist_content_changed.emit(pid)
@@ -465,8 +476,13 @@ class LibraryManager(QObject):
def _remove_tracks(self, tracks: list, snapshots: dict[str, list[int]]): def _remove_tracks(self, tracks: list, snapshots: dict[str, list[int]]):
doomed = {t.track_id for t in tracks} doomed = {t.track_id for t in tracks}
when = _utc_now_iso()
for track in tracks: for track in tracks:
self.library.tracks.pop(track.track_id, None) self.library.tracks.pop(track.track_id, None)
# Same reason as the playlist events: library.json merges as a
# union, so a deletion the other copy hasn't heard about comes
# straight back without a record that it was deliberate.
self.library.deleted_tracks[track.track_id] = when
self._invalidate_artwork(track) self._invalidate_artwork(track)
for pid in snapshots: for pid in snapshots:
playlist = self.library.playlists.get(pid) playlist = self.library.playlists.get(pid)
@@ -482,6 +498,7 @@ class LibraryManager(QObject):
def _restore_tracks(self, tracks: list, snapshots: dict[str, list[int]]): def _restore_tracks(self, tracks: list, snapshots: dict[str, list[int]]):
for track in tracks: for track in tracks:
self.library.tracks[track.track_id] = track self.library.tracks[track.track_id] = track
self.library.deleted_tracks.pop(track.track_id, None)
self._max_track_id = max(self._max_track_id, track.track_id) self._max_track_id = max(self._max_track_id, track.track_id)
for pid, ids in snapshots.items(): for pid, ids in snapshots.items():
if pid in self.library.playlists: if pid in self.library.playlists:
@@ -1016,6 +1033,15 @@ class LibraryManager(QObject):
# amount equal to this machine's journal. # amount equal to this machine's journal.
self.play_journal.load(self.data_dir, disk) self.play_journal.load(self.data_dir, disk)
# Deletion records are additive on both sides: ours are not on disk yet
# if we haven't flushed, and theirs are news to us. Union first, then
# apply, so a track deleted on either machine stays deleted on both.
self.library.deleted_tracks = tombstones.merge_events(
self.library.deleted_tracks, disk.deleted_tracks)
for tid in self.library.deleted_tracks:
self.library.tracks.pop(tid, None)
disk.tracks.pop(tid, None)
for tid, disk_track in disk.tracks.items(): for tid, disk_track in disk.tracks.items():
mem_track = self.library.tracks.get(tid) mem_track = self.library.tracks.get(tid)
if mem_track is None: if mem_track is None:
@@ -1047,7 +1073,7 @@ class LibraryManager(QObject):
self.library.library_settings = disk.library_settings self.library.library_settings = disk.library_settings
self.refresh_music_folder() self.refresh_music_folder()
self._max_track_id = max(self.library.tracks.keys(), default=0) self._max_track_id = self._highest_track_id()
self._recomputing = True self._recomputing = True
try: try:
@@ -1098,6 +1124,8 @@ def _reconcile_playlist(mem_pl, disk_pl):
other machine was quietly undone. Same rule as other machine was quietly undone. Same rule as
conflict_resolver._playlist_newer, one layer up.""" conflict_resolver._playlist_newer, one layer up."""
from lintunes.storage.conflict_resolver import merge_track_order from lintunes.storage.conflict_resolver import merge_track_order
mem_pl.track_events = tombstones.merge_events(mem_pl.track_events,
disk_pl.track_events)
ours, theirs = mem_pl.date_modified, disk_pl.date_modified ours, theirs = mem_pl.date_modified, disk_pl.date_modified
disk_newer = bool(theirs) and (not ours or theirs > ours) disk_newer = bool(theirs) and (not ours or theirs > ours)
mine, other = list(mem_pl.track_ids), list(disk_pl.track_ids) mine, other = list(mem_pl.track_ids), list(disk_pl.track_ids)
@@ -1116,7 +1144,9 @@ def _reconcile_playlist(mem_pl, disk_pl):
# so is what lets the other machine adopt it instead of the two trading # so is what lets the other machine adopt it instead of the two trading
# the same tracks back and forth. # the same tracks back and forth.
mem_pl.date_modified = _utc_now_iso() mem_pl.date_modified = _utc_now_iso()
mem_pl.track_ids = merged # A union would hand back anything the newest event says was removed.
gone = tombstones.removed_ids(mem_pl.track_events)
mem_pl.track_ids = [tid for tid in merged if tid not in gone]
def _utc_now_iso() -> str: def _utc_now_iso() -> str:
+4
View File
@@ -24,6 +24,10 @@ class Library:
music_folder_rel: str = "" music_folder_rel: str = ""
music_folder_set_at: Optional[str] = None music_folder_set_at: Optional[str] = None
import_date: Optional[str] = None import_date: Optional[str] = None
# {track_id: when} for tracks deleted from the library inside the retention
# window. A merge unions two copies of library.json, so without this a track
# deleted here is handed straight back by the other machine's copy.
deleted_tracks: dict = field(default_factory=dict)
# Column/sort settings for the all-tracks Library view # Column/sort settings for the all-tracks Library view
library_settings: PlaylistSettings = field( library_settings: PlaylistSettings = field(
default_factory=lambda: PlaylistSettings(sort_column="artist")) default_factory=lambda: PlaylistSettings(sort_column="artist"))
+10
View File
@@ -62,6 +62,10 @@ class Playlist:
# for column widths or sort order — the conflict merge uses it to decide # for column widths or sort order — the conflict merge uses it to decide
# whose order wins, and the file's mtime moves for cosmetic reasons. # whose order wins, and the file's mtime moves for cosmetic reasons.
date_modified: Optional[str] = None date_modified: Optional[str] = None
# {track_id: [when, "add"|"remove"]} for membership changes inside the
# retention window. Without it a merge is a union and a song removed on one
# machine comes back from the other forever; see lintunes/tombstones.py.
track_events: dict = field(default_factory=dict)
# Set only for SMART playlists. Membership (track_ids) is derived from this # Set only for SMART playlists. Membership (track_ids) is derived from this
# by the LibraryManager; for imported playlists track_ids also doubles as a # by the LibraryManager; for imported playlists track_ids also doubles as a
# snapshot fallback when the criteria couldn't be fully parsed. # snapshot fallback when the criteria couldn't be fully parsed.
@@ -99,6 +103,10 @@ class Playlist:
d["parent_persistent_id"] = self.parent_persistent_id d["parent_persistent_id"] = self.parent_persistent_id
if self.date_modified: if self.date_modified:
d["date_modified"] = self.date_modified d["date_modified"] = self.date_modified
if self.track_events:
# JSON object keys are strings; track ids are ints everywhere else.
d["track_events"] = {str(tid): event
for tid, event in self.track_events.items()}
if self.smart_criteria is not None: if self.smart_criteria is not None:
d["smart_criteria"] = self.smart_criteria.to_dict() d["smart_criteria"] = self.smart_criteria.to_dict()
return d return d
@@ -118,6 +126,8 @@ class Playlist:
settings=PlaylistSettings.from_dict(d.get("settings", {})), settings=PlaylistSettings.from_dict(d.get("settings", {})),
is_system=d.get("is_system", False), is_system=d.get("is_system", False),
date_modified=d.get("date_modified"), date_modified=d.get("date_modified"),
track_events={int(tid): list(event)
for tid, event in (d.get("track_events") or {}).items()},
smart_criteria=smart_criteria, smart_criteria=smart_criteria,
) )
+108 -18
View File
@@ -4,7 +4,7 @@ from dataclasses import dataclass, field
from datetime import datetime, timezone from datetime import datetime, timezone
from pathlib import Path from pathlib import Path
from lintunes import sync_identity from lintunes import sync_identity, tombstones
from lintunes.storage.json_storage import read_json, write_json from lintunes.storage.json_storage import read_json, write_json
@@ -96,6 +96,11 @@ def resolve_conflicts(data_dir: Path) -> list[ConflictSummary]:
conflict_files = _find_conflict_files(data_dir) conflict_files = _find_conflict_files(data_dir)
if not conflict_files: if not conflict_files:
return [] return []
# library_metadata.json carries the record of what was deleted, and
# library.json is merged against it — so it has to be merged first, whatever
# order rglob happened to hand them back in.
conflict_files.sort(
key=lambda item: 0 if item.original_path.name == "library_metadata.json" else 1)
stamp = datetime.now().strftime("%Y%m%d-%H%M%S") stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
backup_dir = data_dir / ".resolved" / stamp backup_dir = data_dir / ".resolved" / stamp
@@ -120,7 +125,8 @@ def resolve_conflicts(data_dir: Path) -> list[ConflictSummary]:
"the other copy is in the backup."], "the other copy is in the backup."],
level=WARNING) level=WARNING)
elif original_path.name == "library.json": elif original_path.name == "library.json":
summary = _merge_library(original_path, conflict_path) summary = _merge_library(original_path, conflict_path,
data_dir, label)
elif original_path.name == "library_metadata.json": elif original_path.name == "library_metadata.json":
summary = _merge_metadata(original_path, conflict_path) summary = _merge_metadata(original_path, conflict_path)
elif original_path.parent.name == "playlists": elif original_path.parent.name == "playlists":
@@ -238,14 +244,18 @@ def _track_labels(data_dir: Path) -> dict[str, str]:
labels = {} labels = {}
try: try:
for tid, track in read_json(data_dir / "library.json").items(): for tid, track in read_json(data_dir / "library.json").items():
title = track.get("name") or f"track {tid}" labels[str(tid)] = _label_for(tid, track)
artist = track.get("artist") or track.get("album_artist")
labels[str(tid)] = f"{artist}{title}" if artist else title
except Exception: except Exception:
pass # no library yet, or an unreadable one: fall back to bare ids pass # no library yet, or an unreadable one: fall back to bare ids
return labels return labels
def _label_for(tid, track: dict) -> str:
title = track.get("name") or f"track {tid}"
artist = track.get("artist") or track.get("album_artist")
return f"{artist}{title}" if artist else title
class _Labeler: class _Labeler:
"""Names tracks for the report, loading library.json only if asked.""" """Names tracks for the report, loading library.json only if asked."""
@@ -258,6 +268,13 @@ class _Labeler:
self._labels = _track_labels(self._data_dir) self._labels = _track_labels(self._data_dir)
return self._labels.get(str(tid)) or f"track {tid}" return self._labels.get(str(tid)) or f"track {tid}"
def remember(self, tid, track: dict):
"""Keep a name that is about to be deleted from library.json — the
playlist merge still has to say which song it dropped."""
if self._labels is None:
self._labels = _track_labels(self._data_dir)
self._labels.setdefault(str(tid), _label_for(tid, track))
def _local_time(stamp: str | None) -> str: def _local_time(stamp: str | None) -> str:
"""A naive-UTC ISO stamp as local wall-clock, the way the window shows it.""" """A naive-UTC ISO stamp as local wall-clock, the way the window shows it."""
@@ -278,14 +295,28 @@ def _utc_now_iso() -> str:
return datetime.now(timezone.utc).replace(tzinfo=None).isoformat() return datetime.now(timezone.utc).replace(tzinfo=None).isoformat()
def _merge_library(original_path: Path, conflict_path: Path) -> ConflictSummary: def _merge_library(original_path: Path, conflict_path: Path,
data_dir: Path = None, label=None) -> ConflictSummary:
original = read_json(original_path) original = read_json(original_path)
conflict = read_json(conflict_path) conflict = read_json(conflict_path)
# Deliberate deletions, from the file merged just before this one. A union
# without them means a track deleted on either machine is handed back by
# whichever copy hadn't heard yet — every time, forever.
deleted = _deleted_tracks(data_dir)
added = 0 added = 0
changed = 0 changed = 0
removed = 0
examples: list[str] = [] examples: list[str] = []
for tid in list(original):
if tid in deleted:
if label is not None:
label.remember(tid, original[tid])
del original[tid]
removed += 1
for tid, conflict_track_data in conflict.items(): for tid, conflict_track_data in conflict.items():
if tid in deleted:
continue
if tid not in original: if tid not in original:
original[tid] = conflict_track_data original[tid] = conflict_track_data
added += 1 added += 1
@@ -305,11 +336,29 @@ def _merge_library(original_path: Path, conflict_path: Path) -> ConflictSummary:
"(play counts kept highest, newest edits win).") "(play counts kept highest, newest edits win).")
if added: if added:
lines.append(f"{added} track(s) present only on the other machine were kept.") lines.append(f"{added} track(s) present only on the other machine were kept.")
if removed:
lines.append(f"{_count(removed)} deleted from the library stayed deleted, "
"though the other copy still listed "
f"{'it' if removed == 1 else 'them'}. "
"No file on this machine was touched.")
if not lines: if not lines:
lines.append("No differences needed reconciling.") lines.append("No differences needed reconciling.")
lines.extend(examples) lines.extend(examples)
return ConflictSummary("library.json", "library", lines, return ConflictSummary(
level=CHANGE if (changed or added) else INFO) "library.json", "library", lines,
level=WARNING if removed else (CHANGE if (changed or added) else INFO))
def _deleted_tracks(data_dir: Path | None) -> dict:
"""The library's deletion record, keyed the way library.json is (strings)."""
if data_dir is None:
return {}
try:
metadata = read_json(data_dir / "library_metadata.json")
except (ValueError, OSError):
return {}
return {str(tid): when
for tid, when in (metadata.get("deleted_tracks") or {}).items()}
# Every field _merge_track_fields reads or writes. If two copies of a track # Every field _merge_track_fields reads or writes. If two copies of a track
@@ -443,6 +492,12 @@ def _playlist_newer(original: dict, conflict: dict,
"timestamps instead.") "timestamps instead.")
def _events(playlist: dict) -> dict:
"""A playlist file's membership events, with ids back in int form."""
return {int(tid): event
for tid, event in (playlist.get("track_events") or {}).items()}
def _positions(merged: list, tids, label) -> list[str]: def _positions(merged: list, tids, label) -> list[str]:
""""Artist — Title → position 12, after “…”" for each of `tids`.""" """"Artist — Title → position 12, after “…”" for each of `tids`."""
index = {} index = {}
@@ -531,6 +586,17 @@ def _merge_playlist(original_path: Path, conflict_path: Path,
merged = merge_track_order(conf_ids, orig_ids) merged = merge_track_order(conf_ids, orig_ids)
else: else:
merged = merge_track_order(orig_ids, conf_ids) merged = merge_track_order(orig_ids, conf_ids)
# A union alone can never honor a removal: with no common ancestor, "they
# removed it" and "we added it" look identical from the two lists. The
# newest recorded event per track breaks the tie — a removal beats a copy
# that merely still had the song, and a later re-add beats the removal.
events = tombstones.merge_events(_events(original), _events(conflict))
gone = tombstones.removed_ids(events) & set(merged)
merged = [tid for tid in merged if tid not in gone]
if events:
original["track_events"] = {str(tid): event
for tid, event in events.items()}
original["track_ids"] = merged original["track_ids"] = merged
renamed_from = "" renamed_from = ""
@@ -567,9 +633,10 @@ def _merge_playlist(original_path: Path, conflict_path: Path,
# merge made the copy in place harder to beat on the next one. # merge made the copy in place harder to beat on the next one.
write_json(original_path, original) write_json(original_path, original)
added = [tid for tid in conf_set - orig_set] added = [tid for tid in conf_set - orig_set if tid not in gone]
only_here = [tid for tid in orig_set - conf_set] only_here = [tid for tid in orig_set - conf_set if tid not in gone]
if orig_ids == conf_ids: removed = sorted(gone)
if orig_ids == conf_ids and not gone:
# The two copies held the same tracks in the same order, so they differed # The two copies held the same tracks in the same order, so they differed
# only in column widths or sort order — a window resize, not an edit. # only in column widths or sort order — a window resize, not an edit.
# Note this is a test on the *inputs*: a merge whose result happens to # Note this is a test on the *inputs*: a merge whose result happens to
@@ -610,19 +677,34 @@ def _merge_playlist(original_path: Path, conflict_path: Path,
# no way to tell which. Both readings are offered rather than guessed. # no way to tell which. Both readings are offered rather than guessed.
headline = (f"{_count(len(only_here))} here that the other copy did " headline = (f"{_count(len(only_here))} here that the other copy did "
f"not have {'was' if len(only_here) == 1 else 'were'} " f"not have {'was' if len(only_here) == 1 else 'were'} "
"kept — a merge never removes:") "kept — nothing recorded a removal:")
it, they, them = (("it", "it", "it") if len(only_here) == 1 it, they = ("it", "it") if len(only_here) == 1 else ("them", "they")
else ("them", "they", "them")) note = (f"Most likely you added {it} here. A removal made on the other "
note = (f"If you added {it} here, that's all this is. If you deleted " f"machine carries across on its own now, so {they} would have "
f"{it} on the other machine, delete {them} here too or " "gone already — unless it happened before that machine "
f"{they} will come back on the next merge.") f"updated, or longer ago than {tombstones.RETENTION_DAYS} days.")
lines.append(headline) lines.append(headline)
lines.extend(_capped(entries)) lines.extend(_capped(entries))
lines.append(note) lines.append(note)
detail.append(headline) detail.append(headline)
detail.extend(f"{entry}" for entry in entries) detail.extend(f"{entry}" for entry in entries)
detail.append(note) detail.append(note)
return ConflictSummary(name, "playlist", lines, level=CHANGE, detail=detail) if removed:
entries = [label(tid) for tid in removed]
headline = (f"{_count(len(removed))} removed from this playlist on the "
"other machine "
f"{'was' if len(removed) == 1 else 'were'} removed here too:")
note = ("Undo (Ctrl+Z) does not reach a merge — add "
f"{'it' if len(removed) == 1 else 'them'} back from the library "
"if this wasn't what you wanted. No file was touched.")
lines.append(headline)
lines.extend(_capped(entries))
lines.append(note)
detail.append(headline)
detail.extend(f"{entry}" for entry in entries)
detail.append(note)
return ConflictSummary(name, "playlist", lines,
level=WARNING if removed else CHANGE, detail=detail)
def _merge_play_journal(original_path: Path, conflict_path: Path) -> ConflictSummary: def _merge_play_journal(original_path: Path, conflict_path: Path) -> ConflictSummary:
@@ -666,6 +748,7 @@ def _merge_play_journal(original_path: Path, conflict_path: Path) -> ConflictSum
# What library_metadata.json actually holds, in words a person recognizes. # What library_metadata.json actually holds, in words a person recognizes.
_METADATA_LABELS = { _METADATA_LABELS = {
"library_settings": "column layout", "library_settings": "column layout",
"deleted_tracks": "record of deleted songs",
"music_folder": "music folder", "music_folder": "music folder",
"music_folder_rel": "music folder", "music_folder_rel": "music folder",
"music_folder_set_at": "music folder", "music_folder_set_at": "music folder",
@@ -701,6 +784,13 @@ def _merge_metadata(original_path: Path, conflict_path: Path) -> ConflictSummary
# machine can silently revert a folder change made on the other. # machine can silently revert a folder change made on the other.
if before is not None and other is not None: if before is not None and other is not None:
kept = read_json(original_path) kept = read_json(original_path)
# Deletions are additive: whichever whole copy the mtime pick kept, the
# other one's record of what was deleted is still true.
union = tombstones.merge_events(before.get("deleted_tracks") or {},
other.get("deleted_tracks") or {})
if union != (kept.get("deleted_tracks") or {}):
kept["deleted_tracks"] = union
write_json(original_path, kept)
# Decide the folder from the two *inputs*, never from whichever copy the # Decide the folder from the two *inputs*, never from whichever copy the
# mtime pick above landed on: when both files are written in the same # mtime pick above landed on: when both files are written in the same
# instant that pick is a coin flip, and it was deciding a deliberate # instant that pick is a coin flip, and it was deciding a deliberate
+25
View File
@@ -1,6 +1,7 @@
import json import json
from pathlib import Path from pathlib import Path
from lintunes import tombstones
from lintunes.models import Track, Playlist, PlaylistSettings, Library from lintunes.models import Track, Playlist, PlaylistSettings, Library
from lintunes.paths import to_relative, to_absolute from lintunes.paths import to_relative, to_absolute
@@ -66,6 +67,10 @@ def save_metadata(library: Library, data_dir: Path):
"track_count": len(library.tracks), "track_count": len(library.tracks),
"playlist_count": len(library.playlists), "playlist_count": len(library.playlists),
"library_settings": library.library_settings.to_dict(), "library_settings": library.library_settings.to_dict(),
# Pruned at the boundary, like the derived-membership gate below: the
# in-memory dict stays honest, the file doesn't grow forever.
"deleted_tracks": {str(tid): when for tid, when
in tombstones.prune(library.deleted_tracks).items()},
} }
write_json(data_dir / "library_metadata.json", metadata) write_json(data_dir / "library_metadata.json", metadata)
@@ -74,6 +79,10 @@ def save_playlist(playlist: Playlist, data_dir: Path):
playlists_dir = data_dir / "playlists" playlists_dir = data_dir / "playlists"
playlists_dir.mkdir(parents=True, exist_ok=True) playlists_dir.mkdir(parents=True, exist_ok=True)
data = playlist.to_dict() data = playlist.to_dict()
if data.get("track_events"):
data["track_events"] = tombstones.prune(data["track_events"])
if not data["track_events"]:
del data["track_events"]
if playlist.has_derived_membership: if playlist.has_derived_membership:
# Membership is a pure function of the criteria and the library, and # Membership is a pure function of the criteria and the library, and
# every machine recomputes it on load — storing it only bought a # every machine recomputes it on load — storing it only bought a
@@ -113,6 +122,7 @@ def load_library(data_dir: Path) -> Library:
music_folder_rel = "" music_folder_rel = ""
music_folder_set_at = None music_folder_set_at = None
import_date = None import_date = None
deleted_tracks = {}
library_settings = PlaylistSettings() library_settings = PlaylistSettings()
metadata_path = data_dir / "library_metadata.json" metadata_path = data_dir / "library_metadata.json"
if metadata_path.exists(): if metadata_path.exists():
@@ -121,6 +131,8 @@ def load_library(data_dir: Path) -> Library:
music_folder_rel = metadata.get("music_folder_rel", "") music_folder_rel = metadata.get("music_folder_rel", "")
music_folder_set_at = metadata.get("music_folder_set_at") music_folder_set_at = metadata.get("music_folder_set_at")
import_date = metadata.get("import_date") import_date = metadata.get("import_date")
deleted_tracks = {int(tid): when for tid, when
in (metadata.get("deleted_tracks") or {}).items()}
if "library_settings" in metadata: if "library_settings" in metadata:
library_settings = PlaylistSettings.from_dict(metadata["library_settings"]) library_settings = PlaylistSettings.from_dict(metadata["library_settings"])
@@ -133,9 +145,22 @@ def load_library(data_dir: Path) -> Library:
playlist = Playlist.from_dict(playlist_data) playlist = Playlist.from_dict(playlist_data)
playlists[playlist.persistent_id] = playlist playlists[playlist.persistent_id] = playlist
# A deleted track can still be sitting in a library.json that synced in
# from a machine which hasn't heard about the deletion yet, and in the
# track_ids of any playlist that held it. Applying the record here covers
# every load path at once — startup and the reload_from_disk reconcile.
for tid in deleted_tracks:
tracks.pop(tid, None)
if deleted_tracks:
for playlist in playlists.values():
if any(tid in deleted_tracks for tid in playlist.track_ids):
playlist.track_ids = [tid for tid in playlist.track_ids
if tid not in deleted_tracks]
return Library( return Library(
tracks=tracks, tracks=tracks,
playlists=playlists, playlists=playlists,
deleted_tracks=deleted_tracks,
music_folder=music_folder, music_folder=music_folder,
music_folder_rel=music_folder_rel, music_folder_rel=music_folder_rel,
music_folder_set_at=music_folder_set_at, music_folder_set_at=music_folder_set_at,
+89
View File
@@ -0,0 +1,89 @@
"""Removals that survive a merge.
Two copies of a playlist and no common ancestor cannot tell "A added this song"
apart from "B deleted it" — which is why every merge until now was a union, and
why a song removed on one machine came back from the other on the next sync.
A deletion you cannot make stick is not a deletion.
So the removal is recorded rather than inferred: each playlist keeps a small
``track_events`` map of ``{track id: (when, "add"|"remove")}``, and the library
keeps ``deleted_tracks``. A merge takes the *newest* event per track across both
copies and applies it, so a removal beats a copy that simply still had the song,
and a deliberate re-add afterwards beats the removal. Tracks nobody has touched
have no event and keep the old union behavior.
Events are pruned after ``RETENTION_DAYS``: by then every machine has seen them,
and keeping them forever would grow the file for nothing. The trade is that a
machine offline longer than that can resurrect a song it never learned was
deleted — which is the same trade every sync system makes, and the merge window
says out loud when a removal is applied.
"""
from datetime import datetime, timedelta, timezone
ADD = "add"
REMOVE = "remove"
# How long a membership event is kept. Long enough that a laptop shut in a bag
# for a month still learns about the deletion; short enough that the map stays
# a handful of entries.
RETENTION_DAYS = 30
def now_iso() -> str:
"""Naive UTC ISO — the one stamp format every machine writes, so the strings
sort correctly against each other without parsing."""
return datetime.now(timezone.utc).replace(tzinfo=None).isoformat()
def _expiry(now: str | None = None) -> str:
base = datetime.fromisoformat(now) if now else datetime.now(timezone.utc).replace(tzinfo=None)
return (base - timedelta(days=RETENTION_DAYS)).isoformat()
def prune(events: dict, now: str | None = None) -> dict:
"""Drop events older than the retention window."""
cutoff = _expiry(now)
return {key: value for key, value in events.items()
if _stamp(value) and _stamp(value) >= cutoff}
def _stamp(value):
"""An event is (when, op); a bare stamp means a deletion in deleted_tracks."""
if isinstance(value, (list, tuple)):
return value[0] if value else None
return value
def _op(value):
if isinstance(value, (list, tuple)) and len(value) > 1:
return value[1]
return REMOVE
def merge_events(ours: dict, theirs: dict) -> dict:
"""Newest event per track wins. Works for ``track_events`` (stamp + op) and
for ``deleted_tracks`` (bare stamps) alike."""
merged = dict(ours or {})
for key, value in (theirs or {}).items():
mine = merged.get(key)
if mine is None or (_stamp(value) or "") > (_stamp(mine) or ""):
merged[key] = value
return merged
def removed_ids(events: dict) -> set:
"""Which tracks the newest event says are gone."""
return {key for key, value in (events or {}).items() if _op(value) == REMOVE}
def record(events: dict, before, after) -> dict:
"""Note what a membership change did, in place. ``before``/``after`` are the
two track_ids lists; a reorder produces no events at all."""
was, now = set(before), set(after)
stamp = now_iso()
for tid in was - now:
events[tid] = [stamp, REMOVE]
for tid in now - was:
events[tid] = [stamp, ADD]
return events
+50
View File
@@ -1,5 +1,55 @@
## Done ## Done
### Round 44 (2026-08-27) — A removal you make sticks (v0.14.0)
Round 43 made the merge report honest, which made its one real limitation
impossible to miss: every merge was a union, so a song removed from a playlist
on one machine was handed straight back by the other on the next sync, and a
track deleted from the library came back with it. A deletion you cannot make
stick is not a deletion.
- [x] **Removals are recorded, not inferred.** The union was there for a real
reason — two copies and no common ancestor cannot tell "A added this" from
"B removed it" — so the missing evidence is now written down. New
`lintunes/tombstones.py`: `Playlist.track_events` holds
`{track id: [when, "add"|"remove"]}` and `Library.deleted_tracks` holds
`{track id: when}` in `library_metadata.json`. A merge takes the newest
event per track across both copies and applies it. A removal beats a copy
that merely still had the song; a deliberate re-add afterwards beats the
removal; a track nobody touched still merges as a union, which is the safe
old behavior for everything that has no evidence either way.
- [x] **Not "the newer copy wins wholesale".** That was the tempting one-line
version and it silently drops a song the other machine added while you
were removing one. `test_an_unrelated_addition_is_not_lost` pins it.
- [x] **Recorded in the existing funnels.** `_set_track_ids` diffs before/after
(so a reorder records nothing), `_remove_tracks` stamps
`deleted_tracks`, and `_restore_tracks` clears it, so Ctrl+Z on a delete
takes the tombstone back with it.
- [x] **Track ids are never reused.** The sharpest edge in the whole design: the
next id came from `max(library.tracks)`, so deleting the highest-numbered
track freed its id for the next import — and that new track would be
dropped on sight by the dead id's own tombstone, on every machine.
`_highest_track_id` counts the deletions too.
- [x] **`library_metadata.json` merges before `library.json`.** It carries the
record the library merge filters against, and `rglob` order is not a plan.
`_merge_metadata` unions the two copies' `deleted_tracks` regardless of
which whole copy the mtime pick kept.
- [x] **A merge applying a deletion never touches a music file.** It drops the
library entry only — the file was already trashed on the machine where the
delete happened, and the music folder is its own Syncthing share.
`test_a_merge_never_touches_a_music_file` fails the run if `send_to_trash`
is so much as called.
- [x] **Reported as a warning, with the song named.** An applied removal grades
WARNING so it opens the merge window, names each track, and says Ctrl+Z
does not reach a merge. `_Labeler.remember` keeps the name of a track
library.json is about to lose, so the playlist merge can still say which
song it dropped rather than "track 4".
- [x] **Events expire after 30 days** (`RETENTION_DAYS`), pruned at the save
boundary like the derived-membership gate. The trade is stated in the
module docstring: a machine offline longer than that can resurrect a song
it never learned was deleted.
### Round 43 (2026-08-27) — The merge report says who, what, and where (v0.13.0) ### Round 43 (2026-08-27) — The merge report says who, what, and where (v0.13.0)
The merge window kept saying `Order kept from this machine (most recently The merge window kept saying `Order kept from this machine (most recently
+3 -1
View File
@@ -242,7 +242,9 @@ class TestNamingTheSongs:
body = "\n".join(summaries[0].lines) body = "\n".join(summaries[0].lines)
assert "the other copy did not have" in body assert "the other copy did not have" in body
assert "Pola — Abeille" in body assert "Pola — Abeille" in body
assert "delete it here too" in body # Round 44: a removal now carries across on its own, so the note no
# longer tells you to delete it here by hand.
assert "Most likely you added it here" in body
def test_the_window_caps_the_list_and_the_backup_holds_all_of_it(self, tmp_path): def test_the_window_caps_the_list_and_the_backup_holds_all_of_it(self, tmp_path):
from lintunes.models import Track from lintunes.models import Track
+247
View File
@@ -0,0 +1,247 @@
"""Round 44: a removal you make sticks.
Round 43 made the merge report honest, and in doing so made its one real
limitation impossible to miss: every merge was a *union*, so a song removed from
a playlist on one machine was handed straight back by the other one on the next
sync, forever. Same for a track deleted from the library. A deletion you cannot
make stick is not a deletion.
The reason it was a union is real — two copies and no common ancestor cannot
tell "A added this" from "B removed it". So the removal is now *recorded*
(`lintunes/tombstones.py`): a playlist keeps `track_events`, the library keeps
`deleted_tracks`, and a merge applies the newest event per track. A removal
beats a copy that merely still had the song; a deliberate re-add afterwards
beats the removal; a track nobody has touched still merges as a union.
"""
import json
import shutil
import pytest
from lintunes import tombstones
from lintunes.library_manager import LibraryManager
from lintunes.models import Library, Playlist, PlaylistType, Track
from lintunes.storage import json_storage, conflict_resolver
from lintunes.storage.conflict_resolver import WARNING, CHANGE
PID = "AAAA1111"
def _machine(tmp_path, name, ids=(1, 2, 3)):
"""A data dir with three tracks and one playlist holding them."""
root = tmp_path / name
lib = Library()
for tid in (1, 2, 3, 4):
lib.tracks[tid] = Track(track_id=tid, name=f"Song {tid}", artist="A")
lib.playlists[PID] = Playlist(persistent_id=PID, name="mix",
playlist_type=PlaylistType.REGULAR,
track_ids=list(ids))
json_storage.save_library(lib, root)
return root
def _manager(root, qapp):
return LibraryManager(json_storage.load_library(root), root)
def _sync_as_conflict(src, dest, rel="playlists/AAAA1111.json"):
"""What Syncthing does when both copies changed: src's version lands beside
dest's own, under a conflict name."""
source = src / rel
name = f"{source.stem}.sync-conflict-20260827-120000-CMFNCIX{source.suffix}"
shutil.copy2(source, dest / rel.rsplit("/", 1)[0] / name)
# library_metadata.json rides along — it carries the deletion record.
meta = src / "library_metadata.json"
if meta.exists():
shutil.copy2(meta, dest / "library_metadata.sync-conflict-"
"20260827-120000-CMFNCIX.json")
return conflict_resolver.resolve_conflicts(dest)
def _ids(root, pid=PID):
return json.loads((root / "playlists" / f"{pid}.json").read_text())["track_ids"]
class TestARemovalSticks:
def test_it_survives_the_round_trip(self, tmp_path, qapp):
"""The whole complaint: remove a song here, go to the other machine, and
it is back. Twice over — the second sync used to re-add it again."""
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
manager = _manager(a, qapp)
manager.remove_tracks_from_playlist(PID, [1]) # row index 1 == track 2
manager.flush()
assert _ids(a) == [1, 3]
_sync_as_conflict(a, b)
assert _ids(b) == [1, 3]
# ...and B's copy, now travelling back, must not resurrect it either.
_sync_as_conflict(b, a)
assert _ids(a) == [1, 3]
def test_a_deliberate_re_add_beats_the_removal(self, tmp_path, qapp):
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
manager_a = _manager(a, qapp)
manager_a.remove_tracks_from_playlist(PID, [1])
manager_a.flush()
_sync_as_conflict(a, b)
assert _ids(b) == [1, 3]
manager_b = _manager(b, qapp)
manager_b.add_tracks_to_playlist(PID, [2])
manager_b.flush()
_sync_as_conflict(b, a)
assert 2 in _ids(a)
def test_an_unrelated_addition_is_not_lost(self, tmp_path, qapp):
"""The reason this isn't just "the newer copy wins wholesale": that would
drop a song the other machine added while we were removing one."""
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
manager_a = _manager(a, qapp)
manager_a.remove_tracks_from_playlist(PID, [1])
manager_a.flush()
manager_b = _manager(b, qapp)
manager_b.add_tracks_to_playlist(PID, [4])
manager_b.flush()
_sync_as_conflict(a, b)
assert _ids(b) == [1, 3, 4]
def test_a_track_nobody_touched_still_merges_as_a_union(self, tmp_path, qapp):
"""No event means no evidence, and the old behavior is still the safe
one: keep it."""
a = _machine(tmp_path, "a", ids=[1, 2])
b = _machine(tmp_path, "b", ids=[1, 2, 3])
_sync_as_conflict(b, a)
assert set(_ids(a)) == {1, 2, 3}
def test_the_removal_is_reported_as_a_warning(self, tmp_path, qapp):
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
manager = _manager(a, qapp)
manager.remove_tracks_from_playlist(PID, [1])
manager.flush()
summaries = _sync_as_conflict(a, b)
playlist = [s for s in summaries if s.kind == "playlist"][0]
assert playlist.level == WARNING
body = "\n".join(playlist.lines)
assert "removed here too" in body
assert "A — Song 2" in body
assert "No file was touched" in body
class TestDeletingFromTheLibrary:
def test_a_deleted_track_stays_deleted_through_a_merge(self, tmp_path, qapp):
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
manager = _manager(a, qapp)
removed, failures = manager.delete_tracks([2], delete_files=False)
manager.flush()
assert removed == [2] and not failures
summaries = _sync_as_conflict(a, b)
# b's library.json still lists track 2; the deletion record must win.
shutil.copy2(a / "library.json",
b / "library.sync-conflict-20260827-120000-CMFNCIX.json")
summaries += conflict_resolver.resolve_conflicts(b)
assert "2" not in json.loads((b / "library.json").read_text())
assert 2 not in _ids(b)
library = [s for s in summaries if s.kind == "library"]
if library: # only when library.json actually conflicted
assert library[0].level == WARNING
def test_load_drops_a_deleted_track_and_its_playlist_rows(self, tmp_path):
"""The clean-sync path: the other machine's library.json arrives with no
conflict at all, still listing the track."""
root = _machine(tmp_path, "a")
metadata = json.loads((root / "library_metadata.json").read_text())
metadata["deleted_tracks"] = {"2": tombstones.now_iso()}
(root / "library_metadata.json").write_text(json.dumps(metadata))
library = json_storage.load_library(root)
assert 2 not in library.tracks
assert library.playlists[PID].track_ids == [1, 3]
def test_undo_takes_the_deletion_back(self, tmp_path, qapp):
root = _machine(tmp_path, "a")
manager = _manager(root, qapp)
manager.delete_tracks([2], delete_files=False)
assert 2 in manager.library.deleted_tracks
manager.undo_stack.undo()
assert 2 not in manager.library.deleted_tracks
assert 2 in manager.library.tracks
def test_a_merge_never_touches_a_music_file(self, tmp_path, qapp, monkeypatch):
"""A deletion syncing in removes the library entry only. The file was
already trashed on the machine where the delete happened, and the music
folder is its own Syncthing share."""
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
manager = _manager(a, qapp)
manager.delete_tracks([2], delete_files=False)
manager.flush()
from lintunes import trash
monkeypatch.setattr(trash, "send_to_trash", lambda *args: pytest.fail(
"a merge must never touch a music file"))
_sync_as_conflict(a, b)
class TestTheRecordItself:
def test_a_reorder_records_nothing(self, tmp_path, qapp):
"""Only membership changes are events; moving a row is not a removal."""
root = _machine(tmp_path, "a")
manager = _manager(root, qapp)
manager.move_tracks_in_playlist(PID, [2], 0)
assert manager.library.playlists[PID].track_events == {}
def test_events_expire_so_the_file_does_not_grow_forever(self, tmp_path):
old = "2020-01-01T00:00:00"
fresh = tombstones.now_iso()
kept = tombstones.prune({1: [old, "remove"], 2: [fresh, "remove"]})
assert kept == {2: [fresh, "remove"]}
def test_the_newest_event_per_track_wins_a_merge(self):
merged = tombstones.merge_events(
{1: ["2026-08-01T00:00:00", "remove"], 2: ["2026-08-01T00:00:00", "add"]},
{1: ["2026-08-02T00:00:00", "add"]})
assert merged[1] == ["2026-08-02T00:00:00", "add"]
assert tombstones.removed_ids(merged) == set()
def test_they_round_trip_through_the_playlist_file(self, tmp_path):
root = tmp_path / "a"
lib = Library()
playlist = Playlist(persistent_id=PID, playlist_type=PlaylistType.REGULAR,
track_ids=[1])
playlist.track_events = {2: [tombstones.now_iso(), "remove"]}
lib.playlists[PID] = playlist
json_storage.save_library(lib, root)
loaded = json_storage.load_library(root)
assert tombstones.removed_ids(loaded.playlists[PID].track_events) == {2}
class TestIdsAreNeverReused:
def test_a_new_track_does_not_inherit_a_deleted_ones_id(self, tmp_path, qapp):
"""The nastiest failure this design could produce: hand a fresh track
the id of a deleted one and every machine drops it on sight, its own
tombstone having outlived it."""
root = _machine(tmp_path, "a")
manager = _manager(root, qapp)
manager.delete_tracks([4], delete_files=False) # 4 was the highest id
manager.flush()
reopened = _manager(root, qapp)
assert reopened.new_track_id() > 4
assert 4 in reopened.library.deleted_tracks