v0.14.0: a removal you make sticks
Round 43 made the merge report honest, and in doing so made its one real
limitation impossible to miss: every merge was a union, so a song removed from a
playlist on one machine was handed straight back by the other on the next sync,
and a track deleted from the library came back with it. A deletion you cannot
make stick is not a deletion.
The union was there for a real reason — two copies with no common ancestor
cannot tell "A added this" from "B removed it" — so the missing evidence is
written down instead of inferred. New lintunes/tombstones.py: a playlist keeps
track_events {track id: [when, add|remove]}, the library keeps deleted_tracks
{track id: when} in library_metadata.json, and a merge applies the newest event
per track across both copies. A removal beats a copy that merely still had the
song; a deliberate re-add afterwards beats the removal; a track nobody touched
still merges as a union, which stays the safe behavior where there is no
evidence either way.
Deliberately not "the newer copy wins wholesale": that one-liner silently drops
a song the other machine added while you were removing one, which
test_an_unrelated_addition_is_not_lost pins.
Events are recorded in the funnels that already exist — _set_track_ids diffs
before/after so a reorder records nothing, _remove_tracks stamps the library,
_restore_tracks clears it so Ctrl+Z takes the tombstone back — and pruned after
30 days at the save boundary.
Three edges worth naming:
* Track ids are never reused. The next id came from max(library.tracks), so
deleting the highest-numbered track freed its id, and the next import would be
dropped on sight by the dead id's own tombstone on every machine.
* library_metadata.json is merged before library.json, since it carries the
record the library merge is filtered against and rglob order is not a plan.
* A merge applying a deletion never touches a music file — it drops the library
entry only, and test_a_merge_never_touches_a_music_file fails the run if
send_to_trash is so much as called. Applied removals grade WARNING and name
the song.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y9ZEFi4qNJ39FMiBtiAxy2
This commit is contained in:
@@ -57,7 +57,8 @@ persistence) → GUI (Qt widgets that read the manager and connect to its signal
|
|||||||
`plays/<machine-id>.json` per machine. Writes are atomic (`*.json.tmp`
|
`plays/<machine-id>.json` per machine. Writes are atomic (`*.json.tmp`
|
||||||
→ rename). **`storage/conflict_resolver.py`** merges Syncthing
|
→ rename). **`storage/conflict_resolver.py`** merges Syncthing
|
||||||
`*.sync-conflict-*` files on startup: play counts take the max, edited fields
|
`*.sync-conflict-*` files on startup: play counts take the max, edited fields
|
||||||
take the newest, playlist membership takes the union. Since Round 39 that
|
take the newest, playlist membership takes the union **except where a removal
|
||||||
|
was recorded** (Round 44, see `tombstones.py`). Since Round 39 that
|
||||||
union is **anchor-based** (`merge_track_order`): a track only one copy has is
|
union is **anchor-based** (`merge_track_order`): a track only one copy has is
|
||||||
re-inserted after the nearest track both share, not appended at the tail, so
|
re-inserted after the nearest track both share, not appended at the tail, so
|
||||||
a middle insert stays in the middle. Whose order wins is decided by
|
a middle insert stays in the middle. Whose order wins is decided by
|
||||||
@@ -169,6 +170,25 @@ persistence) → GUI (Qt widgets that read the manager and connect to its signal
|
|||||||
back to `Path("Music")`, which resolved against a working directory GNOME's
|
back to `Path("Music")`, which resolved against a working directory GNOME's
|
||||||
dash doesn't set predictably.
|
dash doesn't set predictably.
|
||||||
|
|
||||||
|
- **`lintunes/tombstones.py`** — why a deletion sticks. Two copies with no
|
||||||
|
common ancestor cannot tell "A added this" from "B removed it", which is why
|
||||||
|
every merge was a union — and why a song removed on one machine came back from
|
||||||
|
the other forever. So removals are *recorded*: `Playlist.track_events`
|
||||||
|
(`{track id: [when, "add"|"remove"]}`) and `Library.deleted_tracks`
|
||||||
|
(`{track id: when}`, in `library_metadata.json`). A merge takes the **newest
|
||||||
|
event per track** across both copies and applies it, so a removal beats a copy
|
||||||
|
that merely still had the song and a later re-add beats the removal; a track
|
||||||
|
with no event still merges as a union. Events are recorded in the same single
|
||||||
|
funnels as everything else — `_set_track_ids` for playlists, `_remove_tracks`
|
||||||
|
for the library — and pruned after `RETENTION_DAYS` (30) at the save boundary,
|
||||||
|
so a machine offline longer than that can resurrect something. Three rules:
|
||||||
|
**track ids are never reused** (`_highest_track_id` counts deletions, or a new
|
||||||
|
track would be dropped on sight by the dead id's own tombstone),
|
||||||
|
`library_metadata.json` is merged **before** `library.json` (it carries the
|
||||||
|
record the library merge is filtered against — `resolve_conflicts` sorts for
|
||||||
|
it), and a merge applying a deletion **never touches a music file**; it only
|
||||||
|
drops the library entry, and reports at WARNING.
|
||||||
|
|
||||||
- **`lintunes/sync_identity.py`** — turns a conflict filename's 7-char device
|
- **`lintunes/sync_identity.py`** — turns a conflict filename's 7-char device
|
||||||
token into a device name, by reading Syncthing's `config.xml` and deriving
|
token into a device name, by reading Syncthing's `config.xml` and deriving
|
||||||
*our own* device ID from `cert.pem` (base32 of the SHA-256 of the DER cert).
|
*our own* device ID from `cert.pem` (base32 of the SHA-256 of the DER cert).
|
||||||
|
|||||||
@@ -1,3 +1,3 @@
|
|||||||
"""LinTunes — iTunes-style music library manager and player for Linux."""
|
"""LinTunes — iTunes-style music library manager and player for Linux."""
|
||||||
|
|
||||||
__version__ = "0.13.0"
|
__version__ = "0.14.0"
|
||||||
|
|||||||
@@ -25,7 +25,8 @@ from lintunes.storage.conflict_resolver import (
|
|||||||
|
|
||||||
|
|
||||||
INTRO = ("LinTunes found changes made on more than one machine and merged them "
|
INTRO = ("LinTunes found changes made on more than one machine and merged them "
|
||||||
"(play counts kept highest, newest edits win, nothing removed). Both "
|
"(play counts kept highest, newest edits win, and a removal made on "
|
||||||
|
"either machine is applied on both). Both "
|
||||||
"versions were backed up first — restore them if a merge isn't what "
|
"versions were backed up first — restore them if a merge isn't what "
|
||||||
"you wanted. The backup folder holds what-changed.txt: the same merge "
|
"you wanted. The backup folder holds what-changed.txt: the same merge "
|
||||||
"in full, every track named.")
|
"in full, every track named.")
|
||||||
|
|||||||
@@ -8,7 +8,7 @@ from PyQt6.QtCore import QObject, QTimer, pyqtSignal
|
|||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
from lintunes import music_folder as music_folder_mod
|
from lintunes import music_folder as music_folder_mod
|
||||||
from lintunes import smart, tagging, trash
|
from lintunes import smart, tagging, tombstones, trash
|
||||||
from lintunes.importers.file_importer import organized_destination, unique_path
|
from lintunes.importers.file_importer import organized_destination, unique_path
|
||||||
from lintunes.models import Library, Playlist, PlaylistType
|
from lintunes.models import Library, Playlist, PlaylistType
|
||||||
from lintunes.paths import to_relative
|
from lintunes.paths import to_relative
|
||||||
@@ -96,13 +96,19 @@ class LibraryManager(QObject):
|
|||||||
self._own_sigs: dict[str, tuple | None] = {}
|
self._own_sigs: dict[str, tuple | None] = {}
|
||||||
self.snapshot_sync_state()
|
self.snapshot_sync_state()
|
||||||
|
|
||||||
self._max_track_id = max(self.library.tracks.keys(), default=0)
|
self._max_track_id = self._highest_track_id()
|
||||||
|
|
||||||
# ---- id generation ----
|
# ---- id generation ----
|
||||||
|
|
||||||
def new_track_id(self) -> int:
|
def new_track_id(self) -> int:
|
||||||
return self._max_track_id + 1
|
return self._max_track_id + 1
|
||||||
|
|
||||||
|
def _highest_track_id(self) -> int:
|
||||||
|
"""The highest id ever used, deletions included. Ids must never be
|
||||||
|
reused: a new track handed a deleted one's id would be dropped on sight
|
||||||
|
by that id's own tombstone (and on every other machine too)."""
|
||||||
|
return max([*self.library.tracks, *self.library.deleted_tracks], default=0)
|
||||||
|
|
||||||
def new_persistent_id(self) -> str:
|
def new_persistent_id(self) -> str:
|
||||||
while True:
|
while True:
|
||||||
pid = secrets.token_hex(8).upper()
|
pid = secrets.token_hex(8).upper()
|
||||||
@@ -336,6 +342,7 @@ class LibraryManager(QObject):
|
|||||||
playlist = self.library.playlists.get(pid)
|
playlist = self.library.playlists.get(pid)
|
||||||
if playlist is None:
|
if playlist is None:
|
||||||
return
|
return
|
||||||
|
before = list(playlist.track_ids)
|
||||||
playlist.track_ids = list(ids)
|
playlist.track_ids = list(ids)
|
||||||
# The single funnel for every content change (add / remove / reorder /
|
# The single funnel for every content change (add / remove / reorder /
|
||||||
# undo). date_modified moves only where content genuinely changed: here,
|
# undo). date_modified moves only where content genuinely changed: here,
|
||||||
@@ -347,6 +354,10 @@ class LibraryManager(QObject):
|
|||||||
# conflict_resolver._playlist_newer.
|
# conflict_resolver._playlist_newer.
|
||||||
if touch:
|
if touch:
|
||||||
playlist.date_modified = _utc_now_iso()
|
playlist.date_modified = _utc_now_iso()
|
||||||
|
# Record *what* changed, not just that something did. A merge has no
|
||||||
|
# common ancestor, so without this a removal is indistinguishable
|
||||||
|
# from the other copy's addition and the union hands the song back.
|
||||||
|
tombstones.record(playlist.track_events, before, playlist.track_ids)
|
||||||
self._mark_playlist(pid, content=True)
|
self._mark_playlist(pid, content=True)
|
||||||
self.playlist_content_changed.emit(pid)
|
self.playlist_content_changed.emit(pid)
|
||||||
|
|
||||||
@@ -465,8 +476,13 @@ class LibraryManager(QObject):
|
|||||||
|
|
||||||
def _remove_tracks(self, tracks: list, snapshots: dict[str, list[int]]):
|
def _remove_tracks(self, tracks: list, snapshots: dict[str, list[int]]):
|
||||||
doomed = {t.track_id for t in tracks}
|
doomed = {t.track_id for t in tracks}
|
||||||
|
when = _utc_now_iso()
|
||||||
for track in tracks:
|
for track in tracks:
|
||||||
self.library.tracks.pop(track.track_id, None)
|
self.library.tracks.pop(track.track_id, None)
|
||||||
|
# Same reason as the playlist events: library.json merges as a
|
||||||
|
# union, so a deletion the other copy hasn't heard about comes
|
||||||
|
# straight back without a record that it was deliberate.
|
||||||
|
self.library.deleted_tracks[track.track_id] = when
|
||||||
self._invalidate_artwork(track)
|
self._invalidate_artwork(track)
|
||||||
for pid in snapshots:
|
for pid in snapshots:
|
||||||
playlist = self.library.playlists.get(pid)
|
playlist = self.library.playlists.get(pid)
|
||||||
@@ -482,6 +498,7 @@ class LibraryManager(QObject):
|
|||||||
def _restore_tracks(self, tracks: list, snapshots: dict[str, list[int]]):
|
def _restore_tracks(self, tracks: list, snapshots: dict[str, list[int]]):
|
||||||
for track in tracks:
|
for track in tracks:
|
||||||
self.library.tracks[track.track_id] = track
|
self.library.tracks[track.track_id] = track
|
||||||
|
self.library.deleted_tracks.pop(track.track_id, None)
|
||||||
self._max_track_id = max(self._max_track_id, track.track_id)
|
self._max_track_id = max(self._max_track_id, track.track_id)
|
||||||
for pid, ids in snapshots.items():
|
for pid, ids in snapshots.items():
|
||||||
if pid in self.library.playlists:
|
if pid in self.library.playlists:
|
||||||
@@ -1016,6 +1033,15 @@ class LibraryManager(QObject):
|
|||||||
# amount equal to this machine's journal.
|
# amount equal to this machine's journal.
|
||||||
self.play_journal.load(self.data_dir, disk)
|
self.play_journal.load(self.data_dir, disk)
|
||||||
|
|
||||||
|
# Deletion records are additive on both sides: ours are not on disk yet
|
||||||
|
# if we haven't flushed, and theirs are news to us. Union first, then
|
||||||
|
# apply, so a track deleted on either machine stays deleted on both.
|
||||||
|
self.library.deleted_tracks = tombstones.merge_events(
|
||||||
|
self.library.deleted_tracks, disk.deleted_tracks)
|
||||||
|
for tid in self.library.deleted_tracks:
|
||||||
|
self.library.tracks.pop(tid, None)
|
||||||
|
disk.tracks.pop(tid, None)
|
||||||
|
|
||||||
for tid, disk_track in disk.tracks.items():
|
for tid, disk_track in disk.tracks.items():
|
||||||
mem_track = self.library.tracks.get(tid)
|
mem_track = self.library.tracks.get(tid)
|
||||||
if mem_track is None:
|
if mem_track is None:
|
||||||
@@ -1047,7 +1073,7 @@ class LibraryManager(QObject):
|
|||||||
self.library.library_settings = disk.library_settings
|
self.library.library_settings = disk.library_settings
|
||||||
self.refresh_music_folder()
|
self.refresh_music_folder()
|
||||||
|
|
||||||
self._max_track_id = max(self.library.tracks.keys(), default=0)
|
self._max_track_id = self._highest_track_id()
|
||||||
|
|
||||||
self._recomputing = True
|
self._recomputing = True
|
||||||
try:
|
try:
|
||||||
@@ -1098,6 +1124,8 @@ def _reconcile_playlist(mem_pl, disk_pl):
|
|||||||
other machine was quietly undone. Same rule as
|
other machine was quietly undone. Same rule as
|
||||||
conflict_resolver._playlist_newer, one layer up."""
|
conflict_resolver._playlist_newer, one layer up."""
|
||||||
from lintunes.storage.conflict_resolver import merge_track_order
|
from lintunes.storage.conflict_resolver import merge_track_order
|
||||||
|
mem_pl.track_events = tombstones.merge_events(mem_pl.track_events,
|
||||||
|
disk_pl.track_events)
|
||||||
ours, theirs = mem_pl.date_modified, disk_pl.date_modified
|
ours, theirs = mem_pl.date_modified, disk_pl.date_modified
|
||||||
disk_newer = bool(theirs) and (not ours or theirs > ours)
|
disk_newer = bool(theirs) and (not ours or theirs > ours)
|
||||||
mine, other = list(mem_pl.track_ids), list(disk_pl.track_ids)
|
mine, other = list(mem_pl.track_ids), list(disk_pl.track_ids)
|
||||||
@@ -1116,7 +1144,9 @@ def _reconcile_playlist(mem_pl, disk_pl):
|
|||||||
# so is what lets the other machine adopt it instead of the two trading
|
# so is what lets the other machine adopt it instead of the two trading
|
||||||
# the same tracks back and forth.
|
# the same tracks back and forth.
|
||||||
mem_pl.date_modified = _utc_now_iso()
|
mem_pl.date_modified = _utc_now_iso()
|
||||||
mem_pl.track_ids = merged
|
# A union would hand back anything the newest event says was removed.
|
||||||
|
gone = tombstones.removed_ids(mem_pl.track_events)
|
||||||
|
mem_pl.track_ids = [tid for tid in merged if tid not in gone]
|
||||||
|
|
||||||
|
|
||||||
def _utc_now_iso() -> str:
|
def _utc_now_iso() -> str:
|
||||||
|
|||||||
@@ -24,6 +24,10 @@ class Library:
|
|||||||
music_folder_rel: str = ""
|
music_folder_rel: str = ""
|
||||||
music_folder_set_at: Optional[str] = None
|
music_folder_set_at: Optional[str] = None
|
||||||
import_date: Optional[str] = None
|
import_date: Optional[str] = None
|
||||||
|
# {track_id: when} for tracks deleted from the library inside the retention
|
||||||
|
# window. A merge unions two copies of library.json, so without this a track
|
||||||
|
# deleted here is handed straight back by the other machine's copy.
|
||||||
|
deleted_tracks: dict = field(default_factory=dict)
|
||||||
# Column/sort settings for the all-tracks Library view
|
# Column/sort settings for the all-tracks Library view
|
||||||
library_settings: PlaylistSettings = field(
|
library_settings: PlaylistSettings = field(
|
||||||
default_factory=lambda: PlaylistSettings(sort_column="artist"))
|
default_factory=lambda: PlaylistSettings(sort_column="artist"))
|
||||||
|
|||||||
@@ -62,6 +62,10 @@ class Playlist:
|
|||||||
# for column widths or sort order — the conflict merge uses it to decide
|
# for column widths or sort order — the conflict merge uses it to decide
|
||||||
# whose order wins, and the file's mtime moves for cosmetic reasons.
|
# whose order wins, and the file's mtime moves for cosmetic reasons.
|
||||||
date_modified: Optional[str] = None
|
date_modified: Optional[str] = None
|
||||||
|
# {track_id: [when, "add"|"remove"]} for membership changes inside the
|
||||||
|
# retention window. Without it a merge is a union and a song removed on one
|
||||||
|
# machine comes back from the other forever; see lintunes/tombstones.py.
|
||||||
|
track_events: dict = field(default_factory=dict)
|
||||||
# Set only for SMART playlists. Membership (track_ids) is derived from this
|
# Set only for SMART playlists. Membership (track_ids) is derived from this
|
||||||
# by the LibraryManager; for imported playlists track_ids also doubles as a
|
# by the LibraryManager; for imported playlists track_ids also doubles as a
|
||||||
# snapshot fallback when the criteria couldn't be fully parsed.
|
# snapshot fallback when the criteria couldn't be fully parsed.
|
||||||
@@ -99,6 +103,10 @@ class Playlist:
|
|||||||
d["parent_persistent_id"] = self.parent_persistent_id
|
d["parent_persistent_id"] = self.parent_persistent_id
|
||||||
if self.date_modified:
|
if self.date_modified:
|
||||||
d["date_modified"] = self.date_modified
|
d["date_modified"] = self.date_modified
|
||||||
|
if self.track_events:
|
||||||
|
# JSON object keys are strings; track ids are ints everywhere else.
|
||||||
|
d["track_events"] = {str(tid): event
|
||||||
|
for tid, event in self.track_events.items()}
|
||||||
if self.smart_criteria is not None:
|
if self.smart_criteria is not None:
|
||||||
d["smart_criteria"] = self.smart_criteria.to_dict()
|
d["smart_criteria"] = self.smart_criteria.to_dict()
|
||||||
return d
|
return d
|
||||||
@@ -118,6 +126,8 @@ class Playlist:
|
|||||||
settings=PlaylistSettings.from_dict(d.get("settings", {})),
|
settings=PlaylistSettings.from_dict(d.get("settings", {})),
|
||||||
is_system=d.get("is_system", False),
|
is_system=d.get("is_system", False),
|
||||||
date_modified=d.get("date_modified"),
|
date_modified=d.get("date_modified"),
|
||||||
|
track_events={int(tid): list(event)
|
||||||
|
for tid, event in (d.get("track_events") or {}).items()},
|
||||||
smart_criteria=smart_criteria,
|
smart_criteria=smart_criteria,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ from dataclasses import dataclass, field
|
|||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from lintunes import sync_identity
|
from lintunes import sync_identity, tombstones
|
||||||
from lintunes.storage.json_storage import read_json, write_json
|
from lintunes.storage.json_storage import read_json, write_json
|
||||||
|
|
||||||
|
|
||||||
@@ -96,6 +96,11 @@ def resolve_conflicts(data_dir: Path) -> list[ConflictSummary]:
|
|||||||
conflict_files = _find_conflict_files(data_dir)
|
conflict_files = _find_conflict_files(data_dir)
|
||||||
if not conflict_files:
|
if not conflict_files:
|
||||||
return []
|
return []
|
||||||
|
# library_metadata.json carries the record of what was deleted, and
|
||||||
|
# library.json is merged against it — so it has to be merged first, whatever
|
||||||
|
# order rglob happened to hand them back in.
|
||||||
|
conflict_files.sort(
|
||||||
|
key=lambda item: 0 if item.original_path.name == "library_metadata.json" else 1)
|
||||||
|
|
||||||
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
|
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
|
||||||
backup_dir = data_dir / ".resolved" / stamp
|
backup_dir = data_dir / ".resolved" / stamp
|
||||||
@@ -120,7 +125,8 @@ def resolve_conflicts(data_dir: Path) -> list[ConflictSummary]:
|
|||||||
"the other copy is in the backup."],
|
"the other copy is in the backup."],
|
||||||
level=WARNING)
|
level=WARNING)
|
||||||
elif original_path.name == "library.json":
|
elif original_path.name == "library.json":
|
||||||
summary = _merge_library(original_path, conflict_path)
|
summary = _merge_library(original_path, conflict_path,
|
||||||
|
data_dir, label)
|
||||||
elif original_path.name == "library_metadata.json":
|
elif original_path.name == "library_metadata.json":
|
||||||
summary = _merge_metadata(original_path, conflict_path)
|
summary = _merge_metadata(original_path, conflict_path)
|
||||||
elif original_path.parent.name == "playlists":
|
elif original_path.parent.name == "playlists":
|
||||||
@@ -238,14 +244,18 @@ def _track_labels(data_dir: Path) -> dict[str, str]:
|
|||||||
labels = {}
|
labels = {}
|
||||||
try:
|
try:
|
||||||
for tid, track in read_json(data_dir / "library.json").items():
|
for tid, track in read_json(data_dir / "library.json").items():
|
||||||
title = track.get("name") or f"track {tid}"
|
labels[str(tid)] = _label_for(tid, track)
|
||||||
artist = track.get("artist") or track.get("album_artist")
|
|
||||||
labels[str(tid)] = f"{artist} — {title}" if artist else title
|
|
||||||
except Exception:
|
except Exception:
|
||||||
pass # no library yet, or an unreadable one: fall back to bare ids
|
pass # no library yet, or an unreadable one: fall back to bare ids
|
||||||
return labels
|
return labels
|
||||||
|
|
||||||
|
|
||||||
|
def _label_for(tid, track: dict) -> str:
|
||||||
|
title = track.get("name") or f"track {tid}"
|
||||||
|
artist = track.get("artist") or track.get("album_artist")
|
||||||
|
return f"{artist} — {title}" if artist else title
|
||||||
|
|
||||||
|
|
||||||
class _Labeler:
|
class _Labeler:
|
||||||
"""Names tracks for the report, loading library.json only if asked."""
|
"""Names tracks for the report, loading library.json only if asked."""
|
||||||
|
|
||||||
@@ -258,6 +268,13 @@ class _Labeler:
|
|||||||
self._labels = _track_labels(self._data_dir)
|
self._labels = _track_labels(self._data_dir)
|
||||||
return self._labels.get(str(tid)) or f"track {tid}"
|
return self._labels.get(str(tid)) or f"track {tid}"
|
||||||
|
|
||||||
|
def remember(self, tid, track: dict):
|
||||||
|
"""Keep a name that is about to be deleted from library.json — the
|
||||||
|
playlist merge still has to say which song it dropped."""
|
||||||
|
if self._labels is None:
|
||||||
|
self._labels = _track_labels(self._data_dir)
|
||||||
|
self._labels.setdefault(str(tid), _label_for(tid, track))
|
||||||
|
|
||||||
|
|
||||||
def _local_time(stamp: str | None) -> str:
|
def _local_time(stamp: str | None) -> str:
|
||||||
"""A naive-UTC ISO stamp as local wall-clock, the way the window shows it."""
|
"""A naive-UTC ISO stamp as local wall-clock, the way the window shows it."""
|
||||||
@@ -278,14 +295,28 @@ def _utc_now_iso() -> str:
|
|||||||
return datetime.now(timezone.utc).replace(tzinfo=None).isoformat()
|
return datetime.now(timezone.utc).replace(tzinfo=None).isoformat()
|
||||||
|
|
||||||
|
|
||||||
def _merge_library(original_path: Path, conflict_path: Path) -> ConflictSummary:
|
def _merge_library(original_path: Path, conflict_path: Path,
|
||||||
|
data_dir: Path = None, label=None) -> ConflictSummary:
|
||||||
original = read_json(original_path)
|
original = read_json(original_path)
|
||||||
conflict = read_json(conflict_path)
|
conflict = read_json(conflict_path)
|
||||||
|
# Deliberate deletions, from the file merged just before this one. A union
|
||||||
|
# without them means a track deleted on either machine is handed back by
|
||||||
|
# whichever copy hadn't heard yet — every time, forever.
|
||||||
|
deleted = _deleted_tracks(data_dir)
|
||||||
|
|
||||||
added = 0
|
added = 0
|
||||||
changed = 0
|
changed = 0
|
||||||
|
removed = 0
|
||||||
examples: list[str] = []
|
examples: list[str] = []
|
||||||
|
for tid in list(original):
|
||||||
|
if tid in deleted:
|
||||||
|
if label is not None:
|
||||||
|
label.remember(tid, original[tid])
|
||||||
|
del original[tid]
|
||||||
|
removed += 1
|
||||||
for tid, conflict_track_data in conflict.items():
|
for tid, conflict_track_data in conflict.items():
|
||||||
|
if tid in deleted:
|
||||||
|
continue
|
||||||
if tid not in original:
|
if tid not in original:
|
||||||
original[tid] = conflict_track_data
|
original[tid] = conflict_track_data
|
||||||
added += 1
|
added += 1
|
||||||
@@ -305,11 +336,29 @@ def _merge_library(original_path: Path, conflict_path: Path) -> ConflictSummary:
|
|||||||
"(play counts kept highest, newest edits win).")
|
"(play counts kept highest, newest edits win).")
|
||||||
if added:
|
if added:
|
||||||
lines.append(f"{added} track(s) present only on the other machine were kept.")
|
lines.append(f"{added} track(s) present only on the other machine were kept.")
|
||||||
|
if removed:
|
||||||
|
lines.append(f"{_count(removed)} deleted from the library stayed deleted, "
|
||||||
|
"though the other copy still listed "
|
||||||
|
f"{'it' if removed == 1 else 'them'}. "
|
||||||
|
"No file on this machine was touched.")
|
||||||
if not lines:
|
if not lines:
|
||||||
lines.append("No differences needed reconciling.")
|
lines.append("No differences needed reconciling.")
|
||||||
lines.extend(examples)
|
lines.extend(examples)
|
||||||
return ConflictSummary("library.json", "library", lines,
|
return ConflictSummary(
|
||||||
level=CHANGE if (changed or added) else INFO)
|
"library.json", "library", lines,
|
||||||
|
level=WARNING if removed else (CHANGE if (changed or added) else INFO))
|
||||||
|
|
||||||
|
|
||||||
|
def _deleted_tracks(data_dir: Path | None) -> dict:
|
||||||
|
"""The library's deletion record, keyed the way library.json is (strings)."""
|
||||||
|
if data_dir is None:
|
||||||
|
return {}
|
||||||
|
try:
|
||||||
|
metadata = read_json(data_dir / "library_metadata.json")
|
||||||
|
except (ValueError, OSError):
|
||||||
|
return {}
|
||||||
|
return {str(tid): when
|
||||||
|
for tid, when in (metadata.get("deleted_tracks") or {}).items()}
|
||||||
|
|
||||||
|
|
||||||
# Every field _merge_track_fields reads or writes. If two copies of a track
|
# Every field _merge_track_fields reads or writes. If two copies of a track
|
||||||
@@ -443,6 +492,12 @@ def _playlist_newer(original: dict, conflict: dict,
|
|||||||
"timestamps instead.")
|
"timestamps instead.")
|
||||||
|
|
||||||
|
|
||||||
|
def _events(playlist: dict) -> dict:
|
||||||
|
"""A playlist file's membership events, with ids back in int form."""
|
||||||
|
return {int(tid): event
|
||||||
|
for tid, event in (playlist.get("track_events") or {}).items()}
|
||||||
|
|
||||||
|
|
||||||
def _positions(merged: list, tids, label) -> list[str]:
|
def _positions(merged: list, tids, label) -> list[str]:
|
||||||
""""Artist — Title → position 12, after “…”" for each of `tids`."""
|
""""Artist — Title → position 12, after “…”" for each of `tids`."""
|
||||||
index = {}
|
index = {}
|
||||||
@@ -531,6 +586,17 @@ def _merge_playlist(original_path: Path, conflict_path: Path,
|
|||||||
merged = merge_track_order(conf_ids, orig_ids)
|
merged = merge_track_order(conf_ids, orig_ids)
|
||||||
else:
|
else:
|
||||||
merged = merge_track_order(orig_ids, conf_ids)
|
merged = merge_track_order(orig_ids, conf_ids)
|
||||||
|
|
||||||
|
# A union alone can never honor a removal: with no common ancestor, "they
|
||||||
|
# removed it" and "we added it" look identical from the two lists. The
|
||||||
|
# newest recorded event per track breaks the tie — a removal beats a copy
|
||||||
|
# that merely still had the song, and a later re-add beats the removal.
|
||||||
|
events = tombstones.merge_events(_events(original), _events(conflict))
|
||||||
|
gone = tombstones.removed_ids(events) & set(merged)
|
||||||
|
merged = [tid for tid in merged if tid not in gone]
|
||||||
|
if events:
|
||||||
|
original["track_events"] = {str(tid): event
|
||||||
|
for tid, event in events.items()}
|
||||||
original["track_ids"] = merged
|
original["track_ids"] = merged
|
||||||
|
|
||||||
renamed_from = ""
|
renamed_from = ""
|
||||||
@@ -567,9 +633,10 @@ def _merge_playlist(original_path: Path, conflict_path: Path,
|
|||||||
# merge made the copy in place harder to beat on the next one.
|
# merge made the copy in place harder to beat on the next one.
|
||||||
write_json(original_path, original)
|
write_json(original_path, original)
|
||||||
|
|
||||||
added = [tid for tid in conf_set - orig_set]
|
added = [tid for tid in conf_set - orig_set if tid not in gone]
|
||||||
only_here = [tid for tid in orig_set - conf_set]
|
only_here = [tid for tid in orig_set - conf_set if tid not in gone]
|
||||||
if orig_ids == conf_ids:
|
removed = sorted(gone)
|
||||||
|
if orig_ids == conf_ids and not gone:
|
||||||
# The two copies held the same tracks in the same order, so they differed
|
# The two copies held the same tracks in the same order, so they differed
|
||||||
# only in column widths or sort order — a window resize, not an edit.
|
# only in column widths or sort order — a window resize, not an edit.
|
||||||
# Note this is a test on the *inputs*: a merge whose result happens to
|
# Note this is a test on the *inputs*: a merge whose result happens to
|
||||||
@@ -610,19 +677,34 @@ def _merge_playlist(original_path: Path, conflict_path: Path,
|
|||||||
# no way to tell which. Both readings are offered rather than guessed.
|
# no way to tell which. Both readings are offered rather than guessed.
|
||||||
headline = (f"{_count(len(only_here))} here that the other copy did "
|
headline = (f"{_count(len(only_here))} here that the other copy did "
|
||||||
f"not have {'was' if len(only_here) == 1 else 'were'} "
|
f"not have {'was' if len(only_here) == 1 else 'were'} "
|
||||||
"kept — a merge never removes:")
|
"kept — nothing recorded a removal:")
|
||||||
it, they, them = (("it", "it", "it") if len(only_here) == 1
|
it, they = ("it", "it") if len(only_here) == 1 else ("them", "they")
|
||||||
else ("them", "they", "them"))
|
note = (f"Most likely you added {it} here. A removal made on the other "
|
||||||
note = (f"If you added {it} here, that's all this is. If you deleted "
|
f"machine carries across on its own now, so {they} would have "
|
||||||
f"{it} on the other machine, delete {them} here too or "
|
"gone already — unless it happened before that machine "
|
||||||
f"{they} will come back on the next merge.")
|
f"updated, or longer ago than {tombstones.RETENTION_DAYS} days.")
|
||||||
lines.append(headline)
|
lines.append(headline)
|
||||||
lines.extend(_capped(entries))
|
lines.extend(_capped(entries))
|
||||||
lines.append(note)
|
lines.append(note)
|
||||||
detail.append(headline)
|
detail.append(headline)
|
||||||
detail.extend(f" • {entry}" for entry in entries)
|
detail.extend(f" • {entry}" for entry in entries)
|
||||||
detail.append(note)
|
detail.append(note)
|
||||||
return ConflictSummary(name, "playlist", lines, level=CHANGE, detail=detail)
|
if removed:
|
||||||
|
entries = [label(tid) for tid in removed]
|
||||||
|
headline = (f"{_count(len(removed))} removed from this playlist on the "
|
||||||
|
"other machine "
|
||||||
|
f"{'was' if len(removed) == 1 else 'were'} removed here too:")
|
||||||
|
note = ("Undo (Ctrl+Z) does not reach a merge — add "
|
||||||
|
f"{'it' if len(removed) == 1 else 'them'} back from the library "
|
||||||
|
"if this wasn't what you wanted. No file was touched.")
|
||||||
|
lines.append(headline)
|
||||||
|
lines.extend(_capped(entries))
|
||||||
|
lines.append(note)
|
||||||
|
detail.append(headline)
|
||||||
|
detail.extend(f" • {entry}" for entry in entries)
|
||||||
|
detail.append(note)
|
||||||
|
return ConflictSummary(name, "playlist", lines,
|
||||||
|
level=WARNING if removed else CHANGE, detail=detail)
|
||||||
|
|
||||||
|
|
||||||
def _merge_play_journal(original_path: Path, conflict_path: Path) -> ConflictSummary:
|
def _merge_play_journal(original_path: Path, conflict_path: Path) -> ConflictSummary:
|
||||||
@@ -666,6 +748,7 @@ def _merge_play_journal(original_path: Path, conflict_path: Path) -> ConflictSum
|
|||||||
# What library_metadata.json actually holds, in words a person recognizes.
|
# What library_metadata.json actually holds, in words a person recognizes.
|
||||||
_METADATA_LABELS = {
|
_METADATA_LABELS = {
|
||||||
"library_settings": "column layout",
|
"library_settings": "column layout",
|
||||||
|
"deleted_tracks": "record of deleted songs",
|
||||||
"music_folder": "music folder",
|
"music_folder": "music folder",
|
||||||
"music_folder_rel": "music folder",
|
"music_folder_rel": "music folder",
|
||||||
"music_folder_set_at": "music folder",
|
"music_folder_set_at": "music folder",
|
||||||
@@ -701,6 +784,13 @@ def _merge_metadata(original_path: Path, conflict_path: Path) -> ConflictSummary
|
|||||||
# machine can silently revert a folder change made on the other.
|
# machine can silently revert a folder change made on the other.
|
||||||
if before is not None and other is not None:
|
if before is not None and other is not None:
|
||||||
kept = read_json(original_path)
|
kept = read_json(original_path)
|
||||||
|
# Deletions are additive: whichever whole copy the mtime pick kept, the
|
||||||
|
# other one's record of what was deleted is still true.
|
||||||
|
union = tombstones.merge_events(before.get("deleted_tracks") or {},
|
||||||
|
other.get("deleted_tracks") or {})
|
||||||
|
if union != (kept.get("deleted_tracks") or {}):
|
||||||
|
kept["deleted_tracks"] = union
|
||||||
|
write_json(original_path, kept)
|
||||||
# Decide the folder from the two *inputs*, never from whichever copy the
|
# Decide the folder from the two *inputs*, never from whichever copy the
|
||||||
# mtime pick above landed on: when both files are written in the same
|
# mtime pick above landed on: when both files are written in the same
|
||||||
# instant that pick is a coin flip, and it was deciding a deliberate
|
# instant that pick is a coin flip, and it was deciding a deliberate
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import json
|
import json
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
|
from lintunes import tombstones
|
||||||
from lintunes.models import Track, Playlist, PlaylistSettings, Library
|
from lintunes.models import Track, Playlist, PlaylistSettings, Library
|
||||||
from lintunes.paths import to_relative, to_absolute
|
from lintunes.paths import to_relative, to_absolute
|
||||||
|
|
||||||
@@ -66,6 +67,10 @@ def save_metadata(library: Library, data_dir: Path):
|
|||||||
"track_count": len(library.tracks),
|
"track_count": len(library.tracks),
|
||||||
"playlist_count": len(library.playlists),
|
"playlist_count": len(library.playlists),
|
||||||
"library_settings": library.library_settings.to_dict(),
|
"library_settings": library.library_settings.to_dict(),
|
||||||
|
# Pruned at the boundary, like the derived-membership gate below: the
|
||||||
|
# in-memory dict stays honest, the file doesn't grow forever.
|
||||||
|
"deleted_tracks": {str(tid): when for tid, when
|
||||||
|
in tombstones.prune(library.deleted_tracks).items()},
|
||||||
}
|
}
|
||||||
write_json(data_dir / "library_metadata.json", metadata)
|
write_json(data_dir / "library_metadata.json", metadata)
|
||||||
|
|
||||||
@@ -74,6 +79,10 @@ def save_playlist(playlist: Playlist, data_dir: Path):
|
|||||||
playlists_dir = data_dir / "playlists"
|
playlists_dir = data_dir / "playlists"
|
||||||
playlists_dir.mkdir(parents=True, exist_ok=True)
|
playlists_dir.mkdir(parents=True, exist_ok=True)
|
||||||
data = playlist.to_dict()
|
data = playlist.to_dict()
|
||||||
|
if data.get("track_events"):
|
||||||
|
data["track_events"] = tombstones.prune(data["track_events"])
|
||||||
|
if not data["track_events"]:
|
||||||
|
del data["track_events"]
|
||||||
if playlist.has_derived_membership:
|
if playlist.has_derived_membership:
|
||||||
# Membership is a pure function of the criteria and the library, and
|
# Membership is a pure function of the criteria and the library, and
|
||||||
# every machine recomputes it on load — storing it only bought a
|
# every machine recomputes it on load — storing it only bought a
|
||||||
@@ -113,6 +122,7 @@ def load_library(data_dir: Path) -> Library:
|
|||||||
music_folder_rel = ""
|
music_folder_rel = ""
|
||||||
music_folder_set_at = None
|
music_folder_set_at = None
|
||||||
import_date = None
|
import_date = None
|
||||||
|
deleted_tracks = {}
|
||||||
library_settings = PlaylistSettings()
|
library_settings = PlaylistSettings()
|
||||||
metadata_path = data_dir / "library_metadata.json"
|
metadata_path = data_dir / "library_metadata.json"
|
||||||
if metadata_path.exists():
|
if metadata_path.exists():
|
||||||
@@ -121,6 +131,8 @@ def load_library(data_dir: Path) -> Library:
|
|||||||
music_folder_rel = metadata.get("music_folder_rel", "")
|
music_folder_rel = metadata.get("music_folder_rel", "")
|
||||||
music_folder_set_at = metadata.get("music_folder_set_at")
|
music_folder_set_at = metadata.get("music_folder_set_at")
|
||||||
import_date = metadata.get("import_date")
|
import_date = metadata.get("import_date")
|
||||||
|
deleted_tracks = {int(tid): when for tid, when
|
||||||
|
in (metadata.get("deleted_tracks") or {}).items()}
|
||||||
if "library_settings" in metadata:
|
if "library_settings" in metadata:
|
||||||
library_settings = PlaylistSettings.from_dict(metadata["library_settings"])
|
library_settings = PlaylistSettings.from_dict(metadata["library_settings"])
|
||||||
|
|
||||||
@@ -133,9 +145,22 @@ def load_library(data_dir: Path) -> Library:
|
|||||||
playlist = Playlist.from_dict(playlist_data)
|
playlist = Playlist.from_dict(playlist_data)
|
||||||
playlists[playlist.persistent_id] = playlist
|
playlists[playlist.persistent_id] = playlist
|
||||||
|
|
||||||
|
# A deleted track can still be sitting in a library.json that synced in
|
||||||
|
# from a machine which hasn't heard about the deletion yet, and in the
|
||||||
|
# track_ids of any playlist that held it. Applying the record here covers
|
||||||
|
# every load path at once — startup and the reload_from_disk reconcile.
|
||||||
|
for tid in deleted_tracks:
|
||||||
|
tracks.pop(tid, None)
|
||||||
|
if deleted_tracks:
|
||||||
|
for playlist in playlists.values():
|
||||||
|
if any(tid in deleted_tracks for tid in playlist.track_ids):
|
||||||
|
playlist.track_ids = [tid for tid in playlist.track_ids
|
||||||
|
if tid not in deleted_tracks]
|
||||||
|
|
||||||
return Library(
|
return Library(
|
||||||
tracks=tracks,
|
tracks=tracks,
|
||||||
playlists=playlists,
|
playlists=playlists,
|
||||||
|
deleted_tracks=deleted_tracks,
|
||||||
music_folder=music_folder,
|
music_folder=music_folder,
|
||||||
music_folder_rel=music_folder_rel,
|
music_folder_rel=music_folder_rel,
|
||||||
music_folder_set_at=music_folder_set_at,
|
music_folder_set_at=music_folder_set_at,
|
||||||
|
|||||||
@@ -0,0 +1,89 @@
|
|||||||
|
"""Removals that survive a merge.
|
||||||
|
|
||||||
|
Two copies of a playlist and no common ancestor cannot tell "A added this song"
|
||||||
|
apart from "B deleted it" — which is why every merge until now was a union, and
|
||||||
|
why a song removed on one machine came back from the other on the next sync.
|
||||||
|
A deletion you cannot make stick is not a deletion.
|
||||||
|
|
||||||
|
So the removal is recorded rather than inferred: each playlist keeps a small
|
||||||
|
``track_events`` map of ``{track id: (when, "add"|"remove")}``, and the library
|
||||||
|
keeps ``deleted_tracks``. A merge takes the *newest* event per track across both
|
||||||
|
copies and applies it, so a removal beats a copy that simply still had the song,
|
||||||
|
and a deliberate re-add afterwards beats the removal. Tracks nobody has touched
|
||||||
|
have no event and keep the old union behavior.
|
||||||
|
|
||||||
|
Events are pruned after ``RETENTION_DAYS``: by then every machine has seen them,
|
||||||
|
and keeping them forever would grow the file for nothing. The trade is that a
|
||||||
|
machine offline longer than that can resurrect a song it never learned was
|
||||||
|
deleted — which is the same trade every sync system makes, and the merge window
|
||||||
|
says out loud when a removal is applied.
|
||||||
|
"""
|
||||||
|
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
|
||||||
|
ADD = "add"
|
||||||
|
REMOVE = "remove"
|
||||||
|
|
||||||
|
# How long a membership event is kept. Long enough that a laptop shut in a bag
|
||||||
|
# for a month still learns about the deletion; short enough that the map stays
|
||||||
|
# a handful of entries.
|
||||||
|
RETENTION_DAYS = 30
|
||||||
|
|
||||||
|
|
||||||
|
def now_iso() -> str:
|
||||||
|
"""Naive UTC ISO — the one stamp format every machine writes, so the strings
|
||||||
|
sort correctly against each other without parsing."""
|
||||||
|
return datetime.now(timezone.utc).replace(tzinfo=None).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
def _expiry(now: str | None = None) -> str:
|
||||||
|
base = datetime.fromisoformat(now) if now else datetime.now(timezone.utc).replace(tzinfo=None)
|
||||||
|
return (base - timedelta(days=RETENTION_DAYS)).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
def prune(events: dict, now: str | None = None) -> dict:
|
||||||
|
"""Drop events older than the retention window."""
|
||||||
|
cutoff = _expiry(now)
|
||||||
|
return {key: value for key, value in events.items()
|
||||||
|
if _stamp(value) and _stamp(value) >= cutoff}
|
||||||
|
|
||||||
|
|
||||||
|
def _stamp(value):
|
||||||
|
"""An event is (when, op); a bare stamp means a deletion in deleted_tracks."""
|
||||||
|
if isinstance(value, (list, tuple)):
|
||||||
|
return value[0] if value else None
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _op(value):
|
||||||
|
if isinstance(value, (list, tuple)) and len(value) > 1:
|
||||||
|
return value[1]
|
||||||
|
return REMOVE
|
||||||
|
|
||||||
|
|
||||||
|
def merge_events(ours: dict, theirs: dict) -> dict:
|
||||||
|
"""Newest event per track wins. Works for ``track_events`` (stamp + op) and
|
||||||
|
for ``deleted_tracks`` (bare stamps) alike."""
|
||||||
|
merged = dict(ours or {})
|
||||||
|
for key, value in (theirs or {}).items():
|
||||||
|
mine = merged.get(key)
|
||||||
|
if mine is None or (_stamp(value) or "") > (_stamp(mine) or ""):
|
||||||
|
merged[key] = value
|
||||||
|
return merged
|
||||||
|
|
||||||
|
|
||||||
|
def removed_ids(events: dict) -> set:
|
||||||
|
"""Which tracks the newest event says are gone."""
|
||||||
|
return {key for key, value in (events or {}).items() if _op(value) == REMOVE}
|
||||||
|
|
||||||
|
|
||||||
|
def record(events: dict, before, after) -> dict:
|
||||||
|
"""Note what a membership change did, in place. ``before``/``after`` are the
|
||||||
|
two track_ids lists; a reorder produces no events at all."""
|
||||||
|
was, now = set(before), set(after)
|
||||||
|
stamp = now_iso()
|
||||||
|
for tid in was - now:
|
||||||
|
events[tid] = [stamp, REMOVE]
|
||||||
|
for tid in now - was:
|
||||||
|
events[tid] = [stamp, ADD]
|
||||||
|
return events
|
||||||
@@ -1,5 +1,55 @@
|
|||||||
## Done
|
## Done
|
||||||
|
|
||||||
|
### Round 44 (2026-08-27) — A removal you make sticks (v0.14.0)
|
||||||
|
|
||||||
|
Round 43 made the merge report honest, which made its one real limitation
|
||||||
|
impossible to miss: every merge was a union, so a song removed from a playlist
|
||||||
|
on one machine was handed straight back by the other on the next sync, and a
|
||||||
|
track deleted from the library came back with it. A deletion you cannot make
|
||||||
|
stick is not a deletion.
|
||||||
|
|
||||||
|
- [x] **Removals are recorded, not inferred.** The union was there for a real
|
||||||
|
reason — two copies and no common ancestor cannot tell "A added this" from
|
||||||
|
"B removed it" — so the missing evidence is now written down. New
|
||||||
|
`lintunes/tombstones.py`: `Playlist.track_events` holds
|
||||||
|
`{track id: [when, "add"|"remove"]}` and `Library.deleted_tracks` holds
|
||||||
|
`{track id: when}` in `library_metadata.json`. A merge takes the newest
|
||||||
|
event per track across both copies and applies it. A removal beats a copy
|
||||||
|
that merely still had the song; a deliberate re-add afterwards beats the
|
||||||
|
removal; a track nobody touched still merges as a union, which is the safe
|
||||||
|
old behavior for everything that has no evidence either way.
|
||||||
|
- [x] **Not "the newer copy wins wholesale".** That was the tempting one-line
|
||||||
|
version and it silently drops a song the other machine added while you
|
||||||
|
were removing one. `test_an_unrelated_addition_is_not_lost` pins it.
|
||||||
|
- [x] **Recorded in the existing funnels.** `_set_track_ids` diffs before/after
|
||||||
|
(so a reorder records nothing), `_remove_tracks` stamps
|
||||||
|
`deleted_tracks`, and `_restore_tracks` clears it, so Ctrl+Z on a delete
|
||||||
|
takes the tombstone back with it.
|
||||||
|
- [x] **Track ids are never reused.** The sharpest edge in the whole design: the
|
||||||
|
next id came from `max(library.tracks)`, so deleting the highest-numbered
|
||||||
|
track freed its id for the next import — and that new track would be
|
||||||
|
dropped on sight by the dead id's own tombstone, on every machine.
|
||||||
|
`_highest_track_id` counts the deletions too.
|
||||||
|
- [x] **`library_metadata.json` merges before `library.json`.** It carries the
|
||||||
|
record the library merge filters against, and `rglob` order is not a plan.
|
||||||
|
`_merge_metadata` unions the two copies' `deleted_tracks` regardless of
|
||||||
|
which whole copy the mtime pick kept.
|
||||||
|
- [x] **A merge applying a deletion never touches a music file.** It drops the
|
||||||
|
library entry only — the file was already trashed on the machine where the
|
||||||
|
delete happened, and the music folder is its own Syncthing share.
|
||||||
|
`test_a_merge_never_touches_a_music_file` fails the run if `send_to_trash`
|
||||||
|
is so much as called.
|
||||||
|
- [x] **Reported as a warning, with the song named.** An applied removal grades
|
||||||
|
WARNING so it opens the merge window, names each track, and says Ctrl+Z
|
||||||
|
does not reach a merge. `_Labeler.remember` keeps the name of a track
|
||||||
|
library.json is about to lose, so the playlist merge can still say which
|
||||||
|
song it dropped rather than "track 4".
|
||||||
|
- [x] **Events expire after 30 days** (`RETENTION_DAYS`), pruned at the save
|
||||||
|
boundary like the derived-membership gate. The trade is stated in the
|
||||||
|
module docstring: a machine offline longer than that can resurrect a song
|
||||||
|
it never learned was deleted.
|
||||||
|
|
||||||
|
|
||||||
### Round 43 (2026-08-27) — The merge report says who, what, and where (v0.13.0)
|
### Round 43 (2026-08-27) — The merge report says who, what, and where (v0.13.0)
|
||||||
|
|
||||||
The merge window kept saying `Order kept from this machine (most recently
|
The merge window kept saying `Order kept from this machine (most recently
|
||||||
|
|||||||
@@ -242,7 +242,9 @@ class TestNamingTheSongs:
|
|||||||
body = "\n".join(summaries[0].lines)
|
body = "\n".join(summaries[0].lines)
|
||||||
assert "the other copy did not have" in body
|
assert "the other copy did not have" in body
|
||||||
assert "Pola — Abeille" in body
|
assert "Pola — Abeille" in body
|
||||||
assert "delete it here too" in body
|
# Round 44: a removal now carries across on its own, so the note no
|
||||||
|
# longer tells you to delete it here by hand.
|
||||||
|
assert "Most likely you added it here" in body
|
||||||
|
|
||||||
def test_the_window_caps_the_list_and_the_backup_holds_all_of_it(self, tmp_path):
|
def test_the_window_caps_the_list_and_the_backup_holds_all_of_it(self, tmp_path):
|
||||||
from lintunes.models import Track
|
from lintunes.models import Track
|
||||||
|
|||||||
@@ -0,0 +1,247 @@
|
|||||||
|
"""Round 44: a removal you make sticks.
|
||||||
|
|
||||||
|
Round 43 made the merge report honest, and in doing so made its one real
|
||||||
|
limitation impossible to miss: every merge was a *union*, so a song removed from
|
||||||
|
a playlist on one machine was handed straight back by the other one on the next
|
||||||
|
sync, forever. Same for a track deleted from the library. A deletion you cannot
|
||||||
|
make stick is not a deletion.
|
||||||
|
|
||||||
|
The reason it was a union is real — two copies and no common ancestor cannot
|
||||||
|
tell "A added this" from "B removed it". So the removal is now *recorded*
|
||||||
|
(`lintunes/tombstones.py`): a playlist keeps `track_events`, the library keeps
|
||||||
|
`deleted_tracks`, and a merge applies the newest event per track. A removal
|
||||||
|
beats a copy that merely still had the song; a deliberate re-add afterwards
|
||||||
|
beats the removal; a track nobody has touched still merges as a union.
|
||||||
|
"""
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from lintunes import tombstones
|
||||||
|
from lintunes.library_manager import LibraryManager
|
||||||
|
from lintunes.models import Library, Playlist, PlaylistType, Track
|
||||||
|
from lintunes.storage import json_storage, conflict_resolver
|
||||||
|
from lintunes.storage.conflict_resolver import WARNING, CHANGE
|
||||||
|
|
||||||
|
|
||||||
|
PID = "AAAA1111"
|
||||||
|
|
||||||
|
|
||||||
|
def _machine(tmp_path, name, ids=(1, 2, 3)):
|
||||||
|
"""A data dir with three tracks and one playlist holding them."""
|
||||||
|
root = tmp_path / name
|
||||||
|
lib = Library()
|
||||||
|
for tid in (1, 2, 3, 4):
|
||||||
|
lib.tracks[tid] = Track(track_id=tid, name=f"Song {tid}", artist="A")
|
||||||
|
lib.playlists[PID] = Playlist(persistent_id=PID, name="mix",
|
||||||
|
playlist_type=PlaylistType.REGULAR,
|
||||||
|
track_ids=list(ids))
|
||||||
|
json_storage.save_library(lib, root)
|
||||||
|
return root
|
||||||
|
|
||||||
|
|
||||||
|
def _manager(root, qapp):
|
||||||
|
return LibraryManager(json_storage.load_library(root), root)
|
||||||
|
|
||||||
|
|
||||||
|
def _sync_as_conflict(src, dest, rel="playlists/AAAA1111.json"):
|
||||||
|
"""What Syncthing does when both copies changed: src's version lands beside
|
||||||
|
dest's own, under a conflict name."""
|
||||||
|
source = src / rel
|
||||||
|
name = f"{source.stem}.sync-conflict-20260827-120000-CMFNCIX{source.suffix}"
|
||||||
|
shutil.copy2(source, dest / rel.rsplit("/", 1)[0] / name)
|
||||||
|
# library_metadata.json rides along — it carries the deletion record.
|
||||||
|
meta = src / "library_metadata.json"
|
||||||
|
if meta.exists():
|
||||||
|
shutil.copy2(meta, dest / "library_metadata.sync-conflict-"
|
||||||
|
"20260827-120000-CMFNCIX.json")
|
||||||
|
return conflict_resolver.resolve_conflicts(dest)
|
||||||
|
|
||||||
|
|
||||||
|
def _ids(root, pid=PID):
|
||||||
|
return json.loads((root / "playlists" / f"{pid}.json").read_text())["track_ids"]
|
||||||
|
|
||||||
|
|
||||||
|
class TestARemovalSticks:
|
||||||
|
def test_it_survives_the_round_trip(self, tmp_path, qapp):
|
||||||
|
"""The whole complaint: remove a song here, go to the other machine, and
|
||||||
|
it is back. Twice over — the second sync used to re-add it again."""
|
||||||
|
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
|
||||||
|
manager = _manager(a, qapp)
|
||||||
|
manager.remove_tracks_from_playlist(PID, [1]) # row index 1 == track 2
|
||||||
|
manager.flush()
|
||||||
|
assert _ids(a) == [1, 3]
|
||||||
|
|
||||||
|
_sync_as_conflict(a, b)
|
||||||
|
assert _ids(b) == [1, 3]
|
||||||
|
|
||||||
|
# ...and B's copy, now travelling back, must not resurrect it either.
|
||||||
|
_sync_as_conflict(b, a)
|
||||||
|
assert _ids(a) == [1, 3]
|
||||||
|
|
||||||
|
def test_a_deliberate_re_add_beats_the_removal(self, tmp_path, qapp):
|
||||||
|
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
|
||||||
|
manager_a = _manager(a, qapp)
|
||||||
|
manager_a.remove_tracks_from_playlist(PID, [1])
|
||||||
|
manager_a.flush()
|
||||||
|
_sync_as_conflict(a, b)
|
||||||
|
assert _ids(b) == [1, 3]
|
||||||
|
|
||||||
|
manager_b = _manager(b, qapp)
|
||||||
|
manager_b.add_tracks_to_playlist(PID, [2])
|
||||||
|
manager_b.flush()
|
||||||
|
|
||||||
|
_sync_as_conflict(b, a)
|
||||||
|
assert 2 in _ids(a)
|
||||||
|
|
||||||
|
def test_an_unrelated_addition_is_not_lost(self, tmp_path, qapp):
|
||||||
|
"""The reason this isn't just "the newer copy wins wholesale": that would
|
||||||
|
drop a song the other machine added while we were removing one."""
|
||||||
|
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
|
||||||
|
manager_a = _manager(a, qapp)
|
||||||
|
manager_a.remove_tracks_from_playlist(PID, [1])
|
||||||
|
manager_a.flush()
|
||||||
|
|
||||||
|
manager_b = _manager(b, qapp)
|
||||||
|
manager_b.add_tracks_to_playlist(PID, [4])
|
||||||
|
manager_b.flush()
|
||||||
|
|
||||||
|
_sync_as_conflict(a, b)
|
||||||
|
|
||||||
|
assert _ids(b) == [1, 3, 4]
|
||||||
|
|
||||||
|
def test_a_track_nobody_touched_still_merges_as_a_union(self, tmp_path, qapp):
|
||||||
|
"""No event means no evidence, and the old behavior is still the safe
|
||||||
|
one: keep it."""
|
||||||
|
a = _machine(tmp_path, "a", ids=[1, 2])
|
||||||
|
b = _machine(tmp_path, "b", ids=[1, 2, 3])
|
||||||
|
|
||||||
|
_sync_as_conflict(b, a)
|
||||||
|
|
||||||
|
assert set(_ids(a)) == {1, 2, 3}
|
||||||
|
|
||||||
|
def test_the_removal_is_reported_as_a_warning(self, tmp_path, qapp):
|
||||||
|
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
|
||||||
|
manager = _manager(a, qapp)
|
||||||
|
manager.remove_tracks_from_playlist(PID, [1])
|
||||||
|
manager.flush()
|
||||||
|
|
||||||
|
summaries = _sync_as_conflict(a, b)
|
||||||
|
|
||||||
|
playlist = [s for s in summaries if s.kind == "playlist"][0]
|
||||||
|
assert playlist.level == WARNING
|
||||||
|
body = "\n".join(playlist.lines)
|
||||||
|
assert "removed here too" in body
|
||||||
|
assert "A — Song 2" in body
|
||||||
|
assert "No file was touched" in body
|
||||||
|
|
||||||
|
|
||||||
|
class TestDeletingFromTheLibrary:
|
||||||
|
def test_a_deleted_track_stays_deleted_through_a_merge(self, tmp_path, qapp):
|
||||||
|
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
|
||||||
|
manager = _manager(a, qapp)
|
||||||
|
removed, failures = manager.delete_tracks([2], delete_files=False)
|
||||||
|
manager.flush()
|
||||||
|
assert removed == [2] and not failures
|
||||||
|
|
||||||
|
summaries = _sync_as_conflict(a, b)
|
||||||
|
# b's library.json still lists track 2; the deletion record must win.
|
||||||
|
shutil.copy2(a / "library.json",
|
||||||
|
b / "library.sync-conflict-20260827-120000-CMFNCIX.json")
|
||||||
|
summaries += conflict_resolver.resolve_conflicts(b)
|
||||||
|
|
||||||
|
assert "2" not in json.loads((b / "library.json").read_text())
|
||||||
|
assert 2 not in _ids(b)
|
||||||
|
library = [s for s in summaries if s.kind == "library"]
|
||||||
|
if library: # only when library.json actually conflicted
|
||||||
|
assert library[0].level == WARNING
|
||||||
|
|
||||||
|
def test_load_drops_a_deleted_track_and_its_playlist_rows(self, tmp_path):
|
||||||
|
"""The clean-sync path: the other machine's library.json arrives with no
|
||||||
|
conflict at all, still listing the track."""
|
||||||
|
root = _machine(tmp_path, "a")
|
||||||
|
metadata = json.loads((root / "library_metadata.json").read_text())
|
||||||
|
metadata["deleted_tracks"] = {"2": tombstones.now_iso()}
|
||||||
|
(root / "library_metadata.json").write_text(json.dumps(metadata))
|
||||||
|
|
||||||
|
library = json_storage.load_library(root)
|
||||||
|
|
||||||
|
assert 2 not in library.tracks
|
||||||
|
assert library.playlists[PID].track_ids == [1, 3]
|
||||||
|
|
||||||
|
def test_undo_takes_the_deletion_back(self, tmp_path, qapp):
|
||||||
|
root = _machine(tmp_path, "a")
|
||||||
|
manager = _manager(root, qapp)
|
||||||
|
manager.delete_tracks([2], delete_files=False)
|
||||||
|
assert 2 in manager.library.deleted_tracks
|
||||||
|
|
||||||
|
manager.undo_stack.undo()
|
||||||
|
|
||||||
|
assert 2 not in manager.library.deleted_tracks
|
||||||
|
assert 2 in manager.library.tracks
|
||||||
|
|
||||||
|
def test_a_merge_never_touches_a_music_file(self, tmp_path, qapp, monkeypatch):
|
||||||
|
"""A deletion syncing in removes the library entry only. The file was
|
||||||
|
already trashed on the machine where the delete happened, and the music
|
||||||
|
folder is its own Syncthing share."""
|
||||||
|
a, b = _machine(tmp_path, "a"), _machine(tmp_path, "b")
|
||||||
|
manager = _manager(a, qapp)
|
||||||
|
manager.delete_tracks([2], delete_files=False)
|
||||||
|
manager.flush()
|
||||||
|
|
||||||
|
from lintunes import trash
|
||||||
|
monkeypatch.setattr(trash, "send_to_trash", lambda *args: pytest.fail(
|
||||||
|
"a merge must never touch a music file"))
|
||||||
|
_sync_as_conflict(a, b)
|
||||||
|
|
||||||
|
|
||||||
|
class TestTheRecordItself:
|
||||||
|
def test_a_reorder_records_nothing(self, tmp_path, qapp):
|
||||||
|
"""Only membership changes are events; moving a row is not a removal."""
|
||||||
|
root = _machine(tmp_path, "a")
|
||||||
|
manager = _manager(root, qapp)
|
||||||
|
manager.move_tracks_in_playlist(PID, [2], 0)
|
||||||
|
assert manager.library.playlists[PID].track_events == {}
|
||||||
|
|
||||||
|
def test_events_expire_so_the_file_does_not_grow_forever(self, tmp_path):
|
||||||
|
old = "2020-01-01T00:00:00"
|
||||||
|
fresh = tombstones.now_iso()
|
||||||
|
kept = tombstones.prune({1: [old, "remove"], 2: [fresh, "remove"]})
|
||||||
|
assert kept == {2: [fresh, "remove"]}
|
||||||
|
|
||||||
|
def test_the_newest_event_per_track_wins_a_merge(self):
|
||||||
|
merged = tombstones.merge_events(
|
||||||
|
{1: ["2026-08-01T00:00:00", "remove"], 2: ["2026-08-01T00:00:00", "add"]},
|
||||||
|
{1: ["2026-08-02T00:00:00", "add"]})
|
||||||
|
assert merged[1] == ["2026-08-02T00:00:00", "add"]
|
||||||
|
assert tombstones.removed_ids(merged) == set()
|
||||||
|
|
||||||
|
def test_they_round_trip_through_the_playlist_file(self, tmp_path):
|
||||||
|
root = tmp_path / "a"
|
||||||
|
lib = Library()
|
||||||
|
playlist = Playlist(persistent_id=PID, playlist_type=PlaylistType.REGULAR,
|
||||||
|
track_ids=[1])
|
||||||
|
playlist.track_events = {2: [tombstones.now_iso(), "remove"]}
|
||||||
|
lib.playlists[PID] = playlist
|
||||||
|
json_storage.save_library(lib, root)
|
||||||
|
|
||||||
|
loaded = json_storage.load_library(root)
|
||||||
|
|
||||||
|
assert tombstones.removed_ids(loaded.playlists[PID].track_events) == {2}
|
||||||
|
|
||||||
|
|
||||||
|
class TestIdsAreNeverReused:
|
||||||
|
def test_a_new_track_does_not_inherit_a_deleted_ones_id(self, tmp_path, qapp):
|
||||||
|
"""The nastiest failure this design could produce: hand a fresh track
|
||||||
|
the id of a deleted one and every machine drops it on sight, its own
|
||||||
|
tombstone having outlived it."""
|
||||||
|
root = _machine(tmp_path, "a")
|
||||||
|
manager = _manager(root, qapp)
|
||||||
|
manager.delete_tracks([4], delete_files=False) # 4 was the highest id
|
||||||
|
manager.flush()
|
||||||
|
|
||||||
|
reopened = _manager(root, qapp)
|
||||||
|
|
||||||
|
assert reopened.new_track_id() > 4
|
||||||
|
assert 4 in reopened.library.deleted_tracks
|
||||||
Reference in New Issue
Block a user