Initial signal-cal-bot skeleton: iCalendar feed to Signal group
- feed fetch with ETag/Last-Modified caching - iCalendar parsing/expansion with TEC series detection - posting-window scheduler with 3-day reminders and day-of fallback - image attachments from ATTACH, downscaled via Pillow - SQLite outbox with idempotent deliveries - watchdog alerts to Note to Self - Docker Compose with signal-cli-rest-api
This commit is contained in:
+23
@@ -0,0 +1,23 @@
|
||||
# iCal feeds / exports — never commit these
|
||||
*.ics
|
||||
*.icsx
|
||||
|
||||
# Runtime state
|
||||
cache/
|
||||
data/
|
||||
signal-data/
|
||||
|
||||
# Python
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
.venv/
|
||||
venv/
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
dist/
|
||||
build/
|
||||
*.egg-info/
|
||||
|
||||
# Local config holds real phone numbers and group IDs
|
||||
config.toml
|
||||
.env
|
||||
+16
@@ -0,0 +1,16 @@
|
||||
FROM python:3.12-slim
|
||||
|
||||
ENV PYTHONUNBUFFERED=1 \
|
||||
PYTHONDONTWRITEBYTECODE=1 \
|
||||
PIP_NO_CACHE_DIR=1
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY pyproject.toml ./
|
||||
COPY src ./src
|
||||
RUN pip install .
|
||||
|
||||
# The compose file runs the bot as uid/gid 1000; these must be writable.
|
||||
RUN mkdir -p /app/data /app/cache && chown -R 1000:1000 /app/data /app/cache
|
||||
|
||||
CMD ["python", "-m", "calbot", "--config", "/app/config.toml"]
|
||||
@@ -0,0 +1,126 @@
|
||||
# calbot — iCalendar → Signal
|
||||
|
||||
Watches an iCalendar feed (built for The Events Calendar / MCR Commons) and posts
|
||||
events to a Signal group, with the event's poster image attached.
|
||||
|
||||
Posts look like this:
|
||||
|
||||
```
|
||||
Folk Night
|
||||
#November #November14
|
||||
4.30-6.30pm
|
||||
Old Print Works B12 9AH
|
||||
https://commonscalendar.uk/event/folk-night/
|
||||
```
|
||||
|
||||
## How it behaves
|
||||
|
||||
- **Announcements** — a newly seen event is posted within one refresh interval,
|
||||
deferred to the next posting window (`08:00–10:00` or `18:00–21:00` by default).
|
||||
- **Reminders** — every event is posted again 3 days before it starts. If the
|
||||
event was added less than 3 days ahead, the reminder goes out on the morning
|
||||
of the event instead. An imminent event breaks the posting window rather than
|
||||
arriving late.
|
||||
- **Images** — taken from the feed's `ATTACH` property (The Events Calendar
|
||||
emits the featured image), falling back to the event page's JSON-LD / `og:image`.
|
||||
Every post carries an image.
|
||||
- **Recurring events** — the feed pre-expands occurrences, but announcements are
|
||||
deduplicated per series, so adding a weekly event produces one announcement.
|
||||
- **First run** — seeds silently (`announce.backfill = false`); reminders still
|
||||
work for everything already in the feed.
|
||||
- **Watchdog** — messages you on Signal if the feed stops working, parses to
|
||||
nothing, or sending keeps failing.
|
||||
|
||||
## Setup
|
||||
|
||||
### 1. Link a Signal account
|
||||
|
||||
Linking as a secondary device is the quickest route: the bot posts **as your own
|
||||
number**. For a separate bot identity you must register a dedicated number
|
||||
instead (see `signal-cli-rest-api` docs).
|
||||
|
||||
```bash
|
||||
mkdir -p signal-data data cache
|
||||
docker compose up -d signal-api
|
||||
```
|
||||
|
||||
Open <http://localhost:8080/v1/qrcodelink?device_name=calbot> and scan the QR
|
||||
code from **Signal → Settings → Linked devices**.
|
||||
|
||||
```bash
|
||||
# the linked number:
|
||||
curl -s http://localhost:8080/v1/accounts
|
||||
|
||||
# groups for that number — copy the internal_id (starts with "group."):
|
||||
curl -s http://localhost:8080/v1/groups/+447700900000 | jq
|
||||
```
|
||||
|
||||
### 2. Configure
|
||||
|
||||
```bash
|
||||
cp config.example.toml config.toml
|
||||
chmod 600 config.toml
|
||||
$EDITOR config.toml
|
||||
```
|
||||
|
||||
Fill in `signal.number`, `signal.group_id`, and `calendar.url`. `webcal://`
|
||||
URLs are accepted and fetched over https.
|
||||
|
||||
### 3. Check, preview, run
|
||||
|
||||
```bash
|
||||
# is the account linked and the config valid?
|
||||
docker compose run --rm bot python -m calbot --check
|
||||
|
||||
# print the rendered posts for the feed without sending anything
|
||||
docker compose run --rm bot python -m calbot --dump
|
||||
|
||||
# start the bot
|
||||
docker compose up -d bot
|
||||
docker compose logs -f bot
|
||||
```
|
||||
|
||||
`--once` runs a single tick and exits.
|
||||
|
||||
## Install by hand
|
||||
|
||||
```bash
|
||||
python -m venv .venv && . .venv/bin/activate
|
||||
pip install -e '.[dev]'
|
||||
calbot --config config.toml --dump
|
||||
calbot --config config.toml
|
||||
```
|
||||
|
||||
## Files that matter
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `config.toml` | numbers, group, feed, behaviour (never committed) |
|
||||
| `signal-data/` | linked-device keys — **back this up**, losing it means re-linking |
|
||||
| `data/calbot.sqlite3` | event/outbox/delivery state (safe to delete to re-seed) |
|
||||
| `cache/images/` | downscaled posters |
|
||||
|
||||
Deleting `data/calbot.sqlite3` resets dedupe state: on the next run everything
|
||||
is seeded silently again, so you will not get a flood, but new-series detection
|
||||
restarts from that moment.
|
||||
|
||||
## Notes and limits
|
||||
|
||||
- Signal has no bot API; this uses [`signal-cli`](https://github.com/AsamK/signal-cli)
|
||||
via [`signal-cli-rest-api`](https://github.com/bbernhard/signal-cli-rest-api).
|
||||
- Delivery is **at-least-once**. A crash between sending and recording can
|
||||
duplicate one post.
|
||||
- The feed is capped by The Events Calendar (30 events by default) and may omit
|
||||
past events; the bot only knows what the feed contains.
|
||||
- Venue/postcode come from the ICS `LOCATION` when present (this feed often has
|
||||
none), then from the event page's structured data, then from the description.
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
pip install -e '.[dev]'
|
||||
pytest
|
||||
```
|
||||
|
||||
Fixtures are stored as `tests/fixtures/*.ics.txt` so that `*.ics` can stay
|
||||
gitignored.
|
||||
@@ -0,0 +1,63 @@
|
||||
# Copy to config.toml and fill in the signal section. Never commit config.toml.
|
||||
|
||||
[signal]
|
||||
# signal-cli-rest-api (from docker-compose.yml)
|
||||
api_url = "http://signal-api:8080"
|
||||
# Your own number (the linked account's number), international format.
|
||||
number = "+447700900000"
|
||||
# internal_id from GET /v1/groups/<number>, including the "group." prefix.
|
||||
group_id = "group.REPLACE_ME"
|
||||
text_mode = "normal" # or "styled"
|
||||
|
||||
[calendar]
|
||||
url = "https://commonscalendar.uk/?post_type=tribe_events&ical=1&eventDisplay=list"
|
||||
timezone = "Europe/London"
|
||||
refresh_minutes = 15
|
||||
window_days = 60
|
||||
|
||||
[posting]
|
||||
# Local times. Posts are deferred to the next window; the urgent guard breaks
|
||||
# the window when an event is about to start.
|
||||
windows = ["08:00-10:00", "18:00-21:00"]
|
||||
urgent_guard_minutes = 60
|
||||
|
||||
[announce]
|
||||
enabled = true
|
||||
max_days_ahead = 30
|
||||
# false = first run seeds silently; set true to announce everything on startup
|
||||
backfill = false
|
||||
|
||||
[reminders]
|
||||
enabled = true
|
||||
lead_times = ["3d"]
|
||||
# If an event is discovered less than the lead time ahead, remind on the day.
|
||||
fallback_to_day_of = true
|
||||
day_of_time = "09:00"
|
||||
|
||||
[images]
|
||||
enabled = true
|
||||
max_pixels = 1600
|
||||
max_bytes = 400000
|
||||
max_images_per_post = 3
|
||||
scrape_event_page = true
|
||||
|
||||
[format]
|
||||
show_title = true
|
||||
show_hashtags = true
|
||||
show_time = true
|
||||
show_location = true
|
||||
show_link = true
|
||||
shorten_links = false
|
||||
|
||||
[watchdog]
|
||||
enabled = true
|
||||
# Defaults to [signal].number ("Note to Self") when unset.
|
||||
# notify_number = "+447700900001"
|
||||
feed_failures = 3
|
||||
send_failures = 3
|
||||
alert_cooldown_hours = 6
|
||||
daily_heartbeat = false
|
||||
|
||||
[storage]
|
||||
db = "data/calbot.sqlite3"
|
||||
cache = "cache"
|
||||
@@ -0,0 +1,32 @@
|
||||
# signal-cli-rest-api, with the native JSON-RPC mode for low latency.
|
||||
services:
|
||||
signal-api:
|
||||
image: bbernhard/signal-cli-rest-api:latest
|
||||
container_name: calbot-signal
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- MODE=json-rpc-native
|
||||
- SIGNAL_CLI_UID=1000
|
||||
- SIGNAL_CLI_GID=1000
|
||||
- LOG_LEVEL=info
|
||||
ports:
|
||||
- "127.0.0.1:8080:8080"
|
||||
volumes:
|
||||
# Linked-device keys live here. Back it up; losing it means re-linking.
|
||||
- ./signal-data:/home/.local/share/signal-cli
|
||||
|
||||
bot:
|
||||
build: .
|
||||
container_name: calbot
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- signal-api
|
||||
user: "1000:1000"
|
||||
environment:
|
||||
- CALBOT_CONFIG=/app/config.toml
|
||||
- LOG_LEVEL=info
|
||||
- TZ=Europe/London
|
||||
volumes:
|
||||
- ./config.toml:/app/config.toml:ro
|
||||
- ./data:/app/data
|
||||
- ./cache:/app/cache
|
||||
@@ -0,0 +1,29 @@
|
||||
[project]
|
||||
name = "calbot"
|
||||
version = "0.1.0"
|
||||
description = "Announce iCalendar events to a Signal group"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"icalendar>=6.0",
|
||||
"recurring-ical-events>=3.0",
|
||||
"httpx>=0.27",
|
||||
"pillow>=10.0",
|
||||
"beautifulsoup4>=4.12",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = ["pytest>=8.0", "ruff>=0.6"]
|
||||
|
||||
[project.scripts]
|
||||
calbot = "calbot.__main__:main"
|
||||
|
||||
[build-system]
|
||||
requires = ["setuptools>=68"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[tool.setuptools.packages.find]
|
||||
where = ["src"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
pythonpath = ["src"]
|
||||
testpaths = ["tests"]
|
||||
@@ -0,0 +1,3 @@
|
||||
"""calbot: announce iCalendar events to a Signal group."""
|
||||
|
||||
__version__ = "0.1.0"
|
||||
@@ -0,0 +1,187 @@
|
||||
"""Entry point: run the bot loop, or dump what would be posted."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import time
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import httpx
|
||||
|
||||
from .config import Config, ConfigError, load_config
|
||||
from .expand import parse_events
|
||||
from .feed import Feed
|
||||
from .format import find_postcode, render_post, venue_from_description, venue_from_location
|
||||
from .images import ImageService, PageScraper
|
||||
from .scheduler import Bot
|
||||
from .signal_client import SignalClient
|
||||
from .store import Store
|
||||
from .watchdog import Watchdog
|
||||
|
||||
log = logging.getLogger("calbot")
|
||||
|
||||
TICK_SECONDS = 30
|
||||
|
||||
|
||||
def _logging(level: str) -> None:
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, level.upper(), logging.INFO),
|
||||
format="%(asctime)s %(levelname)-7s %(name)s: %(message)s",
|
||||
)
|
||||
|
||||
|
||||
def _http_client() -> httpx.Client:
|
||||
return httpx.Client(
|
||||
follow_redirects=True,
|
||||
headers={"User-Agent": "calbot/0.1 (iCalendar to Signal bridge)"},
|
||||
limits=httpx.Limits(max_connections=8, max_keepalive_connections=4),
|
||||
)
|
||||
|
||||
|
||||
def dump(config: Config) -> int:
|
||||
"""Print rendered posts for the next events, without touching Signal."""
|
||||
http = _http_client()
|
||||
feed = Feed(config.calendar.url, Store(config.db_path), config.calendar.refresh_minutes, http)
|
||||
try:
|
||||
raw = feed.fetch(datetime.now(timezone.utc))
|
||||
except httpx.HTTPError as exc:
|
||||
print(f"feed fetch failed: {exc}", file=sys.stderr)
|
||||
return 1
|
||||
if raw is None:
|
||||
print("feed unchanged (304); nothing to dump")
|
||||
return 0
|
||||
|
||||
now = datetime.now(timezone.utc)
|
||||
events = parse_events(
|
||||
raw,
|
||||
now,
|
||||
now + timedelta(days=config.calendar.window_days),
|
||||
config.calendar.timezone,
|
||||
)
|
||||
local_tz = ZoneInfo(config.calendar.timezone)
|
||||
for event in events:
|
||||
venue = venue_from_location(event.location) or venue_from_description(event.description)
|
||||
postcode = find_postcode(event.location, event.description)
|
||||
print("=" * 60)
|
||||
print(f"{event.start.astimezone(local_tz):%Y-%m-%d %H:%M} [{event.series_id}]")
|
||||
print("-" * 60)
|
||||
print(
|
||||
render_post(
|
||||
event,
|
||||
config.format,
|
||||
local_tz,
|
||||
venue=venue,
|
||||
postcode=postcode,
|
||||
link=event.url,
|
||||
)
|
||||
)
|
||||
print(f"image: {event.image_url or '(none)'}")
|
||||
print("=" * 60)
|
||||
print(f"{len(events)} event(s)")
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
parser = argparse.ArgumentParser(prog="calbot", description=__doc__)
|
||||
parser.add_argument(
|
||||
"--config",
|
||||
default=os.environ.get("CALBOT_CONFIG", "config.toml"),
|
||||
help="path to config.toml (default: $CALBOT_CONFIG or ./config.toml)",
|
||||
)
|
||||
parser.add_argument("--once", action="store_true", help="run a single tick and exit")
|
||||
parser.add_argument("--dump", action="store_true", help="print posts for upcoming events")
|
||||
parser.add_argument("--check", action="store_true", help="validate config and connection")
|
||||
parser.add_argument("--log-level", default=os.environ.get("LOG_LEVEL", "INFO"))
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
try:
|
||||
config = load_config(args.config)
|
||||
except ConfigError as exc:
|
||||
print(f"config error: {exc}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
_logging(args.log_level)
|
||||
|
||||
if args.dump:
|
||||
return dump(config)
|
||||
|
||||
if not config.signal.group_id.startswith("group."):
|
||||
log.warning(
|
||||
"signal.group_id does not start with 'group.' — "
|
||||
"use the internal_id from GET /v1/groups/<number>"
|
||||
)
|
||||
|
||||
store = Store(config.db_path)
|
||||
http = _http_client()
|
||||
sender = SignalClient(config.signal.api_url, config.signal.number)
|
||||
watchdog = Watchdog(config.watchdog, sender, config.signal.number, config.signal.text_mode)
|
||||
|
||||
if args.check:
|
||||
if sender.account_exists():
|
||||
log.info("Signal account %s is linked", config.signal.number)
|
||||
else:
|
||||
log.error(
|
||||
"Signal account %s not found at %s. Link it first: "
|
||||
"open %s/v1/qrcodelink?device_name=calbot and scan the QR code.",
|
||||
config.signal.number,
|
||||
config.signal.api_url,
|
||||
config.signal.api_url,
|
||||
)
|
||||
return 1
|
||||
log.info("config OK: %s", config.config_path)
|
||||
return 0
|
||||
|
||||
bot = Bot(
|
||||
config=config,
|
||||
store=store,
|
||||
feed=Feed(config.calendar.url, store, config.calendar.refresh_minutes, http),
|
||||
sender=sender,
|
||||
images=ImageService(config.images, config.cache_dir, http),
|
||||
scraper=PageScraper(config.images, store, http),
|
||||
watchdog=watchdog,
|
||||
http=http,
|
||||
)
|
||||
|
||||
if args.once:
|
||||
bot.tick()
|
||||
return 0
|
||||
|
||||
stopping = False
|
||||
|
||||
def stop(_signum, _frame):
|
||||
nonlocal stopping
|
||||
stopping = True
|
||||
|
||||
signal.signal(signal.SIGTERM, stop)
|
||||
signal.signal(signal.SIGINT, stop)
|
||||
|
||||
log.info("calbot started (feed=%s, group=%s)", config.calendar.url, config.signal.group_id)
|
||||
if not sender.account_exists():
|
||||
log.warning(
|
||||
"Signal account not linked yet. Open %s/v1/qrcodelink?device_name=calbot",
|
||||
config.signal.api_url,
|
||||
)
|
||||
|
||||
while not stopping:
|
||||
try:
|
||||
bot.tick()
|
||||
except Exception: # noqa: BLE001 - keep the service alive
|
||||
log.exception("tick failed")
|
||||
deadline = time.monotonic() + TICK_SECONDS
|
||||
while not stopping and time.monotonic() < deadline:
|
||||
time.sleep(0.5)
|
||||
|
||||
log.info("calbot stopped")
|
||||
store.close()
|
||||
http.close()
|
||||
sender.close()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@@ -0,0 +1,259 @@
|
||||
"""Configuration loading and defaults."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import tomllib
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import time
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class ConfigError(Exception):
|
||||
"""Raised when the configuration file is invalid."""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# small parsers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_DURATION_RE = re.compile(r"^\s*(\d+)\s*([smhdw])\s*$", re.IGNORECASE)
|
||||
_DURATION_UNITS = {
|
||||
"s": 1,
|
||||
"m": 60,
|
||||
"h": 3600,
|
||||
"d": 86400,
|
||||
"w": 604800,
|
||||
}
|
||||
|
||||
|
||||
def parse_duration(value: str | int) -> int:
|
||||
"""Parse '3d', '2h', '90m', '1w' into seconds. Bare ints are seconds."""
|
||||
if isinstance(value, int):
|
||||
return value
|
||||
match = _DURATION_RE.match(value)
|
||||
if not match:
|
||||
raise ConfigError(f"invalid duration: {value!r} (use e.g. '3d', '2h', '30m')")
|
||||
return int(match.group(1)) * _DURATION_UNITS[match.group(2).lower()]
|
||||
|
||||
|
||||
def parse_clock(value: str) -> time:
|
||||
try:
|
||||
hour, minute = (int(part) for part in value.split(":"))
|
||||
return time(hour, minute)
|
||||
except (ValueError, TypeError) as exc:
|
||||
raise ConfigError(f"invalid time of day: {value!r}") from exc
|
||||
|
||||
|
||||
def parse_window(value: str) -> tuple[time, time]:
|
||||
try:
|
||||
start, end = value.split("-")
|
||||
except ValueError as exc:
|
||||
raise ConfigError(f"invalid window: {value!r} (use '08:00-10:00')") from exc
|
||||
start_time, end_time = parse_clock(start), parse_clock(end)
|
||||
if end_time <= start_time:
|
||||
raise ConfigError(f"window must end after it starts: {value!r}")
|
||||
return start_time, end_time
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# config sections
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SignalConfig:
|
||||
api_url: str
|
||||
number: str
|
||||
group_id: str
|
||||
text_mode: str = "normal"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CalendarConfig:
|
||||
url: str
|
||||
timezone: str = "Europe/London"
|
||||
refresh_minutes: int = 15
|
||||
window_days: int = 60
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PostingConfig:
|
||||
windows: tuple[tuple[time, time], ...] = ((time(8, 0), time(10, 0)), (time(18, 0), time(21, 0)))
|
||||
urgent_guard_minutes: int = 60
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AnnounceConfig:
|
||||
enabled: bool = True
|
||||
max_days_ahead: int = 30
|
||||
backfill: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RemindersConfig:
|
||||
enabled: bool = True
|
||||
lead_times: tuple[str, ...] = ("3d",)
|
||||
fallback_to_day_of: bool = True
|
||||
day_of_time: time = time(9, 0)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ImagesConfig:
|
||||
enabled: bool = True
|
||||
max_pixels: int = 1600
|
||||
max_bytes: int = 400_000
|
||||
max_images_per_post: int = 3
|
||||
timeout_seconds: float = 10.0
|
||||
scrape_event_page: bool = True
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FormatConfig:
|
||||
show_title: bool = True
|
||||
show_hashtags: bool = True
|
||||
show_time: bool = True
|
||||
show_location: bool = True
|
||||
show_link: bool = True
|
||||
shorten_links: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class WatchdogConfig:
|
||||
enabled: bool = True
|
||||
notify_number: str | None = None
|
||||
feed_failures: int = 3
|
||||
send_failures: int = 3
|
||||
alert_cooldown_hours: float = 6.0
|
||||
daily_heartbeat: bool = False
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Config:
|
||||
signal: SignalConfig
|
||||
calendar: CalendarConfig
|
||||
posting: PostingConfig = field(default_factory=PostingConfig)
|
||||
announce: AnnounceConfig = field(default_factory=AnnounceConfig)
|
||||
reminders: RemindersConfig = field(default_factory=RemindersConfig)
|
||||
images: ImagesConfig = field(default_factory=ImagesConfig)
|
||||
format: FormatConfig = field(default_factory=FormatConfig)
|
||||
watchdog: WatchdogConfig = field(default_factory=WatchdogConfig)
|
||||
db_path: Path = Path("data/calbot.sqlite3")
|
||||
cache_dir: Path = Path("cache")
|
||||
config_path: Path | None = None
|
||||
base_dir: Path = Path(".")
|
||||
|
||||
|
||||
def _section(data: dict, name: str) -> dict:
|
||||
value = data.get(name, {})
|
||||
if not isinstance(value, dict):
|
||||
raise ConfigError(f"[{name}] must be a table")
|
||||
return value
|
||||
|
||||
|
||||
def _windows(raw: object) -> tuple[tuple[time, time], ...]:
|
||||
if not isinstance(raw, list) or not raw:
|
||||
raise ConfigError("posting.windows must be a non-empty list of 'HH:MM-HH:MM'")
|
||||
windows = tuple(sorted(parse_window(str(item)) for item in raw))
|
||||
return windows
|
||||
|
||||
|
||||
def load_config(path: str | Path) -> Config:
|
||||
config_path = Path(path).expanduser().resolve()
|
||||
try:
|
||||
data = tomllib.loads(config_path.read_text(encoding="utf-8"))
|
||||
except FileNotFoundError as exc:
|
||||
raise ConfigError(f"config file not found: {config_path}") from exc
|
||||
except tomllib.TOMLDecodeError as exc:
|
||||
raise ConfigError(f"invalid TOML in {config_path}: {exc}") from exc
|
||||
|
||||
base_dir = config_path.parent
|
||||
|
||||
signal = _section(data, "signal")
|
||||
for key in ("api_url", "number", "group_id"):
|
||||
if not signal.get(key):
|
||||
raise ConfigError(f"signal.{key} is required")
|
||||
|
||||
calendar = _section(data, "calendar")
|
||||
if not calendar.get("url"):
|
||||
raise ConfigError("calendar.url is required")
|
||||
|
||||
posting = _section(data, "posting")
|
||||
announce = _section(data, "announce")
|
||||
reminders = _section(data, "reminders")
|
||||
images = _section(data, "images")
|
||||
fmt = _section(data, "format")
|
||||
watchdog = _section(data, "watchdog")
|
||||
storage = _section(data, "storage")
|
||||
|
||||
lead_times = tuple(str(lead) for lead in reminders.get("lead_times", ["3d"]))
|
||||
for lead in lead_times:
|
||||
parse_duration(lead)
|
||||
|
||||
db_path = Path(os.environ.get("CALBOT_DB", storage.get("db", "data/calbot.sqlite3")))
|
||||
cache_dir = Path(os.environ.get("CALBOT_CACHE", storage.get("cache", "cache")))
|
||||
if not db_path.is_absolute():
|
||||
db_path = base_dir / db_path
|
||||
if not cache_dir.is_absolute():
|
||||
cache_dir = base_dir / cache_dir
|
||||
|
||||
return Config(
|
||||
signal=SignalConfig(
|
||||
api_url=str(signal["api_url"]).rstrip("/"),
|
||||
number=str(signal["number"]),
|
||||
group_id=str(signal["group_id"]),
|
||||
text_mode=str(signal.get("text_mode", "normal")),
|
||||
),
|
||||
calendar=CalendarConfig(
|
||||
url=str(calendar["url"]),
|
||||
timezone=str(calendar.get("timezone", "Europe/London")),
|
||||
refresh_minutes=int(calendar.get("refresh_minutes", 15)),
|
||||
window_days=int(calendar.get("window_days", 60)),
|
||||
),
|
||||
posting=PostingConfig(
|
||||
windows=_windows(posting.get("windows", ["08:00-10:00", "18:00-21:00"])),
|
||||
urgent_guard_minutes=int(posting.get("urgent_guard_minutes", 60)),
|
||||
),
|
||||
announce=AnnounceConfig(
|
||||
enabled=bool(announce.get("enabled", True)),
|
||||
max_days_ahead=int(announce.get("max_days_ahead", 30)),
|
||||
backfill=bool(announce.get("backfill", False)),
|
||||
),
|
||||
reminders=RemindersConfig(
|
||||
enabled=bool(reminders.get("enabled", True)),
|
||||
lead_times=lead_times,
|
||||
fallback_to_day_of=bool(reminders.get("fallback_to_day_of", True)),
|
||||
day_of_time=parse_clock(str(reminders.get("day_of_time", "09:00"))),
|
||||
),
|
||||
images=ImagesConfig(
|
||||
enabled=bool(images.get("enabled", True)),
|
||||
max_pixels=int(images.get("max_pixels", 1600)),
|
||||
max_bytes=int(images.get("max_bytes", 400_000)),
|
||||
max_images_per_post=int(images.get("max_images_per_post", 3)),
|
||||
timeout_seconds=float(images.get("timeout_seconds", 10)),
|
||||
scrape_event_page=bool(images.get("scrape_event_page", True)),
|
||||
),
|
||||
format=FormatConfig(
|
||||
show_title=bool(fmt.get("show_title", True)),
|
||||
show_hashtags=bool(fmt.get("show_hashtags", True)),
|
||||
show_time=bool(fmt.get("show_time", True)),
|
||||
show_location=bool(fmt.get("show_location", True)),
|
||||
show_link=bool(fmt.get("show_link", True)),
|
||||
shorten_links=bool(fmt.get("shorten_links", False)),
|
||||
),
|
||||
watchdog=WatchdogConfig(
|
||||
enabled=bool(watchdog.get("enabled", True)),
|
||||
notify_number=(
|
||||
str(watchdog["notify_number"]) if watchdog.get("notify_number") else None
|
||||
),
|
||||
feed_failures=int(watchdog.get("feed_failures", 3)),
|
||||
send_failures=int(watchdog.get("send_failures", 3)),
|
||||
alert_cooldown_hours=float(watchdog.get("alert_cooldown_hours", 6)),
|
||||
daily_heartbeat=bool(watchdog.get("daily_heartbeat", False)),
|
||||
),
|
||||
db_path=db_path,
|
||||
cache_dir=cache_dir,
|
||||
config_path=config_path,
|
||||
base_dir=base_dir,
|
||||
)
|
||||
@@ -0,0 +1,178 @@
|
||||
"""Turn an iCalendar document into concrete Event instances inside a window."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from datetime import date, datetime, time, timedelta, timezone
|
||||
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
|
||||
|
||||
import icalendar
|
||||
import recurring_ical_events
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
TEC_UID_RE = re.compile(r"^(\d+)-\d+-\d+@")
|
||||
DATE_LIKE_RE = re.compile(r"^\d{8}$")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Event:
|
||||
key: str
|
||||
uid: str
|
||||
series_id: str
|
||||
summary: str
|
||||
start: datetime
|
||||
end: datetime
|
||||
all_day: bool
|
||||
location: str = ""
|
||||
description: str = ""
|
||||
url: str = ""
|
||||
image_url: str = ""
|
||||
tz: str = "UTC"
|
||||
status: str = ""
|
||||
|
||||
|
||||
def _zone(name: str) -> ZoneInfo:
|
||||
try:
|
||||
return ZoneInfo(name)
|
||||
except (ZoneInfoNotFoundError, ValueError):
|
||||
log.warning("unknown timezone %r, falling back to UTC", name)
|
||||
return ZoneInfo("UTC")
|
||||
|
||||
|
||||
def _as_datetime(value: datetime | date, tz: ZoneInfo) -> datetime:
|
||||
if isinstance(value, datetime):
|
||||
if value.tzinfo is None:
|
||||
return value.replace(tzinfo=tz)
|
||||
return value
|
||||
return datetime.combine(value, time.min, tzinfo=tz)
|
||||
|
||||
|
||||
def _all_day_flag(component) -> bool:
|
||||
prop = component.get("DTSTART")
|
||||
return prop is not None and not isinstance(getattr(prop, "dt", None), datetime)
|
||||
|
||||
|
||||
def _image_url(component) -> str:
|
||||
for name in ("ATTACH", "IMAGE"):
|
||||
value = component.get(name)
|
||||
if value is None:
|
||||
continue
|
||||
values = value if isinstance(value, list) else [value]
|
||||
for item in values:
|
||||
fmt = ""
|
||||
params = getattr(item, "params", {}) or {}
|
||||
fmt = str(params.get("FMTTYPE", "")).lower()
|
||||
if fmt and not fmt.startswith("image/"):
|
||||
continue
|
||||
uri = str(item)
|
||||
if uri.startswith(("http://", "https://")):
|
||||
return uri
|
||||
return ""
|
||||
|
||||
|
||||
def _series_id(uid: str, summary: str, start: datetime, duration: timedelta) -> str:
|
||||
match = TEC_UID_RE.match(uid)
|
||||
if match:
|
||||
return f"tec:{match.group(1)}"
|
||||
fingerprint = f"{summary}|{start.astimezone(timezone.utc).strftime('%H:%M')}|{duration}"
|
||||
digest = hashlib.sha1(fingerprint.encode("utf-8")).hexdigest()[:12]
|
||||
return f"h:{digest}"
|
||||
|
||||
|
||||
def _build_event(component, default_tz: ZoneInfo) -> Event | None:
|
||||
dtstart = component.get("DTSTART")
|
||||
if dtstart is None:
|
||||
return None
|
||||
raw_start = dtstart.dt
|
||||
|
||||
status = str(component.get("STATUS", "")).upper()
|
||||
if status == "CANCELLED":
|
||||
return None
|
||||
|
||||
all_day = not isinstance(raw_start, datetime)
|
||||
tz_name = default_tz.key
|
||||
if isinstance(raw_start, datetime) and raw_start.tzinfo is not None:
|
||||
tz_name = getattr(raw_start.tzinfo, "key", None) or str(raw_start.tzinfo)
|
||||
|
||||
local_tz = _zone(tz_name)
|
||||
start_local = _as_datetime(raw_start, default_tz)
|
||||
|
||||
dtend = component.get("DTEND")
|
||||
duration_prop = component.get("DURATION")
|
||||
if dtend is not None:
|
||||
end_local = _as_datetime(dtend.dt, default_tz)
|
||||
elif duration_prop is not None:
|
||||
end_local = start_local + duration_prop.dt
|
||||
else:
|
||||
end_local = start_local + timedelta(days=1 if all_day else 0)
|
||||
|
||||
start = start_local.astimezone(timezone.utc)
|
||||
end = end_local.astimezone(timezone.utc)
|
||||
if end < start:
|
||||
end = start
|
||||
|
||||
uid = str(component.get("UID", "")).strip()
|
||||
summary = str(component.get("SUMMARY", "")).strip() or "(untitled event)"
|
||||
duration = end - start
|
||||
if not uid:
|
||||
uid = hashlib.sha1(f"{summary}|{start.isoformat()}".encode()).hexdigest()[:16]
|
||||
|
||||
return Event(
|
||||
key=f"{uid}|{start.isoformat()}",
|
||||
uid=uid,
|
||||
series_id=_series_id(uid, summary, start, duration),
|
||||
summary=summary,
|
||||
start=start,
|
||||
end=end,
|
||||
all_day=all_day,
|
||||
location=str(component.get("LOCATION", "") or "").strip(),
|
||||
description=str(component.get("DESCRIPTION", "") or "").strip(),
|
||||
url=str(component.get("URL", "") or "").strip(),
|
||||
image_url=_image_url(component),
|
||||
tz=local_tz.key,
|
||||
status=status,
|
||||
)
|
||||
|
||||
|
||||
def parse_events(
|
||||
raw: bytes | str,
|
||||
window_start: datetime,
|
||||
window_end: datetime,
|
||||
default_timezone: str = "Europe/London",
|
||||
) -> list[Event]:
|
||||
"""Parse an iCalendar document and return events overlapping the window."""
|
||||
default_tz = _zone(default_timezone)
|
||||
calendar = icalendar.Calendar.from_ical(raw)
|
||||
|
||||
if window_start.tzinfo is None:
|
||||
window_start = window_start.replace(tzinfo=default_tz)
|
||||
if window_end.tzinfo is None:
|
||||
window_end = window_end.replace(tzinfo=default_tz)
|
||||
|
||||
components = []
|
||||
try:
|
||||
components = recurring_ical_events.of(calendar).between(window_start, window_end)
|
||||
except Exception: # pragma: no cover - defensive, feeds vary wildly
|
||||
log.exception("recurrence expansion failed; falling back to raw VEVENTs")
|
||||
|
||||
if not components:
|
||||
components = [
|
||||
item
|
||||
for item in calendar.walk("VEVENT")
|
||||
if getattr(item.get("DTSTART"), "dt", None) is not None
|
||||
]
|
||||
|
||||
events: dict[str, Event] = {}
|
||||
for component in components:
|
||||
event = _build_event(component, default_tz)
|
||||
if event is None:
|
||||
continue
|
||||
if event.end <= window_start or event.start >= window_end:
|
||||
continue
|
||||
events[event.key] = event
|
||||
|
||||
return sorted(events.values(), key=lambda item: item.start)
|
||||
@@ -0,0 +1,61 @@
|
||||
"""Fetch the iCalendar feed with conditional requests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import httpx
|
||||
|
||||
from .store import Store, from_iso, iso
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
USER_AGENT = "calbot/0.1 (iCalendar to Signal bridge)"
|
||||
|
||||
|
||||
def normalize_feed_url(url: str) -> str:
|
||||
"""webcal:// is a hint, not a transport scheme."""
|
||||
if url.startswith("webcal://"):
|
||||
return "https://" + url[len("webcal://") :]
|
||||
return url
|
||||
|
||||
|
||||
class Feed:
|
||||
def __init__(self, url: str, store: Store, refresh_minutes: int, http: httpx.Client):
|
||||
self.url = normalize_feed_url(url)
|
||||
self.store = store
|
||||
self.refresh_minutes = max(1, refresh_minutes)
|
||||
self.http = http
|
||||
|
||||
def due(self, now: datetime) -> bool:
|
||||
last = self.store.get_meta("feed_last_attempt")
|
||||
if not last:
|
||||
return True
|
||||
return now - from_iso(last) >= timedelta(minutes=self.refresh_minutes)
|
||||
|
||||
def fetch(self, now: datetime) -> bytes | None:
|
||||
"""Return new feed bytes, or None when unchanged / not modified."""
|
||||
headers = {"User-Agent": USER_AGENT, "Accept": "text/calendar, */*"}
|
||||
etag = self.store.get_meta("feed_etag")
|
||||
modified = self.store.get_meta("feed_last_modified")
|
||||
if etag:
|
||||
headers["If-None-Match"] = etag
|
||||
if modified:
|
||||
headers["If-Modified-Since"] = modified
|
||||
|
||||
self.store.set_meta("feed_last_attempt", iso(now))
|
||||
response = self.http.get(self.url, headers=headers, timeout=30)
|
||||
|
||||
if response.status_code == 304:
|
||||
log.debug("feed not modified")
|
||||
return None
|
||||
response.raise_for_status()
|
||||
|
||||
if response.headers.get("etag"):
|
||||
self.store.set_meta("feed_etag", response.headers["etag"])
|
||||
if response.headers.get("last-modified"):
|
||||
self.store.set_meta("feed_last_modified", response.headers["last-modified"])
|
||||
self.store.set_meta("feed_last_success", iso(datetime.now(timezone.utc)))
|
||||
log.info("fetched feed: %d bytes", len(response.content))
|
||||
return response.content
|
||||
@@ -0,0 +1,112 @@
|
||||
"""Render an event in the group's posting format."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import calendar as calendar_module
|
||||
import re
|
||||
from datetime import datetime
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from .config import FormatConfig
|
||||
from .expand import Event
|
||||
|
||||
# Loose UK postcode matcher; deliberately permissive because feeds vary.
|
||||
UK_POSTCODE_RE = re.compile(r"\b([A-Z]{1,2}\d[A-Z\d]?)\s*(\d[A-Z]{2})\b", re.IGNORECASE)
|
||||
|
||||
|
||||
def find_postcode(*texts: str) -> str:
|
||||
for text in texts:
|
||||
if not text:
|
||||
continue
|
||||
match = UK_POSTCODE_RE.search(text)
|
||||
if match:
|
||||
return f"{match.group(1).upper()} {match.group(2).upper()}"
|
||||
return ""
|
||||
|
||||
|
||||
def _clean(text: str) -> str:
|
||||
return re.sub(r"\s+", " ", text or "").strip()
|
||||
|
||||
|
||||
def _first_line(text: str) -> str:
|
||||
for line in (text or "").splitlines():
|
||||
if line.strip():
|
||||
return line.strip()
|
||||
return ""
|
||||
|
||||
|
||||
def venue_from_location(location: str) -> str:
|
||||
"""The first line of a TEC LOCATION value is the venue name."""
|
||||
return _clean(_first_line(location)).rstrip(",")
|
||||
|
||||
|
||||
def venue_from_description(description: str) -> str:
|
||||
"""Best-effort fallback: a line in the description that carries a postcode."""
|
||||
for line in (description or "").splitlines():
|
||||
match = UK_POSTCODE_RE.search(line)
|
||||
if not match:
|
||||
continue
|
||||
before = line[: match.start()].strip(" ,-\u2013")
|
||||
parts = [part.strip() for part in re.split(r"[,;]", before) if part.strip()]
|
||||
if parts:
|
||||
return _clean(parts[0])[:60]
|
||||
return ""
|
||||
|
||||
|
||||
def _clock(moment: datetime) -> tuple[str, str]:
|
||||
hour = moment.hour % 12 or 12
|
||||
meridiem = "am" if moment.hour < 12 else "pm"
|
||||
return f"{hour}.{moment.minute:02d}", meridiem
|
||||
|
||||
|
||||
def format_time_range(start: datetime, end: datetime) -> str:
|
||||
start_text, start_meridiem = _clock(start)
|
||||
end_text, end_meridiem = _clock(end)
|
||||
if not end or end <= start:
|
||||
return f"{start_text}{start_meridiem}"
|
||||
if start_meridiem == end_meridiem:
|
||||
return f"{start_text}-{end_text}{end_meridiem}"
|
||||
return f"{start_text}{start_meridiem}-{end_text}{end_meridiem}"
|
||||
|
||||
|
||||
def format_hashtags(start: datetime) -> str:
|
||||
month = calendar_module.month_name[start.month]
|
||||
return f"#{month} #{month}{start.day}"
|
||||
|
||||
|
||||
def render_post(
|
||||
event: Event,
|
||||
config: FormatConfig,
|
||||
local_tz: ZoneInfo,
|
||||
*,
|
||||
venue: str = "",
|
||||
postcode: str = "",
|
||||
link: str = "",
|
||||
) -> str:
|
||||
start = event.start.astimezone(local_tz)
|
||||
end = event.end.astimezone(local_tz)
|
||||
|
||||
lines: list[str] = []
|
||||
|
||||
if config.show_title:
|
||||
title = _clean(event.summary)
|
||||
if title:
|
||||
lines.append(title)
|
||||
|
||||
if config.show_hashtags:
|
||||
lines.append(format_hashtags(start))
|
||||
|
||||
if config.show_time:
|
||||
lines.append("All day" if event.all_day else format_time_range(start, end))
|
||||
|
||||
if config.show_location:
|
||||
name = _clean(venue) or venue_from_location(event.location)
|
||||
code = postcode or find_postcode(event.location, event.description)
|
||||
place = " ".join(part for part in (name, code) if part).strip()
|
||||
if place:
|
||||
lines.append(place)
|
||||
|
||||
if config.show_link and link:
|
||||
lines.append(link)
|
||||
|
||||
return "\n".join(lines)
|
||||
@@ -0,0 +1,224 @@
|
||||
"""Download, downscale and cache event images; scrape event pages for details."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import hashlib
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
from bs4 import BeautifulSoup
|
||||
from PIL import Image, ImageOps, UnidentifiedImageError
|
||||
|
||||
from .config import ImagesConfig
|
||||
from .store import PageInfo, Store
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# Decompression-bomb guard. Real posters are nowhere near this.
|
||||
Image.MAX_IMAGE_PIXELS = 50_000_000
|
||||
|
||||
MAX_DOWNLOAD_BYTES = 20 * 1024 * 1024
|
||||
DOWNLOAD_TIMEOUT = 20.0
|
||||
USER_AGENT = "calbot/0.1 (iCalendar to Signal bridge)"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PageDetails:
|
||||
image_url: str = ""
|
||||
venue: str = ""
|
||||
postcode: str = ""
|
||||
|
||||
|
||||
class PageScraper:
|
||||
"""Best-effort extraction of image and venue from a TEC event page.
|
||||
|
||||
Results are cached forever per URL; pages are stable in practice and this
|
||||
keeps us to at most one request per event.
|
||||
"""
|
||||
|
||||
def __init__(self, config: ImagesConfig, store: Store, http: httpx.Client):
|
||||
self.config = config
|
||||
self.store = store
|
||||
self.http = http
|
||||
|
||||
def details(self, event_url: str) -> PageInfo:
|
||||
if not event_url or not self.config.scrape_event_page:
|
||||
return PageInfo(image_url="", venue="", postcode="")
|
||||
|
||||
cached = self.store.get_page(event_url)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
try:
|
||||
response = self.http.get(
|
||||
event_url,
|
||||
headers={"User-Agent": USER_AGENT},
|
||||
timeout=DOWNLOAD_TIMEOUT,
|
||||
follow_redirects=True,
|
||||
)
|
||||
response.raise_for_status()
|
||||
except httpx.HTTPError as exc:
|
||||
log.warning("page fetch failed for %s: %s", event_url, exc)
|
||||
return PageInfo(image_url="", venue="", postcode="")
|
||||
|
||||
soup = BeautifulSoup(response.text, "html.parser")
|
||||
details = self._extract(soup)
|
||||
info = PageInfo(
|
||||
image_url=details.image_url,
|
||||
venue=details.venue,
|
||||
postcode=details.postcode,
|
||||
)
|
||||
self.store.put_page(event_url, info, datetime.now())
|
||||
return info
|
||||
|
||||
@staticmethod
|
||||
def _extract(soup: BeautifulSoup) -> PageDetails:
|
||||
image_url = ""
|
||||
venue = ""
|
||||
postcode = ""
|
||||
|
||||
for script in soup.find_all("script", type="application/ld+json"):
|
||||
try:
|
||||
payload = json.loads(script.string or "")
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
for node in _walk_json(payload):
|
||||
if not isinstance(node, dict) or not _is_event(node):
|
||||
continue
|
||||
image_url = image_url or _first_image(node.get("image"))
|
||||
place = node.get("location")
|
||||
if isinstance(place, list):
|
||||
place = place[0] if place else None
|
||||
if isinstance(place, dict):
|
||||
venue = venue or str(place.get("name", "") or "").strip()
|
||||
address = place.get("address")
|
||||
if isinstance(address, list):
|
||||
address = address[0] if address else None
|
||||
if isinstance(address, dict):
|
||||
postcode = postcode or str(address.get("postalCode", "") or "").strip()
|
||||
|
||||
if not image_url:
|
||||
for attrs in (
|
||||
{"property": "og:image"},
|
||||
{"name": "og:image"},
|
||||
{"name": "twitter:image"},
|
||||
):
|
||||
meta = soup.find("meta", attrs=attrs)
|
||||
if meta and meta.get("content"):
|
||||
image_url = str(meta["content"])
|
||||
break
|
||||
|
||||
return PageDetails(image_url=image_url, venue=venue, postcode=postcode)
|
||||
|
||||
|
||||
def _walk_json(node):
|
||||
if isinstance(node, dict):
|
||||
yield node
|
||||
for value in node.values():
|
||||
yield from _walk_json(value)
|
||||
elif isinstance(node, list):
|
||||
for value in node:
|
||||
yield from _walk_json(value)
|
||||
|
||||
|
||||
def _is_event(node: dict) -> bool:
|
||||
kind = node.get("@type")
|
||||
if isinstance(kind, list):
|
||||
return any(str(item).lower() == "event" for item in kind)
|
||||
return str(kind).lower() == "event"
|
||||
|
||||
|
||||
def _first_image(value) -> str:
|
||||
if isinstance(value, str):
|
||||
return value
|
||||
if isinstance(value, dict):
|
||||
return str(value.get("url", "") or "")
|
||||
if isinstance(value, list):
|
||||
for item in value:
|
||||
found = _first_image(item)
|
||||
if found:
|
||||
return found
|
||||
return ""
|
||||
|
||||
|
||||
class ImageService:
|
||||
def __init__(self, config: ImagesConfig, cache_dir: Path, http: httpx.Client):
|
||||
self.config = config
|
||||
self.cache_dir = cache_dir / "images"
|
||||
self.cache_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.http = http
|
||||
|
||||
def data_uri(self, url: str) -> str | None:
|
||||
if not self.config.enabled or not url:
|
||||
return None
|
||||
try:
|
||||
payload = self._cached_or_download(url)
|
||||
except (httpx.HTTPError, OSError) as exc:
|
||||
log.warning("image fetch failed for %s: %s", url, exc)
|
||||
return None
|
||||
if payload is None:
|
||||
return None
|
||||
encoded = base64.b64encode(payload).decode("ascii")
|
||||
return f"data:image/jpeg;base64,{encoded}"
|
||||
|
||||
def _cached_or_download(self, url: str) -> bytes | None:
|
||||
digest = hashlib.sha1(url.encode("utf-8")).hexdigest()[:20]
|
||||
path = self.cache_dir / f"{digest}.jpg"
|
||||
if path.exists() and path.stat().st_size > 0:
|
||||
return path.read_bytes()
|
||||
|
||||
raw = self._download(url)
|
||||
if raw is None:
|
||||
return None
|
||||
resized = self._resize(raw)
|
||||
if resized is None:
|
||||
return None
|
||||
path.write_bytes(resized)
|
||||
return resized
|
||||
|
||||
def _download(self, url: str) -> bytes | None:
|
||||
if not url.startswith(("http://", "https://")):
|
||||
return None
|
||||
with self.http.stream(
|
||||
"GET", url, headers={"User-Agent": USER_AGENT}, timeout=DOWNLOAD_TIMEOUT
|
||||
) as response:
|
||||
response.raise_for_status()
|
||||
chunks: list[bytes] = []
|
||||
total = 0
|
||||
for chunk in response.iter_bytes():
|
||||
total += len(chunk)
|
||||
if total > MAX_DOWNLOAD_BYTES:
|
||||
log.warning("image too large, skipping: %s", url)
|
||||
return None
|
||||
chunks.append(chunk)
|
||||
return b"".join(chunks)
|
||||
|
||||
def _resize(self, raw: bytes) -> bytes | None:
|
||||
try:
|
||||
with Image.open(io.BytesIO(raw)) as image:
|
||||
image = ImageOps.exif_transpose(image)
|
||||
image = image.convert("RGB")
|
||||
image.thumbnail((self.config.max_pixels, self.config.max_pixels))
|
||||
|
||||
quality = 82
|
||||
while True:
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="JPEG", quality=quality, optimize=True)
|
||||
if buffer.tell() <= self.config.max_bytes or quality <= 50:
|
||||
return buffer.getvalue()
|
||||
quality -= 8
|
||||
if quality < 50:
|
||||
# Still too big: shrink dimensions and retry once.
|
||||
image.thumbnail(
|
||||
(int(image.width * 0.8), int(image.height * 0.8))
|
||||
)
|
||||
quality = 70
|
||||
except (UnidentifiedImageError, OSError, ValueError) as exc:
|
||||
log.warning("could not decode image: %s", exc)
|
||||
return None
|
||||
@@ -0,0 +1,389 @@
|
||||
"""Decision logic and the main tick loop."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import datetime, time, timedelta, timezone
|
||||
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
|
||||
|
||||
import httpx
|
||||
|
||||
from .config import Config, parse_duration
|
||||
from .expand import Event, parse_events
|
||||
from .feed import Feed
|
||||
from .format import find_postcode, render_post, venue_from_description, venue_from_location
|
||||
from .images import ImageService, PageScraper
|
||||
from .signal_client import SignalClient, SignalError
|
||||
from .store import OutboxItem, Store, iso
|
||||
from .watchdog import Watchdog
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
MAX_SEND_ATTEMPTS = 5
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# posting windows
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def in_window(moment: datetime, windows: tuple[tuple[time, time], ...]) -> bool:
|
||||
clock = moment.timetz().replace(tzinfo=None)
|
||||
return any(start <= clock < end for start, end in windows)
|
||||
|
||||
|
||||
def next_window_start(
|
||||
moment: datetime, windows: tuple[tuple[time, time], ...]
|
||||
) -> datetime:
|
||||
for offset in range(0, 4):
|
||||
day = moment.date() + timedelta(days=offset)
|
||||
for start, _end in windows:
|
||||
candidate = datetime.combine(day, start, tzinfo=moment.tzinfo)
|
||||
if candidate > moment:
|
||||
return candidate
|
||||
return moment + timedelta(days=1)
|
||||
|
||||
|
||||
def clamp_send_at(
|
||||
due: datetime,
|
||||
now: datetime,
|
||||
deadline: datetime,
|
||||
windows: tuple[tuple[time, time], ...],
|
||||
guard: timedelta,
|
||||
) -> datetime:
|
||||
"""Defer a post to the next preferred window unless that misses the deadline."""
|
||||
base = max(due, now)
|
||||
if in_window(base, windows):
|
||||
return base
|
||||
upcoming = next_window_start(base, windows)
|
||||
if upcoming >= deadline - guard:
|
||||
return base
|
||||
return upcoming
|
||||
|
||||
|
||||
def reminder_time(
|
||||
event: Event,
|
||||
lead: str,
|
||||
first_seen: datetime,
|
||||
config: Config,
|
||||
local_tz: ZoneInfo,
|
||||
) -> datetime | None:
|
||||
"""When the reminder for this lead should go out (local time), or None."""
|
||||
start = event.start.astimezone(local_tz)
|
||||
seen = first_seen.astimezone(local_tz)
|
||||
lead_delta = timedelta(seconds=parse_duration(lead))
|
||||
due = start - lead_delta
|
||||
|
||||
if due <= seen:
|
||||
if not config.reminders.fallback_to_day_of:
|
||||
return None
|
||||
due = datetime.combine(start.date(), config.reminders.day_of_time, tzinfo=local_tz)
|
||||
if not event.all_day:
|
||||
latest = start - timedelta(minutes=config.posting.urgent_guard_minutes)
|
||||
if due > latest:
|
||||
due = latest
|
||||
|
||||
deadline = start if not event.all_day else start + timedelta(hours=12)
|
||||
return clamp_send_at(
|
||||
due,
|
||||
seen,
|
||||
deadline,
|
||||
config.posting.windows,
|
||||
timedelta(minutes=config.posting.urgent_guard_minutes),
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# bot
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class Bot:
|
||||
def __init__(
|
||||
self,
|
||||
config: Config,
|
||||
store: Store,
|
||||
feed: Feed,
|
||||
sender: SignalClient,
|
||||
images: ImageService,
|
||||
scraper: PageScraper,
|
||||
watchdog: Watchdog,
|
||||
http: httpx.Client,
|
||||
):
|
||||
self.config = config
|
||||
self.store = store
|
||||
self.feed = feed
|
||||
self.sender = sender
|
||||
self.images = images
|
||||
self.scraper = scraper
|
||||
self.watchdog = watchdog
|
||||
self.http = http
|
||||
try:
|
||||
self.local_tz = ZoneInfo(config.calendar.timezone)
|
||||
except ZoneInfoNotFoundError:
|
||||
self.local_tz = ZoneInfo("UTC")
|
||||
|
||||
# -- loop ------------------------------------------------------------
|
||||
|
||||
def tick(self, now: datetime | None = None) -> None:
|
||||
now = now or datetime.now(timezone.utc)
|
||||
self._refresh_feed(now)
|
||||
self._dispatch(now)
|
||||
self._heartbeat(now)
|
||||
|
||||
# -- feed + planning -------------------------------------------------
|
||||
|
||||
def _refresh_feed(self, now: datetime) -> None:
|
||||
if not self.feed.due(now):
|
||||
return
|
||||
try:
|
||||
raw = self.feed.fetch(now)
|
||||
if raw is None:
|
||||
self.watchdog.feed_ok()
|
||||
return
|
||||
events = parse_events(
|
||||
raw,
|
||||
now,
|
||||
now + timedelta(days=self.config.calendar.window_days),
|
||||
self.config.calendar.timezone,
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001 - feed problems must not kill the loop
|
||||
log.exception("feed refresh failed")
|
||||
self.watchdog.feed_failed(str(exc), now)
|
||||
return
|
||||
|
||||
if not events and self.store.count_events() > 0:
|
||||
self.watchdog.alert(
|
||||
"empty-feed",
|
||||
"Feed parsed to zero events while the database still has events. "
|
||||
"Leaving stored events untouched.",
|
||||
now,
|
||||
)
|
||||
return
|
||||
|
||||
self.watchdog.feed_ok()
|
||||
self._sync(events, now)
|
||||
|
||||
def _sync(self, events: list[Event], now: datetime) -> None:
|
||||
first_run = self.store.get_meta("seeded") is None
|
||||
seeding = first_run and not self.config.announce.backfill
|
||||
|
||||
for event in events:
|
||||
is_new = self.store.upsert_event(event, now)
|
||||
if not is_new:
|
||||
continue
|
||||
|
||||
new_series = self.store.create_series(event.series_id, now)
|
||||
self._plan_reminders(event, now)
|
||||
|
||||
if seeding:
|
||||
self.store.record_delivery("announce", event.series_id, "seed", now)
|
||||
continue
|
||||
|
||||
if new_series:
|
||||
self._plan_announcement(event, now)
|
||||
|
||||
self.store.mark_gone([event.key for event in events], now)
|
||||
if first_run:
|
||||
self.store.set_meta("seeded", iso(now))
|
||||
log.info("first run: seeded %d events", len(events))
|
||||
|
||||
def _plan_announcement(self, event: Event, now: datetime) -> None:
|
||||
if not self.config.announce.enabled:
|
||||
return
|
||||
if event.end <= now:
|
||||
return
|
||||
horizon = now + timedelta(days=self.config.announce.max_days_ahead)
|
||||
if event.start > horizon:
|
||||
return
|
||||
if self.store.delivered("announce", event.series_id, "once"):
|
||||
return
|
||||
if self.store.queued("announce", event.series_id, "once"):
|
||||
return
|
||||
|
||||
start_local = event.start.astimezone(self.local_tz)
|
||||
deadline = (
|
||||
start_local
|
||||
if not event.all_day
|
||||
else start_local + timedelta(hours=12)
|
||||
)
|
||||
send_at = clamp_send_at(
|
||||
now.astimezone(self.local_tz),
|
||||
now.astimezone(self.local_tz),
|
||||
deadline,
|
||||
self.config.posting.windows,
|
||||
timedelta(minutes=self.config.posting.urgent_guard_minutes),
|
||||
)
|
||||
self.store.add_outbox(
|
||||
"announce",
|
||||
event.series_id,
|
||||
"once",
|
||||
event.key,
|
||||
send_at.astimezone(timezone.utc),
|
||||
now,
|
||||
)
|
||||
|
||||
def _plan_reminders(self, event: Event, now: datetime) -> None:
|
||||
if not self.config.reminders.enabled or event.end <= now:
|
||||
return
|
||||
for lead in self.config.reminders.lead_times:
|
||||
if self.store.plan_exists(event.key, lead):
|
||||
continue
|
||||
due = reminder_time(event, lead, now, self.config, self.local_tz)
|
||||
if due is None:
|
||||
continue
|
||||
self.store.add_plan(event.key, lead, due)
|
||||
self.store.add_outbox(
|
||||
"reminder",
|
||||
event.key,
|
||||
lead,
|
||||
event.key,
|
||||
due.astimezone(timezone.utc),
|
||||
now,
|
||||
)
|
||||
|
||||
# -- sending ---------------------------------------------------------
|
||||
|
||||
def _dispatch(self, now: datetime) -> None:
|
||||
items = self.store.due_outbox(now)
|
||||
if not items:
|
||||
return
|
||||
|
||||
grouped: dict[str, list[OutboxItem]] = {}
|
||||
for item in items:
|
||||
grouped.setdefault(item.event_key, []).append(item)
|
||||
|
||||
batch: list[tuple[Event, list[OutboxItem]]] = []
|
||||
for event_key, group in grouped.items():
|
||||
event = self.store.get_event(event_key)
|
||||
if event is None:
|
||||
for item in group:
|
||||
self.store.drop_outbox(item.id)
|
||||
continue
|
||||
|
||||
valid: list[OutboxItem] = []
|
||||
for item in group:
|
||||
usable = event.end > now and (
|
||||
item.kind != "reminder" or event.all_day or event.start > now
|
||||
)
|
||||
if usable:
|
||||
valid.append(item)
|
||||
else:
|
||||
log.info("dropping stale %s for %s", item.kind, event.summary)
|
||||
self.store.record_delivery(item.kind, item.ref, item.period, now)
|
||||
self.store.drop_outbox(item.id)
|
||||
if valid:
|
||||
batch.append((event, valid))
|
||||
|
||||
if not batch:
|
||||
return
|
||||
|
||||
message, attachments = self._compose(batch)
|
||||
try:
|
||||
self.sender.send(
|
||||
message,
|
||||
[self.config.signal.group_id],
|
||||
attachments=attachments,
|
||||
text_mode=self.config.signal.text_mode,
|
||||
)
|
||||
except SignalError as exc:
|
||||
log.error("send failed: %s", exc)
|
||||
self.watchdog.send_failed(str(exc), now)
|
||||
for _event, group in batch:
|
||||
for item in group:
|
||||
attempts = self.store.bump_attempts(item.id)
|
||||
if attempts >= MAX_SEND_ATTEMPTS:
|
||||
log.error("giving up on %s after %d attempts", item.ref, attempts)
|
||||
self.watchdog.alert(
|
||||
"outbox",
|
||||
f"Dropping a post after {attempts} failed attempts.",
|
||||
now,
|
||||
)
|
||||
self.store.drop_outbox(item.id)
|
||||
return
|
||||
|
||||
self.watchdog.send_ok()
|
||||
for event, group in batch:
|
||||
for item in group:
|
||||
self.store.record_delivery(item.kind, item.ref, item.period, now)
|
||||
if item.kind == "announce":
|
||||
self.store.mark_series_announced(item.ref, event.uid, now)
|
||||
self.store.drop_outbox(item.id)
|
||||
log.info("posted %d item(s) covering %d event(s)", len(items), len(batch))
|
||||
|
||||
def _compose(self, batch: list[tuple[Event, list[OutboxItem]]]) -> tuple[str, list[str]]:
|
||||
blocks: list[str] = []
|
||||
attachments: list[str] = []
|
||||
limit = self.config.images.max_images_per_post if len(batch) > 1 else 1
|
||||
|
||||
for event, _items in batch:
|
||||
info = self.scraper.details(event.url)
|
||||
venue = (
|
||||
venue_from_location(event.location)
|
||||
or info.venue
|
||||
or venue_from_description(event.description)
|
||||
)
|
||||
postcode = (
|
||||
find_postcode(event.location, event.description) or info.postcode
|
||||
)
|
||||
link = event.url
|
||||
if self.config.format.shorten_links and link:
|
||||
link = self._shorten(link)
|
||||
|
||||
blocks.append(
|
||||
render_post(
|
||||
event,
|
||||
self.config.format,
|
||||
self.local_tz,
|
||||
venue=venue,
|
||||
postcode=postcode,
|
||||
link=link,
|
||||
)
|
||||
)
|
||||
|
||||
if len(attachments) < limit:
|
||||
image_url = event.image_url or info.image_url
|
||||
data_uri = self.images.data_uri(image_url) if image_url else None
|
||||
if data_uri:
|
||||
attachments.append(data_uri)
|
||||
|
||||
return "\n\n".join(blocks), attachments
|
||||
|
||||
def _shorten(self, url: str) -> str:
|
||||
cached = self.store.get_short_link(url)
|
||||
if cached:
|
||||
return cached
|
||||
try:
|
||||
response = self.http.get(
|
||||
"https://tinyurl.com/api-create.php",
|
||||
params={"url": url},
|
||||
timeout=8,
|
||||
headers={"User-Agent": "calbot/0.1"},
|
||||
)
|
||||
if response.status_code == 200 and response.text.strip().startswith("http"):
|
||||
short = response.text.strip()
|
||||
self.store.put_short_link(url, short, datetime.now(timezone.utc))
|
||||
return short
|
||||
except httpx.HTTPError as exc:
|
||||
log.warning("link shortening failed: %s", exc)
|
||||
return url
|
||||
|
||||
# -- heartbeat -------------------------------------------------------
|
||||
|
||||
def _heartbeat(self, now: datetime) -> None:
|
||||
if not self.config.watchdog.daily_heartbeat:
|
||||
return
|
||||
local = now.astimezone(self.local_tz)
|
||||
today = local.date().isoformat()
|
||||
if local.hour < 9 or self.store.get_meta("last_heartbeat") == today:
|
||||
return
|
||||
try:
|
||||
self.sender.send(
|
||||
f"calbot alive \u2014 {self.store.count_events()} events in the window",
|
||||
[self.watchdog.recipient],
|
||||
text_mode=self.config.signal.text_mode,
|
||||
)
|
||||
except SignalError as exc:
|
||||
log.warning("heartbeat failed: %s", exc)
|
||||
return
|
||||
self.store.set_meta("last_heartbeat", today)
|
||||
@@ -0,0 +1,54 @@
|
||||
"""Minimal client for signal-cli-rest-api."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class SignalError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class SignalClient:
|
||||
def __init__(self, base_url: str, number: str, timeout: float = 60.0):
|
||||
self.base_url = base_url.rstrip("/")
|
||||
self.number = number
|
||||
self.http = httpx.Client(base_url=self.base_url, timeout=timeout)
|
||||
|
||||
def send(
|
||||
self,
|
||||
message: str,
|
||||
recipients: list[str],
|
||||
attachments: list[str] | None = None,
|
||||
text_mode: str = "normal",
|
||||
) -> None:
|
||||
payload: dict = {
|
||||
"message": message,
|
||||
"number": self.number,
|
||||
"recipients": recipients,
|
||||
"text_mode": text_mode,
|
||||
}
|
||||
if attachments:
|
||||
payload["base64_attachments"] = attachments
|
||||
|
||||
response = self.http.post("/v2/send", json=payload)
|
||||
if response.status_code >= 400:
|
||||
raise SignalError(
|
||||
f"send failed ({response.status_code}): {response.text[:500]}"
|
||||
)
|
||||
|
||||
def account_exists(self) -> bool:
|
||||
try:
|
||||
response = self.http.get("/v1/accounts")
|
||||
if response.status_code >= 400:
|
||||
return False
|
||||
return self.number in response.json()
|
||||
except (httpx.HTTPError, ValueError):
|
||||
return False
|
||||
|
||||
def close(self) -> None:
|
||||
self.http.close()
|
||||
@@ -0,0 +1,368 @@
|
||||
"""SQLite persistence: events, series, reminder plans, outbox, deliveries."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sqlite3
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
from .expand import Event
|
||||
|
||||
SCHEMA = """
|
||||
CREATE TABLE IF NOT EXISTS meta (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS series (
|
||||
series_id TEXT PRIMARY KEY,
|
||||
first_seen_at TEXT NOT NULL,
|
||||
announced_at TEXT,
|
||||
announced_uid TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS events (
|
||||
event_key TEXT PRIMARY KEY,
|
||||
uid TEXT NOT NULL,
|
||||
series_id TEXT NOT NULL,
|
||||
summary TEXT NOT NULL,
|
||||
start_utc TEXT NOT NULL,
|
||||
end_utc TEXT NOT NULL,
|
||||
all_day INTEGER NOT NULL,
|
||||
location TEXT NOT NULL DEFAULT '',
|
||||
url TEXT NOT NULL DEFAULT '',
|
||||
image_url TEXT NOT NULL DEFAULT '',
|
||||
event_tz TEXT NOT NULL DEFAULT 'UTC',
|
||||
status TEXT NOT NULL DEFAULT '',
|
||||
first_seen_at TEXT NOT NULL,
|
||||
last_seen_at TEXT NOT NULL,
|
||||
gone INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS events_start ON events(start_utc);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS reminder_plans (
|
||||
event_key TEXT NOT NULL,
|
||||
lead TEXT NOT NULL,
|
||||
due_at TEXT NOT NULL,
|
||||
PRIMARY KEY (event_key, lead)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS outbox (
|
||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||
kind TEXT NOT NULL,
|
||||
ref TEXT NOT NULL,
|
||||
period TEXT NOT NULL,
|
||||
event_key TEXT NOT NULL,
|
||||
send_at TEXT NOT NULL,
|
||||
attempts INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TEXT NOT NULL,
|
||||
UNIQUE (kind, ref, period)
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS outbox_due ON outbox(send_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS deliveries (
|
||||
kind TEXT NOT NULL,
|
||||
ref TEXT NOT NULL,
|
||||
period TEXT NOT NULL,
|
||||
sent_at TEXT NOT NULL,
|
||||
PRIMARY KEY (kind, ref, period)
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS pages (
|
||||
url TEXT PRIMARY KEY,
|
||||
image_url TEXT NOT NULL DEFAULT '',
|
||||
venue TEXT NOT NULL DEFAULT '',
|
||||
postcode TEXT NOT NULL DEFAULT '',
|
||||
fetched_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS short_links (
|
||||
url TEXT PRIMARY KEY,
|
||||
short_url TEXT NOT NULL,
|
||||
created_at TEXT NOT NULL
|
||||
);
|
||||
"""
|
||||
|
||||
|
||||
def iso(moment: datetime) -> str:
|
||||
return moment.astimezone(timezone.utc).isoformat()
|
||||
|
||||
|
||||
def from_iso(value: str) -> datetime:
|
||||
return datetime.fromisoformat(value)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OutboxItem:
|
||||
id: int
|
||||
kind: str
|
||||
ref: str
|
||||
period: str
|
||||
event_key: str
|
||||
send_at: datetime
|
||||
attempts: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PageInfo:
|
||||
image_url: str
|
||||
venue: str
|
||||
postcode: str
|
||||
|
||||
|
||||
class Store:
|
||||
def __init__(self, path: Path):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self.conn = sqlite3.connect(path, timeout=30)
|
||||
self.conn.row_factory = sqlite3.Row
|
||||
self.conn.execute("PRAGMA journal_mode=WAL")
|
||||
self.conn.execute("PRAGMA foreign_keys=ON")
|
||||
self.conn.executescript(SCHEMA)
|
||||
self.conn.commit()
|
||||
|
||||
def close(self) -> None:
|
||||
self.conn.close()
|
||||
|
||||
# -- meta ------------------------------------------------------------
|
||||
|
||||
def get_meta(self, key: str) -> str | None:
|
||||
row = self.conn.execute("SELECT value FROM meta WHERE key = ?", (key,)).fetchone()
|
||||
return row["value"] if row else None
|
||||
|
||||
def set_meta(self, key: str, value: str) -> None:
|
||||
self.conn.execute(
|
||||
"INSERT INTO meta(key, value) VALUES(?, ?) "
|
||||
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
|
||||
(key, value),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
# -- events ----------------------------------------------------------
|
||||
|
||||
def has_events(self) -> bool:
|
||||
row = self.conn.execute("SELECT 1 FROM events LIMIT 1").fetchone()
|
||||
return row is not None
|
||||
|
||||
def event_exists(self, event_key: str) -> bool:
|
||||
row = self.conn.execute(
|
||||
"SELECT 1 FROM events WHERE event_key = ?", (event_key,)
|
||||
).fetchone()
|
||||
return row is not None
|
||||
|
||||
def upsert_event(self, event: Event, now: datetime) -> bool:
|
||||
"""Insert or refresh an event. Returns True if it was new."""
|
||||
is_new = not self.event_exists(event.key)
|
||||
self.conn.execute(
|
||||
"""
|
||||
INSERT INTO events (
|
||||
event_key, uid, series_id, summary, start_utc, end_utc, all_day,
|
||||
location, url, image_url, event_tz, status, first_seen_at, last_seen_at, gone
|
||||
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0)
|
||||
ON CONFLICT(event_key) DO UPDATE SET
|
||||
summary = excluded.summary,
|
||||
start_utc = excluded.start_utc,
|
||||
end_utc = excluded.end_utc,
|
||||
all_day = excluded.all_day,
|
||||
location = excluded.location,
|
||||
url = excluded.url,
|
||||
image_url = excluded.image_url,
|
||||
status = excluded.status,
|
||||
last_seen_at = excluded.last_seen_at,
|
||||
gone = 0
|
||||
""",
|
||||
(
|
||||
event.key,
|
||||
event.uid,
|
||||
event.series_id,
|
||||
event.summary,
|
||||
iso(event.start),
|
||||
iso(event.end),
|
||||
int(event.all_day),
|
||||
event.location,
|
||||
event.url,
|
||||
event.image_url,
|
||||
event.tz,
|
||||
event.status,
|
||||
iso(now),
|
||||
iso(now),
|
||||
),
|
||||
)
|
||||
self.conn.commit()
|
||||
return is_new
|
||||
|
||||
def mark_gone(self, seen_keys: Iterable[str], now: datetime) -> None:
|
||||
seen = set(seen_keys)
|
||||
rows = self.conn.execute("SELECT event_key FROM events WHERE gone = 0").fetchall()
|
||||
for row in rows:
|
||||
if row["event_key"] not in seen:
|
||||
self.conn.execute(
|
||||
"UPDATE events SET gone = 1, last_seen_at = ? WHERE event_key = ?",
|
||||
(iso(now), row["event_key"]),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
def get_event(self, event_key: str) -> Event | None:
|
||||
row = self.conn.execute(
|
||||
"SELECT * FROM events WHERE event_key = ?", (event_key,)
|
||||
).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return Event(
|
||||
key=row["event_key"],
|
||||
uid=row["uid"],
|
||||
series_id=row["series_id"],
|
||||
summary=row["summary"],
|
||||
start=from_iso(row["start_utc"]),
|
||||
end=from_iso(row["end_utc"]),
|
||||
all_day=bool(row["all_day"]),
|
||||
location=row["location"],
|
||||
url=row["url"],
|
||||
image_url=row["image_url"],
|
||||
tz=row["event_tz"],
|
||||
status=row["status"],
|
||||
)
|
||||
|
||||
def count_events(self) -> int:
|
||||
row = self.conn.execute("SELECT COUNT(*) AS n FROM events WHERE gone = 0").fetchone()
|
||||
return int(row["n"])
|
||||
|
||||
# -- series ----------------------------------------------------------
|
||||
|
||||
def create_series(self, series_id: str, now: datetime) -> bool:
|
||||
cursor = self.conn.execute(
|
||||
"INSERT OR IGNORE INTO series(series_id, first_seen_at) VALUES(?, ?)",
|
||||
(series_id, iso(now)),
|
||||
)
|
||||
self.conn.commit()
|
||||
return cursor.rowcount > 0
|
||||
|
||||
def mark_series_announced(self, series_id: str, uid: str, now: datetime) -> None:
|
||||
self.conn.execute(
|
||||
"UPDATE series SET announced_at = ?, announced_uid = ? WHERE series_id = ?",
|
||||
(iso(now), uid, series_id),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
# -- reminder plans --------------------------------------------------
|
||||
|
||||
def plan_exists(self, event_key: str, lead: str) -> bool:
|
||||
row = self.conn.execute(
|
||||
"SELECT 1 FROM reminder_plans WHERE event_key = ? AND lead = ?",
|
||||
(event_key, lead),
|
||||
).fetchone()
|
||||
return row is not None
|
||||
|
||||
def add_plan(self, event_key: str, lead: str, due_at: datetime) -> None:
|
||||
self.conn.execute(
|
||||
"INSERT OR IGNORE INTO reminder_plans(event_key, lead, due_at) VALUES(?, ?, ?)",
|
||||
(event_key, lead, iso(due_at)),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
# -- outbox ----------------------------------------------------------
|
||||
|
||||
def add_outbox(
|
||||
self,
|
||||
kind: str,
|
||||
ref: str,
|
||||
period: str,
|
||||
event_key: str,
|
||||
send_at: datetime,
|
||||
now: datetime,
|
||||
) -> None:
|
||||
self.conn.execute(
|
||||
"""
|
||||
INSERT OR IGNORE INTO outbox(kind, ref, period, event_key, send_at, created_at)
|
||||
VALUES (?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(kind, ref, period, event_key, iso(send_at), iso(now)),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
def due_outbox(self, now: datetime) -> list[OutboxItem]:
|
||||
rows = self.conn.execute(
|
||||
"SELECT * FROM outbox WHERE send_at <= ? ORDER BY send_at, id",
|
||||
(iso(now),),
|
||||
).fetchall()
|
||||
return [
|
||||
OutboxItem(
|
||||
id=row["id"],
|
||||
kind=row["kind"],
|
||||
ref=row["ref"],
|
||||
period=row["period"],
|
||||
event_key=row["event_key"],
|
||||
send_at=from_iso(row["send_at"]),
|
||||
attempts=row["attempts"],
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
|
||||
def bump_attempts(self, item_id: int) -> int:
|
||||
self.conn.execute("UPDATE outbox SET attempts = attempts + 1 WHERE id = ?", (item_id,))
|
||||
self.conn.commit()
|
||||
row = self.conn.execute("SELECT attempts FROM outbox WHERE id = ?", (item_id,)).fetchone()
|
||||
return int(row["attempts"]) if row else 0
|
||||
|
||||
def drop_outbox(self, item_id: int) -> None:
|
||||
self.conn.execute("DELETE FROM outbox WHERE id = ?", (item_id,))
|
||||
self.conn.commit()
|
||||
|
||||
# -- deliveries ------------------------------------------------------
|
||||
|
||||
def delivered(self, kind: str, ref: str, period: str) -> bool:
|
||||
row = self.conn.execute(
|
||||
"SELECT 1 FROM deliveries WHERE kind = ? AND ref = ? AND period = ?",
|
||||
(kind, ref, period),
|
||||
).fetchone()
|
||||
return row is not None
|
||||
|
||||
def record_delivery(self, kind: str, ref: str, period: str, now: datetime) -> None:
|
||||
self.conn.execute(
|
||||
"INSERT OR REPLACE INTO deliveries(kind, ref, period, sent_at) VALUES(?, ?, ?, ?)",
|
||||
(kind, ref, period, iso(now)),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
def queued(self, kind: str, ref: str, period: str) -> bool:
|
||||
row = self.conn.execute(
|
||||
"SELECT 1 FROM outbox WHERE kind = ? AND ref = ? AND period = ?",
|
||||
(kind, ref, period),
|
||||
).fetchone()
|
||||
return row is not None
|
||||
|
||||
# -- scraped page cache ----------------------------------------------
|
||||
|
||||
def get_page(self, url: str) -> PageInfo | None:
|
||||
row = self.conn.execute("SELECT * FROM pages WHERE url = ?", (url,)).fetchone()
|
||||
if row is None:
|
||||
return None
|
||||
return PageInfo(
|
||||
image_url=row["image_url"], venue=row["venue"], postcode=row["postcode"]
|
||||
)
|
||||
|
||||
def put_page(self, url: str, info: PageInfo, now: datetime) -> None:
|
||||
self.conn.execute(
|
||||
"INSERT OR REPLACE INTO pages(url, image_url, venue, postcode, fetched_at) "
|
||||
"VALUES (?, ?, ?, ?, ?)",
|
||||
(url, info.image_url, info.venue, info.postcode, iso(now)),
|
||||
)
|
||||
self.conn.commit()
|
||||
|
||||
# -- short links -----------------------------------------------------
|
||||
|
||||
def get_short_link(self, url: str) -> str | None:
|
||||
row = self.conn.execute(
|
||||
"SELECT short_url FROM short_links WHERE url = ?", (url,)
|
||||
).fetchone()
|
||||
return row["short_url"] if row else None
|
||||
|
||||
def put_short_link(self, url: str, short_url: str, now: datetime) -> None:
|
||||
self.conn.execute(
|
||||
"INSERT OR REPLACE INTO short_links(url, short_url, created_at) VALUES (?, ?, ?)",
|
||||
(url, short_url, iso(now)),
|
||||
)
|
||||
self.conn.commit()
|
||||
@@ -0,0 +1,63 @@
|
||||
"""Watchdog: tell the operator when things stop working."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
from .config import WatchdogConfig
|
||||
from .signal_client import SignalClient
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class Watchdog:
|
||||
def __init__(
|
||||
self,
|
||||
config: WatchdogConfig,
|
||||
sender: SignalClient,
|
||||
fallback_number: str,
|
||||
text_mode: str = "normal",
|
||||
):
|
||||
self.config = config
|
||||
self.sender = sender
|
||||
self.recipient = config.notify_number or fallback_number
|
||||
self.text_mode = text_mode
|
||||
self.feed_failures = 0
|
||||
self.send_failures = 0
|
||||
self._last_alert: dict[str, datetime] = {}
|
||||
|
||||
def feed_ok(self) -> None:
|
||||
self.feed_failures = 0
|
||||
|
||||
def feed_failed(self, reason: str, now: datetime) -> None:
|
||||
self.feed_failures += 1
|
||||
if self.feed_failures >= self.config.feed_failures:
|
||||
self.alert("feed", f"Calendar feed is failing: {reason}", now)
|
||||
|
||||
def send_ok(self) -> None:
|
||||
self.send_failures = 0
|
||||
|
||||
def send_failed(self, reason: str, now: datetime) -> None:
|
||||
self.send_failures += 1
|
||||
if self.send_failures >= self.config.send_failures:
|
||||
self.alert("send", f"Signal sending is failing: {reason}", now)
|
||||
|
||||
def alert(self, key: str, text: str, now: datetime) -> None:
|
||||
if not self.config.enabled:
|
||||
log.error("watchdog (disabled): %s", text)
|
||||
return
|
||||
last = self._last_alert.get(key)
|
||||
cooldown = timedelta(hours=self.config.alert_cooldown_hours)
|
||||
if last is not None and now - last < cooldown:
|
||||
return
|
||||
self._last_alert[key] = now
|
||||
try:
|
||||
self.sender.send(
|
||||
f"\u26a0\ufe0f calbot\n{text}",
|
||||
[self.recipient],
|
||||
text_mode=self.text_mode,
|
||||
)
|
||||
log.warning("watchdog alert sent: %s", text)
|
||||
except Exception as exc: # noqa: BLE001 - never crash the loop
|
||||
log.error("watchdog could not send alert (%s): %s", exc, text)
|
||||
Vendored
+49
@@ -0,0 +1,49 @@
|
||||
BEGIN:VCALENDAR
|
||||
VERSION:2.0
|
||||
PRODID:-//MCR Commons Calendar - ECPv6.18.2//NONSGML v1.0//EN
|
||||
BEGIN:VTIMEZONE
|
||||
TZID:Europe/London
|
||||
BEGIN:STANDARD
|
||||
TZOFFSETFROM:+0100
|
||||
TZOFFSETTO:+0000
|
||||
TZNAME:GMT
|
||||
DTSTART:20251026T010000
|
||||
END:STANDARD
|
||||
BEGIN:DAYLIGHT
|
||||
TZOFFSETFROM:+0000
|
||||
TZOFFSETTO:+0100
|
||||
TZNAME:BST
|
||||
DTSTART:20260329T010000
|
||||
END:DAYLIGHT
|
||||
END:VTIMEZONE
|
||||
BEGIN:VEVENT
|
||||
DTSTART;TZID=Europe/London:20261120T090000
|
||||
DTEND;TZID=Europe/London:20261120T100000
|
||||
DTSTAMP:20261116T192444Z
|
||||
UID:1839-1795165200-1795168800@commonscalendar.uk
|
||||
SUMMARY:Clean up Greater Manchester Pension fund
|
||||
DESCRIPTION:Join us to lobby for GM Pensions free from arms and fossil fuel holdings.
|
||||
URL:https://commonscalendar.uk/event/clean-up-greater-manchester-pension-fund/
|
||||
CATEGORIES:Demonstration
|
||||
ATTACH;FMTTYPE=image/png:https://commonscalendar.uk/wp-content/uploads/2026/11/inbound5975249048432321891.png
|
||||
END:VEVENT
|
||||
BEGIN:VEVENT
|
||||
DTSTART;VALUE=DATE:20261121
|
||||
DTEND;VALUE=DATE:20261122
|
||||
DTSTAMP:20261116T192444Z
|
||||
UID:1902-1795257600-1795344000@commonscalendar.uk
|
||||
SUMMARY:All day gathering
|
||||
DESCRIPTION:Come along any time.
|
||||
URL:https://commonscalendar.uk/event/all-day-gathering/
|
||||
END:VEVENT
|
||||
BEGIN:VEVENT
|
||||
DTSTART;TZID=Europe/London:20261127T090000
|
||||
DTEND;TZID=Europe/London:20261127T100000
|
||||
DTSTAMP:20261116T192444Z
|
||||
UID:1839-1795770000-1795773600@commonscalendar.uk
|
||||
SUMMARY:Clean up Greater Manchester Pension fund
|
||||
DESCRIPTION:Second occurrence, same series.
|
||||
URL:https://commonscalendar.uk/event/clean-up-greater-manchester-pension-fund/
|
||||
ATTACH;FMTTYPE=image/webp:https://commonscalendar.uk/wp-content/uploads/2026/11/poster.webp
|
||||
END:VEVENT
|
||||
END:VCALENDAR
|
||||
@@ -0,0 +1,49 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
from calbot.expand import parse_events
|
||||
|
||||
FIXTURE = Path(__file__).parent / "fixtures" / "tec-like.ics.txt"
|
||||
|
||||
|
||||
def load_bytes() -> bytes:
|
||||
return FIXTURE.read_bytes()
|
||||
|
||||
|
||||
def test_parses_tec_style_feed():
|
||||
events = parse_events(
|
||||
load_bytes(),
|
||||
datetime(2026, 11, 19, tzinfo=timezone.utc),
|
||||
datetime(2026, 12, 1, tzinfo=timezone.utc),
|
||||
"Europe/London",
|
||||
)
|
||||
assert len(events) == 3
|
||||
|
||||
first, all_day, third = events
|
||||
|
||||
assert first.summary == "Clean up Greater Manchester Pension fund"
|
||||
assert first.series_id == "tec:1839"
|
||||
assert not first.all_day
|
||||
assert first.image_url.endswith("inbound5975249048432321891.png")
|
||||
assert first.url.endswith("clean-up-greater-manchester-pension-fund/")
|
||||
# 09:00 London on 20 Nov 2026 is GMT (UTC).
|
||||
assert first.start == datetime(2026, 11, 20, 9, 0, tzinfo=timezone.utc)
|
||||
|
||||
assert all_day.all_day
|
||||
assert all_day.series_id == "tec:1902"
|
||||
assert all_day.start == datetime(2026, 11, 21, 0, 0, tzinfo=timezone.utc)
|
||||
|
||||
assert third.series_id == "tec:1839"
|
||||
assert third.image_url.endswith("poster.webp")
|
||||
|
||||
|
||||
def test_window_excludes_far_future():
|
||||
events = parse_events(
|
||||
load_bytes(),
|
||||
datetime(2026, 11, 19, tzinfo=timezone.utc),
|
||||
datetime(2026, 11, 20, tzinfo=timezone.utc),
|
||||
"Europe/London",
|
||||
)
|
||||
assert len(events) <= 1
|
||||
@@ -0,0 +1,98 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from calbot.config import FormatConfig
|
||||
from calbot.expand import Event
|
||||
from calbot.format import (
|
||||
find_postcode,
|
||||
format_hashtags,
|
||||
format_time_range,
|
||||
render_post,
|
||||
venue_from_description,
|
||||
venue_from_location,
|
||||
)
|
||||
|
||||
LONDON = ZoneInfo("Europe/London")
|
||||
|
||||
|
||||
def make_event(**overrides) -> Event:
|
||||
defaults = dict(
|
||||
key="uid|2026-11-14T16:30:00+00:00",
|
||||
uid="uid",
|
||||
series_id="tec:1",
|
||||
summary="Folk Night",
|
||||
start=datetime(2026, 11, 14, 16, 30, tzinfo=LONDON).astimezone(timezone.utc),
|
||||
end=datetime(2026, 11, 14, 18, 30, tzinfo=LONDON).astimezone(timezone.utc),
|
||||
all_day=False,
|
||||
location="Old Print Works\n12 High St\nBirmingham\nB12 9AH",
|
||||
description="",
|
||||
url="https://commonscalendar.uk/event/folk-night/",
|
||||
image_url="",
|
||||
tz="Europe/London",
|
||||
)
|
||||
defaults.update(overrides)
|
||||
return Event(**defaults)
|
||||
|
||||
|
||||
def test_render_matches_agreed_format():
|
||||
text = render_post(
|
||||
make_event(),
|
||||
FormatConfig(),
|
||||
LONDON,
|
||||
venue="Old Print Works",
|
||||
postcode="B12 9AH",
|
||||
link="https://commonscalendar.uk/event/folk-night/",
|
||||
)
|
||||
assert text == (
|
||||
"Folk Night\n"
|
||||
"#November #November14\n"
|
||||
"4.30-6.30pm\n"
|
||||
"Old Print Works B12 9AH\n"
|
||||
"https://commonscalendar.uk/event/folk-night/"
|
||||
)
|
||||
|
||||
|
||||
def test_time_range_same_meridiem():
|
||||
start = datetime(2026, 11, 14, 16, 30, tzinfo=LONDON)
|
||||
end = datetime(2026, 11, 14, 18, 30, tzinfo=LONDON)
|
||||
assert format_time_range(start, end) == "4.30-6.30pm"
|
||||
|
||||
|
||||
def test_time_range_crosses_meridiem():
|
||||
start = datetime(2026, 11, 14, 11, 30, tzinfo=LONDON)
|
||||
end = datetime(2026, 11, 14, 13, 30, tzinfo=LONDON)
|
||||
assert format_time_range(start, end) == "11.30am-1.30pm"
|
||||
|
||||
|
||||
def test_hashtags():
|
||||
assert format_hashtags(datetime(2026, 11, 14, 10, 0, tzinfo=LONDON)) == (
|
||||
"#November #November14"
|
||||
)
|
||||
assert format_hashtags(datetime(2026, 1, 1, 10, 0, tzinfo=LONDON)) == "#January #January1"
|
||||
|
||||
|
||||
def test_all_day_event_says_all_day():
|
||||
event = make_event(
|
||||
all_day=True,
|
||||
start=datetime(2026, 11, 21, tzinfo=LONDON).astimezone(timezone.utc),
|
||||
end=datetime(2026, 11, 22, tzinfo=LONDON).astimezone(timezone.utc),
|
||||
)
|
||||
text = render_post(event, FormatConfig(), LONDON, venue="", postcode="", link="")
|
||||
assert "All day" in text
|
||||
assert "#November #November21" in text
|
||||
|
||||
|
||||
def test_postcode_and_venue_helpers():
|
||||
assert find_postcode("Old Print Works, B12 9AH") == "B12 9AH"
|
||||
assert find_postcode("no code here") == ""
|
||||
assert venue_from_location("Old Print Works\nB12 9AH") == "Old Print Works"
|
||||
assert venue_from_description("Collier Room, Methodist Central Hall, M1 1JQ") == "Collier Room"
|
||||
|
||||
|
||||
def test_missing_fields_are_omitted():
|
||||
event = make_event(location="", url="")
|
||||
text = render_post(event, FormatConfig(), LONDON, venue="", postcode="", link="")
|
||||
lines = text.splitlines()
|
||||
assert lines == ["Folk Night", "#November #November14", "4.30-6.30pm"]
|
||||
@@ -0,0 +1,246 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, time, timedelta, timezone
|
||||
from pathlib import Path
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import httpx
|
||||
|
||||
from calbot.config import (
|
||||
AnnounceConfig,
|
||||
CalendarConfig,
|
||||
Config,
|
||||
FormatConfig,
|
||||
ImagesConfig,
|
||||
PostingConfig,
|
||||
RemindersConfig,
|
||||
SignalConfig,
|
||||
WatchdogConfig,
|
||||
)
|
||||
from calbot.expand import Event
|
||||
from calbot.feed import Feed, normalize_feed_url
|
||||
from calbot.scheduler import (
|
||||
Bot,
|
||||
clamp_send_at,
|
||||
in_window,
|
||||
next_window_start,
|
||||
reminder_time,
|
||||
)
|
||||
from calbot.store import PageInfo, Store
|
||||
from calbot.watchdog import Watchdog
|
||||
|
||||
LONDON = ZoneInfo("Europe/London")
|
||||
UTC = timezone.utc
|
||||
WINDOWS = ((time(8, 0), time(10, 0)), (time(18, 0), time(21, 0)))
|
||||
GUARD = timedelta(hours=1)
|
||||
|
||||
|
||||
def local(*args) -> datetime:
|
||||
return datetime(*args, tzinfo=LONDON)
|
||||
|
||||
|
||||
def make_event(
|
||||
*,
|
||||
uid: str = "1",
|
||||
series_id: str = "tec:1",
|
||||
summary: str = "Folk Night",
|
||||
start: datetime,
|
||||
end: datetime | None = None,
|
||||
all_day: bool = False,
|
||||
) -> Event:
|
||||
return Event(
|
||||
key=f"{uid}|{start.astimezone(UTC).isoformat()}",
|
||||
uid=uid,
|
||||
series_id=series_id,
|
||||
summary=summary,
|
||||
start=start.astimezone(UTC),
|
||||
end=(end or start + timedelta(hours=2)).astimezone(UTC),
|
||||
all_day=all_day,
|
||||
location="Old Print Works\nB12 9AH",
|
||||
url="https://commonscalendar.uk/event/folk-night/",
|
||||
tz="Europe/London",
|
||||
)
|
||||
|
||||
|
||||
def make_config(tmp_path, **overrides) -> Config:
|
||||
defaults = dict(
|
||||
signal=SignalConfig(
|
||||
api_url="http://signal-api:8080", number="+447700900000", group_id="group.test"
|
||||
),
|
||||
calendar=CalendarConfig(url="https://example.test/feed.ics", timezone="Europe/London"),
|
||||
posting=PostingConfig(),
|
||||
announce=AnnounceConfig(),
|
||||
reminders=RemindersConfig(),
|
||||
images=ImagesConfig(),
|
||||
format=FormatConfig(),
|
||||
watchdog=WatchdogConfig(enabled=False),
|
||||
db_path=tmp_path / "db.sqlite3",
|
||||
cache_dir=tmp_path / "cache",
|
||||
)
|
||||
defaults.update(overrides)
|
||||
return Config(**defaults)
|
||||
|
||||
|
||||
# -- window helpers ---------------------------------------------------------
|
||||
|
||||
|
||||
def test_webcal_is_fetched_over_https():
|
||||
assert normalize_feed_url("webcal://example.test/x.ics") == "https://example.test/x.ics"
|
||||
assert normalize_feed_url("https://example.test/x.ics") == "https://example.test/x.ics"
|
||||
|
||||
|
||||
def test_in_window():
|
||||
assert in_window(local(2026, 11, 18, 9, 0), WINDOWS)
|
||||
assert not in_window(local(2026, 11, 18, 15, 0), WINDOWS)
|
||||
assert not in_window(local(2026, 11, 18, 21, 0), WINDOWS)
|
||||
|
||||
|
||||
def test_next_window_start():
|
||||
assert next_window_start(local(2026, 11, 18, 7, 0), WINDOWS) == local(2026, 11, 18, 8, 0)
|
||||
assert next_window_start(local(2026, 11, 18, 10, 30), WINDOWS) == local(2026, 11, 18, 18, 0)
|
||||
assert next_window_start(local(2026, 11, 18, 21, 30), WINDOWS) == local(2026, 11, 19, 8, 0)
|
||||
|
||||
|
||||
def test_clamp_defers_to_evening():
|
||||
due = local(2026, 11, 18, 14, 0)
|
||||
assert clamp_send_at(due, due, local(2026, 11, 25, 18, 0), WINDOWS, GUARD) == local(
|
||||
2026, 11, 18, 18, 0
|
||||
)
|
||||
|
||||
|
||||
def test_clamp_respects_urgent_guard():
|
||||
due = local(2026, 11, 18, 18, 30)
|
||||
start = local(2026, 11, 18, 19, 0)
|
||||
# Next window is tomorrow 08:00, which is after the event: send now.
|
||||
assert clamp_send_at(due, due, start, WINDOWS, GUARD) == due
|
||||
|
||||
|
||||
# -- reminder planning ------------------------------------------------------
|
||||
|
||||
|
||||
def test_reminder_three_days_before_lands_in_window():
|
||||
event = make_event(start=local(2026, 11, 21, 14, 0))
|
||||
config = make_config(Path("/tmp"), posting=PostingConfig())
|
||||
due = reminder_time(event, "3d", local(2026, 11, 10, 12, 0), config, LONDON)
|
||||
assert due == local(2026, 11, 18, 18, 0)
|
||||
|
||||
|
||||
def test_reminder_falls_back_to_day_of():
|
||||
event = make_event(start=local(2026, 11, 20, 14, 0))
|
||||
config = make_config(Path("/tmp"))
|
||||
due = reminder_time(event, "3d", local(2026, 11, 19, 12, 0), config, LONDON)
|
||||
assert due == local(2026, 11, 20, 9, 0)
|
||||
|
||||
|
||||
def test_reminder_day_of_moves_before_early_event():
|
||||
event = make_event(start=local(2026, 11, 20, 8, 30))
|
||||
config = make_config(Path("/tmp"))
|
||||
due = reminder_time(event, "3d", local(2026, 11, 19, 12, 0), config, LONDON)
|
||||
assert due == local(2026, 11, 20, 7, 30)
|
||||
|
||||
|
||||
def test_reminder_disabled_fallback_returns_none():
|
||||
event = make_event(start=local(2026, 11, 20, 14, 0))
|
||||
config = make_config(
|
||||
Path("/tmp"), reminders=RemindersConfig(fallback_to_day_of=False)
|
||||
)
|
||||
assert reminder_time(event, "3d", local(2026, 11, 19, 12, 0), config, LONDON) is None
|
||||
|
||||
|
||||
# -- end to end -------------------------------------------------------------
|
||||
|
||||
|
||||
class FakeSender:
|
||||
def __init__(self):
|
||||
self.sent: list[dict] = []
|
||||
|
||||
def send(self, message, recipients, attachments=None, text_mode="normal"):
|
||||
self.sent.append(
|
||||
{"message": message, "recipients": recipients, "attachments": attachments}
|
||||
)
|
||||
|
||||
|
||||
class FakeImages:
|
||||
def data_uri(self, url):
|
||||
return None
|
||||
|
||||
|
||||
class FakeScraper:
|
||||
def details(self, url):
|
||||
return PageInfo(image_url="", venue="", postcode="")
|
||||
|
||||
|
||||
def build_bot(tmp_path, sender):
|
||||
config = make_config(tmp_path)
|
||||
store = Store(config.db_path)
|
||||
http = httpx.Client()
|
||||
bot = Bot(
|
||||
config=config,
|
||||
store=store,
|
||||
feed=Feed(config.calendar.url, store, 15, http),
|
||||
sender=sender,
|
||||
images=FakeImages(),
|
||||
scraper=FakeScraper(),
|
||||
watchdog=Watchdog(config.watchdog, sender, "+447700900000"),
|
||||
http=http,
|
||||
)
|
||||
return bot, store
|
||||
|
||||
|
||||
def test_first_run_seeds_silently_then_announces_new_series(tmp_path):
|
||||
sender = FakeSender()
|
||||
bot, store = build_bot(tmp_path, sender)
|
||||
|
||||
existing = make_event(uid="1", series_id="tec:1", start=local(2026, 11, 20, 9, 0))
|
||||
bot._sync([existing], local(2026, 11, 18, 12, 0))
|
||||
assert store.due_outbox(local(2026, 11, 18, 18, 0)) == []
|
||||
|
||||
newcomer = make_event(
|
||||
uid="2", series_id="tec:2", summary="Film Night", start=local(2026, 11, 25, 19, 0)
|
||||
)
|
||||
bot._sync([existing, newcomer], local(2026, 11, 18, 12, 1))
|
||||
pending = store.due_outbox(local(2026, 11, 18, 18, 0))
|
||||
assert [item.kind for item in pending] == ["announce"]
|
||||
|
||||
bot._dispatch(local(2026, 11, 18, 18, 0))
|
||||
assert len(sender.sent) == 1
|
||||
assert sender.sent[0]["recipients"] == ["group.test"]
|
||||
assert sender.sent[0]["message"].startswith("Film Night")
|
||||
assert "#November #November25" in sender.sent[0]["message"]
|
||||
|
||||
# A later occurrence of the same series must not be announced again.
|
||||
third = make_event(
|
||||
uid="3", series_id="tec:2", summary="Film Night", start=local(2026, 12, 2, 19, 0)
|
||||
)
|
||||
bot._sync([existing, newcomer, third], local(2026, 11, 18, 18, 5))
|
||||
assert store.due_outbox(local(2026, 11, 18, 18, 10)) == []
|
||||
|
||||
|
||||
def test_reminder_is_sent_at_its_window(tmp_path):
|
||||
sender = FakeSender()
|
||||
bot, store = build_bot(tmp_path, sender)
|
||||
|
||||
event = make_event(uid="9", series_id="tec:9", start=local(2026, 11, 25, 14, 0))
|
||||
bot._sync([event], local(2026, 11, 18, 12, 0)) # seeds; still plans reminders
|
||||
bot._dispatch(local(2026, 11, 18, 18, 0))
|
||||
assert sender.sent == []
|
||||
|
||||
# Three days before is 22 Nov 14:00, deferred to the 18:00 window.
|
||||
bot._dispatch(local(2026, 11, 22, 18, 5))
|
||||
assert len(sender.sent) == 1
|
||||
assert sender.sent[0]["message"].startswith("Folk Night")
|
||||
|
||||
# Idempotent: dispatching again must not resend.
|
||||
bot._dispatch(local(2026, 11, 22, 18, 10))
|
||||
assert len(sender.sent) == 1
|
||||
|
||||
|
||||
def test_stale_reminder_is_dropped_not_sent(tmp_path):
|
||||
sender = FakeSender()
|
||||
bot, store = build_bot(tmp_path, sender)
|
||||
|
||||
event = make_event(uid="10", series_id="tec:10", start=local(2026, 11, 18, 15, 0))
|
||||
bot._sync([event], local(2026, 11, 18, 12, 0))
|
||||
bot._dispatch(local(2026, 11, 18, 18, 0)) # event already started
|
||||
assert sender.sent == []
|
||||
assert store.due_outbox(local(2026, 11, 18, 18, 5)) == []
|
||||
Reference in New Issue
Block a user