Initial signal-cal-bot skeleton: iCalendar feed to Signal group
- feed fetch with ETag/Last-Modified caching - iCalendar parsing/expansion with TEC series detection - posting-window scheduler with 3-day reminders and day-of fallback - image attachments from ATTACH, downscaled via Pillow - SQLite outbox with idempotent deliveries - watchdog alerts to Note to Self - Docker Compose with signal-cli-rest-api
This commit is contained in:
+23
@@ -0,0 +1,23 @@
|
|||||||
|
# iCal feeds / exports — never commit these
|
||||||
|
*.ics
|
||||||
|
*.icsx
|
||||||
|
|
||||||
|
# Runtime state
|
||||||
|
cache/
|
||||||
|
data/
|
||||||
|
signal-data/
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
.pytest_cache/
|
||||||
|
.ruff_cache/
|
||||||
|
dist/
|
||||||
|
build/
|
||||||
|
*.egg-info/
|
||||||
|
|
||||||
|
# Local config holds real phone numbers and group IDs
|
||||||
|
config.toml
|
||||||
|
.env
|
||||||
+16
@@ -0,0 +1,16 @@
|
|||||||
|
FROM python:3.12-slim
|
||||||
|
|
||||||
|
ENV PYTHONUNBUFFERED=1 \
|
||||||
|
PYTHONDONTWRITEBYTECODE=1 \
|
||||||
|
PIP_NO_CACHE_DIR=1
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
COPY pyproject.toml ./
|
||||||
|
COPY src ./src
|
||||||
|
RUN pip install .
|
||||||
|
|
||||||
|
# The compose file runs the bot as uid/gid 1000; these must be writable.
|
||||||
|
RUN mkdir -p /app/data /app/cache && chown -R 1000:1000 /app/data /app/cache
|
||||||
|
|
||||||
|
CMD ["python", "-m", "calbot", "--config", "/app/config.toml"]
|
||||||
@@ -0,0 +1,126 @@
|
|||||||
|
# calbot — iCalendar → Signal
|
||||||
|
|
||||||
|
Watches an iCalendar feed (built for The Events Calendar / MCR Commons) and posts
|
||||||
|
events to a Signal group, with the event's poster image attached.
|
||||||
|
|
||||||
|
Posts look like this:
|
||||||
|
|
||||||
|
```
|
||||||
|
Folk Night
|
||||||
|
#November #November14
|
||||||
|
4.30-6.30pm
|
||||||
|
Old Print Works B12 9AH
|
||||||
|
https://commonscalendar.uk/event/folk-night/
|
||||||
|
```
|
||||||
|
|
||||||
|
## How it behaves
|
||||||
|
|
||||||
|
- **Announcements** — a newly seen event is posted within one refresh interval,
|
||||||
|
deferred to the next posting window (`08:00–10:00` or `18:00–21:00` by default).
|
||||||
|
- **Reminders** — every event is posted again 3 days before it starts. If the
|
||||||
|
event was added less than 3 days ahead, the reminder goes out on the morning
|
||||||
|
of the event instead. An imminent event breaks the posting window rather than
|
||||||
|
arriving late.
|
||||||
|
- **Images** — taken from the feed's `ATTACH` property (The Events Calendar
|
||||||
|
emits the featured image), falling back to the event page's JSON-LD / `og:image`.
|
||||||
|
Every post carries an image.
|
||||||
|
- **Recurring events** — the feed pre-expands occurrences, but announcements are
|
||||||
|
deduplicated per series, so adding a weekly event produces one announcement.
|
||||||
|
- **First run** — seeds silently (`announce.backfill = false`); reminders still
|
||||||
|
work for everything already in the feed.
|
||||||
|
- **Watchdog** — messages you on Signal if the feed stops working, parses to
|
||||||
|
nothing, or sending keeps failing.
|
||||||
|
|
||||||
|
## Setup
|
||||||
|
|
||||||
|
### 1. Link a Signal account
|
||||||
|
|
||||||
|
Linking as a secondary device is the quickest route: the bot posts **as your own
|
||||||
|
number**. For a separate bot identity you must register a dedicated number
|
||||||
|
instead (see `signal-cli-rest-api` docs).
|
||||||
|
|
||||||
|
```bash
|
||||||
|
mkdir -p signal-data data cache
|
||||||
|
docker compose up -d signal-api
|
||||||
|
```
|
||||||
|
|
||||||
|
Open <http://localhost:8080/v1/qrcodelink?device_name=calbot> and scan the QR
|
||||||
|
code from **Signal → Settings → Linked devices**.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# the linked number:
|
||||||
|
curl -s http://localhost:8080/v1/accounts
|
||||||
|
|
||||||
|
# groups for that number — copy the internal_id (starts with "group."):
|
||||||
|
curl -s http://localhost:8080/v1/groups/+447700900000 | jq
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Configure
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cp config.example.toml config.toml
|
||||||
|
chmod 600 config.toml
|
||||||
|
$EDITOR config.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
Fill in `signal.number`, `signal.group_id`, and `calendar.url`. `webcal://`
|
||||||
|
URLs are accepted and fetched over https.
|
||||||
|
|
||||||
|
### 3. Check, preview, run
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# is the account linked and the config valid?
|
||||||
|
docker compose run --rm bot python -m calbot --check
|
||||||
|
|
||||||
|
# print the rendered posts for the feed without sending anything
|
||||||
|
docker compose run --rm bot python -m calbot --dump
|
||||||
|
|
||||||
|
# start the bot
|
||||||
|
docker compose up -d bot
|
||||||
|
docker compose logs -f bot
|
||||||
|
```
|
||||||
|
|
||||||
|
`--once` runs a single tick and exits.
|
||||||
|
|
||||||
|
## Install by hand
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m venv .venv && . .venv/bin/activate
|
||||||
|
pip install -e '.[dev]'
|
||||||
|
calbot --config config.toml --dump
|
||||||
|
calbot --config config.toml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Files that matter
|
||||||
|
|
||||||
|
| Path | Purpose |
|
||||||
|
|---|---|
|
||||||
|
| `config.toml` | numbers, group, feed, behaviour (never committed) |
|
||||||
|
| `signal-data/` | linked-device keys — **back this up**, losing it means re-linking |
|
||||||
|
| `data/calbot.sqlite3` | event/outbox/delivery state (safe to delete to re-seed) |
|
||||||
|
| `cache/images/` | downscaled posters |
|
||||||
|
|
||||||
|
Deleting `data/calbot.sqlite3` resets dedupe state: on the next run everything
|
||||||
|
is seeded silently again, so you will not get a flood, but new-series detection
|
||||||
|
restarts from that moment.
|
||||||
|
|
||||||
|
## Notes and limits
|
||||||
|
|
||||||
|
- Signal has no bot API; this uses [`signal-cli`](https://github.com/AsamK/signal-cli)
|
||||||
|
via [`signal-cli-rest-api`](https://github.com/bbernhard/signal-cli-rest-api).
|
||||||
|
- Delivery is **at-least-once**. A crash between sending and recording can
|
||||||
|
duplicate one post.
|
||||||
|
- The feed is capped by The Events Calendar (30 events by default) and may omit
|
||||||
|
past events; the bot only knows what the feed contains.
|
||||||
|
- Venue/postcode come from the ICS `LOCATION` when present (this feed often has
|
||||||
|
none), then from the event page's structured data, then from the description.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -e '.[dev]'
|
||||||
|
pytest
|
||||||
|
```
|
||||||
|
|
||||||
|
Fixtures are stored as `tests/fixtures/*.ics.txt` so that `*.ics` can stay
|
||||||
|
gitignored.
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
# Copy to config.toml and fill in the signal section. Never commit config.toml.
|
||||||
|
|
||||||
|
[signal]
|
||||||
|
# signal-cli-rest-api (from docker-compose.yml)
|
||||||
|
api_url = "http://signal-api:8080"
|
||||||
|
# Your own number (the linked account's number), international format.
|
||||||
|
number = "+447700900000"
|
||||||
|
# internal_id from GET /v1/groups/<number>, including the "group." prefix.
|
||||||
|
group_id = "group.REPLACE_ME"
|
||||||
|
text_mode = "normal" # or "styled"
|
||||||
|
|
||||||
|
[calendar]
|
||||||
|
url = "https://commonscalendar.uk/?post_type=tribe_events&ical=1&eventDisplay=list"
|
||||||
|
timezone = "Europe/London"
|
||||||
|
refresh_minutes = 15
|
||||||
|
window_days = 60
|
||||||
|
|
||||||
|
[posting]
|
||||||
|
# Local times. Posts are deferred to the next window; the urgent guard breaks
|
||||||
|
# the window when an event is about to start.
|
||||||
|
windows = ["08:00-10:00", "18:00-21:00"]
|
||||||
|
urgent_guard_minutes = 60
|
||||||
|
|
||||||
|
[announce]
|
||||||
|
enabled = true
|
||||||
|
max_days_ahead = 30
|
||||||
|
# false = first run seeds silently; set true to announce everything on startup
|
||||||
|
backfill = false
|
||||||
|
|
||||||
|
[reminders]
|
||||||
|
enabled = true
|
||||||
|
lead_times = ["3d"]
|
||||||
|
# If an event is discovered less than the lead time ahead, remind on the day.
|
||||||
|
fallback_to_day_of = true
|
||||||
|
day_of_time = "09:00"
|
||||||
|
|
||||||
|
[images]
|
||||||
|
enabled = true
|
||||||
|
max_pixels = 1600
|
||||||
|
max_bytes = 400000
|
||||||
|
max_images_per_post = 3
|
||||||
|
scrape_event_page = true
|
||||||
|
|
||||||
|
[format]
|
||||||
|
show_title = true
|
||||||
|
show_hashtags = true
|
||||||
|
show_time = true
|
||||||
|
show_location = true
|
||||||
|
show_link = true
|
||||||
|
shorten_links = false
|
||||||
|
|
||||||
|
[watchdog]
|
||||||
|
enabled = true
|
||||||
|
# Defaults to [signal].number ("Note to Self") when unset.
|
||||||
|
# notify_number = "+447700900001"
|
||||||
|
feed_failures = 3
|
||||||
|
send_failures = 3
|
||||||
|
alert_cooldown_hours = 6
|
||||||
|
daily_heartbeat = false
|
||||||
|
|
||||||
|
[storage]
|
||||||
|
db = "data/calbot.sqlite3"
|
||||||
|
cache = "cache"
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# signal-cli-rest-api, with the native JSON-RPC mode for low latency.
|
||||||
|
services:
|
||||||
|
signal-api:
|
||||||
|
image: bbernhard/signal-cli-rest-api:latest
|
||||||
|
container_name: calbot-signal
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
- MODE=json-rpc-native
|
||||||
|
- SIGNAL_CLI_UID=1000
|
||||||
|
- SIGNAL_CLI_GID=1000
|
||||||
|
- LOG_LEVEL=info
|
||||||
|
ports:
|
||||||
|
- "127.0.0.1:8080:8080"
|
||||||
|
volumes:
|
||||||
|
# Linked-device keys live here. Back it up; losing it means re-linking.
|
||||||
|
- ./signal-data:/home/.local/share/signal-cli
|
||||||
|
|
||||||
|
bot:
|
||||||
|
build: .
|
||||||
|
container_name: calbot
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
- signal-api
|
||||||
|
user: "1000:1000"
|
||||||
|
environment:
|
||||||
|
- CALBOT_CONFIG=/app/config.toml
|
||||||
|
- LOG_LEVEL=info
|
||||||
|
- TZ=Europe/London
|
||||||
|
volumes:
|
||||||
|
- ./config.toml:/app/config.toml:ro
|
||||||
|
- ./data:/app/data
|
||||||
|
- ./cache:/app/cache
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
[project]
|
||||||
|
name = "calbot"
|
||||||
|
version = "0.1.0"
|
||||||
|
description = "Announce iCalendar events to a Signal group"
|
||||||
|
requires-python = ">=3.11"
|
||||||
|
dependencies = [
|
||||||
|
"icalendar>=6.0",
|
||||||
|
"recurring-ical-events>=3.0",
|
||||||
|
"httpx>=0.27",
|
||||||
|
"pillow>=10.0",
|
||||||
|
"beautifulsoup4>=4.12",
|
||||||
|
]
|
||||||
|
|
||||||
|
[project.optional-dependencies]
|
||||||
|
dev = ["pytest>=8.0", "ruff>=0.6"]
|
||||||
|
|
||||||
|
[project.scripts]
|
||||||
|
calbot = "calbot.__main__:main"
|
||||||
|
|
||||||
|
[build-system]
|
||||||
|
requires = ["setuptools>=68"]
|
||||||
|
build-backend = "setuptools.build_meta"
|
||||||
|
|
||||||
|
[tool.setuptools.packages.find]
|
||||||
|
where = ["src"]
|
||||||
|
|
||||||
|
[tool.pytest.ini_options]
|
||||||
|
pythonpath = ["src"]
|
||||||
|
testpaths = ["tests"]
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
"""calbot: announce iCalendar events to a Signal group."""
|
||||||
|
|
||||||
|
__version__ = "0.1.0"
|
||||||
@@ -0,0 +1,187 @@
|
|||||||
|
"""Entry point: run the bot loop, or dump what would be posted."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
import signal
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from .config import Config, ConfigError, load_config
|
||||||
|
from .expand import parse_events
|
||||||
|
from .feed import Feed
|
||||||
|
from .format import find_postcode, render_post, venue_from_description, venue_from_location
|
||||||
|
from .images import ImageService, PageScraper
|
||||||
|
from .scheduler import Bot
|
||||||
|
from .signal_client import SignalClient
|
||||||
|
from .store import Store
|
||||||
|
from .watchdog import Watchdog
|
||||||
|
|
||||||
|
log = logging.getLogger("calbot")
|
||||||
|
|
||||||
|
TICK_SECONDS = 30
|
||||||
|
|
||||||
|
|
||||||
|
def _logging(level: str) -> None:
|
||||||
|
logging.basicConfig(
|
||||||
|
level=getattr(logging, level.upper(), logging.INFO),
|
||||||
|
format="%(asctime)s %(levelname)-7s %(name)s: %(message)s",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _http_client() -> httpx.Client:
|
||||||
|
return httpx.Client(
|
||||||
|
follow_redirects=True,
|
||||||
|
headers={"User-Agent": "calbot/0.1 (iCalendar to Signal bridge)"},
|
||||||
|
limits=httpx.Limits(max_connections=8, max_keepalive_connections=4),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def dump(config: Config) -> int:
|
||||||
|
"""Print rendered posts for the next events, without touching Signal."""
|
||||||
|
http = _http_client()
|
||||||
|
feed = Feed(config.calendar.url, Store(config.db_path), config.calendar.refresh_minutes, http)
|
||||||
|
try:
|
||||||
|
raw = feed.fetch(datetime.now(timezone.utc))
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
print(f"feed fetch failed: {exc}", file=sys.stderr)
|
||||||
|
return 1
|
||||||
|
if raw is None:
|
||||||
|
print("feed unchanged (304); nothing to dump")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
events = parse_events(
|
||||||
|
raw,
|
||||||
|
now,
|
||||||
|
now + timedelta(days=config.calendar.window_days),
|
||||||
|
config.calendar.timezone,
|
||||||
|
)
|
||||||
|
local_tz = ZoneInfo(config.calendar.timezone)
|
||||||
|
for event in events:
|
||||||
|
venue = venue_from_location(event.location) or venue_from_description(event.description)
|
||||||
|
postcode = find_postcode(event.location, event.description)
|
||||||
|
print("=" * 60)
|
||||||
|
print(f"{event.start.astimezone(local_tz):%Y-%m-%d %H:%M} [{event.series_id}]")
|
||||||
|
print("-" * 60)
|
||||||
|
print(
|
||||||
|
render_post(
|
||||||
|
event,
|
||||||
|
config.format,
|
||||||
|
local_tz,
|
||||||
|
venue=venue,
|
||||||
|
postcode=postcode,
|
||||||
|
link=event.url,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
print(f"image: {event.image_url or '(none)'}")
|
||||||
|
print("=" * 60)
|
||||||
|
print(f"{len(events)} event(s)")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
def main(argv: list[str] | None = None) -> int:
|
||||||
|
parser = argparse.ArgumentParser(prog="calbot", description=__doc__)
|
||||||
|
parser.add_argument(
|
||||||
|
"--config",
|
||||||
|
default=os.environ.get("CALBOT_CONFIG", "config.toml"),
|
||||||
|
help="path to config.toml (default: $CALBOT_CONFIG or ./config.toml)",
|
||||||
|
)
|
||||||
|
parser.add_argument("--once", action="store_true", help="run a single tick and exit")
|
||||||
|
parser.add_argument("--dump", action="store_true", help="print posts for upcoming events")
|
||||||
|
parser.add_argument("--check", action="store_true", help="validate config and connection")
|
||||||
|
parser.add_argument("--log-level", default=os.environ.get("LOG_LEVEL", "INFO"))
|
||||||
|
args = parser.parse_args(argv)
|
||||||
|
|
||||||
|
try:
|
||||||
|
config = load_config(args.config)
|
||||||
|
except ConfigError as exc:
|
||||||
|
print(f"config error: {exc}", file=sys.stderr)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
_logging(args.log_level)
|
||||||
|
|
||||||
|
if args.dump:
|
||||||
|
return dump(config)
|
||||||
|
|
||||||
|
if not config.signal.group_id.startswith("group."):
|
||||||
|
log.warning(
|
||||||
|
"signal.group_id does not start with 'group.' — "
|
||||||
|
"use the internal_id from GET /v1/groups/<number>"
|
||||||
|
)
|
||||||
|
|
||||||
|
store = Store(config.db_path)
|
||||||
|
http = _http_client()
|
||||||
|
sender = SignalClient(config.signal.api_url, config.signal.number)
|
||||||
|
watchdog = Watchdog(config.watchdog, sender, config.signal.number, config.signal.text_mode)
|
||||||
|
|
||||||
|
if args.check:
|
||||||
|
if sender.account_exists():
|
||||||
|
log.info("Signal account %s is linked", config.signal.number)
|
||||||
|
else:
|
||||||
|
log.error(
|
||||||
|
"Signal account %s not found at %s. Link it first: "
|
||||||
|
"open %s/v1/qrcodelink?device_name=calbot and scan the QR code.",
|
||||||
|
config.signal.number,
|
||||||
|
config.signal.api_url,
|
||||||
|
config.signal.api_url,
|
||||||
|
)
|
||||||
|
return 1
|
||||||
|
log.info("config OK: %s", config.config_path)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
bot = Bot(
|
||||||
|
config=config,
|
||||||
|
store=store,
|
||||||
|
feed=Feed(config.calendar.url, store, config.calendar.refresh_minutes, http),
|
||||||
|
sender=sender,
|
||||||
|
images=ImageService(config.images, config.cache_dir, http),
|
||||||
|
scraper=PageScraper(config.images, store, http),
|
||||||
|
watchdog=watchdog,
|
||||||
|
http=http,
|
||||||
|
)
|
||||||
|
|
||||||
|
if args.once:
|
||||||
|
bot.tick()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
stopping = False
|
||||||
|
|
||||||
|
def stop(_signum, _frame):
|
||||||
|
nonlocal stopping
|
||||||
|
stopping = True
|
||||||
|
|
||||||
|
signal.signal(signal.SIGTERM, stop)
|
||||||
|
signal.signal(signal.SIGINT, stop)
|
||||||
|
|
||||||
|
log.info("calbot started (feed=%s, group=%s)", config.calendar.url, config.signal.group_id)
|
||||||
|
if not sender.account_exists():
|
||||||
|
log.warning(
|
||||||
|
"Signal account not linked yet. Open %s/v1/qrcodelink?device_name=calbot",
|
||||||
|
config.signal.api_url,
|
||||||
|
)
|
||||||
|
|
||||||
|
while not stopping:
|
||||||
|
try:
|
||||||
|
bot.tick()
|
||||||
|
except Exception: # noqa: BLE001 - keep the service alive
|
||||||
|
log.exception("tick failed")
|
||||||
|
deadline = time.monotonic() + TICK_SECONDS
|
||||||
|
while not stopping and time.monotonic() < deadline:
|
||||||
|
time.sleep(0.5)
|
||||||
|
|
||||||
|
log.info("calbot stopped")
|
||||||
|
store.close()
|
||||||
|
http.close()
|
||||||
|
sender.close()
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
@@ -0,0 +1,259 @@
|
|||||||
|
"""Configuration loading and defaults."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
import re
|
||||||
|
import tomllib
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from datetime import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
class ConfigError(Exception):
|
||||||
|
"""Raised when the configuration file is invalid."""
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# small parsers
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
_DURATION_RE = re.compile(r"^\s*(\d+)\s*([smhdw])\s*$", re.IGNORECASE)
|
||||||
|
_DURATION_UNITS = {
|
||||||
|
"s": 1,
|
||||||
|
"m": 60,
|
||||||
|
"h": 3600,
|
||||||
|
"d": 86400,
|
||||||
|
"w": 604800,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_duration(value: str | int) -> int:
|
||||||
|
"""Parse '3d', '2h', '90m', '1w' into seconds. Bare ints are seconds."""
|
||||||
|
if isinstance(value, int):
|
||||||
|
return value
|
||||||
|
match = _DURATION_RE.match(value)
|
||||||
|
if not match:
|
||||||
|
raise ConfigError(f"invalid duration: {value!r} (use e.g. '3d', '2h', '30m')")
|
||||||
|
return int(match.group(1)) * _DURATION_UNITS[match.group(2).lower()]
|
||||||
|
|
||||||
|
|
||||||
|
def parse_clock(value: str) -> time:
|
||||||
|
try:
|
||||||
|
hour, minute = (int(part) for part in value.split(":"))
|
||||||
|
return time(hour, minute)
|
||||||
|
except (ValueError, TypeError) as exc:
|
||||||
|
raise ConfigError(f"invalid time of day: {value!r}") from exc
|
||||||
|
|
||||||
|
|
||||||
|
def parse_window(value: str) -> tuple[time, time]:
|
||||||
|
try:
|
||||||
|
start, end = value.split("-")
|
||||||
|
except ValueError as exc:
|
||||||
|
raise ConfigError(f"invalid window: {value!r} (use '08:00-10:00')") from exc
|
||||||
|
start_time, end_time = parse_clock(start), parse_clock(end)
|
||||||
|
if end_time <= start_time:
|
||||||
|
raise ConfigError(f"window must end after it starts: {value!r}")
|
||||||
|
return start_time, end_time
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# config sections
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class SignalConfig:
|
||||||
|
api_url: str
|
||||||
|
number: str
|
||||||
|
group_id: str
|
||||||
|
text_mode: str = "normal"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class CalendarConfig:
|
||||||
|
url: str
|
||||||
|
timezone: str = "Europe/London"
|
||||||
|
refresh_minutes: int = 15
|
||||||
|
window_days: int = 60
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PostingConfig:
|
||||||
|
windows: tuple[tuple[time, time], ...] = ((time(8, 0), time(10, 0)), (time(18, 0), time(21, 0)))
|
||||||
|
urgent_guard_minutes: int = 60
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class AnnounceConfig:
|
||||||
|
enabled: bool = True
|
||||||
|
max_days_ahead: int = 30
|
||||||
|
backfill: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class RemindersConfig:
|
||||||
|
enabled: bool = True
|
||||||
|
lead_times: tuple[str, ...] = ("3d",)
|
||||||
|
fallback_to_day_of: bool = True
|
||||||
|
day_of_time: time = time(9, 0)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class ImagesConfig:
|
||||||
|
enabled: bool = True
|
||||||
|
max_pixels: int = 1600
|
||||||
|
max_bytes: int = 400_000
|
||||||
|
max_images_per_post: int = 3
|
||||||
|
timeout_seconds: float = 10.0
|
||||||
|
scrape_event_page: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class FormatConfig:
|
||||||
|
show_title: bool = True
|
||||||
|
show_hashtags: bool = True
|
||||||
|
show_time: bool = True
|
||||||
|
show_location: bool = True
|
||||||
|
show_link: bool = True
|
||||||
|
shorten_links: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class WatchdogConfig:
|
||||||
|
enabled: bool = True
|
||||||
|
notify_number: str | None = None
|
||||||
|
feed_failures: int = 3
|
||||||
|
send_failures: int = 3
|
||||||
|
alert_cooldown_hours: float = 6.0
|
||||||
|
daily_heartbeat: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Config:
|
||||||
|
signal: SignalConfig
|
||||||
|
calendar: CalendarConfig
|
||||||
|
posting: PostingConfig = field(default_factory=PostingConfig)
|
||||||
|
announce: AnnounceConfig = field(default_factory=AnnounceConfig)
|
||||||
|
reminders: RemindersConfig = field(default_factory=RemindersConfig)
|
||||||
|
images: ImagesConfig = field(default_factory=ImagesConfig)
|
||||||
|
format: FormatConfig = field(default_factory=FormatConfig)
|
||||||
|
watchdog: WatchdogConfig = field(default_factory=WatchdogConfig)
|
||||||
|
db_path: Path = Path("data/calbot.sqlite3")
|
||||||
|
cache_dir: Path = Path("cache")
|
||||||
|
config_path: Path | None = None
|
||||||
|
base_dir: Path = Path(".")
|
||||||
|
|
||||||
|
|
||||||
|
def _section(data: dict, name: str) -> dict:
|
||||||
|
value = data.get(name, {})
|
||||||
|
if not isinstance(value, dict):
|
||||||
|
raise ConfigError(f"[{name}] must be a table")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _windows(raw: object) -> tuple[tuple[time, time], ...]:
|
||||||
|
if not isinstance(raw, list) or not raw:
|
||||||
|
raise ConfigError("posting.windows must be a non-empty list of 'HH:MM-HH:MM'")
|
||||||
|
windows = tuple(sorted(parse_window(str(item)) for item in raw))
|
||||||
|
return windows
|
||||||
|
|
||||||
|
|
||||||
|
def load_config(path: str | Path) -> Config:
|
||||||
|
config_path = Path(path).expanduser().resolve()
|
||||||
|
try:
|
||||||
|
data = tomllib.loads(config_path.read_text(encoding="utf-8"))
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise ConfigError(f"config file not found: {config_path}") from exc
|
||||||
|
except tomllib.TOMLDecodeError as exc:
|
||||||
|
raise ConfigError(f"invalid TOML in {config_path}: {exc}") from exc
|
||||||
|
|
||||||
|
base_dir = config_path.parent
|
||||||
|
|
||||||
|
signal = _section(data, "signal")
|
||||||
|
for key in ("api_url", "number", "group_id"):
|
||||||
|
if not signal.get(key):
|
||||||
|
raise ConfigError(f"signal.{key} is required")
|
||||||
|
|
||||||
|
calendar = _section(data, "calendar")
|
||||||
|
if not calendar.get("url"):
|
||||||
|
raise ConfigError("calendar.url is required")
|
||||||
|
|
||||||
|
posting = _section(data, "posting")
|
||||||
|
announce = _section(data, "announce")
|
||||||
|
reminders = _section(data, "reminders")
|
||||||
|
images = _section(data, "images")
|
||||||
|
fmt = _section(data, "format")
|
||||||
|
watchdog = _section(data, "watchdog")
|
||||||
|
storage = _section(data, "storage")
|
||||||
|
|
||||||
|
lead_times = tuple(str(lead) for lead in reminders.get("lead_times", ["3d"]))
|
||||||
|
for lead in lead_times:
|
||||||
|
parse_duration(lead)
|
||||||
|
|
||||||
|
db_path = Path(os.environ.get("CALBOT_DB", storage.get("db", "data/calbot.sqlite3")))
|
||||||
|
cache_dir = Path(os.environ.get("CALBOT_CACHE", storage.get("cache", "cache")))
|
||||||
|
if not db_path.is_absolute():
|
||||||
|
db_path = base_dir / db_path
|
||||||
|
if not cache_dir.is_absolute():
|
||||||
|
cache_dir = base_dir / cache_dir
|
||||||
|
|
||||||
|
return Config(
|
||||||
|
signal=SignalConfig(
|
||||||
|
api_url=str(signal["api_url"]).rstrip("/"),
|
||||||
|
number=str(signal["number"]),
|
||||||
|
group_id=str(signal["group_id"]),
|
||||||
|
text_mode=str(signal.get("text_mode", "normal")),
|
||||||
|
),
|
||||||
|
calendar=CalendarConfig(
|
||||||
|
url=str(calendar["url"]),
|
||||||
|
timezone=str(calendar.get("timezone", "Europe/London")),
|
||||||
|
refresh_minutes=int(calendar.get("refresh_minutes", 15)),
|
||||||
|
window_days=int(calendar.get("window_days", 60)),
|
||||||
|
),
|
||||||
|
posting=PostingConfig(
|
||||||
|
windows=_windows(posting.get("windows", ["08:00-10:00", "18:00-21:00"])),
|
||||||
|
urgent_guard_minutes=int(posting.get("urgent_guard_minutes", 60)),
|
||||||
|
),
|
||||||
|
announce=AnnounceConfig(
|
||||||
|
enabled=bool(announce.get("enabled", True)),
|
||||||
|
max_days_ahead=int(announce.get("max_days_ahead", 30)),
|
||||||
|
backfill=bool(announce.get("backfill", False)),
|
||||||
|
),
|
||||||
|
reminders=RemindersConfig(
|
||||||
|
enabled=bool(reminders.get("enabled", True)),
|
||||||
|
lead_times=lead_times,
|
||||||
|
fallback_to_day_of=bool(reminders.get("fallback_to_day_of", True)),
|
||||||
|
day_of_time=parse_clock(str(reminders.get("day_of_time", "09:00"))),
|
||||||
|
),
|
||||||
|
images=ImagesConfig(
|
||||||
|
enabled=bool(images.get("enabled", True)),
|
||||||
|
max_pixels=int(images.get("max_pixels", 1600)),
|
||||||
|
max_bytes=int(images.get("max_bytes", 400_000)),
|
||||||
|
max_images_per_post=int(images.get("max_images_per_post", 3)),
|
||||||
|
timeout_seconds=float(images.get("timeout_seconds", 10)),
|
||||||
|
scrape_event_page=bool(images.get("scrape_event_page", True)),
|
||||||
|
),
|
||||||
|
format=FormatConfig(
|
||||||
|
show_title=bool(fmt.get("show_title", True)),
|
||||||
|
show_hashtags=bool(fmt.get("show_hashtags", True)),
|
||||||
|
show_time=bool(fmt.get("show_time", True)),
|
||||||
|
show_location=bool(fmt.get("show_location", True)),
|
||||||
|
show_link=bool(fmt.get("show_link", True)),
|
||||||
|
shorten_links=bool(fmt.get("shorten_links", False)),
|
||||||
|
),
|
||||||
|
watchdog=WatchdogConfig(
|
||||||
|
enabled=bool(watchdog.get("enabled", True)),
|
||||||
|
notify_number=(
|
||||||
|
str(watchdog["notify_number"]) if watchdog.get("notify_number") else None
|
||||||
|
),
|
||||||
|
feed_failures=int(watchdog.get("feed_failures", 3)),
|
||||||
|
send_failures=int(watchdog.get("send_failures", 3)),
|
||||||
|
alert_cooldown_hours=float(watchdog.get("alert_cooldown_hours", 6)),
|
||||||
|
daily_heartbeat=bool(watchdog.get("daily_heartbeat", False)),
|
||||||
|
),
|
||||||
|
db_path=db_path,
|
||||||
|
cache_dir=cache_dir,
|
||||||
|
config_path=config_path,
|
||||||
|
base_dir=base_dir,
|
||||||
|
)
|
||||||
@@ -0,0 +1,178 @@
|
|||||||
|
"""Turn an iCalendar document into concrete Event instances inside a window."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import logging
|
||||||
|
import re
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import date, datetime, time, timedelta, timezone
|
||||||
|
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
|
||||||
|
|
||||||
|
import icalendar
|
||||||
|
import recurring_ical_events
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
TEC_UID_RE = re.compile(r"^(\d+)-\d+-\d+@")
|
||||||
|
DATE_LIKE_RE = re.compile(r"^\d{8}$")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class Event:
|
||||||
|
key: str
|
||||||
|
uid: str
|
||||||
|
series_id: str
|
||||||
|
summary: str
|
||||||
|
start: datetime
|
||||||
|
end: datetime
|
||||||
|
all_day: bool
|
||||||
|
location: str = ""
|
||||||
|
description: str = ""
|
||||||
|
url: str = ""
|
||||||
|
image_url: str = ""
|
||||||
|
tz: str = "UTC"
|
||||||
|
status: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
def _zone(name: str) -> ZoneInfo:
|
||||||
|
try:
|
||||||
|
return ZoneInfo(name)
|
||||||
|
except (ZoneInfoNotFoundError, ValueError):
|
||||||
|
log.warning("unknown timezone %r, falling back to UTC", name)
|
||||||
|
return ZoneInfo("UTC")
|
||||||
|
|
||||||
|
|
||||||
|
def _as_datetime(value: datetime | date, tz: ZoneInfo) -> datetime:
|
||||||
|
if isinstance(value, datetime):
|
||||||
|
if value.tzinfo is None:
|
||||||
|
return value.replace(tzinfo=tz)
|
||||||
|
return value
|
||||||
|
return datetime.combine(value, time.min, tzinfo=tz)
|
||||||
|
|
||||||
|
|
||||||
|
def _all_day_flag(component) -> bool:
|
||||||
|
prop = component.get("DTSTART")
|
||||||
|
return prop is not None and not isinstance(getattr(prop, "dt", None), datetime)
|
||||||
|
|
||||||
|
|
||||||
|
def _image_url(component) -> str:
|
||||||
|
for name in ("ATTACH", "IMAGE"):
|
||||||
|
value = component.get(name)
|
||||||
|
if value is None:
|
||||||
|
continue
|
||||||
|
values = value if isinstance(value, list) else [value]
|
||||||
|
for item in values:
|
||||||
|
fmt = ""
|
||||||
|
params = getattr(item, "params", {}) or {}
|
||||||
|
fmt = str(params.get("FMTTYPE", "")).lower()
|
||||||
|
if fmt and not fmt.startswith("image/"):
|
||||||
|
continue
|
||||||
|
uri = str(item)
|
||||||
|
if uri.startswith(("http://", "https://")):
|
||||||
|
return uri
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def _series_id(uid: str, summary: str, start: datetime, duration: timedelta) -> str:
|
||||||
|
match = TEC_UID_RE.match(uid)
|
||||||
|
if match:
|
||||||
|
return f"tec:{match.group(1)}"
|
||||||
|
fingerprint = f"{summary}|{start.astimezone(timezone.utc).strftime('%H:%M')}|{duration}"
|
||||||
|
digest = hashlib.sha1(fingerprint.encode("utf-8")).hexdigest()[:12]
|
||||||
|
return f"h:{digest}"
|
||||||
|
|
||||||
|
|
||||||
|
def _build_event(component, default_tz: ZoneInfo) -> Event | None:
|
||||||
|
dtstart = component.get("DTSTART")
|
||||||
|
if dtstart is None:
|
||||||
|
return None
|
||||||
|
raw_start = dtstart.dt
|
||||||
|
|
||||||
|
status = str(component.get("STATUS", "")).upper()
|
||||||
|
if status == "CANCELLED":
|
||||||
|
return None
|
||||||
|
|
||||||
|
all_day = not isinstance(raw_start, datetime)
|
||||||
|
tz_name = default_tz.key
|
||||||
|
if isinstance(raw_start, datetime) and raw_start.tzinfo is not None:
|
||||||
|
tz_name = getattr(raw_start.tzinfo, "key", None) or str(raw_start.tzinfo)
|
||||||
|
|
||||||
|
local_tz = _zone(tz_name)
|
||||||
|
start_local = _as_datetime(raw_start, default_tz)
|
||||||
|
|
||||||
|
dtend = component.get("DTEND")
|
||||||
|
duration_prop = component.get("DURATION")
|
||||||
|
if dtend is not None:
|
||||||
|
end_local = _as_datetime(dtend.dt, default_tz)
|
||||||
|
elif duration_prop is not None:
|
||||||
|
end_local = start_local + duration_prop.dt
|
||||||
|
else:
|
||||||
|
end_local = start_local + timedelta(days=1 if all_day else 0)
|
||||||
|
|
||||||
|
start = start_local.astimezone(timezone.utc)
|
||||||
|
end = end_local.astimezone(timezone.utc)
|
||||||
|
if end < start:
|
||||||
|
end = start
|
||||||
|
|
||||||
|
uid = str(component.get("UID", "")).strip()
|
||||||
|
summary = str(component.get("SUMMARY", "")).strip() or "(untitled event)"
|
||||||
|
duration = end - start
|
||||||
|
if not uid:
|
||||||
|
uid = hashlib.sha1(f"{summary}|{start.isoformat()}".encode()).hexdigest()[:16]
|
||||||
|
|
||||||
|
return Event(
|
||||||
|
key=f"{uid}|{start.isoformat()}",
|
||||||
|
uid=uid,
|
||||||
|
series_id=_series_id(uid, summary, start, duration),
|
||||||
|
summary=summary,
|
||||||
|
start=start,
|
||||||
|
end=end,
|
||||||
|
all_day=all_day,
|
||||||
|
location=str(component.get("LOCATION", "") or "").strip(),
|
||||||
|
description=str(component.get("DESCRIPTION", "") or "").strip(),
|
||||||
|
url=str(component.get("URL", "") or "").strip(),
|
||||||
|
image_url=_image_url(component),
|
||||||
|
tz=local_tz.key,
|
||||||
|
status=status,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_events(
|
||||||
|
raw: bytes | str,
|
||||||
|
window_start: datetime,
|
||||||
|
window_end: datetime,
|
||||||
|
default_timezone: str = "Europe/London",
|
||||||
|
) -> list[Event]:
|
||||||
|
"""Parse an iCalendar document and return events overlapping the window."""
|
||||||
|
default_tz = _zone(default_timezone)
|
||||||
|
calendar = icalendar.Calendar.from_ical(raw)
|
||||||
|
|
||||||
|
if window_start.tzinfo is None:
|
||||||
|
window_start = window_start.replace(tzinfo=default_tz)
|
||||||
|
if window_end.tzinfo is None:
|
||||||
|
window_end = window_end.replace(tzinfo=default_tz)
|
||||||
|
|
||||||
|
components = []
|
||||||
|
try:
|
||||||
|
components = recurring_ical_events.of(calendar).between(window_start, window_end)
|
||||||
|
except Exception: # pragma: no cover - defensive, feeds vary wildly
|
||||||
|
log.exception("recurrence expansion failed; falling back to raw VEVENTs")
|
||||||
|
|
||||||
|
if not components:
|
||||||
|
components = [
|
||||||
|
item
|
||||||
|
for item in calendar.walk("VEVENT")
|
||||||
|
if getattr(item.get("DTSTART"), "dt", None) is not None
|
||||||
|
]
|
||||||
|
|
||||||
|
events: dict[str, Event] = {}
|
||||||
|
for component in components:
|
||||||
|
event = _build_event(component, default_tz)
|
||||||
|
if event is None:
|
||||||
|
continue
|
||||||
|
if event.end <= window_start or event.start >= window_end:
|
||||||
|
continue
|
||||||
|
events[event.key] = event
|
||||||
|
|
||||||
|
return sorted(events.values(), key=lambda item: item.start)
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
"""Fetch the iCalendar feed with conditional requests."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from .store import Store, from_iso, iso
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
USER_AGENT = "calbot/0.1 (iCalendar to Signal bridge)"
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_feed_url(url: str) -> str:
|
||||||
|
"""webcal:// is a hint, not a transport scheme."""
|
||||||
|
if url.startswith("webcal://"):
|
||||||
|
return "https://" + url[len("webcal://") :]
|
||||||
|
return url
|
||||||
|
|
||||||
|
|
||||||
|
class Feed:
|
||||||
|
def __init__(self, url: str, store: Store, refresh_minutes: int, http: httpx.Client):
|
||||||
|
self.url = normalize_feed_url(url)
|
||||||
|
self.store = store
|
||||||
|
self.refresh_minutes = max(1, refresh_minutes)
|
||||||
|
self.http = http
|
||||||
|
|
||||||
|
def due(self, now: datetime) -> bool:
|
||||||
|
last = self.store.get_meta("feed_last_attempt")
|
||||||
|
if not last:
|
||||||
|
return True
|
||||||
|
return now - from_iso(last) >= timedelta(minutes=self.refresh_minutes)
|
||||||
|
|
||||||
|
def fetch(self, now: datetime) -> bytes | None:
|
||||||
|
"""Return new feed bytes, or None when unchanged / not modified."""
|
||||||
|
headers = {"User-Agent": USER_AGENT, "Accept": "text/calendar, */*"}
|
||||||
|
etag = self.store.get_meta("feed_etag")
|
||||||
|
modified = self.store.get_meta("feed_last_modified")
|
||||||
|
if etag:
|
||||||
|
headers["If-None-Match"] = etag
|
||||||
|
if modified:
|
||||||
|
headers["If-Modified-Since"] = modified
|
||||||
|
|
||||||
|
self.store.set_meta("feed_last_attempt", iso(now))
|
||||||
|
response = self.http.get(self.url, headers=headers, timeout=30)
|
||||||
|
|
||||||
|
if response.status_code == 304:
|
||||||
|
log.debug("feed not modified")
|
||||||
|
return None
|
||||||
|
response.raise_for_status()
|
||||||
|
|
||||||
|
if response.headers.get("etag"):
|
||||||
|
self.store.set_meta("feed_etag", response.headers["etag"])
|
||||||
|
if response.headers.get("last-modified"):
|
||||||
|
self.store.set_meta("feed_last_modified", response.headers["last-modified"])
|
||||||
|
self.store.set_meta("feed_last_success", iso(datetime.now(timezone.utc)))
|
||||||
|
log.info("fetched feed: %d bytes", len(response.content))
|
||||||
|
return response.content
|
||||||
@@ -0,0 +1,112 @@
|
|||||||
|
"""Render an event in the group's posting format."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import calendar as calendar_module
|
||||||
|
import re
|
||||||
|
from datetime import datetime
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
from .config import FormatConfig
|
||||||
|
from .expand import Event
|
||||||
|
|
||||||
|
# Loose UK postcode matcher; deliberately permissive because feeds vary.
|
||||||
|
UK_POSTCODE_RE = re.compile(r"\b([A-Z]{1,2}\d[A-Z\d]?)\s*(\d[A-Z]{2})\b", re.IGNORECASE)
|
||||||
|
|
||||||
|
|
||||||
|
def find_postcode(*texts: str) -> str:
|
||||||
|
for text in texts:
|
||||||
|
if not text:
|
||||||
|
continue
|
||||||
|
match = UK_POSTCODE_RE.search(text)
|
||||||
|
if match:
|
||||||
|
return f"{match.group(1).upper()} {match.group(2).upper()}"
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def _clean(text: str) -> str:
|
||||||
|
return re.sub(r"\s+", " ", text or "").strip()
|
||||||
|
|
||||||
|
|
||||||
|
def _first_line(text: str) -> str:
|
||||||
|
for line in (text or "").splitlines():
|
||||||
|
if line.strip():
|
||||||
|
return line.strip()
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def venue_from_location(location: str) -> str:
|
||||||
|
"""The first line of a TEC LOCATION value is the venue name."""
|
||||||
|
return _clean(_first_line(location)).rstrip(",")
|
||||||
|
|
||||||
|
|
||||||
|
def venue_from_description(description: str) -> str:
|
||||||
|
"""Best-effort fallback: a line in the description that carries a postcode."""
|
||||||
|
for line in (description or "").splitlines():
|
||||||
|
match = UK_POSTCODE_RE.search(line)
|
||||||
|
if not match:
|
||||||
|
continue
|
||||||
|
before = line[: match.start()].strip(" ,-\u2013")
|
||||||
|
parts = [part.strip() for part in re.split(r"[,;]", before) if part.strip()]
|
||||||
|
if parts:
|
||||||
|
return _clean(parts[0])[:60]
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def _clock(moment: datetime) -> tuple[str, str]:
|
||||||
|
hour = moment.hour % 12 or 12
|
||||||
|
meridiem = "am" if moment.hour < 12 else "pm"
|
||||||
|
return f"{hour}.{moment.minute:02d}", meridiem
|
||||||
|
|
||||||
|
|
||||||
|
def format_time_range(start: datetime, end: datetime) -> str:
|
||||||
|
start_text, start_meridiem = _clock(start)
|
||||||
|
end_text, end_meridiem = _clock(end)
|
||||||
|
if not end or end <= start:
|
||||||
|
return f"{start_text}{start_meridiem}"
|
||||||
|
if start_meridiem == end_meridiem:
|
||||||
|
return f"{start_text}-{end_text}{end_meridiem}"
|
||||||
|
return f"{start_text}{start_meridiem}-{end_text}{end_meridiem}"
|
||||||
|
|
||||||
|
|
||||||
|
def format_hashtags(start: datetime) -> str:
|
||||||
|
month = calendar_module.month_name[start.month]
|
||||||
|
return f"#{month} #{month}{start.day}"
|
||||||
|
|
||||||
|
|
||||||
|
def render_post(
|
||||||
|
event: Event,
|
||||||
|
config: FormatConfig,
|
||||||
|
local_tz: ZoneInfo,
|
||||||
|
*,
|
||||||
|
venue: str = "",
|
||||||
|
postcode: str = "",
|
||||||
|
link: str = "",
|
||||||
|
) -> str:
|
||||||
|
start = event.start.astimezone(local_tz)
|
||||||
|
end = event.end.astimezone(local_tz)
|
||||||
|
|
||||||
|
lines: list[str] = []
|
||||||
|
|
||||||
|
if config.show_title:
|
||||||
|
title = _clean(event.summary)
|
||||||
|
if title:
|
||||||
|
lines.append(title)
|
||||||
|
|
||||||
|
if config.show_hashtags:
|
||||||
|
lines.append(format_hashtags(start))
|
||||||
|
|
||||||
|
if config.show_time:
|
||||||
|
lines.append("All day" if event.all_day else format_time_range(start, end))
|
||||||
|
|
||||||
|
if config.show_location:
|
||||||
|
name = _clean(venue) or venue_from_location(event.location)
|
||||||
|
code = postcode or find_postcode(event.location, event.description)
|
||||||
|
place = " ".join(part for part in (name, code) if part).strip()
|
||||||
|
if place:
|
||||||
|
lines.append(place)
|
||||||
|
|
||||||
|
if config.show_link and link:
|
||||||
|
lines.append(link)
|
||||||
|
|
||||||
|
return "\n".join(lines)
|
||||||
@@ -0,0 +1,224 @@
|
|||||||
|
"""Download, downscale and cache event images; scrape event pages for details."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
import hashlib
|
||||||
|
import io
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from bs4 import BeautifulSoup
|
||||||
|
from PIL import Image, ImageOps, UnidentifiedImageError
|
||||||
|
|
||||||
|
from .config import ImagesConfig
|
||||||
|
from .store import PageInfo, Store
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
# Decompression-bomb guard. Real posters are nowhere near this.
|
||||||
|
Image.MAX_IMAGE_PIXELS = 50_000_000
|
||||||
|
|
||||||
|
MAX_DOWNLOAD_BYTES = 20 * 1024 * 1024
|
||||||
|
DOWNLOAD_TIMEOUT = 20.0
|
||||||
|
USER_AGENT = "calbot/0.1 (iCalendar to Signal bridge)"
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PageDetails:
|
||||||
|
image_url: str = ""
|
||||||
|
venue: str = ""
|
||||||
|
postcode: str = ""
|
||||||
|
|
||||||
|
|
||||||
|
class PageScraper:
|
||||||
|
"""Best-effort extraction of image and venue from a TEC event page.
|
||||||
|
|
||||||
|
Results are cached forever per URL; pages are stable in practice and this
|
||||||
|
keeps us to at most one request per event.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, config: ImagesConfig, store: Store, http: httpx.Client):
|
||||||
|
self.config = config
|
||||||
|
self.store = store
|
||||||
|
self.http = http
|
||||||
|
|
||||||
|
def details(self, event_url: str) -> PageInfo:
|
||||||
|
if not event_url or not self.config.scrape_event_page:
|
||||||
|
return PageInfo(image_url="", venue="", postcode="")
|
||||||
|
|
||||||
|
cached = self.store.get_page(event_url)
|
||||||
|
if cached is not None:
|
||||||
|
return cached
|
||||||
|
|
||||||
|
try:
|
||||||
|
response = self.http.get(
|
||||||
|
event_url,
|
||||||
|
headers={"User-Agent": USER_AGENT},
|
||||||
|
timeout=DOWNLOAD_TIMEOUT,
|
||||||
|
follow_redirects=True,
|
||||||
|
)
|
||||||
|
response.raise_for_status()
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
log.warning("page fetch failed for %s: %s", event_url, exc)
|
||||||
|
return PageInfo(image_url="", venue="", postcode="")
|
||||||
|
|
||||||
|
soup = BeautifulSoup(response.text, "html.parser")
|
||||||
|
details = self._extract(soup)
|
||||||
|
info = PageInfo(
|
||||||
|
image_url=details.image_url,
|
||||||
|
venue=details.venue,
|
||||||
|
postcode=details.postcode,
|
||||||
|
)
|
||||||
|
self.store.put_page(event_url, info, datetime.now())
|
||||||
|
return info
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract(soup: BeautifulSoup) -> PageDetails:
|
||||||
|
image_url = ""
|
||||||
|
venue = ""
|
||||||
|
postcode = ""
|
||||||
|
|
||||||
|
for script in soup.find_all("script", type="application/ld+json"):
|
||||||
|
try:
|
||||||
|
payload = json.loads(script.string or "")
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
continue
|
||||||
|
for node in _walk_json(payload):
|
||||||
|
if not isinstance(node, dict) or not _is_event(node):
|
||||||
|
continue
|
||||||
|
image_url = image_url or _first_image(node.get("image"))
|
||||||
|
place = node.get("location")
|
||||||
|
if isinstance(place, list):
|
||||||
|
place = place[0] if place else None
|
||||||
|
if isinstance(place, dict):
|
||||||
|
venue = venue or str(place.get("name", "") or "").strip()
|
||||||
|
address = place.get("address")
|
||||||
|
if isinstance(address, list):
|
||||||
|
address = address[0] if address else None
|
||||||
|
if isinstance(address, dict):
|
||||||
|
postcode = postcode or str(address.get("postalCode", "") or "").strip()
|
||||||
|
|
||||||
|
if not image_url:
|
||||||
|
for attrs in (
|
||||||
|
{"property": "og:image"},
|
||||||
|
{"name": "og:image"},
|
||||||
|
{"name": "twitter:image"},
|
||||||
|
):
|
||||||
|
meta = soup.find("meta", attrs=attrs)
|
||||||
|
if meta and meta.get("content"):
|
||||||
|
image_url = str(meta["content"])
|
||||||
|
break
|
||||||
|
|
||||||
|
return PageDetails(image_url=image_url, venue=venue, postcode=postcode)
|
||||||
|
|
||||||
|
|
||||||
|
def _walk_json(node):
|
||||||
|
if isinstance(node, dict):
|
||||||
|
yield node
|
||||||
|
for value in node.values():
|
||||||
|
yield from _walk_json(value)
|
||||||
|
elif isinstance(node, list):
|
||||||
|
for value in node:
|
||||||
|
yield from _walk_json(value)
|
||||||
|
|
||||||
|
|
||||||
|
def _is_event(node: dict) -> bool:
|
||||||
|
kind = node.get("@type")
|
||||||
|
if isinstance(kind, list):
|
||||||
|
return any(str(item).lower() == "event" for item in kind)
|
||||||
|
return str(kind).lower() == "event"
|
||||||
|
|
||||||
|
|
||||||
|
def _first_image(value) -> str:
|
||||||
|
if isinstance(value, str):
|
||||||
|
return value
|
||||||
|
if isinstance(value, dict):
|
||||||
|
return str(value.get("url", "") or "")
|
||||||
|
if isinstance(value, list):
|
||||||
|
for item in value:
|
||||||
|
found = _first_image(item)
|
||||||
|
if found:
|
||||||
|
return found
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
class ImageService:
|
||||||
|
def __init__(self, config: ImagesConfig, cache_dir: Path, http: httpx.Client):
|
||||||
|
self.config = config
|
||||||
|
self.cache_dir = cache_dir / "images"
|
||||||
|
self.cache_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.http = http
|
||||||
|
|
||||||
|
def data_uri(self, url: str) -> str | None:
|
||||||
|
if not self.config.enabled or not url:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
payload = self._cached_or_download(url)
|
||||||
|
except (httpx.HTTPError, OSError) as exc:
|
||||||
|
log.warning("image fetch failed for %s: %s", url, exc)
|
||||||
|
return None
|
||||||
|
if payload is None:
|
||||||
|
return None
|
||||||
|
encoded = base64.b64encode(payload).decode("ascii")
|
||||||
|
return f"data:image/jpeg;base64,{encoded}"
|
||||||
|
|
||||||
|
def _cached_or_download(self, url: str) -> bytes | None:
|
||||||
|
digest = hashlib.sha1(url.encode("utf-8")).hexdigest()[:20]
|
||||||
|
path = self.cache_dir / f"{digest}.jpg"
|
||||||
|
if path.exists() and path.stat().st_size > 0:
|
||||||
|
return path.read_bytes()
|
||||||
|
|
||||||
|
raw = self._download(url)
|
||||||
|
if raw is None:
|
||||||
|
return None
|
||||||
|
resized = self._resize(raw)
|
||||||
|
if resized is None:
|
||||||
|
return None
|
||||||
|
path.write_bytes(resized)
|
||||||
|
return resized
|
||||||
|
|
||||||
|
def _download(self, url: str) -> bytes | None:
|
||||||
|
if not url.startswith(("http://", "https://")):
|
||||||
|
return None
|
||||||
|
with self.http.stream(
|
||||||
|
"GET", url, headers={"User-Agent": USER_AGENT}, timeout=DOWNLOAD_TIMEOUT
|
||||||
|
) as response:
|
||||||
|
response.raise_for_status()
|
||||||
|
chunks: list[bytes] = []
|
||||||
|
total = 0
|
||||||
|
for chunk in response.iter_bytes():
|
||||||
|
total += len(chunk)
|
||||||
|
if total > MAX_DOWNLOAD_BYTES:
|
||||||
|
log.warning("image too large, skipping: %s", url)
|
||||||
|
return None
|
||||||
|
chunks.append(chunk)
|
||||||
|
return b"".join(chunks)
|
||||||
|
|
||||||
|
def _resize(self, raw: bytes) -> bytes | None:
|
||||||
|
try:
|
||||||
|
with Image.open(io.BytesIO(raw)) as image:
|
||||||
|
image = ImageOps.exif_transpose(image)
|
||||||
|
image = image.convert("RGB")
|
||||||
|
image.thumbnail((self.config.max_pixels, self.config.max_pixels))
|
||||||
|
|
||||||
|
quality = 82
|
||||||
|
while True:
|
||||||
|
buffer = io.BytesIO()
|
||||||
|
image.save(buffer, format="JPEG", quality=quality, optimize=True)
|
||||||
|
if buffer.tell() <= self.config.max_bytes or quality <= 50:
|
||||||
|
return buffer.getvalue()
|
||||||
|
quality -= 8
|
||||||
|
if quality < 50:
|
||||||
|
# Still too big: shrink dimensions and retry once.
|
||||||
|
image.thumbnail(
|
||||||
|
(int(image.width * 0.8), int(image.height * 0.8))
|
||||||
|
)
|
||||||
|
quality = 70
|
||||||
|
except (UnidentifiedImageError, OSError, ValueError) as exc:
|
||||||
|
log.warning("could not decode image: %s", exc)
|
||||||
|
return None
|
||||||
@@ -0,0 +1,389 @@
|
|||||||
|
"""Decision logic and the main tick loop."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from datetime import datetime, time, timedelta, timezone
|
||||||
|
from zoneinfo import ZoneInfo, ZoneInfoNotFoundError
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from .config import Config, parse_duration
|
||||||
|
from .expand import Event, parse_events
|
||||||
|
from .feed import Feed
|
||||||
|
from .format import find_postcode, render_post, venue_from_description, venue_from_location
|
||||||
|
from .images import ImageService, PageScraper
|
||||||
|
from .signal_client import SignalClient, SignalError
|
||||||
|
from .store import OutboxItem, Store, iso
|
||||||
|
from .watchdog import Watchdog
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
MAX_SEND_ATTEMPTS = 5
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# posting windows
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def in_window(moment: datetime, windows: tuple[tuple[time, time], ...]) -> bool:
|
||||||
|
clock = moment.timetz().replace(tzinfo=None)
|
||||||
|
return any(start <= clock < end for start, end in windows)
|
||||||
|
|
||||||
|
|
||||||
|
def next_window_start(
|
||||||
|
moment: datetime, windows: tuple[tuple[time, time], ...]
|
||||||
|
) -> datetime:
|
||||||
|
for offset in range(0, 4):
|
||||||
|
day = moment.date() + timedelta(days=offset)
|
||||||
|
for start, _end in windows:
|
||||||
|
candidate = datetime.combine(day, start, tzinfo=moment.tzinfo)
|
||||||
|
if candidate > moment:
|
||||||
|
return candidate
|
||||||
|
return moment + timedelta(days=1)
|
||||||
|
|
||||||
|
|
||||||
|
def clamp_send_at(
|
||||||
|
due: datetime,
|
||||||
|
now: datetime,
|
||||||
|
deadline: datetime,
|
||||||
|
windows: tuple[tuple[time, time], ...],
|
||||||
|
guard: timedelta,
|
||||||
|
) -> datetime:
|
||||||
|
"""Defer a post to the next preferred window unless that misses the deadline."""
|
||||||
|
base = max(due, now)
|
||||||
|
if in_window(base, windows):
|
||||||
|
return base
|
||||||
|
upcoming = next_window_start(base, windows)
|
||||||
|
if upcoming >= deadline - guard:
|
||||||
|
return base
|
||||||
|
return upcoming
|
||||||
|
|
||||||
|
|
||||||
|
def reminder_time(
|
||||||
|
event: Event,
|
||||||
|
lead: str,
|
||||||
|
first_seen: datetime,
|
||||||
|
config: Config,
|
||||||
|
local_tz: ZoneInfo,
|
||||||
|
) -> datetime | None:
|
||||||
|
"""When the reminder for this lead should go out (local time), or None."""
|
||||||
|
start = event.start.astimezone(local_tz)
|
||||||
|
seen = first_seen.astimezone(local_tz)
|
||||||
|
lead_delta = timedelta(seconds=parse_duration(lead))
|
||||||
|
due = start - lead_delta
|
||||||
|
|
||||||
|
if due <= seen:
|
||||||
|
if not config.reminders.fallback_to_day_of:
|
||||||
|
return None
|
||||||
|
due = datetime.combine(start.date(), config.reminders.day_of_time, tzinfo=local_tz)
|
||||||
|
if not event.all_day:
|
||||||
|
latest = start - timedelta(minutes=config.posting.urgent_guard_minutes)
|
||||||
|
if due > latest:
|
||||||
|
due = latest
|
||||||
|
|
||||||
|
deadline = start if not event.all_day else start + timedelta(hours=12)
|
||||||
|
return clamp_send_at(
|
||||||
|
due,
|
||||||
|
seen,
|
||||||
|
deadline,
|
||||||
|
config.posting.windows,
|
||||||
|
timedelta(minutes=config.posting.urgent_guard_minutes),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# bot
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class Bot:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
config: Config,
|
||||||
|
store: Store,
|
||||||
|
feed: Feed,
|
||||||
|
sender: SignalClient,
|
||||||
|
images: ImageService,
|
||||||
|
scraper: PageScraper,
|
||||||
|
watchdog: Watchdog,
|
||||||
|
http: httpx.Client,
|
||||||
|
):
|
||||||
|
self.config = config
|
||||||
|
self.store = store
|
||||||
|
self.feed = feed
|
||||||
|
self.sender = sender
|
||||||
|
self.images = images
|
||||||
|
self.scraper = scraper
|
||||||
|
self.watchdog = watchdog
|
||||||
|
self.http = http
|
||||||
|
try:
|
||||||
|
self.local_tz = ZoneInfo(config.calendar.timezone)
|
||||||
|
except ZoneInfoNotFoundError:
|
||||||
|
self.local_tz = ZoneInfo("UTC")
|
||||||
|
|
||||||
|
# -- loop ------------------------------------------------------------
|
||||||
|
|
||||||
|
def tick(self, now: datetime | None = None) -> None:
|
||||||
|
now = now or datetime.now(timezone.utc)
|
||||||
|
self._refresh_feed(now)
|
||||||
|
self._dispatch(now)
|
||||||
|
self._heartbeat(now)
|
||||||
|
|
||||||
|
# -- feed + planning -------------------------------------------------
|
||||||
|
|
||||||
|
def _refresh_feed(self, now: datetime) -> None:
|
||||||
|
if not self.feed.due(now):
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
raw = self.feed.fetch(now)
|
||||||
|
if raw is None:
|
||||||
|
self.watchdog.feed_ok()
|
||||||
|
return
|
||||||
|
events = parse_events(
|
||||||
|
raw,
|
||||||
|
now,
|
||||||
|
now + timedelta(days=self.config.calendar.window_days),
|
||||||
|
self.config.calendar.timezone,
|
||||||
|
)
|
||||||
|
except Exception as exc: # noqa: BLE001 - feed problems must not kill the loop
|
||||||
|
log.exception("feed refresh failed")
|
||||||
|
self.watchdog.feed_failed(str(exc), now)
|
||||||
|
return
|
||||||
|
|
||||||
|
if not events and self.store.count_events() > 0:
|
||||||
|
self.watchdog.alert(
|
||||||
|
"empty-feed",
|
||||||
|
"Feed parsed to zero events while the database still has events. "
|
||||||
|
"Leaving stored events untouched.",
|
||||||
|
now,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
|
||||||
|
self.watchdog.feed_ok()
|
||||||
|
self._sync(events, now)
|
||||||
|
|
||||||
|
def _sync(self, events: list[Event], now: datetime) -> None:
|
||||||
|
first_run = self.store.get_meta("seeded") is None
|
||||||
|
seeding = first_run and not self.config.announce.backfill
|
||||||
|
|
||||||
|
for event in events:
|
||||||
|
is_new = self.store.upsert_event(event, now)
|
||||||
|
if not is_new:
|
||||||
|
continue
|
||||||
|
|
||||||
|
new_series = self.store.create_series(event.series_id, now)
|
||||||
|
self._plan_reminders(event, now)
|
||||||
|
|
||||||
|
if seeding:
|
||||||
|
self.store.record_delivery("announce", event.series_id, "seed", now)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if new_series:
|
||||||
|
self._plan_announcement(event, now)
|
||||||
|
|
||||||
|
self.store.mark_gone([event.key for event in events], now)
|
||||||
|
if first_run:
|
||||||
|
self.store.set_meta("seeded", iso(now))
|
||||||
|
log.info("first run: seeded %d events", len(events))
|
||||||
|
|
||||||
|
def _plan_announcement(self, event: Event, now: datetime) -> None:
|
||||||
|
if not self.config.announce.enabled:
|
||||||
|
return
|
||||||
|
if event.end <= now:
|
||||||
|
return
|
||||||
|
horizon = now + timedelta(days=self.config.announce.max_days_ahead)
|
||||||
|
if event.start > horizon:
|
||||||
|
return
|
||||||
|
if self.store.delivered("announce", event.series_id, "once"):
|
||||||
|
return
|
||||||
|
if self.store.queued("announce", event.series_id, "once"):
|
||||||
|
return
|
||||||
|
|
||||||
|
start_local = event.start.astimezone(self.local_tz)
|
||||||
|
deadline = (
|
||||||
|
start_local
|
||||||
|
if not event.all_day
|
||||||
|
else start_local + timedelta(hours=12)
|
||||||
|
)
|
||||||
|
send_at = clamp_send_at(
|
||||||
|
now.astimezone(self.local_tz),
|
||||||
|
now.astimezone(self.local_tz),
|
||||||
|
deadline,
|
||||||
|
self.config.posting.windows,
|
||||||
|
timedelta(minutes=self.config.posting.urgent_guard_minutes),
|
||||||
|
)
|
||||||
|
self.store.add_outbox(
|
||||||
|
"announce",
|
||||||
|
event.series_id,
|
||||||
|
"once",
|
||||||
|
event.key,
|
||||||
|
send_at.astimezone(timezone.utc),
|
||||||
|
now,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _plan_reminders(self, event: Event, now: datetime) -> None:
|
||||||
|
if not self.config.reminders.enabled or event.end <= now:
|
||||||
|
return
|
||||||
|
for lead in self.config.reminders.lead_times:
|
||||||
|
if self.store.plan_exists(event.key, lead):
|
||||||
|
continue
|
||||||
|
due = reminder_time(event, lead, now, self.config, self.local_tz)
|
||||||
|
if due is None:
|
||||||
|
continue
|
||||||
|
self.store.add_plan(event.key, lead, due)
|
||||||
|
self.store.add_outbox(
|
||||||
|
"reminder",
|
||||||
|
event.key,
|
||||||
|
lead,
|
||||||
|
event.key,
|
||||||
|
due.astimezone(timezone.utc),
|
||||||
|
now,
|
||||||
|
)
|
||||||
|
|
||||||
|
# -- sending ---------------------------------------------------------
|
||||||
|
|
||||||
|
def _dispatch(self, now: datetime) -> None:
|
||||||
|
items = self.store.due_outbox(now)
|
||||||
|
if not items:
|
||||||
|
return
|
||||||
|
|
||||||
|
grouped: dict[str, list[OutboxItem]] = {}
|
||||||
|
for item in items:
|
||||||
|
grouped.setdefault(item.event_key, []).append(item)
|
||||||
|
|
||||||
|
batch: list[tuple[Event, list[OutboxItem]]] = []
|
||||||
|
for event_key, group in grouped.items():
|
||||||
|
event = self.store.get_event(event_key)
|
||||||
|
if event is None:
|
||||||
|
for item in group:
|
||||||
|
self.store.drop_outbox(item.id)
|
||||||
|
continue
|
||||||
|
|
||||||
|
valid: list[OutboxItem] = []
|
||||||
|
for item in group:
|
||||||
|
usable = event.end > now and (
|
||||||
|
item.kind != "reminder" or event.all_day or event.start > now
|
||||||
|
)
|
||||||
|
if usable:
|
||||||
|
valid.append(item)
|
||||||
|
else:
|
||||||
|
log.info("dropping stale %s for %s", item.kind, event.summary)
|
||||||
|
self.store.record_delivery(item.kind, item.ref, item.period, now)
|
||||||
|
self.store.drop_outbox(item.id)
|
||||||
|
if valid:
|
||||||
|
batch.append((event, valid))
|
||||||
|
|
||||||
|
if not batch:
|
||||||
|
return
|
||||||
|
|
||||||
|
message, attachments = self._compose(batch)
|
||||||
|
try:
|
||||||
|
self.sender.send(
|
||||||
|
message,
|
||||||
|
[self.config.signal.group_id],
|
||||||
|
attachments=attachments,
|
||||||
|
text_mode=self.config.signal.text_mode,
|
||||||
|
)
|
||||||
|
except SignalError as exc:
|
||||||
|
log.error("send failed: %s", exc)
|
||||||
|
self.watchdog.send_failed(str(exc), now)
|
||||||
|
for _event, group in batch:
|
||||||
|
for item in group:
|
||||||
|
attempts = self.store.bump_attempts(item.id)
|
||||||
|
if attempts >= MAX_SEND_ATTEMPTS:
|
||||||
|
log.error("giving up on %s after %d attempts", item.ref, attempts)
|
||||||
|
self.watchdog.alert(
|
||||||
|
"outbox",
|
||||||
|
f"Dropping a post after {attempts} failed attempts.",
|
||||||
|
now,
|
||||||
|
)
|
||||||
|
self.store.drop_outbox(item.id)
|
||||||
|
return
|
||||||
|
|
||||||
|
self.watchdog.send_ok()
|
||||||
|
for event, group in batch:
|
||||||
|
for item in group:
|
||||||
|
self.store.record_delivery(item.kind, item.ref, item.period, now)
|
||||||
|
if item.kind == "announce":
|
||||||
|
self.store.mark_series_announced(item.ref, event.uid, now)
|
||||||
|
self.store.drop_outbox(item.id)
|
||||||
|
log.info("posted %d item(s) covering %d event(s)", len(items), len(batch))
|
||||||
|
|
||||||
|
def _compose(self, batch: list[tuple[Event, list[OutboxItem]]]) -> tuple[str, list[str]]:
|
||||||
|
blocks: list[str] = []
|
||||||
|
attachments: list[str] = []
|
||||||
|
limit = self.config.images.max_images_per_post if len(batch) > 1 else 1
|
||||||
|
|
||||||
|
for event, _items in batch:
|
||||||
|
info = self.scraper.details(event.url)
|
||||||
|
venue = (
|
||||||
|
venue_from_location(event.location)
|
||||||
|
or info.venue
|
||||||
|
or venue_from_description(event.description)
|
||||||
|
)
|
||||||
|
postcode = (
|
||||||
|
find_postcode(event.location, event.description) or info.postcode
|
||||||
|
)
|
||||||
|
link = event.url
|
||||||
|
if self.config.format.shorten_links and link:
|
||||||
|
link = self._shorten(link)
|
||||||
|
|
||||||
|
blocks.append(
|
||||||
|
render_post(
|
||||||
|
event,
|
||||||
|
self.config.format,
|
||||||
|
self.local_tz,
|
||||||
|
venue=venue,
|
||||||
|
postcode=postcode,
|
||||||
|
link=link,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
if len(attachments) < limit:
|
||||||
|
image_url = event.image_url or info.image_url
|
||||||
|
data_uri = self.images.data_uri(image_url) if image_url else None
|
||||||
|
if data_uri:
|
||||||
|
attachments.append(data_uri)
|
||||||
|
|
||||||
|
return "\n\n".join(blocks), attachments
|
||||||
|
|
||||||
|
def _shorten(self, url: str) -> str:
|
||||||
|
cached = self.store.get_short_link(url)
|
||||||
|
if cached:
|
||||||
|
return cached
|
||||||
|
try:
|
||||||
|
response = self.http.get(
|
||||||
|
"https://tinyurl.com/api-create.php",
|
||||||
|
params={"url": url},
|
||||||
|
timeout=8,
|
||||||
|
headers={"User-Agent": "calbot/0.1"},
|
||||||
|
)
|
||||||
|
if response.status_code == 200 and response.text.strip().startswith("http"):
|
||||||
|
short = response.text.strip()
|
||||||
|
self.store.put_short_link(url, short, datetime.now(timezone.utc))
|
||||||
|
return short
|
||||||
|
except httpx.HTTPError as exc:
|
||||||
|
log.warning("link shortening failed: %s", exc)
|
||||||
|
return url
|
||||||
|
|
||||||
|
# -- heartbeat -------------------------------------------------------
|
||||||
|
|
||||||
|
def _heartbeat(self, now: datetime) -> None:
|
||||||
|
if not self.config.watchdog.daily_heartbeat:
|
||||||
|
return
|
||||||
|
local = now.astimezone(self.local_tz)
|
||||||
|
today = local.date().isoformat()
|
||||||
|
if local.hour < 9 or self.store.get_meta("last_heartbeat") == today:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
self.sender.send(
|
||||||
|
f"calbot alive \u2014 {self.store.count_events()} events in the window",
|
||||||
|
[self.watchdog.recipient],
|
||||||
|
text_mode=self.config.signal.text_mode,
|
||||||
|
)
|
||||||
|
except SignalError as exc:
|
||||||
|
log.warning("heartbeat failed: %s", exc)
|
||||||
|
return
|
||||||
|
self.store.set_meta("last_heartbeat", today)
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
"""Minimal client for signal-cli-rest-api."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class SignalError(Exception):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
class SignalClient:
|
||||||
|
def __init__(self, base_url: str, number: str, timeout: float = 60.0):
|
||||||
|
self.base_url = base_url.rstrip("/")
|
||||||
|
self.number = number
|
||||||
|
self.http = httpx.Client(base_url=self.base_url, timeout=timeout)
|
||||||
|
|
||||||
|
def send(
|
||||||
|
self,
|
||||||
|
message: str,
|
||||||
|
recipients: list[str],
|
||||||
|
attachments: list[str] | None = None,
|
||||||
|
text_mode: str = "normal",
|
||||||
|
) -> None:
|
||||||
|
payload: dict = {
|
||||||
|
"message": message,
|
||||||
|
"number": self.number,
|
||||||
|
"recipients": recipients,
|
||||||
|
"text_mode": text_mode,
|
||||||
|
}
|
||||||
|
if attachments:
|
||||||
|
payload["base64_attachments"] = attachments
|
||||||
|
|
||||||
|
response = self.http.post("/v2/send", json=payload)
|
||||||
|
if response.status_code >= 400:
|
||||||
|
raise SignalError(
|
||||||
|
f"send failed ({response.status_code}): {response.text[:500]}"
|
||||||
|
)
|
||||||
|
|
||||||
|
def account_exists(self) -> bool:
|
||||||
|
try:
|
||||||
|
response = self.http.get("/v1/accounts")
|
||||||
|
if response.status_code >= 400:
|
||||||
|
return False
|
||||||
|
return self.number in response.json()
|
||||||
|
except (httpx.HTTPError, ValueError):
|
||||||
|
return False
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self.http.close()
|
||||||
@@ -0,0 +1,368 @@
|
|||||||
|
"""SQLite persistence: events, series, reminder plans, outbox, deliveries."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sqlite3
|
||||||
|
from dataclasses import dataclass
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Iterable
|
||||||
|
|
||||||
|
from .expand import Event
|
||||||
|
|
||||||
|
SCHEMA = """
|
||||||
|
CREATE TABLE IF NOT EXISTS meta (
|
||||||
|
key TEXT PRIMARY KEY,
|
||||||
|
value TEXT NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS series (
|
||||||
|
series_id TEXT PRIMARY KEY,
|
||||||
|
first_seen_at TEXT NOT NULL,
|
||||||
|
announced_at TEXT,
|
||||||
|
announced_uid TEXT
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS events (
|
||||||
|
event_key TEXT PRIMARY KEY,
|
||||||
|
uid TEXT NOT NULL,
|
||||||
|
series_id TEXT NOT NULL,
|
||||||
|
summary TEXT NOT NULL,
|
||||||
|
start_utc TEXT NOT NULL,
|
||||||
|
end_utc TEXT NOT NULL,
|
||||||
|
all_day INTEGER NOT NULL,
|
||||||
|
location TEXT NOT NULL DEFAULT '',
|
||||||
|
url TEXT NOT NULL DEFAULT '',
|
||||||
|
image_url TEXT NOT NULL DEFAULT '',
|
||||||
|
event_tz TEXT NOT NULL DEFAULT 'UTC',
|
||||||
|
status TEXT NOT NULL DEFAULT '',
|
||||||
|
first_seen_at TEXT NOT NULL,
|
||||||
|
last_seen_at TEXT NOT NULL,
|
||||||
|
gone INTEGER NOT NULL DEFAULT 0
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS events_start ON events(start_utc);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS reminder_plans (
|
||||||
|
event_key TEXT NOT NULL,
|
||||||
|
lead TEXT NOT NULL,
|
||||||
|
due_at TEXT NOT NULL,
|
||||||
|
PRIMARY KEY (event_key, lead)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS outbox (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
kind TEXT NOT NULL,
|
||||||
|
ref TEXT NOT NULL,
|
||||||
|
period TEXT NOT NULL,
|
||||||
|
event_key TEXT NOT NULL,
|
||||||
|
send_at TEXT NOT NULL,
|
||||||
|
attempts INTEGER NOT NULL DEFAULT 0,
|
||||||
|
created_at TEXT NOT NULL,
|
||||||
|
UNIQUE (kind, ref, period)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE INDEX IF NOT EXISTS outbox_due ON outbox(send_at);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS deliveries (
|
||||||
|
kind TEXT NOT NULL,
|
||||||
|
ref TEXT NOT NULL,
|
||||||
|
period TEXT NOT NULL,
|
||||||
|
sent_at TEXT NOT NULL,
|
||||||
|
PRIMARY KEY (kind, ref, period)
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS pages (
|
||||||
|
url TEXT PRIMARY KEY,
|
||||||
|
image_url TEXT NOT NULL DEFAULT '',
|
||||||
|
venue TEXT NOT NULL DEFAULT '',
|
||||||
|
postcode TEXT NOT NULL DEFAULT '',
|
||||||
|
fetched_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS short_links (
|
||||||
|
url TEXT PRIMARY KEY,
|
||||||
|
short_url TEXT NOT NULL,
|
||||||
|
created_at TEXT NOT NULL
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
|
||||||
|
|
||||||
|
def iso(moment: datetime) -> str:
|
||||||
|
return moment.astimezone(timezone.utc).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
def from_iso(value: str) -> datetime:
|
||||||
|
return datetime.fromisoformat(value)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class OutboxItem:
|
||||||
|
id: int
|
||||||
|
kind: str
|
||||||
|
ref: str
|
||||||
|
period: str
|
||||||
|
event_key: str
|
||||||
|
send_at: datetime
|
||||||
|
attempts: int
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass(frozen=True)
|
||||||
|
class PageInfo:
|
||||||
|
image_url: str
|
||||||
|
venue: str
|
||||||
|
postcode: str
|
||||||
|
|
||||||
|
|
||||||
|
class Store:
|
||||||
|
def __init__(self, path: Path):
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.conn = sqlite3.connect(path, timeout=30)
|
||||||
|
self.conn.row_factory = sqlite3.Row
|
||||||
|
self.conn.execute("PRAGMA journal_mode=WAL")
|
||||||
|
self.conn.execute("PRAGMA foreign_keys=ON")
|
||||||
|
self.conn.executescript(SCHEMA)
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self.conn.close()
|
||||||
|
|
||||||
|
# -- meta ------------------------------------------------------------
|
||||||
|
|
||||||
|
def get_meta(self, key: str) -> str | None:
|
||||||
|
row = self.conn.execute("SELECT value FROM meta WHERE key = ?", (key,)).fetchone()
|
||||||
|
return row["value"] if row else None
|
||||||
|
|
||||||
|
def set_meta(self, key: str, value: str) -> None:
|
||||||
|
self.conn.execute(
|
||||||
|
"INSERT INTO meta(key, value) VALUES(?, ?) "
|
||||||
|
"ON CONFLICT(key) DO UPDATE SET value = excluded.value",
|
||||||
|
(key, value),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
# -- events ----------------------------------------------------------
|
||||||
|
|
||||||
|
def has_events(self) -> bool:
|
||||||
|
row = self.conn.execute("SELECT 1 FROM events LIMIT 1").fetchone()
|
||||||
|
return row is not None
|
||||||
|
|
||||||
|
def event_exists(self, event_key: str) -> bool:
|
||||||
|
row = self.conn.execute(
|
||||||
|
"SELECT 1 FROM events WHERE event_key = ?", (event_key,)
|
||||||
|
).fetchone()
|
||||||
|
return row is not None
|
||||||
|
|
||||||
|
def upsert_event(self, event: Event, now: datetime) -> bool:
|
||||||
|
"""Insert or refresh an event. Returns True if it was new."""
|
||||||
|
is_new = not self.event_exists(event.key)
|
||||||
|
self.conn.execute(
|
||||||
|
"""
|
||||||
|
INSERT INTO events (
|
||||||
|
event_key, uid, series_id, summary, start_utc, end_utc, all_day,
|
||||||
|
location, url, image_url, event_tz, status, first_seen_at, last_seen_at, gone
|
||||||
|
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, 0)
|
||||||
|
ON CONFLICT(event_key) DO UPDATE SET
|
||||||
|
summary = excluded.summary,
|
||||||
|
start_utc = excluded.start_utc,
|
||||||
|
end_utc = excluded.end_utc,
|
||||||
|
all_day = excluded.all_day,
|
||||||
|
location = excluded.location,
|
||||||
|
url = excluded.url,
|
||||||
|
image_url = excluded.image_url,
|
||||||
|
status = excluded.status,
|
||||||
|
last_seen_at = excluded.last_seen_at,
|
||||||
|
gone = 0
|
||||||
|
""",
|
||||||
|
(
|
||||||
|
event.key,
|
||||||
|
event.uid,
|
||||||
|
event.series_id,
|
||||||
|
event.summary,
|
||||||
|
iso(event.start),
|
||||||
|
iso(event.end),
|
||||||
|
int(event.all_day),
|
||||||
|
event.location,
|
||||||
|
event.url,
|
||||||
|
event.image_url,
|
||||||
|
event.tz,
|
||||||
|
event.status,
|
||||||
|
iso(now),
|
||||||
|
iso(now),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
|
return is_new
|
||||||
|
|
||||||
|
def mark_gone(self, seen_keys: Iterable[str], now: datetime) -> None:
|
||||||
|
seen = set(seen_keys)
|
||||||
|
rows = self.conn.execute("SELECT event_key FROM events WHERE gone = 0").fetchall()
|
||||||
|
for row in rows:
|
||||||
|
if row["event_key"] not in seen:
|
||||||
|
self.conn.execute(
|
||||||
|
"UPDATE events SET gone = 1, last_seen_at = ? WHERE event_key = ?",
|
||||||
|
(iso(now), row["event_key"]),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
def get_event(self, event_key: str) -> Event | None:
|
||||||
|
row = self.conn.execute(
|
||||||
|
"SELECT * FROM events WHERE event_key = ?", (event_key,)
|
||||||
|
).fetchone()
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
return Event(
|
||||||
|
key=row["event_key"],
|
||||||
|
uid=row["uid"],
|
||||||
|
series_id=row["series_id"],
|
||||||
|
summary=row["summary"],
|
||||||
|
start=from_iso(row["start_utc"]),
|
||||||
|
end=from_iso(row["end_utc"]),
|
||||||
|
all_day=bool(row["all_day"]),
|
||||||
|
location=row["location"],
|
||||||
|
url=row["url"],
|
||||||
|
image_url=row["image_url"],
|
||||||
|
tz=row["event_tz"],
|
||||||
|
status=row["status"],
|
||||||
|
)
|
||||||
|
|
||||||
|
def count_events(self) -> int:
|
||||||
|
row = self.conn.execute("SELECT COUNT(*) AS n FROM events WHERE gone = 0").fetchone()
|
||||||
|
return int(row["n"])
|
||||||
|
|
||||||
|
# -- series ----------------------------------------------------------
|
||||||
|
|
||||||
|
def create_series(self, series_id: str, now: datetime) -> bool:
|
||||||
|
cursor = self.conn.execute(
|
||||||
|
"INSERT OR IGNORE INTO series(series_id, first_seen_at) VALUES(?, ?)",
|
||||||
|
(series_id, iso(now)),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
|
return cursor.rowcount > 0
|
||||||
|
|
||||||
|
def mark_series_announced(self, series_id: str, uid: str, now: datetime) -> None:
|
||||||
|
self.conn.execute(
|
||||||
|
"UPDATE series SET announced_at = ?, announced_uid = ? WHERE series_id = ?",
|
||||||
|
(iso(now), uid, series_id),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
# -- reminder plans --------------------------------------------------
|
||||||
|
|
||||||
|
def plan_exists(self, event_key: str, lead: str) -> bool:
|
||||||
|
row = self.conn.execute(
|
||||||
|
"SELECT 1 FROM reminder_plans WHERE event_key = ? AND lead = ?",
|
||||||
|
(event_key, lead),
|
||||||
|
).fetchone()
|
||||||
|
return row is not None
|
||||||
|
|
||||||
|
def add_plan(self, event_key: str, lead: str, due_at: datetime) -> None:
|
||||||
|
self.conn.execute(
|
||||||
|
"INSERT OR IGNORE INTO reminder_plans(event_key, lead, due_at) VALUES(?, ?, ?)",
|
||||||
|
(event_key, lead, iso(due_at)),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
# -- outbox ----------------------------------------------------------
|
||||||
|
|
||||||
|
def add_outbox(
|
||||||
|
self,
|
||||||
|
kind: str,
|
||||||
|
ref: str,
|
||||||
|
period: str,
|
||||||
|
event_key: str,
|
||||||
|
send_at: datetime,
|
||||||
|
now: datetime,
|
||||||
|
) -> None:
|
||||||
|
self.conn.execute(
|
||||||
|
"""
|
||||||
|
INSERT OR IGNORE INTO outbox(kind, ref, period, event_key, send_at, created_at)
|
||||||
|
VALUES (?, ?, ?, ?, ?, ?)
|
||||||
|
""",
|
||||||
|
(kind, ref, period, event_key, iso(send_at), iso(now)),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
def due_outbox(self, now: datetime) -> list[OutboxItem]:
|
||||||
|
rows = self.conn.execute(
|
||||||
|
"SELECT * FROM outbox WHERE send_at <= ? ORDER BY send_at, id",
|
||||||
|
(iso(now),),
|
||||||
|
).fetchall()
|
||||||
|
return [
|
||||||
|
OutboxItem(
|
||||||
|
id=row["id"],
|
||||||
|
kind=row["kind"],
|
||||||
|
ref=row["ref"],
|
||||||
|
period=row["period"],
|
||||||
|
event_key=row["event_key"],
|
||||||
|
send_at=from_iso(row["send_at"]),
|
||||||
|
attempts=row["attempts"],
|
||||||
|
)
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
def bump_attempts(self, item_id: int) -> int:
|
||||||
|
self.conn.execute("UPDATE outbox SET attempts = attempts + 1 WHERE id = ?", (item_id,))
|
||||||
|
self.conn.commit()
|
||||||
|
row = self.conn.execute("SELECT attempts FROM outbox WHERE id = ?", (item_id,)).fetchone()
|
||||||
|
return int(row["attempts"]) if row else 0
|
||||||
|
|
||||||
|
def drop_outbox(self, item_id: int) -> None:
|
||||||
|
self.conn.execute("DELETE FROM outbox WHERE id = ?", (item_id,))
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
# -- deliveries ------------------------------------------------------
|
||||||
|
|
||||||
|
def delivered(self, kind: str, ref: str, period: str) -> bool:
|
||||||
|
row = self.conn.execute(
|
||||||
|
"SELECT 1 FROM deliveries WHERE kind = ? AND ref = ? AND period = ?",
|
||||||
|
(kind, ref, period),
|
||||||
|
).fetchone()
|
||||||
|
return row is not None
|
||||||
|
|
||||||
|
def record_delivery(self, kind: str, ref: str, period: str, now: datetime) -> None:
|
||||||
|
self.conn.execute(
|
||||||
|
"INSERT OR REPLACE INTO deliveries(kind, ref, period, sent_at) VALUES(?, ?, ?, ?)",
|
||||||
|
(kind, ref, period, iso(now)),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
def queued(self, kind: str, ref: str, period: str) -> bool:
|
||||||
|
row = self.conn.execute(
|
||||||
|
"SELECT 1 FROM outbox WHERE kind = ? AND ref = ? AND period = ?",
|
||||||
|
(kind, ref, period),
|
||||||
|
).fetchone()
|
||||||
|
return row is not None
|
||||||
|
|
||||||
|
# -- scraped page cache ----------------------------------------------
|
||||||
|
|
||||||
|
def get_page(self, url: str) -> PageInfo | None:
|
||||||
|
row = self.conn.execute("SELECT * FROM pages WHERE url = ?", (url,)).fetchone()
|
||||||
|
if row is None:
|
||||||
|
return None
|
||||||
|
return PageInfo(
|
||||||
|
image_url=row["image_url"], venue=row["venue"], postcode=row["postcode"]
|
||||||
|
)
|
||||||
|
|
||||||
|
def put_page(self, url: str, info: PageInfo, now: datetime) -> None:
|
||||||
|
self.conn.execute(
|
||||||
|
"INSERT OR REPLACE INTO pages(url, image_url, venue, postcode, fetched_at) "
|
||||||
|
"VALUES (?, ?, ?, ?, ?)",
|
||||||
|
(url, info.image_url, info.venue, info.postcode, iso(now)),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
|
|
||||||
|
# -- short links -----------------------------------------------------
|
||||||
|
|
||||||
|
def get_short_link(self, url: str) -> str | None:
|
||||||
|
row = self.conn.execute(
|
||||||
|
"SELECT short_url FROM short_links WHERE url = ?", (url,)
|
||||||
|
).fetchone()
|
||||||
|
return row["short_url"] if row else None
|
||||||
|
|
||||||
|
def put_short_link(self, url: str, short_url: str, now: datetime) -> None:
|
||||||
|
self.conn.execute(
|
||||||
|
"INSERT OR REPLACE INTO short_links(url, short_url, created_at) VALUES (?, ?, ?)",
|
||||||
|
(url, short_url, iso(now)),
|
||||||
|
)
|
||||||
|
self.conn.commit()
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
"""Watchdog: tell the operator when things stop working."""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from datetime import datetime, timedelta
|
||||||
|
|
||||||
|
from .config import WatchdogConfig
|
||||||
|
from .signal_client import SignalClient
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class Watchdog:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
config: WatchdogConfig,
|
||||||
|
sender: SignalClient,
|
||||||
|
fallback_number: str,
|
||||||
|
text_mode: str = "normal",
|
||||||
|
):
|
||||||
|
self.config = config
|
||||||
|
self.sender = sender
|
||||||
|
self.recipient = config.notify_number or fallback_number
|
||||||
|
self.text_mode = text_mode
|
||||||
|
self.feed_failures = 0
|
||||||
|
self.send_failures = 0
|
||||||
|
self._last_alert: dict[str, datetime] = {}
|
||||||
|
|
||||||
|
def feed_ok(self) -> None:
|
||||||
|
self.feed_failures = 0
|
||||||
|
|
||||||
|
def feed_failed(self, reason: str, now: datetime) -> None:
|
||||||
|
self.feed_failures += 1
|
||||||
|
if self.feed_failures >= self.config.feed_failures:
|
||||||
|
self.alert("feed", f"Calendar feed is failing: {reason}", now)
|
||||||
|
|
||||||
|
def send_ok(self) -> None:
|
||||||
|
self.send_failures = 0
|
||||||
|
|
||||||
|
def send_failed(self, reason: str, now: datetime) -> None:
|
||||||
|
self.send_failures += 1
|
||||||
|
if self.send_failures >= self.config.send_failures:
|
||||||
|
self.alert("send", f"Signal sending is failing: {reason}", now)
|
||||||
|
|
||||||
|
def alert(self, key: str, text: str, now: datetime) -> None:
|
||||||
|
if not self.config.enabled:
|
||||||
|
log.error("watchdog (disabled): %s", text)
|
||||||
|
return
|
||||||
|
last = self._last_alert.get(key)
|
||||||
|
cooldown = timedelta(hours=self.config.alert_cooldown_hours)
|
||||||
|
if last is not None and now - last < cooldown:
|
||||||
|
return
|
||||||
|
self._last_alert[key] = now
|
||||||
|
try:
|
||||||
|
self.sender.send(
|
||||||
|
f"\u26a0\ufe0f calbot\n{text}",
|
||||||
|
[self.recipient],
|
||||||
|
text_mode=self.text_mode,
|
||||||
|
)
|
||||||
|
log.warning("watchdog alert sent: %s", text)
|
||||||
|
except Exception as exc: # noqa: BLE001 - never crash the loop
|
||||||
|
log.error("watchdog could not send alert (%s): %s", exc, text)
|
||||||
Vendored
+49
@@ -0,0 +1,49 @@
|
|||||||
|
BEGIN:VCALENDAR
|
||||||
|
VERSION:2.0
|
||||||
|
PRODID:-//MCR Commons Calendar - ECPv6.18.2//NONSGML v1.0//EN
|
||||||
|
BEGIN:VTIMEZONE
|
||||||
|
TZID:Europe/London
|
||||||
|
BEGIN:STANDARD
|
||||||
|
TZOFFSETFROM:+0100
|
||||||
|
TZOFFSETTO:+0000
|
||||||
|
TZNAME:GMT
|
||||||
|
DTSTART:20251026T010000
|
||||||
|
END:STANDARD
|
||||||
|
BEGIN:DAYLIGHT
|
||||||
|
TZOFFSETFROM:+0000
|
||||||
|
TZOFFSETTO:+0100
|
||||||
|
TZNAME:BST
|
||||||
|
DTSTART:20260329T010000
|
||||||
|
END:DAYLIGHT
|
||||||
|
END:VTIMEZONE
|
||||||
|
BEGIN:VEVENT
|
||||||
|
DTSTART;TZID=Europe/London:20261120T090000
|
||||||
|
DTEND;TZID=Europe/London:20261120T100000
|
||||||
|
DTSTAMP:20261116T192444Z
|
||||||
|
UID:1839-1795165200-1795168800@commonscalendar.uk
|
||||||
|
SUMMARY:Clean up Greater Manchester Pension fund
|
||||||
|
DESCRIPTION:Join us to lobby for GM Pensions free from arms and fossil fuel holdings.
|
||||||
|
URL:https://commonscalendar.uk/event/clean-up-greater-manchester-pension-fund/
|
||||||
|
CATEGORIES:Demonstration
|
||||||
|
ATTACH;FMTTYPE=image/png:https://commonscalendar.uk/wp-content/uploads/2026/11/inbound5975249048432321891.png
|
||||||
|
END:VEVENT
|
||||||
|
BEGIN:VEVENT
|
||||||
|
DTSTART;VALUE=DATE:20261121
|
||||||
|
DTEND;VALUE=DATE:20261122
|
||||||
|
DTSTAMP:20261116T192444Z
|
||||||
|
UID:1902-1795257600-1795344000@commonscalendar.uk
|
||||||
|
SUMMARY:All day gathering
|
||||||
|
DESCRIPTION:Come along any time.
|
||||||
|
URL:https://commonscalendar.uk/event/all-day-gathering/
|
||||||
|
END:VEVENT
|
||||||
|
BEGIN:VEVENT
|
||||||
|
DTSTART;TZID=Europe/London:20261127T090000
|
||||||
|
DTEND;TZID=Europe/London:20261127T100000
|
||||||
|
DTSTAMP:20261116T192444Z
|
||||||
|
UID:1839-1795770000-1795773600@commonscalendar.uk
|
||||||
|
SUMMARY:Clean up Greater Manchester Pension fund
|
||||||
|
DESCRIPTION:Second occurrence, same series.
|
||||||
|
URL:https://commonscalendar.uk/event/clean-up-greater-manchester-pension-fund/
|
||||||
|
ATTACH;FMTTYPE=image/webp:https://commonscalendar.uk/wp-content/uploads/2026/11/poster.webp
|
||||||
|
END:VEVENT
|
||||||
|
END:VCALENDAR
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from calbot.expand import parse_events
|
||||||
|
|
||||||
|
FIXTURE = Path(__file__).parent / "fixtures" / "tec-like.ics.txt"
|
||||||
|
|
||||||
|
|
||||||
|
def load_bytes() -> bytes:
|
||||||
|
return FIXTURE.read_bytes()
|
||||||
|
|
||||||
|
|
||||||
|
def test_parses_tec_style_feed():
|
||||||
|
events = parse_events(
|
||||||
|
load_bytes(),
|
||||||
|
datetime(2026, 11, 19, tzinfo=timezone.utc),
|
||||||
|
datetime(2026, 12, 1, tzinfo=timezone.utc),
|
||||||
|
"Europe/London",
|
||||||
|
)
|
||||||
|
assert len(events) == 3
|
||||||
|
|
||||||
|
first, all_day, third = events
|
||||||
|
|
||||||
|
assert first.summary == "Clean up Greater Manchester Pension fund"
|
||||||
|
assert first.series_id == "tec:1839"
|
||||||
|
assert not first.all_day
|
||||||
|
assert first.image_url.endswith("inbound5975249048432321891.png")
|
||||||
|
assert first.url.endswith("clean-up-greater-manchester-pension-fund/")
|
||||||
|
# 09:00 London on 20 Nov 2026 is GMT (UTC).
|
||||||
|
assert first.start == datetime(2026, 11, 20, 9, 0, tzinfo=timezone.utc)
|
||||||
|
|
||||||
|
assert all_day.all_day
|
||||||
|
assert all_day.series_id == "tec:1902"
|
||||||
|
assert all_day.start == datetime(2026, 11, 21, 0, 0, tzinfo=timezone.utc)
|
||||||
|
|
||||||
|
assert third.series_id == "tec:1839"
|
||||||
|
assert third.image_url.endswith("poster.webp")
|
||||||
|
|
||||||
|
|
||||||
|
def test_window_excludes_far_future():
|
||||||
|
events = parse_events(
|
||||||
|
load_bytes(),
|
||||||
|
datetime(2026, 11, 19, tzinfo=timezone.utc),
|
||||||
|
datetime(2026, 11, 20, tzinfo=timezone.utc),
|
||||||
|
"Europe/London",
|
||||||
|
)
|
||||||
|
assert len(events) <= 1
|
||||||
@@ -0,0 +1,98 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
from calbot.config import FormatConfig
|
||||||
|
from calbot.expand import Event
|
||||||
|
from calbot.format import (
|
||||||
|
find_postcode,
|
||||||
|
format_hashtags,
|
||||||
|
format_time_range,
|
||||||
|
render_post,
|
||||||
|
venue_from_description,
|
||||||
|
venue_from_location,
|
||||||
|
)
|
||||||
|
|
||||||
|
LONDON = ZoneInfo("Europe/London")
|
||||||
|
|
||||||
|
|
||||||
|
def make_event(**overrides) -> Event:
|
||||||
|
defaults = dict(
|
||||||
|
key="uid|2026-11-14T16:30:00+00:00",
|
||||||
|
uid="uid",
|
||||||
|
series_id="tec:1",
|
||||||
|
summary="Folk Night",
|
||||||
|
start=datetime(2026, 11, 14, 16, 30, tzinfo=LONDON).astimezone(timezone.utc),
|
||||||
|
end=datetime(2026, 11, 14, 18, 30, tzinfo=LONDON).astimezone(timezone.utc),
|
||||||
|
all_day=False,
|
||||||
|
location="Old Print Works\n12 High St\nBirmingham\nB12 9AH",
|
||||||
|
description="",
|
||||||
|
url="https://commonscalendar.uk/event/folk-night/",
|
||||||
|
image_url="",
|
||||||
|
tz="Europe/London",
|
||||||
|
)
|
||||||
|
defaults.update(overrides)
|
||||||
|
return Event(**defaults)
|
||||||
|
|
||||||
|
|
||||||
|
def test_render_matches_agreed_format():
|
||||||
|
text = render_post(
|
||||||
|
make_event(),
|
||||||
|
FormatConfig(),
|
||||||
|
LONDON,
|
||||||
|
venue="Old Print Works",
|
||||||
|
postcode="B12 9AH",
|
||||||
|
link="https://commonscalendar.uk/event/folk-night/",
|
||||||
|
)
|
||||||
|
assert text == (
|
||||||
|
"Folk Night\n"
|
||||||
|
"#November #November14\n"
|
||||||
|
"4.30-6.30pm\n"
|
||||||
|
"Old Print Works B12 9AH\n"
|
||||||
|
"https://commonscalendar.uk/event/folk-night/"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_time_range_same_meridiem():
|
||||||
|
start = datetime(2026, 11, 14, 16, 30, tzinfo=LONDON)
|
||||||
|
end = datetime(2026, 11, 14, 18, 30, tzinfo=LONDON)
|
||||||
|
assert format_time_range(start, end) == "4.30-6.30pm"
|
||||||
|
|
||||||
|
|
||||||
|
def test_time_range_crosses_meridiem():
|
||||||
|
start = datetime(2026, 11, 14, 11, 30, tzinfo=LONDON)
|
||||||
|
end = datetime(2026, 11, 14, 13, 30, tzinfo=LONDON)
|
||||||
|
assert format_time_range(start, end) == "11.30am-1.30pm"
|
||||||
|
|
||||||
|
|
||||||
|
def test_hashtags():
|
||||||
|
assert format_hashtags(datetime(2026, 11, 14, 10, 0, tzinfo=LONDON)) == (
|
||||||
|
"#November #November14"
|
||||||
|
)
|
||||||
|
assert format_hashtags(datetime(2026, 1, 1, 10, 0, tzinfo=LONDON)) == "#January #January1"
|
||||||
|
|
||||||
|
|
||||||
|
def test_all_day_event_says_all_day():
|
||||||
|
event = make_event(
|
||||||
|
all_day=True,
|
||||||
|
start=datetime(2026, 11, 21, tzinfo=LONDON).astimezone(timezone.utc),
|
||||||
|
end=datetime(2026, 11, 22, tzinfo=LONDON).astimezone(timezone.utc),
|
||||||
|
)
|
||||||
|
text = render_post(event, FormatConfig(), LONDON, venue="", postcode="", link="")
|
||||||
|
assert "All day" in text
|
||||||
|
assert "#November #November21" in text
|
||||||
|
|
||||||
|
|
||||||
|
def test_postcode_and_venue_helpers():
|
||||||
|
assert find_postcode("Old Print Works, B12 9AH") == "B12 9AH"
|
||||||
|
assert find_postcode("no code here") == ""
|
||||||
|
assert venue_from_location("Old Print Works\nB12 9AH") == "Old Print Works"
|
||||||
|
assert venue_from_description("Collier Room, Methodist Central Hall, M1 1JQ") == "Collier Room"
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_fields_are_omitted():
|
||||||
|
event = make_event(location="", url="")
|
||||||
|
text = render_post(event, FormatConfig(), LONDON, venue="", postcode="", link="")
|
||||||
|
lines = text.splitlines()
|
||||||
|
assert lines == ["Folk Night", "#November #November14", "4.30-6.30pm"]
|
||||||
@@ -0,0 +1,246 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, time, timedelta, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from zoneinfo import ZoneInfo
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from calbot.config import (
|
||||||
|
AnnounceConfig,
|
||||||
|
CalendarConfig,
|
||||||
|
Config,
|
||||||
|
FormatConfig,
|
||||||
|
ImagesConfig,
|
||||||
|
PostingConfig,
|
||||||
|
RemindersConfig,
|
||||||
|
SignalConfig,
|
||||||
|
WatchdogConfig,
|
||||||
|
)
|
||||||
|
from calbot.expand import Event
|
||||||
|
from calbot.feed import Feed, normalize_feed_url
|
||||||
|
from calbot.scheduler import (
|
||||||
|
Bot,
|
||||||
|
clamp_send_at,
|
||||||
|
in_window,
|
||||||
|
next_window_start,
|
||||||
|
reminder_time,
|
||||||
|
)
|
||||||
|
from calbot.store import PageInfo, Store
|
||||||
|
from calbot.watchdog import Watchdog
|
||||||
|
|
||||||
|
LONDON = ZoneInfo("Europe/London")
|
||||||
|
UTC = timezone.utc
|
||||||
|
WINDOWS = ((time(8, 0), time(10, 0)), (time(18, 0), time(21, 0)))
|
||||||
|
GUARD = timedelta(hours=1)
|
||||||
|
|
||||||
|
|
||||||
|
def local(*args) -> datetime:
|
||||||
|
return datetime(*args, tzinfo=LONDON)
|
||||||
|
|
||||||
|
|
||||||
|
def make_event(
|
||||||
|
*,
|
||||||
|
uid: str = "1",
|
||||||
|
series_id: str = "tec:1",
|
||||||
|
summary: str = "Folk Night",
|
||||||
|
start: datetime,
|
||||||
|
end: datetime | None = None,
|
||||||
|
all_day: bool = False,
|
||||||
|
) -> Event:
|
||||||
|
return Event(
|
||||||
|
key=f"{uid}|{start.astimezone(UTC).isoformat()}",
|
||||||
|
uid=uid,
|
||||||
|
series_id=series_id,
|
||||||
|
summary=summary,
|
||||||
|
start=start.astimezone(UTC),
|
||||||
|
end=(end or start + timedelta(hours=2)).astimezone(UTC),
|
||||||
|
all_day=all_day,
|
||||||
|
location="Old Print Works\nB12 9AH",
|
||||||
|
url="https://commonscalendar.uk/event/folk-night/",
|
||||||
|
tz="Europe/London",
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def make_config(tmp_path, **overrides) -> Config:
|
||||||
|
defaults = dict(
|
||||||
|
signal=SignalConfig(
|
||||||
|
api_url="http://signal-api:8080", number="+447700900000", group_id="group.test"
|
||||||
|
),
|
||||||
|
calendar=CalendarConfig(url="https://example.test/feed.ics", timezone="Europe/London"),
|
||||||
|
posting=PostingConfig(),
|
||||||
|
announce=AnnounceConfig(),
|
||||||
|
reminders=RemindersConfig(),
|
||||||
|
images=ImagesConfig(),
|
||||||
|
format=FormatConfig(),
|
||||||
|
watchdog=WatchdogConfig(enabled=False),
|
||||||
|
db_path=tmp_path / "db.sqlite3",
|
||||||
|
cache_dir=tmp_path / "cache",
|
||||||
|
)
|
||||||
|
defaults.update(overrides)
|
||||||
|
return Config(**defaults)
|
||||||
|
|
||||||
|
|
||||||
|
# -- window helpers ---------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_webcal_is_fetched_over_https():
|
||||||
|
assert normalize_feed_url("webcal://example.test/x.ics") == "https://example.test/x.ics"
|
||||||
|
assert normalize_feed_url("https://example.test/x.ics") == "https://example.test/x.ics"
|
||||||
|
|
||||||
|
|
||||||
|
def test_in_window():
|
||||||
|
assert in_window(local(2026, 11, 18, 9, 0), WINDOWS)
|
||||||
|
assert not in_window(local(2026, 11, 18, 15, 0), WINDOWS)
|
||||||
|
assert not in_window(local(2026, 11, 18, 21, 0), WINDOWS)
|
||||||
|
|
||||||
|
|
||||||
|
def test_next_window_start():
|
||||||
|
assert next_window_start(local(2026, 11, 18, 7, 0), WINDOWS) == local(2026, 11, 18, 8, 0)
|
||||||
|
assert next_window_start(local(2026, 11, 18, 10, 30), WINDOWS) == local(2026, 11, 18, 18, 0)
|
||||||
|
assert next_window_start(local(2026, 11, 18, 21, 30), WINDOWS) == local(2026, 11, 19, 8, 0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_clamp_defers_to_evening():
|
||||||
|
due = local(2026, 11, 18, 14, 0)
|
||||||
|
assert clamp_send_at(due, due, local(2026, 11, 25, 18, 0), WINDOWS, GUARD) == local(
|
||||||
|
2026, 11, 18, 18, 0
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def test_clamp_respects_urgent_guard():
|
||||||
|
due = local(2026, 11, 18, 18, 30)
|
||||||
|
start = local(2026, 11, 18, 19, 0)
|
||||||
|
# Next window is tomorrow 08:00, which is after the event: send now.
|
||||||
|
assert clamp_send_at(due, due, start, WINDOWS, GUARD) == due
|
||||||
|
|
||||||
|
|
||||||
|
# -- reminder planning ------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
def test_reminder_three_days_before_lands_in_window():
|
||||||
|
event = make_event(start=local(2026, 11, 21, 14, 0))
|
||||||
|
config = make_config(Path("/tmp"), posting=PostingConfig())
|
||||||
|
due = reminder_time(event, "3d", local(2026, 11, 10, 12, 0), config, LONDON)
|
||||||
|
assert due == local(2026, 11, 18, 18, 0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_reminder_falls_back_to_day_of():
|
||||||
|
event = make_event(start=local(2026, 11, 20, 14, 0))
|
||||||
|
config = make_config(Path("/tmp"))
|
||||||
|
due = reminder_time(event, "3d", local(2026, 11, 19, 12, 0), config, LONDON)
|
||||||
|
assert due == local(2026, 11, 20, 9, 0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_reminder_day_of_moves_before_early_event():
|
||||||
|
event = make_event(start=local(2026, 11, 20, 8, 30))
|
||||||
|
config = make_config(Path("/tmp"))
|
||||||
|
due = reminder_time(event, "3d", local(2026, 11, 19, 12, 0), config, LONDON)
|
||||||
|
assert due == local(2026, 11, 20, 7, 30)
|
||||||
|
|
||||||
|
|
||||||
|
def test_reminder_disabled_fallback_returns_none():
|
||||||
|
event = make_event(start=local(2026, 11, 20, 14, 0))
|
||||||
|
config = make_config(
|
||||||
|
Path("/tmp"), reminders=RemindersConfig(fallback_to_day_of=False)
|
||||||
|
)
|
||||||
|
assert reminder_time(event, "3d", local(2026, 11, 19, 12, 0), config, LONDON) is None
|
||||||
|
|
||||||
|
|
||||||
|
# -- end to end -------------------------------------------------------------
|
||||||
|
|
||||||
|
|
||||||
|
class FakeSender:
|
||||||
|
def __init__(self):
|
||||||
|
self.sent: list[dict] = []
|
||||||
|
|
||||||
|
def send(self, message, recipients, attachments=None, text_mode="normal"):
|
||||||
|
self.sent.append(
|
||||||
|
{"message": message, "recipients": recipients, "attachments": attachments}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeImages:
|
||||||
|
def data_uri(self, url):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
class FakeScraper:
|
||||||
|
def details(self, url):
|
||||||
|
return PageInfo(image_url="", venue="", postcode="")
|
||||||
|
|
||||||
|
|
||||||
|
def build_bot(tmp_path, sender):
|
||||||
|
config = make_config(tmp_path)
|
||||||
|
store = Store(config.db_path)
|
||||||
|
http = httpx.Client()
|
||||||
|
bot = Bot(
|
||||||
|
config=config,
|
||||||
|
store=store,
|
||||||
|
feed=Feed(config.calendar.url, store, 15, http),
|
||||||
|
sender=sender,
|
||||||
|
images=FakeImages(),
|
||||||
|
scraper=FakeScraper(),
|
||||||
|
watchdog=Watchdog(config.watchdog, sender, "+447700900000"),
|
||||||
|
http=http,
|
||||||
|
)
|
||||||
|
return bot, store
|
||||||
|
|
||||||
|
|
||||||
|
def test_first_run_seeds_silently_then_announces_new_series(tmp_path):
|
||||||
|
sender = FakeSender()
|
||||||
|
bot, store = build_bot(tmp_path, sender)
|
||||||
|
|
||||||
|
existing = make_event(uid="1", series_id="tec:1", start=local(2026, 11, 20, 9, 0))
|
||||||
|
bot._sync([existing], local(2026, 11, 18, 12, 0))
|
||||||
|
assert store.due_outbox(local(2026, 11, 18, 18, 0)) == []
|
||||||
|
|
||||||
|
newcomer = make_event(
|
||||||
|
uid="2", series_id="tec:2", summary="Film Night", start=local(2026, 11, 25, 19, 0)
|
||||||
|
)
|
||||||
|
bot._sync([existing, newcomer], local(2026, 11, 18, 12, 1))
|
||||||
|
pending = store.due_outbox(local(2026, 11, 18, 18, 0))
|
||||||
|
assert [item.kind for item in pending] == ["announce"]
|
||||||
|
|
||||||
|
bot._dispatch(local(2026, 11, 18, 18, 0))
|
||||||
|
assert len(sender.sent) == 1
|
||||||
|
assert sender.sent[0]["recipients"] == ["group.test"]
|
||||||
|
assert sender.sent[0]["message"].startswith("Film Night")
|
||||||
|
assert "#November #November25" in sender.sent[0]["message"]
|
||||||
|
|
||||||
|
# A later occurrence of the same series must not be announced again.
|
||||||
|
third = make_event(
|
||||||
|
uid="3", series_id="tec:2", summary="Film Night", start=local(2026, 12, 2, 19, 0)
|
||||||
|
)
|
||||||
|
bot._sync([existing, newcomer, third], local(2026, 11, 18, 18, 5))
|
||||||
|
assert store.due_outbox(local(2026, 11, 18, 18, 10)) == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_reminder_is_sent_at_its_window(tmp_path):
|
||||||
|
sender = FakeSender()
|
||||||
|
bot, store = build_bot(tmp_path, sender)
|
||||||
|
|
||||||
|
event = make_event(uid="9", series_id="tec:9", start=local(2026, 11, 25, 14, 0))
|
||||||
|
bot._sync([event], local(2026, 11, 18, 12, 0)) # seeds; still plans reminders
|
||||||
|
bot._dispatch(local(2026, 11, 18, 18, 0))
|
||||||
|
assert sender.sent == []
|
||||||
|
|
||||||
|
# Three days before is 22 Nov 14:00, deferred to the 18:00 window.
|
||||||
|
bot._dispatch(local(2026, 11, 22, 18, 5))
|
||||||
|
assert len(sender.sent) == 1
|
||||||
|
assert sender.sent[0]["message"].startswith("Folk Night")
|
||||||
|
|
||||||
|
# Idempotent: dispatching again must not resend.
|
||||||
|
bot._dispatch(local(2026, 11, 22, 18, 10))
|
||||||
|
assert len(sender.sent) == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_stale_reminder_is_dropped_not_sent(tmp_path):
|
||||||
|
sender = FakeSender()
|
||||||
|
bot, store = build_bot(tmp_path, sender)
|
||||||
|
|
||||||
|
event = make_event(uid="10", series_id="tec:10", start=local(2026, 11, 18, 15, 0))
|
||||||
|
bot._sync([event], local(2026, 11, 18, 12, 0))
|
||||||
|
bot._dispatch(local(2026, 11, 18, 18, 0)) # event already started
|
||||||
|
assert sender.sent == []
|
||||||
|
assert store.due_outbox(local(2026, 11, 18, 18, 5)) == []
|
||||||
Reference in New Issue
Block a user