From b0f77831bd10be9566a95cee889a3c8061295a6e Mon Sep 17 00:00:00 2001 From: rra Date: Thu, 2 Jun 2022 06:45:54 +0200 Subject: [PATCH 1/4] add 'contributors' as metadata category --- lumbunglib/templates/feed.md | 1 + lumbunglib/templates/hashtag.md | 3 ++- lumbunglib/templates/video.md | 1 + 3 files changed, 4 insertions(+), 1 deletion(-) diff --git a/lumbunglib/templates/feed.md b/lumbunglib/templates/feed.md index 71d984b..29d8e91 100644 --- a/lumbunglib/templates/feed.md +++ b/lumbunglib/templates/feed.md @@ -7,6 +7,7 @@ authors: {% if frontmatter.author %} ["{{ frontmatter.author }}"] {% endif %} original_link: "{{ frontmatter.original_link }}" feed_name: "{{ frontmatter.feed_name}}" categories: ["{{ frontmatter.card_type }}", "{{ frontmatter.feed_name}}"] +contributors: ["{{ frontmatter.feed_name}}"] tags: {{ frontmatter.tags }} {% if frontmatter.featured_image %}featured_image: "{{frontmatter.featured_image}}"{% endif %} --- diff --git a/lumbunglib/templates/hashtag.md b/lumbunglib/templates/hashtag.md index bdcacba..4a61a7a 100644 --- a/lumbunglib/templates/hashtag.md +++ b/lumbunglib/templates/hashtag.md @@ -2,11 +2,12 @@ date: {{ post_metadata.created_at }} #2021-06-10T10:46:33+02:00 draft: false authors: ["{{ post_metadata.account.display_name }}"] +contributors: ["{{ post_metadata.account.acct}}"] avatar: {{ post_metadata.account.avatar }} categories: ["shouts"] images: [{% for i in post_metadata.media_attachments %} {{ i.url }}, {% endfor %}] title: {{ post_metadata.account.display_name }} -tags: [{% for i in post_metadata.tags %} {{ i.name }} {% endfor %}] +tags: [{% for i in post_metadata.tags %} "{{ i.name }}", {% endfor %}] --- {% for item in post_metadata.media_attachments %} diff --git a/lumbunglib/templates/video.md b/lumbunglib/templates/video.md index 96761ec..076a1bf 100644 --- a/lumbunglib/templates/video.md +++ b/lumbunglib/templates/video.md @@ -6,6 +6,7 @@ uuid: "{{v.uuid}}" video_duration: "{{ v.duration | duration }} " video_channel: "{{ v.channel.display_name }}" channel_url: "{{ v.channel.url }}" +contributors: ["{{ v.account.display_name }}"] preview_image: "{{ preview_image }}" images: ["./{{ preview_image }}"] categories: ["tv","{{ v.channel.display_name }}"] From 00f795f16de4018632109b6e1415f95afe9d3e75 Mon Sep 17 00:00:00 2001 From: rra Date: Thu, 2 Jun 2022 09:23:58 +0200 Subject: [PATCH 2/4] rename project to konfluks for legibility, add docs --- README.md | 54 ++++++++++++++++++- docs/konfluks.svg | 31 +++++++++++ {lumbunglib => konfluks}/cloudcal.py | 0 {lumbunglib => konfluks}/feed.py | 0 {lumbunglib => konfluks}/hashtag.py | 0 .../templates/calendar.md | 0 {lumbunglib => konfluks}/templates/feed.md | 0 {lumbunglib => konfluks}/templates/hashtag.md | 0 {lumbunglib => konfluks}/templates/video.md | 0 {lumbunglib => konfluks}/video.py | 0 pyproject.toml | 12 ++--- setup.py | 14 ++--- 12 files changed, 96 insertions(+), 15 deletions(-) create mode 100644 docs/konfluks.svg rename {lumbunglib => konfluks}/cloudcal.py (100%) rename {lumbunglib => konfluks}/feed.py (100%) rename {lumbunglib => konfluks}/hashtag.py (100%) rename {lumbunglib => konfluks}/templates/calendar.md (100%) rename {lumbunglib => konfluks}/templates/feed.md (100%) rename {lumbunglib => konfluks}/templates/hashtag.md (100%) rename {lumbunglib => konfluks}/templates/video.md (100%) rename {lumbunglib => konfluks}/video.py (100%) diff --git a/README.md b/README.md index 3deeeb2..85a1be8 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,56 @@ -# lumbunglib +![Konfluks logo is a stylized and schematic representation of a drainage basin](docs/Konfluks.svg) + +# Konfluks + +A drainage basin is a geographical feature that collects all precipitation in an area, first in to smaller streams and finally together in to the large river. Similarly, Konfluks can bring together small and dispersed streams of web content from different applications and websites together in a single large stream. + +Specifically, Konfluks turns Peertube videos, iCal calendar events, other websites through their RSS and OPDS feeds and Mastodon posts under a hashtag in to Hugo page bundles. This allows one to publish from diverse sources to a single stream. + +Konfluks was first made by Roel Roscam Abbing as part of , together with ruangrupa and Autonomic. + +## Philosophy + +Konfluks tries to act as a mirror representation of the input sources. That means that whenever something remote is deleted, changed or becomes unavailable, it is also changed or deleted by Konfluks. + +Konfluks tries to preserve intention. That means the above, but also by requiring explicit ways of publishing. + +Konfluks works by periodically polling the remote sources, taking care not to duplicate work. It caches files, asks for last-modified headers, and skips things it has already. This makes every poll as fast and as light as possible. + +Konfluks is written for clarity, not brevity nor cleverness. + +Konfluks is extendable, a work in progress and a messy undertaking. + +## High-level overview + +Konfluks consists of different Python scripts which each poll a particular service, say, a Peertube server, to download information and convert it in to [Hugo Page Bundles](https://gohugo.io/content-management/page-bundles/) + +Each script part of Konfluks will essentially to the following: + * Parse a source and request posts/updates/videos/a feed + * Taking care of publish ques + * Create a Hugo post for each item returned, by: + * Making a folder per post in the `output` directory + * Formatting post metadata as [Hugo Post Frontmatter](https://gohugo.io/content-management/front-matter/) in a file called `index.md` + * Grabbing local copies of media and saving them in the post folder + * Adding the post content to `index.md` + * According to jinja2 templates (see `Konfluks/templates/`) + +The page bundles created, where possible, are given human friendly names. + + +Here is a typical output structure: +`user@server: ~/Konfluks/output: tree tv/ +tv/ +├── forum-27an-mother-earth-353f93f3-5fee-49d6-b71d-8aef753f7041 +│   ├── 86ccae63-3df9-443c-91f3-edce146055db.jpg +│   └── index.md +├── keroncong-tugu-cafrinho-live-at-ruru-gallery-ruangrupa-jakarta-19-august-2014-e6d5bb2a-d77f-4a00-a449-992a579c8c0d +│   ├── 32291aa2-a391-4219-a413-87521ff373ba.jpg +│   └── index.md +├── lecture-series-1-camp-notes-on-education-8d54d3c9-0322-42af-ab6e-e954d251e076 +│   ├── 0f3c835b-42c2-48a3-a2a3-a75ddac8688a.jpg +│   └── index.md +` -> Python lib which powers `lumbung.space` automation ## hacking diff --git a/docs/konfluks.svg b/docs/konfluks.svg new file mode 100644 index 0000000..1a91ef3 --- /dev/null +++ b/docs/konfluks.svg @@ -0,0 +1,31 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/lumbunglib/cloudcal.py b/konfluks/cloudcal.py similarity index 100% rename from lumbunglib/cloudcal.py rename to konfluks/cloudcal.py diff --git a/lumbunglib/feed.py b/konfluks/feed.py similarity index 100% rename from lumbunglib/feed.py rename to konfluks/feed.py diff --git a/lumbunglib/hashtag.py b/konfluks/hashtag.py similarity index 100% rename from lumbunglib/hashtag.py rename to konfluks/hashtag.py diff --git a/lumbunglib/templates/calendar.md b/konfluks/templates/calendar.md similarity index 100% rename from lumbunglib/templates/calendar.md rename to konfluks/templates/calendar.md diff --git a/lumbunglib/templates/feed.md b/konfluks/templates/feed.md similarity index 100% rename from lumbunglib/templates/feed.md rename to konfluks/templates/feed.md diff --git a/lumbunglib/templates/hashtag.md b/konfluks/templates/hashtag.md similarity index 100% rename from lumbunglib/templates/hashtag.md rename to konfluks/templates/hashtag.md diff --git a/lumbunglib/templates/video.md b/konfluks/templates/video.md similarity index 100% rename from lumbunglib/templates/video.md rename to konfluks/templates/video.md diff --git a/lumbunglib/video.py b/konfluks/video.py similarity index 100% rename from lumbunglib/video.py rename to konfluks/video.py diff --git a/pyproject.toml b/pyproject.toml index 20ba70a..c94fcf0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,9 +1,9 @@ [tool.poetry] -name = "lumbunglib" +name = "konfluks" version = "0.1.0" description = "Python lib which powers lumbung[dot]space automation" authors = ["rra", "decentral1se"] -license = "GPLv3+" +license = "AGPLv3+" [tool.poetry.dependencies] python = "^3.9" @@ -25,7 +25,7 @@ requires = ["poetry-core>=1.0.0"] build-backend = "poetry.core.masonry.api" [tool.poetry.scripts] -lumbunglib-cal = "lumbunglib.cloudcal:main" -lumbunglib-vid = "lumbunglib.video:main" -lumbunglib-feed = "lumbunglib.feed:main" -lumbunglib-hash = "lumbunglib.hashtag:main" +konfluks-cal = "konfluks.cloudcal:main" +konfluks-vid = "konfluks.video:main" +konfluks-feed = "konfluks.feed:main" +konfluks-hash = "konfluks.hashtag:main" diff --git a/setup.py b/setup.py index e1e1af7..34c0444 100644 --- a/setup.py +++ b/setup.py @@ -2,10 +2,10 @@ from setuptools import setup packages = \ -['lumbunglib'] +['konfluks'] package_data = \ -{'': ['*'], 'lumbunglib': ['templates/*']} +{'': ['*'], 'konfluks': ['templates/*']} install_requires = \ ['Jinja2>=3.0.3,<4.0.0', @@ -20,13 +20,13 @@ install_requires = \ 'requests>=2.26.0,<3.0.0'] entry_points = \ -{'console_scripts': ['lumbunglib-cal = lumbunglib.cloudcal:main', - 'lumbunglib-feed = lumbunglib.feed:main', - 'lumbunglib-hash = lumbunglib.hashtag:main', - 'lumbunglib-vid = lumbunglib.video:main']} +{'console_scripts': ['konfluks-cal = konfluks.cloudcal:main', + 'konfluks-feed = konfluks.feed:main', + 'konfluks-hash = konfluks.hashtag:main', + 'konfluks-vid = konfluks.video:main']} setup_kwargs = { - 'name': 'lumbunglib', + 'name': 'konfluks', 'version': '0.1.0', 'description': 'Python lib which powers lumbung[dot]space automation', 'long_description': None, From f162bb946a4e8b61f81d3511f7cb3432eb0712db Mon Sep 17 00:00:00 2001 From: rra Date: Thu, 2 Jun 2022 09:28:37 +0200 Subject: [PATCH 3/4] Update 'README.md' correcting markup / styling --- README.md | 46 ++++++++++++++++++++++++---------------------- 1 file changed, 24 insertions(+), 22 deletions(-) diff --git a/README.md b/README.md index 85a1be8..f8b2b2b 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -![Konfluks logo is a stylized and schematic representation of a drainage basin](docs/Konfluks.svg) +![Konfluks logo is a stylized and schematic representation of a drainage basin](https://git.autonomic.zone/r/konfluks/raw/branch/konfluks-renaming/docs/konfluks.svg) # Konfluks @@ -25,34 +25,36 @@ Konfluks is extendable, a work in progress and a messy undertaking. Konfluks consists of different Python scripts which each poll a particular service, say, a Peertube server, to download information and convert it in to [Hugo Page Bundles](https://gohugo.io/content-management/page-bundles/) Each script part of Konfluks will essentially to the following: - * Parse a source and request posts/updates/videos/a feed - * Taking care of publish ques - * Create a Hugo post for each item returned, by: - * Making a folder per post in the `output` directory - * Formatting post metadata as [Hugo Post Frontmatter](https://gohugo.io/content-management/front-matter/) in a file called `index.md` - * Grabbing local copies of media and saving them in the post folder - * Adding the post content to `index.md` - * According to jinja2 templates (see `Konfluks/templates/`) +* Parse a source and request posts/updates/videos/a feed + * Taking care of publish ques +* Create a Hugo post for each item returned, by: + * Making a folder per post in the `output` directory + * Formatting post metadata as [Hugo Post Frontmatter](https://gohugo.io/content-management/front-matter/) in a file called `index.md` + * Grabbing local copies of media and saving them in the post folder + * Adding the post content to `index.md` + * According to jinja2 templates (see `Konfluks/templates/`) The page bundles created, where possible, are given human friendly names. Here is a typical output structure: -`user@server: ~/Konfluks/output: tree tv/ -tv/ -├── forum-27an-mother-earth-353f93f3-5fee-49d6-b71d-8aef753f7041 -│   ├── 86ccae63-3df9-443c-91f3-edce146055db.jpg -│   └── index.md -├── keroncong-tugu-cafrinho-live-at-ruru-gallery-ruangrupa-jakarta-19-august-2014-e6d5bb2a-d77f-4a00-a449-992a579c8c0d -│   ├── 32291aa2-a391-4219-a413-87521ff373ba.jpg -│   └── index.md -├── lecture-series-1-camp-notes-on-education-8d54d3c9-0322-42af-ab6e-e954d251e076 -│   ├── 0f3c835b-42c2-48a3-a2a3-a75ddac8688a.jpg -│   └── index.md -` +``` + user@server: ~/Konfluks/output: tree tv/ + tv/ + ├── forum-27an-mother-earth-353f93f3-5fee-49d6-b71d-8aef753f7041 + │   ├── 86ccae63-3df9-443c-91f3-edce146055db.jpg + │   └── index.md + ├── keroncong-tugu-cafrinho-live-at-ruru-gallery-ruangrupa-jakarta-19-august-2014-e6d5bb2a-d77f-4a00-a449-992a579c8c0d + │   ├── 32291aa2-a391-4219-a413-87521ff373ba.jpg + │   └── index.md + ├── lecture-series-1-camp-notes-on-education-8d54d3c9-0322-42af-ab6e-e954d251e076 + │   ├── 0f3c835b-42c2-48a3-a2a3-a75ddac8688a.jpg + │   └── index.md +``` -## hacking + +## Hacking Install [poetry](https://python-poetry.org/docs/#osx--linux--bashonwindows-install-instructions): From 845a54787b9dc0b0d272b50c78e072aa04e4add6 Mon Sep 17 00:00:00 2001 From: rra Date: Thu, 2 Jun 2022 09:29:20 +0200 Subject: [PATCH 4/4] Update 'README.md' --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f8b2b2b..1af1154 100644 --- a/README.md +++ b/README.md @@ -6,7 +6,7 @@ A drainage basin is a geographical feature that collects all precipitation in an Specifically, Konfluks turns Peertube videos, iCal calendar events, other websites through their RSS and OPDS feeds and Mastodon posts under a hashtag in to Hugo page bundles. This allows one to publish from diverse sources to a single stream. -Konfluks was first made by Roel Roscam Abbing as part of , together with ruangrupa and Autonomic. +Konfluks was first made by Roel Roscam Abbing as part of [lumbung.space](https://lumbung.space), together with ruangrupa and Autonomic. ## Philosophy