Masto posts with n+1 hashtags are downloaded n+1 times #28

New Issue

decentral1se · 2022-01-25T21:03:30Z

decentral1se commented

2022-01-25 21:03:30 +00:00

tagging a post with #hash1, #hash2 means it is downloaded twice into hash1/ and hash2/ which means it will be shown twice on lumbung.space. we should only download it once for one hashtag? or somehow we should handle duplicates, I guess it should be downloaded but not shown twice on the listing (i.e. if you click #hash1 you should still see it and if you click #hash2 you should still see it...)

tagging a post with `#hash1`, `#hash2` means it is downloaded twice into `hash1/` and `hash2/` which means it will be shown twice on lumbung.space. we should only download it once for one hashtag? or somehow we should handle duplicates, I guess it should be downloaded but not shown twice on the listing (i.e. if you click `#hash1` you should still see it and if you click `#hash2` you should still see it...)

decentral1se changed title from ~~masto posts with n+1 hashtags are downloaded n+1 times~~ to Masto posts with n+1 hashtags are downloaded n+1 times

2022-01-26 08:47:02 +00:00

decentral1se added this to the (deleted) project 2022-01-26 08:48:26 +00:00

r commented

2022-01-26 08:57:08 +00:00

I think at the root it has to do with how the data is pulled in and checked for existence. Probably two seperate posts are now made. However, posts can have multiple categories (tag a, tag b) and a single post can thusly be shown in different category pages. So the issue should be with https://git.autonomic.zone/ruangrupa/lumbunglib/src/branch/master/lumbunglib/hashtag.py rather than the hugo theme.

The solution is to add a logic to check whether:

there are multiple hashtags in the post
then whether multiple of those are in the allowlist
if multiple are in the allow list
check for existence of same post with different tag
if it exits, skip.

I think at the root it has to do with how the data is pulled in and checked for existence. Probably two seperate posts are now made. However, posts can have multiple categories (tag a, tag b) and a single post can thusly be shown in different category pages. So the issue should be with https://git.autonomic.zone/ruangrupa/lumbunglib/src/branch/master/lumbunglib/hashtag.py rather than the hugo theme. The solution is to add a logic to check whether: * there are multiple hashtags in the post * then whether multiple of those are in the allowlist * if multiple are in the allow list * check for existence of same post with different tag * if it exits, skip.

knoflook self-assigned this 2022-01-27 10:22:42 +00:00

decentral1se commented

2022-01-27 10:54:32 +00:00

Hmmm, nice. If we skip downloading the post tho, then the link on the hugo theme side will 404, so I think we still need to download them? Perhaps just filtering on the theming side could also be a fix? Investigating...

knoflook referenced this issue from ruangrupa/konfluks

2022-01-27 16:56:25 +00:00

fix: don't pull the same post twice #19

knoflook commented

2022-01-27 19:20:40 +00:00

see ruangrupa/lumbunglib#19

see https://git.autonomic.zone/ruangrupa/lumbunglib/pulls/19

decentral1se commented

2022-01-28 12:53:19 +00:00

We tried to fix this today but there are still issues in the implementation of lumbunglib, so, for now, we've disabled the socials contente generation in 5ad74ef1cc and removed the duplicates manually. the fix is Coming Soon ™️

We tried to fix this today but there are still issues in the implementation of lumbunglib, so, for now, we've disabled the socials contente generation in https://git.autonomic.zone/ruangrupa/lumbung.space/commit/5ad74ef1cc3b7a86900f6573baf2079a5c495359 and removed the duplicates manually. the fix is Coming Soon ™️

knoflook referenced this issue from ruangrupa/konfluks

2022-01-28 14:50:24 +00:00

fix: don't pull the same post twice but for real #22

knoflook commented

2022-01-31 13:33:16 +00:00

closed in ruangrupa/lumbunglib#22

closed in https://git.autonomic.zone/ruangrupa/lumbunglib/pulls/22

👍 1

knoflook closed this issue

2022-01-31 13:33:16 +00:00

Sign in to join this conversation.