* Add tests
* Ensure deleted statuses are marked as such
* Save some redis memory by not storing URIs in delete_upon_arrival values
* Avoid possible race condition when processing incoming Deletes
* Avoid potential duplicate Delete forwards
* Lower lock durations to reduce issues in case of hard crash of the Rails process
* Check for `lock.aquired?` and improve comment
* Refactor RedisLock usage in app/lib/activitypub
* Fix using incorrect or non-existent sender for relaying Deletes
Prevent new accounts from being created using the same underlying
e-mail as a suspended account using extensions and period
permutations. Stores e-mails as a SHA256 hash
* Update devise-two-factor to unreleased fork for Rails 6 support
Update tests to match new `rotp` version.
* Update nsa gem to unreleased fork for Rails 6 support
* Update rails to 6.1.3 and rails-i18n to 6.0
* Update to unreleased fork of pluck_each for Ruby 6 support
* Run "rails app:update"
* Add missing ActiveStorage config file
* Use config.ssl_options instead of removed ApplicationController#force_ssl
Disabled force_ssl-related tests as they do not seem to be easily testable
anymore.
* Fix nonce directives by removing Rails 5 specific monkey-patching
* Fix fixture_file_upload deprecation warning
* Fix yield-based test failing with Rails 6
* Use Rails 6's index_with when possible
* Use ActiveRecord::Cache::Store#delete_multi from Rails 6
This will yield better performances when deleting an account
* Disable Rails 6.1's automatic preload link headers
Since Rails 6.1, ActionView adds preload links for javascript files
in the Links header per default.
In our case, that will bloat headers too much and potentially cause
issues with reverse proxies. Furhermore, we don't need those links,
as we already output them as HTML link tags.
* Switch to Rails 6.0 default config
* Switch to Rails 6.1 default config
* Do not include autoload paths in the load path
* Bypass MX validation for explicitly allowed domains
This spares some lookups and prevent issues in some edge cases with
local domains.
* Add tests
* Fix test
* Improve account counters handling
* Use ActiveRecord::Base::sanitize_sql to pass values instead of interpolating them
Keep using string interpolation for `key` as it is safe and using
“ActiveRecord::Base::sanitize_sql_hash_for_assignment” would require stitching
bits of SQL in a way that is not more easily checked for safety.
* Add migration hook to catch PostgreSQL versions earlier than 9.5
* Use ActiveRecord::Result#to_ary instead of deprecated to_hash
They do the same thing, and to_hash has been removed from Rails 6.1
* Explicitly name polymorphic indexes to workaround a bug in Rails 6.1
cf. https://github.com/rails/rails/issues/41693
* Fix incorrect usage of “foreign_key” in migration script
* Use `ActiveModel::Errors#delete` instead of deprecated clear method
* Fix link headers tests on Rails 6.1
Rails 6.1 adds values to the Link header by default, thus it is not a
LinkHeader object anymore. Fix the test to parse the Link header instead
of assuming it is a LinkHeader.
* Prepare Mastodon for zeitwerk autoloader (Rails 6)
Add inflections and rename/move a few classes.
In particular, app/lib/exceptions.rb and app/lib/sanitize_config.rb
were manually loaded while still in autoload paths.
* Add inflection for Url → URL
* Fix misuse of foreign_type
* Fix use of removed "add_template_helper"
* Use response.media_type instead of response.content_type in tests
* Fix CSV export controller test on Rails 6
Rails 6 sets a "filename*" field in the Content-Disposition header to
explicitly encode the filename as UTF-8.
This changes checks the first part of the Content-Disposition header so
it matches in both Rails 5 and Rails 6.
* Fix emoji formatting with Rails 6
* Make emoji output more idiomatic and robust
* Switch from redis-rails gem to built-in Rails redis cache storage
* Update twitter-text from 1.14 to 3.1.0
* Disable emoji parsing
* Properly depend on twitter-text for url detection
* Fix some URLs being wrongly detected client-side
* Add test for server-side validation of non-autolinkable URLs
* Fix server-side status length counting
* Change ResolveAccountService's handling of skip_webfinger
Change it so it never makes any webfinger query, as the name would imply.
* Add tests
* Change FollowService to not take an URI for target_account
* Restore domain-block check in FollowService
* Fix tests
* Fix URI of repeat follow requests not being recorded
In case we receive a “repeat” or “duplicate” follow request, we automatically
fast-forward the accept with the latest received Activity `id`, but we don't
record it.
In general, a “repeat” or “duplicate” follow request may happen if for some
reason (e.g. inconsistent handling of Block or Undo Accept activities, an
instance being brought back up from the dead, etc.) the local instance thought
the remote actor were following them while the remote actor thought otherwise.
In those cases, the remote instance does not know about the older Follow
activity `id`, so keeping that record serves no purpose, but knowing the most
recent one is useful if the remote implementation at some point refers to it
by `id` without inlining it.
* Add tests
* Added .deepsource.toml
* Removed bad use of `alias`
* Fixed operand order in the binary expression
* Prefixed unused method arguments with an underscore
* Replaced the old OpenSSL algorithmic constants with the newer strings initializers.
* Removed unnecessary UTF-8 encoding comment
* Fix being able to import more than allowed number of follows
Without this commit, if someone tries importing a second list of accounts to
follow before the first one has been processed, this will queue imports for
the two whole lists, even if they exceed the account's allowed number of
outgoing follows.
This commit changes it so the individual queued imports aren't exempt from
the follow limit check (they remain exempt from the rate-limiting check
though).
* Catch validation errors to not re-queue failed follows
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Delete status records by batches of 50
* Do not precompute values that are only used once
* Do not generate redis events for removal of public toots older than two weeks
* Filter reported toots a priori for polls and status deletion
* Do not process reblogs when cleaning up public timelines
As in Mastodon proper, reblogs don't appear in public TLs
* Clean the deleted account's own feed in one go
* Refactor Account#clean_feed_manager and List#clean_feed_manager
* Delete instead of destroy a few more associations
* Fix preloading
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix external user creation failing when invite request text is required
Also fixes tootctl-based user creation.
* Add test about invites when invite request text is otherwise required
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix deleting polls not deleting notifications
* Fix fav notification deletion when deleting a toot
* Refactor DeleteAccountService spec
* Add DeleteAccountService tests for other associations and notifications
* Add favourite handling spec in status removal
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix ResolveAccountService accepting mismatching acct: URI
* Set attributes that should be updated regardless of suspension
* Fix key fetching
* Automatically merge remote accounts with duplicate `uri`
* Add tests
* Add "tootctl accounts fix-duplicates"
Finds duplicate accounts sharing a same ActivityPub `id`, re-fetch them and
merge them under the canonical `acct:` URI.
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Improve searching for private toots from URL
Most of the time, when sharing toots, people use the toot URL rather than
the toot URI, which makes sense since it is the user-facing URL.
In Mastodon's case, the URL and URI are different, and Mastodon does not
have an index on URL, which means searching a private toot by URL is done
with a slow query that will only succeed for very recent toots.
This change gets rid of the slow query, and attempts to guess the URI from
URL instead, as Mastodon's are predictable.
* Add tests
* Only return status with guessed uri if url matches
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix admin being able to suspend their own instance account
* Add text about the instance's own actor in admin view
* Change instance actor notice from flash message to template
* Do not list local instance actor in account moderation list
* Add honeypot fields to limit non-specialized spam
Add two honeypot fields: a fake website input and a fake password confirmation
one. The label/placeholder/aria-label tells not to fill them, and they are
hidden in CSS, so legitimate users should not fall into these.
This should cut down on some non-Mastodon-specific spambots.
* Require a 3 seconds delay before submitting the registration form
* Fix tests
* Move registration form time check to model validation
* Give people a chance to clear the honeypot fields
* Refactor honeypot translation strings
Co-authored-by: Claire <claire.github-309c@sitedethib.com>
* Fix webfinger redirect handling in ResolveAccountService
ResolveAccountService#process_webfinger! handled a one-step webfinger
redirection, but only accepting the result if it matched the exact URI passed
as input, defeating the point of a redirection check.
Instead, use the same logic as in `ActivityPub::FetchRemoteAccountService`,
updating the resulting `acct:` URI with the result of the first webfinger
query.
* Add tests
If someone tries logging in to an account and is prompted for a 2FA
code or sign-in token, even if the account's password or e-mail is
updated in the meantime, the session will show the prompt and allow
the login process to complete with a valid 2FA code or sign-in token
* Add support for followers synchronization on the receiving end
Check the `collectionSynchronization` attribute on `Create` and `Announce`
activities and synchronize followers from provided collection if possible.
* Add tests for followers synchronization on the receiving end
* Add support for follower synchronization on the sender's end
* Add tests for the sending end
* Switch from AS attributes to HTTP header
Replace the custom `collectionSynchronization` ActivityStreams attribute by
an HTTP header (`X-AS-Collection-Synchronization`) with the same syntax as
the `Signature` header and the following fields:
- `collectionId` to specify which collection to synchronize
- `digest` for the SHA256 hex-digest of the list of followers known on the
receiving instance (where “receiving instance” is determined by accounts
sharing the same host name for their ActivityPub actor `id`)
- `url` of a collection that should be fetched by the instance actor
Internally, move away from the webfinger-based `domain` attribute and use
account `uri` prefix to group accounts.
* Add environment variable to disable followers synchronization
Since the whole mechanism relies on some new preconditions that, in some
extremely rare cases, might not be met, add an environment variable
(DISABLE_FOLLOWERS_SYNCHRONIZATION) to disable the mechanism altogether and
avoid followers being incorrectly removed.
The current conditions are:
1. all managed accounts' actor `id` and inbox URL have the same URI scheme and
netloc.
2. all accounts whose actor `id` or inbox URL share the same URI scheme and
netloc as a managed account must be managed by the same Mastodon instance
as well.
As far as Mastodon is concerned, breaking those preconditions require extensive
configuration changes in the reverse proxy and might also cause other issues.
Therefore, this environment variable provides a way out for people with highly
unusual configurations, and can be safely ignored for the overwhelming majority
of Mastodon administrators.
* Only set follower synchronization header on non-public statuses
This is to avoid unnecessary computations and allow Follow-related
activities to be handled by the usual codepath instead of going through
the synchronization mechanism (otherwise, any Follow/Undo/Accept activity
would trigger the synchronization mechanism even if processing the activity
itself would be enough to re-introduce synchronization)
* Change how ActivityPub::SynchronizeFollowersService handles follow requests
If the remote lists a local follower which we only know has sent a follow
request, consider the follow request as accepted instead of sending an Undo.
* Integrate review feeback
- rename X-AS-Collection-Synchronization to Collection-Synchronization
- various minor refactoring and code style changes
* Only select required fields when computing followers_hash
* Use actor URI rather than webfinger domain in synchronization endpoint
* Change hash computation to be a XOR of individual hashes
Makes it much easier to be memory-efficient, and avoid sorting discrepancy issues.
* Marginally improve followers_hash computation speed
* Further improve hash computation performances by using pluck_each
