Commit Graph

295 Commits

Author SHA1 Message Date
Eugen Rochko
b8e166894b Fix Scheduler::SubscriptionsScheduler (#2834)
* Fix Scheduler::SubscriptionsScheduler, add worker test for it

* Change production log level of Sidekiq to "warn" instead of "info"
2017-05-06 13:05:03 +02:00
Matt Jankowski
3f5b994ff0 Stream entry specs and refactor to use delegate (#2827)
* Add coverage for stream entry delegated methods

* Use delegate with allow_nil to clean up stream entry
2017-05-06 04:00:21 +02:00
Matt Jankowski
484c9709b6 Misc spec coverage improvements (#2821)
* Dont use raise_error by itself (avoids warning)

* Add coverage for AccountFilter

* Improve coverage and refactor for Subscription#lease_seconds

* Improve coverage and refactor for NotificationMailer

* Simplify assignment of min/max threshold on subscription
2017-05-05 14:56:00 -04:00
masarakki
d08f1112d5 use Twitter::Extractor for creating links (#2502) 2017-05-05 19:48:22 +02:00
Matt Jankowski
20c37ed0f9 Add specs (and refactor) of FetchRemoteResourceService and SearchService (#2812)
* Coverage for fetch remote resource service

* Refactor fetch remote resource service

* Coverage for search service

* Refactor search service
2017-05-05 17:26:04 +02:00
Eugen Rochko
81584779cb More robust PuSH subscription refreshes (#2799)
* Fix #2473 - Use sidekiq scheduler to refresh PuSH subscriptions instead of cron

Fix an issue where / in domain would raise exception in TagManager#normalize_domain

PuSH subscriptions refresh done in a round-robin way to avoid hammering a single
server's hub in sequence. Correct handling of failures/retries through Sidekiq (see
also #2613). Optimize Account#with_followers scope. Also, since subscriptions
are now delegated to Sidekiq jobs, an uncaught exception will not stop the entire
refreshing operation halfway through

Fix #2702 - Correct user agent header on outgoing http requests

* Add test for SubscribeService

* Extract #expiring_accounts into method

* Make mastodon:push:refresh no-op

* Queues are now defined in sidekiq.yml

* Queues are now in sidekiq.yml
2017-05-05 02:23:01 +02:00
Matt Jankowski
f9d398e8fb Remove the react-rails gem (#2801)
* Remove react-rails gem

* Fix broken view spec
2017-05-04 23:45:18 +02:00
alpaca-tc
74c8ca699c Delete records in smaller transaction (#2802) 2017-05-04 23:44:39 +02:00
Matt Jankowski
84eb425f38 Remove redis clear from before in spec (#2795) 2017-05-04 20:53:39 +02:00
Matt Jankowski
2bd132d458 Replace best_in_place editor on admin settings page (#2789)
* Remove best_in_place

* Replace best_in_place usage with rails helpers

* Move admin/settings#index to #edit action

* Remove click_to__edit from i18n
2017-05-04 18:12:44 +02:00
Eugen Rochko
13c16b4e95 Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses (#2734)
* Likely fix #2458, fix #2031 - handle out-of-order deletes for statuses

If a delete arrives before the original status, cache that information
for 6h, and if the original status arrives in that window, ignore it

* Add test case
2017-05-04 04:34:57 +02:00
ThibG
8d4e7504b1 Additional specs for URI handling (#2759) 2017-05-03 20:40:14 +02:00
Eugen Rochko
bafd22ecf4 Fix #2706 - Always respond with 200 to PuSH payloads (#2733)
Fix #2196 - Respond with 201 when Salmon accepted, 400 when unverified
Fix #2629 - Correctly handle confirm_domain? for local accounts
Unify rules for extracting author acct from XML, prefer <email>, fall back
to <name> + <uri> (see also #2017, #2172)
2017-05-03 17:02:18 +02:00
Matt Jankowski
8c5ad23b24 Language improvements, replace whatlanguage with CLD (#2753)
* add failing en specs

* add cld2 gem

* Replace WhatLanguage with CLD
2017-05-03 16:59:31 +02:00
Eugen Rochko
f5bf5ebb82 Replace sprockets/browserify with Webpack (#2617)
* Replace browserify with webpack

* Add react-intl-translations-manager

* Do not minify in development, add offline-plugin for ServiceWorker background cache updates

* Adjust tests and dependencies

* Fix production deployments

* Fix tests

* More optimizations

* Improve travis cache for npm stuff

* Re-run travis

* Add back support for custom.scss as before

* Remove offline-plugin and babili

* Fix issue with Immutable.List().unshift(...values) not working as expected

* Make travis load schema instead of running all migrations in sequence

* Fix missing React import in WarningContainer. Optimize rendering performance by using ImmutablePureComponent instead of
React.PureComponent. ImmutablePureComponent uses Immutable.is() to compare props. Replace dynamic callback bindings in
<UI />

* Add react definitions to places that use JSX

* Add Procfile.dev for running rails, webpack and streaming API at the same time
2017-05-03 02:04:16 +02:00
Matt Jankowski
268dd32d76 Auth sign out (#2511)
* Add a spec for signing out

* Add spec showing that suspended user gets a 403 forbidden on sign out

* Allow suspended account users to sign out
2017-05-02 23:37:58 +02:00
ThibG
bea97ea766 Add rspec to further specify FollowRemoteAccountService (#2414) 2017-05-02 23:37:26 +02:00
Kaylee
7880671f35 Add option to disable two factor auth in admin accounts panel. (#2584)
* Add option to disable two factor auth in admin accounts panel.
Closes #2578

* Add @mjankowski's suggestions.
* Moves destroy actions behind User#disable_two_factor!
* Adds spec coverage for Admin:TwoFactorAuthenticationsController and User#disable_two_factor!
2017-05-02 21:07:12 +02:00
Yamagishi Kazutoshi
6f75c8451d Fix subscription expiration condition (#2715)
* Fix subscription expiration condition

* dry and add spec
2017-05-02 18:21:22 +02:00
Matt Jankowski
a4859446ab Coverage for remote follows (#2694)
* Add coverage for create with empty acct value

* Add coverage for create with webfinger failure

* Add coverage for create with webfinger providing bad values

* Add coverage for create when webfinger is good

* Add coverage for session[:remote_follow] having data

* Simplify how remote follow pulls acct from session

* Remote follow behaves more like model

* Move the discovery portions of remote follow out of controller

* Check for suspended accounts
2017-05-02 00:44:23 +02:00
Matt Jankowski
f025cc6782 Filter on allowed user language preferences (#2361)
* Naive approached to timeline filtering

* Convert allowed_languages into a db column

* Allow users to choose languages to see statuses in

* Style list items as two columns

* Add a hint to explain language filtering preference
2017-05-01 17:42:13 +02:00
Matt Jankowski
5259319cf5 Reports spec for media attachments, clean up method (#2660)
* Add coverage for Report#media_attachments

* Direct query on media attachment
2017-04-30 18:15:49 +02:00
Akihiko Odaki
a6788662b0 Revert "Use PostgreSQL inheritance for blocks and mutes (#2520)" (#2634)
This reverts commit 5135d609b7.
2017-04-30 08:49:24 -04:00
Matt Jankowski
fdcf884cf7 Extract user tracking into concern (#2600) 2017-04-30 00:28:16 +02:00
Akihiko Odaki
5135d609b7 Use PostgreSQL inheritance for blocks and mutes (#2520) 2017-04-30 00:27:31 +02:00
Matt Jankowski
f48cb3eb17 More coverage yes more even more (#2627)
* Add coverage for admin/confirmations controller

* Coverage for statuses controller show action

* Add coverage for admin/domain_blocks controller

* Add coverage for settings/profiles#update
2017-04-30 00:25:38 +02:00
Joël Quenneville
27301312a6 Add tests to the Feed model (#2594)
This adds a test for the `Feed#get` method. While the data
transformations in `Feed#get` may seem redundant, they are important to
maintain the order from Redis. The tests I wrote will fail if someone
tries to refactor away this "redundancy" (as I tried to do in the first
iteration of this change).
2017-04-29 00:21:35 +02:00
Patrick Figel
8ac7fca5d0 Set correct attachment type for rejected media (#2599)
In #2110, a new attachment type "unknown" was introduced for
attachments that were rejected due to a domain being blocked using
reject_media. However, the "type" field was never set to "unknown"
because a default value of "0" (image) is set for that column,
causing the `type.blank?` expression to always equal false.

This version uses type_changed? instead, causing the type to be set
to "unknown" unless a type has been explicitly set. This introduces
a small change in behaviour causing the type to be set to unknown
before paperclip calls `before_post_process`. Presumably this
behaviour is more appropriate than the current one because the
attachment type has not been determined by that point.

Included are new tests for `ProcessFeedService` and
`UpdateRemoteProfileService` which now check that remote media is
downloaded for non-blocked domains and is rejected for others.
2017-04-29 00:18:32 +02:00
Matt Jankowski
a823509b99 Fix broken view spec for about/links (#2591) 2017-04-28 17:54:03 +02:00
Matt Jankowski
9262f6968b Fix broken spec for about/links view partial (#2586) 2017-04-28 16:45:24 +02:00
Yamagishi Kazutoshi
71e73e36cd Fix spec for hide get started (#2585) 2017-04-28 16:44:57 +02:00
Matt Jankowski
9566893cc9 More controller specs (#2561)
* Add render_views in more places

* Delegate methods from account to user with allow nil true, so that admin accounts show view renders when missing a user

* Use actual account instances in authorize follow controller spec
2017-04-28 15:12:37 +02:00
Matt Jankowski
429480bb77 Return missing page when tag does not exist (#2563) 2017-04-28 15:11:21 +02:00
Matt Jankowski
effb08edbb More status specs (#2564)
* Add rough outline of coverage needed for public timeline

* Specs for visibility, replies, boosts

* Specs for silenced account

* Specs for local_only option

* Specs for blocks and mutes

* Add tentative spec around including other silenced account statuses

* Add with_public_visibility scope

* Add simple coverage for tag_timeline

* Tag timeline includes replies

* Replace tag.statuses with a tagged_with scope in tag timeline method

* Use with_public_visibility in tag timeline

* Extract common scope between public and tag timelines to method

* Extract local domain check to local_only scope

* Extract local_only check to starting scope method

* Move list of excluded from timeline account ids to account model

* Simplify excluded accounts list on account model

* Only join accounts when needed

* Rename method for account specific filtering

* Extract method for account exclusions

* Fix bug where silenced accounts were not including statuses from other silenced accounts

* DRY up filter application from account or no account

* timeline_scope can be private

* Add spec showing that account can find its excluded accounts ids

* Add spec which fails if local_only does not have a left outer join

* rubocop
2017-04-28 15:10:41 +02:00
Matt Jankowski
a39e719b39 Add render_views to admin/reports controller spec (#2548) 2017-04-28 01:21:38 +02:00
Matt Jankowski
9736753985 Fix remote follow (#2547)
* Add coverage for remote_follow#new

* Fix reference to authorize_follows/card partial
2017-04-28 00:16:44 +02:00
Eugen Rochko
2af4f3c4e2 Improve shared status verification (#2525)
* Instead of parsing shared status contents verbatim, make roundtrip
to purported original URL. Confirm that the "original" URL is from the
same domain as the author it claims to be from.

* Fix obvious typo, add comment

* Use URI look-up first

* Add test, update Goldfinger dependency to make less useless HTTP requests per Webfinger lookup
2017-04-27 17:06:47 +02:00
Matt Jankowski
b48f2cbc8b Catch error when server decryption fails on 2FA (#2512) 2017-04-27 15:18:21 +02:00
Eugen Rochko
88725d6ce8 OEmbed support for PreviewCard (#2337)
* OEmbed support for PreviewCard

* Improve ProviderDiscovery code failure treatment

* Do not crawl links if there is a content warning, since those
don't display a link card anyway

* Reset db schema

* Fresh migrate

* Fix rubocop style issues
Fix #1681 - return existing access token when applicable instead of creating new

* Fix test

* Extract http client to helper

* Improve oembed controller
2017-04-27 14:42:22 +02:00
Matt Jankowski
0618f09939 Add spec coverage and refactor authorize_follows controller (#2505) 2017-04-27 00:19:53 +02:00
Matt Jankowski
8857cabca4 Domain block service cleanup (#2490)
* Add coverage for domain block service with silence

* Get rid of warning about find_each and order

* Move domain_block to attr_reader

* Move optional clear_media into silence_accounts method

* Use blocked_domain method to reduce passed vars

* Extract blocked_domain_accounts method to find accounts on the domain

* Extract media_from_blocked_domain method to find relevant attachments

* Separate destruction of account images and account attachments
2017-04-26 20:09:01 +02:00
Matt Jankowski
affd75936e Add spec for Status.as_home_timeline (#2451) 2017-04-26 20:08:30 +02:00
Francis Chong
193dddb433 Add media dimensions (#2448)
* Fixes #1985

- add migration AddMediaAttachmentMeta, which add meta field to media_attachments
- before saving attachment, set file meta if needed
- add meta in api

* add spec

* align the “size” format for image and video

* fix code climate

* fixes media_attachment_spec.rb
2017-04-26 03:48:12 +02:00
Guillaume Lo Re
7177e37b99 Stricter whitelist rules (#2213)
* Stricter whitelist rules

* Linting

* Added spec for blacklisting

* Test subdomain blacklist on domain whitelist

* No need to split

* Change spec name
2017-04-26 01:22:51 +02:00
alpaca-tc
9317ec8eb1 Localize with i18n for Devise::FailureApp (#2309)
This PR fixes I18n.locale for rake middlewares. Mastodon uses Devise that depends on Warden.
Warden::Manager can be found in rake middleware. It is outside of the controller.

In the case of authentication failed, warden calls throw(:warden). At the time Warden::Manager
delegates request to failure_app to generate response and flash[:alert] after catching it.
Unfortunately, I18n.locale is already reset then because I18n.with_locale is enabled only
inside the controller. If we used I18n.locale=, Devise::FailureApp could get the current locale.
2017-04-25 15:06:41 +02:00
Evan Minto
122d59ac41 Change ActivityPub paging to match spec. Clean up ActivityPub outbox changes. (#2410)
* Change ActivityPub paging to match spec. Clean up ActivityPub outbox changes.

* Fix code style and test failures for OutboxController.

* Attempt to fix CI errors.
2017-04-25 15:06:06 +02:00
Eugen Rochko
8b5179d006 Fix #2402 - Add Idempotency-Key header to PostStatusService that prevents (#2419)
duplicates. Web UI regenerates UUID for that header every time the compose
form is changed or successfully submitted

Also, fix Farsi i18n overwriting the English one
2017-04-25 15:04:49 +02:00
Yamagishi Kazutoshi
3ea5b948a4 Decodes URL containing IDN (#2436) 2017-04-25 15:03:51 +02:00
178inaba
d2159deaf2 Optimize account search (#2421) 2017-04-25 04:44:43 +02:00
Eugen
17c591ffba Punycode URI normalization (#2370)
* Fix #2119 - Whenever about to send a HTTP request, normalize the URI

* Add test for IDN request in FetchLinkCardService

* Perform IDN normalization on domains before they are stored in the DB
2017-04-25 02:47:31 +02:00