Commit Graph

177 Commits

Author SHA1 Message Date
Akihiko Odaki
bb4d005a83 Introduce OStatus::TagManager (#5008) 2017-09-19 18:08:08 +02:00
Eugen Rochko
81cec35dbf Custom emoji (#4988)
* Custom emoji

- In OStatus: `<link rel="emoji" name="coolcat" href="http://..." />`
- In ActivityPub: `{ type: "Emoji", name: ":coolcat:", href: "http://..." }`
- In REST API: Status object includes `emojis` array (`shortcode`, `url`)
- Domain blocks with reject media stop emojis
- Emoji file up to 50KB
- Web UI handles custom emojis
- Static pages render custom emojis as `<img />` tags

Side effects:

- Undo #4500 optimization, as I needed to modify it to restore
  shortcode handling in emojify()
- Formatter#plaintext should now make sure stripped out line-breaks
  and paragraphs are replaced with newlines

* Fix emoji at the start not being converted
2017-09-19 02:42:40 +02:00
unarist
ec36df97c4 Escape URL parts on formatting local status (#4975) 2017-09-16 21:33:52 +02:00
Akihiko Odaki
48d77ea1eb Fix filterable_languages method of SettingsHelper (#4966) 2017-09-16 14:59:41 +02:00
ふぁぼ原
3816943e6b Enable to recognize most kinds of characters as URL paths (#4941) 2017-09-14 18:03:20 +02:00
ThibG
550ff677da Fix ActivityPub handling of replies with WEB_DOMAIN (#4895) (#4904)
* Fix ActivityPub handling of replies when LOCAL_DOMAIN ≠ WEB_DOMAIN (#4895)

For all intents and purposes, `local_url?` is used to check if an URL refers
to the Web UI or the various API endpoints of the local instances. Those things
reside on `WEB_DOMAIN` and not `LOCAL_DOMAIN`.

* Change local_url? spec, as all URLs handled by Mastodon are based on WEB_DOMAIN
2017-09-13 14:22:16 +02:00
unarist
a12572e074 Handle stream_entry URL correctly in ActivityPub (#4854)
In before, the method uses stream_entry id as status id, so replied status was wrongly selected.

This PR uses StatusFinder which was introduced with `Api::Web::EmbedsController`.
2017-09-08 18:20:03 +02:00
Eugen Rochko
a4caa7eb62 Fetch statuses/following/followers numbers from ActivityPub collections (#4840) 2017-09-08 12:00:17 +02:00
Eugen Rochko
e7adbf572a Switch to static URIs, new URI format in both protocols for new statuses (#4815)
* Decouple Status#local? from uri being nil

* Replace on-the-fly URI generation with stored URIs

- Generate URI in after_save hook for local statuses
- Use static value in TagManager when available, fallback to tag format
- Make TagManager use ActivityPub::TagManager to understand new format
- Adjust tests

* Use other heuristic for locality of old statuses, do not perform long query

* Exclude tombstone stream entries from Atom feed

* Prevent nil statuses from landing in Pubsubhubbub::DistributionWorker

* Fix URI not being saved (#4818)

* Add more specs for Status

* Save generated uri immediately

and also fix method order to minimize diff.

* Fix alternate HTML URL in Atom

* Fix tests

* Remove not-null constraint from statuses migration to speed it up
2017-09-06 19:01:28 +02:00
unarist
6a4e2db661 Raise an error for remote url in StatusFinder (#4776)
* Raise an error for remote url in StatusFinder

Previous implementation had allowed remote url with status id which also exists on local.

Then that bug leads /api/web/embed to return wrong embed url.

* Fix oembed_controller_spec
2017-09-02 17:42:47 +02:00
Eugen Rochko
9a5ae09620 Remove identity context from output of LinkedDataSignature (#4753) 2017-08-31 21:32:09 +02:00
Eugen Rochko
7b8f262840 Forward ActivityPub creates that reply to local statuses (#4709)
* Forward ActivityPub creates that reply to local statuses

* Fix test

* Fix wrong signers
2017-08-30 15:37:02 +02:00
Eugen Rochko
e95bdec7c5 Update status embeds (#4742)
- Use statuses controller for embeds instead of stream entries controller
- Prefer /@:username/:id/embed URL for embeds
- Use /@:username as author_url in OEmbed
- Add follow link to embeds which opens web intent in new window
- Use redis cache in development
- Cache entire embed
2017-08-30 10:23:43 +02:00
unarist
938cd2875b Fix Delete activity handling when the status has been reblogged (#4729) 2017-08-28 22:08:11 +02:00
Eugen Rochko
00840f4f2e Add handling of Linked Data Signatures in payloads (#4687)
* Add handling of Linked Data Signatures in payloads

* Add a way to sign JSON, fix canonicalization of signature options

* Fix signatureValue encoding, send out signed JSON when distributing

* Add missing security context
2017-08-26 13:47:38 +02:00
Eugen Rochko
40c45f5dd9 Put ActivityPub alternate link into Atom, prefer it when processing Atom (#4623) 2017-08-18 01:03:18 +02:00
unarist
a855956185 Fix ActivityPub follow interaction and add more specs (#4601) 2017-08-14 16:57:46 +02:00
Eugen Rochko
dd7ef0dc41 Add ActivityPub inbox (#4216)
* Add ActivityPub inbox

* Handle ActivityPub deletes

* Handle ActivityPub creates

* Handle ActivityPub announces

* Stubs for handling all activities that need to be handled

* Add ActivityPub actor resolving

* Handle conversation URI passing in ActivityPub

* Handle content language in ActivityPub

* Send accept header when fetching actor, handle JSON parse errors

* Test for ActivityPub::FetchRemoteAccountService

* Handle public key and icon/image when embedded/as array/as resolvable URI

* Implement ActivityPub::FetchRemoteStatusService

* Add stubs for more interactions

* Undo activities implemented

* Handle out of order activities

* Hook up ActivityPub to ResolveRemoteAccountService, handle
Update Account activities

* Add fragment IDs to all transient activity serializers

* Add tests and fixes

* Add stubs for missing tests

* Add more tests

* Add more tests
2017-08-08 21:52:15 +02:00
Eugen Rochko
a390abdefb Use the same emoji data on the frontend and backend (#4284)
* Use the same emoji data on the frontend and backend

* Move emoji.json to repository, add tests

This way you don't need to install node dependencies if you only
want to run Ruby code
2017-07-21 04:27:40 +02:00
Eugen Rochko
a5a07da892 Correct OStatus inflection (Ostatus -> OStatus) (#4255) 2017-07-19 01:37:26 +02:00
Yamagishi Kazutoshi
3267e4a785 Add unfollow modal (optional) (#4246)
* Add unfollow modal

* unfollowing someone

* remove unnecessary prop
2017-07-18 17:14:43 +02:00
Akihiko Odaki
89b988cab5 Introduce Ostatus name space (#4164)
* Wrap methods of ProcessFeedService::ProcessEntry in classes

This is a change same with 425acecfdb, except
that it has the following changes:

* Revert irrelevant change in find_or_create_conversation
* Fix error handling for RemoteActivity

* Introduce Ostatus name space
2017-07-18 16:39:47 +02:00
Eugen Rochko
de397f3bc1 Fix subsequent replies to unresolved status not being filtered from home (#4190)
Resolves #4177 - smaller changeset
2017-07-14 22:31:38 +02:00
Eugen Rochko
1618b68bfa HTTP signatures (#4146)
* Add Request class with HTTP signature generator

Spec: https://tools.ietf.org/html/draft-cavage-http-signatures-06

* Add HTTP signature verification concern

* Add test for SignatureVerification concern

* Add basic test for Request class

* Make PuSH subscribe/unsubscribe requests use new Request class

Accidentally fix lease_seconds not being set and sent properly, and
change the new minimum subscription duration to 1 day

* Make all PuSH workers use new Request class

* Make Salmon sender use new Request class

* Make FetchLinkService use new Request class

* Make FetchAtomService use the new Request class

* Make Remotable use the new Request class

* Make ResolveRemoteAccountService use the new Request class

* Add more tests

* Allow +-30 seconds window for signed request to remain valid

* Disable time window validation for signed requests, restore 7 days
as PuSH subscription duration (which was previous default due to a bug)
2017-07-14 20:41:49 +02:00
Yamagishi Kazutoshi
2b9721d1b3 Add setting a always mark media as sensitive (#4136) 2017-07-10 14:00:32 +02:00
Eugen Rochko
8b2cad5637 Refactor JSON templates to be generated with ActiveModelSerializers instead of Rabl (#4090) 2017-07-07 04:02:06 +02:00
Damien Erambert
18d3fa953b Add a setting allowing the use of system's default font in Web UI (#4033)
* add a system_font_ui setting on the server

* Plug the system_font_ui on the front-end

* add EN/FR locales for the new setting

* put Roboto after all other fonts

* remove trailing whitespace so CodeClimate is happy

* fix user_spec.rb

* correctly write user_spect this time

* slightly better way of adding the classes

* add comments to the system-font stack for clarification

* use .system-font for the class instead

* don't use multiple lines for comments

* remove trailing whitespace

* use the classnames module for consistency

* use `mastodon-font-sans-serif` instead of Roboto directly
2017-07-06 22:39:56 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
60b2b56d38 Reduce number of commands in FeedManager#trim (#3989) 2017-06-29 01:17:26 +02:00
nightpool
94d0e012de Whitelist allowed classes for federated statuses (#3810)
* Whitelist allowed classes for federated statuses

Allowed classes are currently:

 - Any microformats class (h/p/u/dt/e-*)
 - the classes mention, hashtag, ellipses and invisible.

this last one is somewhat suspect, but Mastodon currently uses it to render hidden link text.

resolved #3790

* Fix code style
2017-06-17 20:26:05 +02:00
Eugen Rochko
8518d005fd Fix regression from #3490 - filter out hidden statuses from ancestors/descendants even if the viewer is anonymous (#3752) 2017-06-15 02:00:08 +02:00
Matt Jankowski
022008a2a6 Language detection defaults to nil (#3666)
* Default to nil for statuses.language

* Language detection defaults to nil instead of instance UI default
2017-06-09 18:09:37 +02:00
Matt Jankowski
6201f96b8a Introduce StatusThreadingConcern (#3490)
* Add a StatusFilter class to identify visibility of statuses by accounts

* Extract StatusThreadingConcern from Status

* Clarify purpose of checking for nil account
2017-06-05 16:07:44 +02:00
Yamagishi Kazutoshi
6a4b224397 Fix randomly fail (regression from #3560) (#3580) 2017-06-05 10:03:36 +02:00
Yamagishi Kazutoshi
6adbd114c1 Skip image length check (regression from #3528) (#3581) 2017-06-05 10:03:01 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
defe4f9bc3 Cover WebfingerResource more (#3560) 2017-06-04 17:13:37 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
e07b57852e Remove some code in TagManager and spec (#3547)
* Do not fall back to StreamEntry if object_type is unavailable in TagManager

Since 6d6a429af8, when Status, the only model
with stream_entry, and StreamEntry got its own logic in uri_for and
url_for, the purpose of the fallbacks to activity_type of StreamEntry
became unclear.

This commit removes the fallbacks. When adding another model with
stream_entry in future, consider to update uri_for and url_for.

* Cover TagManager more
2017-06-04 16:14:25 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
02a0fd5b64 Spec InlineRablScope (#3542) 2017-06-04 15:00:08 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
6505a42be0 Spec HashObject (#3544) 2017-06-04 14:59:52 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
e674608d10 A minor change for ProviderDiscovery and spec (#3543)
* Do not default the format in ProviderDiscovery

The format should be determined when discovering, as it is in the current
implementation, and it is a flaw if it is not determined.

* Spec ProviderDiscovery
2017-06-04 14:59:40 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
c7af8cbc90 Remove some arguments of Formatter.instance.format and spec (#3541)
* Remove some arguments of Formatter.instance.format

* Improve spec for Formatter
2017-06-04 14:58:57 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
9475fbae78 Spec Extractor (#3540) 2017-06-04 14:57:02 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
00e61d6807 Spec action verification of StreamEntryFinder (#3549) 2017-06-04 14:56:31 +02:00
Akihiko Odaki (@fn_aki@pawoo.net)
ce7c0def88 Remove redundant code in AtomSerializer and spec (#3528)
* Remove redundant status.reply? in AtomSerializer#object

* Cover AtomSerializer more
2017-06-03 20:51:08 +02:00
Matt Jankowski
d010e270e6 Remove usernames and hashtags from language detection (#3503)
* Add failing specs for hashtag and username extraction in language detector

* Remove usernames and hashtags from text before language detection

* Handle multiple instances of special case, and reduce whitespace
2017-06-01 09:29:14 -04:00
Matt Jankowski
3576fa0d59 Improve api oembed controller (#3450)
* Add StreamEntryFinder class to parse URLs

* Use StreamEntryFinder and clean up api/oembed controller
2017-05-30 16:30:06 -04:00
Atsushi Yamamoto
402c19a924 Add preference setting for delete toot modal (#3368)
* Set delete_modal preference to true by default
* Does not show confirmation modal if delete_modal is false
* Add ja translation for preference setting page
2017-05-29 17:56:13 +02:00
abcang
bbc3db8b20 Add test when hashtag and URL are concatenated (#3279) 2017-05-24 15:36:10 +02:00
Yamagishi Kazutoshi
f937cad68f Skip formatting for cashtag in status text (#3275)
Resolve #3270
2017-05-24 14:32:53 +02:00
Eugen Rochko
620d0d8029 Account domain blocks (#2381)
* Add <ostatus:conversation /> tag to Atom input/output

Only uses ref attribute (not href) because href would be
the alternate link that's always included also.

Creates new conversation for every non-reply status. Carries
over conversation for every reply. Keeps remote URIs verbatim,
generates local URIs on the fly like the rest of them.

* Conversation muting - prevents notifications that reference a conversation
(including replies, favourites, reblogs) from being created. API endpoints
/api/v1/statuses/:id/mute and /api/v1/statuses/:id/unmute

Currently no way to tell when a status/conversation is muted, so the web UI
only has a "disable notifications" button, doesn't work as a toggle

* Display "Dismiss notifications" on all statuses in notifications column, not just own

* Add "muted" as a boolean attribute on statuses JSON

For now always false on contained reblogs, since it's only relevant for
statuses returned from the notifications endpoint, which are not nested

Remove "Disable notifications" from detailed status view, since it's
only relevant in the notifications column

* Up max class length

* Remove pending test for conversation mute

* Add tests, clean up

* Rename to "mute conversation" and "unmute conversation"

* Raise validation error when trying to mute/unmute status without conversation

* Adding account domain blocks that filter notifications and public timelines

* Add tests for domain blocks in notifications, public timelines
Filter reblogs of blocked domains from home

* Add API for listing and creating account domain blocks

* API for creating/deleting domain blocks, tests for Status#ancestors
and Status#descendants, filter domain blocks from them

* Filter domains in streaming API

* Update account_domain_block_spec.rb
2017-05-19 01:14:30 +02:00
Eugen Rochko
b5a9c6b3d2 Prepend reblogs' wrapper content with "RT @original_author", (#3013)
so that when a reblog parse fails on another instance, it doesn't
look like a misattributed/stolen text
2017-05-12 17:46:44 +02:00
Eugen Rochko
72698bc3b4 Fix regressions from #2683 (#2970)
* Fix regressions from #2683

Properly format spoiler text HTML, while keeping old logic for blankness intact
Process hashtags and mentions in spoiler text
Format spoiler text for Atom
Change "show more" toggle into a button instead of anchor
Fix style regression on dropdowns for detailed statuses

* Fix lint issue

* Convert spoiler text to plaintext in desktop notifications
2017-05-11 00:28:10 +02:00
Akihiko Odaki
d5cabfe5c6 Use CLD3 (#2949)
Compact Language Detector v3 (CLD3) is the successor of CLD2, which was
used in the previous implementation. CLD3 includes improvements since CLD2,
and supports newer compilers. On the other hand, it has additional
requirements and cld3-ruby, the FFI of CLD3 for Ruby, is still new and may
be still inmature.

Though CLD3 is named after CLD2, it is implemented with a neural network
model, different from the old implementation, which is based on a Naïve
Bayesian classifier.

CLD3 supports newer compilers, such as GCC 6. CLD2 is not compatible with
GCC 6 because it assigns negative values to varibales typed unsigned.
(see internal/cld_generated_cjk_uni_prop_80.cc) The support for GCC 6 and
newer compilers are essential today, when some server operating system
such as Ubuntu Server 16.10 has GCC 6 by default.

On the one hand, CLD3 requires C++11 support. Environments with old
compilers such as Ubuntu Server 14.04 needs to update the system or install
a newer compiler.

CLD3 needs protocol buffers as a new dependency. However,it is not
considered problematic because major server operating systems, CentOS and
Ubuntu Server provide them.

The FFI cld3-ruby was written by me (Akihiko Odaki) for use in Mastodon.
It is still new and may be inmature, but confirmed to pass existing tests.
2017-05-09 19:58:03 +02:00
abcang
08e94d1b19 Fixed bug that timeline can not be displayed by InvalidURIError (#2947) 2017-05-09 18:17:41 +02:00
Matt Jankowski
7a7d12d27f Delegate Account#user_locale method and allow nil (#2927) 2017-05-08 23:10:50 +02:00
masarakki
d08f1112d5 use Twitter::Extractor for creating links (#2502) 2017-05-05 19:48:22 +02:00
Matt Jankowski
8c5ad23b24 Language improvements, replace whatlanguage with CLD (#2753)
* add failing en specs

* add cld2 gem

* Replace WhatLanguage with CLD
2017-05-03 16:59:31 +02:00
Akihiko Odaki
a6788662b0 Revert "Use PostgreSQL inheritance for blocks and mutes (#2520)" (#2634)
This reverts commit 5135d609b7.
2017-04-30 08:49:24 -04:00
Akihiko Odaki
5135d609b7 Use PostgreSQL inheritance for blocks and mutes (#2520) 2017-04-30 00:27:31 +02:00
Yamagishi Kazutoshi
3ea5b948a4 Decodes URL containing IDN (#2436) 2017-04-25 15:03:51 +02:00
Eugen
338df98ddf Add more FeedManager#filter? tests (#2413) 2017-04-24 22:37:24 +02:00
Matt Jankowski
629d35e6f5 [WIP] Html lang on statuses (#2297)
* Add html lang attributes around statuses

* Remove urls from language detection
2017-04-22 04:26:25 +02:00
Matt Jankowski
2dda356e3f Clean up settings/preferences controller (#2237)
* Add missing fields group on preferences page

* Clean up settings/preferences controller

* Extract a UserSettingsDecorator
2017-04-21 03:26:52 +02:00
abcang
70891a99a9 Fix html escape characters in the URL (#2138)
* fix character escaping in URL

* add tests

* put a comma after the last item

* add HTML escape test
2017-04-19 14:52:18 +02:00
Eugen
96268e76e6 Fix wording on admin instances view, fix typos in language detector spec (#2106) 2017-04-18 23:39:51 +02:00
Eugen
6bd1019235 Add more tests for AtomSerializer (#2096) 2017-04-18 22:30:17 +02:00
Matt Jankowski
297c11dba2 Language detection refactor (#2099)
* Extract detect_language to separate class

* Use default locale, not just en

* Add spec to confirm that whatlanguage cant identify empty string

* Allow account locale to override default in language detector

* PostStatusService supplies an account to detect language
2017-04-18 22:20:12 +02:00
Eugen
f03fb6c0ae Add forgotten AtomSerializer spec (#2047) 2017-04-18 01:07:46 +02:00
Eugen
5dc73339ae Fix tests issue introduced in #1607 (#1639) 2017-04-13 02:17:40 +02:00
Matt Jankowski
aa90798386 Webfinger resource to extract username from resource string (#1607)
* Add WebfingerResource class to extract usernames

* Use WebfingerResource in xrd#webfinger
2017-04-12 18:22:38 +02:00
Eugen Rochko
3202bdd744 Fix #555 - Use a better URL parser 2017-02-22 19:35:11 +01:00
Eugen Rochko
0518492158 Stop trying to shoehorn all Salmon updates into the poor database-connected
StreamEntry model. Simply render Salmon slaps as they are needed
2017-02-12 01:19:14 +01:00
Eugen Rochko
920ba5fc4e Fix #61 - Add list of blocked users to the UI; clean up failed push notifications API
Try to fix Travis CI setup
2017-02-05 19:18:11 +01:00
Eugen Rochko
80cefd5b3c Fix #204, fix #515 - URL truncating is now a style so copypasting is not
affected, replaced onClick handler with onMouseUp/Down to detect text
selection not trigger onClick handler then
2017-01-24 17:05:44 +01:00
Eugen Rochko
76ec907993 Improved admin UI 2016-12-13 13:42:10 +01:00
Eugen Rochko
33f7e1cf99 Shorten rendered links (strip protocol and www, truncate to 30 chars), redirect
to sign in page after sign up instead of root path which redirects to /about
2016-10-13 16:51:34 +02:00
Eugen Rochko
be86d4e0a3 Preparing feeds for better filtering 2016-10-02 15:28:47 +02:00
Eugen Rochko
3cc47beb6e Refactored generation of unique tags, URIs and object URLs into own classes,
as well as formatting of content
2016-09-09 20:04:34 +02:00