autonomic-cooperative/hometown
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Improve error handling on LinkCrawlWorker (#5250)

Browse Source 
* Improve error handling on LinkCrawlWorker

* Ignore TimeoutError and InvalidURIError too
* Record errors to debug log
* Enable dead job queue on LinkCrawlWorker

Since most of acceptable errors were already ignored, only our side issue should go to dead job queue.

* Ignore all http gem errors
This commit is contained in:
unarist 2017-10-07 03:39:08 +09:00 committed by Eugen Rochko
parent d8b2f89d33
commit 5c8ca024ef
2 changed files with 3 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
app/services/fetch_link_card_service.rb
View File

@ -27,7 +27,8 @@ class FetchLinkCardService < BaseService
end end
attach_card if @card&.persisted? attach_card if @card&.persisted?
rescue HTTP::ConnectionError, OpenSSL::SSL::SSLError rescue HTTP::Error, Addressable::URI::InvalidURIError => e
Rails.logger.debug "Error fetching link #{@url}: #{e}"
nil nil
end end

2
app/workers/link_crawl_worker.rb
View File

@ -3,7 +3,7 @@
class LinkCrawlWorker class LinkCrawlWorker
include Sidekiq::Worker include Sidekiq::Worker
sidekiq_options queue: 'pull', retry: false sidekiq_options queue: 'pull', retry: 0
def perform(status_id) def perform(status_id)
FetchLinkCardService.new.call(Status.find(status_id)) FetchLinkCardService.new.call(Status.find(status_id))