autonomic-cooperative/hometown
0
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Use charlock_holmes instead of nkf at FetchLinkCardService (#4080)

Browse Source 
* Specs for language detection

* Use CharlockHolmes instead of NKF

* Correct mistakes

* Correct style

* Set hint_enc instead of falling back and strip_tags

* Improve specs

* Add dependencies
This commit is contained in:
nullkal
2017-07-09 05:44:31 +09:00
committed by Eugen Rochko
parent 794781d121
commit 007ab330e6
11 changed files with 78 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
spec/fixtures/requests/koi8-r.txt vendored Normal file
View File

@ -0,0 +1,20 @@
HTTP/1.1 200 OK
Server: nginx/1.11.10
Date: Tue, 04 Jul 2017 16:43:39 GMT
Content-Type: text/html
Content-Length: 273
Connection: keep-alive
Last-Modified: Tue, 04 Jul 2017 16:41:34 GMT
Accept-Ranges: bytes
<HTML>
<HEAD>
<META NAME="GENERATOR" CONTENT="Adobe PageMill 3.0J Mac">
<META HTTP-EQUIV="Content-Type" CONTENT="text/html;CHARSET=koi8-r">
<TITLE><3E><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><> XVI <20><>. <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>.</TITLE>
</HEAD>
<BODY>
<P><CENTER><B><FONT SIZE="+2"><3E><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><> XVI <20><>. <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD> <20><><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD><EFBFBD>.</FONT></B><BR>
<HR><BR>
</BODY>
</HTML>