Skip to content

Fix encoding issues #17

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Fix encoding issues #17

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ee47367
Do not force content string encodings, keep them as they are
Oct 2, 2014
8ed777b
Bump version
Oct 2, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions lib/simple-rss.rb
View file
Original file line number Diff line number Diff line change
@@ -151,13 +151,13 @@ def clean_tag(tag)
tag.to_s.gsub(':','_').intern
end

def unescape(content)
if content.respond_to?(:force_encoding) && content.force_encoding("binary") =~ /([^-_.!~*'()a-zA-Z\d;\/?:@&=+$,\[\]]%)/n then
CGI.unescape(content).gsub(/(<!\[CDATA\[|\]\]>)/,'').strip
else
content.gsub(/(<!\[CDATA\[|\]\]>)/,'').strip
end
end
def unescape(content)
if content =~ /([^-_.!~*'()a-zA-Z\d;\/?:@&=+$,\[\]]%)/ then
CGI.unescape(content)
else
content
end.gsub(/(<!\[CDATA\[|\]\]>)/,'').strip
end
end

class SimpleRSSError < StandardError
4 changes: 2 additions & 2 deletions simple-rss.gemspec
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
Gem::Specification.new do |s|
s.name = "simple-rss"
s.version = "1.2.3"
s.version = "1.3.2"
s.version = "#{s.version}-alpha-#{ENV['TRAVIS_BUILD_NUMBER']}" if ENV['TRAVIS']
s.date = "2010-07-06"
s.date = "2014-10-03"
s.summary = "A simple, flexible, extensible, and liberal RSS and Atom reader for Ruby. It is designed to be backwards compatible with the standard RSS parser, but will never do RSS generation."
s.email = "lucas@rufy.com"
s.homepage = "http://github.com/cardmagic/simple-rss"
11 changes: 11 additions & 0 deletions test/base/base_test.rb
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# -*- coding: utf-8 -*-
require 'test_helper'
class BaseTest < Test::Unit::TestCase
def setup
@rss09 = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/rss09.rdf')
@rss20 = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/rss20.xml')
@rss20_utf8 = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/rss20_utf8.xml')
@media_rss = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/media_rss.xml')
@atom = SimpleRSS.parse open(File.dirname(__FILE__) + '/../data/atom.xml')
end
@@ -69,4 +71,13 @@ def test_atom
def test_bad_feed
assert_raise(SimpleRSSError) { SimpleRSS.parse(open(File.dirname(__FILE__) + '/../data/not-rss.xml')) }
end

def test_rss_utf8
assert_equal 2, @rss20_utf8.items.size
assert_equal "SC5 Blog", @rss20_utf8.title
assert_equal Encoding::UTF_8, @rss20_utf8.title.encoding
item = @rss20_utf8.items.first
assert_equal "Mitä asiakkaamme ajattelevat meistä?", item.title
assert_equal Encoding::UTF_8, item.title.encoding
end
end
61 changes: 61 additions & 0 deletions test/data/rss20_utf8.xml
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
xmlns:content="http://purl.org/rss/1.0/modules/content/"
xmlns:wfw="http://wellformedweb.org/CommentAPI/"
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:atom="http://www.w3.org/2005/Atom"
xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
>

<channel>
<title>SC5 Blog</title>
<atom:link href="http://sc5.io/blog/feed/" rel="self" type="application/rss+xml" />
<link>http://sc5.io/blog</link>
<description>HTML5 expertise</description>
<lastBuildDate>Wed, 01 Oct 2014 09:52:48 +0000</lastBuildDate>
<language>fi-FI</language>
<sy:updatePeriod>hourly</sy:updatePeriod>
<sy:updateFrequency>1</sy:updateFrequency>
<generator>http://wordpress.org/?v=4.0</generator>
<item>
<title>Mitä asiakkaamme ajattelevat meistä?</title>
<link>http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/</link>
<comments>http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/#comments</comments>
<pubDate>Mon, 29 Sep 2014 13:19:37 +0000</pubDate>
<dc:creator><![CDATA[Saskia Salomaa]]></dc:creator>
<category><![CDATA[HTML5]]></category>
<category><![CDATA[Organization]]></category>

<guid isPermaLink="false">http://sc5.io/blog/?p=2818</guid>
<description><![CDATA[<p>Toteutimme yhdessä MicroMedian kanssa asiakastyytyväisyystutkimuksen, jossa tiedustelimme asiakkaidemme näkemyksiä toimintatavoistamme. Voimme ylpeänä kertoa, että asiakkaamme tykkäsivät työmme jäljestä. Mutta mihin asiakamme olivat meissä erityisen tyytyväisiä? Vastauksissa korostuivat työntekijöidemme osaaminen ja ammattitaito. Lisäksi kehuttiin kykyämme nähdä... <a class="read-more" href="http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/">Read More</a></p>
<p>The post <a rel="nofollow" href="http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/">Mitä asiakkaamme ajattelevat meistä?</a> appeared first on <a rel="nofollow" href="http://sc5.io/blog">SC5 Blog</a>.</p>
]]></description>
<wfw:commentRss>http://sc5.io/blog/2014/09/mita-asiakkaamme-ajattelevat-meista/feed/</wfw:commentRss>
<slash:comments>0</slash:comments>
</item>
<item>
<title>Tribute to Jyväskylä, City of Lights</title>
<link>http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/</link>
<comments>http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/#comments</comments>
<pubDate>Thu, 25 Sep 2014 10:36:20 +0000</pubDate>
<dc:creator><![CDATA[Arto Liukkonen]]></dc:creator>
<category><![CDATA[JavaScript]]></category>
<category><![CDATA[Jyväskylä]]></category>
<category><![CDATA[Mobile]]></category>
<category><![CDATA[Video]]></category>
<category><![CDATA[Web apps]]></category>
<category><![CDATA[city of lights]]></category>
<category><![CDATA[internet of things]]></category>
<category><![CDATA[iot]]></category>
<category><![CDATA[raspberry pi]]></category>

<guid isPermaLink="false">http://sc5.io/blog/?p=2886</guid>
<description><![CDATA[<p>In Finland the winters are dark so artificial, energy-efficient lights are needed. Good lighting not only boosts enjoyment and the quality of life, but can be a form of art itself. Jyväskylä is a forerunner... <a class="read-more" href="http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/">Read More</a></p>
<p>The post <a rel="nofollow" href="http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/">Tribute to Jyväskylä, City of Lights</a> appeared first on <a rel="nofollow" href="http://sc5.io/blog">SC5 Blog</a>.</p>
]]></description>
<wfw:commentRss>http://sc5.io/blog/2014/09/tribute-to-jyvaskyla-city-of-lights/feed/</wfw:commentRss>
<slash:comments>0</slash:comments>
</item>
</channel>
</rss>