Fix downcasing of unicode tag names

[leo-souza]

I ran into some issues when a tag name contains upcased unicode chars.
It happens in both creation and retrieving:

• It doesn't find the previous tag when a second tag is being created and duplicates it.
• When tagged_with is called, it returns an empty array, even when calling with the exact tag name.
  Examples:

Post.create(title: 'First', tag_list: 'Ruby')
Post.create(title: 'Second', tag_list: 'ruby')
Post.create(title: 'Third', tag_list: 'Ábaco')
Post.create(title: 'Fourth', tag_list: 'ábaco')
ActsAsTaggableOn::Tag.all
#<ActiveRecord::Relation [
  #<ActsAsTaggableOn::Tag id: 1, name: "Ruby">, 
  #<ActsAsTaggableOn::Tag id: 2, name: "Ábaco">, 
  #<ActsAsTaggableOn::Tag id: 3, name: "ábaco">]>

Post.tagged_with('ruby')
#<ActiveRecord::Relation [
  #<Post id: 1, title: "First", body: nil>,
  #<Post id: 2, title: "Second", body: nil>]>
Post.tagged_with('Ruby')
 #<ActiveRecord::Relation [
  #<Post id: 1, title: "First", body: nil>,
  #<Post id: 2, title: "Second", body: nil>]>
Post.tagged_with('ábaco')
#<ActiveRecord::Relation []>
Post.tagged_with('Ábaco')
#<ActiveRecord::Relation []>

This behaviour was introduced when .force_encoding('BINARY') was added in Tag class, which was being called before downcasing the string

When sqlite is being used, the only way around this is loading the ICU extension.

PR #472 only fixes the tagged_with part of this issue.

[seuros]

Tests are failing.

[leo-souza]

My mistake. do I need to repush this commits as one?

[seuros]

Since you will need to push the changelog , rebase and push as one.

[leo-souza]

This is related to #464 as well

[seuros]

Would you mind modifying the change-log as well ?
And add the note about ICU extension in the readme.

[seuros]

Thank you

[seuros]

Sqlite3 tests are failing. :/

[leo-souza]

I'll check this

[leo-souza]

How can I reproduce those testing errors in my local code?

[seuros]

I could not. All tests passed on my machine.
You need use ruby 1.9.3.
cc @bf4 @mbleigh

[leo-souza]

Maybe rolling back #as_8bit_ascii to what it was and relying on database's lower function for queries is another valid approach.
Like this a75a061

[seuros]

Do you want to send a PR ?

[nicolaslazartekaqui]

@leo-souza is returning a string, should not be returning a ActiveSupport::Multibyte::Chars?
Maybe this broke the build.

Suggestion:

def as_8bit_ascii(string, downcase=false)
  string = string.to_s
  string.downcase! if downcase
  if defined?(Encoding)
    string.dup.force_encoding('BINARY')
  else
    string.mb_chars
  end
end

[leo-souza]

#mb_chars have to be called before #downcase or else Á would not become á. But either way, I don't think the else block is being called at all. But this is a good suggestion, maybe just removing #to_s is a good try. If only the tests fail locally

[bf4]

The mb_chars code was legacy 1.8 that I left in there. That's why it's in the elsif Encoding is not defined.

[leo-souza]

Ruby downcase method doesn't apply to multibyte chars unless you call mb_chars before downcase, even for 1.9+

[leo-souza]

Yes, I do.
This build https://travis-ci.org/mbleigh/acts-as-taggable-on/builds/21049248 throw me off of understanding what's going on, a test on mysql broke, and in this other build https://travis-ci.org/mbleigh/acts-as-taggable-on/builds/20835215 a postgres test broke, so not only the sqlite ones are failing.
Should i modify changelog again?

[seuros]

I noticed that. If you restart the tests , they often pass.

[bf4]

Why use mb_chars here? Unnecessary in Ruby 1.9