UnicodeEncodeError, urllib.urlencode doesn't like unicode values

[mongkok]

pyminfraud/init.py", line 180, in _request_arguments
return compatibility.urlencode(arguments)

UnicodeEncodeError: 'ascii' codec can't encode character...

[radzhome]

I also get this problem

  File "/fanx-apis/tests/fanx-apis/../../lib/utils/minfraud_api.py", line 50, in get_minfraud_score
    result = fraud.execute()
  File "/usr/lib/python2.7/site-packages/pyminfraud/__init__.py", line 103, in execute
    self.response = compatibility.urlopen(self._request_url(protocol, host, uri), self._request_arguments(), timeout)
  File "/usr/lib/python2.7/site-packages/pyminfraud/__init__.py", line 180, in _request_arguments
    return compatibility.urlencode(arguments)
  File "/usr/lib64/python2.7/urllib.py", line 1326, in urlencode
    v = quote_plus(str(v))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0308' in position 7: ordinal not in range(128)

The problem is that pyminfraud is trying to urlencode the params, send the data as url parameters and that must be in ascii.

The minFraud API accepts input in a query string or as a form post (application/x-www-form-urlencoded).

See here: http://dev.maxmind.com/minfraud/#Input

Converting the data to ascii works for me, but the actual AVS info will now be changed.. not sure how that affects score. Not sure why minfraud only accepts urlencoded data:

import unicodedata

def convert_text_to_encoding(value, encoding):
    """Normalize text to encoding specified using unicodedata."""
    if value and type(value) in (str, unicode):
        if type(value) == str:
            value = unicode(value, 'utf-8')  # wrap in unicode utf-8 first
        # Replace special characters here as required
        value = value.replace(u'\xdf', 'ss')
        value = value.replace(u'\xe6', 'ae')
        value = value.replace(u'\u0153', 'ce')
        value = value.replace(u'\u0142', 'l')
        return unicode(unicodedata.normalize('NFKD', value).encode(encoding, 'ignore'), 'utf-8')
    else:
        return value

I'm using this with python 2.7.x

[oschwald]

FWIW, minFraud Premium and Standard expect the input to be ISO 8859-1.