CSV export fails

[markhahnfeld]

CSV export fails with the following stacktrace:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python2.7/dist-packages/flask_appbuilder/security/decorators.py", line 26, in wraps
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/caravel/models.py", line 1164, in wrapper
    return f(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/caravel/views.py", line 515, in explore
    payload = obj.get_csv()
  File "/usr/local/lib/python2.7/dist-packages/caravel/viz.py", line 282, in get_csv
    return df.to_csv(index=include_index)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/frame.py", line 1343, in to_csv
    formatter.save()
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/format.py", line 1550, in save
    self._save()
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/format.py", line 1651, in _save
    self._save_chunk(start_i, end_i)
  File "/usr/local/lib/python2.7/dist-packages/pandas/core/format.py", line 1677, in _save_chunk
    lib.write_csv_rows(self.data, ix, self.nlevels, self.cols, self.writer)
  File "pandas/lib.pyx", line 1078, in pandas.lib.write_csv_rows (pandas/lib.c:19848)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 1: ordinal not in range(128)

[andrewhn]

I think the fix should be a matter of adding encoding='utf-8' as an argument to DataFrame.to_csv here.

Would be nice to have a unicode datasource to test these issues with.

[jacopoch]

@markhahnfeld, @andrewhn alternatively you could do something like adding:

 import sys
 sys.setdefaultencoding("utf-8")

to caravel/viz.py, it should do the trick. But we'll need to make a PR to fix the problem.

[andrewhn]

My preference would be not to use setdefaultencoding for these reasons.

I plan to add a unicode data source for testing in the next couple of days.

[mistercrunch]

Yeah if we can sprinkle some unicode in the unit tests that would help.