Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UnicodeEncodeError in Py2 in slicemodelview/list and dashboardmodelview/list, when username carries unicode characters #2554

Closed
2 tasks done
rumbin opened this issue Apr 5, 2017 · 3 comments
Closed
2 tasks done

UnicodeEncodeError in Py2 in slicemodelview/list and dashboardmodelview/list, when username carries unicode characters #2554

rumbin opened this issue Apr 5, 2017 · 3 comments
Labels
inactive Inactive for >= 30 days

Comments

@rumbin
Copy link
Contributor

rumbin commented Apr 5, 2017

Make sure these boxes are checked before submitting your issue - thank you!

  • I have checked the superset logs for python stacktraces and included it here as text if any
  • I have reproduced the issue with at least the latest released version of superset
  • [yes and no] I have checked the issue tracker for the same issue and I haven't found one similar

I found similar issues (#2282,#1935,#543), but they don't seem to match entirely, as in this case only the username seems to be the troublemaker.

Superset version

0.17.3

Expected results

Rendering slicemodelview/list or dashboardmodelview/list succeeds, regardless of any unicode characters present in a users first or last name (as displayed in Creator column).

Actual results

Loading these pages results in 500 - Internal Server Error with the following stacktrace:

/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/forms.py:245: FlaskWTFDeprecationWarning: "flask_wtf.Form" has been renamed to "FlaskForm" and will be removed in 1.0.
  form = self(obj=obj)
[2017-04-05 09:51:53,173] ERROR in app: Exception on /dashboardmodelview/list/ [GET]
Traceback (most recent call last):
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/security/decorators.py", line 26, in wraps
    return f(self, *args, **kwargs)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/views.py", line 453, in list
    widgets=widgets)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/baseviews.py", line 158, in render_template
    return render_template(template, **dict(list(kwargs.items()) + list(self.extra_args.items())))
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
    context, ctx.app)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html", line 2, in top-level template code
    {% import 'appbuilder/general/lib.html' as lib %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/base.html", line 1, in top-level template code
    {% extends base_template %}
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/templates/superset/base.html", line 1, in top-level template code
    {% extends "appbuilder/baselayout.html" %}
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/templates/appbuilder/baselayout.html", line 2, in top-level template code
    {% import 'appbuilder/baselib.html' as baselib %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/init.html", line 46, in top-level template code
    {% block body %}
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/templates/appbuilder/baselayout.html", line 21, in block "body"
    {% block content %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html", line 13, in block "content"
    {% block list_list scoped %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html", line 15, in block "list_list"
    {{ widgets.get('list')()|safe }}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/widgets.py", line 34, in __call__
    return template.render(args)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/list.html", line 2, in top-level template code
    {% extends 'appbuilder/general/widgets/base_list.html' %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/base_list.html", line 23, in top-level template code
    {% block begin_loop_values %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/list.html", line 45, in block "begin_loop_values"
    {% for item in value_columns %}
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/runtime.py", line 452, in __next__
    ctx._after = ctx._safe_next()
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/runtime.py", line 430, in _safe_next
    return next(self._iterator)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/models/base.py", line 87, in get_values
    retdict[col] = self._get_attr_value(item, col)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/models/base.py", line 44, in _get_attr_value
    return getattr(item, col)()
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/models/helpers.py", line 74, in creator
    return self._user_link(self.created_by)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/models/helpers.py", line 70, in _user_link
    return Markup('<a href="{}">{}</a>'.format(url, escape(user) or ''))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 7: ordinal not in range(128)
Exception on /dashboardmodelview/list/ [GET]
Traceback (most recent call last):
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/security/decorators.py", line 26, in wraps
    return f(self, *args, **kwargs)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/views.py", line 453, in list
    widgets=widgets)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/baseviews.py", line 158, in render_template
    return render_template(template, **dict(list(kwargs.items()) + list(self.extra_args.items())))
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/templating.py", line 134, in render_template
    context, ctx.app)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/flask/templating.py", line 116, in _render
    rv = template.render(context)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html", line 2, in top-level template code
    {% import 'appbuilder/general/lib.html' as lib %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/base.html", line 1, in top-level template code
    {% extends base_template %}
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/templates/superset/base.html", line 1, in top-level template code
    {% extends "appbuilder/baselayout.html" %}
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/templates/appbuilder/baselayout.html", line 2, in top-level template code
    {% import 'appbuilder/baselib.html' as baselib %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/init.html", line 46, in top-level template code
    {% block body %}
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/templates/appbuilder/baselayout.html", line 21, in block "body"
    {% block content %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html", line 13, in block "content"
    {% block list_list scoped %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/model/list.html", line 15, in block "list_list"
    {{ widgets.get('list')()|safe }}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/widgets.py", line 34, in __call__
    return template.render(args)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/environment.py", line 1008, in render
    return self.environment.handle_exception(exc_info, True)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/list.html", line 2, in top-level template code
    {% extends 'appbuilder/general/widgets/base_list.html' %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/base_list.html", line 23, in top-level template code
    {% block begin_loop_values %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/list.html", line 45, in block "begin_loop_values"
    {% for item in value_columns %}
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/templates/appbuilder/general/widgets/list.html", line 45, in block "begin_loop_values"
    {% for item in value_columns %}
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/runtime.py", line 452, in __next__
    ctx._after = ctx._safe_next()
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/jinja2/runtime.py", line 430, in _safe_next
    return next(self._iterator)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/models/base.py", line 87, in get_values
    retdict[col] = self._get_attr_value(item, col)
  File "/opt/apps/superset/venv_2.7/lib64/python2.7/site-packages/flask_appbuilder/models/base.py", line 44, in _get_attr_value
    return getattr(item, col)()
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/models/helpers.py", line 74, in creator
    return self._user_link(self.created_by)
  File "/opt/apps/superset/venv_2.7/lib/python2.7/site-packages/superset/models/helpers.py", line 70, in _user_link
    return Markup('<a href="{}">{}</a>'.format(url, escape(user) or ''))
UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 7: ordinal not in range(128)

Notes:

  • Replacing the only 'ü' in a user's surname by 'ue' results in flawlessly loading pages again.
  • This issue is not present in a python 3 venv.
  • Setting export PYTHONIOENCODING=UTF-8 does not help
  • This dirty hack actually helps, but is strongly discouraged:
# Adding to superset_config.py:
import sys  # import sys package, if not already imported
reload(sys)
sys.setdefaultencoding('utf-8')

Steps to reproduce

E.g., add a 'ü' to a user's surname, ensure there is a dashboard owned by this user and then load the list of dashboards.
@xrmx
Copy link
Contributor

xrmx commented Apr 5, 2017 •
edited
Loading

The proper fix is to use unicode strings and not bytestrings in python2 which accidentally work if you live in an ascii world. One workaround is to stick this on top of all python files from __future__ import unicode_literals and make python2 behave like python3 that tend to work.

Said that it would be trivial to open a PR just sticking an u before the string literal that explodes for you.

@luciuschina luciuschina mentioned this issue Jul 27, 2017
Closed
@gangh
Copy link
Contributor

gangh commented Jan 18, 2018

This dirty hack actually helps, but is strongly discouraged:

only this way is useful ,thanks

@sebastianwindeck sebastianwindeck mentioned this issue Mar 1, 2018
Closed
3 tasks
@stale
Copy link

stale bot commented Apr 11, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

@stale stale bot added the inactive Inactive for >= 30 days label Apr 11, 2019
@stale stale bot closed this as completed Apr 18, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
inactive Inactive for >= 30 days
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@xrmx @rumbin @gangh