Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot add database with Superset 1.1.0 installed from PyPi: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf5 in position 1: invalid start byte #14226

Closed
qlands opened this issue Apr 19, 2021 · 6 comments
Labels
data:databases Related to database configurations and connections question & help wanted Use Github discussions instead

Comments

@qlands
Copy link
Contributor

qlands commented Apr 19, 2021

I am trying to add a new database through the web interface with the following parameters:

DATABASE NAME = FS_eecb3567_d4b2_4224_bb42_023b5a1ab368 
SQLALCHEMY URI = mysql+mysqlconnector://root:[email protected]/FS_eecb3567_d4b2_4224_bb42_023b5a1ab368?charset=utf8

The test button returns OK:
INFO:werkzeug:127.0.0.1 - - [18/Apr/2021 19:29:11] "POST /api/v1/database/test_connection HTTP/1.1" 200 -

But the Add button fails:

ERROR:sqlalchemy.pool.impl.QueuePool:Error closing cursor
Traceback (most recent call last):
  File "/home/cquiros/data/projects2017/personal/software/superset/env_superset/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context
    self.dialect.do_execute(
  File "/home/cquiros/data/projects2017/personal/software/superset/env_superset/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
    cursor.execute(statement, parameters)
  File "/home/cquiros/data/projects2017/personal/software/superset/env_superset/lib/python3.8/site-packages/mysql/connector/cursor_cext.py", line 282, in execute
    self._handle_result(result)
  File "/home/cquiros/data/projects2017/personal/software/superset/env_superset/lib/python3.8/site-packages/mysql/connector/cursor_cext.py", line 173, in _handle_result
    self._handle_resultset()
  File "/home/cquiros/data/projects2017/personal/software/superset/env_superset/lib/python3.8/site-packages/mysql/connector/cursor_cext.py", line 681, in _handle_resultset
    self._rows = self._cnx.get_rows()[0]
  File "/home/cquiros/data/projects2017/personal/software/superset/env_superset/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 338, in get_rows
    else self._cmysql.fetch_row()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf5 in position 1: invalid start byte

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cquiros/data/projects2017/personal/software/superset/env_superset/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1368, in _safe_close_cursor
    cursor.close()
  File "/home/cquiros/data/projects2017/personal/software/superset/env_superset/lib/python3.8/site-packages/mysql/connector/cursor_cext.py", line 413, in close
    self._cnx.handle_unread_result()
  File "/home/cquiros/data/projects2017/personal/software/superset/env_superset/lib/python3.8/site-packages/mysql/connector/connection_cext.py", line 749, in handle_unread_result
    raise errors.InternalError("Unread result found")
mysql.connector.errors.InternalError: Unread result found
INFO:werkzeug:127.0.0.1 - - [18/Apr/2021 19:29:16] "POST /api/v1/database/ HTTP/1.1" 422 -

Here is my superset_config.py

from celery.schedules import crontab
from superset.typing import CacheConfig

# Superset specific config
ROW_LIMIT = 5000

SUPERSET_WEBSERVER_PORT = 8088

# Flask App Builder configuration
# Your App secret key
SECRET_KEY = "bEnA7XdTW2xstfUURfvh7P3NQuZYsL64"

# The SQLAlchemy connection string to your database backend
# This connection defines the path to the database that stores your
# superset metadata (slices, connections, tables, dashboards, ...).
# Note that the connection information to connect to the datasources
# you want to explore are managed directly in the web UI
SQLALCHEMY_DATABASE_URI = (
    "mysql+mysqlconnector://root:mypass@localhost/superset?ssl_disabled=True&charset=utf8"
)

# Flask-WTF flag for CSRF
WTF_CSRF_ENABLED = True
# Add endpoints that need to be exempt from CSRF protection
WTF_CSRF_EXEMPT_LIST = []
# A CSRF token that expires in 1 year
WTF_CSRF_TIME_LIMIT = 60 * 60 * 24 * 365

# Set this API key to enable Mapbox visualizations

CACHE_CONFIG: CacheConfig = {"CACHE_TYPE": "redis"}
DATA_CACHE_CONFIG: CacheConfig = {"CACHE_TYPE": "redis"}

DATA_CACHE_CONFIG = {
    "CACHE_TYPE": "redis",
    "CACHE_DEFAULT_TIMEOUT": 60 * 60 * 24,  # 1 day default (in secs)
    "CACHE_KEY_PREFIX": "superset_results",
    "CACHE_REDIS_URL": "redis://localhost:6379/2",
}


class CeleryConfig(object):
    BROKER_URL = "redis://localhost:6379/3"
    CELERY_IMPORTS = (
        "superset.sql_lab",
        "superset.tasks",
    )
    CELERY_RESULT_BACKEND = "redis://localhost:6379/3"
    CELERYD_LOG_LEVEL = "DEBUG"
    CELERYD_PREFETCH_MULTIPLIER = 10
    CELERY_ACKS_LATE = True
    CELERY_ANNOTATIONS = {
        "sql_lab.get_sql_results": {
            "rate_limit": "100/s",
        },
        "email_reports.send": {
            "rate_limit": "1/s",
            "time_limit": 120,
            "soft_time_limit": 150,
            "ignore_result": True,
        },
    }
    CELERYBEAT_SCHEDULE = {
        "email_reports.schedule_hourly": {
            "task": "email_reports.schedule_hourly",
            "schedule": crontab(minute=1, hour="*"),
        },
        "cache-warmup-hourly": {
            "task": "cache-warmup",
            "schedule": crontab(minute=0, hour="*"),  # hourly
            "kwargs": {
                "strategy_name": "top_n_dashboards",
                "top_n": 5,
                "since": "7 days ago",
            },
        },
    }


CELERY_CONFIG = CeleryConfig

Python version: 3.8

Installed libraries:

aiohttp==3.7.4.post0
alembic==1.5.8
amqp==2.6.1
apache-superset==1.1.0
apispec==3.3.2
appdirs==1.4.4
async-timeout==3.0.1
attrs==20.3.0
Babel==2.9.0
backoff==1.10.0
billiard==3.6.4.0
black==20.8b1
bleach==3.3.0
Brotli==1.0.9
cachelib==0.1.1
celery==4.4.7
cffi==1.14.5
chardet==4.0.0
click==7.1.2
colorama==0.4.4
contextlib2==0.6.0.post1
convertdate==2.3.2
cron-descriptor==1.2.24
croniter==1.0.12
cryptography==3.4.7
decorator==5.0.7
defusedxml==0.7.1
dnspython==2.1.0
email-validator==1.1.2
Flask==1.1.2
Flask-AppBuilder==3.2.2
Flask-Babel==1.0.0
Flask-Caching==1.10.1
Flask-Compress==1.9.0
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-Migrate==2.7.0
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.5.1
flask-talisman==0.7.0
Flask-WTF==0.14.3
geographiclib==1.50
geopy==2.1.0
greenlet==1.0.0
gunicorn==20.0.4
holidays==0.10.3
humanize==3.4.1
idna==3.1
isodate==0.6.0
itsdangerous==1.1.0
Jinja2==2.11.3
jsonschema==3.2.0
kombu==4.6.11
korean-lunar-calendar==0.2.1
Mako==1.1.4
Markdown==3.3.4
MarkupSafe==1.1.1
marshmallow==3.11.1
marshmallow-enum==1.5.1
marshmallow-sqlalchemy==0.23.1
msgpack==1.0.2
multidict==5.1.0
mypy-extensions==0.4.3
mysql-connector-python==8.0.23
numpy==1.20.2
packaging==20.9
pandas==1.2.4
parsedatetime==2.6
pathlib2==2.3.5
pathspec==0.8.1
pgsanity==0.2.9
polyline==1.4.0
prison==0.1.3
protobuf==3.15.8
py==1.10.0
pyarrow==3.0.0
pycparser==2.20
PyJWT==1.7.1
PyMeeus==0.5.11
pyparsing==2.4.7
pyrsistent==0.17.3
python-dateutil==2.8.1
python-dotenv==0.17.0
python-editor==1.0.4
python-geohash==0.8.5
python3-openid==3.2.0
pytz==2021.1
PyYAML==5.4.1
redis==3.5.3
regex==2021.4.4
retry==0.9.2
selenium==3.141.0
simplejson==3.17.2
six==1.15.0
slackclient==2.5.0
SQLAlchemy==1.3.24
SQLAlchemy-Utils==0.36.8
sqlparse==0.3.0
toml==0.10.2
typed-ast==1.4.3
typing-extensions==3.7.4.3
urllib3==1.26.4
vine==1.3.0
webencodings==0.5.1
Werkzeug==1.0.1
WTForms==2.3.3
WTForms-JSON==0.3.3
yarl==1.6.3
@qlands qlands added the #bug Bug report label Apr 19, 2021
@qlands
Copy link
Contributor Author

qlands commented Apr 19, 2021

This might be a driver incompatibility as it does not work with mysql-connector-python (8.0.23) but it works with PyMySQL (1.0.2). Both are pure Python implementations.

@junlincc
Copy link
Member

@betodealmeida @nytai 🙏

@junlincc junlincc added question & help wanted Use Github discussions instead data:databases Related to database configurations and connections and removed #bug Bug report labels Apr 21, 2021
@robdiciuccio
Copy link
Member

Can you try this fix? #13830

@srinify
Copy link
Contributor

srinify commented May 6, 2021

hi @qlands FYI I've edited your original comment to remove your Mapbox API key. I'd avoid sharing keys / sensitive info like this publicly :)

@qlands
Copy link
Contributor Author

qlands commented May 6, 2021

Ups... Thanks, @srinify . Sorry, @robdiciuccio I been swamped with work. I will test the fix asap.

@qlands
Copy link
Contributor Author

qlands commented May 18, 2021

Hi @robdiciuccio

I can see that #13830 was merged to Master on the 19th of April. Today I pulled the latest master, tested it, and can confirm that it was corrected. Thanks.

@qlands qlands closed this as completed May 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
data:databases Related to database configurations and connections question & help wanted Use Github discussions instead
Projects
None yet
Development

No branches or pull requests

4 participants