Can't overwrite an existing dataset

[ganczarek]

Superset fails to overwrite an existing dataset in SQL Editor. Request fails with Exception: Dataset [examples].[Test] already exists in before_update function.

How to reproduce the bug

1. Go to SQL Editor
2. Run SELECT table_catalog, table_name FROM information_schema.tables
3. Click Explore. Enter Test in Save as new field and click Save & Explore
4. Go back to SQL Editor
5. Run modified query SELECT table_catalog FROM information_schema.tables
6. Click Explore. Enter Test in Overwrite existing field and click Save & Explore

Expected results

Superset overwrites dataset and redirects to Explore view

Actual results

Browser calls PUT http://localhost:8088/api/v1/dataset/1?override_columns=true with request:

{"sql":"SELECT table_catalog FROM information_schema.tables","columns":[{"column_name":"table_catalog"}],"database_id":1}

and Superset returns 500 Internal Server Error with response:

{"message":"Fatal error"}

Test dataset SQL query gets updated despite the server error.

Exception stacktrace:

superset_app            | 2021-10-13 16:34:08,662:ERROR:root:Dataset [examples].[Test] already exists
superset_app            | Traceback (most recent call last):
superset_app            |   File "/usr/local/lib/python3.7/site-packages/flask_appbuilder/api/__init__.py", line 85, in wraps
superset_app            |     return f(self, *args, **kwargs)
superset_app            |   File "/app/superset/views/base_api.py", line 85, in wraps
superset_app            |     raise ex
superset_app            |   File "/app/superset/views/base_api.py", line 82, in wraps
superset_app            |     duration, response = time_function(f, self, *args, **kwargs)
superset_app            |   File "/app/superset/utils/core.py", line 1429, in time_function
superset_app            |     response = func(*args, **kwargs)
superset_app            |   File "/app/superset/utils/log.py", line 241, in wrapper
superset_app            |     value = f(*args, **kwargs)
superset_app            |   File "/app/superset/datasets/api.py", line 331, in put
superset_app            |     g.user, pk, item, override_columns
superset_app            |   File "/app/superset/datasets/commands/update.py", line 71, in run
superset_app            |     override_columns=self.override_columns,
superset_app            |   File "/app/superset/datasets/dao.py", line 169, in update
superset_app            |     super().update(model, properties, commit=commit)
superset_app            |   File "/app/superset/dao/base.py", line 121, in update
superset_app            |     setattr(model, key, value)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 273, in __set__
superset_app            |     instance_state(instance), instance_dict(instance), value, None
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 1319, in set
superset_app            |     old = self.get(state, dict_, passive=PASSIVE_ONLY_PERSISTENT)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/attributes.py", line 723, in get
superset_app            |     value = self.callable_(state, passive)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 760, in _load_for_state
superset_app            |     session, state, primary_key_identity, passive
superset_app            |   File "<string>", line 1, in <lambda>
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/strategies.py", line 902, in _emit_lazyload
superset_app            |     .with_post_criteria(set_default_params)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 544, in all
superset_app            |     return list(self)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/ext/baked.py", line 439, in __iter__
superset_app            |     self.session._autoflush()
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 1618, in _autoflush
superset_app            |     self.flush()
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2536, in flush
superset_app            |     self._flush(objects)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2678, in _flush
superset_app            |     transaction.rollback(_capture_exception=True)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
superset_app            |     with_traceback=exc_tb,
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
superset_app            |     raise exception
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2638, in _flush
superset_app            |     flush_context.execute()
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
superset_app            |     rec.execute(self)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/unitofwork.py", line 589, in execute
superset_app            |     uow,
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 213, in save_obj
superset_app            |     ) in _organize_states_for_save(base_mapper, states, uowtransaction):
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/persistence.py", line 387, in _organize_states_for_save
superset_app            |     mapper.dispatch.before_update(mapper, connection, state)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/event/attr.py", line 322, in __call__
superset_app            |     fn(*args, **kw)
superset_app            |   File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/events.py", line 719, in wrap
superset_app            |     fn(*arg, **kw)
superset_app            |   File "/app/superset/connectors/sqla/models.py", line 1669, in before_update
superset_app            |     raise Exception(get_dataset_exist_error_msg(target.full_name))
superset_app            | Exception: Dataset [examples].[Test] already exists
superset_app            | 192.168.48.1 - - [13/Oct/2021:16:34:08 +0000] "PUT /api/v1/dataset/1?override_columns=true HTTP/1.1" 500 26 "http://localhost:8088/superset/sqllab/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36"

Environment

I experience this issue consistently in AWS running Superset v1.3.0. However, I also reproduced it with local Docker containers and tags/1.3.1, though it didn't happen the first time I tried, but after Docker containers recreation, so it may require a few tries.

git checkout tags/1.3.1
docker build --target lean --tag apache/superset:latest-dev .
docker-compose -f docker-compose-non-dev.yml up

• browser type and version: Brave v1.29.79
• superset version: v1.3.1
• python version: 3.7.9

Checklist

Make sure to follow these steps before submitting your issue - thank you!

• I have checked the superset logs for python stacktraces and included it here as text if there are any.
• I have reproduced the issue with at least the latest released version of superset.
• I have checked the issue tracker for the same issue and I haven't found one similar.

Additional context

I think that #16859 might have introduced that issue. #16434 is similar, but seems to address something slightly different.

[chathawee]

I also faced on this bug. How to fix this?

[mrcorey-fluency]

Same issue as above, same errors in log.

[agusfigueroa-htg]

Same issue here, happy to contribute anyhow

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

[rusackas]

@andrey-zayats took a look at this and was unable to reproduce the issue. I'm going to go ahead and remove the validation:required label and close the issue. If anyone sees this happening in Superset 2.0.x or later, or on master, we're open to revisiting this conversation, re-opening the ticket, or triaging a new Issue with updated context. Thank you!