Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: unskip command does not work with schemas/databases #3494

Open
1 task done
maruppel opened this issue Jan 7, 2025 · 1 comment · May be fixed by #3567
Open
1 task done

[BUG]: unskip command does not work with schemas/databases #3494

maruppel opened this issue Jan 7, 2025 · 1 comment · May be fixed by #3567

Comments

@maruppel
Copy link

maruppel commented Jan 7, 2025

Is there an existing issue for this?

  • I have searched the existing issues

Current Behavior

When attempting to run unskip on a schema, the following error is shown:

"[REDACTED]databricks\labs\ucx\lib\src\databricks\labs\ucx\hive_metastore\mapping.py", line 192, in unskip_schema
    self._sql_backend.execute(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        f"ALTER SCHEMA hive_metastore.{escape_sql_identifier(schema)} UNSET DBPROPERTIES IF EXISTS('{self.UCX_SKIP_PROPERTY}');"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\sdk\retries.py", line 54, in wrapper
    raise err
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\sdk\retries.py", line 33, in wrapper
    return func(*args, **kwargs)
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\labs\lsql\backends.py", line 220, in execute
    self._sql.execute(sql, catalog=catalog, schema=schema)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\labs\lsql\core.py", line 268, in execute
    self._raise_if_needed(status)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\labs\lsql\core.py", line 505, in _raise_if_needed
    raise error_class(error_message)
databricks.sdk.errors.platform.BadRequest:
[PARSE_SYNTAX_ERROR] Syntax error at or near 'UNSET'. SQLSTATE: 42601 (line 1, pos 38)
 
== SQL ==
ALTER SCHEMA hive_metastore.`[REDACTED]` UNSET DBPROPERTIES IF EXISTS('databricks.labs.ucx.skip');`

Expected Behavior

The unskip command removes the databricks.labs.ucx.skip property from the schema.

Steps To Reproduce

No response

Cloud

Azure

Operating System

Windows

Version

latest via Databricks CLI

Relevant log output

`"[REDACTED]databricks\labs\ucx\lib\src\databricks\labs\ucx\hive_metastore\mapping.py", line 192, in unskip_schema
    self._sql_backend.execute(
    ~~~~~~~~~~~~~~~~~~~~~~~~~^
        f"ALTER SCHEMA hive_metastore.{escape_sql_identifier(schema)} UNSET DBPROPERTIES IF EXISTS('{self.UCX_SKIP_PROPERTY}');"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\sdk\retries.py", line 54, in wrapper
    raise err
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\sdk\retries.py", line 33, in wrapper
    return func(*args, **kwargs)
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\labs\lsql\backends.py", line 220, in execute
    self._sql.execute(sql, catalog=catalog, schema=schema)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\labs\lsql\core.py", line 268, in execute
    self._raise_if_needed(status)
    ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "[REDACTED].databricks\labs\ucx\state\venv\Lib\site-packages\databricks\labs\lsql\core.py", line 505, in _raise_if_needed
    raise error_class(error_message)
databricks.sdk.errors.platform.BadRequest:
[PARSE_SYNTAX_ERROR] Syntax error at or near 'UNSET'. SQLSTATE: 42601 (line 1, pos 38)
 
== SQL ==
ALTER SCHEMA hive_metastore.`[REDACTED]` UNSET DBPROPERTIES IF EXISTS('databricks.labs.ucx.skip');`
@JoelMorton
Copy link

JoelMorton commented Jan 8, 2025

Adding related info if helpful. migrate-tables only checks for the existence of property databricks.labs.ucx.skip. Even if this property is set to false, a schema will be skipped in the migrate-tables job.

migrate-tables job skips "uc_test_2" schema:
INFO [d.l.u.hive_metastore.mapping][checking_databases_for_skip_property_4] Database uc_test_2 is marked to be skipped

That schema's property is set to false:
describe schema extended uc_test_2
((databricks.labs.ucx.skip,false))

@gueniai gueniai removed the bug label Jan 23, 2025
@github-project-automation github-project-automation bot moved this to Todo in UCX Jan 23, 2025
@FastLee FastLee linked a pull request Jan 23, 2025 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Development

Successfully merging a pull request may close this issue.

3 participants