Merge branch 'master' into fix/dashboard-customcss-injection-error

apache · Mar 30, 2022 · ab22200 · ab22200
2 parents 69b23f2 + a4c261d
commit ab22200
Show file tree

Hide file tree

Showing 397 changed files with 22,186 additions and 31,556 deletions.
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -41,7 +41,7 @@ repos:
       - id: trailing-whitespace
         args: ["--markdown-linebreak-ext=md"]
   - repo: https://github.com/psf/black
-    rev: 19.10b0
+    rev: 22.3.0
     hooks:
       - id: black
         language_version: python3

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -17,6 +17,16 @@ specific language governing permissions and limitations
 under the License.
 -->
 ## Change Log
+
+### 1.4.2 (Sat Mar 19 00:08:06 2022 +0200)
+**Features**
+- [#19248](https://github.com/apache/superset/pull/19248) feat: add support for comments in adhoc clauses (@villebro)
+- [#18214](https://github.com/apache/superset/pull/18214) feat(docker-compose): add TAG option (@villebro)
+
+**Fixes**
+- [#17641](https://github.com/apache/superset/pull/17641) fix(sqla): make text clause escaping optional (@villebro)
+- [#18566](https://github.com/apache/superset/pull/18566) fix(plugin-chart-echarts): area chart opacity bug (@villebro)
+
 ### 1.4.1
 **Database Migrations**
 

diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md
@@ -106,7 +106,7 @@ This statement thanks the following, on which it draws for content and inspirati
 
 # Slack Community Guidelines
 
-If you decide to join the [Community Slack](https://join.slack.com/t/apache-superset/shared_invite/zt-uxbh5g36-AISUtHbzOXcu0BIj7kgUaw), please adhere to the following rules:
+If you decide to join the [Community Slack](https://join.slack.com/t/apache-superset/shared_invite/zt-16jvzmoi8-sI7jKWp~xc2zYRe~NqiY9Q), please adhere to the following rules:
 
 **1. Treat everyone in the community with respect.**
 

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -412,7 +412,7 @@ You also need to install MySQL or [MariaDB](https://mariadb.com/downloads).
 
 Ensure that you are using Python version 3.7 or 3.8, then proceed with:
 
-````bash
+```bash
 # Create a virtual environment and activate it (recommended)
 python3 -m venv venv # setup a python3 virtualenv
 source venv/bin/activate
@@ -457,7 +457,7 @@ $ make superset
 
 # Setup pre-commit only
 $ make pre-commit
-````
+```
 
 **Note: the FLASK_APP env var should not need to be set, as it's currently controlled
 via `.flaskenv`, however if needed, it should be set to `superset.app:create_app()`**

diff --git a/README.md b/README.md
@@ -25,7 +25,7 @@ under the License.
 [![PyPI version](https://badge.fury.io/py/apache-superset.svg)](https://badge.fury.io/py/apache-superset)
 [![Coverage Status](https://codecov.io/github/apache/superset/coverage.svg?branch=master)](https://codecov.io/github/apache/superset)
 [![PyPI](https://img.shields.io/pypi/pyversions/apache-superset.svg?maxAge=2592000)](https://pypi.python.org/pypi/apache-superset)
-[![Get on Slack](https://img.shields.io/badge/slack-join-orange.svg)](https://join.slack.com/t/apache-superset/shared_invite/zt-uxbh5g36-AISUtHbzOXcu0BIj7kgUaw)
+[![Get on Slack](https://img.shields.io/badge/slack-join-orange.svg)](https://join.slack.com/t/apache-superset/shared_invite/zt-16jvzmoi8-sI7jKWp~xc2zYRe~NqiY9Q)
 [![Documentation](https://img.shields.io/badge/docs-apache.org-blue.svg)](https://superset.apache.org)
 
 <img
@@ -129,7 +129,7 @@ Want to add support for your datastore or data engine? Read more [here](https://
 ## Get Involved
 
 - Ask and answer questions on [StackOverflow](https://stackoverflow.com/questions/tagged/apache-superset) using the **apache-superset** tag
-- [Join our community's Slack](https://join.slack.com/t/apache-superset/shared_invite/zt-uxbh5g36-AISUtHbzOXcu0BIj7kgUaw)
+- [Join our community's Slack](https://join.slack.com/t/apache-superset/shared_invite/zt-16jvzmoi8-sI7jKWp~xc2zYRe~NqiY9Q)
   and please read our [Slack Community Guidelines](https://github.com/apache/superset/blob/master/CODE_OF_CONDUCT.md#slack-community-guidelines)
 - [Join our [email protected] Mailing list](https://lists.apache.org/[email protected])
 
@@ -144,7 +144,7 @@ how to set up a development environment.
 
 - Getting Started with Superset
   - [Superset in 2 Minutes using Docker Compose](https://superset.apache.org/docs/installation/installing-superset-using-docker-compose#installing-superset-locally-using-docker-compose)
-  - [Installing Database Drivers](https://superset.apache.org/docs/databases/dockeradddrivers)
+  - [Installing Database Drivers](https://superset.apache.org/docs/databases/docker-add-drivers/)
   - [Building New Database Connectors](https://preset.io/blog/building-database-connector/)
   - [Create Your First Dashboard](https://superset.apache.org/docs/creating-charts-dashboards/first-dashboard)
   - [Comprehensive Tutorial for Contributing Code to Apache Superset

diff --git a/RELEASING/changelog.py b/RELEASING/changelog.py
@@ -167,7 +167,10 @@ def _get_changelog_version_head(self) -> str:
         return f"### {self._version} ({self._logs[0].time})"
 
     def _parse_change_log(
-        self, changelog: Dict[str, str], pr_info: Dict[str, str], github_login: str,
+        self,
+        changelog: Dict[str, str],
+        pr_info: Dict[str, str],
+        github_login: str,
     ) -> None:
         formatted_pr = (
             f"- [#{pr_info.get('id')}]"
@@ -355,7 +358,8 @@ def compare(base_parameters: BaseParameters) -> None:
 
 @cli.command("changelog")
 @click.option(
-    "--csv", help="The csv filename to export the changelog to",
+    "--csv",
+    help="The csv filename to export the changelog to",
 )
 @click.option(
     "--access_token",

diff --git a/RELEASING/send_email.py b/RELEASING/send_email.py
@@ -106,7 +106,12 @@ def inter_send_email(
 
 class BaseParameters(object):
     def __init__(
-        self, email: str, username: str, password: str, version: str, version_rc: str,
+        self,
+        email: str,
+        username: str,
+        password: str,
+        version: str,
+        version_rc: str,
     ) -> None:
         self.email = email
         self.username = username

diff --git a/UPDATING.md b/UPDATING.md
@@ -29,6 +29,9 @@ assists people when migrating to a new version.
 
 ### Breaking Changes
 
+- [19168](https://github.com/apache/superset/pull/19168): Celery upgrade to 5.X has breaking changes on it's command line invocation.
+  Please follow: https://docs.celeryq.dev/en/stable/whatsnew-5.2.html#step-1-adjust-your-command-line-invocation
+  Consider migrating you celery config if you haven't already: https://docs.celeryq.dev/en/stable/userguide/configuration.html#conf-old-settings-map
 - [19049](https://github.com/apache/superset/pull/19049): APP_ICON_WIDTH has been removed from the config. Superset should now be able to handle different logo sizes without having to explicitly set an APP_ICON_WIDTH. This might affect the size of existing custom logos as the UI will now resize them according to the specified space of maximum 148px and not according to the value of APP_ICON_WIDTH.
 - [19274](https://github.com/apache/superset/pull/19274): The `PUBLIC_ROLE_LIKE_GAMMA` config key has been removed, set `PUBLIC_ROLE_LIKE` = "Gamma" to have the same functionality.
 - [19273](https://github.com/apache/superset/pull/19273): The `SUPERSET_CELERY_WORKERS` and `SUPERSET_WORKERS` config keys has been removed. Configure celery directly using `CELERY_CONFIG` on Superset
@@ -59,7 +62,6 @@ flag for the legacy datasource editor (DISABLE_LEGACY_DATASOURCE_EDITOR) in conf
 
 ### Deprecations
 
-- [19078](https://github.com/apache/superset/pull/19078): Creation of old shorturl links has been deprecated in favor of a new permalink feature that solves the long url problem (old shorturls will still work, though!). By default, new permalinks use UUID4 as the key. However, to use serial ids similar to the old shorturls, add the following to your `superset_config.py`: `PERMALINK_KEY_TYPE = "id"`.
 - [18960](https://github.com/apache/superset/pull/18960): Persisting URL params in chart metadata is no longer supported. To set a default value for URL params in Jinja code, use the optional second argument: `url_param("my-param", "my-default-value")`.
 
 ### Other

diff --git a/docker/pythonpath_dev/superset_config.py b/docker/pythonpath_dev/superset_config.py
@@ -72,7 +72,7 @@ def get_env_variable(var_name: str, default: Optional[str] = None) -> str:
 
 class CeleryConfig(object):
     BROKER_URL = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_CELERY_DB}"
-    CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks")
+    CELERY_IMPORTS = ("superset.sql_lab",)
     CELERY_RESULT_BACKEND = f"redis://{REDIS_HOST}:{REDIS_PORT}/{REDIS_RESULTS_DB}"
     CELERYD_LOG_LEVEL = "DEBUG"
     CELERYD_PREFETCH_MULTIPLIER = 1

diff --git a/docs/README.md b/docs/README.md
@@ -24,7 +24,7 @@ This website is built using [Docusaurus 2](https://docusaurus.io/), a modern sta
 ### Installation
 
 ```
-$ yarn
+$ yarn install
 ```
 
 ### Local Development

diff --git a/docs/docs/api.mdx b/docs/docs/api.mdx
@@ -4,12 +4,12 @@ hide_title: true
 sidebar_position: 9
 ---
 
-import { Buffer } from "buffer";
+import { Buffer } from 'buffer';
 global.Buffer = Buffer;
-import SwaggerUI from "swagger-ui-react";
-import openapi from "/resources/openapi.json";
-import "swagger-ui-react/swagger-ui.css";
-// import { Alert } from "antd";
+import SwaggerUI from 'swagger-ui-react';
+import openapi from '/resources/openapi.json';
+import 'swagger-ui-react/swagger-ui.css';
+import { Alert } from 'antd';
 
 ## API
 
@@ -18,28 +18,16 @@ Superset's public **REST API** follows the
 documented here. The docs bellow are generated using
 [Swagger React UI](https://www.npmjs.com/package/swagger-ui-react).
 
-<!--
-  TODO: (corbinrobb) Uncomment Alert if/when antd gets added and remove Infima alert. Fix SwaggerUI readability in dark mode.
--->
-
-<!-- <Alert
+<Alert
   type="info"
   message={
-
     <div>
       <strong>NOTE! </strong>
       You can find an interactive version of this documentation on your local Superset
-      instance at <strong>/swagger/v1</strong> (if enabled)
+      instance at <strong>/swagger/v1</strong> (unless disabled)
     </div>
-
- }
-/> -->
-
-<div class="alert alert--info" role="alert">
-  <strong>NOTE! </strong>
-  You can find an interactive version of this documentation on your local Superset
-  instance at <strong>/swagger/v1</strong> (if enabled)
-</div>
+  }
+/>
 
 <br />
 <br />

diff --git a/docs/docs/contributing/contributing-page.mdx b/docs/docs/contributing/contributing-page.mdx
@@ -8,11 +8,11 @@ version: 1
 ## Contributing to Superset
 
 Superset is an [Apache Software foundation](https://www.apache.org/theapacheway/index.html) project.
-The core contributors (or committers) to Superset communicate primarily in the following channels (all of
-which you can join):
+The core contributors (or committers) to Superset communicate primarily in the following channels (
+which can be joined by anyone):
 
 - [Mailing list](https://lists.apache.org/[email protected])
-- [Apache Superset Slack community](https://join.slack.com/t/apache-superset/shared_invite/zt-uxbh5g36-AISUtHbzOXcu0BIj7kgUaw)
+- [Apache Superset Slack community](https://join.slack.com/t/apache-superset/shared_invite/zt-16jvzmoi8-sI7jKWp~xc2zYRe~NqiY9Q)
 - [Github issues and PR's](https://github.com/apache/superset/issues)
 
 More references:

diff --git a/docs/docs/contributing/hooks-and-linting.mdx b/docs/docs/contributing/hooks-and-linting.mdx
@@ -39,7 +39,7 @@ We use [Pylint](https://pylint.org/) for linting which can be invoked via:
 tox -e pylint
 ```
 
-In terms of best practices please advoid blanket disablement of Pylint messages globally (via `.pylintrc`) or top-level within the file header, albeit there being a few exceptions. Disablement should occur inline as it prevents masking issues and provides context as to why said message is disabled.
+In terms of best practices please avoid blanket disablement of Pylint messages globally (via `.pylintrc`) or top-level within the file header, albeit there being a few exceptions. Disablement should occur inline as it prevents masking issues and provides context as to why said message is disabled.
 
 Additionally, the Python code is auto-formatted using [Black](https://github.com/python/black) which
 is configured as a pre-commit hook. There are also numerous [editor integrations](https://black.readthedocs.io/en/stable/integrations/editors.html)

diff --git a/docs/docs/installation/alerts-reports.mdx b/docs/docs/installation/alerts-reports.mdx
@@ -89,12 +89,12 @@ REDIS_HOST = "redis-superset"
 REDIS_PORT = "6379"
 
 class CeleryConfig:
-    BROKER_URL = 'redis://%s:%s/0' % (REDIS_HOST, REDIS_PORT)
-    CELERY_IMPORTS = ('superset.sql_lab', "superset.tasks", "superset.tasks.thumbnails", )
-    CELERY_RESULT_BACKEND = 'redis://%s:%s/0' % (REDIS_HOST, REDIS_PORT)
-    CELERYD_PREFETCH_MULTIPLIER = 10
-    CELERY_ACKS_LATE = True
-    CELERY_ANNOTATIONS = {
+    broker_url = 'redis://%s:%s/0' % (REDIS_HOST, REDIS_PORT)
+    imports = ('superset.sql_lab', "superset.tasks", "superset.tasks.thumbnails", )
+    result_backend = 'redis://%s:%s/0' % (REDIS_HOST, REDIS_PORT)
+    worker_prefetch_multiplier = 10
+    task_acks_late = True
+    task_annotations = {
         'sql_lab.get_sql_results': {
             'rate_limit': '100/s',
         },
@@ -105,7 +105,7 @@ class CeleryConfig:
             'ignore_result': True,
         },
     }
-    CELERYBEAT_SCHEDULE = {
+    beat_schedule = {
         'reports.scheduler': {
             'task': 'reports.scheduler',
             'schedule': crontab(minute='*', hour='*'),

diff --git a/docs/docs/installation/async-queries-celery.mdx b/docs/docs/installation/async-queries-celery.mdx
@@ -23,16 +23,16 @@ and web server processes should have the same configuration.
 
 ```python
 class CeleryConfig(object):
-    BROKER_URL = 'redis://localhost:6379/0'
-    CELERY_IMPORTS = (
+    broker_url = 'redis://localhost:6379/0'
+    imports = (
         'superset.sql_lab',
         'superset.tasks',
     )
-    CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
-    CELERYD_LOG_LEVEL = 'DEBUG'
-    CELERYD_PREFETCH_MULTIPLIER = 10
-    CELERY_ACKS_LATE = True
-    CELERY_ANNOTATIONS = {
+    result_backend = 'redis://localhost:6379/0'
+    worker_log_level = 'DEBUG'
+    worker_prefetch_multiplier = 10
+    task_acks_late = True
+    task_annotations = {
         'sql_lab.get_sql_results': {
             'rate_limit': '100/s',
         },
@@ -43,7 +43,7 @@ class CeleryConfig(object):
             'ignore_result': True,
         },
     }
-    CELERYBEAT_SCHEDULE = {
+    beat_schedule = {
         'email_reports.schedule_hourly': {
             'task': 'email_reports.schedule_hourly',
             'schedule': crontab(minute=1, hour='*'),

diff --git a/docs/docs/installation/cache.mdx b/docs/docs/installation/cache.mdx
@@ -85,11 +85,11 @@ from s3cache.s3cache import S3Cache
 ...
 
 class CeleryConfig(object):
-    BROKER_URL = "redis://localhost:6379/0"
-    CELERY_IMPORTS = ("superset.sql_lab", "superset.tasks", "superset.tasks.thumbnails")
-    CELERY_RESULT_BACKEND = "redis://localhost:6379/0"
-    CELERYD_PREFETCH_MULTIPLIER = 10
-    CELERY_ACKS_LATE = True
+    broker_url = "redis://localhost:6379/0"
+    imports = ("superset.sql_lab", "superset.tasks", "superset.tasks.thumbnails")
+    result_backend = "redis://localhost:6379/0"
+    worker_prefetch_multiplier = 10
+    task_acks_late = True
 
 
 CELERY_CONFIG = CeleryConfig

diff --git a/docs/docs/installation/configuring-superset.mdx b/docs/docs/installation/configuring-superset.mdx
@@ -10,7 +10,7 @@ version: 1
 ### Configuration
 
 To configure your application, you need to create a file `superset_config.py` and add it to your
-`PYTHONPATH`. If your applcation was installed using docker-compose an alternative configuration is required. See [https://github.com/apache/superset/tree/master/docker#readme](https://github.com/apache/superset/tree/master/docker#readme) for details.
+`PYTHONPATH`. If your application was installed using docker-compose an alternative configuration is required. See [https://github.com/apache/superset/tree/master/docker#readme](https://github.com/apache/superset/tree/master/docker#readme) for details.
 
 Here are some of the parameters you can set in that file:
 ```
@@ -251,10 +251,10 @@ A current list of feature flags can be found in [RESOURCES/FEATURE_FLAGS.md](htt
 
 If you want to rotate the SECRET_KEY(change the existing secret key), follow the below steps.
 
-# Add the new SECRET_KEY and PREVIOUS_SECRET_KEY
+Add the new SECRET_KEY and PREVIOUS_SECRET_KEY to `superset_config.py`:
 
 ```python
 PREVIOUS_SECRET_KEY = 'CURRENT_SECRET_KEY' # The default SECRET_KEY for deployment is '21thisismyscretkey12eyyh'
 SECRET_KEY = 'YOUR_OWN_RANDOM_GENERATED_SECRET_KEY'
 ```
-# Then run `superset re-encrypt-secrets`
+Then run `superset re-encrypt-secrets`
diff --git a/docs/docs/installation/running-on-kubernetes.mdx b/docs/docs/installation/running-on-kubernetes.mdx
@@ -344,12 +344,10 @@ configOverrides:
     from celery.schedules import crontab
 
     class CeleryConfig(object):
-      BROKER_URL = f"redis://{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
-      CELERY_IMPORTS = ('superset.sql_lab', )
-      CELERY_RESULT_BACKEND = f"redis://{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
-      CELERY_ANNOTATIONS = {'tasks.add': {'rate_limit': '10/s'}}
-      CELERY_IMPORTS = ('superset.sql_lab', "superset.tasks", "superset.tasks.thumbnails", )
-      CELERY_ANNOTATIONS = {
+      broker_url = f"redis://{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
+      imports = ('superset.sql_lab', "superset.tasks", "superset.tasks.thumbnails", )
+      result_backend = f"redis://{env('REDIS_HOST')}:{env('REDIS_PORT')}/0"
+      task_annotations = {
           'sql_lab.get_sql_results': {
               'rate_limit': '100/s',
           },
@@ -360,7 +358,7 @@ configOverrides:
               'ignore_result': True,
           },
       }
-      CELERYBEAT_SCHEDULE = {
+      beat_schedule = {
           'reports.scheduler': {
               'task': 'reports.scheduler',
               'schedule': crontab(minute='*', hour='*'),

diff --git a/docs/docs/installation/sql-templating.mdx b/docs/docs/installation/sql-templating.mdx
@@ -120,7 +120,7 @@ In this section, we'll walkthrough the pre-defined Jinja macros in Superset.
 The `{{ current_username() }}` macro returns the username of the currently logged in user.
 
 If you have caching enabled in your Superset configuration, then by default the the `username` value will be used
-by Superset when calculating the cache key. A cache key is a unique identifer that determines if there's a
+by Superset when calculating the cache key. A cache key is a unique identifier that determines if there's a
 cache hit in the future and Superset can retrieve cached data.
 
 You can disable the inclusion of the `username` value in the calculation of the
@@ -134,8 +134,8 @@ cache key by adding the following parameter to your Jinja code:
 
 The `{{ current_user_id() }}` macro returns the user_id of the currently logged in user.
 
-If you have caching enabled in your Superset configuration, then by defaul the the `user_id` value will be used
-by Superset when calculating the cache key. A cache key is a unique identifer that determines if there's a
+If you have caching enabled in your Superset configuration, then by default the the `user_id` value will be used
+by Superset when calculating the cache key. A cache key is a unique identifier that determines if there's a
 cache hit in the future and Superset can retrieve cached data.
 
 You can disable the inclusion of the `user_id` value in the calculation of the

diff --git a/docs/docusaurus.config.js b/docs/docusaurus.config.js
@@ -204,7 +204,7 @@ const config = {
               },
               {
                 label: 'Slack',
-                href: 'https://join.slack.com/t/apache-superset/shared_invite/zt-uxbh5g36-AISUtHbzOXcu0BIj7kgUaw',
+                href: 'https://join.slack.com/t/apache-superset/shared_invite/zt-16jvzmoi8-sI7jKWp~xc2zYRe~NqiY9Q',
               },
               {
                 label: 'Mailing List',
-Original file line number
+Diff line change
@@ Expand Up @@
     ### Installation
     ```
-    $ yarn
+    $ yarn install
     ```
     ### Local Development
@@ Expand Down @@