From 9d2518cc8d01ac5ee11d6466fb9fda2ddfb20ebb Mon Sep 17 00:00:00 2001 From: "Mark E. Haase" Date: Tue, 18 Oct 2016 15:47:23 -0400 Subject: [PATCH 1/3] Clean up PEP-505. * Replace emoji with C# spelling for operators. * Remove some unnecessary alternative spellings. * Add proper python code blocks. * Other miscellany. --- pep-0505.txt | 548 +++++++++++++++++++++++---------------------------- 1 file changed, 249 insertions(+), 299 deletions(-) diff --git a/pep-0505.txt b/pep-0505.txt index 334c4e71d45..4da9728d3a4 100644 --- a/pep-0505.txt +++ b/pep-0505.txt @@ -84,7 +84,9 @@ the server; otherwise, the request should be routed through the specified proxy server. This use of ``None`` is preferred here to some other sentinel value or the Null Object Pattern. [3]_ -Examples of this form abound. Consider ``types.py`` in the standard library:: +Examples of this form abound. Consider ``types.py`` in the standard library. + +.. code:: python def prepare_class(name, bases=(), kwds=None): if kwds is None: @@ -107,12 +109,12 @@ represent their respective languages' ``null``. The C language ``null`` often bleeds into Python, too, particularly for thin wrappers around C libraries. For example, in ``pyopenssl``, the ``X509`` class -has `a get_notBefore() method -`_ that returns either a timestamp or ``None``. This -function is a thin wrapper around an OpenSSL function with the return type -``ASN1_TIME *``. Because this C pointer may be ``null``, the Python wrapper must -be able to represent ``null``, and ``None`` is the chosen representation. +has a ``get_notBefore()`` `method `_ that returns +either a timestamp or ``None``. This function is a thin wrapper around an +OpenSSL function with the return type ``ASN1_TIME *``. Because this C pointer +may be ``null``, the Python wrapper must be able to represent ``null``, and +``None`` is the chosen representation. The representation of ``null`` is particularly noticeable when Python code is marshalling data between two systems. For example, consider a Python server that @@ -136,7 +138,7 @@ Behavior In Other Languages Given that ``null``-aware operators exist in other modern languages, it may be helpful to quickly understand how they work in those languages. -C# example:: +.. code:: csharp /* Null-coalescing. */ @@ -204,16 +206,21 @@ these alternatives may be undesirable for some common ``None`` patterns. Similar behavior can be achieved with the ``or`` operator, but ``or`` checks whether its left operand is false-y, not specifically ``None``. This can lead to surprising behavior. Consider the scenario of computing the price of some -products a customer has in his/her shopping cart:: +products a customer has in his/her shopping cart. + +.. code:: python >>> price = 100 - >>> requested_quantity = 5 >>> default_quantity = 1 - >>> (requested_quantity or default_quantity) * price - 500 + # If user didn't specify a quantity, then assume the default. >>> requested_quantity = None >>> (requested_quantity or default_quantity) * price 100 + # The user added 5 items to the cart. + >>> requested_quantity = 5 + >>> (requested_quantity or default_quantity) * price + 500 + # User removed 5 items from cart. >>> requested_quantity = 0 >>> (requested_quantity or default_quantity) * price # oops! 100 @@ -245,10 +252,12 @@ for error handling, they are no more prone to abuse than ``or`` is. Ternary Operator ~~~~~~~~~~~~~~~~ -Another common way to initialize default values is to use the ternary operator. -Here is an excerpt from the popular `Requests package `_:: +Another common way to intialize default values is to use the ternary operator. +Here is an excerpt from the popular `Requests package +`_. + +.. code:: python data = [] if data is None else data files = [] if files is None else files @@ -261,7 +270,9 @@ in an unintuitive order: the brain thinks, "use ``data`` if possible and use ``[]`` as a fallback," but the code puts the fallback *before* the preferred value. -The author of this package could have written it like this instead:: +The author of this package could have written it like this instead. + +.. code:: python data = data if data is not None else [] files = files if files is not None else [] @@ -285,7 +296,9 @@ the drawbacks. This first example is from a Python web crawler that uses the popular Flask framework as a front-end. This function retrieves information about a web site -from a SQL database and formats it as JSON to send to an HTTP client:: +from a SQL database and formats it as JSON to send to an HTTP client. + +.. code:: python class SiteView(FlaskView): @route('/site/', methods=['GET']) @@ -302,7 +315,7 @@ from a SQL database and formats it as JSON to send to an HTTP client:: Both ``first_seen`` and ``last_seen`` are allowed to be ``null`` in the database, and they are also allowed to be ``null`` in the JSON response. JSON -does not have a native way to represent a ``datetime``, so the server's +does not have a native way to represent a ``datetime``, so the the server's contract states that any non-``null`` date is represented as an ISO-8601 string. Note that this code is invalid by PEP-8 standards: several lines are over the @@ -313,7 +326,9 @@ repetition of identifiers on both sides of the ternary ``if`` and the verbosity of the ternary itself (10 characters out of a 78 character line length). One way to fix this code is to replace each ternary with a full ``if/else`` -block:: +block. + +.. code:: python class SiteView(FlaskView): @route('/site/', methods=['GET']) @@ -345,7 +360,9 @@ complicated data model was being used, then it would get tedious to continually write in this long form. The readability would start to suffer as the number of lines in the function grows, and a refactoring would be forced. -Another alternative is to rename some of the identifiers:: +Another alternative is to rename some of the identifiers. + +.. code:: python class SiteView(FlaskView): @route('/site/', methods=['GET']) @@ -369,11 +386,11 @@ aliases. These new identifiers are short enough to fit a ternary expression onto one line, but the identifiers are also less intuitive, e.g. ``fs`` versus ``first_seen``. -As a quick preview, consider an alternative rewrite using a new operator ``๐Ÿ’ฉ``. -(This spelling of the operator is merely a placeholder so that the *concept* can -be debated without arguing about *spelling*. It is not intended to reflect the -public's opinion of said operator. It may, however, bring new meaning to the -phrase "code smell".):: +As a quick preview, consider an alternative rewrite using a new operator. We +will temporarily use the spelling ``?.`` for this operator, but later in the +PEP there will be a full discussion about alternative spellings. + +.. code:: python class SiteView(FlaskView): @route('/site/', methods=['GET']) @@ -381,19 +398,21 @@ phrase "code smell".):: site = db.query('site_table').find(id_) return jsonify( - first_seen=site๐Ÿ’ฉfirst_seen.isoformat(), + first_seen=site.first_seen?.isoformat(), id=site.id, is_active=site.is_active, - last_seen=site๐Ÿ’ฉlast_seen.isoformat(), + last_seen=site.last_seen?.isoformat(), url=site.url.rstrip('/') ) -The ``๐Ÿ’ฉ`` operator behaves as a "safe navigation" operator, allowing a more +The ``?.`` operator behaves as a "safe navigation" operator, allowing a more concise syntax where the expression ``site.first_seen`` is not duplicated. The next example is from a trending project on GitHub called `Grab -`_, -which is a Python scraping library:: +`_, which is a Python scraping library. + +.. code:: python class BaseUploadObject(object): def find_content_type(self, filename): @@ -435,7 +454,9 @@ which is a Python scraping library:: This example contains several good examples of needing to provide default values. It is a bit verbose as it is, and it is certainly not improved by the -ternary operator:: +ternary operator. + +.. code:: python class BaseUploadObject(object): def find_content_type(self, filename): @@ -465,26 +486,27 @@ long that they must be wrapped. The overall readability is worsened, not improved. This code *might* be improved, though, if there was a syntactic shortcut for -this common need to supply a default value. We'll assume the fictitious -operator ``๐Ÿ”‘`` to avoid a premature debate about the spelling of said -operator:: +this common need to supply a default value. We will temporarily spell this +operator ``??``, but alternative spellings will be discussed further down. + +.. code:: python class BaseUploadObject(object): def find_ctype(self, filename): ctype, encoding = mimetypes.guess_type(filename) - return ctype ๐Ÿ”‘ 'application/octet-stream' + return ctype ?? 'application/octet-stream' class UploadContent(BaseUploadObject): def __init__(self, content, filename=None, content_type=None): self.content = content - self.filename = filename ๐Ÿ”‘ self.get_random_filename() - self.content_type = content_type ๐Ÿ”‘ self.find_ctype(self.filename) + self.filename = filename ?? self.get_random_filename() + self.content_type = content_type ?? self.find_ctype(self.filename) class UploadFile(BaseUploadObject): def __init__(self, path, filename=None, content_type=None): self.path = path - self.filename = filename ๐Ÿ”‘ os.path.split(path)[1] - self.content_type = content_type ๐Ÿ”‘ self.find_ctype(self.filename) + self.filename = filename ?? os.path.split(path)[1] + self.content_type = content_type ?? self.find_ctype(self.filename) This syntax has an intuitive ordering of the operands, e.g. ``ctype`` -- the preferred value -- comes before the fallback value. The terseness of the syntax @@ -504,10 +526,12 @@ Usage Of ``None`` In The Standard Library The previous sections show some code patterns that are claimed to be "common", but how common are they? The attached script `find-pep505.py -`_ is meant to +`_ is meant to answer this question. It uses the ``ast`` module to search for variations of the following patterns in any ``*.py`` file. +.. code:: python + >>> # None-coalescing if block ... >>> if a is None: @@ -542,6 +566,8 @@ following patterns in any ``*.py`` file. This script takes one or more names of Python source files to analyze:: +.. code:: shell + $ python3 find-pep505.py test.py $ find /usr/lib/python3.4 -name '*.py' | xargs python3 find-pep505.py @@ -577,11 +603,11 @@ The script prints out any matches it finds. Sample:: hand, we assume that ``and`` is always incorrect for safe navigation. The script has been tested against `test.py -`_ and the Python 3.4 +`_ and the Python 3.4 standard library, but it should work on any arbitrary Python 3 source code. The complete output from running it against the standard library is attached to this -proposal as -`find-pep505.out `_. +proposal as `find-pep505.out `_. The script counts how many matches it finds and prints the totals at the end:: @@ -614,17 +640,11 @@ natural to ask if it should also apply to function invocation syntax. It might be written as ``foo?()``, where ``foo`` is only called if it is not None. This idea was quickly rejected, for several reasons. -First, no other mainstream language has such syntax. Moreover, it would be -difficult to discern if a function call returned ``None`` because the function -itself returned ``None`` or because it was short-circuited. Finally, Python -evaluates arguments to a function before it looks up the function itself, so +First, no other mainstream language has such syntax. Second, Python evaluates +arguments to a function before it looks up the function itself, so ``foo?(bar())`` would still call ``bar()`` even if ``foo`` is ``None``. This behaviour is unexpected for a so-called "short-circuiting" operator. -Instead, the "``None``-severing" operator is proposed below. This operator -offers a concise form for writing ``None``-aware function expressions that is -truly short-circuiting. - ``?`` Unary Postfix Operator ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -633,7 +653,9 @@ introduced, a unary, postfix operator spelled ``?`` was suggested. The idea is that ``?`` might return a special object that could would override dunder methods that return ``self``. For example, ``foo?`` would evaluate to ``foo`` if it is not ``None``, otherwise it would evaluate to an instance of -``NoneQuestion``:: +``NoneQuestion``. + +.. code:: python class NoneQuestion(): def __call__(self, *args, **kwargs): @@ -651,7 +673,9 @@ evaluates to ``NoneQuestion`` if ``foo`` is None. This is a nifty generalization, but it's difficult to use in practice since most existing code won't know what ``NoneQuestion`` is. -Going back to one of the motivating examples above, consider the following:: +Going back to one of the motivating examples above, consider the following. + +.. code:: python >>> import json >>> created = None @@ -665,6 +689,8 @@ At the same time, the ``?`` operator may also be **too general**, in the sense that it can be combined with any other operator. What should the following expressions mean? +.. code:: python + >>> x? + 1 >>> x? -= 1 >>> x? == 1 @@ -687,7 +713,9 @@ doesn't translate cleanly into Python. There is a Python package called `pymaybe `_ that provides a rough approximation. The documentation shows the following example that appears -relevant to the discussion at hand:: +relevant to the discussion at hand. + +.. code:: python >>> maybe('VALUE').lower() 'value' @@ -710,12 +738,11 @@ not nearly as powerful as support built into the language. Specification ============= -This PEP suggests 4 new operators be added to Python: +This PEP suggests 3 new operators be added to Python: 1. ``None``-coalescing operator -2. ``None``-severing operator -3. ``None``-aware attribute access -4. ``None``-aware index access/slicing +2. ``None``-aware attribute access +3. ``None``-aware index access/slicing We will continue to assume the same spellings as in the previous sections in order to focus on behavior before diving into the much @@ -728,85 +755,84 @@ A generalization of these operators is also proposed below under the heading ``None``-Coalescing Operator ---------------------------- -The ``None``-coalescing operator is a short-circuiting, binary operator that behaves -in the following way. +The ``None``-coalescing operator is a short-circuiting, binary operator that +behaves in the following way. 1. Evaluate the left operand first. 2. If the left operand is not ``None``, then return it immediately. 3. Else, evaluate the right operand and return the result. -Some simple examples:: +Consider the following examples. We will continue to use the spelling ``??`` +here, but keep in mind that alternative spellings will be discussed below. + +.. code:: python - >>> 1 ๐Ÿ”‘ 2 + >>> 1 ?? 2 1 - >>> None ๐Ÿ”‘ 2 + >>> None ?? 2 2 - >>> 1 ๐Ÿ”‘ None - 1 Importantly, note that the right operand is not evaluated unless the left -operand is None:: +operand is None. + +.. code:: python >>> def err(): raise Exception('foo') - >>> 1 ๐Ÿ”‘ err() + >>> 1 ?? err() 1 - >>> None ๐Ÿ”‘ err() + >>> None ?? err() Traceback (most recent call last): File "", line 1, in File "", line 1, in err Exception: foo The operator is left associative. Combined with its short circuiting behavior, -this makes the operator easy to chain:: +this makes the operator easy to chain. + +.. code:: python >>> timeout = None >>> local_timeout = 60 >>> global_timeout = 300 - >>> timeout ๐Ÿ”‘ local_timeout ๐Ÿ”‘ global_timeout + >>> timeout ?? local_timeout ?? global_timeout 60 >>> local_timeout = None - >>> timeout ๐Ÿ”‘ local_timeout ๐Ÿ”‘ global_timeout + >>> timeout ?? local_timeout ?? global_timeout 300 - >>> import time - >>> timeout ๐Ÿ”‘ local_timeout ๐Ÿ”‘ global_timeout ๐Ÿ”‘ time.sleep(10) - 300 - -Note in the last example that ``time.sleep(10)`` represents an expensive -function call, e.g. initializing a complex data structure. In this example -``time.sleep`` is not evaluated, and the result ``300`` is returned instantly. - The operator has higher precedence than the comparison operators ``==``, ``>``, ``is``, etc., but lower precedence than any bitwise or arithmetic operators. This precedence is chosen for making "default value" expressions intuitive to -read and write:: +read and write. + +.. code:: python >>> user_flag = None >>> default_flag = True - >>> not user_flag ๐Ÿ”‘ default_flag # Same as next expression. + >>> not user_flag ?? default_flag # Same as next expression. False - >>> not (user_flag ๐Ÿ”‘ default_flag) # Same as previous. + >>> not (user_flag ?? default_flag) # Same as previous. False - >>> (not user_flag) ๐Ÿ”‘ default_flag # Different from previous. + >>> (not user_flag) ?? default_flag # Different from previous. True >>> user_quantity = None >>> default_quantity = 1 - >>> 1 == user_quantity ๐Ÿ”‘ default_quantity # Same as next expression. + >>> 1 == user_quantity ?? default_quantity # Same as next expression. True - >>> 1 == (user_quantity ๐Ÿ”‘ default_quantity) # Same as previous. + >>> 1 == (user_quantity ?? default_quantity) # Same as previous. True - >>> (1 == user_quantity) ๐Ÿ”‘ default_quantity # Different from previous. + >>> (1 == user_quantity) ?? default_quantity # Different from previous. False >>> user_words = None >>> default_words = ['foo', 'bar'] - >>> 'foo' in user_words ๐Ÿ”‘ default_words # Same as next expression. + >>> 'foo' in user_words ?? default_words # Same as next expression. True - >>> 'foo' in (user_words ๐Ÿ”‘ default_words) # Same as previous. + >>> 'foo' in (user_words ?? default_words) # Same as previous. True - >>> ('foo' in user_words) ๐Ÿ”‘ default_words # Different from previous. + >>> ('foo' in user_words) ?? default_words # Different from previous. Traceback (most recent call last): File "", line 1, in TypeError: argument of type 'NoneType' is not iterable @@ -814,120 +840,58 @@ read and write:: >>> user_discount = None >>> default_discount = 0.9 >>> price = 100 - >>> price * user_discount ๐Ÿ”‘ default_discount + >>> price * user_discount ?? default_discount Recall the example above of calculating the cost of items in a shopping cart, and the easy-to-miss bug. This type of bug is not possible with the ``None``- -coalescing operator, because there is no implicit type coersion to ``bool``:: +coalescing operator, because there is no implicit type coersion to ``bool``. + +.. code:: python - >>> price = 100 >>> requested_quantity = 0 >>> default_quantity = 1 - >>> (requested_quantity ๐Ÿ”‘ default_quantity) * price + >>> price = 100 + >>> (requested_quantity ?? default_quantity) * price 0 The ``None``-coalescing operator also has a corresponding assignment shortcut. -The following assignments are semantically equivalent:: +The following assignments are semantically similar, except that `foo` is only +looked up once when using the assignment shortcut. + +.. code:: python - >>> foo ๐Ÿ”‘= [] - >>> foo = foo ๐Ÿ”‘ [] + >>> foo ??= [] + >>> foo = foo ?? [] The ``None`` coalescing operator improves readability, especially when handling -default function arguments. Consider again the example of requests, rewritten to -use ``None``-coalescing:: +default function arguments. Consider again the example from the Requests +library, rewritten to use ``None``-coalescing. + +.. code:: python def __init__(self, data=None, files=None, headers=None, params=None, hooks=None): - self.data = data ๐Ÿ”‘ [] - self.files = files ๐Ÿ”‘ [] - self.headers = headers ๐Ÿ”‘ {} - self.params = params ๐Ÿ”‘ {} - self.hooks = hooks ๐Ÿ”‘ {} + self.data = data ?? [] + self.files = files ?? [] + self.headers = headers ?? {} + self.params = params ?? {} + self.hooks = hooks ?? {} The operator makes the intent easier to follow (by putting operands in an intuitive order) and is more concise than the ternary operator, while still preserving the short circuit semantics of the code that it replaces. -``None``-Severing Operator --------------------------- - -The idea of a ``None``-aware function invocation syntax was discussed on python- -ideas. The idea was not popular, so no such operator is included in this -proposal. (Justification for its exclusion is discussed in a previous section.) -Still, calling a function when it is not ``None`` is a common idiom in Python, -particularly for callback functions. Consider this hypothetical example:: - - import time - - def delay(seconds, callback=None): - time.sleep(seconds) - - if callback is not None: - callback() - -With the rejected ``None``-aware function call syntax, this example might be -written more concisely as:: - - import time - - def delay(seconds, callback=None): - time.sleep(seconds) - callback?() - -Instead, consider a "``None``-severing" operator, however, which is a short- -circuiting, boolean operator similar to the ``None``-coalesing operator, except -it returns ``None`` if the left operand is ``None`` and returns the right -operand otherwise. It has short circuiting behavior that compliments the -``None``-coalescing operator: if the left operand is None, then the right -operand is not evaluated. Let's temporarily spell this operator ``โœ‚`` and -rewrite the example accordingly:: - - import time - - def delay(seconds, callback=None): - time.sleep(seconds) - callback โœ‚ callback() - -At this point, you may be astonished at the mere suggestion of such a strange -operator with limited practical usefulness. It is proposed here because of the -symmetry it has with the ``None``-coalescing operator. This symmetry may be more -apparent if the two operators have complementary spellings. - -In the same way that ``or`` and ``and`` go together, ``None``-coalescing and -``None``- severing might be spelled in a pleasing, symmetric way, e.g. ``or?`` -and ``and?``. If such a spelling can be decided on, then this operator adds very -little cognitive load or special machinery to the language, and it's minor -utility may justify its inclusion in the language. - -Note that ``None``-severing could also be used as an alternative to "safe -navigation", at the expense of some repeated expressions:: - - >>> from datetime import datetime - >>> d = None - >>> type(d โœ‚ d.isoformat()) - - - >>> d = datetime.now() - >>> d โœ‚ d.isoformat() - '2015-10-16T20:53:40.312135' - -The repeated expression ``d`` makes this less useful than a ``None``-aware -attribute access operator, but to repeat what was said at the outset: this -proposal may be approved or rejected in whole or in part. This unlikely operator -is included in the proposal in order to be comprehensive. - -The precedence and associativity of the ``None``-severing operator are the same -as the ``None``-coalescing operator. - - ``None``-Aware Attribute Access Operator ---------------------------------------- The ``None``-aware attribute access operator (also called "safe navigation") checks its left operand. If the left operand is ``None``, then the operator -evaluates to ``None``. If the left operand is not ``None``, then the -operator accesses the attribute named by the right operand. As in the previous -section, we continue to use the temporary spelling ``๐Ÿ’ฉ``:: +evaluates to ``None``. If the the left operand is not ``None``, then the +operator accesses the attribute named by the right operand. We will continue to +use the spelling ``?.`` in this section, but keep in mind that alternative +spellings will be discussed below. + +.. code:: python >>> from datetime import date >>> d = date.today() @@ -940,7 +904,7 @@ section, we continue to use the temporary spelling ``๐Ÿ’ฉ``:: File "", line 1, in AttributeError: 'NoneType' object has no attribute 'year' - >>> d๐Ÿ’ฉyear + >>> d?.year None The operator has the same precedence and associativity as the plain attribute @@ -949,21 +913,26 @@ way: if the left operand is ``None``, then any series of attribute access, index access, slicing, or function call operators immediately to the right of it *are not evaluated*. +.. code:: python + >>> name = ' The Black Knight ' >>> name.strip()[4:].upper() 'BLACK KNIGHT' >>> name = None - >>> name๐Ÿ’ฉstrip()[4:].upper() + >>> name?.strip()[4:].upper() None If this operator did not short circuit in this way, then the second example -would partially evaluate ``name๐Ÿ’ฉstrip()`` to ``None()`` and then fail with +would partially evaluate ``name?.strip()`` to ``None()`` and then fail with ``TypeError: 'NoneType' object is not callable``. -To put it another way, the following expressions are semantically equivalent:: +To put it another way, the following expressions are semantically similar, +except that ``name`` is only looked up once on the first line. + +.. code:: python - >>> name๐Ÿ’ฉstrip()[4:].upper() + >>> name?.strip()[4:].upper() >>> name.strip()[4:].upper() if name is not None else None .. note:: @@ -975,21 +944,23 @@ To put it another way, the following expressions are semantically equivalent:: nearly useless. This operator short circuits one or more attribute access, index access, -slicing, or function call operators that are immediately to its right, but it +slicing, or function call operators that are adjacent to its right, but it does not short circuit any other operators (logical, bitwise, arithmetic, etc.), -nor does it escape parentheses:: +nor does it escape parentheses. + +.. code:: python >>> d = date.today() - >>> d๐Ÿ’ฉyear.numerator + 1 + >>> d?.year.numerator + 1 2016 >>> d = None - >>> d๐Ÿ’ฉyear.numerator + 1 + >>> d?.year.numerator + 1 Traceback (most recent call last): File "", line 1, in TypeError: unsupported operand type(s) for +: 'NoneType' and 'int' - >>> (d๐Ÿ’ฉyear).numerator + 1 + >>> (d?.year).numerator + 1 Traceback (most recent call last): File "", line 1, in AttributeError: 'NoneType' object has no attribute 'numerator' @@ -1003,9 +974,12 @@ The third example fails because the operator does not escape parentheses. In that example, the attribute access ``numerator`` is evaluated and fails because ``None`` does not have that attribute. -Finally, observe that short circuiting adjacent operators is not at all the same thing as propagating ``None`` throughout an expression:: +Finally, observe that short circuiting adjacent operators is not at all the same +thing as propagating ``None`` throughout an expression. - >>> user๐Ÿ’ฉfirst_name.upper() +.. code:: python + + >>> user?.first_name.upper() If ``user`` is not ``None``, then ``user.first_name`` is evaluated. If ``user.first_name`` evaluates to ``None``, then ``user.first_name.upper()`` is @@ -1013,9 +987,11 @@ an error! In English, this expression says, "``user`` is optional but if it has a value, then it must have a ``first_name``, too."" If ``first_name`` is supposed to be optional attribute, then the expression must -make that explicit:: +make that explicit. + +.. code:: python - >>> user๐Ÿ’ฉfirst_name๐Ÿ’ฉupper() + >>> user?.first_name?.upper() The operator is not intended as an error silencing mechanism, and it would be undesirable if its presence infected nearby operators. @@ -1027,7 +1003,9 @@ undesirable if its presence infected nearby operators. The ``None``-aware index access/slicing operator (also called "safe navigation") is nearly identical to the ``None``-aware attribute access operator. It combines the familiar square bracket syntax ``[]`` with new punctuation or a new keyword, -the spelling of which is discussed later:: +the spelling of which is discussed later. + +.. code:: python >>> person = {'name': 'Mark', 'age': 32} >>> person['name'] @@ -1039,10 +1017,12 @@ the spelling of which is discussed later:: File "", line 1, in TypeError: 'NoneType' object is not subscriptable - >>> person๐Ÿ’ฉ['name'] + >>> person?.['name'] None -The ``None``-aware slicing operator behaves similarly:: +The ``None``-aware slicing operator behaves similarly. + +.. code:: python >>> name = 'The Black Knight' >>> name[4:] @@ -1054,7 +1034,7 @@ The ``None``-aware slicing operator behaves similarly:: File "", line 1, in TypeError: 'NoneType' object is not subscriptable - >>> name๐Ÿ’ฉ[4:] + >>> name?.[4:] None These operators have the same precedence as the plain index access and slicing @@ -1071,12 +1051,16 @@ invoke a dunder method, e.g. ``__coalesce__(self)`` that returns ``True`` if an object should be coalesced and ``False`` otherwise. With this generalization, ``object`` would implement a dunder method equivalent -to this:: +to this. + +.. code:: python def __coalesce__(self): return False -``NoneType`` would implement a dunder method equivalent to this:: +``NoneType`` would implement a dunder method equivalent to this. + +.. code:: python def __coalesce__(self): return True @@ -1085,23 +1069,31 @@ If this generalization is accepted, then the operators will need to be renamed such that the term ``None`` is not used, e.g. "Coalescing Operator", "Coalesced Member Access Operator", etc. -The coalescing operator would invoke this dunder method. The following two expressions are semantically equivalent:: +The coalesce operator would invoke this dunder method. The following two +expressions are semantically similar, except `foo` is only looked up once when +using the coalesce operator. + +.. code:: python - >>> foo ๐Ÿ”‘ bar + >>> foo ?? bar >>> bar if foo.__coalesce__() else foo The coalesced attribute and index access operators would invoke the same dunder -method:: +method. + +.. code:: python - >>> user๐Ÿ’ฉfirst_name.upper() + >>> user?.first_name.upper() >>> None if user.__coalesce__() else user.first_name.upper() This generalization allows for domain-specific ``null`` objects to be coalesced -just like ``None``. For example, the ``pyasn1`` package has a type called +just like ``None``. For example the ``pyasn1`` package has a type called ``Null`` that represents an ASN.1 ``null``. +.. code:: python + >>> from pyasn1.type import univ - >>> univ.Null() ๐Ÿ”‘ univ.Integer(123) + >>> univ.Null() ?? univ.Integer(123) Integer(123) In addition to making the proposed operators less specialized, this @@ -1113,38 +1105,50 @@ Operator Spelling ----------------- Despite significant support for the proposed operators, the majority of -discussion on python-ideas fixated on the spelling. No consensus was achieved on -this question, for two reasons. First, Python eschews punctuation for logical -operators. For example, it uses ``not`` instead of ``!`` and ``โ€ฆ if โ€ฆ else โ€ฆ`` -instead of ``?:``. Introducing new punctuation is a major turnoff to many -Pythonistas, including BDFL. Second, adding new keywords to the language is -not backwards compatible. Any new keyword could only be introduced in the next -major version, e.g. Python 4. (Even then, `there would be resistance -`_.) - -Furthermore, nearly every single punctuation character on a standard keyboard -already has special meaning in Python. The only exceptions are ``$``, ``!``, -``?``, and backtick (as of Python 3). This leaves few options for a new, single- -character operator. A two character spelling is more likely, such as the ``??`` -and ``?.`` spellings in other programming languages, but this decreases the -appeal of punctuation even further. - -Finally, other projects in the Python universe assign special meaning to +discussion on python-ideas fixated on the spelling. Many alternative spellings +were proposed, including punctutation and keywords, but each alternative +drew significant criticism. + +Spelling the operators with punctuation has several downsides. First, Python +eschews punctuation for logical operators. For example, it uses ``not`` instead +of ``!`` and ``โ€ฆ if โ€ฆ else โ€ฆ`` instead of ``โ€ฆ ? โ€ฆ : โ€ฆ``. + +Second, nearly every single punctuation character on a standard keyboard already +has special meaning in Python. The only exceptions are ``$``, ``!``, ``?``, and +backtick (as of Python 3). This leaves few options for a new, single- character +operator. A two character spelling is found in other mainstream langauges, e.g. +``??`` and ``?.``, but this might uglier than a single character operator. + +Third, other projects in the Python universe assign special meaning to punctuation. For example, `IPython `_ assigns special meaning to ``%``, ``%%``, ``?``, ``??``, ``$``, and ``$$``, among others. Out of deference to those projects and the large communities using them, introducing conflicting syntax -into Python is undesirable. +into Python is undesirable. All of the proposed spellings below are compatible +with existing IPython shells. + +On the other hand, spelling the operator as a keyword is also unpopular. First, +adding new keywords to the language is not backwards compatible if programs are +already using identifiers with the same name. However, it is not impossible to +introduce a new keyword. For example, `PEP-492 `_ introduced the new keywords ``async`` and ``await`` into Python +3.5. + +It is also possible to craft a new operator out of existing keywords, as was +the case with `PEP-308 `_, which +created a ternary operator by cobbling together the `if` and `else` keywords +into a new operator. PEP-308 is a good inspiration for this PEP, because it +faced similar debates over spelling, particularly choosing whether to use the +same punctuation found in other languages or to choose a more Pythonic spelling +with keywords. -This is not the first PEP to deal with this dilemma. PEP-308 [5]_, which -introduced the ternary operator, faced similar issues. Alternative Spellings ~~~~~~~~~~~~~~~~~~~~~ -In keeping with the spirit of the PEP, many alternative spellings for these -``None``-aware operators are suggested, including some that conflict with each -other. Deconfliction will be handled only if any part of this proposal is +In keeping with the spirit of the PEP, several alternative spellings for each of +these ``None``-aware operators are suggested, including some that conflict with +each other. Deconfliction will be handled only if any part of this proposal is accepted. One caveat noted by several respondents on python-ideas: using similar spelling @@ -1155,8 +1159,14 @@ on non-``None``, while ``None``-aware attribute/index access short circuit on is only a practical concern if any part of this proposal is actually accepted, so there is no need to pontificate any further. -The following spellings are proposed candidates for the ``None``-coalescing -operator. +.. note:: + + Your author is not a CPython hacker nor a compiler developer. Some of the + proposed spellings may be difficult or impossible to implement with Python's + LL(1) parser. This is an area where I could use feedback on the PEP. + +The following spellings have been proposed on the python-ideas list for the +``None``-coalescing operator. 1. ``foo ?? bar ?? baz`` - Pros: same spelling as C# and Dart @@ -1166,45 +1176,12 @@ operator. - Pros: similar to existing ``or`` operator - Cons: the difference between this and ``or`` is not intuitive; punctuation is ugly; different precedence from ``or`` may be confusing -3. ``foo ? bar ? baz`` - - Pros: similar to ``??`` used in other languages - - Cons: punctuation is ugly; possible conflict with IPython; not used by any - other language -4. ``foo $$ bar $$ baz`` - - Pros: pronounced "value operator" because it returns the first operand - that has a "value" - - Cons: punctuation is ugly; not used by any other language -5. ``foo else bar else baz`` +3. ``foo else bar else baz`` - Pros: prettier than punctuation; uses an existing keyword - - Cons: difficult or impossible to implement with Python's LL(1) parser -6. ``foo or else bar or else baz`` - - Pros: prettier than punctuation; use existing keywords - - Cons: difficult or impossible to implement with Python's LL(1) parser -7. ``foo def bar def baz`` - - Pros: pronounced 'default'; prettier than punctuation - - Cons: difficult or impossible to implement with Python's LL(1) parser -8. ``foo then bar then baz`` + - Cons: might be confusing to add another meaning to ``else`` +4. ``foo then bar then baz`` - Pros: prettier than punctuation - - Cons: requires a new keyword, probably can't be implemented until Python 4 - (and maybe not even then) -9. No ``None``-coalescing operator. - - (Pros and cons discussed throughout this document.) - -The following spellings are proposed candidates for the ``None``-severing -operator. Each alternative has symmetry with one of the proposed spellings of -the ``None``- coalescing operator. - -1. ``foo !! bar`` - - Pros: symmetric with ``??`` - - Cons: punctuation is ugly; possible conflict with IPython; difficult to - google to find out what it means -2. ``foo and? bar`` - - Pros: symmetric with ``or?`` - - Cons: punctuation is ugly; possible conflict with IPython; difficult to - google to find out what it means; different precedence from ``and`` may be - confusing -3. No ``None``-severing operator. - - (Pros and cons discussed throughout this document.) + - Cons: requires a new keyword The following spellings are proposed candidates for the ``None``-aware attribute access operator. If you find any of these hard to read, consider that we may @@ -1216,26 +1193,9 @@ improve readability. - Cons: punctuation is ugly; possible conflict with IPython; difficult to google to find out what it means; difficult to differentiate from ``.`` when reading quickly -2. ``foo$.bar``, ``foo $. bar`` - - Pros: symmetry with ``$$`` operator proposed above - - Cons: punctuation is ugly; difficult to google; possible confusion because - it looks a bit like other languages' string interpolation; difficult to - google to find out what it means; difficult to differentiate from ``.`` - when reading quickly -3. ``foo!bar``, ``foo ! bar`` - - Pros: similar to ordinary ``.`` operator - - Cons: punctuation is ugly; possible conflict with IPython; no corresponding - spelling for index access (e.g. ``foo!['bar']`` is ambiguous) -4. ``foo->bar``, ``foo -> bar`` - - Pros: easier to read than other punctuation; less likely to be confused - with ordinary attribute access - - Cons: punctuation is ugly; difficult to google; confusing because it is - spelled the same as C's dereference operator -5. ``foo try .bar`` - - Pros: uses an existing keyword; - - Cons: difficult or impossible to implement in Python's LL(1) parser -6. No ``None``-aware attribute access operator. - - (Pros and cons discussed throughout this document.) +2. ``foo try .bar`` + - Pros: uses an existing keyword + - Cons: might be confusing to add another meaning to ``try`` The following spellings are proposed candidates for the ``None``-aware index access/slicing operator. The punctuation used for this operator ought to @@ -1245,20 +1205,10 @@ resemble the punctuation used for the ``None``-aware attribute access. - Pros: same spelling as C# and Dart - Cons: punctuation is ugly; possible conflict with IPython; difficult to google to find out what it means -2. ``foo$['bar']``, ``foo $ ['bar']`` - - Pros: symmetry with ``$$`` operator proposed above - - Cons: punctuation is ugly; possible confusion because - it looks a bit like other languages' string interpolation -3. ``foo->['bar']``, ``foo -> ['bar']`` - - Pros: easier to read than other punctuation; less likely to be confused - with ordinary attribute access - - Cons: punctuation is ugly; difficult to google; confusing because it is - spelled the same as C's dereference operator -4. ``foo try ['bar']`` +2. ``foo try ['bar']`` - Pros: uses an existing keyword; - Cons: difficult or impossible to implement in Python's LL(1) parser -5. No ``None``-aware index access/slicing operator. - - (Pros and cons discussed throughout this document.) + Community Poll ~~~~~~~~~~~~~~ From b8293620673b9c646583980d62158f5638093bbe Mon Sep 17 00:00:00 2001 From: "Mark E. Haase" Date: Fri, 28 Oct 2016 15:35:58 -0400 Subject: [PATCH 2/3] Update PEP-505 * Fix broken build (remove code:: blocks) * Address feedback from Mariatta --- pep-0505.txt | 183 +++++++++++++-------------------------------------- 1 file changed, 46 insertions(+), 137 deletions(-) diff --git a/pep-0505.txt b/pep-0505.txt index 4da9728d3a4..99a94157347 100644 --- a/pep-0505.txt +++ b/pep-0505.txt @@ -84,9 +84,7 @@ the server; otherwise, the request should be routed through the specified proxy server. This use of ``None`` is preferred here to some other sentinel value or the Null Object Pattern. [3]_ -Examples of this form abound. Consider ``types.py`` in the standard library. - -.. code:: python +Examples of this form abound. Consider ``types.py`` in the standard library:: def prepare_class(name, bases=(), kwds=None): if kwds is None: @@ -136,9 +134,7 @@ Behavior In Other Languages --------------------------- Given that ``null``-aware operators exist in other modern languages, it may be -helpful to quickly understand how they work in those languages. - -.. code:: csharp +helpful to quickly understand how they work in those languages:: /* Null-coalescing. */ @@ -206,9 +202,7 @@ these alternatives may be undesirable for some common ``None`` patterns. Similar behavior can be achieved with the ``or`` operator, but ``or`` checks whether its left operand is false-y, not specifically ``None``. This can lead to surprising behavior. Consider the scenario of computing the price of some -products a customer has in his/her shopping cart. - -.. code:: python +products a customer has in his/her shopping cart:: >>> price = 100 >>> default_quantity = 1 @@ -252,12 +246,10 @@ for error handling, they are no more prone to abuse than ``or`` is. Ternary Operator ~~~~~~~~~~~~~~~~ -Another common way to intialize default values is to use the ternary operator. +Another common way to initialize default values is to use the ternary operator. Here is an excerpt from the popular `Requests package `_. - -.. code:: python +a8149f5/requests/models.py#L212>`_:: data = [] if data is None else data files = [] if files is None else files @@ -270,9 +262,7 @@ in an unintuitive order: the brain thinks, "use ``data`` if possible and use ``[]`` as a fallback," but the code puts the fallback *before* the preferred value. -The author of this package could have written it like this instead. - -.. code:: python +The author of this package could have written it like this instead:: data = data if data is not None else [] files = files if files is not None else [] @@ -296,9 +286,7 @@ the drawbacks. This first example is from a Python web crawler that uses the popular Flask framework as a front-end. This function retrieves information about a web site -from a SQL database and formats it as JSON to send to an HTTP client. - -.. code:: python +from a SQL database and formats it as JSON to send to an HTTP client:: class SiteView(FlaskView): @route('/site/', methods=['GET']) @@ -315,8 +303,8 @@ from a SQL database and formats it as JSON to send to an HTTP client. Both ``first_seen`` and ``last_seen`` are allowed to be ``null`` in the database, and they are also allowed to be ``null`` in the JSON response. JSON -does not have a native way to represent a ``datetime``, so the the server's -contract states that any non-``null`` date is represented as an ISO-8601 string. +does not have a native way to represent a ``datetime``, so the server's contract +states that any non-``null`` date is represented as an ISO-8601 string. Note that this code is invalid by PEP-8 standards: several lines are over the line length limit. In fact, *including it in this document* violates the PEP @@ -326,9 +314,7 @@ repetition of identifiers on both sides of the ternary ``if`` and the verbosity of the ternary itself (10 characters out of a 78 character line length). One way to fix this code is to replace each ternary with a full ``if/else`` -block. - -.. code:: python +block:: class SiteView(FlaskView): @route('/site/', methods=['GET']) @@ -360,9 +346,7 @@ complicated data model was being used, then it would get tedious to continually write in this long form. The readability would start to suffer as the number of lines in the function grows, and a refactoring would be forced. -Another alternative is to rename some of the identifiers. - -.. code:: python +Another alternative is to rename some of the identifiers:: class SiteView(FlaskView): @route('/site/', methods=['GET']) @@ -388,9 +372,7 @@ one line, but the identifiers are also less intuitive, e.g. ``fs`` versus As a quick preview, consider an alternative rewrite using a new operator. We will temporarily use the spelling ``?.`` for this operator, but later in the -PEP there will be a full discussion about alternative spellings. - -.. code:: python +PEP there will be a full discussion about alternative spellings:: class SiteView(FlaskView): @route('/site/', methods=['GET']) @@ -410,9 +392,7 @@ concise syntax where the expression ``site.first_seen`` is not duplicated. The next example is from a trending project on GitHub called `Grab `_, which is a Python scraping library. - -.. code:: python +ab/upload.py>`_, which is a Python scraping library:: class BaseUploadObject(object): def find_content_type(self, filename): @@ -454,9 +434,7 @@ ab/upload.py>`_, which is a Python scraping library. This example contains several good examples of needing to provide default values. It is a bit verbose as it is, and it is certainly not improved by the -ternary operator. - -.. code:: python +ternary operator:: class BaseUploadObject(object): def find_content_type(self, filename): @@ -487,9 +465,7 @@ improved. This code *might* be improved, though, if there was a syntactic shortcut for this common need to supply a default value. We will temporarily spell this -operator ``??``, but alternative spellings will be discussed further down. - -.. code:: python +operator ``??``, but alternative spellings will be discussed further down:: class BaseUploadObject(object): def find_ctype(self, filename): @@ -526,11 +502,9 @@ Usage Of ``None`` In The Standard Library The previous sections show some code patterns that are claimed to be "common", but how common are they? The attached script `find-pep505.py -`_ is meant to -answer this question. It uses the ``ast`` module to search for variations of the -following patterns in any ``*.py`` file. - -.. code:: python +`_ is meant +to answer this question. It uses the ``ast`` module to search for variations of +the following patterns in any ``*.py`` file:: >>> # None-coalescing if block ... @@ -566,8 +540,6 @@ following patterns in any ``*.py`` file. This script takes one or more names of Python source files to analyze:: -.. code:: shell - $ python3 find-pep505.py test.py $ find /usr/lib/python3.4 -name '*.py' | xargs python3 find-pep505.py @@ -603,11 +575,11 @@ The script prints out any matches it finds. Sample:: hand, we assume that ``and`` is always incorrect for safe navigation. The script has been tested against `test.py -`_ and the Python 3.4 -standard library, but it should work on any arbitrary Python 3 source code. The -complete output from running it against the standard library is attached to this -proposal as `find-pep505.out `_. +`_ and the Python +3.4 standard library, but it should work on any arbitrary Python 3 source code. +The complete output from running it against the standard library is attached to +this proposal as `find-pep505.out +`_. The script counts how many matches it finds and prints the totals at the end:: @@ -653,9 +625,7 @@ introduced, a unary, postfix operator spelled ``?`` was suggested. The idea is that ``?`` might return a special object that could would override dunder methods that return ``self``. For example, ``foo?`` would evaluate to ``foo`` if it is not ``None``, otherwise it would evaluate to an instance of -``NoneQuestion``. - -.. code:: python +``NoneQuestion``:: class NoneQuestion(): def __call__(self, *args, **kwargs): @@ -675,8 +645,6 @@ won't know what ``NoneQuestion`` is. Going back to one of the motivating examples above, consider the following. -.. code:: python - >>> import json >>> created = None >>> json.dumps({'created': created?.isoformat()})`` @@ -687,9 +655,7 @@ throughout the standard library and any third party library. At the same time, the ``?`` operator may also be **too general**, in the sense that it can be combined with any other operator. What should the following -expressions mean? - -.. code:: python +expressions mean?:: >>> x? + 1 >>> x? -= 1 @@ -713,9 +679,7 @@ doesn't translate cleanly into Python. There is a Python package called `pymaybe `_ that provides a rough approximation. The documentation shows the following example that appears -relevant to the discussion at hand. - -.. code:: python +relevant to the discussion at hand:: >>> maybe('VALUE').lower() 'value' @@ -763,9 +727,7 @@ behaves in the following way. 3. Else, evaluate the right operand and return the result. Consider the following examples. We will continue to use the spelling ``??`` -here, but keep in mind that alternative spellings will be discussed below. - -.. code:: python +here, but keep in mind that alternative spellings will be discussed below:: >>> 1 ?? 2 1 @@ -773,9 +735,7 @@ here, but keep in mind that alternative spellings will be discussed below. 2 Importantly, note that the right operand is not evaluated unless the left -operand is None. - -.. code:: python +operand is None:: >>> def err(): raise Exception('foo') >>> 1 ?? err() @@ -787,9 +747,7 @@ operand is None. Exception: foo The operator is left associative. Combined with its short circuiting behavior, -this makes the operator easy to chain. - -.. code:: python +this makes the operator easy to chain:: >>> timeout = None >>> local_timeout = 60 @@ -804,9 +762,7 @@ this makes the operator easy to chain. The operator has higher precedence than the comparison operators ``==``, ``>``, ``is``, etc., but lower precedence than any bitwise or arithmetic operators. This precedence is chosen for making "default value" expressions intuitive to -read and write. - -.. code:: python +read and write:: >>> user_flag = None >>> default_flag = True @@ -844,9 +800,7 @@ read and write. Recall the example above of calculating the cost of items in a shopping cart, and the easy-to-miss bug. This type of bug is not possible with the ``None``- -coalescing operator, because there is no implicit type coersion to ``bool``. - -.. code:: python +coalescing operator, because there is no implicit type coersion to ``bool``:: >>> requested_quantity = 0 >>> default_quantity = 1 @@ -856,18 +810,14 @@ coalescing operator, because there is no implicit type coersion to ``bool``. The ``None``-coalescing operator also has a corresponding assignment shortcut. The following assignments are semantically similar, except that `foo` is only -looked up once when using the assignment shortcut. - -.. code:: python +looked up once when using the assignment shortcut:: >>> foo ??= [] >>> foo = foo ?? [] The ``None`` coalescing operator improves readability, especially when handling default function arguments. Consider again the example from the Requests -library, rewritten to use ``None``-coalescing. - -.. code:: python +library, rewritten to use ``None``-coalescing:: def __init__(self, data=None, files=None, headers=None, params=None, hooks=None): self.data = data ?? [] @@ -889,9 +839,7 @@ checks its left operand. If the left operand is ``None``, then the operator evaluates to ``None``. If the the left operand is not ``None``, then the operator accesses the attribute named by the right operand. We will continue to use the spelling ``?.`` in this section, but keep in mind that alternative -spellings will be discussed below. - -.. code:: python +spellings will be discussed below:: >>> from datetime import date >>> d = date.today() @@ -911,9 +859,7 @@ The operator has the same precedence and associativity as the plain attribute access operator ``.``, but this operator is also short-circuiting in a unique way: if the left operand is ``None``, then any series of attribute access, index access, slicing, or function call operators immediately to the right of it *are -not evaluated*. - -.. code:: python +not evaluated*:: >>> name = ' The Black Knight ' >>> name.strip()[4:].upper() @@ -928,9 +874,7 @@ would partially evaluate ``name?.strip()`` to ``None()`` and then fail with ``TypeError: 'NoneType' object is not callable``. To put it another way, the following expressions are semantically similar, -except that ``name`` is only looked up once on the first line. - -.. code:: python +except that ``name`` is only looked up once on the first line:: >>> name?.strip()[4:].upper() >>> name.strip()[4:].upper() if name is not None else None @@ -946,9 +890,7 @@ except that ``name`` is only looked up once on the first line. This operator short circuits one or more attribute access, index access, slicing, or function call operators that are adjacent to its right, but it does not short circuit any other operators (logical, bitwise, arithmetic, etc.), -nor does it escape parentheses. - -.. code:: python +nor does it escape parentheses:: >>> d = date.today() >>> d?.year.numerator + 1 @@ -975,9 +917,7 @@ that example, the attribute access ``numerator`` is evaluated and fails because ``None`` does not have that attribute. Finally, observe that short circuiting adjacent operators is not at all the same -thing as propagating ``None`` throughout an expression. - -.. code:: python +thing as propagating ``None`` throughout an expression:: >>> user?.first_name.upper() @@ -987,9 +927,7 @@ an error! In English, this expression says, "``user`` is optional but if it has a value, then it must have a ``first_name``, too."" If ``first_name`` is supposed to be optional attribute, then the expression must -make that explicit. - -.. code:: python +make that explicit:: >>> user?.first_name?.upper() @@ -1003,9 +941,7 @@ undesirable if its presence infected nearby operators. The ``None``-aware index access/slicing operator (also called "safe navigation") is nearly identical to the ``None``-aware attribute access operator. It combines the familiar square bracket syntax ``[]`` with new punctuation or a new keyword, -the spelling of which is discussed later. - -.. code:: python +the spelling of which is discussed later:: >>> person = {'name': 'Mark', 'age': 32} >>> person['name'] @@ -1020,9 +956,7 @@ the spelling of which is discussed later. >>> person?.['name'] None -The ``None``-aware slicing operator behaves similarly. - -.. code:: python +The ``None``-aware slicing operator behaves similarly:: >>> name = 'The Black Knight' >>> name[4:] @@ -1051,16 +985,12 @@ invoke a dunder method, e.g. ``__coalesce__(self)`` that returns ``True`` if an object should be coalesced and ``False`` otherwise. With this generalization, ``object`` would implement a dunder method equivalent -to this. - -.. code:: python +to this:: def __coalesce__(self): return False -``NoneType`` would implement a dunder method equivalent to this. - -.. code:: python +``NoneType`` would implement a dunder method equivalent to this:: def __coalesce__(self): return True @@ -1071,26 +1001,20 @@ Member Access Operator", etc. The coalesce operator would invoke this dunder method. The following two expressions are semantically similar, except `foo` is only looked up once when -using the coalesce operator. - -.. code:: python +using the coalesce operator:: >>> foo ?? bar >>> bar if foo.__coalesce__() else foo The coalesced attribute and index access operators would invoke the same dunder -method. - -.. code:: python +method:: >>> user?.first_name.upper() >>> None if user.__coalesce__() else user.first_name.upper() This generalization allows for domain-specific ``null`` objects to be coalesced just like ``None``. For example the ``pyasn1`` package has a type called -``Null`` that represents an ASN.1 ``null``. - -.. code:: python +``Null`` that represents an ASN.1 ``null``:: >>> from pyasn1.type import univ >>> univ.Null() ?? univ.Integer(123) @@ -1210,21 +1134,6 @@ resemble the punctuation used for the ``None``-aware attribute access. - Cons: difficult or impossible to implement in Python's LL(1) parser -Community Poll -~~~~~~~~~~~~~~ - -In order to collect data about the Python community's preferences for -``None``-aware operators, and with BDFL's consent, a public poll will be -conducted, just as with PEP-308. The poll is viewed as a data-gathering -exercise, not a democratic vote. - -The poll will allow respondents to rank their favorite options from the previous -section. The results will -be placed in this section of the PEP. - -...TBD... - - Implementation -------------- From a68b437b48216c5e7bd00b5ebbc8cbdd39c52343 Mon Sep 17 00:00:00 2001 From: "Mark E. Haase" Date: Mon, 31 Oct 2016 11:14:52 -0400 Subject: [PATCH 3/3] PEP-0505: Commmit to a specific syntax I've removed all of the alternative syntax discussion and the community poll. I expanded the reasoning for picking the spellings that I did. --- pep-0505.txt | 236 +++++++++++++++++---------------------------------- 1 file changed, 77 insertions(+), 159 deletions(-) diff --git a/pep-0505.txt b/pep-0505.txt index 99a94157347..c21b51e64e6 100644 --- a/pep-0505.txt +++ b/pep-0505.txt @@ -370,9 +370,7 @@ aliases. These new identifiers are short enough to fit a ternary expression onto one line, but the identifiers are also less intuitive, e.g. ``fs`` versus ``first_seen``. -As a quick preview, consider an alternative rewrite using a new operator. We -will temporarily use the spelling ``?.`` for this operator, but later in the -PEP there will be a full discussion about alternative spellings:: +As a quick preview, consider an alternative rewrite using a new operator:: class SiteView(FlaskView): @route('/site/', methods=['GET']) @@ -464,8 +462,7 @@ long that they must be wrapped. The overall readability is worsened, not improved. This code *might* be improved, though, if there was a syntactic shortcut for -this common need to supply a default value. We will temporarily spell this -operator ``??``, but alternative spellings will be discussed further down:: +this common need to supply a default value:: class BaseUploadObject(object): def find_ctype(self, filename): @@ -643,7 +640,7 @@ evaluates to ``NoneQuestion`` if ``foo`` is None. This is a nifty generalization, but it's difficult to use in practice since most existing code won't know what ``NoneQuestion`` is. -Going back to one of the motivating examples above, consider the following. +Going back to one of the motivating examples above, consider the following:: >>> import json >>> created = None @@ -716,6 +713,60 @@ A generalization of these operators is also proposed below under the heading "Generalized Coalescing". +Operator Spelling +----------------- + +Despite significant support for the proposed operators, the majority of +discussion on python-ideas fixated on the spelling. Many alternative spellings +were proposed, both punctuation and keywords, but each alternative drew some +criticism. Spelling the operator as a keyword is problematic, because adding new +keywords to the language is not backwards compatible. + +It is not impossible to add a new keyword, however, and we can look at several +other PEPs for inspiration. For example, `PEP-492 +`_ introduced the new keywords +``async`` and ``await`` into Python 3.5. These new keywords are fully backwards +compatible, because that PEP also introduces a new lexical context such that +``async`` and ``await`` are only treated as keywords when used inside of an +``async def`` function. In other locations, ``async`` and ``await`` may be used +as identifiers. + +It is also possible to craft a new operator out of existing keywords, as was +the case with `PEP-308 `_, which +created a ternary operator by cobbling together the `if` and `else` keywords +into a new operator. + +In addition to the lexical acrobatics required to create a new keyword, keyword +operators are also undesirable for creating an assignment shortcut syntax. In +Dart, for example, ``x ??= y`` is an assignment shortcut that approximately +means ``x = x ?? y`` except that ``x`` is only evaluated once. If Python's +coalesce operator is a keyword, e.g. ``foo``, then the assignment shortcut would +be very ugly: ``x foo= y``. + +Spelling new logical operators with punctuation is unlikely, for several +reasons. First, Python eschews punctuation for logical operators. For example, +it uses ``not`` instead of ``!``, ``or`` instead of ``||``, and ``โ€ฆ if โ€ฆ else โ€ฆ`` +instead of ``โ€ฆ ? โ€ฆ : โ€ฆ``. + +Second, nearly every single punctuation character on a standard keyboard already +has special meaning in Python. The only exceptions are ``$``, ``!``, ``?``, and +backtick (as of Python 3). This leaves few options for a new, single-character +operator. + +Third, other projects in the Python universe assign special meaning to +punctuation. For example, `IPython +`_ assigns +special meaning to ``%``, ``%%``, ``?``, ``??``, ``$``, and ``$$``, among +others. Out of deference to those projects and the large communities using them, +introducing conflicting syntax into Python is undesirable. + +The spellings ``??`` and ``?.`` will be familiar to programmers who have seen +them in other popular programming languages. Any alternative punctuation will be +just as ugly but without the benefit of familiarity from other languages. +Therefore, this proposal spells the new operators using the same punctuation +that already exists in other languages. + + ``None``-Coalescing Operator ---------------------------- @@ -764,39 +815,17 @@ The operator has higher precedence than the comparison operators ``==``, ``>``, This precedence is chosen for making "default value" expressions intuitive to read and write:: - >>> user_flag = None - >>> default_flag = True - >>> not user_flag ?? default_flag # Same as next expression. - False - >>> not (user_flag ?? default_flag) # Same as previous. - False - >>> (not user_flag) ?? default_flag # Different from previous. - True + >>> not None ?? True + >>> not (None ?? True) # Same precedence - >>> user_quantity = None - >>> default_quantity = 1 - >>> 1 == user_quantity ?? default_quantity # Same as next expression. - True - >>> 1 == (user_quantity ?? default_quantity) # Same as previous. - True - >>> (1 == user_quantity) ?? default_quantity # Different from previous. - False - - >>> user_words = None - >>> default_words = ['foo', 'bar'] - >>> 'foo' in user_words ?? default_words # Same as next expression. - True - >>> 'foo' in (user_words ?? default_words) # Same as previous. - True - >>> ('foo' in user_words) ?? default_words # Different from previous. - Traceback (most recent call last): - File "", line 1, in - TypeError: argument of type 'NoneType' is not iterable + >>> 1 == None ?? 1 + >>> 1 == (None ?? 1) # Same precedence - >>> user_discount = None - >>> default_discount = 0.9 - >>> price = 100 - >>> price * user_discount ?? default_discount + >>> 'foo' in None ?? ['foo', 'bar'] + >>> 'foo' in (None ?? ['foo', 'bar']) # Same precedence + + >>> 1 + None ?? 2 + >>> 1 + (None ?? 2) # Same precedence Recall the example above of calculating the cost of items in a shopping cart, and the easy-to-miss bug. This type of bug is not possible with the ``None``- @@ -805,11 +834,11 @@ coalescing operator, because there is no implicit type coersion to ``bool``:: >>> requested_quantity = 0 >>> default_quantity = 1 >>> price = 100 - >>> (requested_quantity ?? default_quantity) * price + >>> requested_quantity ?? default_quantity * price 0 The ``None``-coalescing operator also has a corresponding assignment shortcut. -The following assignments are semantically similar, except that `foo` is only +The following assignments are semantically similar, except that ```foo`` is only looked up once when using the assignment shortcut:: >>> foo ??= [] @@ -837,9 +866,7 @@ preserving the short circuit semantics of the code that it replaces. The ``None``-aware attribute access operator (also called "safe navigation") checks its left operand. If the left operand is ``None``, then the operator evaluates to ``None``. If the the left operand is not ``None``, then the -operator accesses the attribute named by the right operand. We will continue to -use the spelling ``?.`` in this section, but keep in mind that alternative -spellings will be discussed below:: +operator accesses the attribute named by the right operand:: >>> from datetime import date >>> d = date.today() @@ -979,10 +1006,10 @@ operators. They also have the same short-circuiting behavior as the Generalized Coalescing ---------------------- -Making ``None`` a special case may seem too specialized and magical. It is -possible to generalize the behavior by making the ``None``-aware operators -invoke a dunder method, e.g. ``__coalesce__(self)`` that returns ``True`` if an -object should be coalesced and ``False`` otherwise. +Making ``None`` a special case is too specialized and magical. The behavior can +be generalized by making the ``None``-aware operators invoke a dunder method, +e.g. ``__coalesce__(self)`` that returns ``True`` if an object should be +coalesced and ``False`` otherwise. With this generalization, ``object`` would implement a dunder method equivalent to this:: @@ -1025,122 +1052,13 @@ generalization also makes it easier to work with the Null Object Pattern, [3]_ for those developers who prefer to avoid using ``None``. -Operator Spelling ------------------ - -Despite significant support for the proposed operators, the majority of -discussion on python-ideas fixated on the spelling. Many alternative spellings -were proposed, including punctutation and keywords, but each alternative -drew significant criticism. - -Spelling the operators with punctuation has several downsides. First, Python -eschews punctuation for logical operators. For example, it uses ``not`` instead -of ``!`` and ``โ€ฆ if โ€ฆ else โ€ฆ`` instead of ``โ€ฆ ? โ€ฆ : โ€ฆ``. - -Second, nearly every single punctuation character on a standard keyboard already -has special meaning in Python. The only exceptions are ``$``, ``!``, ``?``, and -backtick (as of Python 3). This leaves few options for a new, single- character -operator. A two character spelling is found in other mainstream langauges, e.g. -``??`` and ``?.``, but this might uglier than a single character operator. - -Third, other projects in the Python universe assign special meaning to -punctuation. For example, `IPython `_ assigns special meaning to ``%``, ``%%``, -``?``, ``??``, ``$``, and ``$$``, among others. Out of deference to those -projects and the large communities using them, introducing conflicting syntax -into Python is undesirable. All of the proposed spellings below are compatible -with existing IPython shells. - -On the other hand, spelling the operator as a keyword is also unpopular. First, -adding new keywords to the language is not backwards compatible if programs are -already using identifiers with the same name. However, it is not impossible to -introduce a new keyword. For example, `PEP-492 `_ introduced the new keywords ``async`` and ``await`` into Python -3.5. - -It is also possible to craft a new operator out of existing keywords, as was -the case with `PEP-308 `_, which -created a ternary operator by cobbling together the `if` and `else` keywords -into a new operator. PEP-308 is a good inspiration for this PEP, because it -faced similar debates over spelling, particularly choosing whether to use the -same punctuation found in other languages or to choose a more Pythonic spelling -with keywords. - - -Alternative Spellings -~~~~~~~~~~~~~~~~~~~~~ - -In keeping with the spirit of the PEP, several alternative spellings for each of -these ``None``-aware operators are suggested, including some that conflict with -each other. Deconfliction will be handled only if any part of this proposal is -accepted. - -One caveat noted by several respondents on python-ideas: using similar spelling -for ``None`` coalescing and other ``None``-aware operators may be confusing, -because they have different short circuit semantics: coalescing short circuits -on non-``None``, while ``None``-aware attribute/index access short circuit on -``None``. This is a potential downside to spellings like ``??`` and ``?.``. This -is only a practical concern if any part of this proposal is actually accepted, -so there is no need to pontificate any further. - -.. note:: - - Your author is not a CPython hacker nor a compiler developer. Some of the - proposed spellings may be difficult or impossible to implement with Python's - LL(1) parser. This is an area where I could use feedback on the PEP. - -The following spellings have been proposed on the python-ideas list for the -``None``-coalescing operator. - -1. ``foo ?? bar ?? baz`` - - Pros: same spelling as C# and Dart - - Cons: punctuation is ugly; possible conflict with IPython; difficult to - google to find out what it means -2. ``foo or? bar or? baz`` - - Pros: similar to existing ``or`` operator - - Cons: the difference between this and ``or`` is not intuitive; punctuation - is ugly; different precedence from ``or`` may be confusing -3. ``foo else bar else baz`` - - Pros: prettier than punctuation; uses an existing keyword - - Cons: might be confusing to add another meaning to ``else`` -4. ``foo then bar then baz`` - - Pros: prettier than punctuation - - Cons: requires a new keyword - -The following spellings are proposed candidates for the ``None``-aware attribute -access operator. If you find any of these hard to read, consider that we may -adopt a convention of adding whitespace around a ``None``-aware operator to -improve readability. - -1. ``foo?.bar``, ``foo ?. bar`` - - Pros: same spelling as C# and Dart - - Cons: punctuation is ugly; possible conflict with IPython; difficult to - google to find out what it means; difficult to differentiate from ``.`` - when reading quickly -2. ``foo try .bar`` - - Pros: uses an existing keyword - - Cons: might be confusing to add another meaning to ``try`` - -The following spellings are proposed candidates for the ``None``-aware index -access/slicing operator. The punctuation used for this operator ought to -resemble the punctuation used for the ``None``-aware attribute access. - -1. ``foo?['bar']``, ``foo ? ['bar']`` - - Pros: same spelling as C# and Dart - - Cons: punctuation is ugly; possible conflict with IPython; difficult to - google to find out what it means -2. ``foo try ['bar']`` - - Pros: uses an existing keyword; - - Cons: difficult or impossible to implement in Python's LL(1) parser - - Implementation -------------- -Given that the need for ``None``-aware operators is questionable and the -spelling of said operators is almost incendiary, the implementation details for -CPython will be deferred unless and until we have a clearer idea that one (or -more) of the proposed operators will be approved. +The author of this PEP is not competent with grammars or lexers, and given the +contentiousness of this proposal, the implementation details for CPython will be +deferred until we have a clearer idea that one or more of the proposed +enhancements will be approved. ...TBD...