diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index 2c65098c5ec..9eeca3409cd 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -700,6 +700,8 @@ peps/pep-0819.rst @emmatyping peps/pep-0820.rst @encukou peps/pep-0821.rst @JelleZijlstra peps/pep-0822.rst @methane +# ... +peps/pep-0824.rst @gvanrossum peps/pep-0825.rst @warsaw @dstufft peps/pep-0826.rst @savannahostrowski peps/pep-0827.rst @1st1 diff --git a/peps/pep-0824.rst b/peps/pep-0824.rst new file mode 100644 index 00000000000..fa5b1ac1a02 --- /dev/null +++ b/peps/pep-0824.rst @@ -0,0 +1,585 @@ +PEP: 824 +Title: None-coalescing operators +Author: Marc Mueller +Sponsor: Guido van Rossum +Discussions-To: Pending +Status: Draft +Type: Standards Track +Created: 24-Jun-2026 +Python-Version: 3.16 + + +Abstract +======== + +This PEP proposes adding two new operators. + +* The "``None``-coalescing" operator ``??`` +* The "``None``-coalescing assignment" operator ``??=`` + +The "``None``-coalescing" operator evaluates the left hand side, +checks if it is ``not None`` and, if so, returns the result. If the +value is ``None``, the right hand side is evaluated and returned. + +The "``None``-coalescing assignment" operator will only assign the right +hand side to the left hand side if the left hand side evaluates to ``None``. + +They are roughly equivalent to: + +.. code-block:: python + + # a ?? b + _t if ((_t := a) is not None) else b + + # a ??= b + if a is None: + a = b + +See the `Specification`_ section for more details. + + +Motivation +========== + +First officially proposed over ten years ago in (the now deferred) :pep:`505`, +the idea to add ``None``-coalescing operators has been along for some +time now, discussed at length in numerous threads, most recently in +[#discuss_revisit_505]_ and [#discuss_none_coalescing]_. +This PEP aims to capture the current state of +discussion and proposes a specification for addition to the Python +language. In contrast to :pep:`505`, it will only focus on the two +``None``-coalescing operators. See the `Deferred Ideas`_ section for +more details. + +``None``-coalescing operators are not a new invention. Several other +modern programming languages have so called "``null`` coalescing" +operators, including TypeScript [#ts]_, ECMAScript (a.k.a. JavaScript) +[#js_1]_ [#js_2]_, C# [#csharp]_, Dart [#dart_1]_ [#dart_2]_, +Swift [#swift]_, Kotlin [#kotlin]_, PHP [#php_1]_ [#php_2]_ and more. + +The general idea is to provide a comparison operator, similar to ``or``, +which instead of truthiness, checks for ``None`` values. + +Explicit checks for ``None`` +---------------------------- + +While some data types support inherit default values, e.g. an empty +string or collection literal, for others the default value could +itself be a valid value, e.g. ``0``. Furthermore, custom types +often do not have default values either. To denote the absence of +a value, it is therefore common in Python to use the builtin sentinel +``None``. Any code working with such functions either does some kind of +validation to make sure a valid value is actual returned, maybe +returning early or rasing an exception if it is not, or provides +a default / fallback value. To do this, it is common to use +``is not None`` checks. + +.. code-block:: python + + class User: + name: str | None + def get_age(self) -> int | None: ... + + def show_user_age(user: User): + if user.get_age() is not None: + age = user.get_age() + else: + age = "unknown" + print(f"The user age is {age}") + +Though the intent is clear, it is quite verbose. Additionally, it is +not uncommon to repeat the value expression, like in the example above, +instead of assigning it to a temporary variable first. This could cause +problems if the value expression has side effects which would be +executed twice now. + +It is possible to write this in a more concise form using an +assignment expression. However, it is arguably a bit more difficult +to see what is going on here at first glance. + +.. code-block:: python + + def show_user_age(user: User): + age = (val := user.get_age()) if val is not None else "unknown" + print(f"The user age is {age}") + +An issue both options have in common is that there is not a single +agreed upon way to write these statements / expressions. Instead of +putting the value first, some might prefer to invert the check to use +``is None`` instead. Readers need to constantly be aware of this, +increasing the mental load. + +Using the "``None``-coalescing" operator ``??`` instead, helps to keep +the expression short and predictable while still clearly communicating +the intent. + +:: + + def show_user_age(user: User): + age = user.get_age() ?? "unknown" + print(f"The user age is {age}") + +Overwrite ``None`` values +------------------------- + +Sometimes it might be necessary to assign a fallback value inside an +object. To do so, the expression is usually written twice. Once for +the ``is None`` check, and again for the assignment. + +.. code-block:: python + + def set_user_name(user: User): + if user.name is None: + user.name = "unknown" + +Using the "``None``-coalesce assignment" operator ``??=`` helps to +avoid repeating the expression. Especially for more complex once, +this will make it easier to read and write. + +:: + + def set_user_name(user: User): + user.name ??= "unknown" + +Defaults for function arguments +------------------------------- + +Function argument defaults are evaluated in the parent scope. +That is a common issue in cases where the default value is a +mutable object or depends on the function context itself. +In these cases, a typical solution is to allow ``None`` as argument +and assign the fallback value inside the function itself. + +.. code-block:: python + + def show_user_name(user: User | None): + if user is None: + user = create_default_user() + print(f"The user name is {user.name}") + +This could be rewritten as: + +:: + + def show_user_name(user: User | None): + user ??= create_default_user() + print(f"The user name is {user.name}") + + +Specification +============= + +The ``None``-coalescing operator +-------------------------------- + +The ``??`` operator is added. It first evaluates the left hand side. +The result is cached, so that the expression is not evaluated again. +If the value is ``not None``, it is returned. If it is ``None``, the +right hand side expression is evaluated and returned instead. + +.. code-block:: python + + # a ?? b + _t if ((_t := a) is not None) else b + +Precedence +********** + +The precedence of ``??`` will be lower than ``or`` but higher than +conditional expressions. Parentheses can be added as necessary to +modify the precedence in individual expressions. A few examples of +how implicit parentheses would be placed: + +:: + + # "" or None ?? 2 + ("" or None) ?? 2 + + # "Hello" if None ?? True else 0 + "Hello" if (None ?? True) else 0 + +AST changes +*********** + +A new ``Coalesce`` operator is added to ``boolop`` for use in ``BoolOp`` +nodes. + +:: + + expr = BoolOp(boolop op, expr* values) + | ... + + boolop = And | Or | Coalesce + +Grammar changes +*************** + +A new ``??`` token is added, as well as a new ``coalesce`` rule. Every +rule which previously referenced the ``disjunction`` rule is updated +to refer to the ``coalesce`` rule instead. + +.. code-block:: PEG + + coalesce: + | disjunction ('??' disjunction)+ + | disjunction + + disjunction: + | conjunction ('or' conjunction)+ + | conjunction + + +The ``None``-coalescing assignment operator +------------------------------------------- + +The ``??=`` operator is added. It performs a conditional assignment. +As such it will first evaluate the left hand side and check that the +value is ``None`` and only then assign the result from the right hand +side. If the first value is ``not None``, the assignment is skipped. + +.. code-block:: python + + # a ??= b + if a is None: + a = b + +AST changes +*********** + +A new ``CoalesceAssign`` AST node is added. Similar to ``AugAssign``, +it stores a ``target`` and ``value`` expression. + +:: + + stmt = ... + | AugAssign(expr target, operator op, expr value) + ... + | CoalesceAssign(expr target, expr value) + +Grammar changes +*************** + +A new ``??=`` token is added. Additionally, the ``assignment`` rule +is extended to include the "``None``-coalesce assignment". + +.. code-block:: PEG + + assignment: + | NAME ':' expression ['=' annotated_rhs] + | ('(' single_target ')' + | single_subscript_attribute_target) ':' expression ['=' annotated_rhs] + | (star_targets '=')+ annotated_rhs !'=' [TYPE_COMMENT] + | single_target augassign ~ annotated_rhs + | single_target '??=' ~ annotated_rhs + + +Backwards Compatibility +======================= + +The ``None``-coalescing operators are **opt-in**. Existing programs will +continue to run as is. So far code which used either ``??`` or ``??=`` +raised a ``SyntaxError``. + + +Security Implications +===================== + +There are no new security implications from this proposal. + + +How to Teach This +================= + +In a practical sense it might be helpful to think of the "``None``-coalescing" +operator ``??`` as a special case for the conditional ``or`` +operator, with the caveat that ``??`` checks for ``is not None`` instead +of truthiness. As such it makes sense to include ``??`` when teaching +the other conditional operators ``and`` and ``or``. + +While ``None`` is a convenient way to signal the absence of a value, +it is imperative to point out that it is not always the best option. +E.g. using ``""`` or an empty collection as default where appropriate, +can eliminate unnecessary code. + +.. code-block:: python + + from dataclasses import dataclass, field + + @dataclass + class User: + phone_numbers: list[str] = field(default_factory=list) + + def show_user_phone_numbers(user: User): + for number in user.phone_numbers: + print(f"Phone number {number}") + +Though, as explained earlier in the `Explicit checks for None`_ section, +this is not always possible in which case ``??`` is good option. + +The "``None``-coalescing assignment" operator ``??=`` can be best +thought of as a conditional assignment operator. As it is closely +related to ``??``, explaining these together would make sense. +Though it looks related to binary assignment operators like ``+=`` +as well, it is worth pointing out the distinction between these. +Since ``??=`` is a **conditional** assignment operator, the right +hand side will be skipped entirely in some case, while ``+=`` always +evaluates both sides. + +Reading expressions +------------------- + +Reading expressions out loud is always lossy. This PEP does not +intent to define an unambiguous way of speaking these operators. +The following is therefore merely meant as a suggestion. + +``None``-coalescing operator +**************************** + +:: + + user.get_age() ?? "unknown" + ++--------------------------+-----------------------------------+ +| Pattern | Example | ++==========================+===================================+ +| "... or ... if None" | "call user dot get_age ``or`` | +| | unknown ``if None``" | ++--------------------------+-----------------------------------+ +| "... coalesce with ..." | "call user dot get_age | +| | ``coalesce with`` unknown" | ++--------------------------+-----------------------------------+ + +``None``-coalescing assignment operator +*************************************** + +:: + + user.name ??= "unknown" + ++-----------------------------+-----------------------------------+ +| Pattern | Example | ++=============================+===================================+ +| "if ... is None assign ..." | "``if`` user dot name ``is None`` | +| | ``assign`` unknown" | ++-----------------------------+-----------------------------------+ +| "assign ... to ... if None" | "``assign`` unknown ``to`` user | +| | dot name ``if None``" | ++-----------------------------+-----------------------------------+ + + +Reference Implementation +======================== + +A reference implementation is available at +https://github.com/cdce8p/cpython/tree/pep824-none-coalescing-operators. +A online demo can be tested at https://pep823-and-pep824-demo.pages.dev/. + + +Deferred Ideas +============== + +``None``-aware access operators +------------------------------- + +:pep:`505` also suggest the addition of the "``None``-aware access" +operators ``?.`` and ``?[ ]``. As the "``None``-coalescing" operators +have their own use cases, the "``None``-aware access" operators were +moved into a separate document, see PEP-823. Both proposals can be +adopted independently of one another. + + +Rejected Ideas +============== + +Add new (soft-) keyword +----------------------- + +Python does have a history of preferring keywords over symbols. For +example, though a lot of languages use ``&&`` and ``||`` as conditional +operators, Python uses ``and`` and ``or`` respectively. As such it was +suggested to use a new (soft-) keyword, e.g. ``otherwise``, instead of +``??``. + +While the keywords ``and`` and ``or`` help avoid ambiguity with the +binary operators ``&`` and ``|``, there is not a corresponding binary +operator for ``??``. Furthermore, both keywords are well established +in spoken and written language and therefore immediately obvious to the +reader. Not to mention they are also quite short with just two and three +characters. + +In comparison a new (soft-) keyword would likely not have the same +benefits. There is no established short name for it, so while ideas +like ``otherwise`` could be added, they do not convey an inherit +meaning and for this reason do not provide an immediate advantage. +In contrast, the ``??`` operator is well known in other major +programming languages. + +Lastly, using a (soft-) keyword for the "``None``-coalescing assignment" +operator poses additional questions and readability concerns. + +:: + + a = otherwise b + +Add ``??`` as a binary operator +------------------------------- + +:pep:`505` originally suggested to add ``??`` as another binary operator. +As such it would have bound more tightly than the proposed +`specification `_. + +Though this would have worked fine, it would have suggested that ``??`` +is similar to other binary operators like ``+`` or ``**``. This is not +the case. While binary operators first evaluate the left **and** right +hand side before performing the operation, for ``??`` only the left hand +side is evaluated if the values is ``not None``. The +"``None``-coalescing" operator is much more closely related to the +conditional operators ``or`` and ``and`` which also short-circuit the +expression for truthy and falsy values respectively. + +Furthermore, setting the precedence between ``or`` and conditional +expressions matches other languages which have implemented the operator, +like JS [#js_precedence]_ and C# [#csharp_precedence]_. + +Add ``??=`` as ``AugAssign`` +---------------------------- + +:pep:`505` also suggested to add ``??=`` as an ``AugAssign`` node. + +So far ``AugAssign`` is only used for binary operators, including +``??=`` which is a conditional assignment operator would therefore +be confusing. Furthermore, ``AugAssign`` statements always evaluate +the left **and** right hand side, without any short-circuiting. +This is a major difference compared to ``None``-coalescing assignments. + + +Common objections +================= + +Just use a conditional expression +--------------------------------- + +The "``None``-coalescing" operators can be considered syntactic sugar +for existing conditional expressions and statements. As such some +questioned whether they would add anything meaningful to the +language as a whole. + +As shown in the `Motivation`_ section, there are clear benefits to +using the "``None``-coalescing" operators. To summarize them again: + +- They help avoid repeating the variable expression or having to + introduce a temporary variable. + +- Clear control flow, no more ``if ... is not None else ...`` and the + inverse ``if ... is None else ...`` in the same code blocks, + reducing the mental load while reading code. + +- More concise while also being more explicit. + +Proliferation of ``None`` in code bases +--------------------------------------- + +One of the reasons why :pep:`505` stalled was that some expressed their +concern how "``None``-coalescing" and "``None``-aware" operators will affect +the code written by developers. If it is easier to work with ``None`` +values, this will encourage developers to use them more. They believe that +e.g. returning an optional ``None`` value from a function is usually an +anti-pattern. In an ideal world the use of ``None`` would be limited as +much as possible, for example with early data validation. + +It is certainly true that new language features affect how the language +as a whole develops. Therefore any changes should be considered carefully. +However, just because ``None`` represents an anti-pattern for some, has +not prevented the community as a whole from using it extensively. Rather +the lack of "``None``-coalescing" operators has stopped developers from +writing concise expressions and instead often leads to more complex code +which is more difficult to read than necessary, see the `Motivation`_ +section for more details. + +Furthermore, as also reiterated in the `How to Teach This`_ section, +though ``None`` is not always the best option if appropriate other +default values exist, there are still many cases where it is a good +option. + +``None`` is not special enough +------------------------------ + +Some mentioned that ``None`` is not special enough to warrant dedicated +operators. + +"``None``-coalescing" operators have been added to a number of other +modern programming languages. Furthermore, adding ``??`` and ``??=`` +is something which was suggested numerous times since :pep:`505` was +first proposed over ten years ago. + +In Python ``None`` is frequently used to indicate the absence of +something better or a missing value. As such it is common to look +specifically for ``None`` values, for example, to provide a default +or fallback value. + +Use custom sentinels instead of ``None`` +---------------------------------------- + +For Python 3.15 :pep:`661` added the option to define custom sentinels +using ``sentinel(...)``. This addressed an issue in cases where ``None`` +itself is a valid value and thus could not be used as sentinel itself. + +In general though, ``None`` will still be preferred if a sentinel is +needed, simply because it already exists for that exact purpose and +is easier to use. + +Late-bound function argument defaults +------------------------------------- + +Some suggested :pep:`671`, currently in draft and last updated 2022, +might be a better solution for the problem describe in +`Defaults for function arguments`_. While it arguably could be helpful +in some cases, using the syntax suggested in :pep:`671` here just +shifts the responsibility upstream because the function signature itself +would need to be changed. Instead of ``user: User | None`` as argument, +it would be ``user: User => create_default_user()``. It would be up to +the caller now to make sure ``None`` is never passed to the function +and the argument is omitted instead. + +:pep:`671` cannot help though with other use cases like overwriting +``None`` values inside an object, as shown in `Overwrite None values`_. + + +Footnotes +========= + +.. [#discuss_revisit_505] discuss.python.org: Revisiting PEP 505 - None-aware operators + (https://discuss.python.org/t/revisiting-pep-505-none-aware-operators/74568) +.. [#discuss_none_coalescing] discuss.python.org: None-coalescing operator and null-coalescing assignment + (https://discuss.python.org/t/none-coalescing-operator-and-null-coalescing-assignment/107510) +.. [#ts] TypeScript: Nullish Coalescing + (https://www.typescriptlang.org/docs/handbook/release-notes/typescript-3-7.html#nullish-coalescing) +.. [#js_1] JavaScript: Nullish coalescing operator (??) + (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing) +.. [#js_2] JavaScript: Nullish coalescing assignment (??=) + (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Nullish_coalescing_assignment) +.. [#js_precedence] JavaScript: Operator precedence + (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Operator_precedence#table) +.. [#csharp] C# Reference: ?? and ??= operators + (https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/operators/null-coalescing-operator) +.. [#csharp_precedence] C# Reference: Operator precedence + (https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/operators/#operator-precedence) +.. [#dart_1] Dart: Conditional expressions + (https://dart.dev/language/operators#conditional-expressions) +.. [#dart_2] Dart: Assignment operators + (https://dart.dev/language/operators#assignment-operators) +.. [#swift] Swift: Nil-Coalescing Operator + (https://docs.swift.org/swift-book/documentation/the-swift-programming-language/basicoperators/#Nil-Coalescing-Operator) +.. [#kotlin] Kotlin: Elvis operator + (https://kotlinlang.org/docs/null-safety.html#elvis-operator) +.. [#php_1] PHP: Null Coalesce Operator + (https://wiki.php.net/rfc/isset_ternary) +.. [#php_2] PHP: Null Coalescing Assignment Operator + (https://wiki.php.net/rfc/null_coalesce_equal_operator) + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive.