Skip to content

Deprecate lib2to3 (and 2to3) for future removal #84540

Closed
Closed
Deprecate lib2to3 (and 2to3) for future removal#84540
Labels
3.11only security fixestopic-2to3type-featureA feature request or enhancement

Description

@gpshead
gpshead
opened on Apr 22, 2020
BPO 40360
Nosy @gvanrossum, @gpshead, @vstinner, @carljm, @ambv, @ericsnowcurrently, @hroncok, @corona10, @miss-islington, @tirkarthi, @isidentical, @iritkatriel
PRs
  • bpo-40360: Prepare to deprecate lib2to3. #19645
  • bpo-40360: Deprecate lib2to3 module in light of PEP 617 #19663
  • bpo-40360: Add a What's new entry for lib2to3 pending deprecation #19898
  • bpo-40360: Handle PendingDeprecationWarning in test_lib2to3. #21694
  • [3.9] bpo-40360: Handle PendingDeprecationWarning in test_lib2to3. (GH-21694) #21696
  • [3.9] bpo-40360: Handle PendingDeprecationWarning in test_lib2to3. (GH-21694) #21697
  • bpo-40360: Deprecate the lib2to3 package #28116
  • bpo-40360: [doc] Rephrase deprecation note about lib2to3 #28122
  • [3.10] bpo-40360: [doc] Rephrase deprecation note about lib2to3 (GH-28122) #28127
  • bpo-40360: Make the 2to3 deprecation more obvious. #29064
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = <Date 2021-09-02.16:03:42.312>
created_at = <Date 2020-04-22.04:40:54.230>
labels = ['type-feature', 'expert-2to3', '3.11']
title = 'Deprecate lib2to3 (and 2to3) for future removal'
updated_at = <Date 2021-10-20.22:28:52.390>
user = 'https://github.com/gpshead'

    bugs.python.org fields:

    activity = <Date 2021-10-20.22:28:52.390>
actor = 'iritkatriel'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-02.16:03:42.312>
closer = 'vstinner'
components = ['2to3 (2.x to 3.x conversion tool)']
creation = <Date 2020-04-22.04:40:54.230>
creator = 'gregory.p.smith'
dependencies = []
files = []
hgrepos = []
issue_num = 40360
keywords = ['patch']
message_count = 51.0
messages = ['366973', '367005', '367031', '367051', '367208', '367209', '367230', '367235', '367716', '367726', '367730', '367743', '367744', '367766', '367767', '367770', '367771', '367884', '368077', '368388', '373185', '373198', '373327', '373332', '373334', '373444', '373538', '374348', '374349', '374566', '374572', '374573', '374574', '374611', '374634', '374643', '379014', '400876', '400877', '400879', '400883', '400885', '400892', '400905', '400906', '400923', '400927', '400933', '400937', '404324', '404532']
nosy_count = 14.0
nosy_names = ['gvanrossum', 'gregory.p.smith', 'vstinner', 'carljm', 'lukasz.langa', 'eric.snow', 'davidhalter', 'hroncok', 'corona10', 'miss-islington', 'xtreak', 'BTaskaya', 'Peter Ludemann', 'iritkatriel']
pr_nums = ['19645', '19663', '19898', '21694', '21696', '21697', '28116', '28122', '28127', '29064']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40360'
versions = ['Python 3.11']

    Activity

    1. gpshead commented on Apr 22, 2020

      @gpshead
      gpshead
      on Apr 22, 2020
      MemberAuthor

      Based on the PEP-617 acceptance thread on python-dev, lib2to3 is eventually going to run into trouble parsing modern syntax a few releases from now.

      It would be better off maintained outside of the standard library. It gets used by a lot of things and is generally useful, but would make a lot more sense as a PyPI project than as something only quasi-maintained within the stdlib (it only gained the ability to parse a couple modern syntax features in via bugfix contributions to the stdlib the past month or two... meaning a lot of versions of it out there cannot)

      Black has already forked it.

      goal: PendingDeprecationWarning and documentation as such in 3.9. Move to DeprecationWarning in 3.10 or 3.11 and remove it by ~3.12. Subject to our existing deprecation process guidelines.

    2. gpshead
      added
      3.9 (EOL)end of life
      on Apr 22, 2020
    3. gpshead
      self-assigned this
      on Apr 22, 2020
    4. gpshead
      added
      type-featureA feature request or enhancement
      3.9 (EOL)end of life
      on Apr 22, 2020
    5. gpshead
      self-assigned this
      on Apr 22, 2020
    6. gpshead
      added
      type-featureA feature request or enhancement
      topic-2to3
      on Apr 22, 2020

    7. gvanrossum commented on Apr 22, 2020

      @gvanrossum
      gvanrossum
      on Apr 22, 2020
      Member

      I am in favor of this. We could promote LibCST, which is based on Parso, which uses a forked version of pgen2 (the parser in lib2to3). I believe one of these could switch to a fork of pegen as its parser, so it will be able to handle new PEG based syntax in 3.10+.

      Removal by 3.12 might be feasible.

    8. carljm commented on Apr 22, 2020

      @carljm
      carljm
      on Apr 22, 2020
      Member

      I volunteered in the python-dev thread to write a patch to the docs clarifying future status of lib2to3; happy to include the PendingDeprecationWarning as well.

      Re linking to alternatives, we want to make sure we link to alternatives that are committed to updating to support newer Python versions' syntax. This definitely includes LibCST; I can inquire with the parso maintainer about whether it also includes parso. In future it could also include a third-party-maintained copy of lib2to3, if someone picks that up.

    9. carljm commented on Apr 22, 2020

      @carljm
      carljm
      on Apr 22, 2020
      Member

      I opened a PR. It deprecates the lib2to3 library to discourage future use of it for Python3, but not the 2to3 tool. This of course means that the lib2to3 module will in practice stick around in the stdlib as long as 2to3 is still bundled with Python.

      It seems like the idea in this issue is to deprecate and remove both. I'm not sure what we typically do to deprecate a command-line utility bundled with Python. Given warnings are silent by default, the deprecation warning for lib2to3 won't be visible to users of 2to3. Should I add something to its --help output? Or something more aggressive; an unconditionally-printed warning?

    10. gpshead commented on Apr 24, 2020

      @gpshead
      gpshead
      on Apr 24, 2020
      MemberAuthor

      New changeset 503de71 by Carl Meyer in branch 'master':
      bpo-40360: Deprecate lib2to3 module in light of PEP-617 (GH-19663)
      503de71

    11. gpshead commented on Apr 24, 2020

      @gpshead
      gpshead
      on Apr 24, 2020
      MemberAuthor

      Okay,the pending deprecation is in. Keeping open as a reminder to turn that into a real DeprecationWarning in 3.10 after the 3.9 branch is cut.

      We'll then want to track reminding us to remove it in 3.12.

    12. carljm commented on Apr 24, 2020

      @carljm
      carljm
      on Apr 24, 2020
      Member

      @gregory.p.smith

      What do you think about the question I raised above about how to make this deprecation visible to users of the 2to3 CLI tool, assuming the plan is to remove both?

    13. 48 remaining items

    14. ambv commented on Sep 2, 2021

      @ambv
      ambv
      on Sep 2, 2021
      Contributor

      New changeset f0b63d5 by Łukasz Langa in branch 'main':
      bpo-40360: [doc] Rephrase deprecation note about lib2to3 (GH-28122)
      f0b63d5

    15. miss-islington commented on Sep 2, 2021

      @miss-islington
      miss-islington
      on Sep 2, 2021
      Contributor

      New changeset 559af74 by Miss Islington (bot) in branch '3.10':
      bpo-40360: [doc] Rephrase deprecation note about lib2to3 (GH-28122)
      559af74

    16. vstinner commented on Sep 2, 2021

      @vstinner
      vstinner
      on Sep 2, 2021
      Member

      I close the issue: lib2to3 is now deprecated in Python 3.11. I propose to open a new issue in Python 3.13 or newer to remove it.

    17. vstinner
      closed this as completedon Sep 2, 2021
    18. vstinner
      closed this as completedon Sep 2, 2021

    19. ambv commented on Sep 2, 2021

      @ambv
      ambv
      on Sep 2, 2021
      Contributor

      The "pending" deprecation status of lib2to3 in 3.9 and 3.10 is no worse than a vanilla deprecation in terms of visibility. It will appear just the same when run with pytest or -X dev.

      However, upgrading the deprecation between 3.10.0rc1 and 3.10.0rc2 really was too late. So we'll have it deprecated fully in 3.11 and removed in 3.13. One more year doesn't hurt us much but might be helpful for library maintainers to have more time to move off of the included lib2to3.

      Invoking 2to3 also generates the warning so I guess this can be closed:

      $ ./python.exe Tools/scripts/2to3
/private/tmp/cpy2/Tools/scripts/2to3:3: DeprecationWarning: lib2to3 package is deprecated and may not be able to parse Python 3.10+
  from lib2to3.main import main
At least one file or directory argument required.
Use --help to show usage.

      Thanks for the patches, Gregory, Carl, and Victor! ✨ 🍰 ✨

      Now we just have to remember to actually remove the damn thing in 3.13 😂

    20. ambv commented on Oct 19, 2021

      @ambv
      ambv
      on Oct 19, 2021
      Contributor

      New changeset fdbdf3f by Gregory P. Smith in branch 'main':
      bpo-40360: Make the 2to3 deprecation more obvious. (GH-29064)
      fdbdf3f

    21. iritkatriel commented on Oct 20, 2021

      @iritkatriel
      iritkatriel
      on Oct 20, 2021
      Member

      Created bpo-45544 to close all open issues and list them there.

    22. ezio-melotti
      transferred this issue fromon Apr 10, 2022

    23. johns1c commented on Nov 13, 2023

      @johns1c
      johns1c
      on Nov 13, 2023

      What does this mean for someone who picks up a bit of code that happens to be obsolete and then starts getting warnings about print statements and lots of other stuff?

    24. gvanrossum commented on Nov 14, 2023

      @gvanrossum
      gvanrossum
      on Nov 14, 2023
      Member

      @johns1c -- It means that you should ask for help somewhere else. :-) Seriously, try explaining your problem in our Discourse server at discuss.python.org.

    25. vstinner commented on Nov 14, 2023

      @vstinner
      vstinner
      on Nov 14, 2023
      Member

      For people who still have a Python 2 code base: download/install Python 3.12 and run 2to3 included in this version. Or give a try to 2to3 alternatives.

    26. johns1c commented on Nov 17, 2023

      @johns1c
      johns1c
      on Nov 17, 2023 via email

    27. hlovdal commented on Apr 18, 2025

      @hlovdal
      hlovdal
      on Apr 18, 2025

      2to3 alternatives

      @vstinner Such as?

    28. ambv commented on Apr 18, 2025

      @ambv
      ambv
      on Apr 18, 2025
      Contributor

      Such as modernize.

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.11only security fixestopic-2to3type-featureA feature request or enhancement

      Projects

      No projects

        Milestone

        No milestone

        Relationships

        None yet

        Development

        No branches or pull requests

        Issue actions