Skip to content

gh-160036: Document tokenize.TokenInfo separately - #160037

Merged
lysnikolaou merged 5 commits into
python:mainfrom
johnslavik:tokenize-tokeninfo-docs
Oct 21, 2026
Merged

gh-160036: Document tokenize.TokenInfo separately#160037
lysnikolaou merged 5 commits into
python:mainfrom
johnslavik:tokenize-tokeninfo-docs

Conversation

@johnslavik

@johnslavik johnslavik commented Oct 21, 2026 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

@johnslavik johnslavik added docs Documentation in the Doc dir sprint needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Oct 21, 2026
@github-project-automation github-project-automation Bot moved this to Todo in Docs PRs Oct 21, 2026
@github-project-automation github-project-automation Bot moved this to Todo in Sprint Oct 21, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Oct 21, 2026
Open
@johnslavik johnslavik changed the title gh-160036: Document tokenize.TokenInfo and its attributes gh-160036: Document tokenize.TokenInfo separately Oct 21, 2026
@johnslavik johnslavik changed the title gh-160036: Document tokenize.TokenInfo separately gh-160036: Document tokenize.TokenInfo separately Oct 21, 2026
@read-the-docs-community

read-the-docs-community Bot commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #35308455 | 📁 Comparing 7ceb205 against main (0d35e3b)

  🔍 Preview build  

37 files changed · + 1 added · ± 36 modified

+ Added

± Modified

@johnslavik

johnslavik commented Oct 21, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

This ported pre-existing documentation doesn't capture any crucial invariants (1-based rows, 0-based columns, exclusive end positions -- to name a few). I'll expand the docs on this.

lysnikolaou
lysnikolaou reviewed Oct 21, 2026
View reviewed changes

@lysnikolaou lysnikolaou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @johnslavik! lgtm in general. left some comments.

Comment thread Doc/library/tokenize.rst Outdated
Comment thread Doc/library/tokenize.rst Outdated
Comment thread Doc/library/tokenize.rst Outdated
Comment thread Doc/library/tokenize.rst Outdated
Comment thread Doc/library/tokenize.rst Outdated
Comment thread Doc/library/tokenize.rst Outdated
Comment thread Doc/library/tokenize.rst
lysnikolaou
lysnikolaou approved these changes Oct 21, 2026
View reviewed changes

@lysnikolaou lysnikolaou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! thanks @johnslavik!

@github-project-automation github-project-automation Bot moved this from Todo to In Progress in Sprint Oct 21, 2026
lysnikolaou
lysnikolaou reviewed Oct 21, 2026
View reviewed changes
Comment thread Doc/library/tokenize.rst Outdated
lysnikolaou
lysnikolaou approved these changes Oct 21, 2026
View reviewed changes

@lysnikolaou lysnikolaou left a comment •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm (again 😆)

@lysnikolaou
lysnikolaou merged commit 134a19e into python:main Oct 21, 2026
30 checks passed
@github-project-automation github-project-automation Bot moved this from Todo to Done in Docs PRs Oct 21, 2026
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Sprint Oct 21, 2026
@miss-islington-app

miss-islington-app Bot commented Oct 21, 2026

Copy link
Copy Markdown

Thanks @johnslavik for the PR, and @lysnikolaou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@bedevere-app

bedevere-app Bot commented Oct 21, 2026

Copy link
Copy Markdown

GH-160112 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Oct 21, 2026
@bedevere-app

bedevere-app Bot commented Oct 21, 2026

Copy link
Copy Markdown

GH-160113 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Oct 21, 2026
lysnikolaou pushed a commit that referenced this pull request Oct 21, 2026
@miss-islington @johnslavik
[3.14] gh-160036: Document tokenize.TokenInfo and its attributes (G…
9c282db 
…H-160037) (#160113)

(cherry picked from commit 134a19e)

Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lysnikolaou lysnikolaou lysnikolaou approved these changes
@pablogsal pablogsal Awaiting requested review from pablogsal

Assignees

No one assigned

Labels

docs Documentation in the Doc dir skip news sprint

Projects

Docs PRs
Status: Done
Sprint
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@johnslavik @lysnikolaou