Repository navigation
Tkinter: C API changes are needed for Tcl 8.7 and 9.0 value types #103194
Description
Activity
Christopher, thank you for posting and offering your assistence. My understanding is that 8.7 and 9.0 have been in alpha since 2020. Do you have any updated information on when there might be a beta, let along a final release? IOW, how urgent are the needed changes?
Our _/tkinter maintainer is currently constrained in his python time. _tkinter patches should be welcome. I just hope he will be available to review any. Our devguide should give you any info you need.
I maintain IDLE. From what I have read, the main feature of interest for me in 8.7 is change to Text and in 9.0 full unicode support. Are these in the alphas? If 9.0 does unicode support that works with tkinter, that will be the version I would want out windows and mac installers to install. I wonder though if people would push for 8.7. Should that be an issue?
tkinter requires 8.6 now, so supporting 8.5 is not an issue, but 8.6 will be on Linux for possibly a decade or more. Tip 568 suggests replacing
...GetByteArray....with...GetBytes..., but only in 8.7+. So it would seem that we will need separate code for 8.6 and 8.7/9.0. Correct?My understanding is that 8.7 and 9.0 have been in alpha since 2020. Do you have any updated information on when there might be a beta, let along a final release? IOW, how urgent are the needed changes?
It sounded like there has been an increased push to get 8.7 and 9.0 beta releases out since last year, but I do not know how imminent these really are. Some areas of improvement do not seem completely settled, e.g. Unicode support/error handling, replacing
intwithsize_t/ssize_tin APIs, and maybe other things that do not matter as much to Tkinter. But I believe the changes I brought up here will still apply even if 8.7/9.0 releases are years away, and are likely a prerequisite to any work needed for e.g. TIP 474.Our _/tkinter maintainer is currently constrained in his python time. _tkinter patches should be welcome. I just hope he will be available to review any. Our devguide should give you any info you need.
Thanks, I will look into creating a pull request for these changes.
I maintain IDLE. From what I have read, the main feature of interest for me in 8.7 is change to Text and in 9.0 full unicode support. Are these in the alphas? If 9.0 does unicode support that works with tkinter, that will be the version I would want out windows and mac installers to install. I wonder though if people would push for 8.7. Should that be an issue?
The last alpha releases were a couple years ago, so even if they contain these features, I would think there are noticeable differences by now in the development branches. I am not familiar enough with how 8.7 and 9.0 differ (on things like Unicode support) in ways that matter to Tkinter to summarize here. My understanding is that 8.7 is meant as a choice and/or more gradual transition for 8.6 users that really do not want C API or other breaking changes from 9.0. But even some Tcl core team members raise the idea of abandoning 8.7 in favor of 9.0: https://sourceforge.net/p/tcl/mailman/tcl-core/thread/0fdf01d929c6$9c027350$d40759f0$@yahoo.com/
tkinter requires 8.6 now, so supporting 8.5 is not an issue, but 8.6 will be on Linux for possibly a decade or more.
Is the 8.6 requirement already documented somewhere? I find documentation/comments which still mention 8.5.12 as the minimum.
Tip 568 suggests replacing
...GetByteArray....with...GetBytes..., but only in 8.7+. So it would seem that we will need separate code for 8.6 and 8.7/9.0. Correct?I recently got confirmation that for the use case of wrappers like Tkinter, which only want to use
Tcl_GetByteArrayFromObj()for values that already have "bytearray" type, they can continue usingTcl_GetByteArrayFromObj()for 8.6 compatibility as it will behave no differently fromTcl_GetBytesFromObj(): https://core.tcl-lang.org/tcl/info/3bb3bcf2da5b. The guidance in TIP 568 seems meant for other use cases which want to convert arbitrary Tcl values to "bytearray".There was an idea to require 8.6 instead of 8.5, but yes, we stopped at 8.5.12 in issue 91152.
even some Tcl core team members raise the idea of abandoning 8.7 in favor of 9.0
I think that this would be a lot easier for Python.
- added a commit that references this issue
on Apr 16, 2023 I have a draft of the changes: main...chrstphrchvz:cpython:patch-103194. I have verified that they work under Tcl 8.6.13. The boolean value case in
FromObj()does not appear to exercised by Tkinter’s existing tests, but here is a script which I believe exercises all relevant codepaths:import tkinter i = tkinter.Tcl() # retrieve bytearray i.eval(""" set x [binary decode hex 40] # manually verify value type: puts [::tcl::unsupported::representation $x] """) assert i.getvar('x') == b'@' # retrieve booleanString i.eval(""" # start with pure string set x yes puts [::tcl::unsupported::representation $x] # this command causes x to have booleanString type: expr {!$x} puts [::tcl::unsupported::representation $x] """) assert i.getvar('x') == True # retrieve int i.eval('set x [expr 1+1]') assert i.getvar('x') == 2 # retrieve wideInt (or int if wideInt is not present) i.eval(""" set x [expr 1<<32] puts [::tcl::unsupported::representation $x] """) assert i.getvar('x') == (1 << 32)
I think the only thing left to settle for this set of changes is whether
FromObj()should still handle obsolete registered types in Tcl 8.7--specifically "int" on 32-bit and Tcl 8.4 "boolean"--even though Tcl itself will never create values with obsolete types; see latest comment on https://core.tcl-lang.org/tcl/info/3bb3bcf2da5b.Also, at least one other person appears interested in having Tkinter work with Tk 8.7: one of the Tk Aqua developers, who has been working on extensive improvements which will only appear in 8.7, attempted to use Tkinter with 8.7 but encountered other issues: https://core.tcl-lang.org/tk/info/6b49149b4e7c.
one of the Tk Aqua developers … attempted to use Tkinter with 8.7 but encountered other issues: https://core.tcl-lang.org/tk/info/6b49149b4e7c.
That issue seems to indicate another necessary change:
- TIP 622: Under Tcl 8.7, commands can return values with the unregistered "utf32string" type. (The current "string" type still exists for compatibility, but in Tcl 9.0 it will be dropped, and "utf32string" will be renamed to "string".)
FromObj()likely needs to recognize this type, or else users will find they have to pass the results of various commands throughstr()when there was no need to under Tcl 8.6.
- TIP 622: Under Tcl 8.7, commands can return values with the unregistered "utf32string" type. (The current "string" type still exists for compatibility, but in Tcl 9.0 it will be dropped, and "utf32string" will be renamed to "string".)
A separate report for an attempt to build Tkinter against Tcl 8.7 (https://core.tcl-lang.org/tcl/info/ff255adc4cb5) indicates yet another necessary change:
- TIP 538: Tcl 8.7 can be built to use separately-built libtommath, and no longer recommends building with the bundled libtommath. tclTomMath.h requires tommath.h, which Tcl will not install a copy of even when the bundled libtommath is used; either tommath.h must be obtained separately, or
TCL_NO_TOMMATH_Hmust be defined (although a few changes are currently needed for it to work: https://core.tcl-lang.org/tcl/info/2a5cb49733ff).
Package managers presumably would want Tkinter to build against separately-built Tcl/Tk (and in turn separately-built libtommath once they update to 8.7). But given that nothing else in CPython uses libtommath, how much would the official Python binary distributions prefer that Tcl continue building its bundled libtommath?
Edit: I have created a separate issue for this: #103839. It may make more sense to address that issue before making the other changes on the issue here.
- TIP 538: Tcl 8.7 can be built to use separately-built libtommath, and no longer recommends building with the bundled libtommath. tclTomMath.h requires tommath.h, which Tcl will not install a copy of even when the bundled libtommath is used; either tommath.h must be obtained separately, or
- added a commit that references this issue
on Apr 22, 2023 - TIP 622: Under Tcl 8.7, commands can return values with the unregistered "utf32string" type.
Tcl has agreed to register "utf32string": https://core.tcl-lang.org/tcl/vdiff/?from=27d3b99343&to=9883f837fc
However it does seem like Tkinter code should still use
str()where appropriate, rather than assume that functions relying onFromObj()won’t return a_tkinter.Tcl_Objinstead of a Python string.- added 4 commits that reference this issue
on Apr 24, 2023 More C API changes are needed for
Tcl_Size. I have opened a separate issue for that (gh-112672), so I am narrowing the subject of this issue slightly.- changed the title
[-]Tkinter: C API changes are needed for Tcl 8.7 and 9.0[/-][+]Tkinter: C API changes are needed for Tcl 8.7 and 9.0 value types[/+]on Dec 3, 2023 - added 3 commits that reference this issue
on May 31, 2024 Thank you for your great investigations @chrstphrchvz. I agree with all of your changes.
Bug report
I recently made changes to Tcl.pm (a Tcl wrapper for Perl) to support Tcl 8.7/9.0 (gisle/tcl.pm#42). While I was doing so, I briefly looked at _tkinter.c for comparison (particularly
AsObj()andFromObj()). Although I have not attempted to run Tkinter with Tcl 8.7/9.0, I do notice a few issues in the code which would need to be addressed first.Tcl 9.0 currently renames the Tcl 8.5 "booleanString" value type to "boolean", which will break the existing check in
FromObj(). Tcl has suggested an idiom for retrieving theTcl_ObjType *of unregistered types like "booleanString", regardless of Tcl version (see https://core.tcl-lang.org/tcl/info/3bb3bcf2da5b); although I would think that approach entails initializingv->BooleanTypefromTkapp_New()rather than lazily initializingtkapp->BooleanTypefromFromObj().TIP 568: As of Tcl 9.0, "bytearray" is unregistered; the suggested idiom can be used instead.
TIP 484: As of Tcl 8.7, on platforms with 32-bit
long, there are no longer separate 32-bit "int" and 64-bit "wideInt" value types; there is only a single 64-bit type. And as of Tcl 9.0, the "int" type is unregistered. It is possible to reliably get theTcl_ObjType *of "int" from a value created usingTcl_NewIntObj(0)and of "wideInt" (if present) using e.g.Tcl_NewWideIntObj((Tcl_WideInt)(1) << 32). However it would not be appropriate forFromObj()to continue usingTcl_GetLongFromObj()whenlongis 32-bit, as that would retrieve an overflowed result for values within (LONG_MAX,ULONG_MAX] rather than forceFromObj()to try again withTcl_GetWideIntFromObj()(viafromWideIntObj()); an alternative would be to always useTcl_GetWideIntFromObj()for both "int" and "wideInt" values.I am not familiar with contributing to Python, but I may be able to propose fixes for the above issues if no one else already has or intends to. I believe the fixes for the above issues are backward compatible with Tcl 8.5 and 8.6, and so can be implemented and tested without other changes that are likely also needed (such as for Tcl-syntax APIs) before Tkinter is usable with
Tcl 8.7/9.0 and Tk 8.7 (which will be compatible with both Tcl 8.7 and 9.0)Tcl and Tk 8.7/9.0.Linked PRs