Repository navigation
plistlib rejects strings containing control characters #54942
Description
| BPO | 10733 |
|---|---|
| Nosy | @loewis, @ronaldoussoren, @MLModel |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = 'https://github.com/ronaldoussoren'
closed_at = None
created_at = <Date 2010-12-18.19:39:58.777>
labels = ['OS-mac', 'type-bug', 'library']
title = 'plistlib rejects strings containing control characters'
updated_at = <Date 2015-04-23.18:33:14.694>
user = 'https://github.com/MLModel'bugs.python.org fields:
activity = <Date 2015-04-23.18:33:14.694>
actor = 'Behdad.Esfahbod'
assignee = 'ronaldoussoren'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'macOS']
creation = <Date 2010-12-18.19:39:58.777>
creator = 'MLModel'
dependencies = []
files = ['20104']
hgrepos = []
issue_num = 10733
keywords = []
message_count = 7.0
messages = ['124311', '124313', '124325', '124437', '124608', '124625', '241874']
nosy_count = 4.0
nosy_names = ['loewis', 'ronaldoussoren', 'MLModel', 'Behdad.Esfahbod']
pr_nums = []
priority = 'low'
resolution = None
stage = 'test needed'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue10733'
versions = ['Python 2.7', 'Python 3.2', 'Python 3.3']
Activity
MLModel commentedon Dec 18, 2010
plistlib rejects control characters found in XML plists that Apple's 'plutil lint' accepts. I have attached my Terminal preferences as an example. (plistlib accepts the contents of the default Terminal preferences file)
loewis commentedon Dec 18, 2010
This is tricky. It's clearly ill-formed XML, so I'm not sure that this needs to be bug-compatible with Apple's implementation.
MLModel commentedon Dec 18, 2010
I can see where that does make it tricky. (I also tried reading the plist after opening the file as binary, but no luck.) The problem here, of course, is that the only reason for the existence of this library is to read Apple's plist files, however XML-invalid some may be. (It is only a small number of my very many .plist files that have invalid characters -- I just happened to pick one of them to try to access in order to print a simple summary of its contents.) I guess since the plist is read using xml.parsers.expat, there's not much that can be done, and it wouldn't be worth anyone's time to hack around this for plistlib, especially since nearly all .plist files appear to be conforming. Thanks for the clarification.
ronaldoussoren commentedon Dec 21, 2010
I agree with Martin that this is a tricky one.
The file is problematic because it is invalid XML[1], however Apple's tools are perfectly happy to proces the file and as Mitchell notes plistlib exists to interoperate with Apple's plist files.
I'm therefore reopening the issue, but with a low priority. It is unlikely that I'll work on this in the near future though.
Replacing all control characters by entities before trying to parse the Plist XML would likely be the best way forward. A patch (including testcases) would definitely be appreciated.
BTW. I've checked that Apple's Cocoa libraries will read the file, this is not just a bug in the xml1 output formatter of plutil.
[1] It is invalid XML because it contains control characters which are invalid according to the XML specification (<http://www.w3.org/TR/2000/REC-xml-20001006#NT-Char\>).
terryjreedy commentedon Dec 24, 2010
Mitchell: 2.6 is closed to revision except for security issues
MLModel commentedon Dec 25, 2010
Thanks for letting me know (and with a personalized message, yet!). I wasn't paying attention -- i verified that the problem exists in 2.7 and 3.1 and I just dragged 3.1 down to 2.6. Although I've been working furiously in Python for the past six months, I haven't been writing or teaching so I haven't been combing the documentation or testing examples or using obscure or forgotten features, which together are the source of nearly all my bug reports. So I just automatically did what I used to do. I understand the issue and difference; I just didn't realize 2.6 was closed -- I didn't really mean that this should be fixed in anything other than the current or even next release of anything.
On Dec 24, 2010, at 3:36 PM, Terry J. Reedy wrote:
BehdadEsfahbod commentedon Apr 23, 2015
That wouldn't work. Control characters are disallowed in XML's character set, so they are invalid even if input as entities.
Unfortunately this causes a lot of trouble for clients [0], because it means that XML cannot represent the full Unicode repertoire. I'm curious about alternates. Perhaps the expat module can be extended to allow recovering from this if the client chooses to...
[0] eg. fonttools/fonttools#249
ronaldoussoren commentedon Nov 16, 2022
I'm closing this issue: