Repository navigation
imaplib: incorrect quoting in commands #40038
Description
| BPO | 917120 |
|---|---|
| Nosy | @warsaw, @bitdancer, @soltysh, @xtsimpouris |
| Files |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2004-03-16.06:36:58.000>
labels = ['easy', 'type-bug', 'library', 'expert-email']
title = 'imaplib: incorrect quoting in commands'
updated_at = <Date 2021-12-18.16:46:27.029>
user = 'https://bugs.python.org/anadelonbrin'bugs.python.org fields:
activity = <Date 2021-12-18.16:46:27.029>
actor = 'xtsimpouris'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'email']
creation = <Date 2004-03-16.06:36:58.000>
creator = 'anadelonbrin'
dependencies = []
files = ['18367']
hgrepos = []
issue_num = 917120
keywords = ['patch', 'easy']
message_count = 11.0
messages = ['20244', '20245', '87655', '87656', '112741', '112976', '114330', '181950', '181957', '181977', '408860']
nosy_count = 10.0
nosy_names = ['barry', 'anadelonbrin', 'dmbaggett', 'r.david.murray', 'meatballhat', 'maciej.szulik', 'Mauro.Cicognini', 'dveeden', 'bjshan', 'xtsimpouris']
pr_nums = []
priority = 'low'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue917120'
versions = ['Python 3.3']Linked PRs
- gh-40038: Quote imaplib command arguments when necessary #152703
- [3.15] gh-40038: Quote imaplib command arguments when necessary (GH-152703) #153066
- [3.14] gh-40038: Quote imaplib command arguments when necessary (GH-152703) #153069
- [3.13] gh-40038: Quote imaplib command arguments when necessary (GH-152703) #153071
Activity
anadelonbrin commentedon Mar 16, 2004
imaplib incorrectly chooses to quote some arguments.
In particular, doing "UID FETCH # BODY.PEEK[]" results
in the BODY.PEEK[] being quoted, and it should not
(according to the RFC), which means the command fails.
This is demonstrated below. It's possible (and
likely) that other UID FETCH arguments are incorrectly
quoted.
This occurs with anon cvs python of 16/3/04, and 2.3.3.
Windows XP SP1.
I'm happy to provide more info if required, just let me
know. I could try and work up a patch, but it would be
better from someone really familiar with imaplib so
that I don't screw up legitimate quoting.
anadelonbrin commentedon Mar 16, 2004
Logged In: YES
user_id=552329
Sorry, I missed the bit in the docs that points out that
stuff is always quoted and that using () avoids it. Still,
it does seem that imaplib would be doing it's job better if
it followed correct quoting, rather than always quoting. It
would certainly be easier to use for people familiar with
IMAP, but unfamiliar with imaplib.
dmbaggett commentedon May 12, 2009
I'm not sure this causes the behavior reported here, but I believe there
really is a bug in imaplib.
In particular, it seems wrong to me that this line:
has \w in it. Should that be \s?
I found this when I noticed that SELECT commands on mailboxes with
spaces in their names failed.
dmbaggett commentedon May 12, 2009
OK, I missed the initial caret in the regex. The mustquote regex is
listing everything that needn't be quoted, and then negating. I still
think it's wrong, though. According to BNF given in the Formal Syntax
section of RFC 3501, you must must quote atom-specials, which are
defined thus:
atom-specials = "(" / ")" / "{" / SP / CTL / list-wildcards /
quoted-specials / resp-specials
list-wildcards = "%" / "*"
quoted-specials = DQUOTE / "\"
resp-specials = "]"
So I think this regex should do it:
Changing status to bug.
meatballhat commentedon Aug 4, 2010
I'm attaching a patch which does exactly what dmbaggett recommended w.r.t. the mustquote regex. All current tests pass, but I'm not sure if the current tests even cover this code (how is coverage measured in the stdlib tests?)
On a related note, the
_checkquotemethod which uses themustquoteregex is dead code in Python 3.2+, AFAICT.8 remaining items
serhiy-storchaka commentedon Dec 18, 2024
The behavior was weird almost from beginning, when
_checkquote()method was introduced in 8c06221. You can pass arguments either unquoted or quoted, and the heuristic is used to determine if they need quoting. It is ugly design, the API should always take unquoted arguments and quote them if needed. But this was here so long, that there must be much user code which passes quoted values.This got worse in Python 3. The autoquoting was removed in fb5faf0 (except for the password), so users now are forced to pass quoted strings. Since in most cases quoting is not needed, and the documentation does not match the current behavior, users rarely do this, and their code works only until the quoting is needed, then it fails.
We need to restore autoquoting feature. #6395 does this. We should keep the heuristic for compatibility with the code which passes quoted strings. In long long perspective we can deprecate this and always apply proper quoting, but it will be breaking change. We should wait many years until all users get used to passing non-quoted strings. Meanwhile, we can add support of lists/tuples/sets as arguments instead os strings enclosed in parentheses.
What makes this issue more complicated, and what the original report was about, is that different arguments have different quoting rules. For example, "*" should be quoted in the first LIST argument, but not quoted in the second LIST argument. #6395 does not support this. I am working on an alternative solution.
serhiy-storchaka commentedon Dec 19, 2024
On other hand, we cannot apply auto-quoting even with loosened condition to all arguments, because some arguments that contain spaces should not be quoted. There is an example in RFC 3501:
It corresponds to Python code
Well, in this case
_checkquote()would omit quoting because the argument is enclosed in parentheses, but the following example also satisfies the formal syntax:Python code:
Now the argument is not enclosed in parentheses, but it contains spaces, so even the most lenient variant of
_checkquote()would quote it.pythongh-40038: Quote imaplib command arguments when necessary
pythongh-40038: Quote imaplib command arguments when necessary
gh-40038: Quote imaplib command arguments when necessary (GH-152703)
[3.15] gh-40038: Quote imaplib command arguments when necessary (GH-1…
[3.14] gh-40038: Quote imaplib command arguments when necessary (GH-1…
[3.13] gh-40038: Quote imaplib command arguments when necessary (GH-1…
serhiy-storchaka commentedon Jul 5, 2026
Argument quoting has been restored in GH-152703, reimplemented per the RFC 3501 grammar: arguments that need quoting are escaped and quoted, flags, sequence sets and list wildcards are left intact, and already quoted arguments are passed through for backward compatibility.