gh-158633: say that subprocess text mode guesses the child's encoding

[AlKor13]

Documentation only. Closes #158633, and implements what @zooba described on #105312:

I would add a warning that it will guess the encoding of the child process and you'll get garbage if it guesses wrong, and you are always recommended to specify the encoding to use (on all platforms, though you are most likely to get into trouble on Windows).

One .. warning:: in frequently-used-arguments, after the paragraph on binary mode. Popen's section already refers to that one, so it is covered too.

Two things the current text leaves out, and the reason they are worth a warning rather than a note:

• Where the failure appears. When the guess is wrong the output is either silently mojibake, or a UnicodeDecodeError raised while the stream is read — and that traceback comes from inside subprocess, not from the call that omitted encoding=, which is what sends people looking for a bug in the module.

• Windows has more than one default at once. Measured on Windows 11, ANSI code page cp1251, console output code page cp866, so two children of one process need two different decoders:

  subprocess.run([sys.executable, "-c", "print('Тест')"], capture_output=True, text=True).stdout
  # 'Тест'   — the child wrote UTF-8
  
  subprocess.run("echo Тест", shell=True, text=True, stdout=subprocess.PIPE).stdout
  # '’Ґбв'       — the child wrote cp866

  Under UTF-8 mode, which :pep:686 makes the default in 3.15, the second becomes UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 rather than correct text — so the advice holds after 3.15, which is why the warning says so explicitly instead of presenting UTF-8 as the end of the problem.

The Windows paragraph keeps to @zooba's caveat that GetConsoleOutputCP is only right if you know the child uses it: it says a console program is read with that page, and that a Python child can instead be told what to write through PYTHONUTF8 / PYTHONIOENCODING in its env, which is knowledge rather than a guess.

No behaviour change and no new API, so I believe this is skip news; happy to add a blurb if you would rather have one. I have not signed the CLA yet — doing that now, and I will confirm here once it shows.

• Issue: Docs: subprocess text mode does not say that the default encoding is a guess about the child process #158633

[python-cla-bot]

All commit authors signed the Contributor License Agreement.

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #34911658 | 📁 Comparing d1ea9b3 against main (1a85213)

  🔍 Preview build  

1 file changed

± library/subprocess.html

[AlKor13]

CLA signed — the check is green now, and so are lint, Docs, Check EPUB, the HTML-ID check and the docs preview.

Rendered warning, for review without a local build: https://cpython-previews--158634.org.readthedocs.build/en/158634/library/subprocess.html#frequently-used-arguments — the five cross-references in it (UnicodeDecodeError, :pep:686, locale.getpreferredencoding, PYTHONUTF8, PYTHONIOENCODING) all resolve.

Happy to trim it if it reads long for one admonition; the Windows paragraph is the part I would cut first, since the first one carries the advice on its own.