Read about our upcoming Code of Conduct on this issue

This instance will be upgraded to Heptapod 0.28.1 on 2022-01-26 at 16:00 UTC+1 (a few minutes of down time)

Commit 2b92edd4 authored by Gregory Szorc's avatar Gregory Szorc
Browse files

dispatch: force \n for newlines on sys.std* streams (BC)

The sys.std* streams behave differently on Python 3. On Python 3,
these streams are an io.TextIOWrapper that wraps a binary buffer
stored on a .buffer attribute. These TextIOWrapper instances
normalize \n to os.linesep by default. On Windows, this means
that \n is normalized to \r\n. So functions like print() which
have an implicit end='\n' will actually emit \r\n for line endings.

While most parts of Mercurial go through the ui.write() layer to
print output, some code - notably in extensions and hooks - can use
print(). If this code was using print() or otherwise writing to
sys.std* on Windows, Mercurial would emit \r\n.

In reality, pretty much everything on Windows reacts to \n just fine.
Mercurial itself doesn't emit \r\n when going through the ui layer.
Changing the sys.std* streams to not normalize line endings sounds
like a scary change. But I think it is safe. It also makes Mercurial
on Python 3 behave similarly to Python 2, which did not perform \r\n
normalization in print() by default.

.. bc:: sys.{stdout, stderr, stdin} now use \n line endings on Python 3

Differential Revision:
parent 3cbbfd0bfc17
......@@ -10,6 +10,7 @@ from __future__ import absolute_import, print_function
import difflib
import errno
import getopt
import io
import os
import pdb
import re
......@@ -144,7 +145,50 @@ def run():
if pycompat.ispy3:
def initstdio():
# stdio streams on Python 3 are io.TextIOWrapper instances proxying another
# buffer. These streams will normalize \n to \r\n by default. Mercurial's
# preferred mechanism for writing output (ui.write()) uses io.BufferedWriter
# instances, which write to the underlying stdio file descriptor in binary
# mode. ui.write() uses \n for line endings and no line ending normalization
# is attempted through this interface. This "just works," even if the system
# preferred line ending is not \n.
# But some parts of Mercurial (e.g. hooks) can still send data to sys.stdout
# and sys.stderr. They will inherit the line ending normalization settings,
# potentially causing e.g. \r\n to be emitted. Since emitting \n should
# "just work," here we change the sys.* streams to disable line ending
# normalization, ensuring compatibility with our ui type.
# write_through is new in Python 3.7.
kwargs = {
"newline": "\n",
"line_buffering": sys.stdout.line_buffering,
if util.safehasattr(sys.stdout, "write_through"):
kwargs["write_through"] = sys.stdout.write_through
sys.stdout = io.TextIOWrapper(
sys.stdout.buffer, sys.stdout.encoding, sys.stdout.errors, **kwargs
kwargs = {
"newline": "\n",
"line_buffering": sys.stderr.line_buffering,
if util.safehasattr(sys.stderr, "write_through"):
kwargs["write_through"] = sys.stderr.write_through
sys.stderr = io.TextIOWrapper(
sys.stderr.buffer, sys.stderr.encoding, sys.stderr.errors, **kwargs
# No write_through on read-only stream.
sys.stdin = io.TextIOWrapper(
# None is universal newlines mode.
def _silencestdio():
for fp in (sys.stdout, sys.stderr):
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment