Skip to content
Snippets Groups Projects
Commit 87bb6b76 authored by Katsunori FUJIWARA's avatar Katsunori FUJIWARA
Browse files

minirst: use unicode string as intermediate form for replacement

# this change redones part of 521c8e0c93bf, backed out by 0ad0ebe67815

Some character encodings use ASCII characters other than
control/alphabet/digit as a part of multi-bytes characters, so direct
replacing with such characters on strings in local encoding causes
invalid byte sequences.

[mpm: test changed to simple doctest]
parent d7bfbc92
Branches
Tags
No related merge requests found
......@@ -23,4 +23,22 @@
from i18n import _
def replace(text, substs):
'''
Apply a list of (find, replace) pairs to a text.
>>> replace("foo bar", [('f', 'F'), ('b', 'B')])
'Foo Bar'
>>> encoding.encoding = 'latin1'
>>> replace('\\x81\\\\', [('\\\\', '/')])
'\\x81/'
>>> encoding.encoding = 'shiftjis'
>>> replace('\\x81\\\\', [('\\\\', '/')])
'\\x81\\\\'
'''
# some character encodings (cp932 for Japanese, at least) use
# ASCII characters other than control/alphabet/digit as a part of
# multi-bytes characters, so direct replacing with such characters
# on strings in local encoding causes invalid byte sequences.
utext = text.decode(encoding.encoding)
for f, t in substs:
......@@ -26,6 +44,6 @@
for f, t in substs:
text = text.replace(f, t)
return text
utext = utext.replace(f, t)
return utext.encode(encoding.encoding)
_blockre = re.compile(r"\n(?:\s*\n)+")
......
......@@ -36,3 +36,6 @@
import mercurial.revset
doctest.testmod(mercurial.revset)
import mercurial.minirst
doctest.testmod(mercurial.minirst)
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment