Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
pypy
pypy
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 582
    • Issues 582
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 12
    • Merge Requests 12
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards

This instance will be upgraded to Heptapod 0.19.0rc2 on 2021-01-18 at 11:00 UTC+1 (a few minutes of downtime)

  • PyPy
  • pypypypy
  • Issues
  • #3308

Closed
Open
Opened Sep 24, 2020 by Stefano Rivera@stefanorMaintainer

readline module raises InvalidTerminal with TERM=dumb - test failures and cpython differences

In some build environments, I historically had some readline tests lock up on some architectures, so I build with TERM=dumb. This triggers some bugs:

REPL in 3.6:

site.py initializes readline, causing a trackback to the terminal on REPL start-up:

$ TERM=dumb pypy3
Python 3.6.9 (7.3.2~rc2+dfsg-1, Sep 23 2020, 01:53:42)
[PyPy 7.3.2 with GCC 10.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Traceback (most recent call last):
  File "/usr/lib/pypy3/lib-python/3/site.py", line 437, in register_readline
    if readline.get_current_history_length() == 0:
  File "/usr/lib/pypy3/lib_pypy/pyrepl/readline.py", line 309, in get_current_history_length
    return len(self.get_reader().history)
  File "/usr/lib/pypy3/lib_pypy/pyrepl/readline.py", line 251, in get_reader
    console = UnixConsole(self.f_in, self.f_out, encoding=ENCODING)
  File "/usr/lib/pypy3/lib_pypy/pyrepl/unix_console.py", line 116, in __init__
    self._clear = _my_getstr("clear")
  File "/usr/lib/pypy3/lib_pypy/pyrepl/unix_console.py", line 57, in _my_getstr
    "terminal doesn't have the required '%s' capability"%cap)
pyrepl.unix_console.InvalidTerminal: terminal doesn't have the required 'clear' capability
>>>> 

cpython seems to just handle the situation (and readline functions).

Readline tests:

On 3.6:

======================================================================
ERROR: testHistoryUpdates (test.test_readline.TestHistoryManipulation)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/lib-python/3/test/test_readline.py", line 32, in testHistoryUpdates
    readline.clear_history()
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/readline.py", line 346, in clear_history
    del self.get_reader().history[:]
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/readline.py", line 251, in get_reader
    console = UnixConsole(self.f_in, self.f_out, encoding=ENCODING)
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/unix_console.py", line 116, in __init__
    self._clear = _my_getstr("clear")
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/unix_console.py", line 57, in _my_getstr
    "terminal doesn't have the required '%s' capability"%cap)
pyrepl.unix_console.InvalidTerminal: terminal doesn't have the required 'clear' capability

======================================================================
ERROR: test_nonascii_history (test.test_readline.TestHistoryManipulation)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/lib-python/3/test/test_readline.py", line 94, in test_nonascii_history
    readline.clear_history()
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/readline.py", line 346, in clear_history
    del self.get_reader().history[:]
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/readline.py", line 251, in get_reader
    console = UnixConsole(self.f_in, self.f_out, encoding=ENCODING)
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/unix_console.py", line 116, in __init__
    self._clear = _my_getstr("clear")
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/unix_console.py", line 57, in _my_getstr
    "terminal doesn't have the required '%s' capability"%cap)
pyrepl.unix_console.InvalidTerminal: terminal doesn't have the required 'clear' capability

(2.7 has the same issue, but the second test doesn't exist there)

That's a different behaviour to cPython, but doesn't seem worrying. Using readline without a terminal that supports it wouldn't make much sense. So, I'm just just ignoring it with this patch:

diff --git a/lib-python/3/test/test_readline.py b/lib-python/3/test/test_readline.py
index 17157a16..341fe9f8 100644
--- a/lib-python/3/test/test_readline.py
+++ b/lib-python/3/test/test_readline.py
@@ -10,6 +10,7 @@ import subprocess
 import sys
 import tempfile
 import unittest
+from pyrepl.unix_console import InvalidTerminal
 from test.support import import_module, unlink, temp_dir, TESTFN
 from test.support.script_helper import assert_python_ok
 
@@ -29,7 +30,10 @@ class TestHistoryManipulation (unittest.TestCase):
     """
 
     def testHistoryUpdates(self):
-        readline.clear_history()
+        try:
+            readline.clear_history()
+        except InvalidTerminal as e:
+            raise unittest.SkipTest(e)
 
         readline.add_history("first line")
         readline.add_history("second line")
@@ -91,7 +95,10 @@ class TestHistoryManipulation (unittest.TestCase):
         readline.write_history_file(hfilename)
 
     def test_nonascii_history(self):
-        readline.clear_history()
+        try:
+            readline.clear_history()
+        except InvalidTerminal as e:
+            raise unittest.SkipTest(e)
         try:
             readline.add_history("entrée 1")
         except UnicodeEncodeError as err:

3.6 rlcompleter module test failures

From the description of the rlcompleter module, it should be useful when readline isn't available, but again, it blows up with TERM=dumb.

======================================================================
ERROR: test_attr_matches (test.test_rlcompleter.TestRlcompleter)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/<<PKGBUILDDIR>>/lib-python/3/test/test_rlcompleter.py", line 20, in setUp
    self.stdcompleter.complete('', 0)
  File "/<<PKGBUILDDIR>>/lib-python/3/rlcompleter.py", line 79, in complete
    readline.insert_text('\t')
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/readline.py", line 396, in insert_text
    return self.get_reader().insert(text)
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/readline.py", line 251, in get_reader
    console = UnixConsole(self.f_in, self.f_out, encoding=ENCODING)
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/unix_console.py", line 116, in __init__
    self._clear = _my_getstr("clear")
  File "/<<PKGBUILDDIR>>/lib_pypy/pyrepl/unix_console.py", line 57, in _my_getstr
    "terminal doesn't have the required '%s' capability"%cap)
pyrepl.unix_console.InvalidTerminal: terminal doesn't have the required 'clear' capability

...
FAILED (errors=7)
test_rlcompleter failed

Solveable with a simple (but ugly) patch:

--- a/lib-python/3/rlcompleter.py
+++ b/lib-python/3/rlcompleter.py
@@ -32,6 +32,7 @@ Notes:
 import atexit
 import builtins
 import __main__
+from pyrepl.unix_console import InvalidTerminal
 
 __all__ = ["Completer"]
 
@@ -76,11 +77,13 @@ class Completer:
         if not text.strip():
             if state == 0:
                 if _readline_available:
-                    readline.insert_text('\t')
-                    readline.redisplay()
-                    return ''
-                else:
-                    return '\t'
+                    try:
+                        readline.insert_text('\t')
+                        readline.redisplay()
+                        return ''
+                    except InvalidTerminal:
+                        pass
+                return '\t'
             else:
                 return None

Conclusions

These are all essentially the same bug, which is that the pypy readline module doesn't match cpython's behaviour, and throws this InvalidTerminal exception, where cPython manages to avoid blowing up. Maybe because it isn't checking a return code or something? I haven't dived to the bottom of it, yet.

Edited Sep 24, 2020 by Stefano Rivera
To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: pypy/pypy#3308