Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
pypy
pypy
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 584
    • Issues 584
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 11
    • Merge Requests 11
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
  • Operations
    • Operations
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • PyPy
  • pypypypy
  • Issues
  • #3356

Closed
Open
Opened Dec 02, 2020 by Kirill Smelkov@kirr

`pypy -OO` behaves differently compared to `python -OO`

Hello up there. I was preparing new release of pygolang and discoverd another PyPy bug: PyPy does not remove docstrings when invoked as pypy -OO and unfortunately this breaks zope.interfaces. Please find details below:

no -O is handled equally:

(pypy2.venv) test1@deco:~/pypy/pygolang/gpython/testprog$ /usr/bin/python print_opt.py 
sys.flags.debug:      0
sys.flags.optimize:   0
__debug__:            True
assert:               True
docstrings:           True
import mod.py:        mod.pyc

(pypy2.venv) test1@deco:~/pypy/pygolang/gpython/testprog$ pypy print_opt.py 
sys.flags.debug:      0
sys.flags.optimize:   0
__debug__:            True
assert:               True
docstrings:           True
import mod.py:        mod.pyc

One -O is also handled equally (modulo mod.pyc vs mod.pyo):

(pypy2.venv) test1@deco:~/pypy/pygolang/gpython/testprog$ /usr/bin/python -O print_opt.py 
sys.flags.debug:      0
sys.flags.optimize:   1
__debug__:            False
assert:               False
docstrings:           True
import mod.py:        mod.pyo

(pypy2.venv) test1@deco:~/pypy/pygolang/gpython/testprog$ pypy -O print_opt.py 
sys.flags.debug:      0
sys.flags.optimize:   1
__debug__:            False
assert:               False
docstrings:           True
import mod.py:        mod.pyc

However -OO is handled differently: CPython removes docstrings, while PyPy preserves them:

(pypy2.venv) test1@deco:~/pypy/pygolang/gpython/testprog$ /usr/bin/python -OO print_opt.py 
sys.flags.debug:      0
sys.flags.optimize:   2
__debug__:            False
assert:               False
docstrings:           False
import mod.py:        mod.pyo

(pypy2.venv) test1@deco:~/pypy/pygolang/gpython/testprog$ pypy -OO print_opt.py 
sys.flags.debug:      0
sys.flags.optimize:   2
__debug__:            False
assert:               False
docstrings:           True
import mod.py:        mod.pyc

This is how breakage of zope.interfaces looks like:

(pypy2.venv) test1@deco:~/pypy/pygolang$ gpython -m pytest -vsx -k opt gpython/gpython_test.py
===================================================================== test session starts ======================================================================
platform linux2 -- Python 2.7.18[pypy-7.3.3-final], pytest-4.6.11, py-1.9.0, pluggy-0.13.1 -- /home/test1/pypy/pypy2.venv/bin/gpython
cachedir: .pytest_cache
rootdir: /home/test1/pypy/pygolang
collected 17 items / 16 deselected / 1 selected                                                                                                                

gpython/gpython_test.py::test_pymain_opt Traceback (most recent call last):
  File "/home/test1/pypy/pypy2.venv/bin/gpython", line 3, in <module>
    from gpython import main; main()
  File "/home/test1/pypy/pygolang/gpython/__init__.py", line 395, in main
    pymain(argv, init)
  File "/home/test1/pypy/pygolang/gpython/__init__.py", line 217, in pymain
    init()
  File "/home/test1/pypy/pygolang/gpython/__init__.py", line 364, in init
    import gevent
  File "/home/test1/pypy/pypy2.venv/site-packages/gevent/__init__.py", line 85, in <module>
    from gevent._config import config
  File "/home/test1/pypy/pypy2.venv/site-packages/gevent/_config.py", line 699, in <module>
    Loop().get()
  File "/home/test1/pypy/pypy2.venv/site-packages/gevent/_config.py", line 146, in get
    self.value = self.validate(self._default())
  File "/home/test1/pypy/pypy2.venv/site-packages/gevent/_config.py", line 248, in validate
    return self._import_one_of([self.shortname_map.get(x, x) for x in value])
  File "/home/test1/pypy/pypy2.venv/site-packages/gevent/_config.py", line 219, in _import_one_of
    return self._import_one(item)
  File "/home/test1/pypy/pypy2.venv/site-packages/gevent/_config.py", line 237, in _import_one
    module = importlib.import_module(module)
  File "/home/test1/pypy/pypy2.7-v7.3.3-linux64/lib-python/2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "/home/test1/pypy/pypy2.venv/site-packages/gevent/libev/corecffi.py", line 17, in <module>
    from zope.interface import implementer
  File "/home/test1/pypy/pypy2.venv/site-packages/zope/interface/__init__.py", line 57, in <module>
    _wire()
  File "/home/test1/pypy/pypy2.venv/site-packages/zope/interface/interface.py", line 1115, in _wire
    from zope.interface.interfaces import IElement
  File "/home/test1/pypy/pypy2.venv/site-packages/zope/interface/interfaces.py", line 144, in <module>
    class IAttribute(IElement):
  File "/home/test1/pypy/pypy2.venv/site-packages/zope/interface/interface.py", line 787, in __init__
    self.__attrs = self.__compute_attrs(attrs)
  File "/home/test1/pypy/pypy2.venv/site-packages/zope/interface/interface.py", line 807, in __compute_attrs
    for aname, aval in attrs.items()
  File "/home/test1/pypy/pypy2.venv/site-packages/zope/interface/interface.py", line 816, in <dictcomp>
    and aval is not _decorator_non_return
  File "/home/test1/pypy/pypy2.venv/site-packages/zope/interface/interface.py", line 802, in update_value
    raise InvalidInterface("Concrete attribute, " + aname)
InvalidInterface: Concrete attribute, __doc__
FAILED

=========================================================================== FAILURES ===========================================================================
_______________________________________________________________________ test_pymain_opt ________________________________________________________________________

    @gpython_only
    def test_pymain_opt():
        def check(argv):
            argv += ["print_opt.py"]
            kw = {'cwd': testprog}
            check_gpy_vs_py(argv, **kw)
    
        check([])
        check(["-O"])
>       check(["-OO"])

gpython/gpython_test.py:261: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
gpython/gpython_test.py:257: in check
    check_gpy_vs_py(argv, **kw)
gpython/gpython_test.py:342: in check_gpy_vs_py
    gpyout   = u(pyout(argv, **kw))
golang/golang_test.py:1842: in pyout
    return pyrun(argv, stdin=stdin, stdout=stdout, stderr=stderr, **kw)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

argv = ['-OO', 'print_opt.py'], stdin = None, stdout = '', stderr = None, kw = {'cwd': '/home/test1/pypy/pygolang/gpython/testprog'}, retcode = 1

    def pyrun(argv, stdin=None, stdout=None, stderr=None, **kw):
        retcode, stdout, stderr = _pyrun(argv, stdin=stdin, stdout=stdout, stderr=stderr, **kw)
        if retcode:
>           raise RuntimeError(' '.join(argv) + '\n' + (stderr and str(stderr) or '(failed)'))
E           RuntimeError: -OO print_opt.py
E           (failed)

golang/golang_test.py:1836: RuntimeError

---- 8< ---- (print_opt.py)

# -*- coding: utf-8 -*-
# Copyright (C) 2020  Nexedi SA and Contributors.
#                     Kirill Smelkov <kirr@nexedi.com>
#
# This program is free software: you can Use, Study, Modify and Redistribute
# it under the terms of the GNU General Public License version 3, or (at your
# option) any later version, as published by the Free Software Foundation.
#
# You can also Link and Combine this program with other software covered by
# the terms of any of the Free Software licenses or any of the Open Source
# Initiative approved licenses and Convey the resulting work. Corresponding
# source of such a combination shall include the source code for all other
# software used.
#
# This program is distributed WITHOUT ANY WARRANTY; without even the implied
# warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
#
# See COPYING file for full licensing terms.
# See https://www.nexedi.com/licensing for rationale and options.
"""Program print_opt prints information about optimizations."""

from __future__ import print_function, absolute_import

import sys, os, os.path, tempfile, shutil

def main():
    print('sys.flags.debug:      %s' % sys.flags.debug)
    print('sys.flags.optimize:   %s' % sys.flags.optimize)
    print('__debug__:            %s' % __debug__)
    print('assert:               %s' % is_assert_enabled())
    print('docstrings:           %s' % is_docstrings_enabled())
    print('import mod.py:        %s' % modpy_imports_from())


# is_assert_enabled returns whether assert statements are enabled.
def is_assert_enabled():
    try:
        assert False # must raise AssertionError
    except AssertionError:
        return True
    else:
        return False

# is_docstrings_enabled returns whether docstrings are enabled.
def is_docstrings_enabled():
    def _():
        """hello world"""
    if _.__doc__ is None:
        return False
    if _.__doc__ == "hello world":
        return True
    raise AssertionError(_.__doc__)

# modpy returns name for compiled version of python module mod.py
def modpy_imports_from():
    try:
        import mod
    except ImportError:
        # ok - should not be there
        pass
    else:
        raise AssertionError("module 'mod' is already there")

    tmpd = tempfile.mkdtemp('', 'modpy_imports_from')
    try:
        pymod = "%s/mod.py" % tmpd
        with open(pymod, "w") as f:
            f.write("# hello up there\n")

        sys.path.insert(0, tmpd)
        import mod

        files = set()
        for dirpath, dirnames, filenames in os.walk(tmpd):
            for _ in filenames:
                f = '%s/%s' % (dirpath, _)
                if f.startswith(tmpd+'/'):
                    f = f[len(tmpd+'/'):]
                files.add(f)


        files.remove("mod.py") # must be there | raises if not
        if len(files) == 0:
            from_ = "mod.py"   # source-only import
        else:
            if len(files) != 1:
                raise AssertionError("mod.py -> multiple compiled files (%s)" % (files,))

            from_ = files.pop()

        return from_

    finally:
        shutil.rmtree(tmpd)


if __name__ == '__main__':
    main()

---- 8< ----

(pypy2.venv) test1@deco:~/pypy/pygolang/gpython/testprog$ pypy --version
Python 2.7.18 (a29ef73f9b32, Nov 09 2020, 18:42:06)
[PyPy 7.3.3 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)]

Thanks beforehand,
Kirill

To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: pypy/pypy#3356