compat.html 14.9 KB
Newer Older
Matti Picus's avatar
Matti Picus committed
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
<!DOCTYPE html>
<html prefix="
        og: http://ogp.me/ns# article: http://ogp.me/ns/article#
    " vocab="http://ogp.me/ns" lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Python compatibility | PyPy</title>
<link href="assets/css/rst_base.css" rel="stylesheet" type="text/css">
<link href="assets/css/nikola_rst.css" rel="stylesheet" type="text/css">
<link href="assets/css/code.css" rel="stylesheet" type="text/css">
<link href="assets/css/theme.css" rel="stylesheet" type="text/css">
<link href="assets/css/styles.css" rel="stylesheet" type="text/css">
<meta name="theme-color" content="#5670d4">
<meta name="generator" content="Nikola (getnikola.com)">
<link rel="alternate" type="application/rss+xml" title="RSS" hreflang="en" href="rss.xml">
<link rel="canonical" href="https://www.pypy.org/compat.html">
Matti Picus's avatar
Matti Picus committed
18
19
<link rel="icon" href="favicon2.ico" sizes="16x16">
<link rel="icon" href="favicon32x32.ico" sizes="32x32">
Matti Picus's avatar
Matti Picus committed
20
21
22
23
24
<!--[if lt IE 9]><script src="assets/js/html5shiv-printshiv.min.js"></script><![endif]--><link href="https://fonts.googleapis.com/css?family=Roboto" rel="stylesheet">
<meta name="author" content="The PyPy Team">
<meta property="og:site_name" content="PyPy">
<meta property="og:title" content="Python compatibility">
<meta property="og:url" content="https://www.pypy.org/compat.html">
25
<meta property="og:description" content="PyPy implements the Python language version 2.7.18. It supports all of the core
Matti Picus's avatar
Matti Picus committed
26
27
28
29
30
31
32
33
language, passing Python test suite (with minor modifications that were
already accepted in the main python in newer ver">
<meta property="og:type" content="article">
<meta property="article:published_time" content="2019-12-28T16:14:02Z">
</head>
<body>
    <a href="#content" class="sr-only sr-only-focusable">Skip to main content</a>
    <div id="container">
Matti Picus's avatar
Matti Picus committed
34
             <header id="header"><!-- Adapted from https://www.taniarascia.com/responsive-dropdown-navigation-bar --><section class="navigation"><div class="nav-container">
Matti Picus's avatar
Matti Picus committed
35
36
37
            <div class="brand">
                <a href="index.html">
                    <image id="toplogo" src="images/pypy-logo.svg" width="75px;" alt="PyPy/"></image></a>
Matti Picus's avatar
Matti Picus committed
38
            </div>
Matti Picus's avatar
Matti Picus committed
39
            <nav><ul class="nav-list">
Matti Picus's avatar
Matti Picus committed
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
<li> 
                <a href="#!">Features</a>
                <ul class="nav-dropdown">
<li> <a href="features.html">What is PyPy?</a> </li>  
                    <li> <a href="#">Compatibility</a> </li>  
                    <li> <a href="performance.html">Performance</a> </li>  
                </ul>
</li>
          <li> <a href="download.html">Download</a> </li>  
          <li> <a href="http://doc.pypy.org">Dev Documentation</a> </li>  
          <li> <a href="http://morepypy.blogspot.com">Blog</a> </li>  
            <li> 
                <a href="#!">About</a>
                <ul class="nav-dropdown">
<li> <a href="people.html">People</a> </li>  
55
                    <li> <a href="howtohelp.html">How To Help?</a> </li>  
Matti Picus's avatar
Matti Picus committed
56
57
58
                    <li> <a href="contact.html">Contact</a> </li>  
                </ul>
</li>
Matti Picus's avatar
Matti Picus committed
59

Matti Picus's avatar
Matti Picus committed
60
61
62
63
                </ul></nav><div class="nav-mobile">
                <a id="nav-toggle" href="#!"> <span></span></a>
            </div>
        </div>
Matti Picus's avatar
Matti Picus committed
64
    </section></header><main id="content"><article class="post-text storypage" itemscope="itemscope" itemtype="http://schema.org/Article"><header><h1 class="p-name entry-title" itemprop="headline name"><a href="#" class="u-url">Python compatibility</a></h1>
Matti Picus's avatar
Matti Picus committed
65
66
67
68

        
    </header><div class="e-content entry-content" itemprop="articleBody text">
    <div>
69
<p>PyPy implements the Python language version 2.7.18. It supports all of the core
Matti Picus's avatar
Matti Picus committed
70
71
72
language, passing Python test suite (with minor modifications that were
already accepted in the main python in newer versions). It supports most
of the commonly used Python <a class="reference external" href="http://docs.python.org/library/">standard library modules</a>; details below.</p>
73
<p>PyPy3 implements the Python language version 3.7.9.  It has been released,
Matti Picus's avatar
Matti Picus committed
74
75
76
77
78
79
80
81
82
83
but Python is a large language and it is quite possible that a few things are missing.</p>
<p>PyPy has support for the <a class="reference external" href="http://docs.python.org/c-api/">CPython C API</a>, however there are constructs
that are <cite>not compatible</cite>.  We strongly advise use of <a class="reference external" href="http://cffi.readthedocs.org/">CFFI</a>
instead. CFFI come builtin with PyPy. Many libraries will require
a bit of effort to work, but there are known success stories. Check out
PyPy blog for updates</p>
<p>C extensions need to be recompiled for PyPy in order to work. Depending on
your build system, it might work out of the box or will be slightly harder.</p>
<p>Standard library modules supported by PyPy. Note that large parts of python
library are implemented in pure python, so they don't have to be listed
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
there. Please just check if it imports. If it imports, it should work:</p>
<blockquote>
<p><code class="docutils literal">__builtin__</code>, <code class="docutils literal">__pypy__</code>, <code class="docutils literal">_ast</code>, <code class="docutils literal">_cffi_backend</code>, <code class="docutils literal">_codecs</code>,
<code class="docutils literal">_collections</code>, <code class="docutils literal">_continuation</code>, <code class="docutils literal">_csv</code>, <code class="docutils literal">_file</code>, <code class="docutils literal">_hashlib</code>,
<code class="docutils literal">_io</code>, <code class="docutils literal">_locale</code>, <code class="docutils literal">_lsprof</code>, <code class="docutils literal">_md5</code>, <code class="docutils literal">_minimal_curses</code>,
<code class="docutils literal">_multibytecodec</code>, <code class="docutils literal">_multiprocessing</code>, <code class="docutils literal">_numpypy</code>, <code class="docutils literal">_pickle_support</code>,
<code class="docutils literal">_pypyjson</code>, <code class="docutils literal">_random</code>, <code class="docutils literal">_rawffi</code>, <code class="docutils literal">_sha</code>, <code class="docutils literal">_socket</code>, <code class="docutils literal">_sre</code>,
<code class="docutils literal">_ssl</code>, <code class="docutils literal">_struct</code>, <code class="docutils literal">_testing</code>, <code class="docutils literal">_warnings</code>, <code class="docutils literal">_weakref</code>, <code class="docutils literal">array</code>,
<code class="docutils literal">binascii</code>, <code class="docutils literal">bz2</code>, <code class="docutils literal">cStringIO</code>, <code class="docutils literal">cmath</code>, <code class="docutils literal">cppyy</code>, <code class="docutils literal">cpyext</code>,
<code class="docutils literal">crypt</code>, <code class="docutils literal">errno</code>, <code class="docutils literal">exceptions</code>, <code class="docutils literal">fcntl</code>, <code class="docutils literal">gc</code>, <code class="docutils literal">imp</code>,
<code class="docutils literal">itertools</code>, <code class="docutils literal">marshal</code>, <code class="docutils literal">math</code>, <code class="docutils literal">mmap</code>, <code class="docutils literal">operator</code>, <code class="docutils literal">parser</code>,
<code class="docutils literal">posix</code>, <code class="docutils literal">pwd</code>, <code class="docutils literal">pyexpat</code>, <code class="docutils literal">pypyjit</code>, <code class="docutils literal">select</code>, <code class="docutils literal">signal</code>,
<code class="docutils literal">symbol</code>, <code class="docutils literal">sys</code>, <code class="docutils literal">termios</code>, <code class="docutils literal">thread</code>, <code class="docutils literal">time</code>, <code class="docutils literal">token</code>,
<code class="docutils literal">unicodedata</code>, <code class="docutils literal">zipimport</code>, <code class="docutils literal">zlib</code></p>
</blockquote>
Matti Picus's avatar
Matti Picus committed
99
<p>Supported, and written in pure Python:</p>
100
101
102
103
<blockquote>
<p><code class="docutils literal">cPickle</code>, <code class="docutils literal">ctypes</code>, <code class="docutils literal">datetime</code>, <code class="docutils literal">dbm</code>, <code class="docutils literal">_functools</code>, <code class="docutils literal">grp</code>,
<code class="docutils literal">readline</code>, <code class="docutils literal">resource</code>, <code class="docutils literal">sqlite3</code>, <code class="docutils literal">syslog</code></p>
</blockquote>
Matti Picus's avatar
Matti Picus committed
104
105
106
107
<p>All modules that are pure python in CPython of course work.</p>
<p>The main difference that is not going to be fixed is that PyPy does
not support refcounting semantics. The following code won't fill the
file immediately, but only after a certain period of time, when the GC
108
does a collection and flushes the output:</p>
gitlab-runner's avatar
gitlab-runner committed
109
<pre class="code python"><a name="rest_code_0ca0d63fd465410996f57187a685d4ea-1"></a><span class="nb">open</span><span class="p">(</span><span class="s2">"filename"</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s2">"stuff"</span><span class="p">)</span>
Matti Picus's avatar
Matti Picus committed
110
111
</pre>
<p>The proper fix is</p>
gitlab-runner's avatar
gitlab-runner committed
112
113
<pre class="code python"><a name="rest_code_6d6518fd41904341852c849cab944994-1"></a><span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="s2">"filename"</span><span class="p">,</span> <span class="s2">"w"</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
<a name="rest_code_6d6518fd41904341852c849cab944994-2"></a>    <span class="n">f</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s2">"stuff"</span><span class="p">)</span>
Matti Picus's avatar
Matti Picus committed
114
115
116
117
118
119
120
121
122
123
124
125
126
</pre>
<p>The same problem---not closing your files---can also show up if your
program opens a large number of files without closing them explicitly.
In that case, you can easily hit the system limit on the number of file
descriptors that are allowed to be opened at the same time.</p>
<p>PyPy can be run with the command-line option <code class="docutils literal"><span class="pre">-X</span> <span class="pre">track-resources</span></code> (as in,
<code class="docutils literal">pypy <span class="pre">-X</span> <span class="pre">track-resources</span> myprogram.py</code>). This produces a <code class="docutils literal">ResourceWarning</code>
when the GC closes a non-closed file or socket.  The traceback for the place
where the file or socket was allocated is given as well, which aids finding
places where <code class="docutils literal">close()</code> is missing.</p>
<p>Similarly, remember that you must <code class="docutils literal">close()</code> a non-exhausted
generator in order to have its pending <code class="docutils literal">finally</code> or <code class="docutils literal">with</code>
clauses executed immediately:</p>
gitlab-runner's avatar
gitlab-runner committed
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
<pre class="code python"><a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-1"></a><span class="k">def</span> <span class="nf">mygen</span><span class="p">():</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-2"></a>    <span class="k">with</span> <span class="n">foo</span><span class="p">:</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-3"></a>        <span class="k">yield</span> <span class="mi">42</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-4"></a>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-5"></a><span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">mygen</span><span class="p">():</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-6"></a>    <span class="k">if</span> <span class="n">x</span> <span class="o">==</span> <span class="mi">42</span><span class="p">:</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-7"></a>        <span class="k">break</span>    <span class="c1"># foo.__exit__ is not run immediately!</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-8"></a>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-9"></a><span class="c1"># fixed version:</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-10"></a><span class="n">gen</span> <span class="o">=</span> <span class="n">mygen</span><span class="p">()</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-11"></a><span class="k">try</span><span class="p">:</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-12"></a>    <span class="k">for</span> <span class="n">x</span> <span class="ow">in</span> <span class="n">gen</span><span class="p">:</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-13"></a>        <span class="k">if</span> <span class="n">x</span> <span class="o">==</span> <span class="mi">42</span><span class="p">:</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-14"></a>            <span class="k">break</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-15"></a><span class="k">finally</span><span class="p">:</span>
<a name="rest_code_d470342754c54a9fad6cb7cf49b78dbd-16"></a>    <span class="n">gen</span><span class="o">.</span><span class="n">close</span><span class="p">()</span>
Matti Picus's avatar
Matti Picus committed
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
</pre>
<p>More generally, <code class="docutils literal">__del__()</code> methods are not executed as predictively
as on CPython: they run "some time later" in PyPy (or not at all if
the program finishes running in the meantime).  See <a class="reference external" href="http://pypy.readthedocs.org/en/latest/cpython_differences.html#differences-related-to-garbage-collection-strategies">more details
here</a>.</p>
<p>Note that PyPy returns unused memory to the operating system if there
is a madvise() system call (at least Linux, OS X, BSD) or on Windows.  It is
important to realize that you may not see this in <code class="docutils literal">top</code>.  The unused
pages are marked with <code class="docutils literal">MADV_FREE</code>, which tells the system "if you
need more memory at some point, grab this page".  As long as memory is
plentiful, the <code class="docutils literal">RES</code> column in <code class="docutils literal">top</code> might remains high.  (Exceptions to
this rule are systems with no <code class="docutils literal">MADV_FREE</code>, where we use
<code class="docutils literal">MADV_DONTNEED</code>, which forcefully lowers the <code class="docutils literal">RES</code>.  This includes
Linux &lt;= 4.4.)</p>
<p>A more complete list of known differences is available at <a class="reference external" href="http://pypy.readthedocs.org/en/latest/cpython_differences.html">our dev site</a>.</p>
</div>
    </div>
    
</article></main><footer id="footer"><p>
</p>
163
164
165
166
167
<div class="myfooter">
  <div>
    <img src="images/pypy-logo-nav-grey.png" alt="PyPy Logo">
</div>
  <div class="logotext">
gitlab-runner's avatar
gitlab-runner committed
168
     Contents © 2021 <a href="mailto:pypy-dev@pypy.org">The PyPy Team</a>
169
170
171
172
    Powered by <a href="https://getnikola.com" rel="nofollow">Nikola</a> 
  </div>
</div>

Matti Picus's avatar
Matti Picus committed
173
174
175
            
        </footer>
</div>
Matti Picus's avatar
Matti Picus committed
176
            <script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/2.1.3/jquery.min.js" crossorigin="anonymous"></script><script src="assets/js/styles.js"></script>
Matti Picus's avatar
Matti Picus committed
177
178
</body>
</html>