Skip to content
Snippets Groups Projects
numpydonate.html 13.3 KiB
Newer Older
<!DOCTYPE html>
<html>
<head>
	<title>PyPy - Call for donations - PyPy to support Numpy!</title>
	<meta http-equiv="content-language" content="en" />
	<meta http-equiv="content-type" content="text/html; charset=utf-8" />
	<meta name="author" content="PyPy Team" />
	<meta name="description" content="PyPy" />
	<meta name="copyright" content="MIT" />
	<meta name="document-rating" content="general" />
	<link rel="stylesheet" type="text/css" media="screen" title="default" href="css/site.css" />
	<link rel="alternate" type="application/rss+xml" title="RSS Feed for PyPy" href="http://feeds.feedburner.com/PyPyStatusBlog" />
  <link rel="stylesheet" type="text/css" href="css/jquery-ui-1.8.14.custom.css" />
	<script type="text/javascript" src="https://use.typekit.com/hdt8sni.js"></script>
	<script type="text/javascript">try{Typekit.load();}catch(e){}</script>
	<script type="text/javascript" src="https://ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
  <script type="text/javascript" src="js/jquery-ui-1.8.14.custom.min.js"></script>
  <script type="text/javascript" src="js/script2.js?bust=1"></script>
</head>
<body>
<script type="text/javascript">
	var _gaq = [['_setAccount', 'UA-7778406-3'], ['_trackPageview']];
	if (document.location.protocol !== 'file:') {
		(function() {
			var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
			ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
			(document.getElementsByTagName('head')[0] || document.getElementsByTagName('body')[0]).appendChild(ga);
		})();
	}
</script>
<div id="body-outer"><div id="body-inner"><div id="body" class="clearfix">
<div id="header">
    <div id="menu-follow">
    <div><a href="https://bitbucket.org/pypy/pypy"><img src="https://www.selenic.com/hg-logo/logo-droplets-25.png" width="14px" height="14px" /></a></div>
		<div><a href="http://feeds.feedburner.com/PyPyStatusBlog" title="Subscribe to the RSS Feed"><img src="https://feedburner.google.com/fb/lib/images/icons/feed-icon-12x12-orange.gif" alt="Subscribe to the RSS Feed" width="14px" height="14px" /></a></div>
	<div id="logo"><a href="http://pypy.org"><img src="image/pypy-logo.png" alt="PyPy" height="110px" /></a></div>
	<hr class="clear-left" />
    <div id="menu-sub">
      <a href="index.html">Home</a>
        <span class="menu-sub-sep"> | </span>
      <a href="features.html">What is PyPy?</a>
        <span class="menu-sub-sep"> | </span>
      <a href="download.html">Download</a>
        <span class="menu-sub-sep"> | </span>
      <a href="compat.html">Compatibility</a>
        <span class="menu-sub-sep"> | </span>
      <a href="performance.html">Performance</a>
        <span class="menu-sub-sep"> | </span>
      <a href="http://doc.pypy.org">Dev Documentation</a>
        <span class="menu-sub-sep"> | </span>
      <a href="http://morepypy.blogspot.com">Blog</a>
        <span class="menu-sub-sep"> | </span>
      <a href="people.html">People</a>
        <span class="menu-sub-sep"> | </span>
      <a href="contact.html">Contact</a>
        <br />
    </div>
	<hr class="clear" />
</div>
<div id="content">
<div>
<div id="main">
<h1 class="title">Call for donations - PyPy to support Numpy!</h1>
<p>UPDATE (Febuary 2016):</p>
<blockquote>
<em>We have closed this campaign. We have achieved most of the goals
of the orignal work plan while raising most of the funds. Work
will continue to make Numpy and the rest of the numeric stack more
usable on PyPy.</em></blockquote>
Matti Picus's avatar
Matti Picus committed
<p>UPDATE (May 2014):</p>
Matti Picus's avatar
Matti Picus committed
<em>Thanks to our donors, we have raised 80% of
the total so far.  Work on this topic has been happening, and
continues to happen, within the budget &ndash; even if not within the
timeline described below.  We have simply not found enough time to
work on it as much as we wanted, and thus did not consume the money as
quickly as predicted.  The ratio &ldquo;progress / $ used&rdquo; so far
corresponds roughly to what we expected.  The document below is the
original call for proposal, and we still accept donations for this
topic. See the</em> <a class="reference external" href="http://morepypy.blogspot.com/search/label/numpy">latest status reports</a> <em>on our blog for updates.
There is also an automatically generated</em> <a class="reference external" href="http://buildbot.pypy.org/numpy-status/latest.html">coverage dashboard</a> <em>showing
what parts of NumPy are already usable.</em></blockquote>
<p>This is a proposal to provide a fully compatible working <a class="reference external" href="http://numpy.scipy.org/">NumPy</a> implementation
for PyPy. This has long be en a very commonly <a class="reference external" href="http://morepypy.blogspot.com/2011/06/report-back-from-our-survey.html">requested feature</a> for PyPy
as well as a worthy goal given that PyPy performs extremely well on numeric
workloads.</p>
Matti Picus's avatar
Matti Picus committed
<p>We already had some success providing a very basic <a class="reference external" href="http://morepypy.blogspot.com/2011/05/numpy-in-pypy-status-and-roadmap.html">NumPy implementation</a>,
however we believe that raising funds can significantly speed up its
development, by paying people to work full time on it.</p>
<p>Below you'll find the <a class="reference internal" href="#work-plan">work plan</a> and the associated
fundraising targets we need to make things happen.  Once we reach the
necessary target for each stage, we will start contracting
developers.  Contracts and money are managed by the non-profit
<a class="reference external" href="http://sfconservancy.org/">Software Freedom Conservancy</a> of which the PyPy project is a member.
The current elected representatives are Carl Friedrich Bolz, Holger
Krekel and Jacob Hallen and they will – in close collaboration with
Conservancy and the core developers – select the best developers for
implementing NumPy among well known PyPy contributors.</p>
<p>Should we not receive enough donations to complete all stages by 1st March 2012
at the latest, we will try our best to make PyPy support NumPy anyway.  We
however reserve the right to shift any unused funds to other PyPy activities
when that date is reached.  Of course, since the Conservancy is a
501(c)(3) charitable organization incorporated in NY, USA, all funds will,
regardless of their use, be spent in a way that benefits the general
public, the advancement of Open Source and Free Software,
and in particular the PyPy community and the PyPy codebase.</p>
<p><strong>Note</strong> For donations higher than $1,000, we can arrange for an invoice
and a different payment method to avoid the high Paypal fees.  Please
contact pypy at sfconservancy.org if you want to know details on how
to donate via other means.</p>
<div class="section" id="what-is-numpy">
<h1>What is NumPy?</h1>
<p><a class="reference external" href="http://numpy.scipy.org/">NumPy</a> is a framework for doing numerical calculations in Python. It has
become the de-facto standard for doing any kinds of computations that involve
n-dimensional arrays. Please consult the NumPy website for more details.</p>
</div>
Maciej Fijalkowski's avatar
Maciej Fijalkowski committed
<div class="section" id="why-does-numpy-on-pypy-make-sense">
<h1>Why does NumPy on PyPy make sense?</h1>
<p>NumPy on PyPy makes sense for a couple of reasons: Firstly, it is by
far the most requested feature from PyPy. Secondly, PyPy <a class="reference external" href="http://morepypy.blogspot.com/2011/07/realtime-image-processing-in-python.html">performs
well</a> on numerical loads already.  Therefore bringing NumPy into the
equation is a reasonable next step - as it's a very convenient and popular tool
for doing this kind of work. The resulting implementation should move
Python in scientific world from being a merely &ldquo;glue&rdquo; language into
being the main implementation language for a lot of people in
the scientific/numeric worlds. This will benefit current users of NumPy as
well as people who so far have to cope with lower level languages like C
or Fortran for speed purposes.</p>
</div>
<div class="section" id="speed">
<h1>Speed</h1>
<p>The current implementation of NumPy on PyPy is reasonably fast - it ranges
from roughly the same speed to 2-5x faster for stacked large array operations
to 100-300x for accessing NumPy array elements one by one. The exact speed
depends very much how NumPy is used, but the target would be to be within an
order of magnitude from handwritten C.
To achieve this, we would
need to teach our JIT backends how to use modern vector instructions, like SSE
Matti Picus's avatar
Matti Picus committed
or AVX. Hence, we split the proposal into two parts, first part covers compatibility
with a reasonable approach to keeping current speed achievements, second part
is about teaching the JIT how to vectorize certain operations, which should bring
PyPy's NumPy as a very competitive tool compared to other available
solutions for numerical computations, like matlab or C++ array libraries.</p>
<div class="section" id="id1">
<span id="work-plan"></span><h2>Work plan</h2>
</div>
</div>
<div class="section" id="about-estimates-and-costs">
<h1>About estimates and costs</h1>
<p>For each step, we estimated the time that it would take to complete for an
experienced developer who is already familiar with the PyPy codebase.  From
Alex Gaynor's avatar
Alex Gaynor committed
this number, the money is calculated considering an hourly rate of $60, and a
5% general donation which goes to the <a class="reference external" href="http://sfconservancy.org/">Software Freedom Conservancy</a> itself, the non-profit
organization of which the PyPy project is a member and which manages all the
issues related to donations, payments, and tax-exempt status.</p>
<p>We split the proposal into two parts - we plan to implement them in the same
order, starting by the time we raise the corresponding funding targets:</p>
<ul>
<li><p class="first"><strong>Compatibility</strong>:</p>
<p>This part covers the core NumPy Python API. We'll implement all NumPy APIs
that are officially documented and we'll pass all of NumPy's tests that
cover documented APIs and are not implementation details.
Specifically, we don't plan to:</p>
<ul class="simple">
<li>implement NumPy's C API</li>
<li>implement other scientific libraries, like SciPy, matplotlib or biopython</li>
</ul>
<p>Estimated costs: USD$30,000. Estimated duration: 3 months.</p>
</li>
<li><p class="first"><strong>Speed</strong>:</p>
<p>This part will cover significant speed improvements in the JIT that would
make numeric computations faster. This includes, but is not necesarilly
limited to:</p>
<ul class="simple">
<li>write a set of benchmarks covering various use cases</li>
<li>teaching the JIT backend (or multiple backends) how to deal with vector
operations, like SSE</li>
<li>experiments with automatic parallelization using multiple threads, akin
to numexpr</li>
<li>improving the JIT register allocator that will make a difference, especially
for tight loops</li>
</ul>
<p>As with all speed improvements, it's relatively hard to predict exactly
Alex Gaynor's avatar
Alex Gaynor committed
how it'll cope, however we expect the results to be within an order
of magnitude of handwritten C equivalent.</p>
<p>Estimated costs: USD$30,000. Estimated duration: 3 months.</p>
</li>
</ul>
<div class="section" id="benefits-of-this-work-to-the-python-community-and-the-general-public">
<h2>Benefits of This Work to the Python Community and the General Public</h2>
<p>Python has become one of the most popular dynamic programming languages in
the world.  Web developers, educators, and scientific programmers alike
all value Python because Python code is often more readable and because
Python often increases programmer productivity.</p>
<p>Traditionally, languages like Python ran more slowly than static, compiled
languages; Python developers chose to sacrifice execution speed for ease
of programming.  The PyPy project created a substantially improved Python
language implementation, including a fast Just-in-time (JIT) compiler.
The increased execution speed that PyPy provides has attracted many users,
who now find their Python code runs up to four times faster under PyPy
than under the reference implementation written in C.</p>
<p>Meanwhile, adoption of Python is already underway for those researchers
and developers who work specifically on computing that requires fast
numeric operations.  Numpy support in PyPy will allow for Python's use by
those developers and researchers who want the ease of programming that
Python provides, the speed of PyPy, <em>and</em> the speedups for numerical work
that Numpy can provide.</p>
<p>PyPy's developers make all PyPy software available to the public without
charge, under PyPy's Open Source copyright license, the permissive MIT
License.  PyPy's license assures that PyPy is equally available to
everyone freely on terms that allow both non-commercial and commercial
activity.  This license allows for academics, for-profit software
developers, volunteers and enthusiasts alike to collaborate together to
make a better Python implementation for everyone.</p>
<p>NumPy support for PyPy will be licensed similarly, and therefore NumPy
support can directly help researchers and developers who seek to do
numeric computing but want an easier programming language to use than
Fortan or C, which is typically used for these applications.  Being
licensed freely to the general public means that opportunities to use,
improve and learn about how NumPy works itself will be generally available
to everyone.</p>
</div>
</div>
</div>
<div id="sidebar">
</div>
</div>
</div>
</div></div></div>
</body>
</html>