PyPy-3.8-7.3.7 : Problem with asyncio.ProactorEventLoop on Windows.
The Python 3.8 support with PyPy-3.8-7.3.7 is working and performing great already!
But, already for a while, I've been experiencing a problem with the asyncio.ProactorEventLoop
on Windows. This is the default event loop on Windows in Python 3.8. By manually setting the event loop to asyncio.SelectorEventLoop
(using asyncio.set_event_loop_policy(asyncio.WindowsSelectorEventLoopPolicy())
) the problem disappears completely, the same as if we're running things with PyPy-3.7-7.3.7 say (like with all recent versions of PyPy3 for Python 3.6 and 3.7).
The problem seems to concentrate around the call to _kernel32.GetQueuedCompletionStatus()
in the function GetQueuedCompletionStatus()
in the module Lib/_overlapped.py
:
C:\Users\Berry\Documents\GitHub\mpyc\demos>pypy3 aes.py -M2
2021-12-29 12:31:28,842 Install package gmpy2 for better performance.
AES-128 en/decryption and AES-256 en/decryption.
AES polynomial: x^8+x^4+x^3+x+1
2021-12-29 12:31:28,858 Start MPyC runtime v0.8
2021-12-29 12:31:29,391 All 2 parties connected.
Plaintext: 00112233445566778899aabbccddeeff
AES-128 key: 000102030405060708090a0b0c0d0e0f
Ciphertext: 69c4e0d86a7b0430d8cdb78070b4c55a
Plaintext: 00112233445566778899aabbccddeeff
AES-256 key: 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f
Ciphertext: 8ea2b7ca516745bfeafc49904b496089
Plaintext: 00112233445566778899aabbccddeeff
Traceback (most recent call last):
File "aes.py", line 136, in <module>
mpc.run(main())
File "C:\Users\Berry\Documents\GitHub\mpyc\mpyc\runtime.py", line 185, in run
return self._loop.run_until_complete(f)
File "C:\pypy3.8-v7.3.7-win64\Lib\asyncio\base_events.py", line 603, in run_until_complete
self.run_forever()
File "C:\pypy3.8-v7.3.7-win64\Lib\asyncio\windows_events.py", line 316, in run_forever
super().run_forever()
File "C:\pypy3.8-v7.3.7-win64\Lib\asyncio\base_events.py", line 570, in run_forever
self._run_once()
File "C:\pypy3.8-v7.3.7-win64\Lib\asyncio\base_events.py", line 1823, in _run_once
event_list = self._selector.select(timeout)
File "C:\pypy3.8-v7.3.7-win64\Lib\asyncio\windows_events.py", line 434, in select
self._poll(timeout)
File "C:\pypy3.8-v7.3.7-win64\Lib\asyncio\windows_events.py", line 783, in _poll
status = _overlapped.GetQueuedCompletionStatus(self._iocp, ms)
KeyboardInterrupt
Reducing the timeout from INFINITY to 1000 say, makes the call _kernel32.GetQueuedCompletionStatus()
return, but doesn't resolve the problem.
The problem is nondeterministic. Short-lived programs are still likely to finish, longer ones are more likely to get stuck.
Switching off the JIT compiler doesn't seem to make any difference. Except that the program gets slower;)
To reproduce the problem you could install MpyC with pip install git+https://github.com/lschoe/mpyc
, switch to the demos
directory, and try running pypy3 aes.py -M2
a few times until it gets stuck. The -M2
switch causes two parties to run on localhost. If it is omitted or changed to -M1
everything runs fine.
Hope this info sets you on the right track?!