Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • pypy pypy
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 675
    • Issues 675
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 13
    • Merge requests 13
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • PyPyPyPy
  • pypypypy
  • Issues
  • #3630
Closed
Open
Issue created Jan 01, 2022 by Delgan@Delgan

Deadlock while creating a "multiprocessing.Queue()" with acquired lock

Hi!

I noticed a deadlock on PyPy 7.3.7 (both Python 3.7 and Python 3.8) involving:

  • os.register_at_fork()
  • threading.Lock
  • multiprocessing.SimpleQueue
  • multiprocessing.set_start_method()

I tested it on Arch Linux x64 locally. I'm not sure if it happens on Windows without calling set_start_method() (used here to select the "spawn" method which is the default on Windows).

Here is a minimal reproducible example:

import multiprocessing
import threading
import os


if __name__ == "__main__":
    multiprocessing.set_start_method("spawn")

    lock = threading.Lock()

    def acquire():
        print("Acquiring lock")
        lock.acquire()
        print("Acquired lock")

    def release():
        print("Releasing lock")
        lock.release()
        print("Relased lock")

    os.register_at_fork(before=acquire, after_in_parent=release, after_in_child=release)

    with lock:
        multiprocessing.SimpleQueue()

You should notice it blocks on the "Acquiring lock" message, while it works fine using CPython.

For more context about the use case: acquiring and releasing all existing locks during a fork avoid possible deadlocks due to child process inheriting a acquired lock. It looks creaing a multiprocessing.SimpleQueue() causes registered handlers to be called while no process is spawned.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking