Worker pool stop working after all processes crash
The worker keep a list of task that is running. It takes a new task only when the number of running task is less than the number of processes. This way we have 1 task per process and leave the other tasks to possible other worker process. The problem is that a task may never be removed if the process executing it crash badly (like with a stackoverflow). The threading Event is not set because the crashed process will never write result to the output queue.
If all the processes in the queue crash this way than the worker will never take any new task.
I do not really see what we can do for now. One possibility is to have a timeout on the task which will remove it from the list no matter if it is ready or not.