Signal handling & exceptions in @def_extern callbacks

Hi all,

This is a follow up on #152 (closed).

Consider the following CFFI module:

import cffi
ffi = cffi.FFI()
ffi.cdef('''
extern "Python" void foo_cb(void);
void do_stuff_while_under_lock(void (*cb)(void));
''')
ffi.set_source('_def_extern_cb_signals', r'''
#include <stdio.h>
static void acquire_system_lock(void)
{
    printf("acquired lock\n");
}
static void release_system_lock(void)
{
    printf("released lock\n");
}
static void do_stuff_while_under_lock(void (*cb)(void))
{
    acquire_system_lock();
    cb();
    release_system_lock();
}
''')
ffi.compile()

And then, the following test script to demonstrate the problem:

import time

from _def_extern_cb_signals import ffi, lib

@ffi.def_extern()
def foo_cb():
    print('in foo_cb, sleeping for 5s')
    time.sleep(5)

print('calling do_stuff_while_under_lock(foo_cb)')
lib.do_stuff_while_under_lock(lib.foo_cb)
print('still alive, you have failed interrupting me')

When running the script, interrupt it with ctrl-c:

$ python3 test.py 
calling do_stuff_while_under_lock(foo_cb)
acquired lock
in foo_cb, sleeping for 5s
^CFrom cffi callback <function foo_cb at 0x7f08c1e361e0>:
Traceback (most recent call last):
  File "test.py", line 8, in foo_cb
    time.sleep(5)
KeyboardInterrupt
released lock
still alive, you have failed interrupting me

Same problem occurs when registering a signal handler that performs sys.exit().

Adding a onerror handler to the @def_extern does not solve the problem since the callback cannot return any value nor do anything. The exception has absolutely no way to bubble up the stack to the actual python thread that called do_stuff_while_under_lock. Also, calling os._exit() in a signal handler allows to exit, but the fake lock is not released, which is a problem.

In my opinion, signals should be blocked while invoking python callbacks to avoid such issues. Maybe do that on certain conditions, something like:

@ffi.def_extern(block_signals=True)
def foo_cb():
    time.sleep(5)

What do you think?

By the way, I have attempted a quick & dirty patch to see if it fixes the problem but it does not work. It looks like my code is not called at all. I am missing something here...

diff --git a/c/_cffi_backend.c b/c/_cffi_backend.c
--- a/c/_cffi_backend.c
+++ b/c/_cffi_backend.c
@@ -8,12 +8,14 @@
 #include <windows.h>
 #include "misc_win32.h"
 #else
+#define _POSIX_C_SOURCE 200809L
 #include <stddef.h>
 #include <stdint.h>
 #include <dlfcn.h>
 #include <errno.h>
 #include <ffi.h>
 #include <sys/mman.h>
+#include <signal.h>
 #endif
 
 /* this block of #ifs should be kept exactly identical between
@@ -6192,9 +6194,15 @@
 {
     save_errno();
     {
+        sigset_t set, oset;
+        sigfillset(&set);
+        /* block all signals while invoking python callback */
+        pthread_sigmask(SIG_BLOCK, &set, &oset);
         PyGILState_STATE state = gil_ensure();
         general_invoke_callback(1, result, (char *)args, userdata);
         gil_release(state);
+        /* restore sigmask to original state */
+        pthread_sigmask(SIG_SETMASK, &oset, NULL);
     }
     restore_errno();
 }

Thanks in advance for your help!