Asynchronous KeyboardInterrupt and multithreading

Question

It seems that asynchronous signals in multithreaded programs are not correctly handled by Python. But, I thought I would check here to see if anyone can spot a place where I am violating some principle, or misunderstanding some concept.

There are similar threads I've found here on SO, but none that seem to be quite the same.

The scenario is: I have two threads, reader thread and writer thread (main thread). The writer thread writes to a pipe that the reader thread polls. The two threads are coordinated using a threading.Event() primitive (which I assume is implemented using pthread_cond_wait). The main thread waits on the Event while the reader thread eventually sets it.

But, if I want to interrupt my program while the main thread is waiting on the Event, the KeyboardInterrupt is not handled asynchronously.

Here is a small program to illustrate my point:

#!/usr/bin/python
import os
import sys
import select
import time
import threading

pfd_r = -1
pfd_w = -1
reader_ready = threading.Event()

class Reader(threading.Thread):
    """Read data from pipe and echo to stdout."""
    def run(self):
        global pfd_r
        while True:
            if select.select([pfd_r], [], [], 1)[0] == [pfd_r]:
                output = os.read(pfd_r, 1000)
                sys.stdout.write("R> '%s'\n" % output)
                sys.stdout.flush()
                # Suppose there is some long-running processing happening:
                time.sleep(10)
                reader_ready.set()


# Set up pipe.
(pfd_r, pfd_w) = os.pipe()
rt = Reader()
rt.daemon = True
rt.start()

while True:
    reader_ready.clear()
    user_input = raw_input("> ").strip()
    written = os.write(pfd_w, user_input)
    assert written == len(user_input)
    # Wait for reply -- Try to ^C here and it won't work immediately.
    reader_ready.wait()

Start the program with './bug.py' and enter some input at the prompt. Once you see the reader reply with the prefix 'R>', try to interrupt using ^C.

What I see (Ubuntu Linux 10.10, Python 2.6.6) is that the ^C is not handled until after the blocking reader_ready.wait() returns. What I expected to see is that the ^C is raised asynchronously, resulting in the program terminating (because I do not catch KeyboardInterrupt).

This may seem like a contrived example, but I'm running into this in a real-world program where the time.sleep(10) is replaced by actual computation.

Am I doing something obviously wrong, like misunderstanding what the expected result would be?

Edit: I've also just tested with Python 3.1.1 and the same problem exists.

Thomas Orozco · Accepted Answer · 2011-11-18 17:51:25Z

1

The wait() method of a threading._Event object actually relies on a thread.lock's acquire() method. However, the thread documentation states that a lock's acquire() method cannot be interrupted, and that any KeyboardInterrupt exception will be handled after the lock is released.

So basically, this is working as intended. Threading objects that implement this behavior rely on a lock at some point (including queues), so you might want to choose another path.

answered Nov 18, 2011 at 17:51

Thomas Orozco

55.6k12 gold badges120 silver badges120 bronze badges

Sign up to request clarification or add additional context in comments.

4 Comments

tdenniston Over a year ago

Great, thanks for the explanation. I might make the argument that this particular "feature" is in fact a bug.... I guess I'll have to figure out another way to synchronize my threads. It just seems silly that I can't use a built in synchronization primitive to do that.

Thomas Orozco Over a year ago

Well you could go for the pretty hackish: while 1: (Line break) if reader_ready.wait(1): break

tdenniston Over a year ago

I could do that, or use reader_ready.wait(99999) as the timeout. It seems to then do a sleep occasionally, and so the keyboard interrupt can be handled then. Still, a hack.

Thomas Orozco Over a year ago

Yeah, I noticed that the timeout actually didn't matter after posting this comment. I agree with your comment nonetheless!

GingerNinja23 · Accepted Answer · 2015-06-14 16:14:08Z

0

Alternatively, you could also use the pause() function of the signal module instead of reader_ready.wait(). signal.pause() is a blocking function and gets unblocked when a signal is received by the process. In your case, when ^C is pressed, SIGINT signal unblocks the function.

According to the documentation, the function is not available for Windows. I've tested it on Linux and it works. I think this is better than using wait() with a timeout.

answered Jun 14, 2015 at 16:14

GingerNinja23

1622 silver badges11 bronze badges

Collectives™ on Stack Overflow

Asynchronous KeyboardInterrupt and multithreading

2 Answers 2

4 Comments

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

4 Comments

Comments

Your Answer

Sign up or log in

Post as a guest

Related