Asyncio exception handler: not getting called until event loop thread stopped

Question

I am setting an exception handler on my asyncio event loop. However, it doesn't seem to be called until the event loop thread is stopped. For example, consider this code:

def exception_handler(loop, context):
    print('Exception handler called')

loop = asyncio.get_event_loop()

loop.set_exception_handler(exception_handler)

thread = Thread(target=loop.run_forever)
thread.start()

async def run():
    raise RuntimeError()

asyncio.run_coroutine_threadsafe(run(), loop)

loop.call_soon_threadsafe(loop.stop, loop)

thread.join()

This code prints "Exception handler called", as we might expect. However, if I remove the line that shuts-down the event loop (loop.call_soon_threadsafe(loop.stop, loop)) it no longer prints anything.

I have a few questions about this:

Am I doing something wrong here?
Does anyone know if this is the intended behaviour of asyncio exception handlers? I can't find anything that documents this, and it seems a little strange to me.

I'd quite like to have a long-running event loop that logs errors happening in its coroutines, so the current behaviour seems problematic for me.

did youraise any error ? asyncio.run_coroutine_threadsafe(run(), loop) you started run function without thread(for function run run != run()) — dsgdfg
– dsgdfg, Commented Nov 21, 2017 at 13:00
@dsgdfg: I'm not quite sure what you mean, but bear in mind that run is a coroutine, so calling run() does not run it. — Neil
– Neil, Commented Nov 21, 2017 at 16:06

Udi · Accepted Answer · 2017-11-22 10:17:27Z

There are a few problems in the code above:

stop() does not need a parameter
The program ends before the coroutine is executed (stop() was called before it).

Here is the fixed code (without exceptions and the exception handler):

import asyncio
from threading import Thread


async def coro():
    print("in coro")
    return 42


loop = asyncio.get_event_loop()
thread = Thread(target=loop.run_forever)
thread.start()

fut = asyncio.run_coroutine_threadsafe(coro(), loop)

print(fut.result())

loop.call_soon_threadsafe(loop.stop)

thread.join()

call_soon_threadsafe() returns a future object which holds the exception (it does not get to the default exception handler):

import asyncio
from pprint import pprint
from threading import Thread


def exception_handler(loop, context):
    print('Exception handler called')
    pprint(context)


loop = asyncio.get_event_loop()

loop.set_exception_handler(exception_handler)

thread = Thread(target=loop.run_forever)
thread.start()


async def coro():
    print("coro")
    raise RuntimeError("BOOM!")


fut = asyncio.run_coroutine_threadsafe(coro(), loop)
try:
    print("success:", fut.result())
except:
    print("exception:", fut.exception())

loop.call_soon_threadsafe(loop.stop)

thread.join()

However, coroutines that are called using create_task() or ensure_future() will call the exception_handler:

async def coro2():
    print("coro2")
    raise RuntimeError("BOOM2!")


async def coro():
    loop.create_task(coro2())
    print("coro")
    raise RuntimeError("BOOM!")

You can use this to create a small wrapper:

async def boom(x):
    print("boom", x)
    raise RuntimeError("BOOM!")


async def call_later(coro, *args, **kwargs):
    loop.create_task(coro(*args, **kwargs))
    return "ok"


fut = asyncio.run_coroutine_threadsafe(call_later(boom, 7), loop)

However, you should probably consider using a Queue to communicate with your thread instead.

Great, thanks Udi. To summarise, it was actually my erroneous call to loop.call_soon_threadsafe(loop.stop, loop) that caused the exception handler to be invoked, and not the run() coroutine. So, asyncio.run_coroutine_threadsafe never seems to cause the exception handler to be invoked.

Collectives™ on Stack Overflow

Asyncio exception handler: not getting called until event loop thread stopped

1 Answer 1

1 Comment

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

1 Comment

Your Answer

Sign up or log in

Post as a guest

Linked

Related