Enabling long-running, external asyncio functionality

[mindset-team]

Textual is considered as a front-end consuming events (e.g., using Redis' PUBSUB, full code will be posted as a Show&Tell after clearing issues below as this might be interesting to the community).

Below is generic code attempting a background asyncio task to run indefinitely while preserving all of Textual's functionality:

#!/usr/bin/env python

import asyncio
from datetime import datetime

from rich.align import Align
from textual.app import App
from textual.widget import Widget


class AsyncWidget(Widget):

    counter = 0

    async def async_functionality(self):
        while True:
            await asyncio.sleep(0.2)  # Mock async functionality
            self.counter += 1
            self.refresh()  # This is required for ongoing refresh
            self.app.refresh()  # Also required for ongoing refresh, unclear why, but commenting-out breaks live refresh.

    async def on_mount(self):
        await self.async_functionality()

    def render(self) -> Align:
        now = datetime.strftime(datetime.now(), "%H:%M:%S.%f")[:-5]
        text = f"{now}\nCounter: {self.counter}"
        return Align.center(text, vertical="middle")


class AsyncApp(App):
    async def on_load(self) -> None:
        await self.bind("escape", "quit", "Quit")

    async def on_mount(self) -> None:
        await self.view.dock(AsyncWidget())


AsyncApp.run(title="AsyncApp", log="async_app.log")

Running the app and immediately pressing the bound esc key terminates the program gracefully. Here's the log:

# WITH ASYNC FUNCTIONALITY, NO MOUSE EVENT

driver=<class 'textual.drivers.linux_driver.LinuxDriver'>
Load() >>> AsyncApp(title='Textual')
Mount() >>> AsyncApp(title='AsyncApp')
Mount() >>> DockView(name='DockView#1')
Mount() >>> AsyncWidget(name='AsyncWidget#1')
view.forwarded Key(key='escape')
Key(key='escape') >>> AsyncApp(title='AsyncApp')
ACTION AsyncApp(title='AsyncApp') quit
CLOSED AsyncApp(title='AsyncApp')
PROCESS END

However- a mouse click on the widget's body (which triggers a set_focus event) somehow locks further key functionality (i.e. esc doesn't work, view.forwarded Key doesn't fire as seen in the log below). Moreover, ctrl-c must be used to terminate:

# WITH ASYNC FUNCTIONALITY, MOUSE EVENT BREAKS TUI

driver=<class 'textual.drivers.linux_driver.LinuxDriver'>
Load() >>> AsyncApp(title='Textual')
Mount() >>> AsyncApp(title='AsyncApp')
Mount() >>> DockView(name='DockView#1')
Mount() >>> AsyncWidget(name='AsyncWidget#1')
set_focus AsyncWidget(name='AsyncWidget#1') <--- mouse clicked anywhere on body
Key(key='ctrl+c') >>> AsyncApp(title='AsyncApp') <-- only way to terminate
ACTION AsyncApp(title='AsyncApp') quit
CLOSED AsyncApp(title='AsyncApp')
PROCESS END

This is clearly related to the long-running async functionality we're attempting to implement. Commenting out the await self.async_functionality() from the widget's on_mount directive shows the expected behavior for Textual (mouse click works, down/up events trigger, esc quit works):

# WORKING EXAMPLE, W/O ASYNC FUNCTIONALITY

driver=<class 'textual.drivers.linux_driver.LinuxDriver'>
Load() >>> AsyncApp(title='Textual')
Mount() >>> AsyncApp(title='AsyncApp')
Mount() >>> DockView(name='DockView#1')
Mount() >>> AsyncWidget(name='AsyncWidget#1')
set_focus AsyncWidget(name='AsyncWidget#1')
MouseDown(x=56, y=18, button=1) >>> AsyncWidget(name='AsyncWidget#1')
MouseUp(x=56, y=18, button=1) >>> AsyncWidget(name='AsyncWidget#1')
Click(x=56, y=18, button=1) >>> AsyncWidget(name='AsyncWidget#1')
Key(key='escape') >>> AsyncApp(title='AsyncApp')
ACTION AsyncApp(title='AsyncApp') quit
CLOSED AsyncApp(title='AsyncApp')
PROCESS END

Any advise on how to implement this long-running async functionality while interacting with the TUI would be great.

[update: posted on stackoverflow]

[mharig]

You may have a look at my solution for another, but IMHO related, problem here

[mindset-team]

@mharig , yes I saw the thread pool discussion thread. As even-driven, i/o bound challenges go, the one i'm presenting (using Redis/PubSub instrumentation) seems to be an ideal use-case for asyncio. You could have many streams consumed by the TUI very easily - managing multiple threads for this and sharing objects/data with the main TUI app all seem like a lot of overhead with little benefit. Do you see the advantage to a thread-based approach?

[mharig]

Using asyncio like you did (but without the self.app.refresh(), didn't know that before, and have to try it out, yet), I never get real concurrency. All examples I found and everything I tried blocked until all long-running async updates finished. So the advantage is: it works for me :-)
And it is the only solution I found. But that may be only because of my poor understanding of asyncio.

[mindset-team]

Resolution came from WillM and others (see accepted answer on stackoverflow) - long running async corotines should be wrapped as tasks, and not awaited. Namely, replace this code:

async def on_mount(self):
    await self.async_functionality()

with this one:

async def on_mount(self):
    asyncio.create_task(self.async_functionality())

await is not used as this function never returns a value anyway.
Also, as now the asyncio event loop is non-blocked, it seems that the self.app.refresh() call is unnecessary.

[mharig]

That didn't work for my use case. YMMV.