Preemptible Lua widget scripts?

[jfrickmann]

I have now come to the point in the development of SoarETX where I hit the dreaded CPU limit. And I therefore I have the choice between making my widget script divide jobs like reading a score file over multiple cycles with the added complexity of scripts, or figure out a way to have widget scripts preempt like the other scrips 🤓

Standalone scripts use a "backup" buffer for Lua, and this gets copied over to the active LCD buffer. This was already in place before I implemented preemption, and that was very fortunate for me, because I could easily make the buffer copying contingent on Lua having finished the run function call.

To my surprise, it appears that the above case is the only place that this "backup" buffer is used in EdgeTX (#1009)! So therefore I wonder if we could make the "backup" LCD buffer the Lua buffer for widgets also?

What I have in mind is the following process for making the widget scripts preemptible:

1. Run the widget scripts from guiMain() before MainWindow::run(). The widgets have a visible flag added to tell if they must call background() or refresh(...) here. If a widget script is completed then refreshed is set to true and otherwise false.
2. checkEvents() sets visible to false and calls invalidate().
3. paint() copies its rectangle (or the entire screen in fullscreen mode) from the "backup" to the active LCD buffer if the widget is refreshed, and sets visible to true.

[wimalopaan]

I don't know how the scheduler in FreeRTOS works, but is there now way that the OS preempts the LUA-script without interaction from the script itself? What you suggest is kind of coorparative multitasking if I understand that correctly.

[jfrickmann]

What I suggest here, is the same type of preemption that I made for the other scripts. And you are right that it is a kind of cooperative multitasking in the sense that it is the Lua interpreter that decides when to take a break running the script.
The way it worked in OpenTX, is that a Lua debug hook is set, so every time the interpreter has run 100 instructions, it calls the hook function. The hook function would count from 0 to 100% CPU usage, and if you hit 100% (i.e. after 100 x 100 instructions) then it would kill the script with the "CPU limit" error.
What I did, was I modified this, so instead of killing the script, it looks at the clock, and if it is more than 50 ms since the cycle started, then it will take a break and continue in the next cycle. It still only runs one script at a time, i.e. it finishes one script before it runs the next. So we do not have concurrency issues where two scripts run at once.
So it does not in any way involve the kernel scheduler.

[wimalopaan]

Ok, sounds reasonable.

But why do yyou need the visible-flag for the widget? refresh() is called if visible, background() is called if not.