-
Notifications
You must be signed in to change notification settings - Fork 583
CoroutinesLeveraging
The examples presented in this document required Hammerspoon 0.9.79 or newer; or a development build of Hammerspoon built from https://github.com/Hammerspoon/hammerspoon/tree/ea693f1d25d3a5e793575ecc68f94452255e89e1 or newer.
Now that coroutines are formally being supported within Hammerspoon, this document will outline a way to utilize them to hopefully make your installation more responsive.
This is by no means the only way to utilize Coroutines within Hammerspoon, or within Lua itself. It is however a fairly simple one that will hopefully provide tangable results with a minimum of change to your existing code.
Lua, the scripting language Hammerspoon uses for user defined functionality, is not a multithreaded language. At its simplest, this means that the Lua engine or interpreter (used interchangably here for simplicity) can only do one thing at a time in a sequential manner. When a long running task is being performed, nothing else can occur until the task is completed.
One method that Lua offers to mitigate this is known as Coroutines. Again, simplifying a bit, coroutines are a way to pause a long running task to allow the Lua engine to perform another, usually shorter, task so that both tasks can progress in a close to simultaneous manner.
Complicating things a little bit is the way macOS applications, like Hammerspoon, are structured. A macOS application is comprised of multiple application threads, most of which perform tasks that the user, and often even the developers of the application don't have to worry about. The interesting parts happen on the primary or "main" application thread, such as user interface updates, event handling, and in the case of Hammerspoon, running the Lua engine. While the Lua engine is running lua code, however, the main application thread can't perform its other tasks -- like updating the UI or handling events, timers, delegate callbacks, hotkeys, etc.
What follows is a methodology for breaking up a long running task into a format that utilizes coroutines to allow the main application thread of Hammerspoon time to perform its updates and then quickly get back to working on our long running task.
It should be noted that this approach will work best when much of the processing time for the long running task is spent within the Lua engine processing Lua commands; if your slowdown is occuring within a single invocation of a function or method written in Objective-C provided by one of the Hammerspoon modules, then it will be necessary to figure out a way to fix or speed up that code instead, or break it up into smaller chunks which can be iterated through to invoke the Objective-C code multple times by the Lua engine which can then leverage the approach outlined here.
For this discussion, consider this fairly generic function outline:
myFunction = function(...)
-- initial setup for the task
-- e.g. validate arguments, setup initial variables, etc.
-- long running task, often, but not always within a loop
-- clean up for the task
-- possibly including using the results of long running task
endIt is important to note that for the approach outlined here, the function myFunction is not returning anything -- it is self contained, performing a specific task and acting on the results in whatever way is appropriate before returning from the function. Sharing Coroutine Results describes some possible approaches to consider if it truly makes more sense to code in such a way that another code section or block requires the results of myFunction rather than include such code within the clean-up portion of myFunction.
As described in this generic function outline, there are basically three sections. It is assumed that the initial setup and clean up sections are relatively streamlined and efficient -- if they aren't, you might need to reconsider how you have designed your function.
To leverage coroutines to make this code more responsive, we need to modify the code to something like this:
myFunction = function(...)
-- initial setup for the task
-- e.g. validate arguments, setup initial variables, etc.
local longTask
longTask = coroutine.wrap(function(...)
-- if you refer to local variable defined within the setup section, they
-- will become up-values to the coroutine; or if it makes more sense,
-- pass them in as initial arguments
local exitCondition = false
-- long running task, often, but not always within a loop, e.g.
while not exitCondition do
-- do one iteration of the long running task, changing exitCondition to
-- true when we have finished
coroutine.applicationYield()
end
-- clean up for the task
-- possibly including using the results of long running task
longTask = nil -- by referencing longTask within the coroutine function
-- it becomes an up-value so it won't be collected
end)
longTask(...) -- this is where we pass initial arguments into the coroutine
endThe long running task and the clean up sections have been moved into the coroutine function. For the example here, a while loop is used to repeat the task until exitCondition becomes true.
An important line to note here is coroutine.applicationYield(). This function has been added to Hammerspoon (it is not part of the stock Lua coroutine library) and it performs a couple of tasks for us:
- sets up a timer to restart the coroutine function, by default in
hs.math.minFloatseconds - yields the coroutine with
coroutine.yield
hs.math.minFloat is the smallest floating point number recognized by both Lua and hs.timer and represents the smallest timeslice we can assign to a timer. This ensures that our coroutine is restarted as quickly as possible, but still allows Hammerspoon to respond to other events and perform the necessary application housekeeping that keeps it responsive. You can provide a floating point argument to coroutine.applicationYield(delay) if you know that your code doesn't need to be returned to immediately (for example if you're waiting on a response to an hs.http.asyncGet -- see further exmaple below) and want to give Hammerspoon more time to handle other events.
When iterating and performing whatever data manipulation you require, you don't have to invoke coroutine.applicationYield with every iteration; you could just as easily create a counter (or whatever other logic you want to use) and do something like:
-- define as local count = 0 outside of the while loop; this segment is just
-- to replace the lone coroutine.applicationYield line in the above example
count = count + 1
if count % 10 == 0 then -- only yield every 10 iterations
count = 0
coroutine.applicationYield()
endIf your long running code is not in a loop but is actually just a lot of lua commands, you can intersperse coroutine.applicationYield at various points where it makes sense and the code will resume immediately after where the last coroutine.applicationYield was invoked.
- As mentioned above, we can adjust the time
coroutine.applicationYieldwill delay before resuming the coroutine. This may be useful when we know we're waiting on something that will take a while and don't want to waste cycles immediatly yielding again while waiting for the results. An obvious example ishs.http.asyncGet. An example of how we might handle this follows:
myFunction = function(...)
-- initial setup for the task
-- e.g. validate arguments, setup initial variables, etc.
local longTask
longTask = coroutine.wrap(function(...)
-- if you refer to local variable defined within the setup section, they
-- will become up-values to the coroutine; or if it makes more sense,
-- pass them in as initial arguments
-- predeclare these so they are upvalues to the get request callback
local status, body, hdrs = nil, nil, nil
local getDone = false
hs.http.asyncGet("http://site.com", {}, function(s,b,h)
status, body, hdrs = s, b, h
getDone = true
end)
while not getDone do
coroutine.applicationYield(2.0) -- no reason to jump back here immediatly
end
local exitCondition = false
-- long running task, often, but not always within a loop, e.g.
while not exitCondition do
-- do one iteration of the long running task, changing exitCondition to
-- true when we have finished
coroutine.applicationYield() -- now resume as quickly as possible
end
-- clean up for the task
-- possibly including using the results of long running task
longTask = nil -- by referencing longTask within the coroutine function
-- it becomes an up-value so it won't be collected
end)
longTask(...) -- this is where we pass initial arguments into the coroutine
end- If your coroutine function should be resumed when some other activity occurs, rather than after a set amount of time, (for example when the user types a specific hotkey) then it may make more sense to replace
coroutine.applicationYieldwith something else. Consider:
myFunction = function(...)
-- initial setup for the task
-- e.g. validate arguments, setup initial variables, etc.
local longTask
longTask = coroutine.wrap(function(...)
-- if you refer to local variable defined within the setup section, they
-- will become up-values to the coroutine; or if it makes more sense,
-- pass them in as initial arguments
local hk
local exitCondition = false
-- long running task, often, but not always within a loop, e.g.
while not exitCondition do
-- do one iteration of the long running task, changing exitCondition to
-- true when we have finished
-- make sure we don't recreate the hotkey each time it repeaats
if not hk then
-- resume each time the user taps or holds the spacebar
hk = hs.hotkey.bind({}, "space", function()
-- no arguments even if we initially supplied them
-- as we're just resuming
longTask()
end, function()
hk:disable()
hk = nil
longTask()
end, function()
longTask()
end)
end
coroutine.yield() -- instead of coroutine.applicationYield
end
-- clean up for the task
-- possibly including using the results of long running task
longTask = nil -- by referencing longTask within the coroutine function
-- it becomes an up-value so it won't be collected
end)
longTask(...) -- this is where we pass initial arguments into the coroutine
endFor our purposes, the usage of coroutines as outlined in this document focuses on them being self contained and not passing data in or out while they are executing the long task. The reason for this is because I wanted to show a simple way to make Hammerspoon more responsive and by scheduling the resuming of the coroutine with a timer created by a generic helper function we greatly simplify the supporting code necessary for our coroutines.
To leverage the stock Lua coroutine library's ability to pass arguments in and out of a coroutine with coroutine.yield and coroutine.resume, you need to know exactly what code blocks are issuing the resume commands to make sure that they are able to pass in the arguments or receive the arguments passed out with yield and usually requires specific wrappers for each and every separate coroutine. You will likely need to set up your own timers or triggers to keep your code running as timely as possible.
Handling this type of data passing is beyond the scope of this document, but you can find out more by consulting the official Lua Documentation or a Lua reference book -- I've often used the third and fourth edition of Programming in Lua to good effect. Note that if you refer to the online version of this manual, it only covers Lua 5.1 -- there may be some minor differences, though for coroutines, thus far I've only observed that coroutine.isyieldable is an addition to 5.3.
However, we do have some other options, though they may give us a little less precise control over exactly when and where in the code flow data is received and work best when the long running code utilizes loops.
-
At the simplest end of the spectrum, we can use global variables and check at various points within the coroutine to see if what we need/want has been set or changed; or set a global variable and assume that another code block on a timer or triggered by another event will detect that it has been set or changed when its next triggered; however, many experienced programmers like to avoid using global variables to keep things cleaner and avoid possible name collisions (I'm one of those pesky types, so I won't give an example here).
-
Using callback functions within the coroutine should be a familiar pattern to Hammerspoon users. Callback functions can be used to send data out of the coroutine as well as accept new/updated data as values, which could be used to change or update state within the coroutine, but only when the callbacks are actually invoked. If the callback only sometimes needs to pass data in, you'll need to add logic to determine whether there is new data or not.
myFunction = function(cbInterim, cbAtEnd, ...)
-- initial setup for the task
-- e.g. validate arguments, setup initial variables, etc.
local longTask
longTask = coroutine.wrap(function(...)
-- if you refer to local variable defined within the setup section, they
-- will become up-values to the coroutine; or if it makes more sense,
-- pass them in as initial arguments
local exitCondition = false
-- long running task, often, but not always within a loop, e.g.
while not exitCondition do
-- do one iteration of the long running task, changing exitCondition to
-- true when we have finished
-- pass whatever we need to the interim callback and receive the results
--
-- this doesn't have to occur with each iteration -- we could use a
-- counter and only trigger it every `n` times, or use other logic to
-- detemrine if/when we want to notify an outside code block. Note that
-- this callback is also how we determine if there is new data to be
-- brought in, so factor that into your logic for determining if/when to
-- invoke the interim callback
local results = table.pack(cbInterim(...))
if results.n > 0 then
-- something was returned, decide if we need to update anything
end
coroutine.applicationYield() -- then yield
end
-- pass whatever we need to at completion of the task. As this is a completion
-- event, we'll assume that it's not going to return anything, though adjust as
-- necessary for your requirements
cbAtEnd(...)
-- clean up for the task
-- possibly including using the results of long running task
longTask = nil -- by referencing longTask within the coroutine function
-- it becomes an up-value so it won't be collected
end)
longTask(...) -- this is where we pass initial arguments into the coroutine
end- Using
hs.watchablewe trigger the codeblocks we need when passing data out of the coroutine automatically when values within the published table are changed and more than one codeblock can subscribe for notifications (unlike the callback approach which limits us to just one receiver -- or requires the receiver itself to be written in such a way as to notify other code blocks). The timing within a specific code flow for incoming data is a little less precise, but can still be detected almost immediately with the use of a boolean flag when the long running code is within a loop:
myFunction = function(...)
-- initial setup for the task
-- e.g. validate arguments, setup initial variables, etc.
-- if we pass in true, then we can accept data from the outside
local myFunctionWatchables = hs.watchable.new("myFunctionWatchables" [, true])
-- only required if we set `true` when creating myFunctionWatchables:
-- we set this so outside code can verify we exist and not throw an error
-- for trying to change a non-existent (and thus assumed unidirectional)
-- watchable
myFunctionWatchables.running = true
-- used within long task to see if something has changed
local dataChanged = false
-- to reduce invoking this when assigning "output" values, specify
-- specific keys we want to watch for. Repeat for as many "input" keys
-- as required
-- could specify "*" but then would have to check against things we're
-- passing "out" (in this example `interimData` and `finalData`) and
-- that would incur additional overhead
local internalWatchers = {
hs.watchable.watch("myFunctionWatchables.quit", function(w, p, k, o, n)
dataChanged = true
end),
}
local longTask
longTask = coroutine.wrap(function(...)
-- if you refer to local variable defined within the setup section, they
-- will become up-values to the coroutine; or if it makes more sense,
-- pass them in as initial arguments
local exitCondition = false
-- long running task, often, but not always within a loop, e.g.
while not exitCondition do
-- do one iteration of the long running task, changing exitCondition to
-- true when we have finished
-- only required if we set `true` when creating myFunctionWatchables:
if dataChanged then -- something changed
if myFunctionWatchables.quit == true then -- exit early
exitCondition = true
end
dataChanged = false
end
-- this will trigger notification to subscribed watchers... we don't
-- necessarily have to do it with each iteration, only when something we
-- want to pass out changes or is initially set
myFunctionWatchables.interimData = someValue
coroutine.applicationYield() -- then yield
end
myFunctionWatchables.finalData = someValue -- to pass out to outside watchers
-- clean up for the task
-- possibly including using the results of long running task
longTask = nil -- by referencing longTask within the coroutine function
-- it becomes an up-value so it won't be collected
-- only required if we set `true` when creating myFunctionWatchables:
-- clean up so we don't leave things around that are no longer active
for i,v in ipairs(internalWatchers) do
v:release() -- it's done, so release our watchers
end
internalWatchers = nil
end)
longTask(...) -- this is where we pass initial arguments into the coroutine
endWith this example, outside code can trigger an early termination with:
-- we check to see if `running` has been set so that the change won't trigger an
-- error for trying to change a non-existent (and thus assumed unidirectional)
-- watchable
if hs.watchable.watch("myFunctionWatchables.running"):value() then
hs.watchable.watch("myFunctionWatchables.quit"):change(true)
endAnd to receive the updates to myFunctionWatchables.interimData and myFunctionWatchables.finalData, outside code blocks can use something along the lines of:
local outsideWatchers -- predeclare so we can use it inside ourselves
outsideWatchers = {
hs.watchable.watch("myFunctionWatchables.interimData", function(w, p, k, o, n)
-- see docs for `hs.watchable` for full details, but `n` will contain the new
-- value, so do whatever you need to with it
end),
hs.watchable.watch("myFunctionWatchables.finalData", function(w, p, k, o, n)
-- see docs for `hs.watchable` for full details, but `n` will contain the new
-- value, so do whatever you need to with it
-- clean up so we don't leave things around that are no longer active
for i,v in ipairs(outsideWatchers) do
v:release() -- it's done, so release our watchers
end
outsideWatchers = nil
end),
}This introduction by no means covers all of the possibilities, but should hopefully allow you to get started moving your long running lua code into more responsive and friendly coroutines with minimal changes.
At present, it is not possible to yield or resume within long running Objective-C code used by the Hammerspoon modules for accessing the macOS Objective-C runtime environment; hopefully most of these are already fast enough that by coding your Lua code appropriately these do not cause Hammerspoon to become unnecessarily unresponsive.
If you believe that a compiled function or method needs to be looked at, you are welcome to submit your own tweaks or code updates by submitting a pull request to https://github.com/Hammerspoon/hammerspoon or by submitting an issue to https://github.com/Hammerspoon/hammerspoon/issues and our developers will examine the specifics and determine if a different approach within your code might help or if code changes to Hammerspoon or its modules needs to be considered.