Hammerspoon
Question: Premiere Pro `AXFocusedUIElement`
In the past I was able to get the AXFocusedUIElement element from Adobe Premiere Pro.
But for some reason this does no longer work to figure out if the cursor is in a text field.
I am not sure if Adobe changed something here and they are no longer supporting any of the AX / Accessibility trackable things or if this is something about Hammerspoon?
Or does anybody know a workaround to figure out if the caret is in a text field / or any caret is active on screen besides using AXFocusedUIElement?
My intention is to support single keystroke keyboard shortcuts like just pressing s for some Hammerspoon actions but still be able to type in text fields.
Thanks and I hope we can figure this out together.
Cheers, Chris
Hey Chris, did you ever figure this out? I'm trying to figure out why AXFocusedUIElement can be nil while focused on some applications, and whether that's the fault of the application or the fault of something else.
I'm running into something similar with VSCode and the Vivaldi browser.
I have not figured this out and basically given up. It's sad actually. It'd make my shortcuts and commands so much more powerful if I could only trigger them when outside of a text element.
The root of the issue (as far as I understand it / on a very rudimentary level) is: those applications use some other form of interface rendering engine that does not work with the AXFocusedUIElement stuff macOS provides.
Well it's interesting, because this window also returns nil for AXFocusedUIElement, which is not something I expected:
Test code I used for this:
hs.timer.doEvery(1, function()
print(hs.axuielement.systemWideElement():attributeValue("AXFocusedUIElement"))
end)
I can't look into this further right now, but I find it's sadly telling that either hammerspoon or macOS AX elements are no longer used/supported. I also don't see much action here any more.
I still love my hammerspoon actions but have thought about migrating to different solutions a few times at this point. :(
The AXUIElement API is very much core to Mac accessibility, so it's definitely used. Unfortunately, accessibility is often an afterthought, so it's possible we're seeing that here with these programs. I'm a little surprised to see it in a core Mac UI though, let alone VSCode.
We use the Accessibility API extensively in CommandPost - might be worth having a look at our code:
https://commandpost.io
@latenitefilms thank you for this interesting project link. Though, Premiere Pro seems a tricky beast (not providing any of the AXFocusedUIElement stuff.
I did find this for Electron-based apps though, maybe this helps (@codecat):
-- Google Chrome needs this flag to turn on accessibility in the browser
axApp:setAttributeValue('AXEnhancedUserInterface', true)
-- Electron apps require this attribute to be set or else you cannot read the accessibility tree
axApp:setAttributeValue('AXManualAccessibility', true)
Source: https://balatero.com/writings/hammerspoon/retrieving-input-field-values-and-cursor-position-with-hammerspoon/
Some tests I have done:
This checkFocus function is triggered in my code via a "one key shortcut" inside the application. It's printing out all kinds of trying to find the focused element… but Premiere Pro specifically is super stubborn.
-- TESTING differentiating based on FocusedElement not text input
-- hs.logger.clearConsole() -- Clear console for fresh output
local function checkFocus()
print("\n--- Starting Focus Diagnostics ---")
local currentApp = hs.application.frontmostApplication()
if not currentApp then
print("ERROR: No frontmost application found.")
print("\n--- End of Focus Diagnostics ---")
return
end
print("Frontmost Application: " .. currentApp:title() .. " (ID: " .. currentApp:bundleID() .. ")")
-- Test 1: Hammerspoon's standard hs.uielement.focusedElement()
print("\n--- Test 1: hs.uielement.focusedElement() ---")
local hsFocusedElement = hs.uielement.focusedElement()
if hsFocusedElement then
print(" SUCCESS with hs.uielement.focusedElement():")
print(" Element: " .. tostring(hsFocusedElement))
print(" Role: " .. tostring(hsFocusedElement:attributeValue("AXRole")))
print(" Subrole: " .. tostring(hsFocusedElement:attributeValue("AXSubrole")))
print(" Value: " .. tostring(hsFocusedElement:attributeValue("AXValue")))
print(" Title: " .. tostring(hsFocusedElement:attributeValue("AXTitle")))
print(" Description: " .. tostring(hsFocusedElement:attributeValue("AXDescription")))
-- print(" Full inspection: " .. hs.inspect(hsFocusedElement)) -- Uncomment for very detailed output
else
print(" FAILURE: hs.uielement.focusedElement() returned nil.")
end
-- Test 2: AX API via application element
print("\n--- Test 2: Application's AXFocusedUIElement ---")
local axApp = hs.axuielement.applicationElement(currentApp)
-- Google Chrome needs this flag to turn on accessibility in the browser
axApp:setAttributeValue('AXEnhancedUserInterface', true)
-- Electron apps require this attribute to be set or else you cannot read the accessibility tree
axApp:setAttributeValue('AXManualAccessibility', true)
if not axApp then
print(" ERROR: Could not get AX application element for " .. currentApp:title())
else
-- Ensure these are commented out or removed if they cause issues.
-- axApp:setAttributeValue('AXEnhancedUserInterface', true)
-- axApp:setAttributeValue('AXManualAccessibility', true)
print(" axApp object: " .. tostring(axApp))
local appFocusedElement = axApp:attributeValue("AXFocusedUIElement")
if appFocusedElement then
print(" SUCCESS with axApp:attributeValue('AXFocusedUIElement'):")
print(" Element: " .. tostring(appFocusedElement))
print(" Role: " .. tostring(appFocusedElement:attributeValue("AXRole")))
print(" Subrole: " .. tostring(appFocusedElement:attributeValue("AXSubrole")))
print(" Value: " .. tostring(appFocusedElement:attributeValue("AXValue")))
print(" Title: " .. tostring(appFocusedElement:attributeValue("AXTitle")))
print(" Description: " .. tostring(appFocusedElement:attributeValue("AXDescription")))
-- print(" Full inspection: " .. hs.inspect(appFocusedElement)) -- Uncomment for very detailed output
else
print(" FAILURE: axApp:attributeValue('AXFocusedUIElement') returned nil.")
end
end
-- Test 3: AX API via system-wide element
print("\n--- Test 3: System-Wide AXFocusedUIElement ---")
local systemElement = hs.axuielement.systemWideElement()
if not systemElement then
print(" ERROR: Could not get AX system wide element.")
else
print(" SystemWideElement object: " .. tostring(systemElement))
local systemFocusedElement = systemElement:attributeValue("AXFocusedUIElement")
if systemFocusedElement then
print(" SUCCESS with systemElement:attributeValue('AXFocusedUIElement'):")
print(" Element: " .. tostring(systemFocusedElement))
print(" Role: " .. tostring(systemFocusedElement:attributeValue("AXRole")))
print(" Subrole: " .. tostring(systemFocusedElement:attributeValue("AXSubrole")))
print(" Value: " .. tostring(systemFocusedElement:attributeValue("AXValue")))
print(" Title: " .. tostring(systemFocusedElement:attributeValue("AXTitle")))
print(" Description: " .. tostring(systemFocusedElement:attributeValue("AXDescription")))
-- print(" Full inspection: " .. hs.inspect(systemFocusedElement)) -- Uncomment for very detailed output
else
print(" FAILURE: systemElement:attributeValue('AXFocusedUIElement') returned nil.")
end
end
-- Test 4: Targeting Windows/Panels - Focused Element & Children
print("\n--- Test 4: Window/Panel Focused Element & Children Iteration ---")
if currentApp then
local focusedWindow_hs = currentApp:focusedWindow() -- This is an hs.window object
if focusedWindow_hs then
print(" Focused Window Title (from hs.window): " .. tostring(focusedWindow_hs:title()))
print(" hs.window object: " .. tostring(focusedWindow_hs))
-- local focusedWindow_ax = axApp
local focusedWindow_ax = hs.axuielement.windowElement(focusedWindow_hs) -- Get its AXUIElement representation
if not focusedWindow_ax then
print(" ERROR: Could not get AXUIElement for the focused window.")
else
print(" AXUIElement for Window: " .. tostring(focusedWindow_ax))
print(" Window Role (from AXUIElement): " .. tostring(focusedWindow_ax:attributeValue("AXRole")))
print(" Window Subrole (from AXUIElement): " .. tostring(focusedWindow_ax:attributeValue("AXSubrole")))
local windowFocusedElement_ax = focusedWindow_ax:attributeValue("AXFocusedUIElement")
if windowFocusedElement_ax then
print(" SUCCESS finding focused AXUIElement directly from window's AXUIElement:")
print(" Focused Element (from window): " .. tostring(windowFocusedElement_ax))
print(" Role: " .. tostring(windowFocusedElement_ax:attributeValue("AXRole")))
print(" Subrole: " .. tostring(windowFocusedElement_ax:attributeValue("AXSubrole")))
print(" Value: " .. tostring(windowFocusedElement_ax:attributeValue("AXValue")))
else
print(" FAILURE: focusedWindow_ax:attributeValue('AXFocusedUIElement') returned nil. Will now iterate children of the window's AXUIElement...")
end
print("\n --- Iterating Window AXUIElement Children (max depth 3, max 10 children per level) ---")
-- The inspectChildren function expects an AXUIElement or UIEelement
-- Forward declaration for inspectChildren, used by processChildList
local inspectChildren
-- Helper function to process a list of child elements
local function processChildList(childElementList, parentElement, parentLevel, maxLvl, maxChildPerLvl)
print(string.rep(" ", parentLevel + 2) .. "Processing list of " .. #childElementList .. " items for parent (" .. tostring(parentElement:attributeValue("AXRole") or "UnknownRole") .. "):")
for i, child_element in ipairs(childElementList) do
if i > maxChildPerLvl then
print(string.rep(" ", parentLevel + 3) .. "... and " .. (#childElementList - maxChildPerLvl) .. " more children (not shown for this list)")
break
end
if not child_element then
print(string.rep(" ", parentLevel + 3) .. i .. ") Child element is nil (unexpected). Skipping.")
goto continue_process_child_loop -- Lua's goto requires the label to be visible in the block
end
local success_attr, role, subrole, value, title, id, isFocused
success_attr, role = pcall(function() -- pcall returns success, then actual return values or error
local R,S,V,T,I,F -- Local to the pcall function scope
R = child_element:attributeValue("AXRole")
S = child_element:attributeValue("AXSubrole")
V = child_element:attributeValue("AXValue")
T = child_element:attributeValue("AXTitle")
I = child_element:id()
F = child_element:attributeValue("AXFocused")
return R,S,V,T,I,F
end)
if not success_attr then
print(string.rep(" ", parentLevel + 3) .. i .. ") Error getting attributes for child element: " .. tostring(role)) -- role here is the error msg
goto continue_process_child_loop
else
-- Unpack results if pcall was successful (role still holds the first return value from the pcall'd function)
subrole = select(2, role, subrole, value, title, id, isFocused) -- This is tricky. Pcall returns success, then results.
-- If pcall is successful, 'role' contains the actual first result.
-- Need to re-assign all vars from pcall result.
local pcall_results = {role} -- role is the first result from pcall
for idx = 2, 6 do table.insert(pcall_results, select(idx, child_element:attributeValue("AXRole"))) end -- This is wrong way to get multiple results.
-- Correct way to get multiple return values from pcall:
-- success_attr, r, s, v, t, i, f = pcall(...)
-- if success_attr then role=r; subrole=s ... else err_msg = r end
-- Simpler: re-fetch inside the main print block for clarity, or just use what pcall returned.
-- For now, let's assume 'role' got the primary role, and we'll fetch others as needed, or simplify.
-- The pcall above for attribute fetching needs to be structured to return all values correctly or fetch one by one.
-- To keep it simple, let's fetch them individually with pcall for robustness inside the loop for now.
role = child_element:attributeValue("AXRole") -- Assuming this is safe if child_element is valid
subrole = child_element:attributeValue("AXSubrole")
value = child_element:attributeValue("AXValue")
title = child_element:attributeValue("AXTitle")
id = child_element:id()
isFocused = child_element:attributeValue("AXFocused")
end
print(string.rep(" ", parentLevel + 3) .. i .. ") Child ID: "..tostring(id)..", Role: " .. tostring(role) .. ", Subrole: " .. tostring(subrole))
if title and title ~= "" then print(string.rep(" ", parentLevel + 4) .. "Title: " .. tostring(title)) end
if value and value ~= "" then print(string.rep(" ", parentLevel + 4) .. "Value: [" .. tostring(value) .. "]") end
if isFocused then print(string.rep(" ", parentLevel + 4) .. ">>>> THIS CHILD IS FOCUSED (AXFocused=true) <<<<") end
if role and (role == "AXTextField" or role == "AXTextArea" or role == "AXComboBox" or role == "AXSearchField" or role == "AXStaticText") then
print(string.rep(" ", parentLevel + 4) .. "----> Potential text-related field found.")
end
inspectChildren(child_element, parentLevel + 1, maxLvl, maxChildPerLvl) -- Recursive call
::continue_process_child_loop::
end
end
-- Define the main inspectChildren function
inspectChildren = function(element, level, maxLevel, maxChildrenPerLevel)
if not element then
print(string.rep(" ", level + 2) .. "Debug: inspectChildren called with nil element at level " .. level)
return
end
if level > maxLevel then
return
end
local children_found_and_processed = false
local element_role_for_prints = tostring(element:attributeValue("AXRole") or "UnknownRole")
if type(element.children) == "function" then
local success_children_call, children_from_method = pcall(function() return element:children() end)
if success_children_call then
if children_from_method and #children_from_method > 0 then
print(string.rep(" ", level + 2) .. "Element ("..element_role_for_prints..") has " .. #children_from_method .. " children via standard :children() method at level " .. level)
processChildList(children_from_method, element, level, maxLevel, maxChildrenPerLevel)
children_found_and_processed = true
elseif children_from_method then
print(string.rep(" ", level + 2) .. "Element ("..element_role_for_prints.."): Standard :children() method returned an empty list.")
children_found_and_processed = true
else
print(string.rep(" ", level + 2) .. "Element ("..element_role_for_prints.."): Standard :children() method returned nil.")
children_found_and_processed = true
end
else
print(string.rep(" ", level + 2) .. "Error: Calling :children() on element "..tostring(element).." ("..element_role_for_prints..") failed: " .. tostring(children_from_method))
end
else
print(string.rep(" ", level + 2) .. "Info: Standard :children() method not available for Element " .. tostring(element) .. " ("..element_role_for_prints.."). Type is: " .. type(element.children) .. ".")
end
if not children_found_and_processed then
print(string.rep(" ", level + 2) .. "Attempting to find children in Element ("..element_role_for_prints..") via alternative attributes...")
local alternativeChildrenAttributes = {"AXVisibleChildren", "AXContents"}
local found_via_alternative = false
for _, attrName in ipairs(alternativeChildrenAttributes) do
local success_attr_val, potentialChildren = pcall(function() return element:attributeValue(attrName) end)
if success_attr_val then
if type(potentialChildren) == "table" and #potentialChildren > 0 then
local looksLikeAxuielementTable = true
for _, item_in_val in ipairs(potentialChildren) do
local is_ax, _ = pcall(function() item_in_val:attributeValue("AXRole") end)
if not (type(item_in_val) == "userdata" and is_ax) then
looksLikeAxuielementTable = false
break
end
end
if looksLikeAxuielementTable then
print(string.rep(" ", level + 3) .. "Found " .. #potentialChildren .. " potential children in attribute '" .. attrName .. "'. Iterating them:")
processChildList(potentialChildren, element, level, maxLevel, maxChildrenPerLevel)
found_via_alternative = true
children_found_and_processed = true
break
end
elseif type(potentialChildren) == "table" then
print(string.rep(" ", level + 3) .. "Attribute '" .. attrName .. "' is an empty table.")
else
print(string.rep(" ", level + 3) .. "Attribute '" .. attrName .. "' did not return a table of children (value: "..tostring(potentialChildren)..").")
end
else
print(string.rep(" ", level + 3) .. "Error reading attribute '"..attrName.."': "..tostring(potentialChildren))
end
end
if not found_via_alternative then
print(string.rep(" ", level + 2) .. "No children found via alternative attributes for Element ("..element_role_for_prints..").")
end
end
if not children_found_and_processed then
print(string.rep(" ", level + 2) .. "Element ("..element_role_for_prints..") at level " .. level .. ": No children found or processed through any method.")
end
end
inspectChildren(focusedWindow_ax, 1, 3, 10) -- Start at level 1 for children of window's AX element
end
else
print(" No focused window (hs.window) found for " .. currentApp:title())
end
end
print("\n--- End of Focus Diagnostics ---")
end
checkFocus()
Curious... can you just use Premiere's Extensions API to do what you want to do?
FWIW - I also work on Jumper - https://getjumper.io - so I know enough about the Premiere Extensions API to be dangerous!
Also useful: https://hyperbrew.co/resources/bolt-cep/
@latenitefilms thank you for these resources.
The thing I am specifically trying to do is to hot wire keyboard shortcuts (single key shortcuts) to map them to things like a combination of:
- activate current timeline
- jump to current cursor position
- start playback in 2x speed
I have not yet found a way to do this in Premiere Pro (even with Excalibur).
Have you contact Sir Ivan? This should be possible with the Premiere API.
From what I understand, Excalibur and similar plugins are struggling with the same issues. Even showing notes in the UI that one key shortcuts are not supported because they can't be differentiated in the UI between unfocused text area and focused text area.
I'm pretty flat out this week - but if you can give some more information/context on EXACTLY what you need, then I can try chatting with Sir Ivan and come up with a game plan. We're currently discussing exposing ALL of Spellbook's actions in CommandPost.
From what I understand, Excalibur and similar plugins are struggling with the same issues. Even showing notes in the UI that one key shortcuts are not supported because they can't be differentiated in the UI between unfocused text area and focused text area.
Indeed, I do have this issue in Excalibur (hence the note in the picture). No way to distinguish if text field is activated. I used Inspector to examine UIelements, sadly to no avail.
@chrisspiegl
I did find this for Electron-based apps though, maybe this helps
You can't set this on the system-wide UI element, so this does not help me, unfortunately. For my use-case, I may have to explore other ways to get the focused element.
I did end up reporting accessibility bugs to the applications in question though. Perhaps this is something that you guys could do for Premiere Pro as well?
CommandPost does a heap of tricks... can you use a screenshot of the UI element to detect if it's selected or not?