Persistent Screen Search Labels? Scripting with screen search labels?

[C-Loftus]

I use fluent search for the purposes of computer accessibility. I was wondering if it is possible to use screen search and reload the labels automatically with each press without needing to press the hotkey first each time.

• I dictate the letter names using my voice dictation accessibility software instead of pressing them on my keyboard.
  • However it slows down my workflow quite a bit to need to dictate the hotkey each time and I was wondering if it is possible to just have the screen search labels persist.

As one other question, I was wondering if there is any way in which the screen search labels can be marked with the same letter each time.

• (ie if the to label to click a certain item on the taskbar would always be labeled with AE for some similar letter combination. This would make it possible for me to automate certain GUI tasks that would not otherwise be possible and really improve a lot of accessibility)

Thank you for all the work on this software, it helps me a lot

[adirh3]

Hey @C-Loftus, glad you found Fluent Search helpful!

To answer your questions -

1. If you mean to make Screen Search repeatedly appear every time something is changing on screen, then it's a bit problematic - that's because Screen Search is actually screenshotting your screen to find these labels. If Screen Search persisted on screen, then it would screenshot itself which would be problematic.
   I wonder if other means to trigger Screen Search would help in your scenario, is triggering using a hotkey with your accessibility software a problem? If so, maybe it's possible to somehow integrate Fluent Search into it to invoke screen search directly without a hotkey.

2. Currently Screen Search is trying to keep the same labels for the same objects, but it's nowhere near perfect.
   I am aiming for making it consistent, as it would also improve muscle memory and overall performance.

[C-Loftus]

1: Yes essentially what I mean is that I am looking for the labels to automatically reappear after pressing a key (ie press the fluent search keyboard screen search shortcut, type (through voice) ab and then have the labels automatically be regenerated after without needing to press the shortcut again to relaunch the screen search labels

If I need to do a lot of different commands it becomes tedious to need to press the hotkey each time ( For instance with nested menus that take multiple presses to get to a certain setting )

currently I trigger the hot key from a voice command which is fine but also makes me have to dictate a bit more and slow my work flow down. Since fluent search is listening for the keyboard input, I suppose I was wondering if there is a way to just automatically regenerate the labels after the key combination is pressed

Does fluent search have a library/CLI that exposes if the screen search labels are present? if so it would be possible for me to automatically trigger them again more easily.
Although I think with what you said, thinking about it again there are probably ways that I could do this myself locally ( by just keeping a state variable myself after I trigger a voice command to launch fluent search) and then automatically pressing the hot key if that state variable is true ( and unsetting the variable when I close it)

2: that makes sense thank you for that clarification