nvda
nvda copied to clipboard
nvaccess
Object Labeler Functionality
Reported by ateu on 2012-02-17 00:37 Hello
Sometimes, when using object navigation, we navigate controls which NVDA only anounces the control's name (in my job, for example, i use an application which has many inaccessible buttons, and when I use object navigation, I heard just "button" or simply unknown.) Also, when it ocurrs, the flat review does not detect them. For this reazon, it would be so good nNVDA provides a graphic labeler. It would be associated with the object navigation, as it provides the control tipe. As well as the symbols, the graphic info would be saved in the user config, in a file specially made for it. The command key used to activate the graphic labeler would be NVDA+g.
Thanks Blocking #4391
Comment 1 by orcauser on 2012-02-20 13:22 Yes you are right, it would be quite useful to have this feature, and we have had several requests for this on the arabic-nvda list.
Feature inplemented in the labelObjects branch. If you are able to run from source, then please try it.
@Jamie: please review, it seems to be working ok on the virtual machine, but unable to get accurate timings to see if i am reducing performance considerably. What do you think about using yaml to store the label data, that way it is human readable too.
Thanks.
Comment 2 by ateu on 2012-02-20 14:44 Hello,
Unfortunately I can't run from the source code. Can you please send me the modiffyed files to [email protected] or say the entire adress where I can find them/
Comment 3 by ragb on 2012-02-20 14:54 Quick comments:
I labeled some object s in explorer (files) and it worked. Tried to label some buttons on skype and the start button but labels were not reported...
Regarding the labels file format I'd say that any readable format is good, we can have an idea what NVDa is labeling without requiring many debug activities... Moreover, if you agree, it will be good that label data is stored in userconfig/labels to not cluter the config directory with label files.
Comment 4 by orcauser on 2012-02-20 19:41 Replying to ateu:
Can you please send me the modified files or say the entire adress where I can find them?
I am afraid I don't think sending files will help, because the modified files are part of nvda and therefore you will not be able to replace them because you are running a compiled and prepackaged version. But now that Rui is also able to test, we should be able to find and eliminate problems.
Replying to ragb:
Tried to label some buttons on skype and the start button but labels were > not reported...
Yes, I can not label the start button either, not exactly sure why this is, especially if the desktop has the focus.
Unfortunately Skype always crashes in virtualbox (a well known error) so can't test. Can you find another example of an application that fails to label correctly?
Have moved the logging to log.info until we sort out all problems, so please try again with skype and lets see if the log can explain.
Regarding the labels file format I'd say that any readable format is good, we can have an idea what NVDa
is labeling without requiring many debug activities...
yaml seems nice and very readable, but it would introduce another dependancy.
the changeset,labelObjects:5018 switches from pickle to yaml. Please install pyaml for python 2.7 and try again. http://pyyaml.org/wiki/PyYAML
Moreover, if you agree, it will be good that label data is stored in userconfig/labels to not cluter
the config directory with label files.
Sorry I had a typo (now fixed), they were intended to go into the appModules folder next to the appModule it matches, we can always move them if this is not what we want.
Thanks for your testing.
Comment 5 by jteh on 2012-02-20 21:19 I haven't actually reviewed the code yet, but a few quick comments:
- I am very concerned about performance here. Using object paths will be extremely slow in applications with a lot of objects and a lot of labels. My guess is that it's O(n^3), which is pretty nasty. We're likely to have all sorts of bugs filed about performance problems.
- I've often wondered whether we could come up with code to try to uniquely identify objects using various properties, only resorting to path as the last resort. The problem is that this isn't guaranteed to work without human intervention.
- Related to (1), if we do merge this, I would think labels should never be distributed in this form with the main NVDA distribution.
- Rather than introducing another external dependency, what about using json? This isn't currently used by NVDA, but at least it's part of the standard library.
- One of the issues i have with application specific configuration is that many executables now host many applications. For example, this will be a total disaster if someone tries labelling an object on a web page. As another example, !OpenOffice.org hosts all of its applications within the one executable. Any Python code that runs from source will run in python.exe or pythonw.exe. This trend is only going to grow. Unfortunately, I don't really have a solution to this, but it is one of the biggest reasons I have always been against implementing application specific config.
I'll try to provide more review and feedback soon.
Comment 6 by ragb (in reply to comment 4) on 2012-02-20 21:37 Replying to orcauser:
But now that Rui is also able to test, we should be able to find and eliminate problems.
Replying to ragb:
Tried to label some buttons on skype and the start button but labels were > not reported...
Yes, I can not label the start button either, not exactly sure why this is, especially if the desktop has the focus.
I can't label anything now, with revision 5018
See what contains my explorer_labels (in yaml) after trying to label start menu and recyclebin:
? ''
: !!python/tuple
- [!!python/unicode 'Iniciar'
0.0.0.0.1: !!python/tuple
- - !!python/tuple
- [0](]
-)
- 0
- !!python/unicode 'SHELLDLL_DefView'
- !!python/tuple
- [ - 0
- !!python/unicode 'SHELLDLL_DefView'
- !!python/tuple
- [0](0]
)
- 1
- !!python/unicode 'SysListView32'
- !!python/tuple
- [ - 1
- !!python/unicode 'SysListView32'
- !!python/tuple
- [1](0]
)
- 1
- !!python/unicode 'SysListView32'
- !!python/unicode 'REciclagem'
Is this Python/unicode stuff correct? I don't understand the object path code and how it can be used to identify an object so no idea if it is suposed or not to be this way.
Unfortunately Skype always crashes in virtualbox (a well known error) so can't test.
Can you find another example of an application that fails to label correctly?
Have moved the logging to log.info until we sort out all problems, so please try again with skype and lets see if the log can explain.
Now explorer itself.
Regarding the labels file format I'd say that any readable format is good, we can have an idea what NVDa
is labeling without requiring many debug activities...
yaml seems nice and very readable, but it would introduce another dependancy.
I agree with Jamie (comment:5) that json would be a better option. It avoids that extra dependency.
Comment 7 by orcauser (in reply to comment 5) on 2012-02-21 12:13
Replying to jteh:
- I am very concerned about performance here. Using object paths will be extremely
slow in applications with a lot of objects and a lot of labels. My guess is that
it's O(n^3), which is pretty nasty. We're likely to have all sorts of bugs filed about performance problems.
I don't know if this is feasible, but could a property be added to NVDAObjects to track the path of the object? I realize that not all toolkits provide the needed info, but then we can provide None as answer.
Something like this: When an NVDAObject is created, it fetches the cached value of its previous sibling and increments by 1, otherwise 0. If it is a new child, then it adds a new element to the list. The path would be something like: 1,, 2,0
- I've often wondered whether we could come up with code to try to uniquely identify
objects using various properties, only resorting to path as the last resort. The problem
is that this isn't guaranteed to work without human intervention.
It isn't easy to come up with a neet solution, but maybe its good enough to clearly state that this may effect performance in the appModules where this approach is being used.
The problem is always a tradeoff between having access to information and some other commodity. The application I have been batteling for some time has virtually no text, just icons, and all of them are "unknown", and our users have been forced to count how many keypresses to perform, which is very laborious. Yet they need access to it because it provides an electronic arabic book library.
- Related to (1), if we do merge this, I would think labels should never be distributed in
this form with the main NVDA distribution.
No problem, Its easy to pull out again.
- Rather than introducing another external dependency, what about using json? This isn't currently used by NVDA, but at least it's > part of the standard library.
I tried this but serializing and deserializing doesn't give identical python objects, so can not be used for equality checking. Maybe I am doing something wrong.
- One of the issues i have with application specific configuration is that many executables
now host many applications. For example, this will be a total disaster if someone tries labelling
an object on a web page.
This feature was only intended for badly behaving applications, not the broken web.
As another example, !OpenOffice.org hosts all of its applications within the one executable.
Does this mean we are using the same appModule for all openOffice products?
If you are really against this, then the functionallity can probably be pulled out and provided to anyone wishing to create their own appModule, in which case they are responsible for making sure it works.
In any case, thanks for investigating.
Comment 8 by jteh (in reply to comment 7) on 2012-02-21 21:41 Replying to orcauser:
- I am very concerned about performance here. Using object paths will be extremely
slow in applications with a lot of objects and a lot of labels.
I don't know if this is feasible, but could a property be added to
NVDAObjects to track the path of the object?
I realize that not all toolkits provide the needed info, but then we can provide None as answer.
Sadly, I don't think any toolkits provide the needed info directly, although IA2 and JAB do at least provide indexInParent, which is useful.
Something like this:
When an NVDAObject is created, it fetches the cached value of its previous sibling
Even fetching a previous sibling of an object as soon as it is created is expensive. We normally don't cache objects because we can't guarantee true equality, so calling .previous usually fetches a new object. In other words, there would be no advantage to doing it this way.
- I've often wondered whether we could come up with code to try to uniquely identify
objects using various properties, only resorting to path as the last resort.
One idea Mick had was to compare the Python class name of the object first before comparing object path. This way, it won't match on controls that aren't of that type.
It isn't easy to come up with a neet solution, but maybe its good enough to clearly state that this may effect performance in the appModules where this approach is being used.
As it currently stands, this functionality is user accessible. I guess you could try to explain that using it will cause performance degradation for the entire application, but that suggests users need to be very careful about how they use it, which is a big expectation to place on those who possibly don't understand the technical issues.
The application I have been batteling for some time has virtually no text, just icons, and all of them are "unknown", and our users have been forced to count how many keypresses to perform, which is very laborious.
I assume this is the application you were discussing on nvda-dev. Did none of my last suggestions help?
- Rather than introducing another external dependency, what about using json? This isn't currently used by NVDA, but at least it's > part of the standard library.
I tried this but serializing and deserializing doesn't give identical python objects, so can not be used for equality checking.
Damn. I suspect this relates to tuples versus lists. I suspect json will always deserialise as lists, even if you serialise tuples.
- One of the issues i have with application specific configuration is that many executables
now host many applications. For example, this will be a total disaster if someone tries labelling
an object on a web page.
This feature was only intended for badly behaving applications, not the broken web.
Right, but if there is an auto labelling feature which is user accessible, users aren't going to understand that.
Does this mean we are using the same appModule for all openOffice products?
Yes.
If you are really against this, then the functionallity can probably be pulled out and provided to anyone wishing to
create their own appModule, in which case they are responsible for making sure it works.
At the very least, I've certainly been thinking for a while that it would probably make sense to include your object path functions. This way, it's easier for any app module developer to use them. It may even make sense to have a traversalPath property or similar on NVDAObjects and then show this in devInfo.
In any case, thanks for investigating.
To be clear, I really do appreciate your work. If I thought this would work without adverse consequences, it'd definitely make sense to work to get it included. Unfortunately, performance is a very serious concern for me here.
Comment 9 by Ahiiron on 2012-07-12 17:40 Is there any way to package/make a globalPlugin/addon with these tweaks? I, too, am concerned about performance here; NVDA is snappy and lightening fast most times. We don't want that ruined. I'm not about to go snag source and break things, though. :)
Comment 10 by MHameed on 2012-10-30 08:28 Changes: Milestone changed from None to 2013.1
Comment 11 by jteh on 2012-10-30 08:41 I'm not trying to be intentionally difficult, but I very much doubt this will make it into 2013.1. I haven't seen any hint of a design yet that is acceptable for inclusion into the main code. Any implementation must not negatively impact performance. For me, this absolutely does not block 2013.1. If a suitable design can be devised, then we can reconsider the milestone. Changes: Milestone changed from 2013.1 to None
Comment 12 by jteh on 2012-11-06 07:32 It's also worth noting that the approaches here are object labelling, not graphics labelling in the sense that most users think of it. Some graphics are drawn to the screen and don't appear as objects; e.g. the graphics to indicate states for mail messages in some mail clients.
Comment 13 by Ahiiron on 2012-11-07 23:31 Yep. So what about an addon? Don't addons overwrite original files anyway, or is this not possible? Would be interested in playing with this at least.
Comment 14 by vgjh2005 on 2014-07-25 17:35 Hi: This function is very hopeful in China. More and more software is made by the graphics. If the graphic cannot offer the information, however that give them a number is a good way. The first graphic is named graphic1 and the second one is named by graphic 2 etc. Let the number of each graphic is the same everytime. We could try to click them and remember them by remembering its number. finally, we could give it a label by using NVDA. Please merge it in the maincode quickly if it is possible. Thanks!
Comment 16 by blindbhavya on 2014-08-14 18:49 Hey, Mesar Sir. Since you set the owner to yourself, I am asking you, were you able to go further with the implementation of this feature? I have read all the comments, and according to whatever I comprehended, I certainly understood the performance risk such a feature could bring. If this question is not off-topic and everyone is fine with me asking it here instead of the NVDA Devel/Users list, then I am curious to know why is Windows unable to allow such a feature when Android (Talkback) does? Thanks in advance.
I read that Jaws does have this capability. How does this works in Jaws and why it can not in NVDA?
Yes, this feature is still required by members of portuguese language comunity...
Best regards,
Rui Fontes NVDA portuguese team
Às 00:31 de 26/09/2018, Quentin Christensen escreveu:
Is there any progress on this issue? People still request it.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/2111#issuecomment-424535393, or mute the thread https://github.com/notifications/unsubscribe-auth/APIHxmu-MgVA6LH3DHq9EJIyLCcFIlUnks5uerzSgaJpZM4Lih78.
I hope this is still being thought of, because this would be great for accessibility issues! Is there a way to have an open source database of these labels?
So do I! It is very important... Altough I agree with Joseph Lee, we should ask devellopers to label all controls, we need some tool to bypass that problem untill all devellopers start doing things the right way...
Rui Fontes
Às 13:52 de 15/02/2019, Robert Kingett escreveu:
I hope this is still being thought of, because this would be great for accessibility issues! Is there a way to have an open source database of these labels?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/2111#issuecomment-464057434, or mute the thread https://github.com/notifications/unsubscribe-auth/APIHxhu1dWtXNQ6HBCSp7ndIRb7WPHdqks5vNruNgaJpZM4Lih78.
somehow addressed in #7194. This feature is also requested from many people in the romanian community.
What is the progress on the NVDA object labeler? Will it be in NVDA 2019.3? or 2019.4?
Please consider adding this. JAWS has had this for years and users are forced to work with JAWS for their jobs just because of this.
Talkback is a good example of easy labeling. JAWS too. This wouldn't discourage accessibility advocacy, it may push it forward, but I know many JAWS users that use JAWS because this feature exists. If this can also have an open source database of labels, it would make it easier to try NVDA with indie programs.
Hi. Even some printers and scanners have apps which require image/button-labeling in order to become more accessible. So please kindly consider implementing this functionality.
I am really surprised that this feature has not appeared on NVDA so far, and now NVDA users have to force the implementation of this feature on add-ons. Is it possible for NVAccess to reconsider?
I still maintain this would be a boom to NVDA adoption, combined with an open source repository of labels would be the best. Just out of curiosity, which add ons allow this?
I still maintain this would be a boom to NVDA adoption, combined with an open source repository of labels would be the best. Just out of curiosity, which add ons allow this?
Hello NVDA users in Taiwan have created this add-on, but there is no github repository for the time being. Is there an open source tag repository available? Thanks.
I once created an add-on on behalf of @BabbageCom, not sure about the current state though: https://github.com/BabbageCom/objEnhancer
