asterics
Pictogram Prediction
Option 1:
Example from CoughDrop below. Like with word prediction in keyboards, pictogram prediction fields show possible next words as you click on pictograms.
Option 2:
When typing with letters, word prediction doesn't only show the letters but pictograms too. Example from TD Snap
Having both options would be cool.
Option 3, which is partly what happens if I use the prediction element in a pictogram grid now: After clicking an element, the "prediction" element shows word forms of that element. Clicking on the prediction element then replaces the pictogram label in the collect element with that different word form. This would be I think less relevant than the other two options, even though there are AAC apps who do exactly that to access word forms. But as it requires literacy skills and AG's word forms module provides options to create word forms without literacy skills, I think it's less relevant for Asterics. There may be literate users who prefer this option, though. Definitely not a priority, though.
NB: Just chiming in here. This is doable. But I think first we need to do #79
https://youtu.be/jwX6-X9Tk8M of where this is up to.. works on all the symbol libraries
Looks good! Does this include word forms? Like where did you get the "are" from, does an element with that label exist or does the element have the label "be" and "are" is in the word forms?
I checked the link. Where does one activate the images so that it doesn't show the text only?
I checked the link. Where does one activate the images so that it doesn't show the text only?
Okay, I tested it.
So, compared to the current options in the keyboard, option 2 from the original post would be the normal prediction based on the letters you type.
https://github.com/user-attachments/assets/e0a649e7-5e4f-4984-b8be-02fc5af0fbf1
Whereas option 1 would be the prediction based on what words you've used before
https://github.com/user-attachments/assets/564be40d-520d-4ccc-8ea8-590948ebcfb5
(Ignore the strange image ratio, that got messed up when converting)
After typing that on words-only and switching to images, the option 1 only works for the phrase that I already used in the words-only setting (What do you want Anna),
https://github.com/user-attachments/assets/17898966-a8dc-477a-a183-3f9d1cc33172
it doesn't remember the other words (I like cucumber), at least outside of keyboard-mode, I got mixed results in keyboard mode. For some unknown reason it doesn't even predict the pictogram cucumber.
https://github.com/user-attachments/assets/57e51d19-4642-4dd5-af2c-30d4dcc0508d
https://github.com/user-attachments/assets/f23f180c-8e3a-4d5c-85f3-4a8f6c803e11
It doesn't show the word forms either. Like "does" doesn't come with the correct pictogram as is programmed in the board in the word forms, so only the choice to select the wrong pictogram or keep typing.
https://github.com/user-attachments/assets/828814ce-cbb5-4df9-a739-c6fbedc998a7
It doesn't predict pictograms that have more than one word, e.g. "green beans"
https://github.com/user-attachments/assets/eb162855-8790-4e5f-a783-5cf20e3e6fa9
I think the primary source for these problems is that it is taking the data (pictograms and words) online, instead of from whatever is already programmed in the board (and only that, even if it means that it won't be able to predict pictograms of any words that are not programmed it the board, but that is fine). This is (a) a problem, because one needs internet to use that function and (b) a problem because it doesn't adapt to whatever user-specific pictogram changes are set up for the board, and these user-specific changes are usually really important, including buttons with the names of say my family. As you see in the video, not only does it not show the correct pictogram for Anna, it's not even possible to say "Anna" without having to use the wrong pictogram.
Also, there are similar problems with option 2 in TD. So I don't know if this is actually technically doable or if I need to programm some stuff manually. I was expecting to having to do that and then Benjamin thought that it could be doable.
As for option 1, I don't currently have access to Coughdrop but I think that it was working there offline, so there should be a way to get that working.
Hmmm. I'm not sure if any software prioritises your chosen symbol over the lookup. But that should somehow be possible. I need to think how best to do that. There is a sub problem I see. Global symbols and open symbols you really need to choose a symbol subset. That would help this a bit as it's getting all symbols and the result can look messy
Note. Option 1 & 2 is doable already in this PR
Ok. Think this should work. See https://gridbeta-github.asterics-foundation.org/willwade/symbol-prediction/pr it's now creating an index (although note. Just 200) of symbols used in your user pages and uses this first)
Does "just 200" mean it indexes just 200 of the board set? I made a small test set (same link as for grammar order https://we.tl/t-UbKOZjaLcJ ) if that's what it is, but a robust grid set will have more than 200 elements.
It does indeed now show the correct pictogram for Anna, great! :)
Option 1 definitely only works if you use the keyboard and not when you use the pictograms, which is what I'd like it to do.
https://github.com/user-attachments/assets/f7bab29a-9f8a-4a0b-9c4e-9536140a5074
Word forms (e.g. does) still don't work but I guess you didn't work on that in this update.
...for some random reason it however still doesn't show 'cucumber' in the keyboard, that's strange... (That 'Anna' works but 'cucumber' doesn't...)
Ok think both your issues now fixed - check it out. Ive upped cache size to 1000. https://gridbeta-github.asterics-foundation.org/willwade/symbol-prediction/pr/#grid/grid-data-1757510044388-123 and symbol prediction will now work on regular word cells
Hm, I do get the update notice
but I don't see a change for option 1, it still is not remembering the words I clicked. (Delete and re-import the dictionary to delete any past predictions it remembers).
Are there disadvantages to upping the cache more? The average size of a robust app would be 4000-7000 buttons, mine might be even be somewhat bigger.
No not really. That's wierd. It's working for me using your grd backup and others. I'll double check. I'll up the number
Oh! Go to that URL in a private browser tab. I bet the manifest is caching your experience
Yes, I tried in a private browser. I also deleted cache.
https://github.com/user-attachments/assets/94f11f14-09b9-4479-aba7-1a48fc813053
Hold on @ms-mialingvo - is your video not showing Option 1. That is symbolised prediction for next word in a whole word grid page?
Prediction based on what pictograms I've already used. I want this
after using the pictograms only. At the moment it only works with the keyboard.
Not sure what else you're aiming for with option 1 otherwise? The pictograms that currently appear are pictograms that have similar words to what already was clicked and with non-fitting pictograms too so as such that what would be the use of that?
If the pictograms were fitting and if clicking on it would replace the existing pictogramm in the speak bar, it would be option 3 (https://github.com/asterics/AsTeRICS-Grid/issues/349#issuecomment-2646150156) . Not what I'm aiming for with option 1.
ok got you. you are saying that its not using your cache of symbols for prediction right?
you are saying that its not using your cache of symbols for prediction right?
I'm saying it's not predicting the words I've already used first, like it works for the keyboard. The fact that it's not predicting the correct symbols is an additional issue.
(Removed the needs-code-review tag because neither option 1 nor option 2 are working properly yet. Option 2 as I said is for example not showing the word 'cucumber' despite it being saved in the board set. Option 1, see previous post.)