Text added with iOS dictation disappears

[reginabally]

Describe the bug Reported in 5566465-zd-woothemes. When using the iOS dictation to add text to the app's editor, the text will disappear when tapping another block.

To Reproduce Steps to reproduce the behavior:

1. Go to My Site > Posts and create a new post
2. Activate the iOS dictation feature and speak to the microphone
3. Notice the text added to the editor
4. Tap another block on the editor and notice the text disappear.

Expected behavior I expect the text stays in the editor even if I moved to another block in the editor.

Screen recording

https://user-images.githubusercontent.com/40906847/191393589-fa783c20-33c9-42b4-b54d-324f57115016.MP4

Smartphone (please complete the following information):

• Device: iPhone 11
• OS: iOS 16
• Version: WPiOS 20.7.0.4

Additional context Add any other context about the problem here.

[fluiddot]

Thanks for opening the issue @reginabally! This is definitely an annoying bug for users that use dictation, which by the way it's a very interesting feature for writing text in a post quickly 🙂. I noticed that the text remains if you tap on the microphone button after dictating, but any other action like tapping on a different block, typing text, or scrolling the content makes the text disappear.

[reginabally]

I noticed that the text remains if you tap on the microphone button after dictating

TIL! Thanks for pointing that out, @fluiddot! I will inform the user about this trick.

[supernovia]

I've been running into this a lot too. Do we need more information? I'm happy to provide what I can.

[fluiddot]

I've been running into this a lot too. Do we need more information? I'm happy to provide what I can.

We haven't investigated the issue yet but any extra information is always welcomed 🙂. I hope we can prioritize it soon and address it in the next WordPress-iOS versions 🤞 .

[reginabally]

Another report in p1672863996840939-slack-C03URUK5C. The video recording was too large to be uploaded here. Please check the Slack thread for the video if needed.

Device: iPhone 14: iOS 16.1 WPiOS version: 21.2 (21.2.0.4)

[SiobhyB]

I've been looking at this on/off between other project work and wanted to take the chance to share my findings so far:

• There was a similar issue back in 2019, which was fixed as part of https://github.com/wordpress-mobile/gutenberg-mobile/pull/610. The PR goes into more technical details around the cause of the issue, with the fix setting an isInsertingDictationResult flag after dictationRecordingDidEnd fires. This flag is then used to prevent empty text from being propagated:

https://github.com/WordPress/gutenberg/blob/c73da69cdacab06f5b68596f9157ae87db411316/packages/react-native-aztec/ios/RNTAztecView/RCTAztecView.swift#L356-L358

• I believe this new issue arose following the improvements made to dictation in iOS 16‌.

Apple with ‌iOS 16‌ entirely overhauled the dictation experience, making it easier to swap between voice and touch. When you're dictating text on the ‌iPhone‌ or [iPad](https://www.macrumors.com/roundup/ipad/), you can tap into the text field and type with the keyboard to make edits and insert QuickType suggestions without having to stop the dictation process.

• As dictationRecordingDidEnd is no longer called when a user taps into the text field, there is nothing preventing empty text from being propagated.

To fix this, we need to either:

• Figure out a way to set the isInsertingDictationResult flag to true when a user is dictating and manually entering text. This would likely be a bit hack-ish and we might run into the same problem with future changes to dictation causing issues.
• Investigate ways to refactor the code so that textViewDidChange doesn't send an empty string when a user's dictating. This change would be more complicated, but better in the long-run and avoid adding further hacks around the RichText component.

[fluiddot]

Thanks @SiobhyB for investigating the issue and sharing your findings 🙇 !

I've been looking at this on/off between other project work and wanted to take the chance to share my findings so far:

• There was a similar issue back in 2019, which was fixed as part of Fix dictation issue on iOS #610. The PR goes into more technical details around the cause of the issue, with the fix setting an isInsertingDictationResult flag after dictationRecordingDidEnd fires. This flag is then used to prevent empty text from being propagated:

Ah, it's interesting that we bumped into a similar problem in the past and provided a workaround.

To fix this, we need to either:

• Figure out a way to set the isInsertingDictationResult flag to true when a user is dictating and manually entering text. This would likely be a bit hack-ish and we might run into the same problem with future changes to dictation causing issues.
• Investigate ways to refactor the code so that textViewDidChange doesn't send an empty string when a user's dictating. This change would be more complicated, but better in the long-run and avoid adding further hacks around the RichText component.

Definitely, we should try to address the issue in the long run. However, since this is affecting the editing experience, I'd advocate trying first the approach of using the isInsertingDictationResult flag 👍.

[SiobhyB]

I wanted to update on some investigations into this option:

Figure out a way to set the isInsertingDictationResult flag to true when a user is dictating and manually entering text.

This doesn't seem as straightforward as it may seem, as there doesn't appear to be a straightforward way to detect when dictation is active since iOS 16 (previously, it was possible to use textInputMode for this purpose, but not any more).

I found this StackOverflow post which seems to have a good lead involving method swizzling: https://stackoverflow.com/questions/74024881/detect-text-is-entered-using-dictation-on-ios-16/74024882

However, I'm beginning to wonder whether this would end up being more or less complex than the second option:

Investigate ways to refactor the code so that textViewDidChange doesn't send an empty string when a user's dictating.

I have some upcoming AFK, so won't have time to look further into this in the next week or so, but would be happy to keep looking after that if no one beats me to it. :)

[fluiddot]

This doesn't seem as straightforward as it may seem, as there doesn't appear to be a straightforward way to detect when dictation is active since iOS 16 (previously, it was possible to use textInputMode for this purpose, but not any more).

I found this StackOverflow post which seems to have a good lead involving method swizzling: https://stackoverflow.com/questions/74024881/detect-text-is-entered-using-dictation-on-ios-16/74024882

However, I'm beginning to wonder whether this would end up being more or less complex than the second option

I totally agree, if the first hack-ish option becomes more complex than the second I'd advocate directly going with the second one to address the issue 👍.

[twstokes]

👋 Hey @SiobhyB - I looked at this briefly and wondered what would happen if we didn't provide the insertDictationResult method and instead let the system handle it. (I just commented it out entirely and recompiled)

This may only work on iOS 16, but it seems to be an improvement. I'm not quite sure what happened at the 00:38 mark to uncapitalize the first letter...

Video contains audio

https://user-images.githubusercontent.com/2092798/225055493-2e9447b1-5847-4a32-abcd-396dd2707478.mov

[SiobhyB]

@twstokes, you could well be right here, I did a quick test and saw the same improvements as you. I wonder if the changes to dictation in iOS 16 negated any need for these previous hacks. I'll do some more thorough testing tomorrow.

...and also go hide in a corner for potentially making this way more complicated than needs be. 🤦‍♀️

I really appreciate you taking the time to take a look, thank you.

[twstokes]

I wonder if the changes to dictation in iOS 16 negated any need for these previous hacks.

I wonder the same!

for potentially making this way more complicated than needs be

In your defense Aztec bugs typically do mean complicated 😆, plus the jury's still out on whether what I proposed will work. I regret saying "looked at this briefly" because it sounds like it downplays previous efforts. It helped that I had a long battle or two with Aztec previously. 😄

I'll do some more thorough testing tomorrow.

Thanks for digging into this! 🙇

[SiobhyB]

To follow up, I couldn't find any issues with removing the insertDictationResult hack for iOS 16 🎉

I've updated the PR at https://github.com/WordPress/gutenberg/pull/49056 and will set it as ready for review shortly, after a bit more testing.

Thanks so much @twstokes for your help here 🙇‍♀️

[SiobhyB]

I'm not quite sure what happened at the 00:38 mark to uncapitalize the first letter...

Just wanted to note, too, that I've been able to replicate this sporadically. I'll create a separate issue for this as, though not ideal, it still seems a lot better than the current state of dictation for iOS.

[SiobhyB]

Re-opened as we needed to revert the proposed fix in https://github.com/WordPress/gutenberg/pull/49154.

[SiobhyB]

Making a note of a 1 star review that I've offered to follow up with when a fix has been merged:

Hello, if you enter something by voice in the WordPress and Jetpack app, i.e. via the Apple dictation function, and then tap somewhere or on the keyboard, the completely entered text disappears. This also only happens in the WordPress/Jetpack app. This error is very annoying because you often have to retype everything. I have had this on my iPhone 11 Pro and on my 14 pro Max. Greetings heiko

[SiobhyB]

Going ahead to close as this was fixed in https://github.com/WordPress/gutenberg/pull/49452. There are still some edge cases where the cursors placement appears off, which will be addressed separately in https://github.com/WordPress/gutenberg/issues/51227.

The main fix will be included in the 22.6 version of the app, set to be available on June 26th. If we have a beta release for Gutenberg Mobile, we could also include this as part of that.

I also followed up with the review noted above.