Handle LocationData in chat via Twilio/WhatsApp Channel

[m-mohsin-ali]

What problem are you trying to solve?

With growing using of chatbots Specially wide of RASA framework many business are adapting this new mode of reaching out to customers. now business type like restaurants or delivery like services make use of these channels as well now to server there customer best they need to have accurate location of the customer to better server their customers. for that they can utilize the sharing current location features such as in WhatsApp.

Now Rasa Does offer support for WhatsApp channel via Twilio but the the current implementation of the channel does not allow the use of this feature. if you were to try sending this message it will simply not respond and you would see an exception occurred on your console.

Now this is cause by the current implementation here.

https://github.com/RasaHQ/rasa/blob/main/rasa/core/channels/twilio.py#L138

But the Twilio does send the coordinates just not in the body tag of the request.

The keywords used are Latitude and longitude and current implantation only read the Body keyword as text here

https://github.com/RasaHQ/rasa/blob/main/rasa/core/channels/twilio.py#L130

one minor thing I also noticed here

https://github.com/RasaHQ/rasa/blob/main/rasa/core/channels/twilio.py#L134

I think this condition should be changed because as its not catering the None type for text variable. which throws a exception for many other type messages where body tag is not present such as call backs, location, images etc. this makes debugging hard and throws not so useful exceptions.

What's your suggested solution?

I suggest we make little changes to how the Twilio channel handles incoming requests. first need to change the condition here. https://github.com/RasaHQ/rasa/blob/main/rasa/core/channels/twilio.py#L134 To

if sender is not None and text is not None:

or

if sender is not None and text is not None and message is not None:

or we can throw a more relevant message " rasa received a message it can not understand {With the request data it received}" if text is None.

Second for the location I suggest as location data is important for such businesses we extract the location data accordingly and pass it as metadata to the bot where it can be dealt with action server in addition to this we replace the Text from being None to carry '/locationdata' for triggering an intent to be implemented by developer to handle the location data.

now the location data is in metadata which is already being handled here

https://github.com/RasaHQ/rasa/blob/main/rasa/core/channels/twilio.py#L145

now we implement an intent locationdata and handle it in an custom action like

Reference https://forum.rasa.com/t/how-to-pass-metadata-in-post-request-and-then-extract-it/15092/6

Examples (if relevant)

No response

Is anything blocking this from being implemented? (if relevant)

No response

Definition of Done

No response

[sara-tagger]

Thanks for submitting this feature request 🚀 @kedz will get back to you about it soon! ✨

[m-mohsin-ali]

Hey @tmbo i wanted to bring this to any ones attention rasa. i opened it while back and not sure if this even a valid thing to be contributed. maybe you can look at it let me know if we can move forward with it like i can contribute this piece or i can just discard this Issue as a contribution and just post this as article for some one who wants to achieve similar results.

thank you.

[aymalkhalid]

@m-mohsin-ali Thank you sharing this.

[aymalkhalid]

How can someone access the location/metadata in rasa actions, as my whole chatbot is within the rasa with Twilio WhatsApp Integration.

[m-mohsin-ali]

@aymalkhalid when you go through the whole article you can see that at the end I have provided a custom actions class method to do so.

[m-mohsin-ali]

i think another way we can receive the data in custom actions is, if we have 2 slots lets say "lat" and "long" and we can just set the slots directly with out the meta data requirement like.

text='/locationdata{"lat": "0.000","long": "0.000"}'

and then read the slot values in the custom actions

lat = tracker.get_slot('lat')
long =  tracker.get_slot('long')

i haven't tested this but i think this should also do the job

[aymalkhalid]

I will try it out and let you thanks :) Pakistan Zindabad

[ShaidaMuhammad]

much-needed feature. can we workaround it with custom actions in the current implementation without changing Rasa framework files?

[m-mohsin-ali]

much-needed feature. can we workaround it with custom actions in the current implementation without changing Rasa framework files?

No, I don't think we can do that without changing a bit of how channels handle incoming messages.

[sync-by-unito[bot]]

➤ Maxime Verger commented:

:bulb: Heads up! We're moving issues to Jira: https://rasa-open-source.atlassian.net/browse/OSS.

From now on, this Jira board is the place where you can browse (without an account) and create issues (you'll need a free Jira account for that). This GitHub issue has already been migrated to Jira and will be closed on January 9th, 2023. Do not forget to subscribe to the corresponding Jira issue!

:arrow_right: More information in the forum: https://forum.rasa.com/t/migration-of-rasa-oss-issues-to-jira/56569.