apertium-apy
apertium-apy copied to clipboard
apertium
Support multiple q's
We used to (?) support multiple q's per query (or maybe scalemt did). People are expecting us to, anyway:
Hello,
today I discovered that apertium is sending wrong response for batch interface (several q=). For this: https://www.apertium.org/apy/translate?format=html&markUnknown=no&langpair=en%7Ces&q=seconds+with&q=queries.+Memory+Usage%3A&q=8.12+MB
I get: {"responseStatus": 200, "responseDetails": null, "responseData": {"translatedText": "8.12 MB"}}
ONLY LAST PART IS TRANSLATED!!! And according to this information: http://wiki.apertium.org/wiki/Apertium_scalable_service Each part should be translated.
Please fix it or inform ke does batch interface changed. Anyway please send me some info about sitiation :)
Cheers Michał Podbielski
http://wiki.apertium.org/wiki/Apertium-apy says "meant as a drop-in replacement for ScaleMT", and since ScaleMT supports multiple q, so should APy. Or the docs need to change.
http://wiki.apertium.org/wiki/Apertium-apy refers to being drop-in for http://wiki.apertium.org/wiki/ScaleMT which in turns tells people to read the API specification at http://wiki.apertium.org/wiki/Apertium_scalable_service
Multiple q was supported for sure by apertium - we used it in our product (vBET). It is really important to support it again - without it performance of translation will go down and your serwers will have lot of unnecessary requests (why to make 10 requests when we can make 1 request and get 10 translations - it gives results fasster and serwers do not suffer unnecesary traffic)
Hello, I was working on this issue and have made quite a progress in it but there are some things which are not completely clear to me regarding the batch interface. I wanted to know that if there is some error in any of the translation(s) then what should be the output. It would be great if you could provide the output for both the cases when only some translations are erroneous and when all are. Currently, for non-error cases, the output is as follows:-
Also, it would be great if you can tell me what is to be done when we get 'q' parameter and 'langpair' parameter not equal in number.
Also, it would be great if you can tell me what is to be done when we get 'q' parameter and 'langpair' parameter not equal in number.
This should just error. I wouldn't bother translating anything.
It would be great if you could provide the output for both the cases when only some translations are erroneous and when all are.
What do you suggest?
I was thinking of the same to just show the error but in the batch interface description it is defined quite differently and no example output is given for the same. That's why I was asking this. The batch interface description is given on the wiki page whose link is given in one of the comments above.
I was thinking of the same to just show the error but in the batch interface description it is defined quite differently and no example output is given for the same.
Which part of it exactly?
The returned JSON changes a bit when using the batch interface. Now the field responseData contains an array of JSON objects, each one with the usual fields: responseData, responseStatus and responseDetails. Note that we have particular values of responseStatus and responseDetails for each translation, but global values too. If all the translation are OK, these values match, but if there is an error in any translation, global values of these fields take the value of the erroneous translation. If there is more than one erroneous translation, global fields take the value of one the the erroneus translations.
I was talking about this part.
Indeed. Which part of it is the source of your confusion? It makes sense to me.
`The returned JSON changes a bit when using the batch interface. Now the field responseData contains an array of JSON objects, each one with the usual fields: responseData, responseStatus and responseDetails. Note that we have particular values of responseStatus and responseDetails for each translation, but global values too. If all the translation are OK, these values match,
This is clear to apart from but I was just wondering what should happen in cases when both 'q' parameters and 'langpair' parameter received are of unequal size. For example, 'q' parameters received are 2 and 'langpair' parameter received are 4.
but if there is an error in any translation, global values of these fields take the value of the erroneous translation. If there is more than one erroneous translation, global fields take the value of one the the erroneus translations.
This part is particularly unclear to me since there are no examples provided for the above statement.
This is clear to apart from but I was just wondering what should happen in cases when both 'q' parameters and 'langpair' parameter received are of unequal size. For example, 'q' parameters received are 2 and 'langpair' parameter received are 4.
I think that one of the following behaviors is fine:
- don't return any translations and set the global responseStatus != 200.
- return as many translations as you can match and for the rest responseStatus != 200.
This part is particularly unclear to me since there are no examples provided for the above statement.
Say you have two translations that fail with responseDetails=A, responseStatus=X and responseDetails=B, responseStatus=Y and one successful with responseDetails=C,responseStatus=200. Then, the global responseDetails=A, responseStatus=X OR responseDetails=B, responseStatus=Y.
Ok I got the first part but for the second part. There are many reasons for a translation to fail like language pair not found and other such problems, but in those cases APy uses the send_error() method of tornado to send the response and there's no mention of responseDetails and responseStatus in those responses. So how can I set the global parameters without those.
For example, 'q' parameters received are 2 and 'langpair' parameter received are 4.
If there is more than one langpair, isn't that an error? I interpreted the original as saying "we allow more than one q, which will all be translated with the same langpair".
@michalpodbielski can you verify? Have you ever used more than one langpair-parameter in a request?
I never used more than one langpair in same request. One request can translate several texts (q) from same langpair. For new langpair we just send new request.
So, should I assume that we are supporting multiple q's and only single langpair with each request now. Also what about showing error about which I asked in my previous comment
There are many reasons for a translation to fail like language pair not found and other such problems, but in those cases APy uses the send_error() method of tornado to send the response and there's no mention of responseDetails and responseStatus in those responses. So how can I set the global parameters without those.
I think we should send_error if at least one of the q's is an error, ie. never send a "half-way" complete answer. Errors should be exceptional, we shouldn't expect people to have to deal with them in any other way than sending the whole request over again.
Whether there should be some change in the contents of the send_error response should go in a separate Issue from this one.