Error exporting to jsonnested

[arelstone]

After adding a new term and have translated it and calling the api/v1/projects/{PROJECT_ID}/exports?locale=en&format=jsonnested-endpoint i get the following error.

Error:

POST /api/v1/auth/token HTTP/1.0 200 270 - 95.481 ms
TypeError: Cannot create property 'HOURS' on string ''
at Object.<anonymous> (/opt/traduora/src/formatters/jsonnested.js:60:42)
at Generator.next (<anonymous>)
at /opt/traduora/src/formatters/jsonnested.js:7:71
at new Promise (<anonymous>)
at __awaiter (/opt/traduora/src/formatters/jsonnested.js:3:12)
at Object.exports.jsonNestedExporter (/opt/traduora/src/formatters/jsonnested.js:44:40)
at ExportsController.<anonymous> (/opt/traduora/src/controllers/exports.controller.js:100:47)
at Generator.next (<anonymous>)
at /opt/traduora/src/controllers/exports.controller.js:19:71
at new Promise (<anonymous>)
GET /api/v1/projects/{PROJECT_ID}/exports?locale=en&format=jsonnested HTTP/1.0 500 83 - 47.890 ms

I've tried this with both the cli and thru the webapp, same error occurs.

The term i've added is: UPDATE_SCHEDULER_SCREEN.EDIT_EVENT.HOURS.

I thought all-right. It might be the the 255 character limit set my mysql that is reported in #6 so I did a lookup for top 10 longest terms and top 10 longest translations (SELECT value, LENGTH(value) as length FROM term ORDER BY length DESC and SELECT value, LENGTH(value) as length FROM translation ORDER BY length DESC). Neither of these is past the 255 character limit. The longest translation we're using is 216. The longest term is 63. So this was not the issue.

Next up was renaming the term to have an underscore instead of a dot UPDATE_SCHEDULER_SCREEN.EDIT_EVENT_HOURS and did an export again. PROFIT. It worked.

So the issue is not a translation but the term. The weird thing is that we're using terms with multiple dots and underscores. For example ONBOARDING.PAGES.CASE_MATERIALS.HEADLINE or CASE_SCREEN.HOURS.FORM.HOUR_TYPES.EMPTY_LIST_NON_BILLABLE that doesn't give the same error.

What could be the cause of this?

[d4rth-v4d3r]

I have this same problem with a string like this:

term: data.VWAnalyticssaleorder.docdate.day

Message:

TypeError: Cannot create property 'day' on string '[Órdenes de Venta] Fecha '
    at Object.<anonymous> (/opt/traduora/src/formatters/jsonnested.js:58:42)
    at Generator.next (<anonymous>)
    at /opt/traduora/src/formatters/jsonnested.js:7:71
    at new Promise (<anonymous>)
    at __awaiter (/opt/traduora/src/formatters/jsonnested.js:3:12)
    at Object.exports.jsonNestedExporter (/opt/traduora/src/formatters/jsonnested.js:42:40)
    at ExportsController.<anonymous> (/opt/traduora/src/controllers/exports.controller.js:100:47)
    at Generator.next (<anonymous>)
    at /opt/traduora/src/controllers/exports.controller.js:19:71
    at new Promise (<anonymous>)

[anthonynsimon]

I added a few test cases on https://github.com/traduora/traduora/pull/130 but I am unable to reproduce the issue on my end. Could you guys post some info on the OS/node version that you are using (or if using docker) and which traduora version?

[d4rth-v4d3r]

I have two terms:

term0: data.VWAnalyticssaleorder.docdate
term1: data.VWAnalyticssaleorder.docdate.day

Could it be related to this?

[anthonynsimon]

It seems like the nesting operation breaks at some on your side, but I’m unable to reproduce it on my side / CircleCI.

[arelstone]

@d4rth-v4d3r I'm courious if you are using using the webinterface or the cli

[d4rth-v4d3r]

Hi @arelstone I did a test today, in webinterface I added this terms:

In webinterface I can't export correctly, however when I executed export via API /api/v1/projects/{{projectId}}/exports?locale=es_GT&format=jsonnested (with proper project id):

{
    "error": {
        "code": "Internal",
        "message": "Something wen't wrong, that's all we know"
    }
}

In logs, I get the following message:

TypeError: Cannot create property 'test' on string 'Botón'
    at Object.<anonymous> (/opt/traduora/src/formatters/jsonnested.js:60:42)
    at Generator.next (<anonymous>)
    at /opt/traduora/src/formatters/jsonnested.js:7:71
    at new Promise (<anonymous>)
    at __awaiter (/opt/traduora/src/formatters/jsonnested.js:3:12)
    at Object.exports.jsonNestedExporter (/opt/traduora/src/formatters/jsonnested.js:44:40)
    at ExportsController.<anonymous> (/opt/traduora/src/controllers/exports.controller.js:100:47)
    at Generator.next (<anonymous>)
    at /opt/traduora/src/controllers/exports.controller.js:19:71
    at new Promise (<anonymous>)
172.25.0.19 - GET /api/v1/projects/999d01be-56ff-4117-af25-d34640adaeea/exports?locale=es_GT&format=jsonnested HTTP/1.1 500 83 - 20.676 ms

[anthonynsimon]

Thanks for the detailed example!

In this particular case, it seems to be due conflicting terms when exporting it as json nested.

The term “button” has a value assigned, but also “button.test”, which the exporter cannot resolve because that term already has a translation value assigned and cannot be nested.

I agree that the error message is not very helpful in this case, I will add a few test cases and if this is the issue then will improve the provided error.

[d4rth-v4d3r]

Will we going to be able to have such cases in the future? I have an API that generates names like that, I ask this question because in web interface it works well.

[anthonynsimon]

Internally they are stored in a flat format, it’s only when exporting that the per-format rules would apply.

The issue with nested JSON is that there is no way to nest the term if there is a value already defined there:

In the following example, where should button: Bóton go? Notice the button term is already defined as a nested object, and cannot have a string value at the same time.

{
  "button": {   <--- "Botón" string would have to go here
    "test": "Botón de prueba"
  }
}

Likewise, if we reverse following example there is no place to put the nested value button.test: Botón de prueba:

{
  "button": "Botón"  <--- Nested object should go here
}

Of course, it is then possible to use a nested value to represent this case like this:

{
  "button": {
    "_value": "Botón"
    "test": "Botón de prueba"
  }
}

So that button._value and button.test can exist on the same term key. But that complicates things further because: 1) The application that would use this would also need to understand this edge case, 2) what happens if someone actually wants to have the term button._value (it is now a reserved case and cannot be used properly).

[arelstone]

@anthonynsimon This makes sense. I think think it would make sense to use the ._value.

Maybe the error message should be changed to tell the user what is actually the issue instead of throwing an Internal Error. Well I know it is an Internal Error, but this error message makes no sense to the user.

[coeing]

We had the issue with keys overlapping in nested JSON a few times as well. An error message about the keys which caused the error would be very helpful. Right now we have to check our (hundreds of) keys manually :)