Some /api/texts requests with commentary=1 return truncated json

[ronshapiro]

Some Calls to /api/texts seem to produce invalid json, presumably because their results are just too big? I've seen this for at least Genesis 1-4, though not 100% consistently.

E.g. curl https://www.sefaria.org/api/texts/Genesis.1\?commentary\=1\&pad\=0 | python -mjson.tool

Results:

curl: (92) HTTP/2 stream 0 was not closed cleanly: INTERNAL_ERROR (err 2)
Unterminated string starting at: line 1 column 18050017 (char 18050016)

I wonder if this is a client http/machine issue (it also fails in Chrome on Mac and Safari for me too). But perhaps above a certain size, an error should be returned instead of a bad response?

[ronshapiro]

Or perhaps a way to specify a list of which commentaries to select instead of selecting all of them, in case the list is quiet large?

[ronshapiro]

I think using StreamingHttpResponse may fix this: https://andrewbrookins.com/django/how-does-djangos-streaminghttpresponse-work-exactly

https://devdocs.io/django~1.11/ref/request-response#django.http.StreamingHttpResponse