yii-api icon indicating copy to clipboard operation
yii-api copied to clipboard

API: Batch queries with transactions support and error handling.

Open cebe opened this issue 11 years ago • 21 comments

follow up on #303

cebe avatar Jul 27 '14 20:07 cebe

POST /batch could accept JSON of the following format:

[
    "type": "DELETE",
    "url": "/article",
    "data": {
        "ID": 123,
    }
],
[
    "type": "POST",
    "url": "/user",
    "contentType": "json",
    "tempID": "%userID%", 
    "data": {
        "name": "Alex",
    }
],
[
    "type": "POST",
    "url": "/article",
    "data": {
        "created_by": "%userID%",
    }
],

Here we're passing tempID "%userID%" to the batch. That means ID of the created record will be assigned to a placeholder. Later we're using it in another request.

Response should give back responses in the same order as requests were made:

[
    {"success": true, "id": "1234"},
    {"success": true, "id": "1235"},
]

Response content is to be defined (I'm not sure the info above is enough).

samdark avatar Jul 27 '14 20:07 samdark

@samdark

Response should give back responses in the same order as requests were made:

More convenient way is to mark each request and response.

dizews avatar Jul 27 '14 21:07 dizews

@dizews you mean like the following?

{
    'r01': [
        "type": "DELETE",
        "url": "/article",
        "data": {
            "ID": 123,
        }
    ],
    'r02': [
        "type": "POST",
        "url": "/user",
        "contentType": "json",
        "tempID": "%userID%", 
        "data": {
            "name": "Alex",
        }
    ],
    'r03': [
        "type": "POST",
        "url": "/article",
        "data": {
            "created_by": "%userID%",
        }
    ]
}

and

{
    'r01': {"success": true, "id": "1234"},
    'r02': {"success": true, "id": "1235"}
}

samdark avatar Jul 27 '14 21:07 samdark

@samdark exactly.

dizews avatar Jul 27 '14 21:07 dizews

Makes sense.

samdark avatar Jul 27 '14 23:07 samdark

I always though batching would only apply to requests of the same type. Like reading a batch of users. Inserting a batch of products. ...

Doing it this way looks way too much as tunneling to me. When do you start batching? Once you have 50 commands in your queue? 10, 5, 2? If you once implemented batching, why not using it all alone? It would still work with single messages, wouldn't it? An it wouldn't require a second implementation dealing with single messages. Only that we're basically doing something more SOAP-like now.

I might be wrong with this, but it just does not feel right to me. Is this a common approach for batching? Any other frameworks taking this route? Or any books/ articles proposing such a solution?

bwoester avatar Jul 28 '14 08:07 bwoester

@bwoester batching is usually used when application is able to work offline so it accumulates commands till it's online when it sends all these to batch. I've used it in practice and, indeed, it's used for everything.

samdark avatar Jul 28 '14 08:07 samdark

@bwoester Like Any POST request a batch query do not use http cache. It is one of reason to use single GET request.

dizews avatar Jul 28 '14 08:07 dizews

@samdark I think batch response item should be a wrapper around a single response.

{
    "r01":{"status":201,"data":{}},
    "r02":{"status":200,"data":{}}
}

by this way we can have more flexible container. We can add a headers section or something else

dizews avatar Jul 28 '14 08:07 dizews

@dizews makes sense.

samdark avatar Jul 28 '14 09:07 samdark

Seems like Facebook uses a similar approach.

However, Google on the other hand simply uses multipart/mixed messages for their batches. What I like about their approach is: you don't need to introduce new JSON structures (resembling http messages, containing headers, body, ...). Instead, you simply use an already existing mechanism to group your messages.

https://developers.google.com/storage/docs/json_api/v1/how-tos/batch

Also, we could batch transparently in the background: just intercept sent messages and depending on how many chatting is going on, group them and send as multipart/mixed (or custom batch format), or send the intercepted message as is.

bwoester avatar Jul 28 '14 11:07 bwoester

Google's way seems to be a bit more mature. I like it.

samdark avatar Jul 28 '14 13:07 samdark

+1

dizews avatar Jul 28 '14 13:07 dizews

what about parsing the response on the client ? If using google's approach then too much unneeded parsing need to be done by client (compare to simple json_decode and so on), its syntax also confusing and cant be used easily in debug . Also another question is how to get request body , it is unavailable from get / post, should we read php://input instead ? I like @dizews suggestion about request and json format

Ragazzo avatar Aug 31 '14 13:08 Ragazzo

I have a scenario where 4 tables needs to be created and updated simultaneously. I am using transactions for creation where id of one table is foreign key of another table. But i found no solution for update . I wish this feature will be part of Yii ASAP.

mushahidh avatar Nov 25 '16 15:11 mushahidh

+1 This seems being on hold for a long time, any plans for this?

decimoseptimo avatar Jan 19 '17 22:01 decimoseptimo

Not for the next release unless someone from community will take care of it.

samdark avatar Jan 19 '17 22:01 samdark

Damn, not skilled for the task for the time being.

decimoseptimo avatar Jan 20 '17 00:01 decimoseptimo

hey, Sorry if posted on wrong place. I am newbie with yii2 api, I tried but not able to get how to call and make batch processing works.

If my url is abc.com/message-log POST and process single record is working fine, and can load records with GET on abc.com/message-log but as provided above abc.com/message-log/batch respond 404

Please let me know how to process batch records.

muet84 avatar Mar 29 '17 16:03 muet84

@muet84 because it's not implemented.

samdark avatar Mar 29 '17 21:03 samdark

Hi,

i created some kind of solution, it is based on google api batch solution. I would appreciate your comments.

still in first release. i need to do some testing, but it works for me.

https://github.com/p4it-kft/yii2-rest-batch

papppeter avatar Sep 02 '19 22:09 papppeter