orders in pending cancel state

[d-e-s-o]

Just today I accumulated two orders in pending_cancel state, that I neither can cancel nor do ANYTHING with, really.

GET https://paper-api.alpaca.markets/v2/orders/e4f99109a1414575a029a39a8ee2ea7f: body={"id":"e4f99109-a141-4575-a029-a39a8ee2ea7f","client_order_id":"94905de0-dc58-425e-8345-aaf869e3f7d2","created_at":"2020-03-23T13:50:53.230965Z","updated_at":"2020-03-23T18:07:35.842588Z","submitted_at":"2020-03-23T13:50:53.478662Z","filled_at":null,"expired_at":null,"canceled_at":null,"failed_at":null,"replaced_at":null,"replaced_by":null,"replaces":"b8639e43-64a9-428a-b829-4726a05acd6a","asset_id":"a87b8711-d78f-48e3-b8b9-3b88483f2057","symbol":"CGC","asset_class":"us_equity","qty":"83","filled_qty":"0","filled_avg_price":null,"order_class":"simple","order_type":"limit","type":"limit","side":"buy","time_in_force":"day","limit_price":"12.03","stop_price":null,"status":"pending_cancel","extended_hours":false,"legs":null}

Here is how it came to be:

PATCH https://paper-api.alpaca.markets/v2/orders/b8639e4364a9428ab8294726a05acd6a: status=200
PATCH https://paper-api.alpaca.markets/v2/orders/b8639e4364a9428ab8294726a05acd6a: body={"id":"e4f99109-a141-4575-a029-a39a8ee2ea7f","client_order_id":"94905de0-dc58-425e-8345-aaf869e3f7d2","created_at":"2020-03-23T13:50:53.230965Z","updated_at":"2020-03-23T13:50:53.504311386Z","submitted_at":"2020-03-23T13:50:53.478662499Z","filled_at":null,"expired_at":null,"canceled_at":null,"failed_at":null,"replaced_at":null,"replaced_by":null,"replaces":"b8639e43-64a9-428a-b829-4726a05acd6a","asset_id":"a87b8711-d78f-48e3-b8b9-3b88483f2057","symbol":"CGC","asset_class":"us_equity","qty":"83","filled_qty":"0","filled_avg_price":null,"order_class":"simple","order_type":"limit","type":"limit","side":"buy","time_in_force":"day","limit_price":"12.03","stop_price":null,"status":"new","extended_hours":false,"legs":null}
DELETE https://paper-api.alpaca.markets/v2/orders/e4f99109a1414575a029a39a8ee2ea7f}: status=204
trade_update: {"event":"order_cancel_rejected","order":{"asset_class":"us_equity","asset_id":"a87b8711-d78f-48e3-b8b9-3b88483f2057","canceled_at":null,"client_order_id":"94905de0-dc58-425e-8345-aaf869e3f7d2","created_at":"2020-03-23T13:50:53.230965Z","expired_at":null,"extended_hours":false,"failed_at":null,"filled_at":null,"filled_avg_price":null,"filled_qty":"0","id":"e4f99109-a141-4575-a029-a39a8ee2ea7f","legs":null,"limit_price":"12.03","order_class":"simple","order_type":"limit","qty":"83","replaced_at":null,"replaced_by":null,"replaces":"b8639e43-64a9-428a-b829-4726a05acd6a","side":"buy","status":"pending_cancel","stop_price":null,"submitted_at":"2020-03-23T13:50:53.478662Z","symbol":"CGC","time_in_force":"day","type":"limit","updated_at":"2020-03-23T13:50:54.065242Z"},"reason":"UNKNOWN_ORDER","timestamp":"2020-03-23T13:50:54.101543927Z"}}

So the order cancellation got rejected and since then the order is an "UNKNOWN ORDER". Every repeated cancellation since then is producing the same result.

1. Why is it stuck?
2. Why is order cancellation (DELETE) ALWAYS returning 204 (success; when clearly there was no success)? That's not specific to this issue but a general observation.
3. Please don't close this issue tomorrow saying the order is cancelled now. It does not matter. It has to happen when the request came in and not when somebody tinkered with the backend or the market got closed in between.
4. Please fix your state machine to no longer report an UNKNOWN ORDER as active. It's either this or that. Tests, anybody?

[umitanuki]

Hmm it's an interesting case and I believe this is a race condition that exists in the paper trading simulation. Please note that any order interaction behind the scene is asynchronous and some of the behavior may sound surprising to you from REST standpoint but this is how orders are processed. That said,

0. Seems like order A is being replaced by order B (PATCH) and B got a cancel request (DELETE).
1. An order (B in this case) needs to receive cancel acknowledgement from the venue to be released as canceled order, but somehow the acknowledgement was not sent back.
2. There is a slight confusion about DELETE method. This is a cancel request that doesn't mean success code tells the order has been canceled. It is success because the request is made.
3. I understand
4. So the question is how this order got into this state. It has to do with the replace request while order is being filled.

[d-e-s-o]

Thanks for your response.

3. There is a slight confusion about DELETE method. This is a cancel request that doesn't mean success code tells the order has been canceled. It is success because the request is made.

I figured. But that's not in line with the behavior of other APIs, is it? If I look at an order POST or a position DELETE I get back the status immediately, if I don't have enough funds, for example. That's also just a request, is it not? Case in point:

DELETE https://paper-api.alpaca.markets/v2/positions/TSLA: response={"available":"0","code":40310000,"existing_qty":"23","held_for_orders":"23","message":"insufficient qty available for order (requested: 23, available: 0)","related_orders":["47448ebc-491c-4f59-b716-75f8c3949c53"],"symbol":"TSLA"}

or

POST https://paper-api.alpaca.markets/v2/orders}: body={"symbol":"GE","qty":"459","side":"buy","type":"limit","time_in_force":"day","limit_price":"6.5271","stop_price":null,"extended_hours":false,"client_order_id":null}
status: 403
response={"available":"0","code":40310000,"existing_qty":"1367","held_for_orders":"1367","message":"insufficient qty available for order (requested: 459, available: 0)","related_orders":["fe845565-6e1d-4894-8642-2fef279f6aeb"],"symbol":"GE"}

Why have so vastly different behavior? A guiding principle for API design should be the principle of least astonishment. I'd argue you are violating it pretty badly.

[umitanuki]

I hear you, but there are different layers behind the API and I know that's confusing. The poistion API failure of your example is telling that such request was not even made due to the preliminary check. Note DELETE /v2/orders can also fail and return error code if one of such check fails or more fundamental issue occurs (such as the case when the system cannot access the order information). Similarly, the position API that you show can also succeed and later the orders made by such request fail and you will get rejection message in the async websocket.

[d-e-s-o]

The poistion API failure of your example is telling that such request was not even made due to the preliminary check.

And I would hope that there would be such a preliminary check that an order IS KNOWN before anything is done with it :-)

So I have a total of four orders being stuck now:

5a70e163-9fc5-47bd-8653-d42031d61560 e4f99109-a141-4575-a029-a39a8ee2ea7f 7f5b4ca7-6555-45a0-85c7-46e8958fceef 48168df1-bd76-44e2-a78a-0259962fb814

Can this please be fixed?

[d-e-s-o]

Counting nine now.

a66c959f-5f09-42da-9ab1-a368a7586539 16cd41b8-8744-4fb0-8b18-c347947e5f26 fecad748-b710-4b4d-b0b3-2c36d871ab30 562cd929-e730-4cb2-86e8-da55e0701ee2 a21215df-dae1-4c8a-8560-5501bb711222 5a70e163-9fc5-47bd-8653-d42031d61560 e4f99109-a141-4575-a029-a39a8ee2ea7f 7f5b4ca7-6555-45a0-85c7-46e8958fceef 48168df1-bd76-44e2-a78a-0259962fb814

Can't tell me you are lacking reproductions.

Is this how live customers are supported, too, then, eh? Because, allow me to quote from https://github.com/alpacahq/Alpaca-API/issues/104#issuecomment-590487194:

we take the paper trading as seriously as the live trading

Yeah, right. I believe every word.

[danieljrich]

Let's get this going again. I'm having this issue in an Alpaca Live account. I wasn't able to liquidate one of my positions as it was just returning insufficient qty available for order (requested: xxx, available: 0) In the dashboard, there are no open or pending orders. Yet when I hit the API endpoint, I can see the order. I hit it with a cancel request, and now it's just stuck in the pending_cancel status. Again, this is a live account. Kinda sucky to have this happening with real money. To add it it, this pending_cancel order still doesn't show up in the dashboard. Just returned in API calls.

[joeatbayes]

Has this been resolved? I faced a similar issue with TDA today and had to call into customer support. I was considering a different broker because I need a reliable API behavior or it could cost money when an existing sell order could not be replaced with a new sell order as market conditions change.

[d-e-s-o]

Has this been resolved?

Of course not. What a question is that? Fixing a race condition in seven months. C'mon.

New features is what we need!!1 Trading View integration!! Heck yeah! Tradetron! Yippee ki-yay!! Finally the last non-coder can also have his/her orders stuck!

Foresight that this could affect a few more people? Well, you know...it has been rare so far, hasn't it? I mean three users....a couple of orders here, a couple more there...nothing that support can't handle. Plus, I am sure they are happy interacting with a human for a change, those nerds. And what are these guys doing anyway, submitting an order and cancelling it in rapid succession? Is that even a blessed workflow? Has it been signed off? Nope?

If only this guy had not explicitly asked not to close this issue in his first comment; we would look like a assholes just doing exactly that; everything could have been so easy: we would have had "an update" and it was magically solved without further comment...now people actually find this stupid issue and it no longer seems like a one-off *grrrrr*

[/sarcasm]

(I am not bitter, just having fun :D Oh, and if you think that's bad...see how they handle stock splits; another exceedingly rare event that, you know, just doesn't quite meet the shininess bar to be done right)

[danieljrich]

Sorry, never came back and shared my resolution. I just ran a DELETE on the /v2/orders endpoint to delete all orders, and then DELETE on /v2/positions to close all positions.

Have you tried turning it off and on again?

[jeffwlandry]

paper trading. I have two orders for the same ticker stuck in pending cancel. I have a position in the same stock. A race condition may have created this situation, I'm certain a coding error on my part is at fault. HOWEVER, not being able to clear out outstanding orders is not acceptable. At this time the orders have been outstanding for 2+ hours of open trading and 8 hours after trading closed. I have run delete all orders and all positions multiple times and not yet succeeded.

[jabowery]

Is there a finite state machine description of the order process?

[leonarduschen]

Got the same error when running a DELETE on /positions?cancel_orders=True

This is not resolved yet? It's been a full year now.