hummingbot icon indicating copy to clipboard operation
hummingbot copied to clipboard
verified publisher icon hummingbot

V2 strategy - Stuck on creating orders after stop command initiated

Open rapcmia opened this issue 1 year ago • 1 comments

Describe the bug

When testing V2 strategies, found a behavior when a user initiated the stop command while the executor is in the process of creating orders, it continuously creating order but no order created event.

2024-05-27 17:27:02,390 - 12451 - hummingbot.client.hummingbot_application - INFO - start command initiated.
2024-05-27 17:27:03,001 - 12451 - hummingbot.strategy.script_strategy_base - WARNING - binance is not ready. Please wait...
2024-05-27 17:27:03,634 - 12451 - hummingbot.connector.exchange.binance.binance_exchange.BinanceExchange - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2024-05-27 17:27:04,001 - 12451 - hummingbot.strategy.script_strategy_base - WARNING - binance is not ready. Please wait...
2024-05-27 17:27:04,248 - 12451 - hummingbot.connector.exchange.binance.binance_api_order_book_data_source.BinanceAPIOrderBookDataSource - INFO - Subscribed to public order book and trade channels...
2024-05-27 17:27:04,248 - 12451 - hummingbot.connector.exchange.binance.binance_api_user_stream_data_source.BinanceAPIUserStreamDataSource - INFO - Successfully obtained listen key rD7fjuwbnOOd9r2A9vEtzXOQftCIVIqInmazTYh2xUEoDj7R6ojS4LvUdLih
2024-05-27 17:27:04,589 - 12451 - hummingbot.core.data_type.order_book_tracker - INFO - Initialized order book for FIRO-USDT. 1/1 completed.
2024-05-27 17:27:05,000 - 12451 - hummingbot.strategy.script_strategy_base - WARNING - binance is not ready. Please wait...
2024-05-27 17:27:05,872 - 12451 - hummingbot.client.hummingbot_application - INFO - stop command initiated.
2024-05-27 17:27:05,915 - 12451 - hummingbot.core.rate_oracle.rate_oracle - INFO - Network status has changed to NetworkStatus.CONNECTED. Starting networking...
2024-05-27 17:27:06,428 - 12451 - hummingbot.strategy.script_strategy_base - INFO - Creating FIRO-USDT buy order: price: 1.780143499999999999888495444 amount: 19.66133629114731481040282908.
2024-05-27 17:27:06,428 - 12451 - hummingbot.strategy_v2.executors.position_executor.position_executor - ERROR - Market object for buy order is not in the whitelisted markets set.
Traceback (most recent call last):
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/runnable_base.py", line 69, in control_loop
    await self.control_task()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 258, in control_task
    self.control_open_order()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 326, in control_open_order
    self.place_open_order()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 358, in place_open_order
    order_id = self.place_order(
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/executor_base.py", line 285, in place_order
    return self._strategy.buy(connector_name, trading_pair, amount, order_type, price, position_action)
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy/script_strategy_base.py", line 108, in buy
    return self.buy_with_specific_market(market_pair, amount, order_type, price, position_action=position_action)
  File "hummingbot/strategy/strategy_base.pyx", line 505, in hummingbot.strategy.strategy_base.StrategyBase.buy_with_specific_market
    return self.c_buy_with_specific_market(market_trading_pair_tuple, amount,
  File "hummingbot/strategy/strategy_base.pyx", line 528, in hummingbot.strategy.strategy_base.StrategyBase.c_buy_with_specific_market
    raise ValueError(f"Market object for buy order is not in the whitelisted markets set.")
ValueError: Market object for buy order is not in the whitelisted markets set.
2024-05-27 17:27:06,431 - 12451 - hummingbot.strategy.script_strategy_base - INFO - Creating FIRO-USDT sell order: price: 1.790856500000000000111504555 amount: 19.54372111891712149902618444.
2024-05-27 17:27:06,431 - 12451 - hummingbot.strategy_v2.executors.position_executor.position_executor - ERROR - Market object for sell order is not in the whitelisted markets set.
Traceback (most recent call last):
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/runnable_base.py", line 69, in control_loop
    await self.control_task()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 258, in control_task
    self.control_open_order()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 326, in control_open_order
    self.place_open_order()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 358, in place_open_order
    order_id = self.place_order(
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/executor_base.py", line 287, in place_order
    return self._strategy.sell(connector_name, trading_pair, amount, order_type, price, position_action)
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy/script_strategy_base.py", line 131, in sell
    return self.sell_with_specific_market(market_pair, amount, order_type, price, position_action=position_action)
  File "hummingbot/strategy/strategy_base.pyx", line 550, in hummingbot.strategy.strategy_base.StrategyBase.sell_with_specific_market
    return self.c_sell_with_specific_market(market_trading_pair_tuple, amount,
  File "hummingbot/strategy/strategy_base.pyx", line 573, in hummingbot.strategy.strategy_base.StrategyBase.c_sell_with_specific_market
    raise ValueError(f"Market object for sell order is not in the whitelisted markets set.")
ValueError: Market object for sell order is not in the whitelisted markets set.
2024-05-27 17:27:07,432 - 12451 - hummingbot.strategy.script_strategy_base - INFO - Creating FIRO-USDT buy order: price: 1.780143499999999999888495444 amount: 19.66133629114731481040282908.
2024-05-27 17:27:07,432 - 12451 - hummingbot.strategy_v2.executors.position_executor.position_executor - ERROR - Market object for buy order is not in the whitelisted markets set.
Traceback (most recent call last):
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/runnable_base.py", line 69, in control_loop
    await self.control_task()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 258, in control_task
    self.control_open_order()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 326, in control_open_order
    self.place_open_order()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 358, in place_open_order
    order_id = self.place_order(
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/executor_base.py", line 285, in place_order
    return self._strategy.buy(connector_name, trading_pair, amount, order_type, price, position_action)
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy/script_strategy_base.py", line 108, in buy
    return self.buy_with_specific_market(market_pair, amount, order_type, price, position_action=position_action)
  File "hummingbot/strategy/strategy_base.pyx", line 505, in hummingbot.strategy.strategy_base.StrategyBase.buy_with_specific_market
    return self.c_buy_with_specific_market(market_trading_pair_tuple, amount,
  File "hummingbot/strategy/strategy_base.pyx", line 528, in hummingbot.strategy.strategy_base.StrategyBase.c_buy_with_specific_market
    raise ValueError(f"Market object for buy order is not in the whitelisted markets set.")
ValueError: Market object for buy order is not in the whitelisted markets set.
2024-05-27 17:27:07,433 - 12451 - hummingbot.strategy.script_strategy_base - INFO - Creating FIRO-USDT sell order: price: 1.790856500000000000111504555 amount: 19.54372111891712149902618444.
2024-05-27 17:27:07,434 - 12451 - hummingbot.strategy_v2.executors.position_executor.position_executor - ERROR - Market object for sell order is not in the whitelisted markets set.
Traceback (most recent call last):
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/runnable_base.py", line 69, in control_loop
    await self.control_task()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 258, in control_task
    self.control_open_order()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 326, in control_open_order
    self.place_open_order()
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/position_executor/position_executor.py", line 358, in place_open_order
    order_id = self.place_order(
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy_v2/executors/executor_base.py", line 287, in place_order
    return self._strategy.sell(connector_name, trading_pair, amount, order_type, price, position_action)
  File "/Users/rapcomia/github/hummingbot/7046/hummingbot/strategy/script_strategy_base.py", line 131, in sell
    return self.sell_with_specific_market(market_pair, amount, order_type, price, position_action=position_action)
  File "hummingbot/strategy/strategy_base.pyx", line 550, in hummingbot.strategy.strategy_base.StrategyBase.sell_with_specific_market
    return self.c_sell_with_specific_market(market_trading_pair_tuple, amount,
  File "hummingbot/strategy/strategy_base.pyx", line 573, in hummingbot.strategy.strategy_base.StrategyBase.c_sell_with_specific_market
    raise ValueError(f"Market object for sell order is not in the whitelisted markets set.")
ValueError: Market object for sell order is not in the whitelisted markets set.
  • After initiating stop command, the executor continuously creating order.. but no order xxxOrderCreatedEvent
    • No active nor orders stuck on exchange when this issue occurred
    • This issue happened most of the time only after stop command is initiated for all tests today
  • Run stop command again does not stop the script
  • Run exit command ok ✅

Note: For monitoring purpose

Steps to reproduce

  1. Setup any controller config using default setup
  2. Start the script with controller config
    • (This needs timing) When the connector is about to do creating orders..
    • Run stop command and observed the script will continuously creating orders (see recording on link below)

Release version

latest and development 1.28.0

Type of installation

Source

Attach required files

Logs and recording files https://drive.google.com/drive/u/0/folders/1dFshfygRZDcDidMU8a-vxdTxtihYOnyp

rapcmia avatar May 28 '24 05:05 rapcmia

Reproduced when testing using diff connector on xemm controller. Same behavior after stop command, it continuously trying to create orders but failed with Market object for sell order is not in the whitelisted markets set.

2024-07-24 18:53:58,946 - 21123 - hummingbot.client.hummingbot_application - INFO - stop command initiated.
2024-07-24 18:53:59,248 - 21123 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1721818439.0, "order_id": "hbotBXPER61dfbb81d3f192196079e784c1cb71cb662acf56edb90", "exchange_order_id": "1773493652451328", "event_name": "OrderCancelledEvent", "event_source": "bitstamp"}
2024-07-24 18:53:59,262 - 21123 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order hbotBXPER61dfbb81d3f192196079e784c1cb71cb662acf56edb90.
2024-07-24 18:53:59,688 - 21123 - hummingbot.strategy_v2.executors.xemm_executor.xemm_executor - INFO - Trade profitability -0.3013162458653871493312526093 is below minimum profitability. Cancelling order.
2024-07-24 18:53:59,689 - 21123 - hummingbot.strategy.script_strategy_base - INFO - (XRP-EUR) Canceling the limit order hbotBXPER61dfbb81d3f192196079e784c1cb71cb662acf56edb90. [clock=NaT]
2024-07-24 18:53:59,976 - 21123 - hummingbot.core.event.event_reporter - EVENT_LOG - {"timestamp": 1721818439.0, "order_id": "hbotSXPER61dfbde87328b2196079e784c1cb71cb662acf56edb90", "exchange_order_id": "1773496292196352", "event_name": "OrderCancelledEvent", "event_source": "bitstamp"}
2024-07-24 18:53:59,986 - 21123 - hummingbot.connector.client_order_tracker - INFO - Successfully canceled order hbotSXPER61dfbde87328b2196079e784c1cb71cb662acf56edb90.
2024-07-24 18:54:00,690 - 21123 - hummingbot.strategy.script_strategy_base - INFO - Creating XRP-EUR buy order: price: 0.4015819424945737753584248149 amount: 27.12869854590175793966577443.
2024-07-24 18:54:00,691 - 21123 - hummingbot.strategy_v2.executors.xemm_executor.xemm_executor - ERROR - Market object for buy order is not in the whitelisted markets set.
Traceback (most recent call last):
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/runnable_base.py", line 69, in control_loop
    await self.control_task()
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/xemm_executor/xemm_executor.py", line 122, in control_task
    await self.control_maker_order()
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/xemm_executor/xemm_executor.py", line 128, in control_maker_order
    await self.create_maker_order()
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/xemm_executor/xemm_executor.py", line 196, in create_maker_order
    order_id = self.place_order(
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/executor_base.py", line 285, in place_order
    return self._strategy.buy(connector_name, trading_pair, amount, order_type, price, position_action)
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy/script_strategy_base.py", line 108, in buy
    return self.buy_with_specific_market(market_pair, amount, order_type, price, position_action=position_action)
  File "hummingbot/strategy/strategy_base.pyx", line 505, in hummingbot.strategy.strategy_base.StrategyBase.buy_with_specific_market
    return self.c_buy_with_specific_market(market_trading_pair_tuple, amount,
  File "hummingbot/strategy/strategy_base.pyx", line 528, in hummingbot.strategy.strategy_base.StrategyBase.c_buy_with_specific_market
    raise ValueError(f"Market object for buy order is not in the whitelisted markets set.")
ValueError: Market object for buy order is not in the whitelisted markets set.
2024-07-24 18:54:00,698 - 21123 - hummingbot.strategy_v2.executors.xemm_executor.xemm_executor - INFO - Trade profitability -0.3012180336859789268274121025 is below minimum profitability. Cancelling order.
2024-07-24 18:54:00,698 - 21123 - hummingbot.strategy.script_strategy_base - INFO - (XRP-EUR) Canceling the limit order hbotSXPER61dfbde87328b2196079e784c1cb71cb662acf56edb90. [clock=NaT]
2024-07-24 18:54:01,698 - 21123 - hummingbot.strategy.script_strategy_base - INFO - Creating XRP-EUR buy order: price: 0.4016014983625843472654628221 amount: 27.12869854590175793966577443.
2024-07-24 18:54:01,699 - 21123 - hummingbot.strategy_v2.executors.xemm_executor.xemm_executor - ERROR - Market object for buy order is not in the whitelisted markets set.
Traceback (most recent call last):
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/runnable_base.py", line 69, in control_loop
    await self.control_task()
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/xemm_executor/xemm_executor.py", line 122, in control_task
    await self.control_maker_order()
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/xemm_executor/xemm_executor.py", line 128, in control_maker_order
    await self.create_maker_order()
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/xemm_executor/xemm_executor.py", line 196, in create_maker_order
    order_id = self.place_order(
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/executor_base.py", line 285, in place_order
    return self._strategy.buy(connector_name, trading_pair, amount, order_type, price, position_action)
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy/script_strategy_base.py", line 108, in buy
    return self.buy_with_specific_market(market_pair, amount, order_type, price, position_action=position_action)
  File "hummingbot/strategy/strategy_base.pyx", line 505, in hummingbot.strategy.strategy_base.StrategyBase.buy_with_specific_market
    return self.c_buy_with_specific_market(market_trading_pair_tuple, amount,
  File "hummingbot/strategy/strategy_base.pyx", line 528, in hummingbot.strategy.strategy_base.StrategyBase.c_buy_with_specific_market
    raise ValueError(f"Market object for buy order is not in the whitelisted markets set.")
ValueError: Market object for buy order is not in the whitelisted markets set.
2024-07-24 18:54:01,700 - 21123 - hummingbot.strategy.script_strategy_base - INFO - Creating XRP-EUR sell order: price: 0.7564414074017443862412641000 amount: 27.12869854590175793966577443.
2024-07-24 18:54:01,700 - 21123 - hummingbot.strategy_v2.executors.xemm_executor.xemm_executor - ERROR - Market object for sell order is not in the whitelisted markets set.
Traceback (most recent call last):
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/runnable_base.py", line 69, in control_loop
    await self.control_task()
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/xemm_executor/xemm_executor.py", line 122, in control_task
    await self.control_maker_order()
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/xemm_executor/xemm_executor.py", line 128, in control_maker_order
    await self.create_maker_order()
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/xemm_executor/xemm_executor.py", line 196, in create_maker_order
    order_id = self.place_order(
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy_v2/executors/executor_base.py", line 287, in place_order
    return self._strategy.sell(connector_name, trading_pair, amount, order_type, price, position_action)
  File "/Users/rapcomia/github/hummingbot/7102/hummingbot/strategy/script_strategy_base.py", line 131, in sell
    return self.sell_with_specific_market(market_pair, amount, order_type, price, position_action=position_action)
  File "hummingbot/strategy/strategy_base.pyx", line 550, in hummingbot.strategy.strategy_base.StrategyBase.sell_with_specific_market
    return self.c_sell_with_specific_market(market_trading_pair_tuple, amount,
  File "hummingbot/strategy/strategy_base.pyx", line 573, in hummingbot.strategy.strategy_base.StrategyBase.c_sell_with_specific_market
    raise ValueError(f"Market object for sell order is not in the whitelisted markets set.")
ValueError: Market object for sell order is not in the whitelisted markets set.

logs_conf_v2_xemm_bitstamp_1.log

rapcmia avatar Jul 24 '24 11:07 rapcmia