Poloniex: "Invalid limit" when requesting order book

[lazydroid]

Describe your environment

• Operating system: Linux (Ubuntu flavour)
• Python Version: 3.10.12
• CCXT version: 4.2.3
• Freqtrade Version: 2024.7-dev-3b05d680c

Note: All issues other than enhancement requests will be closed without further comment if the above template is deleted or not filled out.

Describe the problem:

When trying to get an order book from Poloniex, freqtrade uses "limit=1", while legal values are "5, 10 etc..."

Reference to the docs: https://api-docs.poloniex.com/spot/api/public/market-data#order-book

Steps to reproduce:

1. this fails: https://api.poloniex.com/markets/WIF_USDT/orderBook?limit=1
2. this works: https://api.poloniex.com/markets/WIF_USDT/orderBook?limit=5
3. this works too: https://api.poloniex.com/markets/WIF_USDT/orderBook

Observed Results:

• What happened?

When running in dry-run mode, when requesting order book, the request fails.

• What did you expect to happen?

Order book should be requested flawlessly.

Relevant code exceptions or logs

Note: Please copy/paste text of the messages, no screenshots of logs please.

Traceback (most recent call last):
  File "/home/lenik/freqtrade/.venv/lib/python3.10/site-packages/ccxt/base/exchange.py", line 651, in fetch
    response.raise_for_status()
  File "/home/lenik/freqtrade/.venv/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.poloniex.com/markets/WIF_USDT/orderBook?limit=1

[lazydroid]

apparently, setting "order_book_top": 5 in the config does not help, probably because of

exchange/exchange.py: line 1117               orderbook = self.fetch_l2_order_book(pair, 1)

changing 1 -> 5 made things better, but I believe this is just a hack, while the possible values should come from the exchange.

[xmatthias]

In theory, freqtrade supports every exchange ccxt supports, which would also include poloniex. In practice, there's often nuanced differences which will require slightly different configurations - usually done through the exchange's subclass _ft_has.

Now you're the first to ask for that (at least as far as i remember), so you're probably the first to have a look at this. we have a rough guideline on things to check in the contributors section of the documentation. While it's probably not complete, it should give you a good idea if the exchange will work for spot trading.

If you encounter specific problems, we're happy to assist up to a certain point - but the expectation for new exchanges is that the user interested is doing the legwork / most of the testing work - which also assumes a somewhat familiarity with python / testing/debugging things in python.

If you're new to python / development - you're almost certainly better of with one of the supported exchanges.

---

For what you're asking - the proper key in _ft_has is l2_limit_range (for example configured on binance https://github.com/freqtrade/freqtrade/blob/b88b1cbbdb8fb0ca28dbbaadedf90db80d221f82/freqtrade/exchange/binance.py#L32). Feel free to submit a pull request adding a subclass for Poloniex and configuring what's necessary - however please make sure to also perform at least the basic, unauthenticated tests in the above link (e.g. what's the correct candle-limit).

Please note: adding that subclass will not make it a supported exchange, and won't guarantee further success - as you can still encounter tons of minor hickups that all need manual investigation and solutions.

[lazydroid]

@xmatthias thank you very much for your detailed answer, I want to make this exchange work and I'll try writing missing parts, since so far I was able to use that exchange with my own hand-written scripts.

I have one question, though, and I'm sorry that probably only you know the correct answer. Poloniex is marked by default as unsupported because it's lacking "fetch_order" that returns open/closed orders in one call. I saw there's an emulation function in the source that makes two separate requests for open and for closed orders, then unites both responses. Is using this emulation function enough or there's something else missing that I don't know about?

[xmatthias]

You won't be happy with emulation of THAT endpoint. it'll make the bot slow (doing 2 calls) when it counts most (when you got an open order).

From the API itself - are you sure that call ain't there? that "hard limit" was set 4 years ago (based on commit history!) - and things may have changed (however we do not constantly monitor for these things).

Purely from looking at the api docs - it would appear that the order Details endpoint might provide an order independent of it's state.

I saw there's an emulation function in the source that makes two separate requests for open and for closed orders

Careful about details here ;) emulation is there for fetch_orders - the (manual) limit talks about fetch_order (singular) - which is quite a difference - as the latter requires an order-id.

this endpoint can be emulated - and probably will need to be - but it's not really a problem, as this is not called frequently. you'll probably have to manually update fetchOrders so it triggers the emulation code.

[xmatthias]

Going to close this as there's been no further updates.

however - please let us know (or submit a PR) if you're still diving into this and find that poloniex is working fine (with some changes to freqtrade, obviously - as we're currently hard-blocking them).

[lazydroid]

Thank you very much! I have found an interesting bug in CCXT that prevented proper candle downloading. Fixed that, keep experimenting. Looks feasible, but might take some time.

[xmatthias]

feels kinda like almost every new exchange has it's own candle download bug in ccxt :laughing: can tell from experience - though they're usually quite quick to fix these

[lazydroid]

sorry for asking for your time answering things that were probably already discussed hundred of times, still I could not figure the answer:

1. candle downloading, the docs say something that I can append historical data, but cannot fill the possible holes in the data -- where this limitation came from, from what I've tried, I can download just the missing candles to get the full set, no matter where the missing parts are -- maybe I'm missing something or other exchanges are not as easy as Poloniex / Binance ?

2. When I have 50 pairs trading and enough money to open maybe 10 trades, is there any way to intentionally select and choose the most profitable (right now) pairs for trading as opposed to shuffling the list and using the top 10 (which seems to me the current approach). Again, sorry if I miss the obvious.

3. I dearly miss the old long output from hypeopting where all traded pairs are listed along with the stats for every pair -- is it gone completely or is there any option to turn it on?

Thank you in advance!!

[xmatthias]

well neither 2 nor 3 has something to do with poloniex.

for 1 - i'm not sure what you're refering to - freqtrade tries pretty hard to not cause any holes - HOWEVER - it assumes default settings of 500 candles per call to OHLCV (unless set differently in the exchange class). for some exchanges - that's too much (causing holes) - for most - they COULD provide more - so it should be configured to what's actually used - which will both give you more data in live trading - and speed up data downloading.