Endpoint request: /markets/{region_id}/history/{date}

[ETristram]

Feature request

Route

/markets/{region_id}/history/{date}

Authentication

No

Use case

I thought I remembered a discussion about this some time ago, but couldn't find an issue for this. Ran across this again while currently porting the access to this endpoint from CREST to ESI.

Adam4eve.eu stores EVE market information of all types in all regions in its local database in order to pre-process it for example for trend analysis etc. Due to this on each day ideally A4E would require the market history information of all items in all regions for the previous day, since for older days the information is already in the local DB.

Using the existing market history endpoint in order to get yesterdays market data (mainly order_count and volume) A4E needs to query ESI (previously CREST) for each item individually, gets ~400 rows from ESI of which only 1 row is needed and inserted into the A4E DB.

Thus getting yesterdays market history values for "The Forge" for all items with active market orders results in ~12000 queries. Doing that for all regions increases that number massively.

Proposition

If a date is supplied instead of a typeID then ESI would return the market history information of all traded types during that day for that day.

This would cut down the needed requests for the above use case to the number of regions, thus max 100, and only transfer the needed information without the 99% redundant data.

Example returns

[ { "type_id": 37, "order_count": 1634, "volume": 58468885, "highest": 399.58, "average": 390, "lowest": 380.09 }, { "type_id": 38, "order_count": 1790, "volume": 82197469, "highest": 399.09, "average": 394.98, "lowest": 382 }, { "type_id": 39 "order_count": 1980, "volume": 84509059, "highest": 397.8, "average": 395, "lowest": 385 }, .... all other traded types ... ]

[Aidansavage]

this runs into caching issues iirc. So generally it's left up to developers to handle isolating the most recent data they actually want. I think it's also the reason other history related endpoints dont take dates as arguments or parameters.

[trixieflatline]

I agree with the OP. The current history API is perfect for the use case of reproducing the in-game history graph for an item on demand, but very inefficient if you want to keep a continuously updated history for all items and regions.

[Seavert]

I would rather suggest to split new and old data. Optimal term can be decided (in-game market browser allready has 5, 10 days). For example, 10 days. Then recent 10 days can be returned in first endpoint, and earlier days - stored in second static one.

[Havenard]

That would be nice, I hate having to download all history data every time I want to update my stuff, given I already have most of it stored locally and only need the changes of the past few days.

[p-groarke]

bump, the history route is overtaxing when doing analysis on recent data (last week, last month, ect).