Uniques selector just like in v0.1

[thirdy]

Add an easy way to select uniques categorized into types. In the first version of Durian, there was a list that had all the uniques which also shows the graphics:

[zocke1r]

that looks nice why was that ever abandoned?

[thirdy]

It was only something for me to try out. A prototype to see if I can pull off a tool using exiletools.

Now with GGG's new api, we'll be leaping into the future very soon and be done with this arcane technology we're using.

Right now I'm thinking of incorporating Durian into Blackmarket:

Well originally, Blackmarket had this search by terms idea while Durian only had the automation. I might rename the project into Durian-Blackmarket, cause I like using crazy names. What do you think?

[zocke1r]

blackmarket durian sounds better, as it implies that the durian was bought on the black market, and i really hope ggg releases the new API soon, but i dont know if i prefer the tiles over the list display

[kyri0]

That was my hope from the start, making them both into one.

Thing is, we gotta think of what will stay and what will go away. I believe the way to go is importing the Durian features to Blackmarket, as it already is more comercial, there are many visual feedbacks and the option to chose from drop down menus, as in poe.trade, making the transition easier.

Of course, the hability of searching through search terms has to come to it.

About displaying, would it be hard to make both options? List and tiles?

And if it's possible, try to keep the option to search at both indexers, maybe at the same time? hahahaha

[thirdy]

with the upcoming ggg api, the data from exiletools will be almost real time; rendering poe.trade obsolete as it is. unless poe.trade starts providing an api like exiletools does, there's no more reason to grab data from poe.trade.

[zocke1r]

man that would be a great day, loosening the tight grip this site has on poe trading,after all competetion is good for the consumer, and we are the consumer

[thirdy]

we now have a new lands to conquer:

exiletools provides an Elastic Search API. Elasticsearch is what you would use if you want to build a database that can be searched in advanced ways including a search engine like google search. It also provides an API out of the box. (btw, ES is open source).

Elasticsearch is big, and most of it, we probably don't need, like ranking search results. And so their documentation is big. But hopefully this one is the hidden gem to use:

https://www.elastic.co/guide/en/elasticsearch/reference/2.2/query-dsl.html

[thirdy]

all the stuff I said is probably really alien. I'll be posting a much simpler manifesto on the direction I'd like to go.

But for sure,

• we'll use ExileTools which provides us Elastic Search API.
• put some time thinking on how search terms can be used for this API

[thirdy]

One advanced possible with elastic is the ability to sum modifiers on multiple items. It might be possible to search "130% fire res from the total fire res from boots helm gloves"

But the elastic search doc too much for me to understand as I didn't listen to my statistics class much:

https://www.elastic.co/blog/out-of-this-world-aggregations

[zocke1r]

yay the first few updates to the trade api are coming even before ascendancy

[thirdy]

Did some playing around (and im late for work :D)

curl -XPOST -H 'Authorization: DEVELOPMENT-Indexer' -d '{
    "query": {
        "query_string": {
           "query": "attributes.baseItemType:Card AND shop.updated:>1456191110199"
        }
    }
}' 'http://api.exiletools.com/index/_search'

This here:

attributes.baseItemType:Card AND shop.updated:>1456191110199

is an example of a simple query to exiletools. That large number there is the millisec I got using this website (http://currentmillis.com/).

Here's a sample item from this query:

{
  "_index": "poe20160130",
  "_type": "item",
  "_id": "1512065:53adf8b625cfea6aa04e4a21a02753e9",
  "_score": 4.9783344,
  "_source": {
    "info": {
      "fullName": "The Catalyst",
      "icon": "http:\/\/webcdn.pathofexile.com\/image\/Art\/2DItems\/Divination\/InventoryIcon.png",
      "flavourText": "Simple actions can lead the world to an early grave.",
      "typeLine": "The Catalyst",
      "name": "The Catalyst",
      "tokenized": {
        "DivinationReward": "currencyitem: vaal orb",
        "fullName": "the catalyst",
        "flavourText": "simple actions can lead the world to an early grave."
      }
    },
    "shop": {
      "lastUpdateDB": "2016-02-16 15:31:30",
      "sellerIGN": "",
      "added": 1455654687000,
      "priceChanges": 0,
      "chaosEquiv": 0,
      "modified": 1455654687000,
      "saleType": "offer",
      "currency": "NONE",
      "threadTitle": "Batty's Shop",
      "amount": 0,
      "sellerAccount": "Battanious",
      "threadid": "1512065",
      "updated": 1456192404000,
      "verified": "YES"
    },
    "mods": {
      "Card": {
        "DivinationReward": "currencyitem: Vaal Orb"
      }
    },
    "attributes": {
      "equipType": "Card",
      "support": true,
      "identified": true,
      "corrupted": false,
      "explicitModsCount": 1,
      "baseItemType": "Card",
      "inventoryHeight": 1,
      "rarity": "Divination Card",
      "mirrored": false,
      "frameType": 6,
      "itemType": "Card",
      "league": "Talisman",
      "lockedToCharacter": false,
      "inventoryWidth": 1
    },
    "md5sum": "53adf8b625cfea6aa04e4a21a02753e9",
    "uuid": "1512065:53adf8b625cfea6aa04e4a21a02753e9",
    "properties": {
      "Card": {
        "Stack Size": "1\/3"
      }
    }
  }
}

The syntax is documented here: https://www.elastic.co/guide/en/elasticsearch/reference/2.2/query-dsl-query-string-query.html#query-string-syntax

@trackpete, this should be the same as the longcut where we send json documents huh?

[kyri0]

How do you guys know exactly what the new apis will do? And where have you seen the ETA on it?

The reason for keeping poe.trade is giving the user choice, that simple. But if that's not possible or too hard, then, no problem.

[pete-wn]

I have early access to the new API's for testing. I can't spoil anything, but basically as indicated the ExileTools index will be updating changes to items within 10-20s in most cases. The new API for items is pretty sweet - unfortunately I don't know anything about on-line/off-line stuff yet.

The v3 indexer is re-written from the ground up, but the actual document information should be almost all the same to v2 documents in Elastic Search. There will be one or two new fields to give some additional information, but the impact will be minimal.

Basically everything from my gist page (https://gist.github.com/trackpete) should still work. The mapping field tools and entire mapping documented at http://api.exiletools.com/info/indexer-docs.html will be accurate, except some of the shop attributes will go away. You can also see some example data reports at http://exiletools.com/uniques

The problem with going against poe.trade is that he doesn't want you to. ^_^ Not to mention it's terrible to load some poor guy's full web page then parse it instead of using an API service. And gives you so many less options.

Cheers!

[thirdy]

@pete, thanks for the reply. Btw, do have the websocket code in-place now? Would love to start working against it. I now have ideas on how I might use it.

I played around with the possibility of search terms in the browser. And here's what I've come up:

http://thirdy.github.io/search/index.html

@Eruyome, would awesome if you can work with me on this.

Not sure yet, but I would probably port Durian to use exile tools as I'm no web dev. But this definitely needs porting the search term definitions.

[pete-wn]

I don't have any of the filter stuff ready - right now the only code I have is a direct stream of every item changed, but it's all on the preview stuff that I can't share yet.

Also, I'm aware that searches are a bit slower than they should be right now - that's because GGG changed the item JSON data on me a few times so the index is over 300GB and I only have ~90GB of RAM available on the three nodes running ES. I probably need to write some aging-out code to get rid of 1+mo old data at some point.