Import tradables (& want-list) from SteamTrades

[Revadike]

Allow (new) users to import their tradables list directly from SteamTrades. When selecting this option, barter.vg scrapes https://www.steamtrades.com/trades/search?user=<USER_STEAM_ID64> and displays the trade topic titles on its page. The user will select which topic to import from. Then, barter.vg will scrape the topic page for a list of tradables and attempt its best to identify them. This will hopefully encourage more SteamTrades users to try out barter.vg!

[bartervg]

Wouldn't this be more appropriate for a browser extension? Export to appID CSV button? I could create / improve API to convert titles to appIDs.

[Revadike]

Wouldn't this be more appropriate for a browser extension? Export to appID CSV button? I could create / improve API to convert titles to appIDs.

No why? Best to offer this feature in the site natively!

[Revadike]

I'd still really like to see this option being offered! It's worth to mention that steamgifts' rate limits, don't apply to steamtrades, so you don't have to worry about that.

[bartervg]

It's worth to mention that steamgifts' rate limits, don't apply to steamtrades

source?

[Revadike]

It's worth to mention that steamgifts' rate limits, don't apply to steamtrades

source?

I went ahead and tested this for you. Just a simple script to do surpass the 120 req/min limit.

[bartervg]

I went ahead and tested this for you.

I didn't mean if it's technically possible, but if it is within ST's terms of service.

[Revadike]

ST is a wild west, surely you must know this. It's self-moderated and pretty much abandoned by its creator.

[bartervg]

Someone still pays for the server bills and the users suffer if performance degrades.

Beyond that, there is the problem is accuracy. How well is a ST trade list going to parse into Barter items? Should the importer have a chance to review before adding to tradables? Should this be a new sync type?

[Revadike]

I think these importing options, should cover most, if not all cases:

• Text detection vs link detection
• User selects link detection if (s)he is linking each item. This import option is likely most accurate.
• Text detection requires additional options:
  • Find and remove user defined keywords (like (DLC), or - Steam Key, or [x3])
  • List vs table
    • Table requires additional options:
      • Which column to detect game titles (default 1)

Some of these options are nice to have with regular importing too.

[bartervg]

Best to offer this feature in the site natively!

Remind me why this is true.

I can see how it could be useful, but it seems that it would require a large initial effort to make it work and frequent refinements.

Request https://www.steamtrades.com/trades/search?user=76561198042965266 then display all of the topics for the user to select the topics to import? Request https://www.steamtrades.com/trade/7Ypes/ search for the div with "have markdown"? use AI* to parse arbitrary HTML into item IDs? Present items to user to edit or import into tradable?

If the new user can setup a trade thread on ST, couldn't the same new user copy and paste their tradables into Barter.vg?

* more likely, an ever growing monstrosity of regex

[Revadike]

Lol, AI... I think you're massively complicating this in your head. It's not that hard. I could even make the PHP script myself, if you want. This is possible, since I do not really have to interact with the barter.vg codebase, except at the very end. The output will be a regular list of appids or game titles. Output will then go through the same algorithms as other import input.

[Revadike]

Best to use a DOM parser, instead of regex.

[bartervg]

I could even make the PHP script myself, if you want.

👍

[Revadike]

Oh boy, I'm a bit rusty on my php scripting.

[Revadike]

I could even make the PHP script myself, if you want.

👍

<?php
/* Text detection vs link detection
User selects link detection if (s)he is linking each item. This import option is likely most accurate.
Text detection requires additional options:
Find and remove user defined keywords (like (DLC), or - Steam Key, or [x3])
List vs table
Table requires additional options:
Which column to detect game titles (default 1) */

// input
$st_code = '7Ypes'; // Use $_POST or $_REQUEST
// $st_mode = 'link'; // Use $_POST or $_REQUEST
$st_mode = 'text'; // Use $_POST or $_REQUEST
$st_blacklist = 'DLC, Alpha, Beta, Demo'; // Use $_POST or $_REQUEST
$st_blacklist =  array_map('trim', explode(',', $st_blacklist));
$st_table = 1; // Use $_POST or $_REQUEST
$st_table_col = 0;  // Use $_POST or $_REQUEST, 0 = first column

// output
$st_output_ids = [];
$st_output_names = [];

$html = file_get_contents('https://www.steamtrades.com/trade/' . $st_code . '/');
$dom = new DomDocument();
@$dom->loadHTML($html);
$xpath = new DOMXpath($dom);
$have = $xpath->query("//div[contains(@class,'have')]")[0];

if ($st_mode == 'link') {
    $links = $have->getElementsByTagName('a');
    foreach ($links as $link) {
        $url = $link->getAttribute('href');
        $parsed_url = parse_url($url);
        if (isset($parsed_url['host']) && ($parsed_url['host'] === 'store.steampowered.com'  || $parsed_url['host'] === 'steamdb.info')) {
            $st_output_ids[] = $parsed_url['path'];
        }
    }
} else if ($st_mode == 'text') {
    if ($st_table) {
        $rows = $have->getElementsByTagName('tr');
        foreach ($rows as $row) {
            $item = $row->getElementsByTagName('td')[$st_table_col];
            $st_output_names[] = $item->textContent;
        }
    } else {
        $texts = $have->getElementsByTagName('p');
        foreach ($texts as $text) {
            $lines = explode('\n', $text->textContent);
            $st_output_names = array_merge($st_output_names, $lines);
        }
    }
    for ($i = 0; $i < count($st_output_names); $i++) {
        $st_output_names[$i] = str_replace($st_blacklist, '', $st_output_names[$i]);
        $st_output_names[$i] = trim($st_output_names[$i]);
    }
}

$st_output_ids = array_unique($st_output_ids);
$st_output_names = array_unique($st_output_names);

if (count($st_output_ids) > 0) {
    // use id import with $st_output_ids
} else if (count($st_output_names) > 0) {
    // use regular import with $st_output_names
}

echo var_dump($st_output_ids);
echo var_dump($st_output_names);

Test it yourself: https://phpsandbox.io/n/st-barter-dxits