Private tracker support

[abique]

Design and implement private tracker support.

1. How to extract the key?
   • Has the key to respect a standard like http://host/announce/key/ ?
   • The administrator could provide a regular expression to extract the key from the URL.
2. How to check authenticate and control resources access?
   • Here we probably need to communicate with an external process, probably a database to check the key, and allow or reject the announce.
3. How to count users upload and download?
   • This can be done by producing a simple stream in CSV format: timestamp ip port downloaded uploaded key

All of this can be done with a plugin interface, or with IPCs. But as we can build a plugin which forwards requests by IPCs to an other process, then the plugin interface solves both solutions.

[Zorlin]

1. http://blog.bitcomet.com/bitcomet/post_417/ provides some hints as to how it works on a protocol level.

There's also this definition here: http://www.bittorrent.org/beps/bep_0027.html#bep-12

Examples of announce URLs:

http://tracker.what.cd:34000/igmax2lw0vabtj0y9w8box2tvxwj4fws/announce http://please.passthepopcorn.me:2710/rhoymaldjuxixvyf2vgq3sf0r9rvjhi5/announce

(keys have been obfuscated for anonymity).

I don't believe the private key needs to actually be in the announce URL at all, but it makes it easier for users to tell what private key a torrent is using, as those are not normally exposed to the user directly. (I've seen a couple of "private" torrents that do NOT have the private key in the URL, but do use private keys).

1. My two primary concerns would be speed and scalability. Checking against a database many times a second could rapidly hurt performance; but I believe that's how Ocelot does it it and Ocelot is ridiculously fast... hmm.

Here's the relevant config stuff from Ocelot (an excellent but restrictively licensed tracker):

config::config() { host = "127.0.0.1"; port = 34000; max_connections = 512; max_read_buffer = 4096; timeout_interval = 20; schedule_interval = 3; max_middlemen = 5000;

announce_interval = 1800;
peers_timeout = 2700; //Announce interval * 1.5

reap_peers_interval = 1800;
del_reason_lifetime = 604800;

// MySQL
mysql_db = "gazelle";
mysql_host = "127.0.0.1:3306";
mysql_username = "***";
mysql_password = "***";

site_host = "localhost";
site_password = "********************************"; // MUST BE 32 CHARS

}

I'll try to get a dump of the DB schema it uses.

I'm fairly certain it gets around the DB performance hit by basically caching MySQL queries in memory - allowing the flexibility of a DB but without as much performance overhead. It flushes the caches every X seconds (schedule_interval?).

It largely follows the DB format XBTT uses, but with some neat variations.

I used MySQL logs to gather some info on how it works - these are in no particular order but give you some idea of how it might be implemented.

Grab list of users allowed to leech: SELECT ID, can_leech, torrent_pass FROM users_main WHERE Enabled='1';

Reset seeder and leecher count in DB: UPDATE torrents SET Seeders = 0, Leechers = 0;

Grab list of torrents, including freeleech/neutral status: SELECT ID, info_hash, freetorrent, Snatched FROM torrents ORDER BY ID; (if freetorrent = 1, freeleech mode; torrent only counts the uploaded data in the DB) (if freetorrent = 2, neutral mode; torrent does not count any data) (if freetorrent = 0, regular mode; both upload and download counts).

Clear XBTT user list?! TRUNCATE xbt_files_users;

There's lots of stuff besides that, but that's the basic stuff near as I can see.

So; to answer 3... Write it to SQL as you clear the buffer :P

[abique]

Maybe the db for private trackers could be into hefur? It would allow a custom and high performances format.

[abique]

By the way, checking against the database can scale if your table+index can fit in memory ;-)

How many users do/will you have?

[abique]

I think that with an interface like this one, we can implement anything: https://github.com/abique/hefur/blob/master/hefur/private-tracker-controller.hh

[Zorlin]

That seems like the easiest idea. I think aiming for Gazelle support is good - it's the most popular private tracker CMS now :)

• B

Maybe the db for private trackers could be into hefur?

— Reply to this email directly or view it on GitHubhttps://github.com/abique/hefur/issues/5#issuecomment-11609826.