KVIrc
KVIrc copied to clipboard
kvirc
Convert the serverdb file to an intermediate format
There are two problems with our current serverdb format:
-
It's hard to read and unparsable by anything other than KVIrc itself,
-
It's safe to assume half the entries, if not more, are dead.
So, I thought: I'll convert it to some other format a Python script can understand, and that way I'll have 1) a more readable format, and 2) a way to automatically check that all servers 2.1) resolve, and 2.2) can be connected to.
The result is here, but I think it's uglier than the butt of a baboon, that's why I don't want to commit it yet. So I'll leave this ticket here in case anybody has anything to comment. The idea is: when someone wants to change the server list, they change the .json file, execute a Python script, and that script converts the .json file to the .kvc file that KVIrc understands.
I was suggested to try YAML instead but there's no builtin module in Python to read that format and installing 3rd party modules in Python is annoying.
This is important anyway: checking which servers are still alive would be nice.
idea: collab with @tingping (HexChat) and others on a format that all (more than one) clients can use and maintain.
Well, I don't know what to say.
https://github.com/hexchat/hexchat/blob/ccd19be4b1ac70c9c4e2c91501efdc992257e9e6/src/common/servlist.c#L51-L365
HexChat can't directly consume JSON as that requires a library. So that only leaves ini files which seems like what you had.
I was thinking mIRC uses ini files but their implementation is horrible.
; For providing suggestions, new servers or networks see http://www.mirc.com/help/serverlist.html
[timestamp]
date=08/08/2008
[networks]
n0=Quakenet
n1=Undernet
n2=IRCnet
n3=EFnet
n4=DALnet
n5=WebChat
n6=GameSurge
n7=LinkNet
n8=UniBG
[servers]
n0=Random serverSERVER:irc.abjects.net:6666-6669,7000,9000GROUP:Abjects
n1=Random serverSERVER:irc.ablenet.org:6667GROUP:AbleNET
n2=Random serverSERVER:irc.absurd-irc.net:6667,7000GROUP:Absurd-IRC
n3=Random serverSERVER:irc.accessirc.net:6667GROUP:Accessirc
n4=Random serverSERVER:irc.acknet.org:6667-6669GROUP:ACKnet
n5=Random serverSERVER:irc.afternet.org:6667GROUP:AfterNET
n6=Random serverSERVER:irc.afterx.net:6667GROUP:AfterX
n7=Random serverSERVER:irc.allnetwork.org:6667GROUP:AllNetwork
n8=EU, SwedenSERVER:irc.ambernet.se:6667-6669GROUP:AmberNet
n9=Random serverSERVER:irc.amcool.net:6660-6669GROUP:AmCool
n10=Random serverSERVER:irc.angeleyez.net:6661-6669,7000GROUP:AngelEyez
n11=Random serverSERVER:irc.animeirc.de:6667GROUP:AnimeIRCnet
n12=Random serverSERVER:irc.aniverse.com:6661-6670GROUP:Aniverse
n13=Random serverSERVER:irc.arabchat.net:6667GROUP:ArabChat
n14=Random serverSERVER:irc.ascends.net:6666-6669,7000GROUP:Ascends
n15=AS, MYSERVER:irc.aseanchat.net:7000GROUP:AseanChat
n16=Random serverSERVER:irc.ashnet.org:6660-6669,7000GROUP:AshNet
n17=Random serverSERVER:irc.astrolink.org:6660-6667GROUP:AstroLink
n18=Random serverSERVER:irc.asylo.com:6660-6669GROUP:Asylo
n19=Random serverSERVER:irc.asylum-net.org:6661-6669,7000,7777GROUP:Asylumnet
n20=Random serverSERVER:irc.atrum.org:6667GROUP:Atrum
n21=Random serverSERVER:irc.austchat.net:6667GROUP:AustChat
n22=Random serverSERVER:irc.austnet.org:6667GROUP:AustNet
n23=Random AU serverSERVER:au.austnet.org:6667GROUP:AustNet
n24=Random SG serverSERVER:sg.austnet.org:6667GROUP:AustNet
n25=Random serverSERVER:irc.awesomechat.net:6661-6670,7001GROUP:AwesomeChat
n26=Random serverSERVER:irc.axenet.org:6660-6667,7000GROUP:Axenet
n27=Random serverSERVER:irc.eu.azzurra.org:6666-6669GROUP:Azzurra
n28=Random serverSERVER:irc.backwerds.net:6667,7000GROUP:BackwerdsIRC
n29=Random serverSERVER:irc.bdsm-net.com:6667GROUP:BDSMnet
...
But how would you do something like this otherwise using an ini file? It's an array of networks, and each network has an array of servers in it. Arrays don't fit naturally in ini files.
[NetworkName]
servers=foo.bar.net,baz.bar.net:+7000
port=6697 ; default if not specified
ssl=true ; default if not specified
encoding=utf-8
Split servers by ,.
If a network has different encodings per server.. well they should feel bad.
The biggest problem with ini is honestly how inconsistent implementations are. Python's configparser is 'ini-like', GLib's GKeyFile is a freedesktop ini-like format, etc.
We have settings that are specific to the individual server as well as general to the network.
You can have multiple servers in one network where some are ssl, some are ipv6, some aren't.
My example supports mixing ssl/ports/ips, but not encodings/descriptions which isn't valuable to me personally.
For what its worth I actually like JSON the best. It has high quality and consistent parsers for every language, supports lists natively, isn't large and complex like YAML, etc.
That's funny because the only reason I don't like json is because I believe it to be ugly and more awkward than ini files. But I guess we'll have to go with that.
Format aside if we want multiple clients to be on board sharing the default some policies would probably need to be established on what is included.
Standardized client specifications? It's like v3 specs for clients :D. Fwiw I agree with json, it's easy and doesn't have 10000 different implementations like ini and TOML I just weird in my opinion. It's like if ini had an enforced style consistency.
On Feb 19, 2017, at 5:05 PM, TingPing [email protected] wrote:
Format aside if we want multiple clients to be on board sharing the default some policies would probably need to be established on what is included.
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
JSON does not really have a style consistency regarding indenting/braces. What you see in that gist I linked to was originally one line (output from json.dump in Python), but then I ran it through http://jsbeautifier.org. The point is that it also should be easily diff-able, no?
I don't see why it needs to be diffable when we can just straight up parse it and/or write a python script to get the difference between two client json configs
On Feb 19, 2017, at 5:13 PM, wodim [email protected] wrote:
JSON does not really have a style consistency regarding indenting/braces. What you see in that gist I linked to was originally one line (output from json.dump in Python), but then I ran it through http://jsbeautifier.org. The point is that it also should be easily diff-able, no?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
Yeah, okay. In any case what I originally wanted is to have a format that Python can read so automatic checks can be done on whether the default servers are still up or not.
JSON would be your best bet then. Python natively supports it back to 2.6. From there parsing, resolving, checking is basically just a slam dunk. If we do this, we should probably include the script in the repo as well (just because).
On Feb 19, 2017, at 5:19 PM, wodim [email protected] wrote:
Yeah, okay. In any case what I originally wanted is to have a format that Python can read so automatic checks can be done on whether the default servers are still up or not.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
I already wrote the scripts (from kvc to json, and vice versa). But no checking. I could include a simple host resolve, idk.
You could simply just resolve the sub domain to an A record, or after you get the record, connect to 6667 and check for :hostname NOTICE * :stuff
Although note IRCu's implementation is different (something something NOTICE AUTH)
On Feb 19, 2017, at 5:24 PM, wodim [email protected] wrote:
I already wrote the scripts (from kvc to json, and vice versa). But no checking. I could include a simple host resolve, idk.
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
XML can be rather annoying to parse, while JSON can be very easily parsed into a Python dictionary without needing to install any extra libraries.
- It's hard to read and unparsable by anything other than KVIrc itself,
From what I see the popular opinion leans in favor of JSON so I do not have much to contribute in that discussion.
- It's safe to assume half the entries, if not more, are dead.
To check if the servers are online you could ping them but it is possible that some of them have it deactivated, so this method should be more reliable:
import socket
# Returns a Boolean depending on whether a connection could be established or not.
# If None is returned, it is because there is a problem not contemplated
def is_online(server, port):
status = None
irc = socket.socket(socket.AF_INET, socket.SOCK_STREAM) #defines the socket
try:
irc.connect((server, port))
status = True
except(socket.gaierror, socket.error): status = False
# close the socket no matter if it was able to connect to the server
finally: irc.close()
return status
I suppose that other IRC modules will have this better implemented but I preferred to use socket to not add additional dependencies. It may be necessary to add exceptions and extra checks, but as is it should be able to check servers with and without SSL.
