Catacomb-Snatch
Catacomb-Snatch copied to clipboard
Maescool
Master server ?
So, I thought it would be awesome if actually everything would run on some kind of master server which manages the games and acts as a lobby for everyone who wants to play etc. Sadly i haven't really got in experience in Networked Programming, but what do you think about the idea?
i already programmed something like a central point, but since nobody answered in the forums i stopped working on it. Sadly the forum is currently closed but you can access my work here: http://spaceemotion.zapto.org/ci/ The plans were a central location for map and server listing (for maps also downloading) which data can be also accessed in-game.
That looks really neat but I thought about some kind of "server-browser" ingame with pair-matching (like Portal 2 or so). I think it would be really hard to implement, so it's just a thought.
well, my system is API friendly so this wouldn't be a problem (server-side). But I also think that this whole project kinda went down, so I think implementing sth. like this would be more effort than success.
Although, it would potentially encourage more people to play it. Making it super simple to just hop into an online game would be much funner than only being able to play with someone on your network.
If the wish is there i can code it right into my system, nothing big to do :D
Edit: According to my site stats there are still 1k visitors each week. Maybe it won't result in such a fail than i thought :)
Yeah. It would be nice to start with a simple map-downloading-feature from http://spaceemotion.zapto.org/ci/ and then implementing some kind of master-server-concept
i will be not at home next week so the site will be down (its my localhost btw) but when i will get back i can start working on the API (list of maps, downloads, etc.)
Is it possible for you to put the code on GitHub so we can take a look at it etc.?
that would be possible indeed :D I am currently storing it on a subversion server (also local), i can copy it over to github if you want.
okay its up:https://github.com/spaceemotion/CatacombSnatch-Repo please mind that the database could not be uploaded (of course)
for the api: i thought using the REST structure, what do you think about?
oh and if you want you could also help me out solving this problem (i wanted to create a map preview using the image lib and the raw xml data...): http://stackoverflow.com/questions/10613996/decompressing-tiled-tmx-file-contents-with-php
About the master server, i found something that could help http://smartfoxserver.com/ check it out and post what you think about it..
so this smartfox server is basically just an extension for the server but still can get all the data from the website? or does it have its own way of storing information? anyways, i think this is a great tool for both, server-side and client-side. It just takes some time to recode the parts in the game i think.
I'm currently researching and trying to add to the code. here is some good reading material http://www.smartfoxserver.com/downloads/sfs2x/documents/SFS2X_FPS_Walkthrough.pdf i also recommend http://docs2x.smartfoxserver.com/
this can do everything we need for a lobby master server system. smartfox community (free) is licensed for 100 users to play at once.
I took a quick look, @Maescool but not sure what you're looking for. let me know!
i also took a look at it and installed the server on my own. the thing that i am worried about is the database. how should i access it when i cannot find it? where does SFS saves its data?
btw we are onl getting about 10-20 people on the website atm. i think its because the forum has been closed :(
I did some initial writing and including the client API @spaceemotion I upgraded the forum yesterday, asked @Austin01 to fix the theme (got borked with upgrade) He did this, i'll see if i can set the forum live again.
To test this, you need to locally install smartfoxserver (for now) i guess this is not a problem for you guys.. just drop it in /opt/ (make sure you also upgrade after the initial install, which is just untarring the file xD) The documentation is great, but we need a stratagie how to implement, cause we will also have to write extensions for the master..
i had my birthday today so lot of family and friends were around but i can make some simple sketches tomorrow. for the extensions: which ones do we need? for the custom protocoll data?
First of all @spaceemotion Happy bday! :D
I notice you guys saw the new branch i added.
There is a lot to be done.. a short list, yes i could go more in detail, but i guess this needs to be done in a wikipage or something (easier to update) or i can make new issues with sfs milestone..
- some kind of login system.. (registering can be done on the site)
- design for the lobby screen, chat-screens needs to be in there, you need to be able to find your friendlist, etc..
- if you have read the sfs docs you know there is a gameroom per game, and that also supports in game chat ;) and all the gamecalls that happen, the gamecalls are passed to the extensions to handle, so the gameroom simulates the game, so it can confirm the calls (work against cheaters) Also that way we can keep ranking between the gamers, also will be easier to add more players, i guess we can go up to 4.
this is a lot of work to be done.. cause the gamecalls need to be handled by an extension on the serverside.. but i have no clue how to start on this..
Yes we are starting this back up! :D
To offload the smartfox work a bit, it can make webservice calls to verify login, handle a part of the friends, handle the ranking system.. so the game also can call that service. Note: we need to be very carefully about the security in this..
the server side stuff will be easy (the repo i mean) but we need sth like and api auth key and session timeouts. ive never done sth like this so far...
since the api will be rest based i found a way to authorize: http encrypted aurhentication (http://en.m.wikipedia.org/wiki/Basic_access_authentication) the username will be encrypted thought and the pass will be a sh1 salted encryption.
Btw, there is also an alternative server: http://www.electrotank.com/es5.html but i'm not sure which to take.. smartfox or es5
I'll see what you guys think before we commit to for smartfox..
I can see a huge advantage in having the master server with user registration and game host lobby and I'll kind of list them out a little bit.
- With user name we can display user name plates above characters heads when in game.
- This will allow us to create a simple achievement system with a viewable achievement list on the main website.
- For the way off into the distance future, we can possibly see implementation of customer character sprites.
- people can easily distribute and play custom maps
- This will pique interest in the game as it has waned since the humble bundle days
- implementation of lobby chat, options menu, ect will finally be able to be realized.
With all of this, this allows us to bring in a new wave of viewers and possible programmers to the project. To think this project has gone this far on just volunteer work is amazing. We need to keep this going.
before we dive too much into the SFS stuff: @Verexa made a good point when we talked about the master server.
About SFS:
- API-friendly
- multi-language
- no access to Db (we hadnt found one)
- no dynamic or flexible way to handle custom stuff (except the extensions)
- no master-server API (huge minus for us)
So we thought about using StreamSocketServers in PHP (example: http://www.php.net/manual/en/function.stream-socket-server.php - see comment 1 for a ssl login)
- DB-friendly
- master-server is flexible
- repo-integration made easy
So far we have not found any negative points for the PHP solution.
What do you guys think?
For the stuff that @godleydemon said: Achievements can be easily stored in the db then, with readable output on the user pages custom maps can be downloaded via PHP very easily (via the stream or a URL)
I'd prefer the PHP solution, although it would probably be alot more work, or are there any libraries supporting the development of a Masterserver? Additionally, I could maybe help a bit with the programming because I've already got some experience in PHP (might need to refresh it a bit).
well there is some kind of library already in PHP: http://php.net/manual/de/book.stream.php @Verexa is currently trying it out, but has some issues with multi-threading the streams
The "friends" function works on my local copy of the repo already, ill upload the changes later.
It is possible to create this solution using PHP. The only few problems that I've currently come across is that multiple connections will need a bit of work. I know that if we had a REST like API in place for the client to use we could do it very easily. But I think that dealing with a lot of information and requests we're better to use streams. As it would mean only authenticating once.
@Verexa started an awsome test code for read/write sockets in PHP. Because I had trouble to re-use ports once the script has been stopped I changes some things.
The work can be seen here: https://github.com/spaceemotion/CSMS (Lots of credit to @Verexa :) )
The best thing is: I opened the game just for fun to see if the game can connect to the script - and it does! The server currently does not recieve the right packages and does not send sth back, but the game recognizes the PHP "server" as a legit game-server! Currently its only working on localhosts, but we are sure that this can be fixed very soon. If this thing really works we made a huge step forward!
holy crap dude, did we literally just jump foreward in time!? I thought it would have taken a month to get this far at least, lol. Nice going Verexa and SpaceEmotion! :D
I think some of the first few things that should be worked on though after figuring out how to make the client handshake the server. We need to design the login screen and or launcher screen. We can start with a simple user name to begin with. But this will allow us to begin programming the db to accept incoming connections based on user names. This will also let us experiment a little with the handshaking process. I've been playing around with some images last night and a little today. I'm designing a few screens for MP. Just so we can kind of get an idea what we're going for here. We can also change little things here and there to adjust it. So that way everyone will be happy with the final outlook. Alot of things won't be able to be programmed for MP until the master server and client hand shake is figured out through the php. Because I'm deffinantly seeing some benefits of using the php.
so far we made the server OOP based. we have classes for commands and packets just like in java - for better handling. we already can read and write packets using this new system (still locally). new stuff can be added very easily now :)
hat we definitly need is a better packagestructure. for example sth like the handshake paket, then a clientinformation paket (game version,...) and then login pakets.
we now have devided the server in: master and game server. the game server handles the game. the master handles the creation of a new game, authentication, map listing and so on. the way of communicating can be done via 1) pakets or 2) commands. the pakets are only being sent by the client and server, commands are morr like admin stuff vis the console to halt/debug the server. thats why we devided the server into server and client parts: client folder holds anything coient related, the server folder anything server related (also db stuff)
The reason I propose Smartfox is that it can handle massive amounts of connections and handle more than a php-server could. The smartfox doesn't need direct contact to the database, but can do it over json calls to a php webservice script, which can handle the database part etc..
The main reason i stand behind this server is that it can first of all, do decent TCP and UDP connections, provides an client-server API, and has proven to handle a lot of gamers at the same time.. which i'm sure php read write sockets can handle that well.. and it saves a lot of development time :)
Really no offense to you php guys.. but like i said, there will be still a lot of php work anyway, so we need to write out first what the dataflow will be, i'll try to write something up this week how i have it in mind and believe can scale.
I think what Maes purposing is a good way to look at. Making the json calls to a php webservice script I think might be the best route to take. I was actually talking about the server with a friend last night and was expressing the same concerns of the php being able to handle a massive intake of connection at once. The conclusion... theres going to be a lot of dropped packets and or commands not reaching the server. But if we use smart fox and program a json call to php, we can deffinantly see an improvement in connection and less packets dropped.
I think what one of the things we should kind of start on though is a .exe launching script thats akin to minecraft. The actual jar is stored in %appdata% you run the game through the .exe which handles the hand shaking to the master server for authentication. Once authenticated, the .exe launches the jar with the appropriated information from the master db information on user.
I can see you have expressed concerns an made very valid points. The only issue I can see is how fast it deals witch connections. There is only a limitation of sockets available as there is to your computer. I believe that we wouldn't have issues with lost packets but on the rare occasion (usually client fault though) we could deal with it well. I can't exactly test a system that isn't finished, but I see many other advantages to using PHP.
With Smartfox server I can't seem to find API documentatio for the server side and believe there might be some licensing issues. As far as it goes I believe we can do what we're doing in PHP but in Java and have it a multi-threaded server to increase performance. I'm just thinking that we can produce a more custom masterserver to what we need.
The problems I saw with using SFS are very important ones: Database integration (like I said, I don't know how i can get the data of open rooms from it without having a client that sends me the data to a php script). the second is the user numbers, the free edition is very limited in my eyes, so SFS could handle this stuff but then stops because the server is full. If only 100 players can be connected that would make 4 waiting in the lobby and 96 others are playing (that makes 48 games). Maybe you are right when you say that the server my not be able to handle all the sockets, but what we currently are making is only the master server (for getting rooms, maps and stuff). the game server itself is another thing that maybe is not even based on php.
hmmm, you have a major point there SpaceEmotion. My thought though is this, what about instead of having everything hosted in a master server. Let the client handle the server side calculations but make periodic calls to the master server on event hooks. Basically the only thing a master server would do is keep track of live games, players and so on. It will also record user data in the database for achievements (like how many snakes have I killed) This way we minimize the amount of stress on the server and instead divide it onto the client computer.
That's a good idea too, but then we'd have to deal with the ports the player who is hosting the game would need to open, won't we?
Yeah me and @spaceemotion had decided to make the masterserver a REST API. We're currently working on it in PHP just now. https://github.com/spaceemotion/SnatchAPI
We think that it should be down to a dedicated server or the client with the server classes to host a game lobby. Then report back to the master server. This is what I thought you guys wanted originally but was lead astray thinking it was meant to be something else haha.
Yeah we'd have to deal with that so... We thought 3031 for the game port, and a HTTP 80 request for the master server aka API?
3031 is pretty easy to remember and its obscure enough nothing is going to interfere with it. -does a google search for port 3031 then face desks- apparently that port is used on Remote AppleEvents/PPC Toolbox. I'm not sure how obscure or how used this is >.0. Let me do a little more research on it just to confirm.
ok this is what I've dug up on port 3031 http://www.speedguide.net/port.php?port=3031
Then let us do port 3040: http://www.speedguide.net/port.php?port=3040 Already used by other games, so it would fit in there :D (and I dont want to let the game be recognized as a virus)
By the way: I just wanted to say that the API is almost finished (server side). The only things left are filling it up with commands and implementing an authorization library (like OAuth). The next thing would be the Java code then I think :D
sounds awesome Space. I can't wait to see a little dev build of this to test her out, any idea on where we are going to run the master server at for the server lists?
umm, i dont think that you got it quite right. there is no real server for only the lists. there are only two sites to handle the requests (not the games): the repo and the api. the repository handles the data while the api handles the requests from the game (login, getting maps, servers and so on). The repo is only a website utility to display and edit stuff. It also is the home of the users - to play online you have to register in order to get a UUID. So this whole thing is running on the homepage server while the games are handled somewhere else.
And for the repository: may be sound a bit hard - but i am thinking about a rewrite. the way its now is too complex and not flexible. i am also seeing code igniter as a not-fast framework for this kind of website so it may be a custom site...
Another alternative would be writing the repo with Rails in Ruby. I don't know if you know ruby, but that would be quite cool too. There's a plugin (gem) which's only purpose is to serve APIs. Here's a good tutorial for it: http://railscasts.com/episodes/348-the-rails-api-gem Maybe you could make a small list of requirements of the API and a can start to work on it in rails. Help is really appreciated if anyone knows rails/ruby. :D But you can use PHP too, this is just an idea.
I am coding in PHP/JS/CSS and HTML for 5-7 years now. Using Ruby would be a bit of a problem because I would have to learn it first, however I know that ruby is preferred when using APIs. If the PHP system fails (too much traffic, other problems) we can still think about it. For the repo i would like to code a system that is specialized in creating "game-based" communities and managements.
Just wanted to say that the framework is nearly done :) Had some free hours today, so I started working on a lightweight framework with nearly no footprint, but the efficiency of our SnatchAPI and the flexibility of CodeIgniter in mind (I only have 9 files so far - and it still works ^^).
I will commit the first version when a single website output can be generated :)
The framework is only a base though. Once its running stable and got some plugins and helpers I will rewrite the site (you can help too!). Like I said, the reason for this is the massive weight of CodeIgniter - it runs benchmarking and some other (for me) useless stuff that I am not using anyways which takes some more time to load the site.
edit: basic output works - only models are left
edit 2: I finally committed the first parts of the framework: https://github.com/spaceemotion/CodeFire. Please mind that this is only the framework - not the repository code!
congrats Space! This is looking really good. I'm loving it so far, and I have to say, your coding as always is very remarkable and well organized. Can't wait to see this in the actual game and I see what you mean about how everything is going to be handled. Its actually pretty smart really, can't wait for this to get to a point were we can begin writing the code to display the lobby framework :D
Had some other things to do the last weeks (a clanserver asked me to help out setting up their gameserver homepage) which took me quite some time, also the achool stuff sucked up my time. Back on track i will have some time to work on the framework (and the resulting repo). i still have to think about some parts and how i will make them, but since the framework system is completely placed in just one system i can just replace that easily on backend updates in the future ^^ I am currently thinking about the overall load of the framework (since it should be lightweight) and think a blank site taking 2ms not too bad but also not that fast, what do you guys think?
it really is looking pretty good Space. Can't wait for this to start getting thrown into the main program. XD
Okay uploaded my changes, its now finally working ^^. I will do some performance related coding later (default page generated in 0.002328 seconds :))
woo hoo! sounds awesome! lol. I sent you a friend request on gtalk Space. I'm working on some java and could really use some direction XD
I think I can finally say that the framework is done. It has all what i need to start a fresh new page :) Only things that can be added as extra are Active Records (Wikipedia: http://en.wikipedia.org/wiki/Active_record_pattern) and some more basic helper functions.
I will start with a fresh new repository including the main website since I want to be it a part of the community system (login, logout).
I know it's quite late to ask. From what I saw you developed a standard MVC framework... Why again did you have to reinvent the wheel when there are tons of (tested!) other frameworks in any language you like? From an engineering point of view this is not a good decision if you don't have special needs that can't be fulfilled by an existing software.Yours is less tested, less documented an has potentially more security holes. No offence, but I just don't get it
First there was the challenge of course, the second thing is the overload.
For the existing system: I will take two big systems I know, CakePHP and CodeIgniter, as an example. CakePHP is well known for its flexibility, but the first thing that annoys me are the namespaces. I am used to write code with functions and classes and not "namespace" or "use" statements. Of course you could get into, but it also has a (for me) negative point that it shares with CodeIgniter: its too overpowered. Like i said in my comments above I wrote the currently running site with it but I had some issues with it. I looked at the code and saw tons of stuff that i would never need but potentially slow down the site (benchmarking class, log file class, pre and post hooks, ...) which annoyed me.
I am a guy that hate sites that are loading slow, if a normal page script takes more than 2 seconds to load its pretty bad coded in my opinion.
These to points were the reason why i started my own framework from scratch. The current site load is very low (< 0.002s of a cached site) and has one more thing that i missed: modules. I implemented them because i missed a way to place content that i often need on any page that i want (like a CMS can do). I saw no framework which had such support.
For the documentation: it has an example with all the stuff that it needs, i will write a short documentation if i have some more time. basically it is working like any other framework - i added just a few more things ;)
Well I think designing our own framework actually isn't a horrid idea. I mean it fits right along with what we've been doing this entire time with catacomb snatch. I mean hell, Notch kind of recreated the wheel with multiplayer functions. Of course they didn't work to extremely well (with the tic problems and all) But he did it for the challenge to see what he could come up with. In a way, that's what we're all doing here. Challenging ourselves to create something extraordinary. I mean look at it like this. Where was catacomb snatch a little under a year ago? Barely finished, missing sprites, a barely there MP system, bug every where as far as the eye could see. Now look at her. It is looking like the game it should have been. We have two more playable characters that were never even planned to be in the game, We have fixed hundreds of bugs, New weapons and animations, New tiles and a map editor! Well you get my point. All we are is a bunch of misfits around the world getting together to work on something we saw worth in that not a whole lot of people decided to stick with. I think we're doing something special and if we're going to do something special, might as well go all the way and create our own MVC framework for the lawls ^.-
I made a small post about a new Website (-Design): https://github.com/Maescool/Catacomb-Snatch/issues/981
I think the integration of user name and registration to the new framework for the server to handle in game names is a pretty much must. This would allow people to register on the website, then use that user name and password in our client so they can play multiplayer games. This will also allow us in the future to implement the much requested after achievement system that everyone and there mother at the beginning was talking about, lol. Will also allow us to create user profiles and possibly in the future custom player skins for MP. I think we can do alot by linking the game to a server on launch with user name and password. Also as a side note I happened to notice this on the issue list https://github.com/Maescool/Catacomb-Snatch/issues/887 its basically end game stats, this would be really neat to have and doesn't sound like it would be to awfully hard to have. In the future we can also have these stats sent to user profiles as a sort of "bragging" rights. There is alot we can do with the game being linked with a server and I think we can really open up the user base on who plays this game and why they should want to play it.
Yup that registering stuff nails pretty much the wait I first thought about when creating the "repo": A global user profile to log into - once logged in it downloads a temporarly cached user file to get the data (so more than one user can use the same pc with the same game) - achievements and friends ("buddies") can be added and are marked in-game with some kind of noticable tag (a star for example). The gameover stat thing can be also pretty handy: to get the top 100 players and see how many times you played.
However I would like to be able to play offline; Either I do not want to register (other than the system Minecraft uses) or I am not connected. Either way the game still collects data about the user - if he decides to register some days later all his saved games wont be lost then.
Hey Space how are we coming on the new website design and layout? Would love to see this implemented soon XD
Already working on it :) I had some class tests to do for school, but I created a new repository for the whole new website - it will be a "All-in-One" website (means: repository, news and login are on the same website - without repo.* or blog.*)
Link to my local version: http://spaceemotion.zapto.org/git/Catacomb-Snatch.net/ Link to the github repo: http://github.com/spaceemotion/Catacomb-Snatch.net
I have to say I'm impressed XD let me know if you need any graphics for it. I'm sure I could whip something up in the next couple of days for you
Is this system going to be self installing, IE would I be able to install this onto a webserver and play around with it? lol
So how is everything going? Is that stuff still in work? When will there be any implementation of the "master server" / Framework stuff?
Well it looks like according to Space Emotions Github, he hasn't updated the repo in 2 months. Thing kind of died away while we were waiting for it to be done unfortunantly >.o
The last few months were very hard for me - I am in my last year of school (this month are the final exams), but I worked on the project on my own, without uploading commits to github. Same thing with @Verexa - we skyped a few times. We already decided that we will start working on the project when school stuff is done.
Unfortunately I haven't had a lot of time to work on it and being unable to access, what I had started of, the server. I'll be trying to start on it again and hopefully something that we can integrate and work on.
The server is in development, as well as the website. For the game, we already started on the Ouya / Android port. Check it out here: https://github.com/Catacomb-Snatch/Catacomb-Snatch
Thought I'd throw up a message here and ask how its all going Space. Would love to hear a quick update on how everythings going on.
I've been coding as hell, for my own Minecraft-Server. Turns out to be two weeks now. Man, that's a lot. I had holidays and said to myself "hey, time for CS", nope - when I wanted to work on the project I was sick :( Anyway, it's school time again but I think I can still manage to finally add some more INPUT related stuff to the game. The problem: You should be able to use keyboard, mouse and joystick alltogether or seperately (as a 2-player split screen mode). thats the problem I am facing atm. @Maescool linked me to an interesting article, i will see what I can do.
For the website: it will most likely use the same system as my own minecraft server website (i will just add and remove small things, also change the template), since it's using the same basic layout (community, news system,...).
All in all, I am still alive :D