Some feedback and suggestions

You have suggestions or want to give feedback to the game or the forum? Post it here!
User avatar
Dr Gains
Posts: 85
Joined: Tue Dec 08, 2015 5:43 pm
Location: Canada

Re: Some feedback and suggestions

Post by Dr Gains » Tue May 03, 2016 10:21 pm

I redid some work for the ladder I was talking about if anyone is interested, includes an updated points system with 6 difficulties, rank titles, list of all maps with difficulties, and points possible to get. Theres also every record time I can find, links to videos, downloads, and name of authors that other users might want to see.
All in this google spreadsheet:
[+] Spoiler
https://docs.google.com/spreadsheets/d/ ... sp=sharing
I-dunno, tell me what you think about it. :|
Get involved in speedrunning Celaria and compete against friends and others at
https://www.speedrun.com/Celaria
Full Map List
https://docs.google.com/spreadsheets/d/ ... fUJIZ4WITU
Last Update: Tuesday June 20, 2017.

User avatar
Lewa
Site Admin
Posts: 163
Joined: Sun Nov 15, 2015 1:24 am

Re: Some feedback and suggestions

Post by Lewa » Wed May 04, 2016 5:25 pm

Dr Gains wrote:I redid some work for the ladder I was talking about if anyone is interested, includes an updated points system with 6 difficulties, rank titles, list of all maps with difficulties, and points possible to get. Theres also every record time I can find, links to videos, downloads, and name of authors that other users might want to see.
All in this google spreadsheet:
[+] Spoiler
https://docs.google.com/spreadsheets/d/ ... sp=sharing
I-dunno, tell me what you think about it. :|
Wow, that's... very detailed!
Seems like you put a lot of work into this.

The current system only tracks the achieved medals of each map per user. Some sort of ladderpoints (like you visualized in this table) would be an additional incentive to play and get better at the game. (I also like the titles which depend on the ladderpoints.)

The issue is that there are a few caveats which i have to sort out before implementing such a laddersystem. (Technically as well as in terms of balancing.)

First the technical problem:

It is easy to keep track of medals. The player achieves a certain time on a map, so this time/score is saved in the database. By looking at the map on which the player played on i can calculate which medal he/she achieved. As the server is going to have to process a lot of requests at once, minimizing the load for each request is crucial. One of the things i do is to store the calculated medal alongside with the time/score in the database. It's redundant (as i can calulate the medal by comparing the players time with the map-medal times in the database) but the problem is that doing this costs a lot of time (especially later when the database gets bloated.)

I can do a similar thing with ladderpoints which are calculated from the medals of each map (like platin which can get you 500 points) but it's difficult to add those additional points which depend on current rank of the map.
Why is that so? Well, everytime a player beats the time of another player, all the leaderboard entries shift by one place.
This means that i have to calculate the ladderpoints of each player which got "downgraded" by one place due to one player getting a new record.
That can be very heavy on performance. (As this can happen with each request to the server.)

I don't know how to solve this problem as of yet. I will search for a solution though.

Balancing
Well, there is only one thing:
How is the difficulty of a map determined? Does the uploader of the custom map set the difficulty? If so, how can we make sure that the difficulty is accurate?
(This can be abused: Maps with a difficulty rating of "very hard" are more valuable in terms of ladderpoints. Someone could be uploading very easy maps and set the difficulty rating to "very hard" and beat them in order to boost their own ladderpoint ranking.)

Another solution would be to allow players to rate the difficulty of the maps, but this is an even more complex task and also has the exact same performance problem as mentioned above (shifting difficulty > recalculating all ladderpoints of each user who played on this particular map.)

I have to iron out those issues before i can implement that into the current system.

User avatar
Dr Gains
Posts: 85
Joined: Tue Dec 08, 2015 5:43 pm
Location: Canada

Re: Some feedback and suggestions

Post by Dr Gains » Wed May 04, 2016 11:04 pm

Lewa wrote: Wow, that's... very detailed!
Seems like you put a lot of work into this.
Bored and trying to kill time after a night shift tbh.
Lewa wrote:First the technical problem:

It is easy to keep track of medals. The player achieves a certain time on a map, so this time/score is saved in the database. By looking at the map on which the player played on i can calculate which medal he/she achieved. As the server is going to have to process a lot of requests at once, minimizing the load for each request is crucial. One of the things i do is to store the calculated medal alongside with the time/score in the database. It's redundant (as i can calulate the medal by comparing the players time with the map-medal times in the database) but the problem is that doing this costs a lot of time (especially later when the database gets bloated.)

I can do a similar thing with ladderpoints which are calculated from the medals of each map (like platin which can get you 500 points) but it's difficult to add those additional points which depend on current rank of the map.
Why is that so? Well, everytime a player beats the time of another player, all the leaderboard entries shift by one place.
This means that i have to calculate the ladderpoints of each player which got "downgraded" by one place due to one player getting a new record.
That can be very heavy on performance. (As this can happen with each request to the server.)

I don't know how to solve this problem as of yet. I will search for a solution though.
Honestly dont think there will be much of a load on the server, since only the top 20(+?) players will be shuffled around per map and then have total points shifted accordingly. Going back to the tempus server I was talking about before, there are 12 or so servers running which relay back and forth to eachother that update points continuously with as much as 20+ players per server and 60+ players overall.
Lewa wrote:Balancing
Well, there is only one thing:
How is the difficulty of a map determined? Does the uploader of the custom map set the difficulty? If so, how can we make sure that the difficulty is accurate?
(This can be abused: Maps with a difficulty rating of "very hard" are more valuable in terms of ladderpoints. Someone could be uploading very easy maps and set the difficulty rating to "very hard" and beat them in order to boost their own ladderpoint ranking.)

Another solution would be to allow players to rate the difficulty of the maps, but this is an even more complex task and also has the exact same performance problem as mentioned above (shifting difficulty > recalculating all ladderpoints of each user who played on this particular map.)
Setting the difficulty of a map I think should be completely on your end. Also, only once the final version of a map is out. (So that you dont have 6 of the same map being used for points) The difficulty should be rated by either just you, or the community, and if no maps have a set difficulty than no points can be awarded.
Lewa wrote:I have to iron out those issues before i can implement that into the current system.
Fair enough buddy, this system is just what made TF2 Jump so addicting for me. :)
Get involved in speedrunning Celaria and compete against friends and others at
https://www.speedrun.com/Celaria
Full Map List
https://docs.google.com/spreadsheets/d/ ... fUJIZ4WITU
Last Update: Tuesday June 20, 2017.

User avatar
Lewa
Site Admin
Posts: 163
Joined: Sun Nov 15, 2015 1:24 am

Re: Some feedback and suggestions

Post by Lewa » Thu May 05, 2016 8:48 pm

Honestly dont think there will be much of a load on the server, since only the top 20(+?) players will be shuffled around per map and then have total points shifted accordingly. Going back to the tempus server I was talking about before, there are 12 or so servers running which relay back and forth to eachother that update points continuously with as much as 20+ players per server and 60+ players overall.
Well, it isn't quite that simple. Yes only the top 20 players need to be shuffled, but imagine that there are multiple players online and each of them beats the record on a map.

The leaderboard calculation also isn't that cheap:
For each beaten highscore i need to recalculate the ladderpoints of the top 20 players.
How does that happen?:

First we calculate the medal ladderpoints:

For each player (only talking about the ladderpoints for the medal here):
- find out on which map he played
- find out what type of difficulty the map has
- lookup the ladderpoints which the player would recieve
- add the ladderpoint value to a temporary variable which would represent the total ladderpoints
- repeat until all maps (on which the player played once in his lifetime) were processed.

And then we have to calculate the ladderpoints for the top 20 players (which is more complex as the actual ranking isn't directly stored in the database. Instead it is calculated on the fly due to redundancy.)


I'm currently looking into a system/ an algorithm which would do that, but it's a very complex task (due to the amount of data which needs to be processed for each player). I'm not sure if i can manage to pull that of. (Well, i can, but the question is if the server is going to be able to handle it.)
One way to save performance would be to do this calculation in longer intervals (like every 2 hours or so.) This would free up some performance in between for other tasks. But still, the calculation itself is still very heavy on performance so i need to find a way to optimize that as good as possible.

Ladderpoints for each medal are somewhat possible. But the top 20 players thing is a different story.

Also note that i don't have 12-20 servers which could do this task. Servers aren't cheap. (Sadly...) I'll only be able to afford a single server (and that isn't even going to be a dedicated server, instead it will be a shared one.). So i have to make sure that i don't run any performance-heavy scripts on there.


I'm researching this topic right now. So we will see...

Setting the difficulty of a map I think should be completely on your end. Also, only once the final version of a map is out. (So that you dont have 6 of the same map being used for points) The difficulty should be rated by either just you, or the community, and if no maps have a set difficulty than no points can be awarded.
I could of course set all the difficulty ratings by myself, but that's a time consuming task. (Checking every day all the new maps, playtesting them and then setting the rating costs a lot of time.) It would be better to use the sparetime i have to add new content/fix bugs in the game rather than going through all this data manually.

User avatar
Lewa
Site Admin
Posts: 163
Joined: Sun Nov 15, 2015 1:24 am

Re: Some feedback and suggestions

Post by Lewa » Sat May 07, 2016 3:21 pm

So i was thinking about that ladderpointsystem for the past few hours and created a few concepts as to how this could be implemented on the game-server.

So yes, i'll implement that. But i'll do some minor changes to your existing concept.
  • Ladderpoint calculation
    Due to the higher performancecost of calculating those points, i'll have to split the calculation up into intervals.
    The player who beat a new record will recieve the ladderpoints immediately, but additional calculations (like players loosing ladderpoints due to losing a rank on the leaderboard) will happen later (depending on the interval.) This might be every 30 minutes/every hour or every 2 hours. Depends a lot on the complexity of the calculation and on the server capabilitys. In the worst-case scenario it could take up to 24 hours (or even longer) to synchronize all ladderpoints.
  • Map-difficulty
    As i don't have the time to set the difficulty of each uploaded map manually, i could implement a voting system.
    Basically the original map-creator can set the difficulty rating by himself. But players (once they played on this specific map) can rate the difficulty afterwards (so they can say that this map was easy or way to hard). Depending on the votes of the players the difficulty rating can/will be adjusted (to avoid any kind of cheating of the system). This will also result in adjustments in the ladderpoints of all affected players.

    Also it would make more sense to reduce the difficulty rating to 3 categorys: easy/medium/hard.
    This makes it easier for players to categorize the maps. (Otherwise you need to define the term "hard" and "very hard". What is the ultimate difference?)

    So by splitting the ladderpoint synchronization up into intervals and reducing the difficulty to 3 categorys, this might be indeed possible.
If it turns out that the calculation of those ladderpoints causes performance problems on the server, i'll be forced to alter/optimize/simplify (or remove that system altogether). Of course, the latter is the worst-case scenario which i'll try to avoid as much as possible.

I'll start to implement this system into the current server today. We will see how it turns out.

User avatar
Dr Gains
Posts: 85
Joined: Tue Dec 08, 2015 5:43 pm
Location: Canada

Re: Some feedback and suggestions

Post by Dr Gains » Sat May 07, 2016 7:11 pm

Lewa wrote:So i was thinking about that ladderpointsystem for the past few hours and created a few concepts as to how this could be implemented on the game-server.

So yes, i'll implement that. But i'll do some minor changes to your existing concept.
Oh wicked! :)
Lewa wrote:Map-difficulty
As i don't have the time to set the difficulty of each uploaded map manually, i could implement a voting system.
Basically the original map-creator can set the difficulty rating by himself. But players (once they played on this specific map) can rate the difficulty afterwards (so they can say that this map was easy or way to hard). Depending on the votes of the players the difficulty rating can/will be adjusted (to avoid any kind of cheating of the system). This will also result in adjustments in the ladderpoints of all affected players.

Also it would make more sense to reduce the difficulty rating to 3 categorys: easy/medium/hard.
This makes it easier for players to categorize the maps. (Otherwise you need to define the term "hard" and "very hard". What is the ultimate difference?)

So by splitting the ladderpoint synchronization up into intervals and reducing the difficulty to 3 categorys, this might be indeed possible.
I think this is all a good idea and all, but what if new players just think everything is hard when it should be medium or even still easy? I think the difference between hard and very hard would still be completely skill based, something that you just dont see in a hard map. Like you would be able to beat a hard map now, but would really struggle or give up on a very hard map this early into the game. And since there isnt really anything I myself would call very hard, maybe it would be a good idea to go from Easy to Hard or even Very Easy to Hard, whether its temporary or permanent. But overall your voting idea is still a very good idea, as long as its handled right.
Lewa wrote:If it turns out that the calculation of those ladderpoints causes performance problems on the server, i'll be forced to alter/optimize/simplify (or remove that system altogether). Of course, the latter is the worst-case scenario which i'll try to avoid as much as possible.
Rats, it be worth a shot I suppose.
Lewa wrote:I'll start to implement this system into the current server today. We will see how it turns out.
Cool, if you would like a hand testing let me know :) Although, Id imagine you already have a handful of people for that.

Oh! Nice job on the leaderboard, just seen it on twitter!
Get involved in speedrunning Celaria and compete against friends and others at
https://www.speedrun.com/Celaria
Full Map List
https://docs.google.com/spreadsheets/d/ ... fUJIZ4WITU
Last Update: Tuesday June 20, 2017.

User avatar
Lewa
Site Admin
Posts: 163
Joined: Sun Nov 15, 2015 1:24 am

Re: Some feedback and suggestions

Post by Lewa » Sat May 07, 2016 11:31 pm

I think this is all a good idea and all, but what if new players just think everything is hard when it should be medium or even still easy? I think the difference between hard and very hard would still be completely skill based, something that you just dont see in a hard map. Like you would be able to beat a hard map now, but would really struggle or give up on a very hard map this early into the game. And since there isnt really anything I myself would call very hard, maybe it would be a good idea to go from Easy to Hard or even Very Easy to Hard, whether its temporary or permanent. But overall your voting idea is still a very good idea, as long as its handled right.
A possible solution to the first problem is to only allow users to vote who played a certain amount of maps. (Like >= 20 maps.)
This way i can make sure that players gain (at least some) experience in the game so that they don't vote randomly on the first maps due to being a complete novice. This voting concept can of course be rendered completely useless if people start to vote randomly or by selecting the wrong difficulty on purpose. But i hope that the majority of players do the right thing and vote with honesty. (Setting a minimum limit of played maps before being able to do so could help in that regard.)

And yeah, you're right, the whole easy-medium-hard difficulty is very generalised. But you have to rate the difficulty of a map in one way or another. Adding more difficulty options would make it only more complicated for players. (The would see 6 buttons for the difficulty rating and they have to think about if the map was now "hard" or "very hard" or "impossible hard" if such an option even exists.) This in turn discourages players from voting. (Would you like to vote while playing a game if you had like 6-8 different options to choose?)
The less options they have, the easier it is for the player. If you want to get feedback from someone without loosing his interest/his patience it is best practice to give him only a simple question with 2 options to select: "Yes" and "No". As a player, you don't have to think about all those other minor details. You only have two options. Like black and white. No different shades of grey in between.
That's why i wanted to reduce it to those 3 difficulty options as it is way more clearer and simpler for the player. (In terms of voting.)

How would players select those options?
  • Easy - a map which wasn't really a challenge for the player. (First try and already gold and silber. No complex shortcuts or jumps, etc...)
  • Medium - a map which wasn't that easy. (More complex maps or maps with jumps/paths which had to be repeated multiple times with a respawn in order to beat it.)
  • Hard - very hard maps. (Getting bronze/silver was a challenge. The player had to put a lot of effort into beating it.)
Rats, it be worth a shot I suppose.
Definitely! It adds a metagame ontop of the game (leaderboard hunting.) It certainly wouldn't be a worthless. :)
Cool, if you would like a hand testing let me know :) Although, Id imagine you already have a handful of people for that.
Currently i don't have anyone to test all those new features. I only did "closed alpha" test maybe 2 max 3 times. (Multiplayer server.)
The current leaderboard/mapupload system works, but it isn't error-prone. (If an error happens like the connection to the server breaks up, you will have to restart the game completely in order to get it to work, etc...) Basically: it works, but it isn't ready for public testing.

But i will do an beta test in the near future which i'll use to (stress-)test all of those things. (probably in the next weeks. Depends on my sparetime.)
Oh! Nice job on the leaderboard, just seen it on twitter!
Thanks! :)

/Edit:
I also created a simple google-sheet for the current ladderpoint system (which i'll update as time goes on.).
Still WIP (a lot of stuff is missing) and i'm also not sure if the medals are really balanced against each other. I'm open for suggestions. :)
https://docs.google.com/spreadsheets/d/ ... VchoHOe4Y/

(Basically, i need to scale your current system down from 6 difficulty levels to 3.)

User avatar
Dr Gains
Posts: 85
Joined: Tue Dec 08, 2015 5:43 pm
Location: Canada

Re: Some feedback and suggestions

Post by Dr Gains » Sun May 08, 2016 8:17 pm

Looked at your sheet, seems like a pretty heavy range going from no medal to platinum. I guess that'd be alright once all of the times on your maps have been balanced, but now you will have to look at other players maps and see how the times differ. Since before you were saying how you dont want to set difficulty because of the time it requires, cant see how you would step into times for every custom map. Id even go as far as saying that the platinum times on half of the maps arnt even close to what they should be.

I also thought of an idea about how difficulty can be chosen. Perhaps you can have like a vote weight? Like better players (even higher ranked) have a stronger vote than new players. I dont know if Id do this by points, but maybe refer to what I had under titles where rank 1 would get say 100 vote points on a map, rank 2 would get 95... rank 51~75 would get 60... 501~700 would get 30... etc. (Think Ill add this to my sheet later just for a reference later..) Id still keep the idea of having a minimum requirement before being alowed to vote, too.

Im still convinced on Very Easy to Hard, just because of the first few maps.. especially Beginning and Passage. Might also give people a better idea on what they can complete with a larger range. But this is all your creation and you can probably see the big picture better than I, haha!

I also added a small part to my sheet last night of what the leaderboard would look like in theory: Page 3 (Titles) Row 41.
Get involved in speedrunning Celaria and compete against friends and others at
https://www.speedrun.com/Celaria
Full Map List
https://docs.google.com/spreadsheets/d/ ... fUJIZ4WITU
Last Update: Tuesday June 20, 2017.

User avatar
Lewa
Site Admin
Posts: 163
Joined: Sun Nov 15, 2015 1:24 am

Re: Some feedback and suggestions

Post by Lewa » Mon May 09, 2016 12:48 am

Looked at your sheet, seems like a pretty heavy range going from no medal to platinum.
Yeah, i'll have to adjust those values.
I guess that'd be alright once all of the times on your maps have been balanced, but now you will have to look at other players maps and see how the times differ. Since before you were saying how you dont want to set difficulty because of the time it requires, cant see how you would step into times for every custom map. Id even go as far as saying that the platinum times on half of the maps arnt even close to what they should be.
Difficulty of a map shouldn't be determined by the platin medal time. It may be a factor which adds to the overall difficulty (the longer the map, the higher chance to screw up a run) but it isn't the right way to determine that.
As an example, you could have a very simple map (like only running in a straight line) for 5 minutes. Or you could have a 10-20 second long map with very hard jumps/routes.

If i myself had to determine the difficulty, it would require me to play every single map which was uploaded by the user and then rate it. It's very time-consuming and not feasible due to the fact that i simply wouldn't have the time to do so.

Also, what exactly would your preffered platin times be?
I also thought of an idea about how difficulty can be chosen. Perhaps you can have like a vote weight? Like better players (even higher ranked) have a stronger vote than new players. I dont know if Id do this by points, but maybe refer to what I had under titles where rank 1 would get say 100 vote points on a map, rank 2 would get 95... rank 51~75 would get 60... 501~700 would get 30... etc. (Think Ill add this to my sheet later just for a reference later..) Id still keep the idea of having a minimum requirement before being alowed to vote, too.
That's definitely a possibility to enhance the rating system. I'll look into it once the basic votingsystem is done.
Im still convinced on Very Easy to Hard, just because of the first few maps.. especially Beginning and Passage. Might also give people a better idea on what they can complete with a larger range. But this is all your creation and you can probably see the big picture better than I, haha!
I'm open minded about that. You are right that categorizing it into more detailed difficultys is better for the overall scoring system (and fairer.) But on the other hand it is more confusing for players in terms of votes. (As the barriers between each difficulty is more "blurred".)
How exactly would a player know if a map was "medium" or "hard"? Or "hard" and "very hard"?

This in turn could lead to wrong votes (players who voted "very hard" on a map which wasn't really that hard, etc...). And this "false" votes could also affect the difficulty of the maps.

But as i said, i'm open minded about that. (And your points are totally valid.) My tendency to the 3-difficutly rating instead of the 6 comes from previous experience in "ratings" from users. (Basically, if you want to get information out of the player, simplicity always wins.) Otherwise i would choose the 6-difficulty rating as well.

There is of course the possibility to simplify the voting system for the player, but still have the 6-difficulty rating behind the scenes. (Although i have to think about how this would work. It is possible though.)

I'll do a bit more research on that topic. If i manage to create a simple voting system (which doesn't create wrong results) while still keeping the more detailed difficulty levels, i'll opt for that option as it combines the best of both worlds. (Heck, i think i might do that this way.)

I also added a small part to my sheet last night of what the leaderboard would look like in theory: Page 3 (Titles) Row 41.
I'll look that up. :)

I also made progress on the ladderpoint system. I updated the database to store all those values (with caching to increase performance) and ladderpoints are already added to the player once he gets a new record on a map. There is still a lot of work to do, but the fundamentals are already there.

User avatar
Dr Gains
Posts: 85
Joined: Tue Dec 08, 2015 5:43 pm
Location: Canada

Re: Some feedback and suggestions

Post by Dr Gains » Mon May 09, 2016 3:34 am

Lewa wrote:Difficulty of a map shouldn't be determined by the platin medal time. It may be a factor which adds to the overall difficulty (the longer the map, the higher chance to screw up a run) but it isn't the right way to determine that.
As an example, you could have a very simple map (like only running in a straight line) for 5 minutes. Or you could have a 10-20 second long map with very hard jumps/routes.

If i myself had to determine the difficulty, it would require me to play every single map which was uploaded by the user and then rate it. It's very time-consuming and not feasible due to the fact that i simply wouldn't have the time to do so.

Also, what exactly would your preffered platin times be?
Oh I wasnt relating difficulty to platinum. It should be 100% about skill required to even complete the map. And I wasnt suggesting you run through other custom maps. What I am saying is there are maps now, and more to come where the platinum times too high, and could use a way to be adjusted. Heres some of the custom maps for an example:
[+] Spoiler
Name | Platinum Time | Personal Record
Freecity | 1:00;46 | 0:38;88
TowerOfChoices | 57:14;73 | 3:09;55
Trails_1 | 1:40;50 | 0:55;78
Trails_2 | 1:11;93 | 0:46;01
TwoWays | 0:34;58 | 0:18;18
All of your maps have a spot on time, have very little room for error. But what Im getting at, is people will keep getting platinum times and thats it. There isnt that struggle to get a better and better time until you reach platinum.
Lewa wrote:I'll do a bit more research on that topic. If i manage to create a simple voting system (which doesn't create wrong results) while still keeping the more detailed difficulty levels, i'll opt for that option as it combines the best of both worlds. (Heck, i think i might do that this way.)
Right on dude. :)

Oh! I was going to mention this earlier but completely forgot. The folders you have in the roaming folder for celaria. You should make them 1 permanent folder in the future so that people wont have to redo all there times. But this might already be kind of resolved with an account system that you may have already rigged up.
Get involved in speedrunning Celaria and compete against friends and others at
https://www.speedrun.com/Celaria
Full Map List
https://docs.google.com/spreadsheets/d/ ... fUJIZ4WITU
Last Update: Tuesday June 20, 2017.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest