I’ve mentioned many times on this blog that I’m a huge fan of the idea of gathering usage statistics, and I do my best to display them in a way that makes sense for others.
I recently read an article on Coding Horror that reinforced my belief that gathering actual "real-world" usage statistics is invaluable. To attempt to boil it down to what’s relevant for this blog, Jeff Atwood describes important lessons learned by Valve’s gathering of hardware and gameplay metrics to discover things about their users and their game that likely could not have been gathered or understood in any other way.
Take, for instance, this (unattributed) quote from Valve about the use of Steam to collect information about Team Fortress II:
We’ve traditionally relied on things like written feedback from players to help decide which improvements to focus on. More recently, Steam has allowed us to collect more information than was previously possible. TF2 includes a reporting mechanism which tells us details about how people are playing the game. We’re sharing the data we collect because we think people will find it interesting, and because we expect to spot emergent problems earlier, and ultimately build better products and experiences as a result.
While less relevant to my point here, even last September GameProducer.net had an article on Steam’s statistics and how some surprises and quite useful information could be found there.
While Second Life® doesn’t really provide any clean way to gather those kinds of statistics, and C:SI doesn’t really require quite that level of detail, I still believe that gathering some data is not only good for the system as a whole but could be very interesting for the community members.
I’ve already discovered one flaw in the data that I’m currently gathering : It doesn’t take into account weapons that don’t get stunned, like the ninjaken. It assumes that a blocked slash results in a stun.
I’ll just put that on the list of things I need to fix / want to enhance
I would like to mention as someone who has avidly gamed online for the last nine years. That statistics both individual and group based add a whole other level of integrative competitiveness to a game.
The more stats you can gather I think the better c:si system will be.
I don’t know how this could be done, but I think it would be really helpful to see data that shows what patterns each person is falling into and what patterns each person is falling for, maybe by registering any combination of button pushes that happen more than three times in a spar and highlighting the other person’s reaction to it. I think I’m asking for too much. lol. But I think it would be a great teaching tool.
It might be possible to do that some day, but I’m not sure I know enough about pattern matching and pattern recognition to do that in LSL at this point.
Humans are by nature exceptionally good at it, though, so maybe it would be sufficient in the short term to simply display the keys that were pressed and let those who wish to review them and draw their own conclusions?
But then, we get into an issue with the amount of data that needs to be transferred to the web site for storage, and llHTTPRequest has a built-in limit of 2048 bytes per message, and a throttle on how many messages can be sent.
I’m already doing some tricky things to try to ensure that all of the duel’s data arrives successfully, but this is a pretty interesting idea with possible real benefit to developers as well, so I’ll add it to the list of 50k*
Thinking more on the issue, I think Malachi’s idea is pretty important, so I will add it to the next version. I have had misunderstandings before about what sequence of keystrokes was used to duplicate an issue, and being able to record this might help a great deal when I’m trying to understand something a user is trying to describe.
Yayyy. ^^
Well, I figured out a way to “compress” the data in order to be able to add the keys, but so far I’m only able to track the keys in use at the time of each attack. I will work on expanding that, but at least we can track the sequence of keys at those points.
Hmmm… Actually, just knowing which keys are in use at the point of an attack will allow me to differentiate between a left slash, right slash, etc. I didn’t have any way to do that before.
/me wanders off in deep thought….
I think that’ll be really helpful with the BD and any future weapons that make use of unique button combinations for certain combos. I found a second 4-part combo that archanox put into the BD: 3w then 4a (I’m finding it’s more useful than the other 4-part combo, especially following 2s.)
The way it’s currently setup, say someone uses 2d on a BD or 3a/3d/(sometimes 3w) on a katana, does it only register the two slashes, not the single key that triggered them?
I have in mind a fairly simple way to detect commonly used patterns. If you have the sequence of actions the player performed, it’s just a matter of counting n-grams and seeing which ones occur most often (I guess what’s really important is how much more they occur).
I just had a crazy idea about taking sequences from different fights and comparing them using pair-wise sequence alignment to determine how similar the fighting style is. It might be able to detect alts, or maybe even detect that one of the fighters was taught by another fighter.
I had a thought at one point about using suffix-trees (see the “longest common substring” and “longest repeated substring” sections) to do something similar.
(thinking out loud): I suppose, now that I think on it, that if I just collect the raw key information and pass it along to the web server, the computations necessary for this could be done on the server side.. That would still allow the fight reporter to be low-lag and not increase the complexity. Hmm..
haha that would be awesome.
I don’t know if that is possible given some of the limitations we have, but it surely would be cool.
We need to get the core functionality in place first, of course, but that’s definitely something I would like to incorporate if I can figure it out.
Interesting… I should be working right now, but I couldn’t resist a quick Google search for code that gathers and compares “n-grams”, and this is what I found : basic PHP N-gram Functions.
It would seem that perhaps some of the more exotic uses of such information could indeed be done without mind-blasting complexity, though that’s perhaps a bit premature and optimistic.
Still, I love research and experimentation, it’s my favorite thing in the world to do
I’m not sure I fully understand the question, but the way the version with key information currently works is that it records all keys in use at the time of a slash. So, for instance if someone pressed AWD simultaneously, it would record all three keys for each of the four slashes that result.
ugh… I just realized that in order to do pattern matching using ngrams, suffix-trees (or arrays), or any other method, I need way more data than this project is likely to generate. I got carried away with the exciting possibilities, haha.
This isn’t going to be built into the swords, or at least I seriously doubt it will be anytime soon; rather, it’s just a HUD device that testers and developers can use to validate the correct operation of the C:SI weapons and identify problems and (hopefully) the causes.
It’s unlikely that we’d have enough usage to generate the kind of data necessary for fingerprinting, for instance, or much meaningful statistics.
Nevertheless, gathering the key data along with the other data is useful, and is already implemented, so that will remain. I’m trying to find a good way to display that information now
identifying someones pattern automatically? i always liked the idea that this is up to the players.. i’m afraid that it comes down to “oh i’m gonna fight XYZ.. lemme check the website to see what hes doing and what i need to do to fight him”.
the next stage is to include a “automatic counter” into the csi-swords that works with the pattern-matching stuff thats supposed to be gathered ;-D maybe one day the swords can fight without player-input *lol
just kidding. i like the idea about stats like reaction-time, hp-damage, duration of a spar and so on.. but this pattern-stuff is really ciritical from my point of view. recognizing someones pattern is skill and should stay this way.
if someone wants to know “my pattern” he should spar me. if hes good he’ll find out. if he isnt he wont find out. its that simple.
Yup, good point, though there are other reasons to want to identify certain patterns that have nothing to do with wanting to publish them for anyone to see.
I’m such a geek that I love seeing how far I can go with stuff like data mining, so I love to speculate about things like being able to identify certain patters so that my bots (if I ever get them working again) might actually have some semblance of “personalized” strategy. But it’s mostly speculation.
And as I said above, what I’m working on is not something that tracks everyone’s duels. It’s something that a single person intentionally uses to track a single duel. I need such a thing for diagnostic and testing purposes, and will provide it to people who are C:SI beta testers, and I thought others might find it interesting.
There are some things that, in my mind at least, make more sense when laid out “by the numbers”. It’s one thing to know that my bot is broken, for instance, and another to look at this fight report and see just how vulnerable and blind he is to kicks. This provides a more objective look at the bot’s weaknesses as well as my progress (or lack thereof) in fixing it. I can look at this report after the fact, over a cup of coffee at my leisure, and gain far more insight than I could “in the heat of battle”.
There are implications there for those who train others, I think. We all know that two people can teach the same subject and yet for any given student one teacher will be the “right fit” whereas the other may not, and sometimes that has to do with nothing more than the way one teacher explains something just subtly different but in such a way that it “clicks” for the student. Fight reports may (or may not) do that for some people. I am one of those people, so I’m biased, lol.
Everything you are talking about here sounds exciting to me, especially when it comes to giving you guys the information you need to fix bugs and improve the performance of the system. But I really am with Makk, at least in that some of this does make me nervous from the stand point of using skill, insight, intuition, etc to do these things. I worry that IF a style, strategy, or favorite patterns of attack could be known using statistics, you take away from people one of the paths of skill development that makes these swords so rich and rewarding. Once you get to the point where you can duel with your eyes closed, beyond the mechanics of the spar are all those mental skills that keep C:SI interesting, and just cringe at the idea of short cuts to those skills.
Its the same thing with knowing your opponents HP and Stam on some meter, which I also agree with Makk about. If you take away the mental skill of counting hits, or reading subtleties of behavior (and being able to bluff those subtleties to make someone assume you are better/worse off than you actually are), you suddenly strip the need of developing those higher level skills that keep C:SI from getting boring. I know you know all that, and I say this all the time, and you aren’t really planning on mathematizing everything, but I still get freaked a little when you talk like this, lol!
One thing I gotta whine about a little while I’m here…
8 Lynkis Aimee KICK No Aimee takes 12 damage
27 Aimee Lynkis SLASH No Lynkis takes 10 damage
I might be alone on this but I’m pretty sure, prior to the spring 07 update, that a kick would not potentially beat a slash like this. I remember after that update came out I ran into an oldbie who hadn’t been around for a while. I asked him to spar, and he said “ok, but don’t use the kick exploit”, lol! (BTW, this person is well known for finding and perfecting exploits). He noticed right away that a person that was relying heavily on kicks didn’t suffer the same consequences as they used to and actually thought it was an exploit.
So say I have 10 HP left, and my opponent does as well. We are blocking, he kicks as I slash, and he wins, when I was the one that did the right thing. It might seem small but to me this should never happen. I think the diversity of what a kick can do needs to carry with it more risk than it has since last Spring. Its not a huge gripe really, and the effect is somewhat marginal, but it can and does lead to people winning on dumb luck, in that the new damage ranges have led to the possibility of winning when you have actually made a tactical mistake.
I understand, I agree, and I will try to state my position : There is not going to be any way for anyone else to look up all of Kasumi’s (for instance) fights and gather such statistics. Any statistics that are gathered would have gone into making C:SI a better system “behind the scenes” by allowing the developers to have a deeper understanding of how the weapons are used in actual combat, and even then only by people who choose to record a duel. I had thought that it might make sense to allow people to use such things in the training of themselves and others, but the more I think about it the more I think it should only be available to beta testers and developers exclusively.