So far, Blizzard has given the community amazing tools for doing replay analysis, and the community has compiled a lot of stats and built services to work with that data. Even so, I think there’s a gap in levels of analysis. At the top, we have aligulac, which compiles win-loss data into accessible predictions but doesn’t give much insight into a playstyle or why the data is the way it is. At the bottom, we have ggtracker, which provides graphs and data for individual games and can compile stats from low-level mechanics in games, but is missing higher-level analysis.

In-between is where I think theorycrafting reigns: strategy. What build orders work against what? Which players are the best at worker harass? Who has the best forcefield placement? These details lie below win-loss analysis, and although these may be extractable from the replay data, this analysis requires some qualitative analysis to determine what constitutes specific tactics or plays. Even the best machine learning techniques (supervised learning) require labeled examples to learn from.

There was an interesting thread by anderhyo a few weeks ago asking what sorts of stats could be compiled, and the type of stats they described fit nicely into this level of analysis. The process of annotating these replays is somewhat laborious, but I think it could be accomplished by having in-game scorekeeping. In addition to the players and commentators, another person could be tasked in-game with marking, labeling, and drawing in things such as major engagements, who won the engagements, huge blunders or mis-micros, and other interesting stats. This data can either be encoded in the replay or shipped off to another service where community members, tournament organizers and commentators, and players could trawl through the data to find interesting things.

For comparison, I think this type of scorekeeping is common in other games. Baseball scorekeeping is a beloved pastime by some fans and is critical to modern-day sabermetrics. Even Chess, which can be entirely represented by simple, discrete, algebraic notation, has annotations for mistakes and good plays to more subjectively draw attention to specific things.

As of a few months ago, I think this type of scorekeeping could be possible thanks to Blizzard’s inclusion of extension mods. Just as GameHeart can be plopped on any map by setting the extension, presumably, we could add scorekeeping to any game, and tournaments could add scorekeepers alongside the commentators and observer to mark certain plays.

So what do you guys think? Do you think something like this would be valuable to you? What sort of things should we scored in a game? For the mapmakers out there, I myself have never used the map editor and would need a hand getting this put together, though I would be happy to inform and help on the format of the data and getting it stored and available on an external service or website.

TL;DR Anyone interested in to helping to make an extension mod for scorekeeping in StarCraft?

(Discussion at http://www.reddit.com/r/starcraft/comments/213v1h/ingame_scorekeeping_for_starcraft/)