Comments on Lost Garden: Soul Bubbles: A classic game ill treated by exper...

Long and interesting post. Reminds me of "Blink" b...

2008-10-27T17:32:00.000-06:00

Long and interesting post. Reminds me of "Blink" by Malcom Gladwell. Have you ever read it before?

Ah, Tomb Raider! This has really been some kind of experience.

The thing is regardless of what the average user e...

2008-10-11T13:17:00.000-06:00

The thing is regardless of what the average user experiences when they play a game for the first time, I've been playing games for 18 or 19 years. The reviewer's "expert bias" is exactly the same bias I'll be experiencing myself when I play the game. I couldn't care less how new players play the game and more importantly I don't think new players could care less about how new players play the game after 10 hours of play.

While being fundamentally enticing and good on a level that appeals to everyone is a worthwhile design objective and an objective that creates markets and profits, all this article points out is that there are multiple audiences for games, a key case of stating the obvious. You don't review movies from the point of view of someone who's never seen a movie before so why should games be that different? There's already a big divide between mainstream reviews and "gamer" orientated reviews and unsurprisingly the mainstream will, on the whole, read mainstream reviews tailored to mainstream gaming experiences while "gamers" will read gamer-orientated reviews. I'll read reviews relevant to my circumstances and the last thing I want is for concrete reviews I rely upon to start pandering to the broadest possible spectrum, because it's simply not feasible to write a review for everyone. Furthermore, in my opinion, from an artistic point of view, there is little to no validity in evaluating something in it's most basic terms devoid from the environment from which it's born. If someone rants and raves about a new album it's virtually meaningless without reference to the artistic vista from which it has been born as well as comparison to it's peers and predecessors. Soul Bubbles seems like a fun and well put together experience but there's a reason that reviewers who play a lot of games have scored it as average and that's simply because out of all the games they've played it is nothing but an average experience, perhaps a well presented and fun average experience, but not much more than that. I'm curious though and will probably buy it, mainly on the grounds that swapping DS games around with friends is a major enough factor that a short play time isn't too big a deal.

Ultimately there are some interesting questions of what a reviewer's purpose truly is, but the fact is that "expert bias", consciously recognised or not, does not invalidate the reviewer's experience in any way shape or form.

@nathan: You make a good point, and upon re-readin...

2008-09-22T01:08:00.000-06:00

@nathan: You make a good point, and upon re-reading my post, I see too much hand-wringing "Games are Art!" self-righteousness in it. I was reacting negatively to the notion that the game reviewer's duty is solely to judge how a game's supposed target audience will receive the game - that is, to predict the aggregate enjoyment that the game will elicit. This seems to be the argument put forth by the article.

Certainly, there are plenty of game reviewers out there who attempt to function in just this way. The Soul Bubbles article was written about them, and my response may be out of place. I just wish we saw more game *criticism* targeted at a mainstream audience; I wish games had a Roger Ebert or an Anthony Lane.

But shouldn't we be encouraging real game criticis...

2008-09-21T22:47:00.000-06:00

But shouldn't we be encouraging real game criticism? Do we want to regard games as a medium and capable of being true cultural artifacts, or do we simply want them to be mindless escapism that we fill our free time with?
This is a false dichotomy, albeit a pervasive one. There's no reason that one precludes the other. And, for that matter, Bejeweled could certainly be considered an artifact of today's culture - naturally, it lacks literary merit and provides no thoughtful social commentary, but commonplace popular entertainment is as much a part of our culture as "true art" (whatever that means) is.
Look at the progress of film as an artistic medium - you have profound, meaningful works of art, and you have light entertainment, and naturally everything in between. Neither precludes the other. To me, at least, games already seem to be headed toward a similar path.

When a game takes itself seriously, it's appropriate that a reviewer would do so as well. But similarly, when a game sets out to be easy, fun entertainment, why should a reviewer hold it to some other standard? The hardcore gaming sites have rated the game in accordance with what hardcore gamers - not the target audience - will get out of it. I suspect the casual gamers don't spend time investigating game reviews in depth anyway - one of the key features of casual gaming is that it's low-investment.

Hi Danc,Just wondering if you'd had a chance to ch...

2008-08-11T06:13:00.000-06:00

Hi Danc,

Just wondering if you'd had a chance to check out Johnathan Blow's long awaited Braid on Xbox Live Arcade.

Cheers~

Not to harp on this too long, but I assumed you we...

2008-07-23T18:59:00.000-06:00

Not to harp on this too long, but I assumed you were using Metacritic for your sample, which still shows only 5 user votes (now at 9.8) My apologies if you were pointing to a different site, as 40 user votes is more solid.

Andrew above makes a good point: every "expert review" of Soul Bubbles comes from an "expert" (hardcore/fan) publication. It seems like non-expert reviews would find a home in publications with a broader audience, like Entertainment Weekly, regional newspapers and the like.

Don't you think just like games have a target mark...

2008-07-20T14:19:00.000-06:00

Don't you think just like games have a target market that reviews also have a target market?

I can't imagine that the people who like to go to sites like gamespot or 1up and read reviews, and look at screenshots and listen to podcasts are this alternative audience of non-expert gamers.

The expert bias is only a bias if there are non-experts in the audience. I've never met anyone who wasn't a pretty avid gameplayer who had a subscription to EGM.

Certainly, there's a place for a Consumer Reports-...

2008-07-20T14:12:00.000-06:00

Certainly, there's a place for a Consumer Reports-like approach to game reviews; that's essentially the role that Gamespot, IGN, and friends are trying to fill. And when it comes to more casual, mass-market games, those reviewers' hardcore biases prevent them from doing a very good job, as you point out. If we're going to review a game as a product, then simply focus-testing it and assigning it a score based on "aggregate enjoyment" seems as good an approach as any -- but it's an utterly soulless one.

But shouldn't we be encouraging real game criticism? Do we want to regard games as a medium and capable of being true cultural artifacts, or do we simply want them to be mindless escapism that we fill our free time with?

@SteveThere are currently 5 written user reviews, ...

2008-07-19T18:41:00.000-06:00

@Steve

There are currently 5 written user reviews, but 40 user votes. I'm assuming that the score of 9.1 is based off the ratings not the reviews. (otherwise the average would be 9.5).

It is still an interesting comment. I didn't calculate the margin of error so it is quite possible that this is just a statistical fluke. A sample of 40 isn't bad, but my stats teacher would be horrified at the eyeballing. :-)

There's also the chance of vote stacking. That's one of the problems with user rating systems.

take care
Danc.

I'd say one other factor in the low reviews is the...

2008-07-19T13:23:00.000-06:00

I'd say one other factor in the low reviews is the approach to playing the game. I'm playing Soul Bubbles a level at a time, one or two levels a day, and really enjoying it. A critic, by sheer necessity, is going to play through the game in one or two sittings. I can easily see you get bored doing that; it just isn't how the game is meant to be played.
Which is a shame, since it may well be the best game the DS has.

I'm afraid your premise here is flawed at the outs...

2008-07-18T09:35:00.000-06:00

I'm afraid your premise here is flawed at the outset. One major disparity between expert and user scores in your example is simply proportional: when you posted this, only two users had reviewed Soul Bubbles, while a dozen or so expert reviewers had weighed in. Now that a total of three user scores have been posted, the user score is 9.6.

A tiny sample of online user reviews is a poor measuring stick against, well, most anything, since a couple of anonymous fans mashing the 10.0 button is basically meaningless. Seems like a rickety foundation to base so many broad conclusions upon.

As someone who keeps the method of criticism in mi...

2008-07-17T13:52:00.000-06:00

As someone who keeps the method of criticism in mind most of the time (and exercising it with varying degrees of success online), I found this article extremely interesting. It pours a slew of ideas into my Pot O' Discourse.

It's what I stand in line here for, and dammit Danc you keep ladling it out!

In some cases, people just need to read the full t...

2008-07-17T07:20:00.000-06:00

In some cases, people just need to read the full text of several reviews, look at some gameplay video on Youtube, troll forums about the game, and make a decision for themselves.

This post makes me envision some strange future where designers write academic journal articles about the relative merits of Design Element X or Design Methodology Y and support their conclusions with data gathered from gamers crooning over video screens with electrodes strapped all over them. Hopefully in the near future we can at least form a common critical vocabulary that isn't entirely based on hazy recollections of past games like it seems to be today, and instead tries to decouple critique from the games being critiqued.

Your point about how modern games are multiplying the 'possibility space' beyond the critical scope of linear media is interesting, and makes me wonder how, or if, reviewers can hope to really properly review a game which could contain a large deal of emergent "content" the reviewer has no hope of seeing.

Elsdoerfer, I think you're quite right. The proble...

2008-07-16T04:37:00.000-06:00

Elsdoerfer, I think you're quite right. The problem is that the subjective point of view expressed by Metacritic scores is taken to be the ultimate measure of quality of a title.
We need to realize that videogames have matured to the point where one viewpoint no longer suits all.
It is probably not an easy thing to do, as I myself am emotionally conflicted at the fact that videogames as a medium are no longer equivalent to the gamer culture I identify with.

There seems to be this notion that film reviews ar...

2008-07-16T04:20:00.000-06:00

There seems to be this notion that film reviews are better suited for making an informed buying decision.

However, looking at metacritic, you'll see review film scores commonly diverging across the whole scale, while in the case of games, it's mainly more of the same. A ~70 game gets reviews from 60 to 80 etc.

Wouldn't that indicate that the opposite is true, that films are a much *more* subjective experience, and therefore harder to review?

In fact, I think it is clear that game reviews traditionally have attempted to be more of an objective test, like testing a piece of hardware. I have never been a fan of that.

So I would actually like to see them take a slice from film, and *embrace* their subjectivity.

Sure, we are probably all influenced one way or another by reviews in our buying decisions (at the very least indirectly). But does anyone actually think that it's guaranteed that he will not enjoy a game because it has a 50 score? I certainly don't. I don't put any pressure on a reviewer to be "right".

Ultimately, it comes down to a very simple recommendation system for me. And there is a reason why personal recommendations (i.e. word of mouth) has always, and will always be much more effective in convincing people to go see a movie, or play a videogame.

As a reader of this blog as well as 'another guy w...

2008-07-16T04:02:00.000-06:00

As a reader of this blog as well as 'another guy with an intricately detailed, heavily biased opinion', I'm flattered that you've linked to my small essay on the state of game criticism. However, I'm a little perplexed that I'm expressing a fallacy of epic proportions.

I will agree I've disregarded the place of the representative user in game development - at least for the purpose of that opinion piece. But the search for Citizen Kane was written to deride reviewers for merely trying to compare games to art and literature while ignoring the unique properties of games. I wrote: 'game criticism will come to resemble an ongoing dialog' with the intention that game criticism would involve more interaction between players and critics regardless of the skill levels of either. And equating game reviewers to game designers was to suggest that game design and design techniques had as much to contribute to the art of game reviewing as the converse - including (but again not stated) the iterative user testing methods of Valve and co.

At least you and I both agree that games are not movies.

While I think it's quite valid to say that using t...

2008-07-16T00:55:00.000-06:00

While I think it's quite valid to say that using the same model for reviewing games and movies is a flawed concept, I think the flaw is a quantitative one, not a qualitative one. (I would characterize your description of the "learning" that games cause as fitting in with "reacting" that movies cause, but I think that's just a semantic distinction) Because games require more personal involvement from their audience, the flaw of the process of reviewing something (subjectivity) becomes more noticable. It's still there when someone reviews a movie, just not quite as noticable.

You refer to the universality of the human experience as what makes movies (or any stories, really) satisfying, and I strongly agree with this, but stories about various things can resonate more or less with certain people. If I can understand that something would resonate with someone else, even if it doesn't impact me significantly, all that does is make me a halfway decent movie critic. I guess I'm just not clear on what inalienable quality makes this game reviewing process fundamentally different.

Then again, I'm one of those elitist jerks who happens to think the only reason why games usually don't have anything meaningful to say about the human condition is because the job of writing is shoved onto some designer as a second thought. Show's what I know...

Contact me if you have Traveled both Time and Spac...

2008-07-15T18:15:00.000-06:00

Contact me if you have Traveled both Time and Space to be where you have been at Cookey_1400@hotmail.com

An off topic question for DancHave you ever been t...

2008-07-15T17:26:00.000-06:00

An off topic question for Danc

Have you ever been to, or do you know anyone from Long Island, New York.

-Daniel "Zeed" Joseph Cook

Let's assume that indeed it was just a matter of "...

2008-07-15T00:20:00.000-06:00

Let's assume that indeed it was just a matter of "lack of challenge" that led the game to be underappreciated by the core gamer audience.
Would it have been that hard to include a "challenge mode" that imposed, say, moderately hard time limits on level completion? You could have a prompt to choose between "adventure mode" and "challenge mode" at the beginning of the game, in order to funnel players into the experience best suited for them.
It seems like they could have added at least part of the core gamer audience to their focus (and maybe 10 metacritic points to their score) with just a little extra effort.

And next time, I'll have only one 'finally' paragr...

2008-07-14T10:28:00.000-06:00

And next time, I'll have only one 'finally' paragraph. :)

I think that Danc has hit onto something here, and...

2008-07-14T10:26:00.000-06:00

I think that Danc has hit onto something here, and it's been mirrored in some of the comments left by others. Reviewers and critics are, by definition, 'experts' in a field, whether it's the culinary arts, literature, film, or games. And just like critics in other media, it becomes important to know how a particular reviewer/critic matches the likes/dislikes of the individual.
For example, if you really dislike everything that Roger Ebert loves, you'll not give his 'thumbs up' much weight. On the other hand, you may have a local movie reviewer who has loved the same films you have. That person's opinion and analysis would have more weight. The trouble that I see is that the gaming industry doesn't have the maturity in critique or review that other media do. Coupled with the instability of the Internet in general as people come and go and everyone has a soapbox, no matter the size, it would seem impossible to find a game reviewer that you can consistently expect to look at games the same way you do.
Finally, as Danc pointed out, the aggregate scoring used on some sites is a statistical fallacy. Amazon has spent millions trying to get their recommendations down to a science – and they miss more often than hit, I think, but the work is being done. And it can be done, but not generally by hobbyists, which are the people who start most of these review sites in the first place. That these sites may move into more corporate territory (IGN, for example) is no indication that they are applying the same business models and analysis that will help someone really understand whether they will like a game or not.
And finally, the analyst group that Danc mentions comprises, in my mind, an extremely small set of individuals. In fact, in all of my years developing software, I’ve met one person who meets those criteria. And he, of course, is the author of this blog. Perhaps if we had more analysts, we could begin to better help the average user who Googles the title to see if it’s worth buying. As the ‘experts’ in the field, I think it’s incumbent upon us to do what we can to help them understand. Whether it requires a re-envisioning of how reviews/critiques are done or a more/better/expanded categorization model (or some other deconstruction) remains to be explored.

I disagree. 'Soul Bubbles' does stand out with it'...

2008-07-14T04:45:00.000-06:00

I disagree. 'Soul Bubbles' does stand out with it's beautiful art direction and a general 'feel' which isn't one of fear or constant over-stimulation, but of exploration and relaxation. As a gamer, I'm usually 'stressed' and end up speeding throught some levels to finish quicker. With 'Soul Bubbles', I truly felt like I could play at my own pace, on my terms, which is rare (and pretty nice).

Still, 'Soul Bubbles', ultimately, is not this grand classic. The novelty factor wears thin way too quickly and in the end, the product feels more like a high-quality tech demo than anything else.

I was lucky to also play an early version of Soul ...

2008-07-14T03:54:00.000-06:00

I was lucky to also play an early version of Soul Bubbles and I wholeheartedly agree with Daniel.

Soul Bubbles is a rare game. Not only does it teaches you new gameplay skills, as Daniel writes, but it also allows you to experience new feelings. The rhythm the game induces through the dragging motion that controls the winds is something truly new. The whole game is tailored around this experience. While I guess I fall into the "hardcore gamer" category, I couldn't help but be captivated by the simple ideas and twists the game offers. Very few games give you the feeling that they're built for exploration, that you should be rewarded for testing stuff and not only for finding the right answer. Playing Soul Bubbles brought up the same feelings in me that the first hours of Super Mario 64, Tomb Raider or Portal gave me: I had to explore all the nooks and crannies of every level.

If you're interested in game design, you should definitely play this game.

Stephane

It is always good to read a note about game review...

2008-07-13T23:28:00.000-06:00

It is always good to read a note about game reviews. I consider myself a strange mixture between casual user and very hardcore one: I have habits like a casual user, buying a couple of games every three or four months, preferring small games like Picross DS than Contra 4, etc. But my favorite genre is the dungeon crawling, and take real pleasure in extremely difficult games like Etrian Odyssey and Shiren.

At work we have a small testing group who checks our software builds for bugs. However, they don't test usability, that is left to our beta testers (clients who signed a NDA), because testers already expect certain things to work in a certain manner that is not always what the final client wants. For example, the tester may think a button would do something because it makes sense to do it at the current dialog screen. However, the client may think otherwise not because of the screen, but because of the situation within the screen.

The main flaw of game reviewers nowadays is that they review every game as a gamer. A casual friendly game like Carnival Games is awful from a gamer perspective, but it is a million seller because the target public accepts it. The site where the review is published is targeted at gamers, true, and therefore their opinion may match that of the gamers. However, it does fail at analyzing the potential of the title. Just like the gaming industry before the casual-gaming craziness, the gaming media focuses in a single environment instead of opening to the world. As an example, EGM did not review Endless Ocean because it was not a game, you only explored, there was no danger, you could not die at all.

Hardcore titles should be analyzed from a hardcore point of view. A game for girls should be analyzed from their perspective. A casual game should be analyzed from their perspective. The media, right now, analyzes everything as hardcore. And while the reader of the site is hardcore, some day daddy and mommy will try to reach them to know what they think about this or that game and the site won't be able to welcome them.

Just as companies found their own divisions to build family-friendly games, so game sites should have their own divisions of casual gaming reviewers. They don't make an Xbox 360 reviewer analyze a Game Boy Advance title, so why they let a hardcore gamer review a casual game? That is the only way those titles will be reviewed correctly.