Bless you Shawn. It's gonna take me a while to read through all of this, but I will. There is such a need for this type of conversation in gaming. I think as, sadly, more and more sources of gaming criticism and journalism dry up, it is important for the respected and experienced voices to get together and continue examining and growing the industry. That's why I LOVE Slate's Gaming Club and any podcast that can bring together multiple, informed perspectives on video games. Continue the good work. Can't wait to see the fruits of your labors from 2K Boston as well.
Having only read up to John Davison's second entry, I apologise if this has been discussed in the latter half of the post, but I just wanted to get this out there.
I believe the reason for all the stress about scores is precisely because of that disconnect between reviewer and critic. What games journalists as a whole seem to be doing right now is trying to merge the two, which is a task without merit. Both the ten-point or five star or whatever scale and the critical analysis of a game's content have a place in the industry, but it's vitally important not to confuse the two as they cater to wildly different audiences.
At a base level the job of a review, as was stated here, is to inform a purchase choice. The majority of gamers have limited disposable income. Comparing Prince of Persia to Sonic Unleashed is ridiculous from a critical standpoint but imperative when you only have sixty bucks in your pocket.
The person who wants to discuss how an individual game affects the medium as a whole or storytelling in general or any one of a thousand other factors is a wholly different person to that first audience, and I think that any attempt to appeal to both audiences at the same time is on a path to frustration.
Although, a spell checker would be nice. There are a ton of typos even in the short pieces.
For example, "shouldn't it that exemplify."
I realize these are emails and you want to maintain the original correspondence, but I keep getting caught up on it. Not trying to troll, trying to give some constructive suggestions.
One of the things left unsaid here is to what extents exclusives are unique to the gaming industry. Apple has a handful of favorite outlets (NYT, WSJ, Newsweek, USAT) who get to review new product, and they've done exclusives (flat screen iMac to Time) before.
We just saw Activision give USAT the exclusive first look on GH: Metallica. To what extent are broader publications inoculated from these pressures? Could not newspapers and general press magazines be able to more accurately do exclusives because they're not as dependent on the advertising from the people they are covering?
I've advocated Tribune Company (here in Chicago) to aggressively go after gaming reviews (either as the CT, the LAT, or a company-wide reviewer) because they can, due to being less reliant on gaming advertising, be more objective even with an exclusive.
I'd love to see your thoughts about whether or not Seth and N'Gai can be freer to do fair exclusives than Shoe or Leigh could at their publications.
-Sachin Agarwal, Dawdle (sorry about the Jerome Walton name; that was my Deadspin nom de plume)
December 18, 2008 at 10:50 AM
Anonymous said...
Thanks guys. As an avid consumer of gaming coverage, this is fascinating stuff. I guess I can count myself in minority in that I don't much see the point in numerical scores.
The extent to which a score ultimately is useful to me would be a bunch of statements like "buy/rent/don't bother..." broken down with qualifying clauses like: ...if you do/don't like the genre ...if you do/don't like other games in the series ...if you don't/only care about multiplayer/singleplayer/coop"
I thought I might help to see what an average gamer/consumer read reviews for. I can typically only afford one game at a time.
If there are a few games out that peak my interest I will go to GameRankings and find the highest reviewed one, read snippets about it and typically go purchase it.
If there is one game I am particularly interested in a I will go to GameRankings and see what overall score the game has. If it is above an 84 or so I will have no issues with buying it. If the score is under that however I do further research by reading the actual review.
If there a few games I definitely want to buy I go through and read every review on each. Like Fable 2 vs Fallout 3, I knew I would play one of them but which to buy first, I had to research to find out. I read everything I can on each and it is then that a review is truly helpful.
If there are two sports games and I have no idea which I want to get (NHL 09 vs NHL 2k9) I will read reviews which typically compare them anyways.
If there is a game out for multiple systems and I am not sure which to buy it for I will consult reviews to help me with my decision.
I don't read "reviews" for critiques, I read blog posts or forum threads or listen to podcast's. That is much more interesting to me. I don't want a critique going over every little thing a review typically would. I want to hear what the writer thought of the game as a whole not broken down into small reviewable sections.
The message that I keep seeing here is that it's all about economics.
"Publishers want good reviews. Editors want exclusives. Magazines and websites want advertising. Advertisers want good reviews."
This is the shortest quote that could encapsulate it all. Review scores are simply a symptom of the audience: yes, it is regrettable that a score means people don't read it, but remember (as if needs to be said): we give the Internet and gamers too much credit if we assume they fully read the majority of anything that is posted. Remember the comments in Game|Life's top 5 360 exclusives?
If you want to write more reviews that give little or no consideration to the scores, you need the audience. Is that audience there? If so, how can you get it? The only answers I can think of are economic ones; I hate to be so reductive, but I don't see how business and money aren't what makes review scores go round.
Shawn, I know this is a bit early on, but is there any plans on binding this symposium and printing it? I found this first part fascinating, and I know with much more to come I would love to have a copy of it sitting on my coffee table to make me look intelligent.
Fascinating stuff, and I'm eager to watch as the rest of this symposium series unfolds.
One thing I'd like to see tackled is why the hell game scores are so seemingly inflated. To pull out its easiest analogue, let's look at typical film scores. Rarely do I see as many perfect 10s (or four stars or what have you) for a film, especially when the accompnaying review is a list of things that are terribly flawed about it. From the reactions I see, a 7.5 review score for a game is practically a kiss of death, whereas a film reviewer (like Roger Ebert) can make a case for the value of a 2.5 star film.
Review scores only matter as an aggregate, at least for me. They're a way to summarize a large group of reviews in order to get an idea of what critics thought of it. When I'm looking at a specific review, the words matter far more.
December 18, 2008 at 1:08 PM
Anonymous said...
Personally, I knew that I'd probably end up sounding like a mix between John D. and Robert A. here.
I'd love you guys to keep this going with all the topics and issues floating around the games industry. It’s an effing goldmine of reading for someone like me.
I'll spare you the long winded personal rant I have on the matter (as you guys spanked it pretty nicely) and just provide some random observations to ponder over.
1 - The idealism that some arguments form here makes me feel a bit snootish for the thoughts that immediately strolled through my head. I consider myself a fairly open-minded person, yet I still found myself sighing at some of the more idealistic based statements here.
2 - "we're all human" got put forth just enough for me to notice it. Not that it's bad or anything, it actually made me smile with a bit of malice in my heart =). Someone go back and count it, it can’t be that many…
3 - I wondered when that damn art debate will come back. Harry's last comments came in like Elika at the last minute for me...They won’t be art until 400 years from now. Seems none of us are smart enough to realistically have it function as a means of existence now. People like Ebert are exempt, as are all people who talk out of blind ignorance.
4 - Psychology and Subjectivity...I prefer to see that to be honest. When those things are exposed, THEN you're really starting to approach whatever sad little reality we hold dear.
5 - Also, acknowledging rationalizations and spurious objectivity is something that surprised me here. It's nice to see the editors knowingly bowing before it (while whispering mean things under their breath).
6 - I’m sure that centuries ago when paintings were the “highest” form of art there was, there was some trendy asshole that walked around pointing at canvases going “OH, THAT’S AN SEVEN!”
I'm reminded of the proposed Cheap Ass Gamer scale for reviews - if the number is meant to reflect some kind of consumer impulse, review the game on a 60-point scale.
The idea then is that the number tells you at what price you should purchase the game. A game that deserves an instant, day one purchase would get a 60. A game that you should wait for bargain bin status would get a 10. A game which is "good" but not worth "full price" might get a 40.
Arguably, Cheap Ass Gamer is the most consumer focused gaming based portal on the Internet, so assigning a dollar value to a game might be crass... but if that's the ultimate end goal of a score, why not?
Assigning a dollar value on a game, much in the same way that an "expert" on Antiques Roadshow would assign a dollar value to a painting or a coin seems to be what people who want scores are looking for anyway and it unifies the seemingly arbitrary scales that each publication imposes on its reviews.
For what its worth, from an outsider perspective, I don't even read reviews anymore. As a consumer, forums like the 1up Boards or GameFAQs or NeoGAF provide much more useful feedback. If a dozen people in a thread for a game agree that the controls for that game are inherently broken, then most likely the controls are broken. The "non-hardcore" version of this phenomenon is the user review on websites such as Amazon.com. In the "Web 2.0" age of "social networking" and "crowd sourcing", consumers are looking to each other for advice rather.
Anyway, I'm really looking forward to - if it happens - the Game Criticism vs Game Reviewing topic. I'll fully admit that I'd love to "critique" games for a living... unfortunately, the only way to do that right now is to get a PhD and become Ian Bogost.
I do not think people will buy videogames of the early ages in 100 years as they buy paintings. Paintings were part of a culture that was led by interconnected, highly educated people, elitists if you will.
Nowadays, culture is a thing off the masses, and elitist culture is split into many niches. A far mor appropiate medium to compare this to is the comic book, as it also started as a mass culture, not lead by intellectuals.
On that not, I do not think that games match the narrative experience of the other narrative media. I still love games with a liner narrative, though, because they do things other media stopped doing long ago: they improve in quality as time passes and newer games get made. Stories are getting more serious, immesion-breakers minimized.
Games have become better and better in both technology and story presentation. Because there are such clean, highly visible differences in quality of those divisions, I think a score may be legitimized. Ore so than a movie review having a score, at least.
Look up tautology in the dictionary. It does not mean what you think it means.
But to be a little more constructive: I was really disoriented by the structure of the responses. Direct questions were asked and never answered (much less acknowledged by the recipient) and whole conversation threads were dropped, only to be picked up again without warning. N'Gai references Harry, and it's half a dozen responses before Harry puts in his first words.
I'm really interested in what you guys have to say, and you say a whole lot in this post worth reading, but I'm wondering if there isn't a more legible way to present it.
December 18, 2008 at 2:52 PM
Anonymous said...
It’s a shame that blogger doesn't have a more sophisticated comments system, such as only giving comments to specific entries. There’s a large amount that I would like to address on here but it's much harder in this manner.
I do like the "Rolling Stone" scoring method that was mentioned here. It would give a feel of validity to reviews that personally I feel that any score lacks. Personally I don't trust review scores at all, they work on a comparative scale that differs in each reviews outlet, although I do have greater faith in the new 1UP grading system, which is a much more general indication of quality, (rather than a specific indication of a less understood scale of quality). When I read reviews I've always read it in its entirety to see what the reviewer thought, usually in order to see if it was a game that I would like and also to see if there was anything that would put me off wanting to play it, often reviews would decide whether I brought a game that I wasn’t sure about. I think that review scores cause a lot of problems, while they aren’t inherently evil they are quite possibly the root cause of stagnation and corruption for video game reviews based industry, possibly even the games industry itself. While this scoring system is in place PR will remain fixated on them, trying to influence and even control the outcome of scores. Sometimes this will work and even if it doesn’t it will often mean relinquishing exclusive reviews and as mentioned above magazines need to co-operate to succeed. Since many publishers now are internally ascertaining the quality of a game and the ability of its design team by its Meta critic score, in a worst case scenario this would mean that the actual content of a game is irrelevant it will receive a certain score regardless. It would also mean most reviews might as well post a number and add some text to make it look official, maybe the games press description (a review Shawn mentioned some time ago actually did this, maybe he’ll link it in a later topic).
I believe we have this bullying of reviewers in video games but not so much in other mediums such as film is due to the youth of the industry, and also of its audience. Games are not yet universally understood and accepted as films are, they don’t have the wide spread and varied audiences quite yet, although this is changing at a exponentially increasing rate, mainly due to Nintendo. I don’t believe there is yet a large enough demand for high quality reviews or mentioning of certain types of games in non-gaming publications. Such sports games reviews in sport magazines and critiques talking about the “best entertainment experiences of 2008”.
Video games aren’t to the point where a review could be in a general newspaper and most people will have a general understanding of what you’re taking about. Because when gaming reaches that benchmark when it’s reviewed as any entertainment would be to a world that thinks of games as a regular entertainment, that will be when specialist press will need to either become where you would look for high quality gaming reviews, or appeal to specialist or niche audiences within gaming.
Thanks for the post - this is a very interesting conversation. I'm looking forward to reading future symposiums. Jeff's appraisal of review scores' purpose as shortcuts for an audience unwilling to think critically about a reviewer's prose seems like a bigger discussion and, possibly, the real root of this conversation. Is "games journalism" shackled by its history and its younger audience? Why is it that thoughtful conversation about games is much more likely to be found on blogs like this and in British gaming publications?
What does one of the "normal" people (as Dan Hsu describes us) feel about review scores? From someone who has read reviews since 1989 for purchasing decisions to now, my attitude has changed
At first, review scores have become the litmus test of a game that I am on the fence about or now nothing of. I want a review to answer the questions of a cynic, a skeptic or just a confused gamer if the game is worth one's time. It is especially true for sequels, sports games and now, all those Activision games that get once a year treatment.
Reviews are no longer the only source that influences my purchasing decision, I often listen to podcasts, the vidcasts, demos and/or follow the "buzz." Even previews have time to time influenced a decision, just recently, Shawn Elliott's Far Cry 2 preview spark an enough interest to buy the game. Stephen Totilo's daily dairies have become like a book club, as I play similar games, for example, I experience the same frustrations that Stephen faced in Dead Space.
Reviews will never go away, as Dan Hsu points out, "It’s ingrained in society and it’s pointless and stubborn to fight it. People don’t always have time to read a 2000-word, well-crafted review to get inside the brain of the reviewer. For most folks in this short-attention-span world, that “4 out of 10” usually says more than enough." The problem with Review Scores are not the scores themselves, but the fact that one person can never review the bulk of weekly game, as Roger Ebert does with the movies he watches for the Chicago Tribune. The last point is better reserve for the next topics.
Reviews for the most part are completely pointless, at least for the enthusiastic gaming consumer, which I'm sure anybody on here reading this is. I don't need your review to tell me that Metal Gear Solid 4 is going to be worth playing. I also don't need your review to tell me that Grand Theft Auto is going to be amazing as well. To be honest, the only time I look at reviews are for the games that don't have the marketing of these mega titles. In this day and age where games pretty much sell themselves, I rarely if ever need a review to merit my purchase.
Then of course there's the possibility of being burnt by the review process. Far Cry 2 got a B+ from 1up, 90+ scores from other sites and what not, and I ended up really not liking that game much at all. What I want to see in critiques is a critic's ability to point out the design of the game, how it compliments its other parts, what certain decisions of the designers worked, what didn't. Why the game was designed the way it was, and how it affects a persons experiences of it. Mainly I just want to see critics question the way the game was design, instead of simply absorbing it and reflecting it back to consumers. Maybe this is a lofty claim, I don't know, but I rarely if ever seem to get this. I do think reviews are needed in this industry, but I just wish people would lend an ear to critiques of games as well.
Why can't we just have both? A review score for a game upon its initial release to the market, then when its shelf life has diminished and there's a sense of detachment from the product, then the criticism can kick in. Idk, perhaps this logic is backwards but I think after the consumer has had a chance to play the game, perhaps constructive conversations can then start on message boards. People just invest too much in a titles release, they need time to sit on it just as much as reviewers/critics do...
It seems to me the internets makes this discussion a bit irrelevant (albeit more possible and more practical.)
I'm skeptical along the lines of Totilo.
There's plenty of more in-depth & "alternative" coverage of games to be found on blogs, forums, small sites and podcasts.
For anyone wanting that type of content you can find it. Or you can create it.
So it doesn't seem to matter if the big mainstream sites & mags are heavily score focused, publish nutritionally-lite reviews and broker exclusive reviews, etc.
Nowadays, thanks to Al Gore, the thirstier of us gamers have plenty of alternatives.
Thanks so much for a thoroughly deep and fascinating discussion.
There was one point touched upon that still left me thinking though... there seems to be a dichotomy in all this between writing for the enthusiast audience and writing for the "average" gamer (not casual just... less hardcore). The purpose for the score, in my experience, is taken differently by these groups. Hardcore gamers are the kind you generally find on message boards defending and refuting the score but, as mentioned, have probably already pre-ordered or will play the game regardless to be part of the discussion. The gamer who is less invested in being a part of the society of games discussion/review/news, however, will probably use it as an aid to his/her buying decision, if they are aware of it at all.
If this is the case, the score plays a dual function of ranking the quality of the game and recommending the game for purchase, which I don't believe are the same thing. It's little wonder then that they provoke such a wide range of opinion. If a writer speaks to the value of the game for the kid who buys 2-3 games a year they ignore the use of the score by the person who buys 10 games a year and doesn't need that return on investment from every game and vice versa. Certainly these two goals can overlap but I don't think it's possible to consistently serve both masters.
By the way, I almost never comment on blogs or reviews but the quality of this symposium, even at this early stage, is so satisfying I couldn't help myself. Looking eagerly forward to the next installment.
"Unless I'm missing something, that's quite an indictment of a portion of your audience. Why would you write for people who won't or can't read an entire review?"
Because those people generate ad revenue, which pays wages. Do you live in the real world?
Not trying to offend, but really, we don't all have the freedom to write thoughtful stuff for people who actually appreciate it. Someone has to write for everyone else.
This symposium reinvents the compelling wheel and truly is greater than the sum of its parts. However the graphics are last gen and there is no sound. 10/10
:P
Just wanted to say thank you very much to everyone who is participating in this. There is allot here & I am going to read through this again.
I hate scores because they're too easy. Being able to cross-refrence a 6 vis a vis a 7 is a comfort, I submit, *only* due to it's immediacy. There isn't any subtlety, philosophy, or art applied in strictly numerical evaluations (though Gillen's thoughts on 100-point scales is incredibly thought-provoking for me) and while I can't for a moment dismiss the commercial applications of a publication which focuses on scoring reviews I yearn for a space where the paragraph replaces the digit. I think a continuing sign of a medium of expression in it's adolescence (I think we've at least moved out of our infancy) is the lack of a truly viable space in the likes of what Harry Allen referenced. There's an intellectual and--dare I say--spiritual need for such an endeavor and that is where I see at least one of the "problems" Totilo is asking for; right now we've *only* got review scores. There isn't an adequate counterweight.
That took a while to read, and after burning my brain on that all I can really say is, fantastic.
I wonder, though, what significance you all see in the increasing popularity of podcasts and their influence on reviews and what readers take from them. Seeing as most of the contributors to this symposium regularly or at least occasionally appear on reputable podcasts, I'm sure there's a little something to be said. As for myself, I've found that in the past year or so that I've more and more been basing my purchases what's being talked about positively on my favorite podcasts. If a game is on topic for weeks surrounding it's release, and is constantly being raved about, that means massively more to me than a 10 ever will, and by the same token when a game is regularly mentioned I'm more likely to check out a few reviews even if I would have otherwise dismissed the game.
I don't know if I'm alone in that, but if I'm not I'd like to read what the symposium-ers (Is there a word for one who contributes to a symposium? I'm having trouble finding an easy way to refer to these fine folks.) think might be the reason behind that (I've got an idea or two about why it's so in my case) or whether they place any importance on podcasts at all for that matter.
I look forward to the future postings.
December 18, 2008 at 11:01 PM
Anonymous said...
My favorite part of the above unreadable wall of text has to be everyone avoiding the obvious answer. People want review scores because any halfway decent writer wouldn't be writing about videogames for Christ's sake.
The job consists of putting a number on a review, you should have a 10 in 10 chance of getting this right before dedicating your time to fart/boob jokes on the screen captures, yet some still manage to fail. How often can you write "it depends on your tastes" (spoiler: every time) before you sit back, stroke your chin, and look at your legacy of awful reviews before just deciding to finish college/snap and start killing people?
"I hope there's an anecdote about the reviewers terribly interesting life in this one!" Do you recognize this quote? That's because it's never been said, ever. Fight through the tears, think of how valid an opinion on something you can't be bothered to finish is, and just give it a 7 already.
If someone wallpapered my house with this I would burn it down before reading it.
I feel that reviews are generally too eager to gush, mostly because they are written around the same time as the game itself.
It makes sense to do it that way, but think of all the cases where the hype has died down. When people have looked back at Bioshock and gone, “You know, maybe it wasn’t the most important game of all time” or Oblivion which went from “Oh my God this game is sooooo amazing” to a whopping about-face “Holy shit this game is terribly broken and emotionally sterile to its very core” It’s this weird nerd high that goes around. The same one that had everyone convinced for about a week that Star Wars Episode 2 was a good movie.
9/10s are handed out like candy on Halloween. Somewhere along the line it all just becomes a blur of hyperbole. Plus, and no offense, but most of the time the writing is so bad and obvious that one glance at “9/10″ and you can predict the details through intuition so what’s the point of reading on? Oh, you say it’s really really good but has small flaws in a few key areas that keep it from being perfect? Brilliant!
To tell the truth, the only people who really, genuinely care about reviews are the doofs who argue about it on the internet and are worried only about having to do damage control over a low score and thus protect whatever sliver of self-worth they get from that. I’d say far more “real” people get their purchasing decisions from personal recommendations than reviews.
I’d greatly prefer games to be “read” instead of “reviewed”. Something along the lines of a dissertation on how moving the Fallout series from an objective, detached camera to a subjective one has all but ruined the subversive atmosphere. (Which is true, btw) It’s just something you can’t sum up in a tidy score and does a much greater service to the people who read it.
Regarding scores on reviews, I often use them along with the reviews name to decide whether I’m interested in reading the text. The score is often an indicator of what to expect and to be honest it’s when that deviates from an average when I’d be interested in reading. I’ll definitely agree with Kieron’s point about hyped games being 9/10 and then losing points and others starting at 5/10 and gaining them. I use that metric internally to decide whether to read. Learning what it is that would make a game lost those points or conversely gain them is a matter than can only be learnt from reading. Seeing the expected score can make even a small review a chore to read. I didn’t feel the need to read anything many reviews about GTA IVs 10/10s, 5/5s or A+s . I was expecting a gushing report about how everything was amazing and it might be flawless.
The other problem with reviews like those, is that coming back after the fact the reviews, to me, felt incorrect. Personally I’m not a massive fan of GTA IV. The opening moments and up to half way through I was with every reviewer but after that I felt the game lost its way, dragged on and stopped being fun. The only place I found this sentiment I felt was in the GFW podcast by Shawn Elliot.
Podcasts are where I get the majority of game opinion from. Those “what we’ve been playing” sections are great for getting a feel for a game and the reviewer. As I mentioned at the start the reviewer is key for me to start reading a review. Learning what things interests any of the reviewers and how their viewpoint compares to mine influences what I believe their score or text could mean to me. Review text though, would be an extension to the discussion given about a game on a podcast.
For example, learning how Ryan and Jeff of Giant Bomb were put off immediately by the first person perspective of Mirrors Edge meant that I knew I personally wouldn’t be able to agree with their assessment of the game as our views severely differ.
I use podcasts for learning about a game because I can do it while I’m doing other things, because there will be a discussion between people about the game and because you don’t need to put a score at the end to know what the person is thinking.
I was listening to Radio Lab a while ago. They had a great segment on emergent thought in ants and humans. Essentially, if you look at an ant colony, there is no leader. It's braindead. And yet if you get a bunch of ants together, they're suddenly smart. Individually they are stupid, blind insects. Together they do infinitely complex tasks that most humans couldn't do.
The same thing, the program explains, happens when you get a bunch of humans together. If you get enough people in one room, ask them to guess how much an ox weighs or how many jellybeans are in a jar you'll find some thing funny; The average of a lot of stupid guesses is more accurate than any educated guess made by an individual.
So in terms of game reviews, the group may be far more knowledgeable than any single person. I'm not just talking about journalistic groupthink or the metacritic score either; I'm talking about kids yammering about the DS on the playground, about the recommendations of minimum wage EBGames employees and about the chatter of people that haven't owned a console since Atari. The value of a game is not determined on a microcosmic level but a macrocosmic one and at the end of the day we shouldn't worry about a review by in isolation, we should worry about the discourse we're having and what influence that discourse has on the group as a whole.
So despite how much I love reading thought-provoking reviews, we need to realize that this is not a solipsistic medium - this is a choir of opinions leading ultimately to a conclusion. If we're going to get anywhere, we need to be conscious of that.
As a total wannabe, writer, this conversation is inspiring on several levels.
I think most of the issues involved with reviewing video games come back to the idea of reviews being treated, by dev. studios, game publishers, or magazines, as total commodities.
The utter lack of transparency that most outlets have regarding the hoops they have to jump through to obtain exclusive reviews contributes to a severe culture of mistrust towards those reviews among those informed about the review process. Sure, we know it's an unfortunate necessity, but that doesn't make me wonder any less if those early scores are valid or simple money-makers.
Ultimately, though, the scoring process is just one symptom of an entire culture centered on game reviews that needs to be overhauled. So what if you take a 10/10 score off a game that doesn't actually deserve it, but got one based on the fact that's is a hyped AAA title? Does that mean that the review is suddenly more valid, that the flaws glossed over to give it a 10 are still not ignored in the text?
I don't think the problem ultimately lies in what scores games are given but rather in what appears to me, as a semi-well informed reader, to be the process behind arriving at those scores.
I know this reads like a long-winded way to say "scores aren't the problem, you guys just write shitty reviews," but that's really not what I'm trying to get at. It just seems to me that, for whatever reason, a lot of games are not judged equally. This is touched on by you guys in your discussion about having preconceptions before you begin reviewing, but it seems to be way more prevalent in reviews than the symposium makes it out to be.
Stephen Totilo, MTV News Multiplayer: Have any of you come across gamers who won't buy a game they were curious about because you gave it a 7 and not a 9?
Well, speaking as someone who doesn't buy many games at $60 (the theoretical target audience of reviews), frankly, if a game I'm curious about gets a 7, I'm not gonna buy it for $60. Instead, I would wait six to twelve months, buy it off the bargain shelf at EB or Future Shop for $20 or $30. Crackdown and Assassin's Creed both had flaws that would probably bother me at $60 that I really didn't care about at $30, because I still felt I was getting my money's worth.
I think that's really what I look for in a review: am I going to get my money's worth? And in a lot of cases, I approach it from the perspective of how much gameplay I'm going to get out of it. Portal was one of the best three-hour gaming experiences I've ever had, but if I'd paid $60 for that alone, I might have felt ripped off; for $15 or as part of the Orange Box, I think I actually got more than I paid for, because of the fantastic writing that backed up the solid gameplay mechanics. At the other end of the scale, ESPN NHL 2K5, Burnout 3, Guitar Hero III, and Diablo II are all games that I've sunk hundreds of hours into at various times over the years (to say nothing of the almost two years I lost to WoW), and one of those was $30 off the bat. I'm more inclined to pay $60 for something that I either know is going to last me a long time, or that I can reasonably assume I'm going to enjoy immensely, based on past experience with the genre or developer*. Games that get a 7 are games that I may be interested in trying out because they sound interesting (or because they're from a genre/developer I know/like/trust), but because of my student budget, I'm just not willing to invest in as an early adopter for fear of disappointment.
It occurs to me to wonder now if that makes me part of the problem: I tend to avoid online forums like the plague, having passed the age of 14 many years ago, but maybe I put more stock into a review score than it really warrants. That being said, in recent months, I've gotten into a number of gaming podcasts (Giant Bombcast, 1UP Yours, 1UP FM, GFW Radio/LAN Party, Evil Avatar Radio/In-Game Chat, Australian Gamer Show), and the more in-depth discussions that occur there have changed my perspective a bit on review scores and what I might like to invest my minimal funds into; besides, it's easier to listen to a podcast on the train than set aside fifteen minutes to read three or four reviews, and you often get more fleshed-out, "real" perspectives by way of interaction in the process. To borrow Totilo's own example, Too Human got excoriated by the press for its repetitiveness, awkward controls, unfulfilled story concept, and numerous gameplay delays/complications (the 20-second resurrection animation and the menu system) but the guys of EAR, as it was known at the time, all loved it for its co-op aspect and loot-whore reward system, and it occurred to me that if it was basically a sci-fi Diablo clone, I might like it, too, even if I have no one to co-op with myself. Because of the differing opinions, I'm inclined to download a demo or wait for the price to hit $30 before buying it, but I'm more open to the concept of a purchase than I might have been a year ago, which is something.
So to bring it back to the subject at hand, I wouldn't say I'd never touch a game if it scored less than an 8 or 7.5 or whatever, but I'd definitely be more wary of investing a lot of money into it for budgetary concerns, and that the "shit filter," as one person called it, does work to help me choose between games I'd buy for $60, games I'd buy for $30, and games I wouldn't buy at all. As long as it's understood that that's what the score is for, I think I'm fine with using it as a proxy for that sort of decision, in the most basic sense, but at the same time, nothing can replace an honest-to-goodness discussion, and I do think a lot of people tend to avoid that discussion because it might assault their preconceived notions (though that's no doubt a subject for another day, and one that's far from gaming-exclusive).
* - Ignored in all this discussion, for obvious reasons, is the store-packaged deal: I got Doom 3, Knights of the Old Republic, and a SoundBlaster Audigy XL with Hitman 2, all for $70, which is what Doom 3 would have cost by itself in Canada at the time, all for being one of the first ten people into Future Shop that day. KotOR turned out to be the winner of that four-item package, but how was I to know that at the time? I went there to buy Doom.
In the postscript, he explains how a four star review was changed to a three star review by the editors of Rolling Stone, but his tone makes it sound as if this was a rare exception. At least as late as 2002, reviewers were submitting their own numerical scores for that publication. So, uh, there you go.
Nice work, just reading through it now and may comment later. Just a note though, could you add some anchors to the ** you've put in so I can bookmark my position as I dip in and out of this.
Well, now I've read this pretty much top to bottom.
I have to say that I right out disagree with N'Gai's position that scores are harming games media, and I've decided that I also disagree with the degree to which certain people (particularly Dan) are getting worked up about this issue as a whole.
Frankly, I rarely read reviews anymore. I pick and choose posts to read off of Joystiq and Giant Bomb, subscribe to a few podcasts (Joystiq, LAN Party, Retronauts, skim 1Up FM and I listen to episodes of 1Up Yours until Garnett gets liquored up and goes apeshit yet again). I might even download a demo, but that's getting pretty rare as well.
I feel like I'm still getting a reasonably good feel for what is going to be good and what isn't. I can't exactly explain how, but there's an overall tone of popular discourse aside from whatever might be happening on "popular message boards" (which I don't read) that fills me in on what I should be paying attention to.
I've also found that in lieu of relying on scores, I just "buy-in" at a price point that's more appropriate to the level of risk that I'm willing to take on a game. If I'm not sure I'm going to like a game, I buy it when it's available for twenty dollars (Army of Two). If it's a sure thing (in my mind, a good example was Rock Band 2), then I'll shop around, but I'll still pick it up at launch.
I wonder if the elephant in the room is that no one reads these reviews anymore anyway, except juveniles, marketing people, and other videogame reviewers.
I couldn't care less what you reviewers are doing, I'm not relying on you to tell me what's good and what isn't in your reviews, I'm getting it from everything else you do and say.
Great discussion so far. Not sure if you meant for this to turn into a sprawling discussion about review scores in general, but I think it needed to, at least initially. I'd love to see a panel come from this, even though there's no appropriate venue for that right now.
To add my own 2 cents, I don't think you guys will ever find what you're looking for. Several different notions of what functions these scores perform emerged throughout the discussion, but the one thing agreed upon is that people need a bottom line. Consequently, publications need to provide a bottom line. The only question is, how can this be justified?
It seems like reviewers want to reconcile meaningful scores with thoughtful review (and and an audience who wants thoughtful review) within the context of a feasible business model. Do we have any proof that such a notion is anything more than modern day alchemy?
I'm personally a fan of many of the people involved in this symposium, and I appreciate all the thought you put into your reviews. But it's been made clear by Metacritic, pressure from developers and publishers, and responses from the enthusiast community that overwhelmingly, the most important part of your job is the score you assign to games. Truly shitty, but perhaps in this case, it would just be best to let sleeping dogs lie. You are paid to contribute your thoughts on an exciting medium. If people only care about the score, then they'll get exactly what they want.
I was wondering if you could get Jeff Green in on the next part of this symposium. I think he was serious about wanting to be a part of this on the GWJ podcast, Shawn (yes, I listen to way too many podcasts and know entirely too much about everyone on here.) I think it would be interesting to hear him chime in as well.
Having only read the first 3 so far... I figured I'd throw my 2 cents in... in point form, because no one could be bothered to read this anyway.
-scores suggest infalibily, in so far as they are set values (9 or 10 out of 10 is not debatably in its worth). People percieve numbers as facts. Letter scores are automatically converted to numerical representation in the brain. - because of the previous, the review gains exclusivity and authority. What should be an opinion becomes a fact: game x is 9/10, so how can 9/10 be 6/10? -a term paper is marked by a lecturer with a degree or three, an olympic gymnastics competition is judged by professionals; yet any tom, dick or harry can pass out a game score-> anyone can claim authority, and because the world of the internet is grounded in anonymity anyone can claim expert status. -Print represented an expert status in games. If you worked for CGW your talents and discression were vouched for by the magazine. On the internet everything is temporary and only valuated by how many clicks it could get, and controversy gets clicks.
Im rambling now... for clarification, I study Arts at Sydney University; majoring in political science, sociology and digital cultures. So while my bullshit may be bullshit, in RL i could at least back it up.
"a term paper is marked by a lecturer with a degree or three, an olympic gymnastics competition is judged by professionals"
and we must remember that "professionals" still have bias and personal opinion. A degree or job doesn't really make their insights more poignant or real.
So there's a question: What constitutes a professional in the gaming world?
Just a quick point. I like video game scoring and despite its problems I find them very useful.
Sure they are inflated by the various measures stated by the panelist which is why sources like metacritic are a great moderators.
I personally use game scores as a Table of Contents. You see I'm still going to purchase certain game whether the press gives them good scores or not. My own opinion is just as valid as theirs, in fact its more valid because they aren't me and likely have different tastes. But review scores allow me to discover games I wouldn't normally pick up.
Not having a score would mean that I'm not going to read your review for "Generic Game Name Here". However giving it a score shows me instantly that this may be something I might be interested in. I will then read your review, read other publications reviews and watch the video reviews.
Honestly if you didn't post a score I'd likely not visit your site. I am looking for a gauge on the products experience (as you saw it), and at that point, if I'm interested, i will read your opinions because that's where the real heart of the subject will lie.
Shawn, thank you for assembling such a concise yet varied discussion of this topic, which has been so ill-researched in the past. Thanks also to the contributors, whose interplay makes this feature diverse and thought-provoking.
That said, I find myself agreeing most with N'Gai: reviews do not need scores. When we look at the assignment of reviews as an art in its infancy and don't hold fast to established norms, it becomes clear that the advantages to eliminating numerical designations across all critical outlets are significant and plenty.
The issue of garnering exclusives or pleasing advertisers fades immediately; if there is no competition based on scores whatsoever, coverage will be given to whichever outlet the publisher deems most appropriate to reach the game's intended audience. This has the added benefit of encouraging the largely homogenized gaming press to specialize, be it by genre, platform, or whatever else divides the gaming public (Mitch Krpata's taxonomy of gamers comes to mind).
What's more, freeing reviewers from assigning scores allows them to be candid and straightforward, rather than sugarcoating their criticisms to fit an overly inflated score. The fun-to-read very high and very low reviews are evidence of this problem: in these situations, the reviewer doesn't have to please both his own conscience and the public, resulting in sharper opinions and better writing. The in-between reviews are littered with this conflict, and removing scores from them would solve this problem cleanly and efficiently. If the reviewer truly is conflicted about a game, let him say so, rather than trying to reconcile his beliefs into a number. In this way removing scores will improve game review writing, and give it a greater significance to the non-gaming public.
Of course, the implementation of such a system is incredibly difficult, as every sect of the business has become dependent on scores in some capacity. What is required is an outlet free of commercial obligations, so as not to be bound to please buyers instantly and unconditionally, to drop scores altogether. Theoretically, the improved writing quality will bring prestige to this maverick outlet, and others will emulate. It's a dream, yes, but I think one worth believing in.
What an excellent and thought-provoking discussion; I commend you all for your participation.
Yet in this entire discussion, there are two absences which I find troubling:
-The comments offered by the primary participants are all originating from the same basic perspective, that of a reviewer. While these insights are varied and interesting, I would have liked to see some representation from the other parties involved: a developer, a full-time PR guy, a common reader. What are those people looking for and how do they view review scores?
-Tom Chick (and others) mentioned the "context" under which he writes reviews. That context is obviously unique to every reviewer, as it is a culmination of all past experience and personal prejudice.
Yet there has been no mention in this entire discussion of the simple fact that the /reason/ a single review score is an innapropriate expression is that it is devoid of all context. Did the reviewer give that racing game a 7.0 because it was buggy or did they perhaps just prefer a different genre?
Since there is no way for the reader to determine /why/ a game is scored the way it is (short of reading the entire review), there is no way for them to evaluate on score basis alone whether or not that number holds any real meaning.
(Case in point: I saw a Metacritic score for Assassin's Creed and elected to hold off my purchase; later I read some reviews and found that while the scores were low, the positives of the game were right down my alley and the negatives were for things I didn't really care about.)
Additionally, it is clear from all the responses on this issue, that reviewers clearly each have a different focus when it comes to actually giving a review score.
So my question for everyone is this:
What can be done to to create a consistent & concise rating system from which all parties can derive meaning? Or is it futile to even try?
[Disclaimer: I realized that after finishing writing this comment, I appear very score focused, but this was not the intent. I am merely concerned with certain trends very prevalent in games reviewing. This post has nothing much to do with games criticism, which is a separate topic.]
I think a major problem with videogame reviews vs. reviews in another medium, such as film, is that there is too much consensus between different outlets' videogame reviews for me to take them seriously.
There are few movies that garner truly universal acclaim or get consistent scores for nearly every major critic yet in games, it seems to be the rule. Amongst film reviews, I can almost always find a reviewer that liked or disliked a movie for many of the same reasons I did, but with videogames, this is quite often not the case. I don't need or necessarily want to just read someone's restating of my own opinion, but I find it very strange that not one professional reviewer exists who feels even remotely like I do about a game.
For example, a quick glance at the review scores for Little Big Planet reveals that no critic among those aggregated on metacritic gave the game anything lower than an 8/10 or equivalent. While I generally object to metacritic style aggregation, this seems absurd.
Regarding LBP, there a quite a few people on internet message boards that I frequent as well as people in my group of friends who play videogames, who have expressed dissatisfaction with the game similar to mine: the core game isn't that interesting, and no amount of level creation or level editing will amend that, and yet despite being a commonly encountered criticism of the game, no professional reviewer seemed to have these issues.
It's things like that that make reviews so suspect at times. If the gaming community has a full range of hypothetical scores they might award a game, why do professional game reviewers' opinions not reflect anywhere near this range, especially for certain, very high profile games?
Papapishu makes an interesting point, but it still implies that aggregation of opinion will yield some kind of scientific fact, which for all his interesting points about group behavior, is absurd, no matter how it is stated.
Sometimes reviews truly seem to encompass the full range of opinion; divisive titles like Mirror's Edge or Assassin's Creed being such games, but when that sort of range is the exception, how can I possibly rely on reviewers?
It seems to me that the problem in reviewing games is the inherent subjectivity of the game's quality. I know that there are certain games that, for one reason or another, I can't get enough of, but if I think about them critically they wouldn't get a "10" from any sane reviewer. I have probably logged more hours in Puzzle Quest than any other game, but it does not have nearly as much time, art or storytelling put into it as, say, Half-Life 2, so which one is the "better" game? I would say that Half-Life 2 is the better game in terms of gameplay, but Puzzle Quest is the better game in terms of replay-ability and mindlessness. Shawn Elliott can not say enough good Company of Heroes and, while I respect his opinion, I know that CoH is not my kind of game so his opinion is moot. Because of this subjectivity reviews must rely on seemingly quantifiable aspects of the game. Graphics, sound, controls, etc. The problem is that, in the end, these aspects don't make or break the game. As gaming technology gets better it is expected that the graphics will improve, but if the antialiasing is a little janky, if the fingers of a model aren't articulated, if the blood splatter isn't realistic, does it actually affect how much fun the game will be? The answer should be no, but there are gamers (and reviewers) who take minor graphical flaws as personal affronts. Clipping is tantamount to punching their dog.
I really enjoyed the conversation in a GFW podcasts between Shawn Elliott and Sean Malloy about BioShock. So rarely do gamers get an intelligent conversation about a game, rather than a one-sided review that usually boils down the graphics rather than the content. Shawn and Sean discussed things that I had not thought about while playing the game and, on the next run through, I was more aware of the underlying themes and perhaps even a bit more critical of the binary good/evil aspect of the game.
I think what should be important to gamers and reviewers is how much fun, thought inspiring, or interesting a game truly is and not any so-called "niggles" that can be found. Attaching numbers to a review is arbitrary and cannot be considered a valid reflection or substitution of the review's content. Scores are the TV dinners of game journalism, they may satiate a basic desire, but they offer little real nourishment.
I wrote a blog post influenced somewhat by this discussion i think its pretty interesting but who knows maybe it suck...check it out http://www.1up.com/do/blogEntry?bId=8976319&publicUserId=5753283
I treat scores as indicators of the value (to me) of the reviews they accompany, and not as a means of judging games. They could be replaced perfectly effectively with a recommendation system, showing me the reviews I might like to read, without reference to a number.
When reading a magazine, I look at the scores first and will jump to the text for the reviews with a high score, or with a score that differs from my expectations. This helps to focus my attention on reviews that might have something new to tell me about a game. Scores of 10/10 for a game I am certain to buy, or 6/10 for a game in a genre I don't usually play, will not catch my eye.
I've got a bit of a dissenting opinion as far as this Symposium goes. I can appreciate the overarching focus, but I think the methods are somewhat lacking. I've posted some of my thoughts on my Wordpress blog; rather than reproducing the entire thing here in the comments, I think it would be best to just provide the link. I'm curious to hear what some of you participants think in particular.
I was going to write a long comment but it was fillied with stuff everyone here already knows. Instead I'll just say thanks for the virtual symposium. I found it far more interesting to read this discussion than almost any other videogame related article in recent memory.
Someone call Leo Laporte and get advice on a virtualized podcast similar to This week in Tech but starring these veteran's of the gaming world and I'd listen religiously. I might even pay a tithe.
That was a very interesting discussion. I love peeking into the mind of a games journalist.
At the age of 40 and still a gamer (although I don't have the time to devote to it that I'd like) I will say review scores themselves are less important to me but at the same time I want to see them. If for no other reason than sometimes it gives me a reason to dig down into a review to see why it got this score.
Case in point was Tom Chick's review of Red Alert 3 on Crispy Gamer. I wasn't even looking for the review but came across it and decided to read it because I was curious about the "Fry It" score. It seemed other sites and magazines were rating it pretty high from just casually looking around. So I read his review and very much enjoyed the depth Tom got into and not holding any punches on his opinions of the game and its flaws as he saw it. I can't judge the review other than to say I think he backed up his score. I think that is the best and ideal relationship between a score and body of a review. Can you back it up. Especially given a scale like Crispy Gamer's. A reviewer can't fudge things. There is an obvious demarcation between Buy It, Try it, Fry it. That is a scale that works perfectly for me because I feel it makes me want to read a review to see why it got that grade. I've gotten to where I hate the 10 point or 100 point scale. Even the 5 point. I think 1UP made a great decision to go to letter grades. I prefer that or Crispy Gamers scale to anything else out there.
In the end though, who cares about the score. I mean really, as critics and journalist, what I want most from you is the text. The in depth review of a game either as buying guide or as a critique. They can either be combined or seperate things.
What I would love to see is an outlet that does an in depth critique and investigation into a game after a game has come out. After the review/buying-guide. Pull no punches. Take the time to really dig in. I know of no outlet that truly does that. It is almost as if a game is forgotten about after it is published and reviewed. The only post mortems out there of a game I know of are the ones that consist of an interview with a developer after the release or a write up the dev after release. While both enjoyable as a peek into the mind of the dev, they don't serve much of a journalistic or critical importance. Something I'm looking for more and more as I get older.
Another case in point but a bit off topic. Nothing to do with reviews but about depth. Geoff Keighley's Behind the Games series he did for Gamespot years back. I enjoyed those to no end and miss those very much. Maybe the tight PR machine that surrounds games now would never allow a reporter to get that deep into the workings of a developer but that is something I yearn for in games journalism today but haven't found since then.
What a brilliant read. I've shown this to all of my friends that are involved in gaming one way or another, and they agreed that this is absolutely amazing. I don't have to contribute anything to this symposium myself though.
What struck me as odd was that I read everything in one session. I am not used to reading anything on my computer screen, but I was completely hooked.
I wish you the best of luck with your endeavours at 2K Boston.
January 15, 2009 at 7:11 PM
Anonymous said...
Greeting! What do you think of wow gold? I always lack of wow gold to buy some great item for my char before the wow patch 3.1 and I need to do wow power leveling for the cheapest wow gold or I have to search cheap wow gold on google search. The world changed since the 3.1 patch released. Short of buy wow gold for me wasn't so high frequency now. And I’m curious why this situation happened? The items became cheap? Or we can farm wow gold much easy? Hope someone can give me an answer.
Do you know Archlord gold? I like it. My brother often go to the internet bar to buy Archlord money and play it. After school, He likes playing games using these archlord online Gold with his friend. I do not like to play it. Because I think that it not only costs much money but also spend much time. One day, he give me many cheap Archlord gold and play the game with me. I came to the bar following him and found buy Archlord gold was so cheap. After that, I also go to play game with him.
Do you know cabal online alz? I like it. My brother often go to the internet bar to buy cabal alz and play it. After school, He likes playing games using these cabal gold with his friend. I do not like to play it. Because I think that it not only costs much money but also spend much time. One day, he give me many cabal money and play the game with me. I came to the bar following him and found buy cabal alz was so cheap. After that, I also go to play game with him.
April 20, 2009 at 10:04 PM
Anonymous said...
Do you want to know the magic of online games, and here you can get more aoc gold. Do you want to have a try? Come on and conan gold can make you happy.You can change a lot age of conan gold for play games. And you will enjoy yourself at the sametime. Playing online games can make much cheap aoc gold. Come and have a try. You will feel a different feeling. And you can aoc money do what you want to do in the online game. Do you want to know the magic of online games, and here you can get more FFXI Gil. Do you want to have a try? Come on and FFXI gold can make you happy. You can change a lot Final Fantasy XI gold for play games. Playing online games can make much buy FFXI Gil. And you will be happy at the same time. And you can use the cheap Final Fantasy XI Gold do what you want to do in the online game.
I like holic gold very much. Since I entered into this game, I learnt skills to earn holic money. Thanks to holic online gold let me know a lot of friends. It is my habit to buy holic online money, and I get some cheap holic gold from my friends and Internet. I like to play last chaos, because I like its name, also I like last chaos gold. My friend told me that she would buy lastchaos gold for me, and I was so happy. I do not like to go shopping, because it always spends a lot of money, but I never hesitate to buy last chaos gold. You can buy cheap lastchaos gold; it is so easy and convenient. I believe the future of lastchaos money is bright.
April 27, 2009 at 9:21 PM
Anonymous said...
To buy rappelz rupees , it is not the aim of the rappelz. It is the game which brings a lot of happiness to me. We buy the rappelz gold together. The beautiful story, we can not be forgotten buy rupees . You will not regret to have rappelz money . So I will continue to have cheap rappelz rupees , it is a beautiful fairy tale, it looks like my life.
On account of I have rohan crone , I am not afraid of the lonely. At that time I do not know the meaning of it. Later I buy rohan gold . And every time, I have no rohan online crone , I will feel very anxious. Why I buy rohan online gold , please it gives me such a warm place. At the moment, I have rohan money .
Their studio is often overlooked, however, when discussing long lasting and well made games, to the great detriment of the MMO industry. With both games, it would be a true tragedy to ignore the story and fun that are available at a cost that is both reasonable and affordable compared to most other subscription MMOs. You will like the game and wakfu kama, I think.
April 28, 2009 at 1:30 AM
Anonymous said...
polo ralph lauren RED! Series of design criteria - stylish, simple, self-cultivation. For the first time in 2009 in New York Fashion Week debut, RED! Series has a new Polo Shirts On Sale series from the traditional burberry polos LOGO, LOGO traditional crocodile logo at the Ralph Lauren Polo Shirts letter above, and the new series of letters with the cheappolos logo in line to be between a red symbol of Burberry Polo Shirts.
Although many people criticize ed hardy clothing for having nothing but tattoo patterns, being filled with waste and useless things, it is widely acknowledged that ed hardy clothes draws the most attention of both common young people as well as many Hollywood welcomed stars for its punk ed hardy t-shirts rock styles. As a matter of fact, in such a world where all ed hardy shirts are much like each other, ed hardy mens creates yet another totally new global sensation in fashion crowd for ed hardy womens fancy design. An increasing number of people are charmed by the magic of amazing tattoo and impressive color of ed hardy sunglasses.
Our company is always making effort on wholesale and retail of Air Jordan Shoes, NFL Jerseys, MBT Shoes, Nike TN and Puma Shoes Men . I would like to inform everyone likes sports apparels, we will offer discount up to 50% discount for large order, especially in NFL Jerseys, if you like these, please don’t hesitate to tell you friend, welcome group purchase in our online store.
Sean Michael Elliott (born February 2, 1968 in Tucson, Arizona) is a retired basketball player of NBA.Elliott played basketball at the Institute in Tucson Cholla, and the University of Arizona under the tutelage of Lute Olson. sport fishing fishing guide
Thanks, it is very nice and helpful .i agree with you very much ! but I have something to share with you too ! you can login our website to look around , all our products are top quality and low prices . would you like to buy something ? ok follow me ! better choice better life ! do not leave something to regret in yourfuture! NFL Jerseys tn chaussures chaussures puma Tn Requin Cheap Polo Shirts better choice better life !Thanks for your post , it is very nice and helpful .see you !
Thanks, it is very nice and helpful .i agree with you very much ! but I have something to share with you too ! you can login our website to look around , all our products are top quality and low prices . would you like to buy something ? ok follow me ! better choice better life ! do not leave something to regret in yourfuture! NFL Jerseys tn chaussures chaussures puma Tn Requin Cheap Polo Shirts chaussures pas cher better choice better life !Thanks for your post , it is very nice and helpful .see you !
I feel honored and privileged after reading your posts.You know,summer is coming! chaussures homme online .The chaussures homme nike will become fashionable soon.We offer chaussures homme nike,all kinds,all colors available.You can also find homme chaussure in our shop,including hommes chaussures,Nike Shox NZ,Nike Shox R4.Best wishes to you!
Guccio Gucci,son of an Italian trader founded the House of Gucci gucci boots outlet in Florence in 1906. Guccio, authentic gucci bags an extraordinary craftsman, started selling luxury goods in the 1920s. cheap gucci shoes In 1938, Gucci opened a boutique in Rome discount Gucci bags.Aldo expanded the company by opening more stores. newest gucci handbags In 1953, New York got its first Gucci store gucci handbags 2010 and by the late '60s, more such stores were launched in London, Paris and Tokyo. Being a proud owner of a watch Armani watches outlet is a wondrous thing that many take for granted. watch wholesale No one remembers back when things were simple and discount DG watches they had to call "time and temperature" to get an accurate time cheap Chanel watches. Now certain devices run on their own newest GUCCI watches with no batteries. Juicy couture handbags are among the brands Juicy couture which have gained a great repute over a longer period of time. The quality authentic Juicy bags and material used for their manufacturing is up to the mark Juicy handbags outlet and has no match. Thus with the holiday shopping season approaching rapidly tiffany 2010, as an investor, the best gift you may give yourself this year is not a ring from tiffany, but shares from Tiffany. With a fairly good fundamentals, tiffany outlet a strong presence over its competitors, discount tiffany charms and a favorable economic situation to feed upon, I would not be hesitant to take a short term risk cheap tiffany Bangles in locating some luck with. stylers are often available with free gifts, discount ghds or other incentives, which can sometimes on offer ghd outlet seems too good to be true, ghd then it probably is. There are many imitation cheap ghd flooding the market , so you need to keep ghd straighteners your wits about you. wholesale edhardy t shirt Whether you like the colour or want to do your bit for charity, Moncler T-shirtsvery popular when they are available. jumpman 23
Since building my Asteroids arcade cabinet replica/ television cabinet, My videogame collection has grown almost double the size, I've completely filled the storage space in the cabinet and now there are boxes, consoles and loose carts all over the floor. wholesale caps
Air max chaussures : http://www.air-max-chaussures.com/ Air max 1 chaussures : http://www.air-max-chaussures.com/air-max-1-c-86.html Air max 90 chaussures: http://www.air-max-chaussures.com/air-max-90-c-89.html Air max light chaussures : http://www.air-max-chaussures.com/air-max-light-c-94.html Air pegasus 89 chaussures: http://www.air-max-chaussures.com/air-pegasus-89-c-95.html Air 180 chaussures: http://www.air-max-chaussures.com/air-180-chaussures-c-96.html Air structure triax 91 chaussures: http://www.air-max-chaussures.com/air-structure-triax-91-c-91.html Air classic BW chaussures: http://www.air-max-chaussures.com/air-classic-bw-c-97.html Air Tailwind 92 chaussures: http://www.air-max-chaussures.com/air-tailwind-92-c-93.html Air max 93 chaussures http://www.air-max-chaussures.com/air-max-93-c-85.html Air max 95 chaussures http://www.air-max-chaussures.com/air-max-95-c-99.html Air max 97 chaussures http://www.air-max-chaussures.com/air-max-97-c-101.html Air max tn chaussures http://www.air-max-chaussures.com/air-max-tn-c-103.html Air max ltd chaussures http://www.air-max-chaussures.com/air-max-ltd-c-108.html Air max 360 chaussures http://www.air-max-chaussures.com/air-max-360-c-112.html nice work.
This blog is very useful for me. the content is full of meaning and the blog template is so cool. What's more, I am deeply influenced by this great article.
My favorite brand is ed hardy wholesale, and I always want this brand of wholesale handbags. And its T-shirts are very special too. I really hope I can wear its wholesale t-shirtsto go to school and go shopping with my friends, and it will be awesome. And most of my friends like the Ed Hardy style too. However, this brand is too expensive for me to buy. An Ed Hardy Nike wholesale cost nearly 250 dollars. It is almost a cost of my living of mouth. I am only an ordinary student who has no trust fund; no job and no income that I can’t afford to pay such high price for its sunglasses wholesaleor wholesale ghd. So, I won’t buy the Ed Hardy brand series at least for now.
Every woman knows that you can never own too many pairs of wholesale CL high heel sandals. And high heels represent the ultimate in shoe fashion for many women. However, it's just a question of finding the right style for you--there's a pump for everyone. For example, a pair of basic black pumps--with round or pointed toes--is a must-have basic for every woman's wholesale LV handbags. As for finding a comfortable pair of cheap Nike sport shoes, I can testify that this is indeed possible. I've also bought two pairs of cheap Ed Hardy sunglasses, though I always feared my feet would slip out of them and sprain an ankle. wholesale Rolex watches are my pump of choice, because the straps are adjustable--and, in a good shoe, the straps are also padded. I like the "breathing room" of a sandal, even though the look of a pointed toe pump is very stylish.
2010 the best wholesale market 1. About Ed Hardy t shirt wholesale Why have wholesale edhardy t shirt become so popular among the rich and famous? The idea of clothing designs all coming from tattoos are in and of itself, a great idea. This is taking unique artwork and placing it on great, fashionable clothing. 2. How to Curl Hair With GHD Straighteners After buying your ghd in the GHD hair straightener wholesale, you no longer need a separate curling iron. Style your hair sleek and straight or create fabulous curls whenever you feel like it. 3. Air Jordans are Not Just an Ordinary Shoe Every kid wants to be like Michael Jordan,it is not a problem. The wholesale air jordan shoes priced at over $100, only the spoiled kids could parade these exclusive shoes around school. 4. Have you ever had any Louis Vuitton products? If no, you will find the answer when you try on them! It doesn't matter; you can solve this problem by searching our b2cwholesale, discount LV handbag is just hot selling! 5.Vibram Five Fingers Running Here cheap five finger shoes now, you can find the cheap professional shoes from our site, and we believe that you will satisfied with the high quality shoes and the lower price!
2010 the best wholesale market 1. About Ed Hardy t shirt wholesale Why have wholesale edhardy t shirt become so popular among the rich and famous? The idea of clothing designs all coming from tattoos are in and of itself, a great idea. This is taking unique artwork and placing it on great, fashionable clothing. 2. How to Curl Hair With GHD Straighteners After buying your ghd in the GHD hair straightener wholesale, you no longer need a separate curling iron. Style your hair sleek and straight or create fabulous curls whenever you feel like it. 3. Air Jordans are Not Just an Ordinary Shoe Every kid wants to be like Michael Jordan,it is not a problem. The wholesale air jordan shoes priced at over $100, only the spoiled kids could parade these exclusive shoes around school. 4. Have you ever had any Louis Vuitton products? If no, you will find the answer when you try on them! It doesn't matter; you can solve this problem by searching our b2cwholesale, discount LV handbag is just hot selling! 5.Vibram Five Fingers Running Here cheap five finger shoes now, you can find the cheap professional shoes from our site, and we believe that you will satisfied with the high quality shoes and the lower price!
Introduction Are reviews primarily a consumer guide, or should they serve another purpose? Do review scores deter intelligent discussion of videogames? Is the presence or absence of a review score the only difference between a reviewer and a critic? What is the role of the reviewer when the Internet is democratizing published opinion? How should reviews and reviewers evolve in light of the emergence and growth of Flash games, small games, indie games and user-generated games?
These questions and more were on the mind of N'Gai Croal, John Davison and Shawn Elliott last summer when they decided to expand their conversation to a number of noted reviewers, writers, bloggers and journalists for a published email symposium on game reviews. (See below for the full list of participants.) The planned list of topics include Review Scores; Review Policy, Practice and Ethics; Reader Backlash; Reviews in the Age of Social media; Reviews in the Mainstream Media; Casual, Indie, and User-Generated Games; Reviews vs. Criticism; and Evolving the Review. Round 1's topic: Review Scores.
Shawn Elliott, 2K Boston: How much is on our minds before we begin playing any given game for review purposes? Will we imagine a range of probable scores that a heavily marketed, highly budgeted, and hugely anticipated game will get? What when the game is branded “budget” or is the work of a lesser-known, less-storied studio? If so, how closely have actual scores correlated with our assumptions?
Kieron Gillen, Rock, Paper, Shotgun: As others have said before—but Troy Goodfellow put most snappily, so I'm stealing his phrasing—the games press has a presentist/futurist bias. The vast majority of press coverage is for games that either aren't available, or are only just available. Even if we haven't seen or played the game personally, our peers will have. And we'll have seen comments threads full of people saying what *they think* of the edited information of the game we (and their PR) have presented. And with all that, when you throw a score out you know it's going to be read with those expectations in mind. When Eurogamer's Metal Gear Solid 4 review gave it an 8/10 there were 2000-post threads and actual death-threats. And Oli [Welsh], when he wrote that review, knew exactly what response he could expect. Games without the hype have lower expectations. I remember the attitude being crystallized by a comment I saw ages ago on Kotaku which stuck with me, when they linked to a B-game someone had 9/10ed: "It can't be any good, as I haven't heard of it". It's an ugly, but common, tautology.
You can't avoid knowing what the score is on that point, without becoming a true hermit. In terms of coloring your actual expectations of the game per se... well, unless someone's actually paying me to research a feature, I ignore 95% of previews. So when reviews come up, I try to review what's there rather than the hype... but that's going onto a whole different question.
Quick thought regarding the indie/AAA dichotomy, though: I often think that AAA-popular-sequels tend to start with 9/10 and lose marks, while games with less expectations start with 5/10 and have to gain them. And... oh, I'll shut up. More on this later, I suspect.
***
Leigh Alexander, Gamasutra/Sexy Videogameland/Variety: So, as far as preconceptions go, I just thought it worth noting that a game's marketing machine, whether through its fierceness or its clumsiness, would very much for like for us to have a preconception going into a review.
Unfortunately for them, they can't necessarily pick what impression they create. I like to think we react to the fashion in which we're being messaged, rather than devouring piecemeal the messaging itself. Or, most of us do.
So, I agree with Kieron that the right answer is "no preconception"—i.e, the reviewing process doesn't begin until you start playing the full version of the game, period. But sometimes I wonder whether background factors should be considered as context for a review. For example, for months a hyperbolic individual promises that his game will revolutionize ludology. Are we allowed (or, conversely, obligated?) to consider his lofty goals when evaluating the end result? If a company creates an "identity" for a game ahead of time, shouldn't that exemplify what the game is aiming to be, and shouldn't we try and consider whether or not it achieves it?
There's a line, I think, between making a prejudgment, and bringing with you a context within which to make an evaluation. Games are an industry and a culture, not a fragmented, compartmentalized list of disparate products, and rather than pretend we have no early opinions, I wonder if it's not beneficial to be prepared to bring that context—which also applies, perhaps to being aware of budgets, of team sizes, of other challenges?
***
Shawn Elliott, 2K Boston: Because I believe that self-enhancing, self-serving, egocentric biases are normal, and that people are prone to see themselves as being immune to the influences that move everybody else, I'll happily admit—along with Kieron—that I have preconceptions before playing. I'm human.
I'd argue that our preconceptions are active when we decide which games we want to review. That's not to suggest that, when given the choice, all critics go straight for the gravy (I've often volunteered to review games that I imagined would be interesting but not the best available). But what, if not a preconception of some sort, drives these decisions?
In addition, I believe that my assumptions are active as I play. For instance, I'm less likely to immediately doubt the wisdom of a given design choice in a Valve game than I am with the work of second-rate studios. An analogy: Say you're competing against someone with sorry win-loss stats in a strategy game. His opening moves seem odd, so you assume he's stupid. When his record is intimidating, you take the time to study his seemingly odd tactics until you're certain you're not missing something. In my mind, "the right answer" isn't a realistic answer.
Leigh, I have a problem with holding a loud developer to his hyperbolic promises (and it has nothing to do with the dozens of programmers, designers, producers, artists, and animators hanging their heads behind him): intentional fallacy. I'm interested in the degree to which game maker's games match their ambitions, but I wouldn't want to evaluate them on this basis. What New Critics wrote of poems seems sensible for games: "It is detached from the author at birth and goes about the world beyond his power to intend about it or control it. The poem belongs to the public."
Should we consider budgets and staff sizes? Certainly not when the critic's intent is strictly to inform consumer shopping sprees.
***
N’Gai Croal, Level Up/Newsweek: I’ve never liked assigning scores as part of any critical assessment, and the times I’ve had to do so in the past, it’s always been under duress. I started out as a journalist by writing movie reviews for my college paper, and none of the critics after whom I tried to pattern myself—Pauline Kael, J. Hoberman, Stanley Kauffmann, John Simon, Andrew Sarris, Armond White—used stars or points or thumbs. They didn’t provide you with any shortcuts or shorthand. You had to read what they wrote in its entirety in order to figure out what they thought. I said to myself, when I grow up, that’s the kind of critic that I want to be. So because I’m not obligated to dole out review scores in print or online, I only have two things on my mind when I start playing a game that I know I’m going to write about.
First, am I going to enjoy this game? In that sense, it’s not dissimilar from when I take in a movie. Or a TV show. Or a play. Or a book. Even when it’s a shared experience, playing a game is intensely personal, and no matter the developer’s pedigree, no matter the budget, I start each new title the same way: on the precipice between hope and fear. I hope that it will be good or great. I fear that it will be mediocre or worse. And as I give myself over to that series of firsts—the first image, the first sound, those first bits of gameplay, that first accomplishment—any and all external influences evaporate, leaving me only the thrum of my internal gauge, the one that tells me just how much I’m enjoying myself. I trust that gauge implicitly, and while external factors might influence precisely how I /articulate/ my opinion, I don’t believe it goes much beyond that.
Second, how much of this game am I going to be able to complete before my deadline? That’s very different from how I approach plays, television, theater or literature—I wouldn’t dream of critically assessing a piece of work from those media without having completed it. Why doesn’t that stop me from doing the same with videogames?
The explanation—or is it an excuse?—that I offer is that I don’t review games. We’ll get into this more in the Reviews vs. Criticism section of our symposium, but the way I see it, a reviewer answers the question, how well does this game work, but a critic answers the question, how does this game work? A reviewer helps consumers decide whether or not they should buy a game; a critic helps players think about a game that they’ve played—in its entirety /or/ in part—and that is the end of the spectrum where I believe my writing lies. (That’s also why, on a game by game basis, I don’t think I need to have completed a game to have some insights about it—but I do think that if I were advising someone on how to spend their money, I’d feel obligated to play most or all of the game.) Scores can serve as a valid form of shorthand for the work of the reviewer, but I’m not convinced that scores have much to offer the work of the critic.
***
Kieron Gillen, Rock, Paper, Shotgun: Leigh, I agree with Shawn. You can mention the hyped intention and mention whether it measures up—but that's not what you're rating. Marketing doesn't necessarily understand their games and what's interesting about it. And occasionally a game is fascinating despite what their creators were trying—Jim Rossignol loving the deeply buggy unpatched release of Boiling Point for its sheer constant surreality comes to mind as an extreme example of that.
N'Gai, it's far too early for me to do my You Don't Need To Complete A Game To Review It piece, I suspect. Methodology of reviews is a question all of itself.
***
Stephen Totilo, MTV News: I wonder why Shawn dragged me into this. I seldom write reviews. I don't put scores on games. My main gig's reporting, a.k.a journalism, a.k.a. the thing most people don't really mean when they want to talk about "games journalism" because the thing they really mean to muddle over and improve upon is what we're talking about here: games-reviewing. I'll give it a go, nonetheless! Scores, who are they for? What do they do?
The question we're answering is whether those who review games pick a number before writing a word. Kieron says the ideal reviewer would not; he and Leigh agree it's hard not to pick a figure already. Shawn's acknowledging the humanity of having preconceived notions but dodging his own question about whether that made him start with a number. But I guess it's hard in some ways to pick a figure at all when it's so unclear what the point of it is.
What does it mean to select -- prematurely or even at the "right" moment—a seven for a game? Or to see a game and, at first sight, have your gut gurgle that it’s a nine?
A review score number may be for the fans, a shopping guide metric that informs a purchase or justifies one already made. It may get used for the dastardly purpose of comparing a game to another—even though it never quite works to pit a 2008 sports game that got an eight against a 1998 role-playing game that got a nine, especially if neither is as good as Tetris. A numerical score might, in isolation, even indicate if a game's any good, but not always.
We're talking about arriving at a number, and, frankly, I don't know how you all do it. A decade ago I worked at a boxing magazine and sat in press row for many fights. Scoring vexed me then. I'd score rounds for my coverage on the "10-point must" system: 10 for the winner of the round, nine for the loser unless he got knocked down or really took a beating, which would dock him to an eight. In that system we see the Gillen-described method of scoring-by-reduction. We also saw the great gaming tradition of grade inflation. Give a judge (or a reporter aping the actions of the official judge) a 10-point scale and all kinds of psychology comes into play.
The other thing I saw at the fights—the thing that really stuck with me—was how hard it was to score any of it. Boxing matches aren't like Rocky fights. It's often hard to see who is winning or which fighter is doing the better work. Sometimes it's all boring or repetitious, but you still must score each three-minute round. Putting numbers on these things—and the official judges had to, in case it went the distance and, god forbid, the paying public needed to know who won—was a murky and unpleasant job. Try it some time. I'd root for the knockout, which would render scores moot and sweep any errors in numerical judgment away. The scorecards didn't matter then. Any scoring biases we had would be secret. The fallacy of putting a number on things would be dodged, and everyone would go home happy. No one would have to know that I gave a 10 to fighter B because I felt bad that he'd gotten beaten up for the three previous rounds or that I gave the wrong guy the first round because I bought into his pre-fight hype.
***
Robert Ashley, freelancer: I took a break from enthusiast press game reviews for a couple of years. What a fucking relief. No more death threats from insane superfans who think my evaluation of their favorite game is some kind of paid-for hit job by a shadowy corporate network. No more forcing myself to play through a 40-hour game in three days. No more tearing my hair out trying to avoid the clichéd language of a form of writing frozen in its awkward adolescence 15 years ago. Free to play whatever I wanted, I fell in love with games all over again. Hard.
Now that I'm back and picking up the occasional review, I simply refuse to engage in the bullshit that used to drive me insane. Review scores have one use: driving traffic from message boards and social networks to your site and giving those people an excuse to argue out their fan beefs in the comments section. I treat them as such.
I have no methodology for choosing a review score. I certainly don't think about it much. Your gut feeling (after either beating the game or the game beating you) is more accurate than whatever you might come up with after careful consideration. This is how the rest of the gaming community arrives at an opinion—and probably why so many people feel that critics are out of touch. When you sit at your computer, running down all the plusses and the minuses—technical issues, story concerns, lovable roughness, annoying roughness—you can end up talking yourself into a score that doesn't really represent your true reaction. You can't explain the magical pixie dust that made the empirically bad game good. You can't explain the soullessness and sterility that made the empirically good game bad. You let your stupid logical brain take the wheel and explain yourself into a lie.
When I say you, I mean me.
Anyway, I say be gutsy and honest with a score, and save your careful thinking for the text.
***
Jeff Gerstmann, Giant Bomb: Well, I won't deny that scores stir up message boards and social networks and such. But to claim that's the only reason they exist is a pretty narrow, jaded view. I think scores are primarily there to serve as shorthand for folks that won't or can't read the full review. They're meant to serve as part of the summary. A deck, a score, and, depending on your publication's review style, some pros and cons or whatever. They aren't rocket science, and were never really meant to be treated as such. The key is to not let the different ways that scores are misused get in the way of what you're trying to accomplish with your reviews. I don't care if the scores I give fit in with the rest of the industry on the review aggregator sites. I don't care if people infer the score to mean that I'm playing favorites because I'm obviously "TEH BIAS" or whatever. I care about the people out there who haven't been following a game from day one, and the people who haven't already pre-ordered the game and are just looking for validation. As soon as you start bending your review systems in order to cater to those extremist segments of the audience, you're getting away from the thing that reviews are designed to accomplish: assist average, everyday people in their purchasing decisions.
I say assist because we've reached a point where one review can't possibly work for every single person that reads it. The audience for video games is too widespread and varied now for reviewers to think that their review is the only one that matters, or that it will be able to directly state if a person should or shouldn't buy a game. This, more than anything, is what should be driving a change in the way games are reviewed, not a bunch of reviewers who are tired of all the weak-ass game review clichés that are still out there. Getting rid of scores because people who write reviews are tired of assigning them and dealing with the fanboy rage that invariably ensues hurts the consumers that actually use reviews for their intended purpose.
But to answer the core questions, I don't really think too much about scores when I'm playing a game. I attempt to go in feeling cautiously optimistic about the game in question, and as I'm playing, I think about text, and things in the game that need to be specifically called out. I start to think about the best way to mention those moments, and the best way to call out its flaws. At some point, all that text swirling around in my head starts to sound like a range of scores, so maybe around halfway through playing a game I start thinking a little more about the score. But it isn't until after the review is written that the score is actually assigned. The score is meant to sum up the text. If I've just written a review full of harsh criticisms, well, then that sounds like a pretty low score. Assigning a score and then attempting to justify it with text puts the cart before the horse.
Assuming a score (or range of scores) before actually playing the final game is pretty dangerous territory. Carefully controlled publisher-run demos usually paint a pretty rosy picture of a game, and games often don't live up to that. Case in point: every time I saw Mercenaries 2 prior to its release, I thought it looked awesome. The missions seemed smart, the co-op was fun, and it felt like a game that would offer a lot of variety. The final product turned out a collection of dopey missions that showcased the game's boneheaded AI, the co-op didn't make much sense, and a lot of the missions were pretty boring. I didn't review Mercs 2, but not letting pre-release exposure to a game color your review with overt disappointment or a sense of smug "I totally called it" satisfaction can get a bit tricky.
So I agree that, ideally, a reviewer should start with no preconceived notions about a game based on budget, hype, promises made by the developer, and so on. But at the end of the day, we're all human, and I'd expect that some form of disappointment over a game that fails to deliver on promises or excitement over a sequel that's turned out better than the last leaks into some of our reviews. The key is in owning up to that and presenting your reviews as informed opinions, rather than hiding behind the old paradigm of rigid objectivity.
***
Shawn Elliott, 2K Boston: I didn't mean to duck the question, Stephen, and I definitely don't start with a specific rating in mind. However, I'm sure that I have imagined ranges of scores that a given game would receive whether I or anyone else was to write the review. That's not to suggest that I once forced the square peg of a game to fit the round hole of my presumptions. I never did. Or I don't think I did. What I'm acknowledging is that, all the same, something was on my mind, both before I began and while I was playing. I think this is the case for every videogame critic. And while that something isn't necessarily decisive, it's nonetheless worth investigating.
I should also add that our predictions regarding meta-ratings and the reviews of other critics are on the mark more often than not. (In these instances, self-fulfilling prophecy isn't an issue.) Some companies are so confident of our ability to make these calls that they're willing to pay us for our input as consultants.
Jeff is correct in that sometimes PR-controlled preview demonstrations are smoke-and-mirrors magic shows. But what about when we're allowed to play near-complete code for prolonged periods? I'm not talking about performance issues—commenting on the framerate of an unfinished game is almost as pointless as it is for an Entertainment Weekly writer to assure her audience that King Kong may or may not appear in place of a green screen. Sometimes design, locked down years prior to a game's preview phase, is apparently dopey. Again, I have to emphasize that holding some assumptions in no way necessitates my maintaining them in the face of final evidence.
You also imply that an aversion to cliché shouldn't drive change in the way that we review games. I won't argue that cliché is the one and only reason to reconsider our habits, however, I count it among the many. The paragraphs on a game's graphics, sound, and so on in previews and reviews produce recognizably generic writing devoid of the discovery and perception that might make them worth reading. They are lazy in that they eliminate both the need to transition thoughts and to interpret a game as the complex product of interconnected components (instead of simply summarizing these parts).
Even worse is when the paragraphs that constitute a template are themselves composed of yet more methods of avoiding actual analysis. I mock the overuse of words such as compelling not because there is anything wrong with the words themselves but rather with the way that they're used to replace real explanation. We know that any guy in the game store can say he likes or doesn't like a game's graphics or story. We recognize that it's our responsibility as paid writers to say something more than "I like it" or "it's good." Replacing "like" and "good" with "compelling" isn't even trying.
***
John Davison, What they Play: If nothing else, review scores serve as the starting point of a discussion for readers. As Jeff says, they serve as a shorthand for those that have no interest in digging deeper than a fundamental thumbs up or thumbs down gauge of quality. I think we can all safely assume this, but back in my time at Ziff we experimented sufficiently that we got absolute, empirical proof.
Jeff Green and I spent a lot of time talking to Computer Gaming World readers, and trawling through our message boards to really try and put together the ultimate reviews section for the audience. We wanted to do something a bit different, but more than anything we wanted to acknowledge what a large group of our readers were telling us. That was, essentially, that "we're older" and "we're smarter" than the average gamer, so "treat us like that." They wanted longer, more considered think pieces about games, and it appeared, anecdotally at least, that review scores were not high on their list of priorities. They wanted, they said, to really understand what the reviewers were trying to convey. They wanted to really dig in.
So we gave them that. We took the scores off, and made the reviews longer. We actually went a step further, and tried to acknowledge the broader critical spectrum, and talk about what caused other reviews to express particularly positive or negative comments. It was our own little expression of idyllic critical idealism. A utopia of reviewing and we dreamt that it would spark enlightened and intelligent debate about specific qualities and opinion.
The reaction was spectacular. The readers really, really fucking HATED it. The most common complaint (I'm paraphrasing, but it was pretty consistent) was "How do I know what you think if you don't give it a score?" That and "you guys are retarded." We figured at first that it was simply a bit of culture shock and that it would wear off, but the negativity increased over time. After three months or so, we had to go back to putting a score out of five on the reviews just to stem the tide of vitriolic hatred.
On a separate note, I was speaking to someone recently who had some connection to Rolling Stone, and he told me that the reviewing process for albums there was that the critics only submit the text, but do not submit a score. The number of stars is assigned by the reviews editor based on the tone of the review. He was drunk at the time, so might have been talking out of his arse though. Does anyone know for sure if this is the case? Even if it's not true, it's certainly an interesting approach—and something I'd like to discuss in this context. If a reviewer is freed from thinking about assigning a score, but knows one will be applied later—would it necessitate a more disciplined approach to how thoughts are expressed? I know it would for me. But are we ready to relinquish that kind of control?
***
Robert Ashley, freelancer: I don't advocate putting an end to scoring. Scores seem to be the one thing that today's online audience can easily form a conversation around, and I think, ideally, a review should be like a conversation between reader and critic. I just think the incredible seriousness surrounding scores (born in no small part, I'm guessing, from the fact that retailers stock their shelves based on review scores, ratcheting up pressure on critics to treat scores like jury sentences) is irritating. Handing out a mediocre score to a mega-hyped game can brand you a heretic (a Crispin Boyer, if you will) when you're just trying to be honest about your reaction.
***
Stephen Totilo, MTV News Multiplayer: Who is actually upset about review scores?
Offended publishers who wanted a 9? Stressed developers whose bonuses depend or ability to get another deal require them to get at least an 8? Superfans whose feelings you hurt by giving the game they are going to buy anyway a 7?
Before we do any more debating about the merits of putting a number on a score I want to know who cares.
Have any of you come across gamers who won't buy a game they were curious about because you gave it a 7 and not a 9? And, if so, would they have made a different decision if your review didn't include a score? Was it review scores that did in Too Human or put Wii Music slow out the gate?
Do scores ever really hurt or help games? Or are we just debating the best way to describe a game's quality, be it through numbers, words or faces in various stages of excitement?
***
Leigh Alexander, Gamasutra/Sexy Videogameland/Variety: Backing Stephen on this. Scores do stress the reviewer, but the stress doesn't come from any source that really counts—excepting maybe the anxiety I get sometimes knowing people's jobs depend on, say, Metacritic, and hoping that I was as thorough and fair as I could possibly be.
Like some others who've chimed in, I write the text first, and then see what score the text supports. In a way, I'm not assigning the number as a be-all measure of the game itself, but as a shorthand for my evaluation, and I think that's the function for which scores are most useful—as Jeff said, a single value that makes sense from a macro viewpoint for people who don't read text. There are a lot of those people (which makes me wonder if we shouldn't be trying to create more accessible, readable text, but that's probably a whole 'nother issue).
Finally, because I talked about preconceptions, I just wanted to clarify that while I may sometimes go in with ideas, hopes, dare I say biases, I never go in with a number in mind. The number is the last thing I come up with.
***
Dan "Shoe" Hsu, Sore Thumbs: To Stephen: Yes, yes and yes. They all care. But in the end, that review and score aren’t for them anyways -- they’re for your “normal” audience—so it really shouldn’t matter how bunched up their panties get.
At EGM, we’ve had plenty of readers who told us they would not even consider a game purchase if the reviews didn’t average a certain score they had set in their minds. Now, this “certain score” is usually a moving target—higher if it’s a game that reader wasn’t originally interested in, lower if he already had that title on his Amazon wishlist. This overreliance and faith in this one rating (scored by someone you probably don’t know intimately well) may seem silly, but I’m absolutely with Jeff and John on this: Despite complaints from a vocal minority, the vast majority of readers really want that number, letter, or direction the thumb’s pointing. It’s ingrained in society and it’s pointless and stubborn to fight it. People don’t always have time to read a 2000-word, well-crafted review to get inside the brain of the reviewer. For most folks in this short-attention-span world, that “4 out of 10” usually says more than enough.
Are we off-topic here, by the way? To answer question #2 above, yes, I sometimes change my score after I write the text. I do it like Robert initially: I score with my gut. But then while writing a review, I get to reflect upon my play time, think back to my progress 10, 20, or 30 hours ago, check back on my notes from last week, etc....and then I might adjust my score (usually by one increment up or down) based on my experience in its entirety. This may seem obvious—to score the whole game, not just how it finishes -- but I also have that same short-attention-span problem. And this helps me keep too much emotion out of the process, too, so a game’s high-note finish doesn’t unduly inflate the score.
***
Francesca Reyes, OXM: I can’t tell you how much I personally hate coming up with scores. It’s putting a quantitative label on something that’s qualitative. But that said, it’s a necessary and completely understandable function of game reviews in enthusiast pubs. (Hell, when I read movie reviews or book reviews, I do the same thing—look at the score to justify whatever half-ass, cobbled together pre-opinion I may or may not publicly admit to having.) The reviews I’m used to writing aren’t criticism in the sense that you’re a “reader” and you’re “reading” a game like you would text, or a movement, or even a movie—they’re practical forms of consumer advice. So, I agree with Jeff in that we’re here to inform our audience if a game is worth X amount of money based on whether it delivers, and to what extent, on its back-of-the-box promises, as well as how it handles as a game. And that demands a score, really. A signpost for what the review contains.
I’ve been lucky that all the pubs I’ve worked for fall in the realm of “enthusiast,” so in a lot of ways, you’re writing for gamers like yourself when you score a product. You’re also using a set of established criteria, depending on your publication’s ranking scale. If a 7 in a certain magazine means “average,” while a 5/5 stars in another means “must-have,” etc., this is what you’re working off of.
I remember writing a review of one game across multiple publications. One had a 100-point scale, another had a four-star scale, and yet another had something else. The thing that sticks out in my mind is that the 100 point one was for Ultra Gameplayers and the star-scale was Next Generation. UGP was your regular “review as a gamer.” Next Gen was “review for innovation and uniqueness, like ‘does this push the boundaries of its genre, etc.?’”. One game, reviewed against two very different sets of criteria. Interesting contrast.
So yeah, it’s totally ideal and utopian to think that you can sum up a game in a one-word or one-sentence definition from a pre-existing list of rankings, but this ain’t science. It’s voodoo magic in a lot of ways, no matter how hard we try to justify what the numbers, letters, or stars mean.
I try not to go into a review with a preconceived score in my head, but like Shawn said early in the thread—the result of months of pre-release hype or non-hype may or may not play a role in expectations from the reviewer and the reader. You may never really know. It’s just human nature, really, and publishers know this. But the trap is that we all play a shit ton of games, right? But our readers do not. Yeah, some of them play a lot of games that we haven’t. Some of them may play as many as or more than we have. But that’s the small portion of our audiences. Most of them pick and choose what they buy and we have to understand that spending their cash may rely heavily on what reviewers say. You have to respect that and go in to a game with the same expectations that someone without the months of exposure to a title might have.
Maybe that’s idealistic to expect this of writers who are supposed to be “experts” on their field (how do you become an expert in a field or medium if you’re not exposed to everything it has to offer, right?), but when it comes to reviews—in a lot of ways the boss is your reader and you have to kind of get in their skin. Am I always successful at doing this? Hell no. But it’s what I always aim for.
As for the actual process of coming up with a score—sometimes you just know, based on games you played before or a gut feeling when you’re playing it. I sometimes wait to put the score in the review until after I’ve written it so I can step back and get some perspective. If I have the luxury of time (shyeah), then I can let it sit for a bit and return to it for another pass to see if the score still holds. I like how Shoe mentioned the “high-note finish” and I agree. Games are experiences and once you see the entire narrative a developer has to tell you, there’s a sense of accomplishment that sometimes make you review through rose-colored glasses.
***
N'Gai Croal, Level Up/Newsweek: Robert, I’ll raise my hand and say that I do advocate putting an end to scoring. You wrote, "Scores seem to be the one thing that today's online audience can easily form a conversation around, and I think, ideally, a review should be like a conversation between reader and critic." Yes, a review should be a conversation between reader and writer. But /what kind/ of conversation do review scores foster? Judging by the Metal Gear Solid 4 example Kieron cited above, not much. (We’ll get into that more when we tackle Reader Backlash.)
Jeff, you wrote, "scores are primarily there to serve as shorthand for folks that won't or can't read the full review." Unless I'm missing something, that's quite an indictment of a portion of your audience. Why would you write for people who won't or can't read an entire review? You also say that scores "assist average, everyday people in their purchasing decisions." I don’t object to heds and deks; pros and cons; bullet points; final words; buy, try, fry; and other forms of giving readers a succinct take on a reviewer’s opinion about a game’s value. Heck, you could just tell them how much you think the game is worth. And like you, Fran, I firmly support the consumer guide function of game reviews. But anyone who won't or can't read an entire review isn't making an informed decision by looking at a single review score, so I’m not convinced that the reason review scores are “necessary” is to genuinely inform those consumers who can only focus on a letter grade, a number or a star rating.
In a different context above, Shawn talked about intentional fallacy. I think that's what's happening with review scores in the age of the Internet. Those of you who assign scores intend them to perform a certain function, but in the real world, the use to which they're being put by the most vocal portion of your readership is pernicious. Scores help bring out the worst in readers. They shut down conversations; foster silly debates; and they encourage meaningless comparisons. For too many readers, the very presence of scores turns the text of a review into a sideshow for the main event: this number, those stars, the orientation of that thumb. The text becomes a caption and the score becomes the photograph, en route to becoming the final cog in the Gamer Metrics-Metacritic-GameStop machine. Why would any of us want to perpetuate that?
This may seem easy for me to say from the perch of a blog and a magazine that isn’t dependent on gamers for its survival. But many of us work for or were previously employed by outlets that have been struggling. None of us are safe. Newsweek had a round of buyouts this year, the third in my nearly 14-year tenure at the magazine. Time Inc recently had layoffs. MTV had layoffs last week. Ziff-Davis closed Games For Windows magazine. The Tribune Company just declared bankruptcy. For years, magazines and newspapers have been moving towards bigger photos, more charts and even “charticles.” The Associated Press has imposed a 500-word limit on its entertainment writers. All of this devalues the importance of the word. Review scores are yet another signal to your readers that your words don’t matter.
If we don’t think scores are genuinely meaningful—there may be a robust defense of the inherent value of review scores; of the 6.5 versus the 7.0 and the 82 as compared to the 89, but no one has offered it yet—why do we continue this charade? Shoe, you wrote that the desire for scores is “ingrained in society and it’s pointless and stubborn to fight it” and that “For most folks in this short-attention-span world, that ‘4 out of 10’ usually says more than enough.” Apparently it’s not enough for most folks that we slit our own throats; we’re expected to provide the knives as well. I won’t pretend that yanking review scores will bring an age of genteel conversation or Socratic debate to the intertubes. But if message boards must be clogged with pointless argument, I’d rather it be fuelled by the words you wrote rather than the numbers you assigned.
***
Kieron Gillen, Rock, Paper, Shotgun: Since you ask for a justification for scores and the fetishized graduations of hundred-points scale, N'Gai, let me give it try. I'm from the wanky English tradition, and was anti-scores for most of my adolescence for the obvious reasons. They're stupid and putting any kind of scale over a subjective experience was ridiculous. For game reviews, at the current point in the medium, I've come around to them on solely utilitarian grounds.
For scores, per se: reviews exist fundamentally as buying guides, the proverbial shit-filter. They can do other things, but the reason to exist is to spend other people's money. When I score, for the pure consumer magazine (as opposed to one with a more critical leaning) I score for one thing only: should you buy it. For those less-prominent games, a high score is a direct tool to even make them read it - World of Goo and Braid getting a string of 10s is an obvious flare for ATTENTION! in a way which just giving them a glowing review isn't. If reviews are a shit filter, the mark is a very blunt tool for achieving that. That, to many gamers, is just what they want. In a real way, N'Gai, that's the job.
(This will come up later, but the thing about classical game reviews is that it isn't like film or music reviewing, but a hybrid between some stuff that's purely subjective and some stuff that's objective. You don't get a band's new CD which doesn't work on most CD players. You don't get movies which freeze randomly. For a review, that stuff genuinely matters and we're betraying our readers and being deeply disingenuous if we pretend otherwise. Are there other forms of games writing for other readers? Hell, yeah. But dismantling the review isn't the solution. We should just go and build something else.)
And the hundred-point scale? Perversely, what I most like about it is actually its weakness. It's inherently ludicrous. Who can tell the difference between 83 and 86 percent? No-one. In other words, its subjectivity is totally clear. The 10 or 5 point scale has a way of actually tricking people into thinking there's some science at work. The hundred-point scale--and calling it "percentage" scale is another thing that's deeply deceptive—is very silly. The fact it's a rough tool rather than a scientific implement is blatant. If you try to argue a few percentage scores you look as if you're suffering from some obsessive-compulsive disorder in a way you don't necessarily look like if you try to argue between a 7 and a 10. 100-point scales, treated correctly, are an ideal way to both act as a shorthand for the review, and simultaneously make it clear the mark isn't the review.
***
Shawn Elliott, 2K Boston: To add to N'Gai's thoughts....
Some of us suggest that our audiences sees scores as buyers' advice. Actual sales rarely correlate with review scores in cases where games are not also heavily hyped and marketed. Increasingly, gamers pre-order games prior to the publication of reviews. Interactive demos allow our audiences to decide for themselves whether or not a game will be worth their dollars. In addition, word of mouth and message board discussions inform our potential audiences' purchasing decisions with an intimacy and directness that we cannot provide. Finally, review aggregation sites such as Metacritic mute the bias of individual reviewers and provide a bigger picture.
I suspect these circumstances suggest that our self-perception is, well—a throwback to a time when magazines and websites were gaming's gatekeepers.
And yet we have John's anecdote about the angry reactions of some Computer Gaming World readers when the magazine dropped its scores. Robert introduced the idea that "review scores have one use: driving traffic from message boards and social networks to your site and giving those people an excuse to argue out their fan beefs in the comments section." Jeff countered that ratings "are primarily there to serve as shorthand for folks that won't or can't read the full review," which prompted N'Gai to ask why anyone would want to write for such an audience. Maybe our audiences aren't a homogeneous monolith—not in the sense that different readers look to a Level Up or Gamespot for different reasons, but that we (and our bottomlines) want or need different readers to look to a single site or magazine for many reasons. Is this part of the problem?
***
Stephen Totilo, MTV News: Shawn concluded his note with "Is this part of the problem?" I ask again, as I did in my previous note (with slightly different phrasing), what's the problem? Who or what are review scores hurting? N'Gai makes a passionate argument against the damage he sees review scores doing to the discourse about games on some message boards and comment threads. And he pitches a convincing case that such damage obscures the value of the scorers' words. All told, though, that doesn't seem like a whole lot of pain.
But, again, what's the problem? Are quality games not being appreciated because of the existence of review scores? Are quality critics not being read because of the proliferation of scores? Are talented game creators losing their jobs because of review scores? If yes to any of those questions, then would the abolition of scores remedy those situations? If not, I see no more reason for Giant Bomb and IGN to ditch scores than I see them needing to have their reviewers append to their reviews drawings of whatever flower the game they just played makes them think of. Whatever info the readers find useful and edifying, you know?
To the score haters, though, I direct you to Kotaku's reviews for support to your arguments. The mad bloggers there found a way to write reviews that don't use scores but can still somehow be comprehended in the time it takes to tie one's shoes. See their Far Cry 2 review, and, aside from too many puns, it gets the job done: http://kotaku.com/5071946/far-cry-2-review-hurry-boy-its-waiting-there-for-you
***
John Davison, What They Play: Kieron's point that game reviews have to be a hybrid between purely subjective commentary and some stuff that's objective is an extremely important one. The innovation and creativity of games design (or lack thereof) is more than just artistic expression, and an assessment of the overall experience is often incomplete without some mention of the mechanics. Criticism of games reviews often focuses on the fact that we spend too much time on this stuff, but it is often a more important consideration than the "art". I think we'd all love to spend more time on digging into what a game is trying to "say"—but we're often faced with impenetrable control mechanisms, or distracting technological problems that cannot go unmentioned. There are also still an awful lot of games, particularly in the current climate of "casual" style games on DS and Wii, that are practically *all* mechanics, and an intelligent discussion of how well they work is all that's needed.
Take a game like Prince of Persia, though. While there's much to be said about the art style, the way the narrative unfolds, and the co-dependence of the lead characters—a review of the "experience" cannot ignore the fact that the game has some nuts and bolts mechanical issues. It has a tendency to play itself, for a start. The controls are simplified to the point that it can be distracting to an experienced player, and that there are some unavoidable issues with the camera. They're boring topics compared to something that, say, focuses on an emotional response to the player's growing bond with Elika, but I think there's an expectation from our audiences for us to convey and assess the overall experience, not just the "heart."
Shawn, your point about the "homogeneous monolith" is a really important one. Times *have* changed since the days of magazines and websites being gaming's gatekeepers, and while we all tend to identify with our own outlets, and maintaining a sense of community that is distinct, we all have a part to play in the broader scheme of things. Given that Google is the window through which the world views the vast majority of content, there's less and less loyalty to specific outlets, and instead people are simply looking for opinions on specific topics. For this growing audience, a score is still important for exactly the reasons that Robert and Jeff have mentioned. It's shorthand. It's a hook. It's a way to get people to look at our reviews. We are all, after all, businesses. We need traffic. We need people to come to our sites and read our stuff.
N'Gai, you asked "Why would you write for people who won't or can't read an entire review?" I think that's oversimplifying. Often, our audiences (both the hardcore, and the not-so-hardcore) go trawling for review scores purely for validation of their taste. An abundance of high scores serves that purpose, and makes them feel vindicated whether it's because they've made a pre-order, or are simply fans of the franchise.
***
Robert Ashley, freelancer: I wouldn't want to separate "art" from game mechanics and interactive feel. Atmosphere and storytelling have been a fixation for games critics lately, but the connection between the player and the machine—the mysteriously engrossing feeling that makes people scrunch their faces up and open their mouths like zombies—is the real "art" of gaming. Technology and all its various failures play a huge role in that feeling. Anyone can tell when something isn't running smoothly, but I wouldn't call these observations entirely objective. Badly behaving cameras can drive some people insane. Other people not so much. The same thing applies to many common technical problems. The question is, did you have that great zen feeling of being inside and part of the game, or did technical problems and poor interface constantly break the spell? Making a game feel like Super Mario Galaxy isn't a simple question of technology and time. There's something special going on there.
***
Tom Chick, freelancer: On the subject of review scores and expectations, I have a lot on my mind when I review a game. I'm crammed full of preconceptions, expectations, prejudices, hopes, and fears. I call it "context" and it's probably the single most valuable thing I have to offer as a guy who writes about games. I've got thirty years of it under my belt. I don't let loose of it when I play a game, and I certainly don't let loose of it when I write about a game. I write as a hobbyist, and I write mostly for other hobbyists. We know how Spore was hyped, we're aware that Haze was from the Timesplitters guys, we've heard about the problems with Killzone, and we remember Trespasser. It's how we talk. When I have my druthers (i.e. when I'm not writing for a wider audience), it's exactly the sort of stuff that goes into what I write. And, yeah, it figures into whatever number or letter I have to slap onto a review. Like any gamer worth his salt, I have the bias of experience.
As for how much and when I think about scores, I think about them all the time. I think about how much I hate them and how much damage they do to the state of videogame discourse. Scores are an end run around saying anything meaningful. I hate when someone says (almost always on the internet), "I liked your review, but I would have given it an 8 instead of a 7". Because that's an unborn conversation that will never happen. If I didn't have to come up with that insufferable 7, the comment would have had to go as follows: "I liked your review, but I disagree with what you said about it being too hard" or "I liked your review, but I disagree with what you said about the graphics being too much like Fable" or "I liked your review but I disagree with what you said about the ending feeling out of place" or even "I liked your review, but I liked the game more than you did". Those are all starters for at least a line of thought and at best a conversation, and in either instance, we can both be the wiser for it.
Review scores are for the lazy, the unengaged, and the inarticulate. They're for stickers on boxes and press releases. They're understood differently by different people, and they're applied differently by different publications. They're an attempt to inject some sort of science into someplace it doesn't belong and the sad irony is that they mean nothing. I don't know if games are art, but so long as we're branding numbers into their flanks, they're certainly consumer products.
Now I have the luxury of saying all this, because unlike some of you (well, John and Dan, at one point), I don't have to run a magazine or website. To folks dealing with lazy, unengaged, and inarticulate readers, I don't envy you your job selling stuff to them.
***
N'Gai Croal, Level Up/Newsweek: Harry, you came up as a journalist and critic alongside hip-hop. You wrote for The Source, with its famous one-to-five-mics rating system and all the arguments—inside the magazine and among its readers--that were stemmed from its reviews. You also worked for the Village Voice, an outlet that didn't apply scores to any of its reviews—with the notable exception of Robert Christgau's monthly Consumer Guide, which introduced letter grades to music criticism—yet fueled legendary debates nonetheless. You worked for Rockstar Games. You're even writing a book about architecture in computer and videogames. (I bow.)
What do you make of all of this? There's nothing new under the sun, so you must have been in or around discussions like the one we're having right here—does it bring back any memories you can share? Are Jeff and Francesca correct when they say that scores are a legitimate part of consumer reviews, regardless of how some readers may respond to them? Are Tom and I right to argue that review scores help engender the all-too-often juvenile discourse that surrounds videogames? Is Stephen onto something when he suggests that the case against scores is tenuous, and that as long as some readers find scores edifying, reviewers who choose to score games shouldn't agonize terribly over doing so? Is John right that Rolling Stone's editors, not its writers, assign the star ratings? (I had to try, man.) And finally, what's it like being on the receiving end, watching scores and reviews trickle in, and—tell the truth, now--which mattered more?
***
Shawn Elliott, 2K Boston: Before we finish this section with a final question on the real consequences that review scores can carry (an elliptical response to Stephen's repeated "What's the problem?"), I want to take a moment to address any unanswered inquiries. Note that, for the moment, I'm withholding my thoughts on the intersection of mechanics, meaning, and the stories that games try to tell (the short, unsubstantiated version is that I agree with Robert that it's largely fruitless to look at these in isolation).
Kieron commented that he thinks "AAA-popular-sequels tend to start with 9/10 and lose marks, while games with less expectations start with 5/10 and have to gain them." This strikes me as especially true for enthusiasts. Anecdotally, it also seems as though critics are more inclined to take the gloves off with less-anticipated, lightly marketed games or, conversely, to forgive their faults. In addition, Leigh wondered about the wisdom in bringing an awareness of a game's budget to our analyses. Thoughts?
And then there's John's Rolling Stone rumor. What do we make of a critic submitting copy and his or her editor supplying the score? It's an interesting thought experiment. I do think that the policy would encourage writers to explain themselves more carefully, and that it might erase much of the discrepancy in words and numbers, but what ramifications would it carry?
***
Jeff Gerstmann, Giant Bomb: I find Kieron's comment about big budget games catching a bit of a break in reviews to be possible in isolated cases, but I don't think it's the norm at most publications. I've also seen the opposite, where the big budget game gets trashed for not living up to the insurmountable mountain of marketing hype while the low-budget indie darling catches a break because it was made by a team of five people or something. And I've certainly seen little, unmarketed games get absolutely thrashed in reviews. Sometimes it seems like the reviewer is doing this because it's "safe" to do so, like it was part of some sort of "see? We totally use our entire 1-10 scale" chest pounding. This is why it's important for reviewers to have an editor (or editors) that can keep them in check and ask questions about a review and its score before it gets published, especially at outlets where that one review is meant to represent the entire publication's view on a game.
While I've never been a party to anything quite like this Rolling Stone rumor, I've spent a great deal of time as an assigning editor for reviews. The most important part of that position is working with the authors on clarity, to make sure they actually mean what they say, and that they aren't coming off as more positive or negative than they intend. Sometimes that involves changing the score to make it match the words more closely. This would happen most often with freelancers, as they can't really be expected to be experts of how one publication's scoring system differs from another. But except in extremely rare cases where the reviewer wasn't available at post time, those changes were made after discussing it both with the internal staff (as part of a review vetting process) and with the author. I wouldn't want to do things the way Rolling Stone supposedly does them, but that might say more about my faith in most freelancers than it does about the policy itself.
Lastly, I really don't think a game's budget matters when reviewing a game. High-budget console games and mid-budget console games cost the same $60. The only dollar amount that matters is the retail price. While bad games certainly don't get dramatically better as their retail price drops, it's a lot easier to overlook some of a game's flaws if you're getting it for $20 instead of $60. But at the same time, it's probably fine to mention a game's budget in passing. There's a big difference between a brief mention of the budget and using it as the centerpiece of your entire article.
***
Leigh Alexander, Gamasutra/Sexy Videogameland/Variety: I'm super eager for Shawn's discussion point on the impact of the scores, and I don't want to drag everyone down a side trail, but wanted to note something on the Rolling Stone rumor.
No idea what they do at Rolling Stone, but I've actually been in situations more than once where the final star ranking or number was suggested by my editor. It's not as sinister as one might imagine—it isn't as if a person who didn't play the game is independently applying a number with no input from me. In fact, it's more of a collaborative review process between the editor and the writer to be sure that the final rating really does correlate to the text as it's written. Limits in what you can express within a word count, which I'm sure Rolling Stone is constrained by, can make it useful to have two pairs of eyes on the situation.
In fact—and I think this is especially true for those who write for more mainstream print—many reviewers in that context want to avoid the wham-boom core market number hysteria. They want to write articles, they prefer to write crit, and don't want to calculate numerics. They may be non-traditional reviewers (as I'd assume Rolling Stone's are). In that case, especially where Metacritic is involved, in my experience an editor may volunteer to apply a score that's correlative with the review text simply to offer an option for those writers who don't want anything to do with the numbers game.
In those cases I have always elected to have input because I feel comfortable that way—but keep in mind that scores can cause headaches and PR arguments for reviewers, and in that case, an editor stepping up and saying "point them to me if they hate the number" may be one way of allowing the writer to do his or her own most honest work with impunity. When an editor plays a role in the score, he or she's essentially "backing" the writer's text by shouldering all those burdens a number can provoke. It's a pleasant reversal from the nightmare scenarios we hear about where writers catch flack from their bosses or even get canned because some publisher flipped their lid.
Again, I don't even know anyone who writes for Rolling Stone. I wish I did, because I'd tell them to write about good music again. But my suspicion is that scenario doesn't so much involve some sinister overlord stealing scoring power from the reviewer, rather an editor who wants to make room for the writer to do what the writer's being paid to do—write well and thoroughly on a title.
***
Shawn Elliott, 2K Boston: The editor in me empathizes with Jeff, especially in cases where a green writer's copy is in apparent conflict with the rating he or she attaches to it. Discussion is helpful here, as the writer may reveal anecdotes and a level of analysis lacking in their initial draft. How many of us remember the teachers who told us they wanted to read in our revisions whatever it was we'd just said?
The writer in me, however, is just as wary of knee-jerk editing, and editors who've played -- and have formed their own opinions of -- the games they've assigned to freelancers. Here http://www.gamesradar.com/pc/call-of-duty-4-modern-warfare/review/call-of-duty-4-modern-warfare/a-2007110512090683184815/g-20070214121215902085 , for example, we find text apparently taken from PC Gamer UK's Call of Duty 4 review ( http://www.computerandvideogames.com/article.php?id=175050&site=pcg) transported to GamesRadar's 360 review page where a presumably miserly British 8.5 becomes a prodigal American 10. Bylines differ, too, as though we're to believe that two writers, an ocean apart, have arrived at the same sequence of words and altogether different assessments of their meaning (the editor's note states that the GamesRadar score was once a 9, which still doesn't explain discrepancy). That Francis's copy came first is fairly obvious in lines like, "If the whole game had been like that, or even just as inventive throughout, you'd find a frankly silly score at the end of this review. Instead it's a more restrained one[....]" My suspicion is that a comatose editor couldn't be bothered to read let alone edit the original. Frankly, that's a fucking insult to writers and readers if there ever was one. It says, 'Hey, fill some space for the cretins naive enough to not notice. We'll handle the hard work."
Hopefully, this is an anomaly. The idea of dismissing the Rolling Stone idea for it is as dumb as it would be to eliminate governorship because of Blagojevich, but, again, we're all human.
***
N'Gai Croal, Level Up/Newsweek: Regarding the Rolling Stone rumor, I'm think it's legitimate for reviewers and editors to discuss scores before assigning them, but I'm wary of editors assigning scores all by their lonesome. This point was driven home for me when, in the wake of Jeff's firing from GameSpot following his Kane & Lynch review (http://blog.newsweek.com/blogs/levelup/archive/2007/12/05/reflections-on-videogame-publisher-and-employer-contempt-towards-the-enthusiast-press.aspx), Tor Thorson issued a statement explaining that "The copy was adjusted several days following its publication so that it better meshed with its score, which remained unchanged." (http://www.joystiq.com/2007/12/05/gamespot-addresses-gerstmann-gate-concerns-in-depth/) Even given the force majeure of this situation, the question that immediately popped into my head was this: if changes had to be made at all, why wasn't the score changed to better mesh with the copy? If there are going to be scores, the individual reviewer should always be intimately involved in that process.
*****
Shawn Elliott, 2K Boston: To finish this section and finally return to Stephen's point that scores, despite 'damaging the discourse about games' and 'obscuring the value of words,' aren't an actual problem, I'd like to ask one last question.
Review writing carries real consequence because some publishers do base developer and PR bonus pay on aggregated ratings. This shouldn't concern critics, but once-warm PR people and game producers can become cold upon our publication of undesirable review scores, diminishing or eliminating our ability to secure subsequent interviews and access. Postmortem discussions and exclusive looks at the publisher and/or developer's forthcoming products are less likely. Conversely, a few publishers will permit us to post reviews before competitors, provided our review scores are favorable. Do such pressures produce a subliminal background—especially among members of the enthusiast press—or even enter our thoughts as we write or edit reviews and assign reviewers or scores? The stock answer says, "Only if you're a bad apple, and I'm not." But isn't the seeming impropriety of business in a bad barrel a problem in itself?
Consider special instances such as the Gamespot Kane and Lynch review that N'Gai cites. Jeff, are you contractually able to discuss that episode in any detail? John, Francesca, and Dan, as well, serve/served as EICs of enthusiast publications and presumably face/faced such pressures and repercussions.
***
Dan Hsu, Sore Thumbs Blog: Exclusive reviews stink...it’s a lose-lose situation. If you score high, then readers call shenanigans...even if there were none. If you score low, then the game makers get mad and won’t want to work with you in the future.
Plus, most companies want a guaranteed score or range of scores. We did play with this fire once at EGM...the very first time I was faced with it. I thought I could deal with it by protecting the reviewers from that discussion—I’d let them review the game independently and then see if the scores were high enough for us to secure that exclusive. But because I had final say on all magazine content, I’m still a part of the reviews process. In the end, I didn’t feel comfortable promising any certain scores, so I backed off and made a rule to never entertain these offers again. If we don’t even have those discussions to begin with, then we’ll have preserved the integrity of the reviews from start to finish.
Besides...if the reviewer caught wind of the deal, how can it not spoil things, however so slightly? Let’s say the reviewer’s 50-50 wavering between an 8.5 and a 9.0, but way back in the deep recesses of his mind, he knows that a 9.0 will get his website or magazine the exclusive review, which translates into revenue-producing traffic or sales. Will that load the dice? Maybe not for everyone, but that’s not the point. Once that process is tainted, it’s tainted. Just ask the reader what he or she thinks.
***
Leigh Alexander, Gamasutra/Sexy Videogameland/Variety: I think in Jeff Gerstmann's case, the fact that the editors took liberty with the final text and scores are evidence of the "Rolling Stone System" not working in practice—again, I support the idea of editors protecting their writers (as Shoe tried to do with his folks, it seems), and doing whatever they can do to keep the heat of publisher response to scores out of the way so that the writer can do his best work. The "protection" obviously didn't take place with Jeff, and while I've no inside knowledge of that situation it sure looks to me like they let him take a bullet, which is at the very least evidence that making any kind of bargain with a publisher regarding a review—its exclusivity or otherwise—is a BS proposition.
I hope that this reason, coupled with the evolution of Internet media, means that exclusive reviews are going to go the way of the dinosaur (or of the console exclusive, har har). I wholly believe we now work in an environment where quality and depth can be the primary competitive advantage, and less timeliness.
I also can't speak for Totilo, and don't mean to put words in his mouth, but when I agreed heartily with his response, I was agreeing with the idea that scores only cause problems for the industry and for us—neither of whom is the audience we serve, and therefore that they create stress is not a compelling argument against the fact that plenty of readers find them useful. Hell, I hate scoring as much as most of us do, and I prefer scoreless criticism and blah blah blah—but in all honesty, when I really want to know whether I should pay attention to a game, the very first thing I do is go to GameSpot and see what number they gave it, for whatever it's worth.
I never buy based on a number (let's take bets on how many—or how few—year-end top ten lists the 10-rated MGS4 or GTA IV get), and I don't think readers do either, which further takes the piss out of this idea of their damages. Believe me, I'd love to stop scoring. I'd love if we could enforce an industry-wide moratorium on scores, so that we wouldn't have to think about 'em and so that our audience could re-learn to focus on the words we write.
But I think we hate scores because of the undue importance that has been placed on them and the ways they've distracted from our work, and this is credited in part to editors balancing the needs of the publishers with their management of irrationally hostile reader reactions, neither of which should be their focus. None of this means they in and of themselves cause harm to the readership or to the art of the review.
***
Shawn Elliott, 2K Boston: Dan, do you see where this can apply to cases less extreme than exclusive reviews? Let's say that a publisher puts one game in the market this month and is also looking for an outlet to announce a forthcoming game a few months from now. No conspiratorial conversations or conniving occurs. Nonetheless, you have to know that by giving the available game a negative review, you could risk reducing the likelihood that the company will allow you to reveal their other title. Scruples aside, the cost-benefit equation is simple: A first look at the unannouced game is probably better for business than the bad review, provided nobody calls bullshit. And compromise comes in shades. An EIC can always assign the review to a specific staff writer who he suspects will appreciate the game more than his peers, and argue afterwards that he hadn't touched the score itself. I have no idea whether or not this happens. I do, however, know that commercial publishing is cutthroat.
***
N’Gai Croal, Level Up/Newsweek: From the outside looking in, I’ve been conditioned to expect that review cover = exclusive review = 8 or above. That led to a bizarre reading experience last holiday when I picked up a copy of Official Xbox Magazine with Turok on the cover, touting the first review. I can’t lie, I immediately flipped to the end of the review to see what score had been handed out…only to be shocked to see that it had been tagged with a 7! What went through my mind next was:
a) Damn, that took guts for Francesca to stick to her guns and let reviewer Paul Curthoys call it like he saw it. b) there goes Christmas for Josh Holmes and the rest of the guys over at Propaganda Games, to say nothing of Graham Hopper and the folks at Disney Interactive. c) I would not want to be the publicist who brokered that exclusive. Her/his job is to suss out the most favorable outlet when determining who’s going to get that first review, because you only get one chance to make a first impression. But a first review score of 7 means that something went terribly wrong—and from PR and marketing’s perspective, the “something went terribly wrong” in this kind of situation is /not /the finished game’s quality.
At last glance, the Metacritic rating for the Xbox 360 version of Turok was 69, so OXM’s review was right in line with the critical consensus as calculated by that aggregation engine. I’m sure that was little consolation for Propaganda and Disney, though.
John, didn't you write about the pressure surrounding review exclusives when you first started blogging?
***
Dan “Shoe” Hsu, Sore Thumbs: Shawn, that’s why this business is so messed up! Publishers want good reviews. Editors want exclusives. Magazines and websites want advertising. Advertisers want good reviews.
Sometimes, the companies know their products aren’t that great. They have internal research (via their own gut checks or “mock reviews”—early evaluations done by outside consultants) that give them a rough idea of how a game may score. If you’re dealing with a reasonable PR team and you’ve scored within their expected range, then you’re usually spared the grief. But woe is the outlet who’s at the bottom of the Game Rankings or Metacritic lists. We’ve had PR people complain to us that we’re “outside the average” (on the low end) on those sites, which John and I would always laugh at. Of course someone has to be outside of the average. If everyone was ON the average, there’d be no point in averaging them!
***
Jeff Gerstmann, Giant Bomb: Exclusives are yet another by-product of this business' print roots. Between magazines stories getting scanned, website text getting pasted into message boards, and the way that most of a site's traffic doesn't come in through the front door (where the big promotion of the exclusive always is), exclusive stories are waaaaay more trouble than they're worth. Exclusive reviews doubly so. On top of that, in my experience, most big exclusives don't really move the needle that much, traffic-wise. They've always felt like a huge waste of time to me. Attracting users by differentiating your coverage from the pack has always seemed like a better goal.
But are we talking about a problem that doesn't really exist anymore? Isn't IGN the only publication still doing exclusive reviews? And if so, they're probably getting them because they're the only ones asking for them, right? I think they've deflected questions about their integrity with the ol' "well, we're not on the take, case closed" routine, which the majority of the public seems completely fine with.
***
Stephen Totilo, MTV News: Many of you were struggling to make a decent argument for the elimination of review scores. Here, I think you just did. Don't put scores on your exclusive reviews in your magazines. Add them via your website two weeks later when everyone else's reviews run. Then everyone's happy.
Or…just don't take exclusive reviews. This would be a challenge for a magazine that needs to secure something notable months in advance of press time. But for websites, is there really a need? You can get the game when it comes out, play through it and have a review up within a week of it's release. No tricky politics. No stink of compromised values. And your readers are served.
The threat of publishers withholding cooperation in the future because of a review strikes me as absurd. I don't doubt that it's a real threat, but there are countless ways to report and edit around any blackballing. Besides, getting blackballed usually makes for a good story.
***
Robert Ashley, freelancer: Just to add to the list of reasons why reviews create editorial drama, it's my understanding that retailers take into account review scores when stocking new games, which means that scores can directly affect sales. I had a PR rep from Deep Silver tell me last week that Target bases their game orders exclusively on Gamespot scores (so, of course, he was bitching about all the snooty divas at Gamespot). I can't vouch for his claim, but just having the guy go off in a room full of other pro enthusiasts noticeably changed the mood of everyone in earshot.
While reviewing for EGM, I felt like there was always barrier between me and PR, thanks to the reviews editor. But when I would venture out to write other stories, visiting studios and going to game reveal events, I would often get snippy comments from people about my reviews. I was once accosted by a developer from the True Crime: New York team about my review of his game. We had, by chance, ended up in the same two-person sleeper car on a train from Paris to Venice. And I was on a family vacation, not some gaming press junket.
***
Dan “Shoe” Hsu, Sore Thumbs: Stephen, getting that first review up the weekend before a game’s out versus a week after launch can be a huge, huge difference in traffic. So most website editors would find an exclusive review very appealing. Can’t blame them.
Those threats from publishers can come in many different forms. Sometimes, it’s just one very specific team or division that wants to blacklist you. For example, we were supposed to get an online exclusive as part of a package deal with our Saints Row 2 cover story, but after they read Robert’s article, THQ told us that the developers didn’t want to work with us anymore. So that online exclusive went to someone else instead.
The Mortal Kombat team and Sony’s sports division banned us as well. But the interesting thing is, these three blacklistings didn’t carry over to the rest of THQ, Midway, or Sony Computer Entertainment. They were very specific to those specific products, because they felt we had it out for them.
Then there are situations where the blacklistings go company-wide, like with Ubisoft after our Assassin’s Creed reviews...and after we wanted to do a story about how outlets were allowed to break the universal AC reviews embargo if they scored the game high enough. That “non-cooperation” on our part was the last straw for Ubi, and when I reported on this blacklisting in EGM, it just further cemented their hatred for me as an individual (even though Sam Kennedy, Patrick Klepek, and just about every other editor wanted to run those stories as well).
Now, I don’t really think Ubisoft or any of those other companies have done anything wrong here. It’s their right to work with whomever they want. But it just goes to show how uncomfortably and inappropriately cozy the industry and enthusiast press are expected to be. You cooperate, you benefit. Simple as that.
***
Kieron Gillen, Rock Paper Shotgun: *Yeah, Shawn, regarding exclusives, on the UK side it's pretty similar. On the print side, perhaps even more intensely. There's just more magazines and the competition has always been fiercer. The former is primarily due to the size of the country - the shipping costs don't cripple you, and a smaller magazine can be more profitable. The second is due to how British games magazines are sold. The vast majority of magazines are sold at newstand, not subscription. Subscriptions are cheaper, but not a straight loss leader for ads. When I was on Gamer, the majority of our money came from actual people buying the bloody thing rather than via advertisers. On the bright side, this abstractly means that in a real way, the bills are paid by people who want to read your mag. On the bad side, since there's an enormous floating readership which you have to fight for every single issue, meaning even more importance is placed on exclusives. An enormous amount of effort is thrown into chasing them, and it can totally lead to the sort of issues others have picked up on.
When I'm chatting to modern day magazine staff, one of my favorite stories to tell is about an old pre-me Future major-mag editor. He didn't speak to any PRs. The Dep Ed did all of that. If any of them tried to speak to him, he just blanked them. He just made the magazine. And when I roll out that anecdote, the look of disbelief which it's always greeted with is akin to what I'd have got if I claimed he was capable of flight.
As I said, they're an enormous waste of energy—I always remember that it seemed that due to the inevitable pissing off of PRs, the exclusives we chased went to our major rival as much as not, and the ones that they went for went to us. But it's also a game which I think magazines—and publishers—are loathe to get out of it. Because as long as one other organ in the room is making exclusives, you are at a distinct disadvantage. And it works both ways. One major games publisher, working on similar logic to me above, stopped doing exclusives for a few year. The amount of coverage suffered—because their rivals were still making deals and they weren't—and they went back on it.
That's the problem with stopping exclusives. It requires a conspiracy of doves for it to hold.
Regarding changing marks and having marks changed... much like others have said, I've done it. It's really not a problem per se. Discussing it with the writer is fine (And an art form in and of itself). Explaining how they're not marking to your mark-scheme is fine too. And most commonly, the review and score just don't match up, where you have to ask that one or the other is changed. If a reviewer is having fun comparing part of the game's mechanics to cancer and still give it in the sixties, either the overwrought writing goes or—if they actually mean the overwroughtness—the mark needs to drop like a stone. In my experience, I've argued far more marks down than up.
On the other side, I'm fine with that too. Sometimes I really don't agree, in which case I ask for my name to be removed from the review. Not in a prima-donna way—just because I don't believe in it, and ultimately, your name is all you've got and you're going to give enough review scores the readers will tear you apart on your lonesome without someone else making them for you. This is one of the first things I say to any writer getting into the business, because—bless 'em—not many are even aware that it's something they can ask to do. And if the editor says no, it's about as good a sign that you should get out of that job sharpish.
And, of course, some editors do change marks without asking and keep your name on it. Just don't work for them again.
***
Harry Allen, Media Assassin: Greetings, everyone.
I just want to say that it's really a privilege to be able to read all of your thoughts. I'm a fan of you all. I read your words and listen to your podcasts, in many cases. So, it's great for me to hear your thoughtfully written ideas about this very trenchant aspect of your work.
I don't know that I have anything to add. Much of what I was thinking has been said by other people, and I know we're about to change topics. For example, a very early concern of mine was the relationship between scores and advertising, so I'm glad that some of that was addressed recently.
I will say that one of the first statements I read that directly connected to my own work was N'Gai's comparison of the reviewer and the critic. Though I use different language, I know that when I started writing about hip-hop professionally, in the late '80s, I made it my objective to never talk about an album in terms of whether I liked it or not. Instead, I always saw it as my job to explain the artist's intent to the readers. I've never called myself a "music critic." I've always said that I am a Hip-Hop Activist.
To me, number scores are mostly an attempt at giving the illusion of numerical precision to functions that cannot possibly bear the same.
Most curators and art historians would call the bold canvas "a masterpiece," and leave it, more or less, at that. If they did quantify it, or one of the others in the series, they would only do so in extremely hushed whispers, among themselves, if at all, but never do so for public consumption. This is actually part of the way that they frame their field, which is commercial, but which is mostly a repository of serious, high culture: By avoiding quantifying numbers ("It's an 8"), which would be seen as crass.
Quantifying a Motherwell, then, would be analogous to asking how much the Maybach is: If you have to ask, you can't afford it. They're not saying you can't afford the Motherwell financially, but you "can't afford it" aesthetically. In other words, the experience of the painting is far too rich to reduce to a digit, and if you don't know that, you shouldn't be here.
I think a reason similar to this is also why I always resisted, during that brief period of my life, when male friends would ask me to assign a number to a girl I've seen. In hip-hop / Black slang, a "dime" is a girl who's a 10...but what does that mean? According to what objective scale?
And indeed, isn't that the core idea that disproves the fantasy: That without an actual 10 to which one can point—the theoretical perfect game—the numbers become meaningless?
That is, on a foot-long ruler, "4" only means something because there's a "7," and a "9," but, most of all, because there's a "12." However, "12" only means something because there's a "13" and "25"; an agreed-upon metric, in other words.
When it comes down to it, game numbers facilitate the purchasing decisions of the buying public, and advertising, and I think that's it. I could have told John Davison that his CGW readers would revolt. It's like all those Americans who claim they only watch PBS and the news, yet, somehow, reality shows are a phenomenon, and porn even more so.
But, clearly, the focus on numbers is leading people to overlook something important, namely the subtle interplay of parts that is a videogame.
I've not played Gears of War 2 yet, though I did have the nerve to write about the game's marketing on Media Assassin, here (http://harryallen.info/?p=1863).
But when I look at stills and video from it, and hear descriptions of what Cliff Bleszinski and crew were trying to do, it's clear to me, Roger Ebert be damned, that we're dealing with a moment here. Gears is a work of art expressed in the videogame medium. It's that simple.
Here's a prediction: A hundred years from now, collectors will purchase copies of Gears, Katamari Damacy, System Shock, Crysis, Ico, Dead Space, and every Mario iteration the way many, today, buy coin banks, first edition books, fine watches, and music boxes.
I'm sorry to say this, but, by that point, those collectors will not care if these games were given sixes or nines. They will collect them as beautiful art expressions of human craft and intellect. For many of the artists who work on these games, it will only be then that, aside from the thoughtful critique of people like you, they get their just due.
###
posted by Shawn Elliott at 10:02 AM on Dec 18, 2008
"Symposium Part One: Review Scores"
421 Comments -
-
Jomolungma said...
-
Duffyside said...
-
Teanaholic said...
-
Will Templeton said...
-
Unknown said...
-
Unknown said...
Sachin Agarwal said...-
Anonymous said...
Jonathan said...
Etelmik said...-
Nate said...
John D. Moore said...
Travis Megill said...-
Anonymous said...
Macroe said...-
Allen said...
-
Das Dorsch said...
-
Arby said...
-
Andrew said...
-
Anonymous said...
-
Anonymous said...
-
Mike said...
-
class of 89 said...
-
Jake said...
-
Juan Letona said...
-
TenaciousCK said...
-
trip1ex said...
Colin said...-
Jonny said...
-
Jonny said...
-
Jonny said...
Justin said...-
Unknown said...
-
Shawn Elliott said...
-
Laez said...
-
Anonymous said...
-
Unknown said...
-
frrrr said...
-
Unknown said...
Chris "Papapishu" Person said...-
Matt Jett said...
Oliver Snyders said...
Doogie2K said...-
HOSB said...
-
Unknown said...
-
Shawn Elliott said...
-
J.Goodwin said...
-
blackwax said...
Nicholas James West said...-
Anthony said...
-
Marc said...
-
Shawn Elliott said...
Nicholas James West said...
Allen Goode said...-
Bib said...
-
Unknown said...
-
Dan Kaplan said...
-
Andrew said...
-
Hunter said...
-
Gabriel R-T said...
-
Jon K said...
~ george said...-
Adam R said...
- Comment deleted
-
Unknown said...
-
BJ said...
-
Adam R said...
-
Bobby K said...
cityshlacker said...-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Toni said...
More Console said...-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
kimberly sayer said...
-
Anonymous said...
-
Anonymous said...
-
Anonymous said...
-
macafe said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Anonymous said...
-
Unknown said...
-
Anonymous said...
-
Anonymous said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
men's clothing said...
-
Unknown said...
-
aai333 said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Cathy Mena said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
men's clothing said...
-
Unknown said...
Christian Hymon said...-
Ya qin said...
-
Unknown said...
-
Unknown said...
-
agatha said...
-
ClothingJeans said...
-
ClothingJeans said...
-
ClothingJeans said...
-
ClothingJeans said...
-
Unknown said...
-
woteam said...
trustme said...-
Unknown said...
-
janewangleilei said...
-
Unknown said...
-
Unknown said...
Christian Hymon said...-
aai333 said...
-
ai said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
ai said...
-
ClothingJeans said...
-
ClothingJeans said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
air max nike said...
-
Unknown said...
-
Unknown said...
-
zxdzxdzxz said...
-
ClothingJeans said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
angela said...
-
Unknown said...
-
zxdzxdzxz said...
-
ai said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
ClothingJeans said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
Unknown said...
-
ClothingJeans said...
-
Unknown said...
1 – 200 of 421 Newer› Newest»Bless you Shawn. It's gonna take me a while to read through all of this, but I will. There is such a need for this type of conversation in gaming. I think as, sadly, more and more sources of gaming criticism and journalism dry up, it is important for the respected and experienced voices to get together and continue examining and growing the industry. That's why I LOVE Slate's Gaming Club and any podcast that can bring together multiple, informed perspectives on video games. Continue the good work. Can't wait to see the fruits of your labors from 2K Boston as well.
December 18, 2008 at 10:22 AM
I'd give this symposium an 8. Compelling.
December 18, 2008 at 10:36 AM
Third!
December 18, 2008 at 10:39 AM
Having only read up to John Davison's second entry, I apologise if this has been discussed in the latter half of the post, but I just wanted to get this out there.
I believe the reason for all the stress about scores is precisely because of that disconnect between reviewer and critic. What games journalists as a whole seem to be doing right now is trying to merge the two, which is a task without merit. Both the ten-point or five star or whatever scale and the critical analysis of a game's content have a place in the industry, but it's vitally important not to confuse the two as they cater to wildly different audiences.
At a base level the job of a review, as was stated here, is to inform a purchase choice. The majority of gamers have limited disposable income. Comparing Prince of Persia to Sonic Unleashed is ridiculous from a critical standpoint but imperative when you only have sixty bucks in your pocket.
The person who wants to discuss how an individual game affects the medium as a whole or storytelling in general or any one of a thousand other factors is a wholly different person to that first audience, and I think that any attempt to appeal to both audiences at the same time is on a path to frustration.
December 18, 2008 at 10:42 AM
Thanks for these posts. A good LONG read.
December 18, 2008 at 10:43 AM
Although, a spell checker would be nice. There are a ton of typos even in the short pieces.
For example, "shouldn't it that exemplify."
I realize these are emails and you want to maintain the original correspondence, but I keep getting caught up on it. Not trying to troll, trying to give some constructive suggestions.
December 18, 2008 at 10:50 AM
One of the things left unsaid here is to what extents exclusives are unique to the gaming industry. Apple has a handful of favorite outlets (NYT, WSJ, Newsweek, USAT) who get to review new product, and they've done exclusives (flat screen iMac to Time) before.
We just saw Activision give USAT the exclusive first look on GH: Metallica. To what extent are broader publications inoculated from these pressures? Could not newspapers and general press magazines be able to more accurately do exclusives because they're not as dependent on the advertising from the people they are covering?
I've advocated Tribune Company (here in Chicago) to aggressively go after gaming reviews (either as the CT, the LAT, or a company-wide reviewer) because they can, due to being less reliant on gaming advertising, be more objective even with an exclusive.
I'd love to see your thoughts about whether or not Seth and N'Gai can be freer to do fair exclusives than Shoe or Leigh could at their publications.
-Sachin Agarwal, Dawdle (sorry about the Jerome Walton name; that was my Deadspin nom de plume)
December 18, 2008 at 10:50 AM
Thanks guys. As an avid consumer of gaming coverage, this is fascinating stuff. I guess I can count myself in minority in that I don't much see the point in numerical scores.
The extent to which a score ultimately is useful to me would be a bunch of statements like "buy/rent/don't bother..." broken down with qualifying clauses like:
...if you do/don't like the genre
...if you do/don't like other games in the series
...if you don't/only care about multiplayer/singleplayer/coop"
December 18, 2008 at 10:57 AM
I thought I might help to see what an average gamer/consumer read reviews for. I can typically only afford one game at a time.
If there are a few games out that peak my interest I will go to GameRankings and find the highest reviewed one, read snippets about it and typically go purchase it.
If there is one game I am particularly interested in a I will go to GameRankings and see what overall score the game has. If it is above an 84 or so I will have no issues with buying it. If the score is under that however I do further research by reading the actual review.
If there a few games I definitely want to buy I go through and read every review on each. Like Fable 2 vs Fallout 3, I knew I would play one of them but which to buy first, I had to research to find out. I read everything I can on each and it is then that a review is truly helpful.
If there are two sports games and I have no idea which I want to get (NHL 09 vs NHL 2k9) I will read reviews which typically compare them anyways.
If there is a game out for multiple systems and I am not sure which to buy it for I will consult reviews to help me with my decision.
I don't read "reviews" for critiques, I read blog posts or forum threads or listen to podcast's. That is much more interesting to me. I don't want a critique going over every little thing a review typically would. I want to hear what the writer thought of the game as a whole not broken down into small reviewable sections.
December 18, 2008 at 11:57 AM
The message that I keep seeing here is that it's all about economics.
"Publishers want good reviews. Editors want exclusives. Magazines and websites want advertising. Advertisers want good reviews."
This is the shortest quote that could encapsulate it all. Review scores are simply a symptom of the audience: yes, it is regrettable that a score means people don't read it, but remember (as if needs to be said): we give the Internet and gamers too much credit if we assume they fully read the majority of anything that is posted. Remember the comments in Game|Life's top 5 360 exclusives?
If you want to write more reviews that give little or no consideration to the scores, you need the audience. Is that audience there? If so, how can you get it? The only answers I can think of are economic ones; I hate to be so reductive, but I don't see how business and money aren't what makes review scores go round.
December 18, 2008 at 12:01 PM
Shawn, I know this is a bit early on, but is there any plans on binding this symposium and printing it? I found this first part fascinating, and I know with much more to come I would love to have a copy of it sitting on my coffee table to make me look intelligent.
December 18, 2008 at 12:28 PM
Fascinating stuff, and I'm eager to watch as the rest of this symposium series unfolds.
One thing I'd like to see tackled is why the hell game scores are so seemingly inflated. To pull out its easiest analogue, let's look at typical film scores. Rarely do I see as many perfect 10s (or four stars or what have you) for a film, especially when the accompnaying review is a list of things that are terribly flawed about it. From the reactions I see, a 7.5 review score for a game is practically a kiss of death, whereas a film reviewer (like Roger Ebert) can make a case for the value of a 2.5 star film.
December 18, 2008 at 12:31 PM
Review scores only matter as an aggregate, at least for me. They're a way to summarize a large group of reviews in order to get an idea of what critics thought of it. When I'm looking at a specific review, the words matter far more.
December 18, 2008 at 1:08 PM
Personally, I knew that I'd probably end up sounding like a mix between John D. and Robert A. here.
I'd love you guys to keep this going with all the topics and issues floating around the games industry. It’s an effing goldmine of reading for someone like me.
I'll spare you the long winded personal rant I have on the matter (as you guys spanked it pretty nicely) and just provide some random observations to ponder over.
1 - The idealism that some arguments form here makes me feel a bit snootish for the thoughts that immediately strolled through my head. I consider myself a fairly open-minded person, yet I still found myself sighing at some of the more idealistic based statements here.
2 - "we're all human" got put forth just enough for me to notice it. Not that it's bad or anything, it actually made me smile with a bit of malice in my heart =). Someone go back and count it, it can’t be that many…
3 - I wondered when that damn art debate will come back. Harry's last comments came in like Elika at the last minute for me...They won’t be art until 400 years from now. Seems none of us are smart enough to realistically have it function as a means of existence now. People like Ebert are exempt, as are all people who talk out of blind ignorance.
4 - Psychology and Subjectivity...I prefer to see that to be honest. When those things are exposed, THEN you're really starting to approach whatever sad little reality we hold dear.
5 - Also, acknowledging rationalizations and spurious objectivity is something that surprised me here. It's nice to see the editors knowingly bowing before it (while whispering mean things under their breath).
6 - I’m sure that centuries ago when paintings were the “highest” form of art there was, there was some trendy asshole that walked around pointing at canvases going “OH, THAT’S AN SEVEN!”
~sLs~
December 18, 2008 at 1:12 PM
Thank you Shawn for your initiative, time and dedication to this project.
December 18, 2008 at 1:21 PM
I'm reminded of the proposed Cheap Ass Gamer scale for reviews - if the number is meant to reflect some kind of consumer impulse, review the game on a 60-point scale.
The idea then is that the number tells you at what price you should purchase the game. A game that deserves an instant, day one purchase would get a 60. A game that you should wait for bargain bin status would get a 10. A game which is "good" but not worth "full price" might get a 40.
Arguably, Cheap Ass Gamer is the most consumer focused gaming based portal on the Internet, so assigning a dollar value to a game might be crass... but if that's the ultimate end goal of a score, why not?
Assigning a dollar value on a game, much in the same way that an "expert" on Antiques Roadshow would assign a dollar value to a painting or a coin seems to be what people who want scores are looking for anyway and it unifies the seemingly arbitrary scales that each publication imposes on its reviews.
For what its worth, from an outsider perspective, I don't even read reviews anymore. As a consumer, forums like the 1up Boards or GameFAQs or NeoGAF provide much more useful feedback. If a dozen people in a thread for a game agree that the controls for that game are inherently broken, then most likely the controls are broken. The "non-hardcore" version of this phenomenon is the user review on websites such as Amazon.com. In the "Web 2.0" age of "social networking" and "crowd sourcing", consumers are looking to each other for advice rather.
Anyway, I'm really looking forward to - if it happens - the Game Criticism vs Game Reviewing topic. I'll fully admit that I'd love to "critique" games for a living... unfortunately, the only way to do that right now is to get a PhD and become Ian Bogost.
December 18, 2008 at 1:26 PM
I do not think people will buy videogames of the early ages in 100 years as they buy paintings. Paintings were part of a culture that was led by interconnected, highly educated people, elitists if you will.
Nowadays, culture is a thing off the masses, and elitist culture is split into many niches. A far mor appropiate medium to compare this to is the comic book, as it also started as a mass culture, not lead by intellectuals.
On that not, I do not think that games match the narrative experience of the other narrative media. I still love games with a liner narrative, though, because they do things other media stopped doing long ago: they improve in quality as time passes and newer games get made. Stories are getting more serious, immesion-breakers minimized.
Games have become better and better in both technology and story presentation. Because there are such clean, highly visible differences in quality of those divisions, I think a score may be legitimized. Ore so than a movie review having a score, at least.
December 18, 2008 at 1:39 PM
http://www.telegraph.co.uk/news/newstopics/howaboutthat/3816785/Performing-monkeys-attack-trainer-in-China.html
December 18, 2008 at 1:59 PM
tl;dr
December 18, 2008 at 2:44 PM
Kieron,
Look up tautology in the dictionary. It does not mean what you think it means.
But to be a little more constructive: I was really disoriented by the structure of the responses. Direct questions were asked and never answered (much less acknowledged by the recipient) and whole conversation threads were dropped, only to be picked up again without warning. N'Gai references Harry, and it's half a dozen responses before Harry puts in his first words.
I'm really interested in what you guys have to say, and you say a whole lot in this post worth reading, but I'm wondering if there isn't a more legible way to present it.
December 18, 2008 at 2:52 PM
It’s a shame that blogger doesn't have a more sophisticated comments system, such as only giving comments to specific entries. There’s a large amount that I would like to address on here but it's much harder in this manner.
I do like the "Rolling Stone" scoring method that was mentioned here. It would give a feel of validity to reviews that personally I feel that any score lacks. Personally I don't trust review scores at all, they work on a comparative scale that differs in each reviews outlet, although I do have greater faith in the new 1UP grading system, which is a much more general indication of quality, (rather than a specific indication of a less understood scale of quality). When I read reviews I've always read it in its entirety to see what the reviewer thought, usually in order to see if it was a game that I would like and also to see if there was anything that would put me off wanting to play it, often reviews would decide whether I brought a game that I wasn’t sure about.
I think that review scores cause a lot of problems, while they aren’t inherently evil they are quite possibly the root cause of stagnation and corruption for video game reviews based industry, possibly even the games industry itself. While this scoring system is in place PR will remain fixated on them, trying to influence and even control the outcome of scores. Sometimes this will work and even if it doesn’t it will often mean relinquishing exclusive reviews and as mentioned above magazines need to co-operate to succeed. Since many publishers now are internally ascertaining the quality of a game and the ability of its design team by its Meta critic score, in a worst case scenario this would mean that the actual content of a game is irrelevant it will receive a certain score regardless. It would also mean most reviews might as well post a number and add some text to make it look official, maybe the games press description (a review Shawn mentioned some time ago actually did this, maybe he’ll link it in a later topic).
I believe we have this bullying of reviewers in video games but not so much in other mediums such as film is due to the youth of the industry, and also of its audience. Games are not yet universally understood and accepted as films are, they don’t have the wide spread and varied audiences quite yet, although this is changing at a exponentially increasing rate, mainly due to Nintendo. I don’t believe there is yet a large enough demand for high quality reviews or mentioning of certain types of games in non-gaming publications. Such sports games reviews in sport magazines and critiques talking about the “best entertainment experiences of 2008”.
Video games aren’t to the point where a review could be in a general newspaper and most people will have a general understanding of what you’re taking about. Because when gaming reaches that benchmark when it’s reviewed as any entertainment would be to a world that thinks of games as a regular entertainment, that will be when specialist press will need to either become where you would look for high quality gaming reviews, or appeal to specialist or niche audiences within gaming.
December 18, 2008 at 3:17 PM
Really need a print version of this.
December 18, 2008 at 3:18 PM
Holy shit. Erickson was right.
December 18, 2008 at 3:25 PM
Thanks for the post - this is a very interesting conversation. I'm looking forward to reading future symposiums.
Jeff's appraisal of review scores' purpose as shortcuts for an audience unwilling to think critically about a reviewer's prose seems like a bigger discussion and, possibly, the real root of this conversation.
Is "games journalism" shackled by its history and its younger audience? Why is it that thoughtful conversation about games is much more likely to be found on blogs like this and in British gaming publications?
December 18, 2008 at 3:28 PM
What does one of the "normal" people (as Dan Hsu describes us) feel about review scores? From someone who has read reviews since 1989 for purchasing decisions to now, my attitude has changed
At first, review scores have become the litmus test of a game that I am on the fence about or now nothing of. I want a review to answer the questions of a cynic, a skeptic or just a confused gamer if the game is worth one's time. It is especially true for sequels, sports games and now, all those Activision games that get once a year treatment.
Reviews are no longer the only source that influences my purchasing decision, I often listen to podcasts, the vidcasts, demos and/or follow the "buzz." Even previews have time to time influenced a decision, just recently, Shawn Elliott's Far Cry 2 preview spark an enough interest to buy the game. Stephen Totilo's daily dairies have become like a book club, as I play similar games, for example, I experience the same frustrations that Stephen faced in Dead Space.
Reviews will never go away, as Dan Hsu points out, "It’s ingrained in society and it’s pointless and stubborn to fight it. People don’t always have time to read a 2000-word, well-crafted review to get inside the brain of the reviewer. For most folks in this short-attention-span world, that “4 out of 10” usually says more than enough." The problem with Review Scores are not the scores themselves, but the fact that one person can never review the bulk of weekly game, as Roger Ebert does with the movies he watches for the Chicago Tribune. The last point is better reserve for the next topics.
December 18, 2008 at 4:14 PM
Reviews for the most part are completely pointless, at least for the enthusiastic gaming consumer, which I'm sure anybody on here reading this is. I don't need your review to tell me that Metal Gear Solid 4 is going to be worth playing. I also don't need your review to tell me that Grand Theft Auto is going to be amazing as well. To be honest, the only time I look at reviews are for the games that don't have the marketing of these mega titles. In this day and age where games pretty much sell themselves, I rarely if ever need a review to merit my purchase.
Then of course there's the possibility of being burnt by the review process. Far Cry 2 got a B+ from 1up, 90+ scores from other sites and what not, and I ended up really not liking that game much at all. What I want to see in critiques is a critic's ability to point out the design of the game, how it compliments its other parts, what certain decisions of the designers worked, what didn't. Why the game was designed the way it was, and how it affects a persons experiences of it. Mainly I just want to see critics question the way the game was design, instead of simply absorbing it and reflecting it back to consumers. Maybe this is a lofty claim, I don't know, but I rarely if ever seem to get this. I do think reviews are needed in this industry, but I just wish people would lend an ear to critiques of games as well.
Why can't we just have both? A review score for a game upon its initial release to the market, then when its shelf life has diminished and there's a sense of detachment from the product, then the criticism can kick in. Idk, perhaps this logic is backwards but I think after the consumer has had a chance to play the game, perhaps constructive conversations can then start on message boards. People just invest too much in a titles release, they need time to sit on it just as much as reviewers/critics do...
Anyway, hope that all made sense...
December 18, 2008 at 4:46 PM
It seems to me the internets makes this discussion a bit irrelevant (albeit more possible and more practical.)
I'm skeptical along the lines of Totilo.
There's plenty of more in-depth & "alternative" coverage of games to be found on blogs, forums, small sites and podcasts.
For anyone wanting that type of content you can find it. Or you can create it.
So it doesn't seem to matter if the big mainstream sites & mags are heavily score focused, publish nutritionally-lite reviews and broker exclusive reviews, etc.
Nowadays, thanks to Al Gore, the thirstier of us gamers have plenty of alternatives.
December 18, 2008 at 4:59 PM
Thanks so much for a thoroughly deep and fascinating discussion.
There was one point touched upon that still left me thinking though... there seems to be a dichotomy in all this between writing for the enthusiast audience and writing for the "average" gamer (not casual just... less hardcore). The purpose for the score, in my experience, is taken differently by these groups. Hardcore gamers are the kind you generally find on message boards defending and refuting the score but, as mentioned, have probably already pre-ordered or will play the game regardless to be part of the discussion. The gamer who is less invested in being a part of the society of games discussion/review/news, however, will probably use it as an aid to his/her buying decision, if they are aware of it at all.
If this is the case, the score plays a dual function of ranking the quality of the game and recommending the game for purchase, which I don't believe are the same thing. It's little wonder then that they provoke such a wide range of opinion. If a writer speaks to the value of the game for the kid who buys 2-3 games a year they ignore the use of the score by the person who buys 10 games a year and doesn't need that return on investment from every game and vice versa. Certainly these two goals can overlap but I don't think it's possible to consistently serve both masters.
By the way, I almost never comment on blogs or reviews but the quality of this symposium, even at this early stage, is so satisfying I couldn't help myself. Looking eagerly forward to the next installment.
December 18, 2008 at 5:11 PM
"which makes me wonder if we shouldn't be trying to create more accessible, readable text, but that's probably a whole 'nother issue"
Yes.
December 18, 2008 at 5:24 PM
"Unless I'm missing something, that's quite an indictment of a portion of your audience. Why would you write for people who won't or can't read an entire review?"
Because those people generate ad revenue, which pays wages. Do you live in the real world?
Not trying to offend, but really, we don't all have the freedom to write thoughtful stuff for people who actually appreciate it. Someone has to write for everyone else.
December 18, 2008 at 5:31 PM
Sorry, I should also add that despite my reservations, I did find this very interesting and I'm looking forward to the next part.
December 18, 2008 at 5:44 PM
This symposium reinvents the compelling wheel and truly is greater than the sum of its parts. However the graphics are last gen and there is no sound. 10/10
:P
Just wanted to say thank you very much to everyone who is participating in this. There is allot here & I am going to read through this again.
December 18, 2008 at 6:49 PM
I hate scores because they're too easy. Being able to cross-refrence a 6 vis a vis a 7 is a comfort, I submit, *only* due to it's immediacy. There isn't any subtlety, philosophy, or art applied in strictly numerical evaluations (though Gillen's thoughts on 100-point scales is incredibly thought-provoking for me) and while I can't for a moment dismiss the commercial applications of a publication which focuses on scoring reviews I yearn for a space where the paragraph replaces the digit. I think a continuing sign of a medium of expression in it's adolescence (I think we've at least moved out of our infancy) is the lack of a truly viable space in the likes of what Harry Allen referenced. There's an intellectual and--dare I say--spiritual need for such an endeavor and that is where I see at least one of the "problems" Totilo is asking for; right now we've *only* got review scores. There isn't an adequate counterweight.
December 18, 2008 at 8:05 PM
I appreciate all responses. They will inform future questions.
December 18, 2008 at 8:56 PM
That took a while to read, and after burning my brain on that all I can really say is, fantastic.
I wonder, though, what significance you all see in the increasing popularity of podcasts and their influence on reviews and what readers take from them. Seeing as most of the contributors to this symposium regularly or at least occasionally appear on reputable podcasts, I'm sure there's a little something to be said. As for myself, I've found that in the past year or so that I've more and more been basing my purchases what's being talked about positively on my favorite podcasts. If a game is on topic for weeks surrounding it's release, and is constantly being raved about, that means massively more to me than a 10 ever will, and by the same token when a game is regularly mentioned I'm more likely to check out a few reviews even if I would have otherwise dismissed the game.
I don't know if I'm alone in that, but if I'm not I'd like to read what the symposium-ers (Is there a word for one who contributes to a symposium? I'm having trouble finding an easy way to refer to these fine folks.) think might be the reason behind that (I've got an idea or two about why it's so in my case) or whether they place any importance on podcasts at all for that matter.
I look forward to the future postings.
December 18, 2008 at 11:01 PM
My favorite part of the above unreadable wall of text has to be everyone avoiding the obvious answer. People want review scores because any halfway decent writer wouldn't be writing about videogames for Christ's sake.
The job consists of putting a number on a review, you should have a 10 in 10 chance of getting this right before dedicating your time to fart/boob jokes on the screen captures, yet some still manage to fail. How often can you write "it depends on your tastes" (spoiler: every time) before you sit back, stroke your chin, and look at your legacy of awful reviews before just deciding to finish college/snap and start killing people?
"I hope there's an anecdote about the reviewers terribly interesting life in this one!" Do you recognize this quote? That's because it's never been said, ever. Fight through the tears, think of how valid an opinion on something you can't be bothered to finish is, and just give it a 7 already.
If someone wallpapered my house with this I would burn it down before reading it.
December 18, 2008 at 11:26 PM
Dear sir, may I recommend the following past time? http://i42.tinypic.com/2yod4dv.jpg
December 18, 2008 at 11:59 PM
I feel that reviews are generally too eager to gush, mostly because they are written around the same time as the game itself.
It makes sense to do it that way, but think of all the cases where the hype has died down. When people have looked back at Bioshock and gone, “You know, maybe it wasn’t the most important game of all time” or Oblivion which went from “Oh my God this game is sooooo amazing” to a whopping about-face “Holy shit this game is terribly broken and emotionally sterile to its very core” It’s this weird nerd high that goes around. The same one that had everyone convinced for about a week that Star Wars Episode 2 was a good movie.
9/10s are handed out like candy on Halloween. Somewhere along the line it all just becomes a blur of hyperbole. Plus, and no offense, but most of the time the writing is so bad and obvious that one glance at “9/10″ and you can predict the details through intuition so what’s the point of reading on? Oh, you say it’s really really good but has small flaws in a few key areas that keep it from being perfect? Brilliant!
To tell the truth, the only people who really, genuinely care about reviews are the doofs who argue about it on the internet and are worried only about having to do damage control over a low score and thus protect whatever sliver of self-worth they get from that. I’d say far more “real” people get their purchasing decisions from personal recommendations than reviews.
I’d greatly prefer games to be “read” instead of “reviewed”. Something along the lines of a dissertation on how moving the Fallout series from an objective, detached camera to a subjective one has all but ruined the subversive atmosphere. (Which is true, btw) It’s just something you can’t sum up in a tidy score and does a much greater service to the people who read it.
December 19, 2008 at 12:33 AM
Thanks for a most interesting read.
Regarding scores on reviews, I often use them along with the reviews name to decide whether I’m interested in reading the text. The score is often an indicator of what to expect and to be honest it’s when that deviates from an average when I’d be interested in reading. I’ll definitely agree with Kieron’s point about hyped games being 9/10 and then losing points and others starting at 5/10 and gaining them. I use that metric internally to decide whether to read. Learning what it is that would make a game lost those points or conversely gain them is a matter than can only be learnt from reading. Seeing the expected score can make even a small review a chore to read. I didn’t feel the need to read anything many reviews about GTA IVs 10/10s, 5/5s or A+s . I was expecting a gushing report about how everything was amazing and it might be flawless.
The other problem with reviews like those, is that coming back after the fact the reviews, to me, felt incorrect. Personally I’m not a massive fan of GTA IV. The opening moments and up to half way through I was with every reviewer but after that I felt the game lost its way, dragged on and stopped being fun. The only place I found this sentiment I felt was in the GFW podcast by Shawn Elliot.
Podcasts are where I get the majority of game opinion from. Those “what we’ve been playing” sections are great for getting a feel for a game and the reviewer. As I mentioned at the start the reviewer is key for me to start reading a review. Learning what things interests any of the reviewers and how their viewpoint compares to mine influences what I believe their score or text could mean to me. Review text though, would be an extension to the discussion given about a game on a podcast.
For example, learning how Ryan and Jeff of Giant Bomb were put off immediately by the first person perspective of Mirrors Edge meant that I knew I personally wouldn’t be able to agree with their assessment of the game as our views severely differ.
I use podcasts for learning about a game because I can do it while I’m doing other things, because there will be a discussion between people about the game and because you don’t need to put a score at the end to know what the person is thinking.
December 19, 2008 at 3:05 AM
Shawn and N'gai touched on this briefly but I think that on some level this critique misses the forest for the trees.
First, listen to this:
http://feeds.wnyc.org/~r/radiolab/~5/143890856/radiolab081407pod.mp3
I was listening to Radio Lab a while ago. They had a great segment on emergent thought in ants and humans. Essentially, if you look at an ant colony, there is no leader. It's braindead. And yet if you get a bunch of ants together, they're suddenly smart. Individually they are stupid, blind insects. Together they do infinitely complex tasks that most humans couldn't do.
The same thing, the program explains, happens when you get a bunch of humans together. If you get enough people in one room, ask them to guess how much an ox weighs or how many jellybeans are in a jar you'll find some thing funny; The average of a lot of stupid guesses is more accurate than any educated guess made by an individual.
So in terms of game reviews, the group may be far more knowledgeable than any single person. I'm not just talking about journalistic groupthink or the metacritic score either; I'm talking about kids yammering about the DS on the playground, about the recommendations of minimum wage EBGames employees and about the chatter of people that haven't owned a console since Atari. The value of a game is not determined on a microcosmic level but a macrocosmic one and at the end of the day we shouldn't worry about a review by in isolation, we should worry about the discourse we're having and what influence that discourse has on the group as a whole.
So despite how much I love reading thought-provoking reviews, we need to realize that this is not a solipsistic medium - this is a choir of opinions leading ultimately to a conclusion. If we're going to get anywhere, we need to be conscious of that.
But then again, what do I know? I'm just an ant.
December 19, 2008 at 3:14 AM
As a total wannabe, writer, this conversation is inspiring on several levels.
I think most of the issues involved with reviewing video games come back to the idea of reviews being treated, by dev. studios, game publishers, or magazines, as total commodities.
The utter lack of transparency that most outlets have regarding the hoops they have to jump through to obtain exclusive reviews contributes to a severe culture of mistrust towards those reviews among those informed about the review process. Sure, we know it's an unfortunate necessity, but that doesn't make me wonder any less if those early scores are valid or simple money-makers.
Ultimately, though, the scoring process is just one symptom of an entire culture centered on game reviews that needs to be overhauled. So what if you take a 10/10 score off a game that doesn't actually deserve it, but got one based on the fact that's is a hyped AAA title? Does that mean that the review is suddenly more valid, that the flaws glossed over to give it a 10 are still not ignored in the text?
I don't think the problem ultimately lies in what scores games are given but rather in what appears to me, as a semi-well informed reader, to be the process behind arriving at those scores.
I know this reads like a long-winded way to say "scores aren't the problem, you guys just write shitty reviews," but that's really not what I'm trying to get at. It just seems to me that, for whatever reason, a lot of games are not judged equally. This is touched on by you guys in your discussion about having preconceptions before you begin reviewing, but it seems to be way more prevalent in reviews than the symposium makes it out to be.
December 19, 2008 at 8:49 AM
Holy Holy! Fantastic read (and I'm only halfway!).
Everybody balances everybody out. Will continue reading ASAP!
December 19, 2008 at 10:10 AM
Stephen Totilo, MTV News Multiplayer: Have any of you come across gamers who won't buy a game they were curious about because you gave it a 7 and not a 9?
Well, speaking as someone who doesn't buy many games at $60 (the theoretical target audience of reviews), frankly, if a game I'm curious about gets a 7, I'm not gonna buy it for $60. Instead, I would wait six to twelve months, buy it off the bargain shelf at EB or Future Shop for $20 or $30. Crackdown and Assassin's Creed both had flaws that would probably bother me at $60 that I really didn't care about at $30, because I still felt I was getting my money's worth.
I think that's really what I look for in a review: am I going to get my money's worth? And in a lot of cases, I approach it from the perspective of how much gameplay I'm going to get out of it. Portal was one of the best three-hour gaming experiences I've ever had, but if I'd paid $60 for that alone, I might have felt ripped off; for $15 or as part of the Orange Box, I think I actually got more than I paid for, because of the fantastic writing that backed up the solid gameplay mechanics. At the other end of the scale, ESPN NHL 2K5, Burnout 3, Guitar Hero III, and Diablo II are all games that I've sunk hundreds of hours into at various times over the years (to say nothing of the almost two years I lost to WoW), and one of those was $30 off the bat. I'm more inclined to pay $60 for something that I either know is going to last me a long time, or that I can reasonably assume I'm going to enjoy immensely, based on past experience with the genre or developer*. Games that get a 7 are games that I may be interested in trying out because they sound interesting (or because they're from a genre/developer I know/like/trust), but because of my student budget, I'm just not willing to invest in as an early adopter for fear of disappointment.
It occurs to me to wonder now if that makes me part of the problem: I tend to avoid online forums like the plague, having passed the age of 14 many years ago, but maybe I put more stock into a review score than it really warrants. That being said, in recent months, I've gotten into a number of gaming podcasts (Giant Bombcast, 1UP Yours, 1UP FM, GFW Radio/LAN Party, Evil Avatar Radio/In-Game Chat, Australian Gamer Show), and the more in-depth discussions that occur there have changed my perspective a bit on review scores and what I might like to invest my minimal funds into; besides, it's easier to listen to a podcast on the train than set aside fifteen minutes to read three or four reviews, and you often get more fleshed-out, "real" perspectives by way of interaction in the process. To borrow Totilo's own example, Too Human got excoriated by the press for its repetitiveness, awkward controls, unfulfilled story concept, and numerous gameplay delays/complications (the 20-second resurrection animation and the menu system) but the guys of EAR, as it was known at the time, all loved it for its co-op aspect and loot-whore reward system, and it occurred to me that if it was basically a sci-fi Diablo clone, I might like it, too, even if I have no one to co-op with myself. Because of the differing opinions, I'm inclined to download a demo or wait for the price to hit $30 before buying it, but I'm more open to the concept of a purchase than I might have been a year ago, which is something.
So to bring it back to the subject at hand, I wouldn't say I'd never touch a game if it scored less than an 8 or 7.5 or whatever, but I'd definitely be more wary of investing a lot of money into it for budgetary concerns, and that the "shit filter," as one person called it, does work to help me choose between games I'd buy for $60, games I'd buy for $30, and games I wouldn't buy at all. As long as it's understood that that's what the score is for, I think I'm fine with using it as a proxy for that sort of decision, in the most basic sense, but at the same time, nothing can replace an honest-to-goodness discussion, and I do think a lot of people tend to avoid that discussion because it might assault their preconceived notions (though that's no doubt a subject for another day, and one that's far from gaming-exclusive).
* - Ignored in all this discussion, for obvious reasons, is the store-packaged deal: I got Doom 3, Knights of the Old Republic, and a SoundBlaster Audigy XL with Hitman 2, all for $70, which is what Doom 3 would have cost by itself in Canada at the time, all for being one of the first ten people into Future Shop that day. KotOR turned out to be the winner of that four-item package, but how was I to know that at the time? I went there to buy Doom.
December 19, 2008 at 12:28 PM
Regarding the "Rolling Stone" rumor mentioned numerous times here. I remembered this entry from veteran rock critic Robert Christgau's website:
http://www.robertchristgau.com/xg/cdrev/kimya-rs.php
In the postscript, he explains how a four star review was changed to a three star review by the editors of Rolling Stone, but his tone makes it sound as if this was a rare exception. At least as late as 2002, reviewers were submitting their own numerical scores for that publication. So, uh, there you go.
December 19, 2008 at 9:14 PM
Nice work, just reading through it now and may comment later. Just a note though, could you add some anchors to the ** you've put in so I can bookmark my position as I dip in and out of this.
December 20, 2008 at 12:28 AM
Great point, Papapishu.
December 20, 2008 at 11:25 AM
Well, now I've read this pretty much top to bottom.
I have to say that I right out disagree with N'Gai's position that scores are harming games media, and I've decided that I also disagree with the degree to which certain people (particularly Dan) are getting worked up about this issue as a whole.
Frankly, I rarely read reviews anymore. I pick and choose posts to read off of Joystiq and Giant Bomb, subscribe to a few podcasts (Joystiq, LAN Party, Retronauts, skim 1Up FM and I listen to episodes of 1Up Yours until Garnett gets liquored up and goes apeshit yet again). I might even download a demo, but that's getting pretty rare as well.
I feel like I'm still getting a reasonably good feel for what is going to be good and what isn't. I can't exactly explain how, but there's an overall tone of popular discourse aside from whatever might be happening on "popular message boards" (which I don't read) that fills me in on what I should be paying attention to.
I've also found that in lieu of relying on scores, I just "buy-in" at a price point that's more appropriate to the level of risk that I'm willing to take on a game. If I'm not sure I'm going to like a game, I buy it when it's available for twenty dollars (Army of Two). If it's a sure thing (in my mind, a good example was Rock Band 2), then I'll shop around, but I'll still pick it up at launch.
I wonder if the elephant in the room is that no one reads these reviews anymore anyway, except juveniles, marketing people, and other videogame reviewers.
I couldn't care less what you reviewers are doing, I'm not relying on you to tell me what's good and what isn't in your reviews, I'm getting it from everything else you do and say.
December 20, 2008 at 10:52 PM
Great discussion so far. Not sure if you meant for this to turn into a sprawling discussion about review scores in general, but I think it needed to, at least initially. I'd love to see a panel come from this, even though there's no appropriate venue for that right now.
To add my own 2 cents, I don't think you guys will ever find what you're looking for. Several different notions of what functions these scores perform emerged throughout the discussion, but the one thing agreed upon is that people need a bottom line. Consequently, publications need to provide a bottom line. The only question is, how can this be justified?
It seems like reviewers want to reconcile meaningful scores with thoughtful review (and and an audience who wants thoughtful review) within the context of a feasible business model. Do we have any proof that such a notion is anything more than modern day alchemy?
I'm personally a fan of many of the people involved in this symposium, and I appreciate all the thought you put into your reviews. But it's been made clear by Metacritic, pressure from developers and publishers, and responses from the enthusiast community that overwhelmingly, the most important part of your job is the score you assign to games. Truly shitty, but perhaps in this case, it would just be best to let sleeping dogs lie. You are paid to contribute your thoughts on an exciting medium. If people only care about the score, then they'll get exactly what they want.
December 21, 2008 at 5:21 AM
On accident I posted this query on the previous blog, but I was wondering:
What would the discussion be like if a bunch of movie critics round-tabled on the subject of review scores?
I appreciate the distinct personalities and sharp insights.
December 21, 2008 at 12:12 PM
I was wondering if you could get Jeff Green in on the next part of this symposium. I think he was serious about wanting to be a part of this on the GWJ podcast, Shawn (yes, I listen to way too many podcasts and know entirely too much about everyone on here.) I think it would be interesting to hear him chime in as well.
December 21, 2008 at 3:35 PM
Having only read the first 3 so far... I figured I'd throw my 2 cents in... in point form, because no one could be bothered to read this anyway.
-scores suggest infalibily, in so far as they are set values (9 or 10 out of 10 is not debatably in its worth). People percieve numbers as facts. Letter scores are automatically converted to numerical representation in the brain.
- because of the previous, the review gains exclusivity and authority. What should be an opinion becomes a fact: game x is 9/10, so how can 9/10 be 6/10?
-a term paper is marked by a lecturer with a degree or three, an olympic gymnastics competition is judged by professionals; yet any tom, dick or harry can pass out a game score-> anyone can claim authority, and because the world of the internet is grounded in anonymity anyone can claim expert status.
-Print represented an expert status in games. If you worked for CGW your talents and discression were vouched for by the magazine. On the internet everything is temporary and only valuated by how many clicks it could get, and controversy gets clicks.
Im rambling now...
for clarification, I study Arts at Sydney University; majoring in political science, sociology and digital cultures. So while my bullshit may be bullshit, in RL i could at least back it up.
December 21, 2008 at 5:43 PM
Marc, we certainly are reading every comment here. Thanks for your feedback and thoughts.
December 21, 2008 at 8:04 PM
Marc, good insights but you said:
"a term paper is marked by a lecturer with a degree or three, an olympic gymnastics competition is judged by professionals"
and we must remember that "professionals" still have bias and personal opinion. A degree or job doesn't really make their insights more poignant or real.
So there's a question: What constitutes a professional in the gaming world?
December 21, 2008 at 9:05 PM
Just a quick point. I like video game scoring and despite its problems I find them very useful.
Sure they are inflated by the various measures stated by the panelist which is why sources like metacritic are a great moderators.
I personally use game scores as a Table of Contents. You see I'm still going to purchase certain game whether the press gives them good scores or not. My own opinion is just as valid as theirs, in fact its more valid because they aren't me and likely have different tastes. But review scores allow me to discover games I wouldn't normally pick up.
Not having a score would mean that I'm not going to read your review for "Generic Game Name Here". However giving it a score shows me instantly that this may be something I might be interested in. I will then read your review, read other publications reviews and watch the video reviews.
Honestly if you didn't post a score I'd likely not visit your site. I am looking for a gauge on the products experience (as you saw it), and at that point, if I'm interested, i will read your opinions because that's where the real heart of the subject will lie.
December 22, 2008 at 9:21 AM
Thank you for this very impressive and valuable collection of thoughts.
December 22, 2008 at 10:30 AM
Shawn, thank you for assembling such a concise yet varied discussion of this topic, which has been so ill-researched in the past. Thanks also to the contributors, whose interplay makes this feature diverse and thought-provoking.
That said, I find myself agreeing most with N'Gai: reviews do not need scores. When we look at the assignment of reviews as an art in its infancy and don't hold fast to established norms, it becomes clear that the advantages to eliminating numerical designations across all critical outlets are significant and plenty.
The issue of garnering exclusives or pleasing advertisers fades immediately; if there is no competition based on scores whatsoever, coverage will be given to whichever outlet the publisher deems most appropriate to reach the game's intended audience. This has the added benefit of encouraging the largely homogenized gaming press to specialize, be it by genre, platform, or whatever else divides the gaming public (Mitch Krpata's taxonomy of gamers comes to mind).
What's more, freeing reviewers from assigning scores allows them to be candid and straightforward, rather than sugarcoating their criticisms to fit an overly inflated score. The fun-to-read very high and very low reviews are evidence of this problem: in these situations, the reviewer doesn't have to please both his own conscience and the public, resulting in sharper opinions and better writing. The in-between reviews are littered with this conflict, and removing scores from them would solve this problem cleanly and efficiently. If the reviewer truly is conflicted about a game, let him say so, rather than trying to reconcile his beliefs into a number. In this way removing scores will improve game review writing, and give it a greater significance to the non-gaming public.
Of course, the implementation of such a system is incredibly difficult, as every sect of the business has become dependent on scores in some capacity. What is required is an outlet free of commercial obligations, so as not to be bound to please buyers instantly and unconditionally, to drop scores altogether. Theoretically, the improved writing quality will bring prestige to this maverick outlet, and others will emulate. It's a dream, yes, but I think one worth believing in.
December 22, 2008 at 1:04 PM
What an excellent and thought-provoking discussion; I commend you all for your participation.
Yet in this entire discussion, there are two absences which I find troubling:
-The comments offered by the primary participants are all originating from the same basic perspective, that of a reviewer. While these insights are varied and interesting, I would have liked to see some representation from the other parties involved: a developer, a full-time PR guy, a common reader. What are those people looking for and how do they view review scores?
-Tom Chick (and others) mentioned the "context" under which he writes reviews. That context is obviously unique to every reviewer, as it is a culmination of all past experience and personal prejudice.
Yet there has been no mention in this entire discussion of the simple fact that the /reason/ a single review score is an innapropriate expression is that it is devoid of all context. Did the reviewer give that racing game a 7.0 because it was buggy or did they perhaps just prefer a different genre?
Since there is no way for the reader to determine /why/ a game is scored the way it is (short of reading the entire review), there is no way for them to evaluate on score basis alone whether or not that number holds any real meaning.
(Case in point: I saw a Metacritic score for Assassin's Creed and elected to hold off my purchase; later I read some reviews and found that while the scores were low, the positives of the game were right down my alley and the negatives were for things I didn't really care about.)
Additionally, it is clear from all the responses on this issue, that reviewers clearly each have a different focus when it comes to actually giving a review score.
So my question for everyone is this:
What can be done to to create a consistent & concise rating system from which all parties can derive meaning? Or is it futile to even try?
December 22, 2008 at 4:17 PM
[Disclaimer: I realized that after finishing writing this comment, I appear very score focused, but this was not the intent. I am merely concerned with certain trends very prevalent in games reviewing. This post has nothing much to do with games criticism, which is a separate topic.]
I think a major problem with videogame reviews vs. reviews in another medium, such as film, is that there is too much consensus between different outlets' videogame reviews for me to take them seriously.
There are few movies that garner truly universal acclaim or get consistent scores for nearly every major critic yet in games, it seems to be the rule. Amongst film reviews, I can almost always find a reviewer that liked or disliked a movie for many of the same reasons I did, but with videogames, this is quite often not the case. I don't need or necessarily want to just read someone's restating of my own opinion, but I find it very strange that not one professional reviewer exists who feels even remotely like I do about a game.
For example, a quick glance at the review scores for Little Big Planet reveals that no critic among those aggregated on metacritic gave the game anything lower than an 8/10 or equivalent. While I generally object to metacritic style aggregation, this seems absurd.
Regarding LBP, there a quite a few people on internet message boards that I frequent as well as people in my group of friends who play videogames, who have expressed dissatisfaction with the game similar to mine: the core game isn't that interesting, and no amount of level creation or level editing will amend that, and yet despite being a commonly encountered criticism of the game, no professional reviewer seemed to have these issues.
It's things like that that make reviews so suspect at times. If the gaming community has a full range of hypothetical scores they might award a game, why do professional game reviewers' opinions not reflect anywhere near this range, especially for certain, very high profile games?
Papapishu makes an interesting point, but it still implies that aggregation of opinion will yield some kind of scientific fact, which for all his interesting points about group behavior, is absurd, no matter how it is stated.
Sometimes reviews truly seem to encompass the full range of opinion; divisive titles like Mirror's Edge or Assassin's Creed being such games, but when that sort of range is the exception, how can I possibly rely on reviewers?
December 23, 2008 at 5:48 AM
It seems to me that the problem in reviewing games is the inherent subjectivity of the game's quality. I know that there are certain games that, for one reason or another, I can't get enough of, but if I think about them critically they wouldn't get a "10" from any sane reviewer. I have probably logged more hours in Puzzle Quest than any other game, but it does not have nearly as much time, art or storytelling put into it as, say, Half-Life 2, so which one is the "better" game? I would say that Half-Life 2 is the better game in terms of gameplay, but Puzzle Quest is the better game in terms of replay-ability and mindlessness. Shawn Elliott can not say enough good Company of Heroes and, while I respect his opinion, I know that CoH is not my kind of game so his opinion is moot. Because of this subjectivity reviews must rely on seemingly quantifiable aspects of the game. Graphics, sound, controls, etc. The problem is that, in the end, these aspects don't make or break the game. As gaming technology gets better it is expected that the graphics will improve, but if the antialiasing is a little janky, if the fingers of a model aren't articulated, if the blood splatter isn't realistic, does it actually affect how much fun the game will be? The answer should be no, but there are gamers (and reviewers) who take minor graphical flaws as personal affronts. Clipping is tantamount to punching their dog.
I really enjoyed the conversation in a GFW podcasts between Shawn Elliott and Sean Malloy about BioShock. So rarely do gamers get an intelligent conversation about a game, rather than a one-sided review that usually boils down the graphics rather than the content. Shawn and Sean discussed things that I had not thought about while playing the game and, on the next run through, I was more aware of the underlying themes and perhaps even a bit more critical of the binary good/evil aspect of the game.
I think what should be important to gamers and reviewers is how much fun, thought inspiring, or interesting a game truly is and not any so-called "niggles" that can be found. Attaching numbers to a review is arbitrary and cannot be considered a valid reflection or substitution of the review's content. Scores are the TV dinners of game journalism, they may satiate a basic desire, but they offer little real nourishment.
December 23, 2008 at 9:06 AM
I wrote a blog post influenced somewhat by this discussion i think its pretty interesting but who knows maybe it suck...check it out
http://www.1up.com/do/blogEntry?bId=8976319&publicUserId=5753283
December 24, 2008 at 8:50 PM
I treat scores as indicators of the value (to me) of the reviews they accompany, and not as a means of judging games. They could be replaced perfectly effectively with a recommendation system, showing me the reviews I might like to read, without reference to a number.
When reading a magazine, I look at the scores first and will jump to the text for the reviews with a high score, or with a score that differs from my expectations. This helps to focus my attention on reviews that might have something new to tell me about a game. Scores of 10/10 for a game I am certain to buy, or 6/10 for a game in a genre I don't usually play, will not catch my eye.
December 25, 2008 at 5:28 PM
much reading was done, and still the question remained unanswered... why so low?
December 26, 2008 at 11:47 PM
I've got a bit of a dissenting opinion as far as this Symposium goes. I can appreciate the overarching focus, but I think the methods are somewhat lacking. I've posted some of my thoughts on my Wordpress blog; rather than reproducing the entire thing here in the comments, I think it would be best to just provide the link. I'm curious to hear what some of you participants think in particular.
Blog link
December 27, 2008 at 11:04 AM
This comment has been removed by the author.
January 4, 2009 at 1:02 PM
Shawn Elliot, you've raised the level of debate and increased the amount of intelligent discussion around gaming. Thank you.
January 6, 2009 at 6:31 AM
I was going to write a long comment but it was fillied with stuff everyone here already knows. Instead I'll just say thanks for the virtual symposium. I found it far more interesting to read this discussion than almost any other videogame related article in recent memory.
Someone call Leo Laporte and get advice on a virtualized podcast similar to This week in Tech but starring these veteran's of the gaming world and I'd listen religiously. I might even pay a tithe.
January 13, 2009 at 1:53 PM
Where do you Symposi-ites fall on this:
Critic in Exile: Is It OK to Finally Admit That I Didn't Really Like Fallout 3 All That Much?
January 14, 2009 at 7:20 AM
That was a very interesting discussion. I love peeking into the mind of a games journalist.
At the age of 40 and still a gamer (although I don't have the time to devote to it that I'd like) I will say review scores themselves are less important to me but at the same time I want to see them. If for no other reason than sometimes it gives me a reason to dig down into a review to see why it got this score.
Case in point was Tom Chick's review of Red Alert 3 on Crispy Gamer. I wasn't even looking for the review but came across it and decided to read it because I was curious about the "Fry It" score. It seemed other sites and magazines were rating it pretty high from just casually looking around. So I read his review and very much enjoyed the depth Tom got into and not holding any punches on his opinions of the game and its flaws as he saw it. I can't judge the review other than to say I think he backed up his score. I think that is the best and ideal relationship between a score and body of a review. Can you back it up. Especially given a scale like Crispy Gamer's. A reviewer can't fudge things. There is an obvious demarcation between Buy It, Try it, Fry it. That is a scale that works perfectly for me because I feel it makes me want to read a review to see why it got that grade. I've gotten to where I hate the 10 point or 100 point scale. Even the 5 point. I think 1UP made a great decision to go to letter grades. I prefer that or Crispy Gamers scale to anything else out there.
In the end though, who cares about the score. I mean really, as critics and journalist, what I want most from you is the text. The in depth review of a game either as buying guide or as a critique. They can either be combined or seperate things.
What I would love to see is an outlet that does an in depth critique and investigation into a game after a game has come out. After the review/buying-guide. Pull no punches. Take the time to really dig in. I know of no outlet that truly does that. It is almost as if a game is forgotten about after it is published and reviewed. The only post mortems out there of a game I know of are the ones that consist of an interview with a developer after the release or a write up the dev after release. While both enjoyable as a peek into the mind of the dev, they don't serve much of a journalistic or critical importance. Something I'm looking for more and more as I get older.
Another case in point but a bit off topic. Nothing to do with reviews but about depth. Geoff Keighley's Behind the Games series he did for Gamespot years back. I enjoyed those to no end and miss those very much. Maybe the tight PR machine that surrounds games now would never allow a reporter to get that deep into the workings of a developer but that is something I yearn for in games journalism today but haven't found since then.
January 15, 2009 at 12:24 PM
What a brilliant read. I've shown this to all of my friends that are involved in gaming one way or another, and they agreed that this is absolutely amazing. I don't have to contribute anything to this symposium myself though.
What struck me as odd was that I read everything in one session. I am not used to reading anything on my computer screen, but I was completely hooked.
I wish you the best of luck with your endeavours at 2K Boston.
January 15, 2009 at 7:11 PM
Greeting! What do you think of wow gold? I always lack of wow gold to buy some great item for my char before the wow patch 3.1 and I need to do wow power leveling for the cheapest wow gold or I have to search cheap wow gold on google search. The world changed since the 3.1 patch released. Short of buy wow gold for me wasn't so high frequency now. And I’m curious why this situation happened? The items became cheap? Or we can farm wow gold much easy? Hope someone can give me an answer.
April 10, 2009 at 2:09 AM
電報opiopuy探偵entyouyouliaolaoren
qwowertofor名刺livehiskypercou名刺作成xieweishilookenw
April 14, 2009 at 1:37 AM
govlog
桜の花
探偵
電報
名刺
オーク
フレッツ
腰痛
不動産
名刺作成
April 14, 2009 at 1:40 AM
lorockmikrombardgovloggovlog 03170govlogoverAll ofraidagain
Fredericksodaswill be set freekeeping
April 14, 2009 at 1:44 AM
Do you know Archlord gold? I like it.
My brother often go to the internet bar to buy Archlord money and play it.
After school, He likes playing games using these archlord online Gold with his friend.
I do not like to play it. Because I think that it not only costs much money but also spend much time. One day, he give me many cheap Archlord gold and play the game with me.
I came to the bar following him and found buy Archlord gold was so cheap. After that, I also go to play game with him.
Do you know cabal online alz? I like it.
My brother often go to the internet bar to buy cabal alz and play it.
After school, He likes playing games using these cabal gold with his friend.
I do not like to play it. Because I think that it not only costs much money but also spend much time. One day, he give me many cabal money and play the game with me.
I came to the bar following him and found buy cabal alz was so cheap. After that, I also go to play game with him.
April 20, 2009 at 10:04 PM
Do you want to know the magic of online games, and here you can get more aoc gold. Do you want to have a try? Come on and conan gold can make you happy.You can change a lot age of conan gold for play games. And you will enjoy yourself at the sametime. Playing online games can make much cheap aoc gold. Come and have a try. You will feel a different feeling. And you can aoc money do what you want to do in the online game.
Do you want to know the magic of online games, and here you can get more FFXI Gil. Do you want to have a try? Come on and FFXI gold can make you happy. You can change a lot Final Fantasy XI gold for play games. Playing online games can make much buy FFXI Gil. And you will be happy at the same time. And you can use the cheap Final Fantasy XI Gold do what you want to do in the online game.
April 27, 2009 at 6:39 PM
Have you heared about a game which you need use Entropiauniverse ped to play, and you can also borrow Entropia Universe Gold from other players? But you can Buy Entropia Universe Gold, or you will lose the choice if you do not have Entropia Universe Money. If you get cheap Entropiauniverse ped, you can continue this game.
Have you heared about a game which you need use kal geons to play, and you can also borrow kal gold from other players? But you can buy kal online geons, or you will lose the choice if you do not have kal online gold. If you get kalonline Geons, you can continue this game.
April 27, 2009 at 8:03 PM
I like holic gold very much. Since I entered into this game, I learnt skills to earn holic money. Thanks to holic online gold let me know a lot of friends. It is my habit to buy holic online money, and I get some cheap holic gold from my friends and Internet.
I like to play last chaos, because I like its name, also I like last chaos gold. My friend told me that she would buy lastchaos gold for me, and I was so happy. I do not like to go shopping, because it always spends a lot of money, but I never hesitate to buy last chaos gold. You can buy cheap lastchaos gold; it is so easy and convenient. I believe the future of lastchaos money is bright.
April 27, 2009 at 9:21 PM
To buy rappelz rupees , it is not the aim of the rappelz. It is the game which brings a lot of happiness to me. We buy the rappelz gold together. The beautiful story, we can not be forgotten buy rupees . You will not regret to have rappelz money . So I will continue to have cheap rappelz rupees , it is a beautiful fairy tale, it looks like my life.
On account of I have rohan crone , I am not afraid of the lonely. At that time I do not know the meaning of it. Later I buy rohan gold . And every time, I have no rohan online crone , I will feel very anxious. Why I buy rohan online gold , please it gives me such a warm place. At the moment, I have rohan money .
April 27, 2009 at 11:36 PM
It is the LOTRO Gold which make me very happy these days, my brother says Lord Of The Rings Gold is his favorite games gold he likes, he usually buy some buy LOTRO Gold to start his game and most of the time he will win the cheap Lord Of The Rings Gold back.
I am so happy to get some mabinogi gold and the cheap mabinogi is given by my close friend who tells me that the mabinogi money is the basis to enter into the game. Therefore, I should buy mabinogi gold with the spare money and I gain some mabinogi online gold from other players.
April 28, 2009 at 12:37 AM
Their studio is often overlooked, however, when discussing long lasting and well made games, to the great detriment of the MMO industry. With both games, it would be a true tragedy to ignore the story and fun that are available at a cost that is both reasonable and affordable compared to most other subscription MMOs. You will like the game and wakfu kama, I think.
April 28, 2009 at 1:30 AM
polo ralph lauren RED! Series of design criteria - stylish, simple, self-cultivation. For the first time in 2009 in New York Fashion Week debut, RED! Series has a new Polo Shirts On Sale series from the traditional burberry polos LOGO, LOGO traditional crocodile logo at the Ralph Lauren Polo Shirts letter above, and the new series of letters with the cheappolos logo in line to be between a red symbol of Burberry Polo Shirts.
June 16, 2009 at 9:10 PM
In this version of the ralph lauren polo, the styling and fabric is advertised as being made out of "playdry" material. Burberry Polo Shirts is another performance fabric which is intended to add luxury Cheap polo shirts, coolness, and moisture wicking qualities. Performance fabrics wholesale polo shirtswere originally created primarily for athletes and other sports enthusiasts who work up a good sweat ralph lauren polo shirts. The moisture wicking fabric in cheap ralph lauren polos draws the moisture away from the body which then evaporates quickly discount ralph lauren polos. ralphlaurenpoloshirts keep the body cooler and drier even when the athlete is under physical duress polo clothing.
June 16, 2009 at 9:11 PM
Although many people criticize ed hardy clothing for having nothing but tattoo patterns, being filled with waste and useless things, it is widely acknowledged that ed hardy clothes draws the most attention of both common young people as well as many Hollywood welcomed stars for its punk ed hardy t-shirts rock styles. As a matter of fact, in such a world where all ed hardy shirts are much like each other, ed hardy mens creates yet another totally new global sensation in fashion crowd for ed hardy womens fancy design. An increasing number of people are charmed by the magic of amazing tattoo and impressive color of ed hardy sunglasses.
June 16, 2009 at 9:12 PM
The larger that the head size is for your babolat pure drive, the more power you will be provided with. Additionally, a larger wilson k six size means a larger sweet spot which is why beginners as well as intermediate players often prefer to work with a head microgel with a larger head size. Additionally, thicker frames are also capable of generating more power as well. Smaller head sizes on prince-o3-tour on the other hand are capable of providing less power, but significantly more control cheap tennis racquets.new polos men poloswomen polosdiscount polossummer polospolo shirts whosalepolo fashionembroodered polostennis racketsclothing poloclothingedhardyshirtedhardyclothingsummer ed hardy clothingcheap shirtsed hardy brand.
June 16, 2009 at 9:14 PM
Laptop Batteries
Acer Laptop Batteries
Apple Laptop Batteries
Compaq Laptop Batteries
Dell Laptop Batteries
HP Laptop Batteries
IBM Laptop Batteries
Lenovo Laptop Batteries
Samsung Laptop Batteries
Sony Laptop Batteries
Toshiba Laptop Batteries
ASUS Laptop Batteries
Gateway Laptop Batteries
LG Laptop Batteries
NEC Laptop Batteries
HITACHI Laptop Batteries
Panasonic Laptop Batteries
BenQ Laptop Batteries
Fujitsu Laptop Batteries
July 27, 2009 at 8:39 AM
Check out my interview with Leigh Alexander. Those of you with a sensitive gag reflex may wish to steer clear:
http://digitalgigolo.blogspot.com/2009/07/exclusive-leigh-alexander-interview.html
August 2, 2009 at 5:27 PM
You are right, I believe that there will be many readers like you
Chanel Cambon?Christian Louboutin?Chanel Bags Chanel Clutches Jimmy Choo
Chanel Denim Handbags?Manolo Blahnik Chanel nappa handbags
Yves Saint Lauret?Christian Louboutin Boots Gucci Handbagsdiscount christianlouboutinManolo Blahnik Gucci Bags?Jimmy Choo? Gucci Wallets?Louis Vuitton BagsYves Saint Lauret? YSL?Christian
Louboutin?http://www.bestlouisvuitton.com/http://www.buylouboutin.com/http://www.yitingbuy.com/http://www.discount-christianlouboutin.com/
October 3, 2009 at 6:16 PM
Our company is always making effort on wholesale and retail of Air Jordan Shoes, NFL Jerseys, MBT Shoes, Nike TN and Puma Shoes Men . I would like to inform everyone likes sports apparels, we will offer discount up to 50% discount for large order, especially in NFL Jerseys, if you like these, please don’t hesitate to tell you friend, welcome group purchase in our online store.
January 27, 2010 at 9:57 PM
Do not mean bad.Thank you so much! I just want to show some fashion Chaussures Sportto all of you. I like Chaussures Sports. Here are the great Chaussure de Sport to you!
February 1, 2010 at 11:38 PM
Perfect!! You are a outstanding person! Do you want to wear stunning tn chaussures, Nike Air Rift?? Here are the most popular nike tn, cheap ed hardyand Tennis Racquet. We also Wholesale Polo Shirts and Cheap Wedding Dresses.
February 1, 2010 at 11:38 PM
gucci replica handbags
men gucci shoes
Gucci men sneakers
Gucci men moccasins
gucci women sneakers
gucci women boots
Gucci men boots
Gucci
Gucci shop
Gucci bags
Gucci shoes
Replica GUCCI SHOES
wholesale gucci shoes
cheap Gucci handbags
Gucci ON sale
Gucci Belts
Gucci small accessories
Gucci hats & scarves
Gucci wallets
Gucci Handbags
Women Gucci shoes
Men Gucci shoes
discount gucci shoes
cheap Gucci shoes
February 22, 2010 at 5:31 PM
I had never see a blog batter than this blog, I like this blog very very much.By the way, do you like my products: street lights, street lighting, street lamps, ball valves, butterfly valves, butterfly valve handles, check valves, sanitary fittings, butterfly valve handles, butterfly valve handles, solar street lights, street light, valve manufacturer, butterfly valve handles, valve manufacturers, butterfly valve handles, street lamp, solar street light, ball valve, butterfly valve handles, butterfly valve, check valve,valves manufacturers, butterfly valve handles
February 25, 2010 at 1:34 AM
Remise Chaussures Sport
nike tn requin
chaussure nike tn requin
February 26, 2010 at 11:08 PM
I had never see a blog batter than this blog, I like this blog very very much.By the way, do you like my products: street lights, street lighting, street lamps, ball valves, butterfly valves, butterfly valve handles, check valves, sanitary fittings, butterfly valve handles, butterfly valve handles, solar street lights, street light, valve manufacturer, butterfly valve handles, valve manufacturers, butterfly valve handles, street lamp, solar street light, ball valve, butterfly valve handles, butterfly valve, check valve,valves manufacturers, butterfly valve handles
February 28, 2010 at 9:55 PM
do you like my products:
Air Max
wholesale polo shirts
Tn Requin
Ralph Lauren
Chaussures Nike
Wholesale Polo Shirts
cheap polo shirts
Polo Shirts
Air Shoes
Tennis Racquet Shop
Cheap Polo Shirts
The North Face Jackets
Cheap Nike Shoes
Nike Air Rift
chaussures puma
polo shirts
cheap nike shox
jeans diesel
Tn Requin
Tennis Racquet
ed hardy
March 3, 2010 at 9:44 PM
China Wholesale
tn chaussures
nike tn
ed hardy clothing
cheap ed hardy
Chaussures Sport
Chaussures Sport
tn requin
Air Jordan Shoes
March 4, 2010 at 12:24 AM
I had never see a blog batter than this blog, I like this blog very very much.By the way, do you like my products: sanitary valve, street lights, street lighting, street lamps, ball valves, butterfly valves, butterfly valve handles, check valves, sanitary fittings, butterfly valve handles, butterfly valve handles, solar street lights, street light, valve manufacturer, butterfly valve handles, street lamp, solar street light, ball valve, butterfly valve handles, butterfly valve, check valve, butterfly valve handles
March 4, 2010 at 7:01 PM
Sean Michael Elliott (born February 2, 1968 in Tucson, Arizona) is a retired basketball player of NBA.Elliott played basketball at the Institute in Tucson Cholla, and the University of Arizona under the tutelage of Lute Olson.
sport fishing
fishing guide
http://www.fishingcostaricaexperts.com
March 7, 2010 at 9:25 AM
China Wholesale
tn chaussures
nike tn
ed hardy clothing
cheap ed hardy
March 9, 2010 at 7:02 PM
Chaussures Sport
Chaussures Sport
tn requin
Air Jordan Shoes
NFL Jerseys
March 9, 2010 at 7:02 PM
MBT shoes
nike tn
puma chaussures hommes
NFL Jerseys
Affliction Jeans
NFL Jerseys
March 9, 2010 at 7:03 PM
Can't wait to see the fruits of your labors from 2K Boston as well.
That'Good Post.
China Business
Hong Kong Business
International Business
Business Opportunities
China Manufacturer
China Market
Picture
Computer
Computer Peripheral
Electronic
That a thank you
March 17, 2010 at 9:28 PM
designer sunglasses
Wholesale Polo Shirts
Chaussures Sport
MBT shoes
cheap cell phones
Cheap shoes online
March 24, 2010 at 6:27 PM
clothes jeans
Nike Sports Shoes
Nike Shox Shoes
Chaussures Sport
MBT shoes
Wholesale Polo Shirts
cheap cell phones
cheap shoes
polo shirts
nike tn
The North Face Jackets
Tennis Racquet
wholesale polo shirts
Air Jordan Shoes
March 28, 2010 at 7:41 PM
Look,Cheap GHD Straighteners,Discount GHD Styler,GHD Outlet,CHI hair Straighteners,2010 Cheap Sunglasses,Discount Sunglasses，Sunglasses Outlet,wholesale Sunglasses,Sunglasses Wholesale. Do not worry. The urge to buy these goods will be very strong once you spot the shoes or clothes that you like. I am also a big fan for these products, although I like ED Hardy,cheap ED Hardy Clothes,discount ED Hardy Outlet,Wholesale Ed Hardy,Ed hardy wholesale, cheap Prada,discount Prada,Prada Outlet,newest Prada handbags,Prada handbags 2010, so I can understand. Online wholesale dealers have quite an array for you to choose from, including wholesale jewelry and handbags. And these are bags that bear the most famous names in the world of fashion, by the way -Prada, Spyder, Moncler, GHD, ED Hardy, designer Sunglasses, Ankh Royalty, Twisted Heart, or sayAnkhRoyalty,Ankh Royalty,Ankh Royalty Clothing ,Ankh Royalty Clothes,AnkhRoyalty Outlet. Such variety that comes at incredibly affordable prices is a treat you can't possibly pass up on.
March 31, 2010 at 12:43 AM
Look,Cheap GHD Straighteners,Discount GHD Styler,GHD Outlet,CHI hair Straighteners,2010 Cheap Sunglasses,Discount Sunglasses，Sunglasses Outlet,wholesale Sunglasses,Sunglasses Wholesale. Do not worry. The urge to buy these goods will be very strong once you spot the shoes or clothes that you like. I am also a big fan for these products, although I like ED Hardy,cheap ED Hardy Clothes,discount ED Hardy Outlet,Wholesale Ed Hardy,Ed hardy wholesale, cheap Prada,discount Prada,Prada Outlet,newest Prada handbags,Prada handbags 2010, so I can understand. Online wholesale dealers have quite an array for you to choose from, including wholesale jewelry and handbags. And these are bags that bear the most famous names in the world of fashion, by the way -Prada, Spyder, Moncler, GHD, ED Hardy, designer Sunglasses, Ankh Royalty, Twisted Heart, or sayAnkhRoyalty,Ankh Royalty,Ankh Royalty Clothing ,Ankh Royalty Clothes,AnkhRoyalty Outlet. Such variety that comes at incredibly affordable prices is a treat you can't possibly pass up on.
March 31, 2010 at 12:43 AM
Look,Cheap GHD Straighteners,Discount GHD Styler,GHD Outlet,CHI hair Straighteners,2010 Cheap Sunglasses,Discount Sunglasses，Sunglasses Outlet,wholesale Sunglasses,Sunglasses Wholesale. Do not worry. The urge to buy these goods will be very strong once you spot the shoes or clothes that you like. I am also a big fan for these products, although I like ED Hardy,cheap ED Hardy Clothes,discount ED Hardy Outlet,Wholesale Ed Hardy,Ed hardy wholesale, cheap Prada,discount Prada,Prada Outlet,newest Prada handbags,Prada handbags 2010, so I can understand. Online wholesale dealers have quite an array for you to choose from, including wholesale jewelry and handbags. And these are bags that bear the most famous names in the world of fashion, by the way -Prada, Spyder, Moncler, GHD, ED Hardy, designer Sunglasses, Ankh Royalty, Twisted Heart, or sayAnkhRoyalty,Ankh Royalty,Ankh Royalty Clothing ,Ankh Royalty Clothes,AnkhRoyalty Outlet. Such variety that comes at incredibly affordable prices is a treat you can't possibly pass up on.
March 31, 2010 at 12:44 AM
Thanks, it is very nice and helpful .i agree with you very much ! but I have something to share with you too ! you can login our website to look around , all our products are top quality and low prices . would you like to buy something ? ok follow me ! better choice better life ! do not leave something to regret in yourfuture!
NFL Jerseys
tn chaussures
chaussures puma
Tn Requin
Cheap Polo Shirts
better choice better life !Thanks for your post , it is very nice and helpful .see you !
March 31, 2010 at 6:34 PM
I feel honored and privileged after reading your posts.You know,summer is coming! The homme chaussure will become fashionable soon.We offer hommes chaussures discount,all kinds,all colors available.You can also find hommes chaussures Nike,hommes chaussures pas cher.fashionable Chaussures Homme 2010,Best wishes to you!
Acheter hommes chaussures in our shop
March 31, 2010 at 7:09 PM
Thanks, it is very nice and helpful .i agree with you very much ! but I have something to share with you too ! you can login our website to look around , all our products are top quality and low prices . would you like to buy something ? ok follow me ! better choice better life ! do not leave something to regret in yourfuture!
NFL Jerseys
tn chaussures
chaussures puma
Tn Requin
Cheap Polo Shirts
chaussures pas cher
better choice better life !Thanks for your post , it is very nice and helpful .see you !
April 1, 2010 at 7:38 PM
Like the asics onitsuka tiger australia range, these boots are used in a variety of asics onitsuka tiger mexico 66 yellow black colors including Dusty Rose asics onitsuka tiger shoes australia and indigo, with black and gray oatmeal asics tiger mexico 66 available. What makes these shoes so different than other shoes in the range of onitsuka tiger mexico 66 yellow black make 3 large wooden buttons you some onitsuka tiger mexico 66 by asics really good looks and different styles with them.
April 3, 2010 at 7:21 PM
asics onitsuka tiger ultimate 81 began as one man's asics onitsuka tiger yellow dream over 50 years asics onitsuka tiger black ago and now has become asics onitsuka tiger green a global leader in the asics mexico 66 baja sporting goods industry. Kihachiro onitsuka tiger mexico 66 brown founded Onitsuka Tiger in 1949 in Kobe, Japan. His vision for the onitsuka tiger mexico 66 silver company was to inspire the country's youth to gain a healthier onitsuka tiger mexico 66 white more positive free shipping there is asics mexico 66 no this big profit for every asics mini cooper boday you should know this.we will with different styles and colors in our website,such as mexico 66 onitsuka tiger shoes,mexico 66 baja, onitsuka tiger shoes and asics mexico 66 mid runner Gel-Stratus The following 101 quotes are some ASICS GEL-Stratus 2.1 hat I have stumbled upon on the web, or ASICS GEL-Stratus 2.1 Mens shoes seen in books / popular culture.
April 3, 2010 at 7:23 PM
https://www.blogger.com/comment.g?blogID=10592227&postID=112144985420972854&page=1&token=1270620524559_AIe9_BHNiyUthsRPfHrFPpmsnmUp400GaDINhxy_L2whqAaTha8kq1SWbvvJbtyydNOBvxisJlWLF7T_lDPqUxvIufYSPzMy6Js7JseJ_PC-pwMEUG33vphDmpkeEoqyV2j2fSPkX-Kp7TuZDWwnaTUO0e6YIQVIkewz2gyk1gQVukpnMrkvHtIZaqyj_DyVf3G2AI9S5UxYyNWLTPHMWXO7EEcDkRdX0SA5cUdG-_LaHq-adjj8Niw
April 6, 2010 at 11:10 PM
Remise Chaussures Sport
nike tn requin
nike max tn
nike tn
nikemaxtn
tn requin
chaussures sport
nike femmes chaussures
chaussure nike tn requin
chaussure nike air max
nike tn
chaussure nike tn
April 6, 2010 at 11:10 PM
Discount Handbags On Sale-Beautycasa Online : - Hermes Handbags Miu Miu Handbags Fendi Handbags Prada Handbags Balenciaga Handbags Chloe Handbags Marc Jacobs Handbags Jimmy Choo Handbags Mulberry Luxury Handbags Christian Dior Handbags Hermes Wallet Gucci Handbags FREE GUCCI WALLET Difference the amount Designer Handbags Chanel Handbags Burberry Handbags Louis Vuitton Wallet ecommerce, open source, shop, online shopping
April 7, 2010 at 7:59 PM
I feel honored and privileged after reading your posts.You know,summer is coming! chaussures homme online .The chaussures homme nike will become fashionable soon.We offer chaussures homme nike,all kinds,all colors available.You can also find homme chaussure in our shop,including hommes chaussures,Nike Shox NZ,Nike Shox R4.Best wishes to you!
chaussures mbt homme
chaussures homme nike
homme chaussure
nike air max 180
nike shox r6
nike shox r3
nike shox r4
nike shox r5
nike shox nz
chaussures femme nike
April 8, 2010 at 1:45 AM
cheap cocktail dresses
Cheap Wedding Dresses
China Wholesale
April 8, 2010 at 8:12 PM
Nice article written by you
cheap Nike dunkofdiscount nike dunkyounike dunk outlet
arediscount nike shoes
oncheap nike shoes
atChristian Louboutin shoes
youChloe bags outlet
cancheap Chloe handbags
bediscount Chloe bags
herenewest Chloe shoes
Chloe bags 2010
bape shoes outlet
bape clothingdiscount bape shoes
cheap bape shoes
bape jackets
ghd
cheap ghd
ghd straighteners
ghd outlet
discount ghds
ed hardy fashion story
ed hardy wholesale
cheap ed hardy wholesale
ghd
ghd outlet
cheap ghd
discount ghds
ghd straighteners
benefit GHD
GHD IV Salon Styler
GHD Mini Styler
kiss GHD
Pink GHD
Pure Black GHD
Pure White GHD
Purple GHD
Rare GHD
discount moncler Tshirt
air jordan
jordan shoes
nike shoes
air jordan 1
air jordan 11
air jordan 4
April 9, 2010 at 6:20 PM
Only 21 years old abercrombie & fitch clothingstriped shirt, white abercrombie clothingby the ghd hair straighteners the British cheap ghd single-handedly founded ghd Burberry, Burberry (ghd outlet Brand: Burberry, silk Bo Li) as a discount ghd representative of the fashion brand sunglasses brand. BURBERRY founded in 1856LOGO, is the Royal gucci sunglasses outlet royal family supplies, in 1879, BURBERRY wrinkle developed a waterproof, breathable durable fabric: gabardine, discount Oaklay sunglasses win everyone's cheap Ray Ban sunglasses .1901, BURBERRY designed the first Armani Sunglasses, the first World War, the trench coat is discount Armani Sunglasses as the senior British army Burberry Sunglasses. Today opened the Oxford Dictionary,BURBERRY discount Burberry Sunglasses coat has become another Cartier Sunglasses for. Today, BURBERRY by constantly discount Cartier Sunglasses its design and innovative plan to Chanel Sunglasses the coveted decorative appeal of discount Chanel Sunglasses classic sensibility with the perfect combination of Coach sunglasses times, in the fashion into the discount Coach sunglasses of a timeless brand!1890 Burberry jacket has been D&G Sunglasses for wet and dry enduring trend. In post-war discount D&G Sunglasses cop become a hobby-horse of many Dior Sunglasses stars, including HEPBURN and "Casablanca" discount Dior Sunglasses Humphrey Bogart and Ingrid Bergman actress. In addition to creation of the ED Hardy Sunglasses in 1901 known trademark
April 9, 2010 at 7:25 PM
as a sign of Burberry. discount ED Hardy Sunglasses, of course, must be mentioned in 1924 created the Burberry Fendi Sunglasses "Nova" - on behalf of discount Fendi Sunglasses plaid. The first time in this plaid Gucci sunglasses for wet and dry inside, while in 1967 cheap Gucci sunglasses stered as a trademark after LV sunglasses more widely used in other products, such as umbrella and suitcase.Burberry - discount LV sunglasses on the burberry (BURBERRY Prorsum), the most classical models is that Oakley sunglasses is windbreaker and that was unique but it discount Oakley sunglasses been copied to the Check everywhere, and many Hollywood stars will be wearing it Police sunglasses the film coat and skirt.Can not be denied, the British veteran Burberry - Poisson (BURBERRY Prorsum) Christopher discount Police sunglasses (Christopher Bailey), led by a Prada sunglasses at international, and sales have discount Prada sunglasses to rise, while the men in Ray Ban Sunglasses spring and summer of 2005, discount Ray Ban Sunglasses stopher Bailey Roberto Cavalli Sunglasses ideas, mainly from all the discount Roberto Cavalli Sunglasses he was pleasant, through the clothing's performance, with Versace Sunglasses sharing. The performance of discount Versace Sunglasses most important is that "color" sky blue, light green, bright yellow, bright orange discount abercrombie and fitch pink and so on, with a white abercrombie fitch outlet, people feel the vitality of cheap abercrombie fitch and summer. In addition,discount LV handbag.
April 9, 2010 at 7:27 PM
here shoes college abercrombie clothingBURBERRY abercrombie outletcenturies abercrombie and fitch shirtsand abercrombie fitch outlet fluorabercrombie & fitch clothing leather abercrombie and fitch clothesman-made abercrombie and fitch clothing flexibiabercrombie fitch fabri any abercrombie & fitchcan feel abercrombie fitch clothing Christopher abercrombie and fitchBURBERRY abercrombie and fitch outlet clothing cheap abercrombie fitch tinuously abercrombie shirt the UK and global explorers, adventurers discount abercrombie and fitch clothes sports to ed hardy wholesale throughs cheap ed hardy wholesaleNew BURBERRY discount ed hardy wholesale embodies wholesale ed hardy vitality BURBERRY ed hardy outlet attitude paulsmith British paulsmith outlet BURBERRY paulsmith 2010productcheap paulsmith national discount paulsmith brand (paul smith shoes and Burberry Lookbook Burberry outletoutspoken cheap Burberry Burberry discount Burberry to brand. Burberry (Burberry) newest Burberrygirls Burberry Shoes outlet fascination vest Burberry bagsshortsBurberry purseinnovative Burberry sunglasses Oh will newest Burberry suit it will moncler T-shirts
April 9, 2010 at 7:28 PM
Only 21 years old abercrombie & fitch clothingstriped shirt, white abercrombie clothingby the ghd hair straighteners the British cheap ghd single-handedly founded ghd Burberry, Burberry (ghd outlet Brand: Burberry, silk Bo Li) as a discount ghd representative of the fashion brand sunglasses brand. BURBERRY founded in 1856LOGO, is the Royal gucci sunglasses outlet royal family supplies, in 1879, BURBERRY wrinkle developed a waterproof, breathable durable fabric: gabardine, discount Oaklay sunglasses win everyone's cheap Ray Ban sunglasses .1901, BURBERRY designed the first Armani Sunglasses, the first World War, the trench coat is discount Armani Sunglasses as the senior British army Burberry Sunglasses. Today opened the Oxford Dictionary,BURBERRY discount Burberry Sunglasses coat has become another Cartier Sunglasses for. Today, BURBERRY by constantly discount Cartier Sunglasses its design and innovative plan to Chanel Sunglasses the coveted decorative appeal of discount Chanel Sunglasses classic sensibility with the perfect combination of Coach sunglasses times, in the fashion into the discount Coach sunglasses of a timeless brand!1890 Burberry jacket has been D&G Sunglasses for wet and dry enduring trend. In post-war discount D&G Sunglasses cop become a hobby-horse of many Dior Sunglasses stars, including HEPBURN and "Casablanca" discount Dior Sunglasses Humphrey Bogart and Ingrid Bergman actress. In addition to creation of the ED Hardy Sunglasses in 1901 known trademark
April 9, 2010 at 7:30 PM
as a sign of Burberry. discount ED Hardy Sunglasses, of course, must be mentioned in 1924 created the Burberry Fendi Sunglasses "Nova" - on behalf of discount Fendi Sunglasses plaid. The first time in this plaid Gucci sunglasses for wet and dry inside, while in 1967 cheap Gucci sunglasses stered as a trademark after LV sunglasses more widely used in other products, such as umbrella and suitcase.Burberry - discount LV sunglasses on the burberry (BURBERRY Prorsum), the most classical models is that Oakley sunglasses is windbreaker and that was unique but it discount Oakley sunglasses been copied to the Check everywhere, and many Hollywood stars will be wearing it Police sunglasses the film coat and skirt.Can not be denied, the British veteran Burberry - Poisson (BURBERRY Prorsum) Christopher discount Police sunglasses (Christopher Bailey), led by a Prada sunglasses at international, and sales have discount Prada sunglasses to rise, while the men in Ray Ban Sunglasses spring and summer of 2005, discount Ray Ban Sunglasses stopher Bailey Roberto Cavalli Sunglasses ideas, mainly from all the discount Roberto Cavalli Sunglasses he was pleasant, through the clothing's performance, with Versace Sunglasses sharing. The performance of discount Versace Sunglasses most important is that "color" sky blue, light green, bright yellow, bright orange discount abercrombie and fitch pink and so on, with a white abercrombie fitch outlet, people feel the vitality of cheap abercrombie fitch and summer. In addition,discount LV handbag.
April 9, 2010 at 7:42 PM
here shoes college abercrombie clothingBURBERRY abercrombie outletcenturies abercrombie and fitch shirtsand abercrombie fitch outlet fluorabercrombie & fitch clothing leather abercrombie and fitch clothesman-made abercrombie and fitch clothing flexibiabercrombie fitch fabri any abercrombie & fitchcan feel abercrombie fitch clothing Christopher abercrombie and fitchBURBERRY abercrombie and fitch outlet clothing cheap abercrombie fitch tinuously abercrombie shirt the UK and global explorers, adventurers discount abercrombie and fitch clothes sports to ed hardy wholesale throughs cheap ed hardy wholesaleNew BURBERRY discount ed hardy wholesale embodies wholesale ed hardy vitality BURBERRY ed hardy outlet attitude paulsmith British paulsmith outlet BURBERRY paulsmith 2010productcheap paulsmith national discount paulsmith brand (paul smith shoes and Burberry Lookbook Burberry outletoutspoken cheap Burberry Burberry discount Burberry to brand. Burberry (Burberry) newest Burberrygirls Burberry Shoes outlet fascination vest Burberry bagsshortsBurberry purseinnovative Burberry sunglasses Oh will newest Burberry suit it will moncler T-shirts
April 9, 2010 at 7:43 PM
Designer purses and handbags cheap LV bagsTommy and Kate handbagsdiscount
Gucci bags you wear it to work with friends newest Dooney&Bourke handbags
Kate is a designer labelChanel handbags outlet Tommy and Kate handbags Burberry handbags 2010
Shox Technology Nike Nike air jordan 2010 The effect is more traction basketball air jordan shoesLunarLite cheap Jordan shoes outlet Zoom Airjordan
Depending on the application discount jordan shoesmaintaining extreme responsivenessjordan 2010Flywire
This is one of the newer technologies introduced by Nike. jumpman23to ensure that the foot is kept in place.
Timberland shoes or boots
cheap timberlandSince the Timberland branddiscount timberlands
These boots are specially timberland BootsHow we tall them apart?cheap timberland ukBut on the other hand there are usually quite few real Timberland discount timberland outlet lines being sold for a fraction of the price. Sunglasses discounttimberland outlet are not limited for the summer season alone. This is basically because sunglasses newest Gucci sunglasses
Here are five simple steps discount coach sunglasses
In Marketing, discount Oakley sunglasses
Now because of customized sunglassRay Ban
sunglasses outlet racks, unreclaimed creature, without refinement,
Nike shoesair max nike
in 1987 first time. Since then Nike newest air max has been frequently introducing new as well as updated models in it.cheap air max shoesJust because air max Clearly, consumers are just as happy as athletes.air max nike Also on this pair they Moncler T-shirts
April 9, 2010 at 8:27 PM
Guccio Gucci,son of an Italian trader founded the House of Gucci gucci boots outlet in Florence in 1906.
Guccio, authentic gucci bags an extraordinary craftsman, started selling luxury goods in the 1920s. cheap gucci shoes In 1938, Gucci opened a boutique in Rome discount Gucci bags.Aldo expanded the company by opening more stores. newest gucci handbags In 1953, New York got its first Gucci store gucci handbags 2010 and by the late '60s, more such stores were launched in London, Paris and Tokyo.
Being a proud owner of a watch Armani watches outlet is a wondrous thing that many take for granted. watch wholesale No one remembers back when things were simple and discount DG watches they had to call "time and temperature" to get an accurate time cheap Chanel watches. Now certain devices run on their own newest GUCCI watches with no batteries.
Juicy couture handbags are among the brands Juicy couture which have gained a great repute over a longer period of time. The quality authentic Juicy bags and material used for their manufacturing is up to the mark Juicy handbags outlet and has no match.
Thus with the holiday shopping season approaching rapidly tiffany 2010, as an investor, the best gift you may give yourself this year is not a ring from tiffany, but shares from Tiffany. With a fairly good fundamentals, tiffany outlet a strong presence over its competitors, discount tiffany charms and a favorable economic situation to feed upon, I would not be hesitant to take a short term risk cheap tiffany Bangles in locating some luck with. stylers are often available with free gifts, discount ghds or other incentives, which can sometimes on offer ghd outlet seems too good to be true, ghd then it probably is. There are many imitation cheap ghd flooding the market , so you need to keep ghd straighteners your wits about you. wholesale edhardy t shirt Whether you like the colour or want to do your bit for charity, Moncler T-shirtsvery popular when they are available. jumpman 23
April 9, 2010 at 11:42 PM
jordan 8
There are cheap shoes to choose
jordan 9
You can have a look at it.
jordan 10
Funny things like this.
jordan 11
jordan 12
jordan 13
April 10, 2010 at 1:15 AM
cheap cocktail dresses
Cheap Wedding Dresses
China Wholesale
April 11, 2010 at 6:38 PM
Since building my Asteroids arcade cabinet replica/ television cabinet, My videogame collection has grown almost double the size, I've completely filled the storage space in the cabinet and now there are boxes, consoles and loose carts all over the floor.
wholesale caps
April 13, 2010 at 4:17 AM
There are cheap shoes to choose
jordan 6
jordan 7
Good quality with low price.
air jordan 2010
Air Jordan 2009
You can have a look at it.
jordan shoes
jordan ajf shoes
We offer different styles.
jordan 1
jordan 2
If you like,you can contact us.
jordan 3
jordan 4
Thanks.
jordan 5
jumpman23
April 13, 2010 at 7:11 PM
Teenagers adidas tennis shoes can discount adidas sneakers teenage newest adidas shoes 2010 shoes discount moncler outlet cross cheap adidas shoes important wholesale adidas shoes growing adidas shoes outlet they discount moncler t-shirt shoes cheap jordan shoes changes new jordan shoes every time new jordan shoes shoes air jordan 2010 qualities discount jordan shoes soles jumpman 23 material Jordan ajf shoes breathes Newest Timberland shoes 2010 extended cheap Nike shox shoes support discount brand shoes ankles cheap Nike Dunk shoes injury AIR MAX shoes outlet when cheap ghd store ghd hair straightener home shoes discount ghd hair straightener properly ghd outlet returnable ghd running shoe cheap bourke dooney handbags good grip authentic dooney bourke bags breathes easily newest dooney bourke handbags should be moncler vest light enough dooney bourke new purses still support discount dooney bourke handbags outlet especially injury wholesale Rolex watches But just moncler hair iron,moncler jackets money.moncler coats
April 13, 2010 at 7:46 PM
MBT will not only change,MBT boots, the way you use your musclesMBT Shoes in fashion, but will improve the use of your joints and spineCheap MBT Shoes sale, The uniquely designed sole combined withDiscount MBT outlet 2010, correct training achieves a more active andMBT Walking Shoes, healthy posture and walk. What are the benefits?Cheap Dior shoes, Free shipping and free return shippingDiscount Dior bags, on all dior. Included with each pair of isDior sunglasses, an instructional DVD. What is Dior?Dior sunglasses outlet 2010, Technology or dior is not just a shoe in the ordinary sense ?Newest Dior sunglasses,
Do not worry. The urge to buy these goods will be very strong once you spot the shoes or clothes that you likeDiscount New Balance shoes，revolutionary fitness aid from Swiss Masai，New Balance 580 outlet, which may help reduce cellulite andCheap New Balance 850, Relieves muscular tension back and joint problems Leads to a relaxed upright，New Balance running shoes, Through its unique design of its multilayered soleNew Balance shoes 2010, transforms flat hard artificial surfaces into natural uneven surfaces theDiscount PUMA Outlet, with top quality and cheap price. Cheap PUMA running shoes, innovative sole design includes thePuma sneaker 2010, curved sole which is theKids Puma Shoes, is good for our foot
Puma Walking Shoes, back ，nice，good and knee is Puma Shoes. Here you can buy wide rangewholesale cl high heel sandals，quality and cheap car GPS navigation systemsMoncler，Very Cool, Comfortable and lightmoncler jackets original packing you can rest assured.moncler coats，You might say that thediscount moncler vestNo one ever thought bothmoncler outlet，As with everything that comes fmonmoncler t-shirtThe new store has been decorated
GHDYou might say that thecheap GHDyou may get the newDiscount GHD hair straightenermany new color options of theGHD hair straighteners your choice will be rightGHD outlet
April 13, 2010 at 10:57 PM
MBT will not only change,MBT boots, the way you use your musclesMBT Shoes in fashion, but will improve the use of your joints and spineCheap MBT Shoes sale, The uniquely designed sole combined withDiscount MBT outlet 2010, correct training achieves a more active andMBT Walking Shoes, healthy posture and walk. What are the benefits?Cheap Dior shoes, Free shipping and free return shippingDiscount Dior bags, on all dior. Included with each pair of isDior sunglasses, an instructional DVD. What is Dior?Dior sunglasses outlet 2010, Technology or dior is not just a shoe in the ordinary sense ?Newest Dior sunglasses,
Do not worry. The urge to buy these goods will be very strong once you spot the shoes or clothes that you likeDiscount New Balance shoes，revolutionary fitness aid from Swiss Masai，New Balance 580 outlet, which may help reduce cellulite andCheap New Balance 850, Relieves muscular tension back and joint problems Leads to a relaxed upright，New Balance running shoes, Through its unique design of its multilayered soleNew Balance shoes 2010, transforms flat hard artificial surfaces into natural uneven surfaces theDiscount PUMA Outlet, with top quality and cheap price. Cheap PUMA running shoes, innovative sole design includes thePuma sneaker 2010, curved sole which is theKids Puma Shoes, is good for our foot
Puma Walking Shoes, back ，nice，good and knee is Puma Shoes. Here you can buy wide rangewholesale cl high heel sandals，quality and cheap car GPS navigation systemsMoncler，Very Cool, Comfortable and lightmoncler jackets original packing you can rest assured.moncler coats，You might say that thediscount moncler vestNo one ever thought bothmoncler outlet，As with everything that comes fmonmoncler t-shirtThe new store has been decorated
GHDYou might say that thecheap GHDyou may get the newDiscount GHD hair straightenermany new color options of theGHD hair straighteners your choice will be rightGHD outlet
April 13, 2010 at 10:57 PM
a bold piece of art
Ed Hardy Wholesale
excellent customer service
wholesale Ed Hardy
Charmed by the magic of amazing tattoo and impressive color
ed hardy wholesale clothing
April 14, 2010 at 10:25 PM
Fashion trends change on daily basis, like Cheap GHD Straighteners (you can get it from a GHD Outlet), Why not get rid of your old straighteners to buy new GHDs? There is another kind hair straightener, called CHI hair Straighteners. Do you want to buy Discount GHD Styler? Following the latest in designer shades has become a passion of everyone, now 2010 Cheap ED Hardy Sunglasses（Cheap Ed Hardy Sunglasses), or we can say 2010 DiscountOakley Sunglasses. If you are the type of a woman who loves to explore in fashion, our Dior Sunglasses Outlet will definitely satisfy your taste, because as you can see, we wholesale Sunglasses. And we have discount Armani Sunglasses, discount Burberry Sunglasses, discount Cartier Sunglasses , discount Chanel Sunglasses, discount Coach Sunglasses, discount D&G Sunglasses, discount Dior Sunglasses, discount ED Hardy Sunglasses and discount Fendi Sunglasses, discount Gucci Sunglasses，discount LV Sunglasses, cheap Oakley Sunglasses, discount Police Sunglasses, discount Prada Sunglasses, discount Ray Ban Sunglasses, discount Roberto Cavalli Sunglasses, discount Versace Sunglasses. Designer shades with optical grade lenses are important to protect our eyes from the sun. Don’t forget us--Gucci Sunglasses outlet.
April 17, 2010 at 12:01 AM
Ed hardy streak of clothing is expanded into its wholesale ED Hardy chain so that a large number of fans and users can enjoy the cheap ED Hardy Clothes range easily with the help of numerous secured websites, actually, our discount ED Hardy Outlet. As we all know, in fact Wholesale Ed Hardy,is based on the creations of the world renowned tattoo artist Don Ed Hardy. Why Ed hardy wholesale? Well, this question is bound to strike the minds of all individuals. Many people may say cheap Prada shoes is a joke, but we can give you discount Prada Sunglasses , because we have authentic Pradas bags Outlet. Almost everyone will agree that newest Pradas Purses are some of the most beautiful designer handbags ( Pradas handbags 2010) marketed today. The reason is simple: fashion prohibited by AnkhRoyalty, in other words, we can write it as Ankh Royalty the Cultural Revolution. Straightens out the collar, the epaulette epaulet, the Ankh Royalty Clothing two-row buckle. Would you like to wear Ankh Royalty Clothes?Now welcome to our AnkhRoyalty Outlet. And these are different products that bear the most famous names in the world of fashion, like Ankh Royalty T-Shirt, by the way-Prada, Spyder, Moncler(Moncler jackets，or you can say Moncler coats, Moncler T-shirt, Moncler vest，and you can buy them from our discount Moncler outlet), GHD, ED Hardy, designer Sunglasses, Ankh Royalty, Twisted Heart.
April 17, 2010 at 12:03 AM
I feel honored and privileged after reading your posts.You know,summer is coming! chaussures homme online .The chaussures homme
nike will become fashionable soon.We offer chaussures homme nike,all kinds,all colors
available.You can also find homme chaussure
in our shop,including hommes chaussures,Nike
Shox NZ,Nike Shox R4.Best wishes to you!
chaussures mbt
homme
chaussures mbt
femme
chaussures
homme pas cher
homme chaussure
nike air max 180
nike air max 95
nike air max 90
nike shox r6
nike shox r3
nike shox r4
nike shox r5
nike shox nz
nike tn dollar
nike tn spider
chaussures femme
nike
nike dunk sb
low
April 17, 2010 at 7:28 AM
cheap cocktail dresses
Cheap Wedding Dresses
China Wholesale
April 18, 2010 at 6:39 PM
thanks for the post!!!
Air max chaussures : http://www.air-max-chaussures.com/
Air max 1 chaussures : http://www.air-max-chaussures.com/air-max-1-c-86.html
Air max 90 chaussures: http://www.air-max-chaussures.com/air-max-90-c-89.html
Air max light chaussures : http://www.air-max-chaussures.com/air-max-light-c-94.html
Air pegasus 89 chaussures: http://www.air-max-chaussures.com/air-pegasus-89-c-95.html
Air 180 chaussures: http://www.air-max-chaussures.com/air-180-chaussures-c-96.html
Air structure triax 91 chaussures:
http://www.air-max-chaussures.com/air-structure-triax-91-c-91.html
Air classic BW chaussures: http://www.air-max-chaussures.com/air-classic-bw-c-97.html
Air Tailwind 92 chaussures:
http://www.air-max-chaussures.com/air-tailwind-92-c-93.html
Air max 93 chaussures
http://www.air-max-chaussures.com/air-max-93-c-85.html
Air max 95 chaussures
http://www.air-max-chaussures.com/air-max-95-c-99.html
Air max 97 chaussures
http://www.air-max-chaussures.com/air-max-97-c-101.html
Air max tn chaussures
http://www.air-max-chaussures.com/air-max-tn-c-103.html
Air max ltd chaussures
http://www.air-max-chaussures.com/air-max-ltd-c-108.html
Air max 360 chaussures
http://www.air-max-chaussures.com/air-max-360-c-112.html
nice work.
April 19, 2010 at 5:31 AM
This blog is very useful for me. the content is full of meaning and the blog template is so cool. What's more, I am deeply influenced by this great article.
clothes jeans
AF Jeans
Aff jeans
Amani Jeans
Artful Dodger Jeans
Bape Jeans
BBC Jeans
Christian Audigier Jeans
D&G Jeans
Ed hardy Jeans
Ed-hardy Mens Jeans Ed hardy Jeans
Ed-hardy Women Jeans Ed hardy Jeans
G-Star Raw Jeans
Laguna Beach Jeans
laguna beach men jeans Laguna Beach Jeans
laguna beach women jeans Laguna Beach Jeans
RED MONKEY Jeans
Seven Jeans
True Religion Jeans
True Religion Mens Jeans True Religion Jeans
True Religion Womens Jeans True Religion Jeans
April 20, 2010 at 12:20 AM
cheap cocktail dresses
Cheap Wedding Dresses
China Wholesale
April 20, 2010 at 6:35 PM
a bold piece of art
Ed Hardy Wholesale
excellent customer service
wholesale Ed Hardy
Charmed by the magic of amazing tattoo and impressive color
ed hardy wholesale clothing
April 20, 2010 at 11:46 PM
There are cheap jordan shoes to choose
replica jordan shoes
so cheap,
ed hardy wholesale
i can't believe it.
nike shoes wholesale
it is wonderful.
April 21, 2010 at 6:29 PM
We provide flare True Religion Jeans of very low Price. And also Wholesale Evisu JeansYou can choose the cheap True Religion Jeans from our online shop.
April 21, 2010 at 6:52 PM
We provide flare True Religion Jeans of very low Price. And also Wholesale Evisu JeansYou can choose the cheap True Religion Jeans from our online shop.
April 21, 2010 at 6:52 PM
We provide flare True Religion Jeans of very low Price. And also Wholesale Evisu JeansYou can choose the cheap True Religion Jeans from our online shop.
April 21, 2010 at 6:54 PM
We provide flare True Religion Jeans of very low Price. And also Wholesale Evisu JeansYou can choose the cheap True Religion Jeans from our online shop.
April 21, 2010 at 6:55 PM
cheap cocktail dresses
Cheap Wedding Dresses
China Wholesale
April 21, 2010 at 6:58 PM
There are cheap jordan shoes to choose
china wholesale suppliers
i can't nelieve it.
jordan shoes wholesale
You can have a look at it.
nike shoes wholesale
April 21, 2010 at 7:51 PM
Helo, your blog is really good, I like it very much!By the way, if you like nike chaussures tn you
can come here to have a look!
nike tn
tn chaussures
nike tn requin chaussures
nike air max tn chaussures
nike homme chaussures
nike femme chaususres
nike enfant chaussres
MBT France
tn chaussures
nike air max
nike shox TL1
nike shox TL3
nike shox TL3
nike tn 8
nike dunk low femme
nike air max 92
nike shox r4 cartoon
nike tn 10
nike tn limpide
nike tn net
nike tn 8 homme
You also can choose the most fashionable sunglasses here, it really can make you different from other
people.
sunglasses
designer sunglasses
wholesale sunglasses
sunglasses discount in USA
men's sunglasses
women's sunglasses
burbbery sunglasses
bvlgari sunglasses
chanel sunglasses
coach sunglasses
dg sunglasses
April 21, 2010 at 10:17 PM
Nike TN Chaussures
Chaussures de Sport
ed hardy clothes
Nike chaussures
Victoria Secret Pink
NFL Jerseys
Chaussures Sport
puma chaussures hommes
NFL Jerseys
Ed Hardy
NIKE MAX TN
Tn Requin
Nike Air Rift
jeans diesel
Chaussures puma
Ed Hardy
NFL Jerseys
MBT Shoes
Remise Chaussures Sport
chaussures pas cher
Nike Shox Shoes
Air Max Shoes
Nike shoes
Chaussures Sports
nike tn requin chaussures
NIKE MAX TN
MBT Shoes On Sale
Athletic Shoes
Clothing Jeans
ed hardy clothing
MBT Shoes On Sale
Victoria's Secret Underwear
cosplay costumes
cheap cosplay costumes
April 22, 2010 at 2:27 AM
You can have a look at it.
jordan shoes
jordan ajf shoes
There are cheap shoes to choose
jordan 6
jordan 7
Good quality with low price.
air jordan 2010
Air Jordan 2009
If you like,you can contact us.
jordan 3
jordan 4
We offer different styles.
jordan 1
jordan 2
Thanks.
jordan 5
jumpman23
April 24, 2010 at 4:23 AM
chaussures mbt homme
chaussures mbt femme
chaussures homme nike
April 24, 2010 at 7:35 PM
I feel honored and privileged after reading your posts.You know,summer is coming! chaussures homme online .The chaussures homme nike will become fashionable soon.We offer chaussures homme nike,all kinds,all colors available.You can also find homme chaussure in our shop,including hommes chaussures,Nike Shox NZ,Nike Shox R4.Best wishes to you!
chaussures mbt homme
chaussures mbt femme
chaussures homme nike
nike air max 180
nike air max 95
nike air max 90
nike air max 2009
nike shox r6
nike shox r3
nike shox r4
nike shox r5
nike shox nz
nike tn dollar
nike tn spider
chaussures femme nike
nike dunk sb low
nike dunk sb low
April 24, 2010 at 7:42 PM
thanks for the post,and be keeping eyes on it.
Air Max Chaussures
Air Max Chaussures
Air Max 1 Chaussures
Air Max light chaussures
Air Max 90 chaussures
Air 180 chaussures
Air classic BW chaussures
Air max 93 chaussures
Air max 95 chaussures
Air max 97 chaussures
ED Hardy
Air max 360 chaussures
Air Max Schuhe
Air Max 2009 Schuhe
Air max ltd chaussures
Air max tn chaussures
April 25, 2010 at 7:59 PM
Nice article written by you
Nicediscount nike dunk
articlecheap nike shoes
writtennike dunk outlet
byChristian Louboutin shoes
youChloe outlet
cheap Chloe bags
discount Chloe bags
newest Chloe shoes
Chloe bags 2010
Chloe totes
bape shoes outlet
bape clothing
discount bape shoes
cheap bape shoes
bape jackets
wholesale ed hardy
ed hardy wholesale
discount ed hardy
MBT boots
MBT shoes in fashion
cheap mbt shoes sale
discount mbt outlet 2010
MBT Walking Shoes
Purple GHD
Moncler
Moncler jackets
Moncler coats
Moncler Vest
Moncler outlet
discount moncler Tshirt
air jordan 2010
cheap jordan shoes
discount jordan shoes
jump man 23
cheap ugg boots
discount ugg boots
ugg boots
discount nike shoes
April 26, 2010 at 5:56 PM
ugg boots
polo boots
polo shoes
herve leger
herve leger bandage dress
chanel outlet
chanel bandbags
chanel bags
chanel iman
ralph Lauren polo
ralph lauren outlet
lacoste polo
polo raplh lauren
air jordan 2010
cheap jordan shoes
jordan ajf shoes
discount jordan shoes
jumpman23
moncler
moncler jackets
moncler coats
moncler vest
moncler outlet
moncler Polo t shirt
cheap five finger shoes
kiss ghd
April 26, 2010 at 6:00 PM
Dior sunglasses
Ray Ban sunglasses
Fendi Handbags
Hermes handbags
Miu miu Handbags
Timberland Boots
timberland outlet
Moncler Jackets
Moncler coats
discount Moncler Vest
Moncler outlet
moncler polo t-shirt
NFL JERSEYS
NBA JERSEYS
NHL JERSEYS
MLB JERSEYS
FOOTBALL JERSEYS
Nike air jordan 2010
cheap Jordan shoes
discount jordan shoes
jordan
jumpman23
Jordan ajf shoes
April 26, 2010 at 7:28 PM
AirMax BW
AirMax Huarache
AirMax LTD
AirMax Skyline
AirMax TN
AirMax Zength
AirMax 09 Sneakers
AirMax 180
AirMax 2003
AirMax 2006
AirMax 2009
AirMax 2010
AirMax 360
AirMax 87
AirMax 90
AirMax 91
AirMax 92
AirMax 93
AirMax 95
cheap ugg boots
discount ugg boots
ugg boots
classic ugg boots
ugg classic tall boots
April 26, 2010 at 7:31 PM
I love your blog very much,and i also want to introduce my products to all of you,them are nice and very fashion,if you like them,please contact us on our website or eamil us,thks very much for your attention.And this is my products below:
gucci outlet
gucci boots
gucci bags
gucci shoes
Armani watches
ugg boots
cheap ugg boots
discount ugg boots
classic ugg boots
ugg classic tall boots
Juicy couture outlet
Juicy couture bags
Tiffany Necklacks
Tiffany pendants outlet
discount Tiffany Earrings
cheap Tiffany Rings
Moncler jackets
Moncler
cheap Moncler coats
Moncler Vest
Moncler outlet
moncler polo t shirt
MBT boots
MBT shoes in fashion
cheap mbt shoes sale
discount mbt outlet 2010
MBT Walking Shoes
Benefit GHD
wholesale edhardy t shirt
thanks very much for your attention,and good luck,guys!!!
April 26, 2010 at 8:06 PM
a bold piece of art
Ed Hardy Wholesale
excellent customer service
wholesale Ed Hardy
Charmed by the magic of amazing tattoo and impressive color
ed hardy wholesale clothing
April 26, 2010 at 10:35 PM
ugg boots
polo boots
polo shoes
herve leger
herve leger bandage dress
chanel outlet
chanel handbags
chanel bags
chanel iman
ralph Lauren polo
ralph lauren outlet
lacoste polo
polo raplh lauren
air jordan 2010
cheap jordan shoes
jordan ajf shoes
discount jordan shoes
jumpman23
moncler
moncler jackets
moncler coats
moncler vest
moncler outlet
moncler Polo t shirt
cheap five finger shoes
kiss ghd
April 28, 2010 at 1:45 AM
We are a large online shop which sells
true religion boot cut jeans
Ricky True Religion Jeans
true religion joey flare
Coogi mens jeans
Cheap evisu jeans for sale
ed hardy jeans women
Christian Audigier Mens Jeans
Mens g star raw jeans
RED MONKEY Jeans
Christian Audigier Jeans
Ed-hardy Jeans
Evisu Jeans
April 28, 2010 at 7:02 PM
We are a large online shop which sells
true religion boot cut jeans
Ricky True Religion Jeans
true religion joey flare
Coogi mens jeans
Cheap evisu jeans for sale
ed hardy jeans women
Christian Audigier Mens Jeans
Mens g star raw jeans
RED MONKEY Jeans
Christian Audigier Jeans
Ed-hardy Jeans
Evisu Jeans
April 28, 2010 at 7:02 PM
My favorite brand is ed hardy wholesale, and I always want this brand of
wholesale handbags. And its T-shirts are very special too. I really hope I can wear its
wholesale t-shirtsto go to school and go shopping with my friends, and it will be awesome. And most of my friends like the Ed Hardy style too.
However, this brand is too expensive for me to buy. An Ed Hardy Nike wholesale cost nearly 250 dollars. It is almost a cost of my living of mouth. I am only an ordinary student who has no trust fund; no job and no income that I can’t afford to pay such high price for its sunglasses wholesaleor wholesale ghd. So, I won’t buy the Ed Hardy brand series at least for now.
April 28, 2010 at 11:28 PM
Teenagers in a track adidas shoes can go through shoes rate adidas outlet teenage runner well knows that adidas sneakers to wear the right shoes air jordan 2010 any cross country among world jordan shoes especially important to the sportsman discount jordan shoes a young person growing up Pure GHD Not only do they want to bourke dooney handbags grow out of their shoes dooney bourke bags but shape of their feet changes which means every time we dooney bourke outlet the time comes to get dooney bourke totes a new pair of shoes eveyone NIKE SHOX a good choice to get NIKE SHOX SHOES it with a high material SHOX SHOES that breathes easily on random moncler They should be extended runs moncler jackets give enough support to athlets moncler coats especially around the ankles area moncler vest to avoid injury your ankles discount moncler outlet when you make your trip discount moncler t-shirt something in store possible fit cheap ugg boots his is also the way to discount ugg boots that don't fit properly only ugg boots it focuses on your convenience classic ugg boots but they are not returnable ugg classic tall boots
April 29, 2010 at 7:31 PM
MBT will not only change,MBT boots,the way you use your MTB Shoes,What are the benefits?christian dior,Free shipping and free return shippingdior shoes,on all diorIncluded with each pair ofdior handbags,an instructional DVDWhat is Dior?dior sunglasses,look at another good paoduct such as Dior totesDo not worryThe urge to buy these goodsnew balancerevolutionary fitness aid from Swiss Masai，new balance shoes,which may help reduce cellulite andnew balance outlet,transforms flat hard artificial surfaces Puma Shoes, with top quality and cheap price. puma outlet,innovative sole design includes thePuma Sneaker,Here you can buy wide rangewholesale cl high heel sandalsquality and cheap car GPS navigation systemsMoncler，Very Cool, Comfortable and lightmoncler jacketsoriginal packing you can rest assured.moncler coatsYou might say that thediscount moncler vestNo one ever thought bothmoncler outlet，As with everything that comes fmonmoncler t-shirtThe new store has been decorated.north faceWe are offering you a wide range ofnorth face outletSome color combinations seem to getnorthfaceBut within the same community north face jacketsbecause it features just the right amount of north face coats,A little of these are given below.ugg bootswas a very well-known French fashionable boot,cheap ugg bootsbecause of the wisdom featuresdiscount ugg bootswhich you are buying is uniqueclassic ugg bootsthey obtain materials from domestic suppliers ugg classic tall boots,A stroll around the park with the GHD IV Salon Styler，We are offering you coach outlet，our store has been decorated coach handbags，Nicecoach totes
May 2, 2010 at 7:48 PM
Every woman knows that you can never own too many pairs of wholesale CL high heel sandals. And high heels represent the ultimate in shoe fashion for many women. However, it's just a question of finding the right style for you--there's a pump for everyone. For example, a pair of basic black pumps--with round or pointed toes--is a must-have basic for every woman's wholesale LV handbags.
As for finding a comfortable pair of cheap Nike sport shoes, I can testify that this is indeed possible.
I've also bought two pairs of cheap Ed Hardy sunglasses, though I always feared my feet would slip out of them and sprain an ankle. wholesale Rolex watches are my pump of choice, because the straps are adjustable--and, in a good shoe, the straps are also padded. I like the "breathing room" of a sandal, even though the look of a pointed toe pump is very stylish.
May 3, 2010 at 2:09 AM
Fashion is all about lookingmoncler jackets good and being comfortable at the same time. monclermoncler coatsEvery woman loves being complimentedmoncler t-shirtmoncler vestmoncler outlet for her sense of style and elegance. ugg bootscheap ugg bootsdiscount ugg bootsNo woman wants to be labeled as outdated and old fashioned.ugg classic tall bootsclassic ugg bootsWomen are expected to be in constant touch with the latest trends in ordermoncler jacketsmonclernew moncler coats to adopt those trends and look fashionable and stylish. moncler vestmoncler outletmoncler polo t-shirtWomen need to make sure that whatever they wear, coach outletcoach handbagscoach bagsit should compliment their personality and they do not feel awkward while wearing it. coach totescoach outletlouis vuitton handbagsAn article about Gothic costume clothing and the importance of choosing Gothic choker. LV handbags 2010Louis Vuitton bagsLouis Vuitton totesThis article explores the wide variety of Gothicrain boots
rainweardiesel jeans chokers available from everyday to special occasions. Ture Religion Jeanslevis jeansabercrombie and fitch outletIt should inspire people to go shopping for their own Gothic chokers. ed hardy wholesaleed hardy outletcheap ed hardy wholesaleThe origins of the Flat cap span discount ed hardy wholesalewholesale ed hardy
May 3, 2010 at 8:32 PM
Fashion trends change on daily basis, like Gold GHD. Following the latest in designer shades has become a passion of everyone, now Burberry Sunglasses. If you are the type of a woman who loves to explore in fashion, our ED Hardy Sunglasses will definitely satisfy your taste. Cheap Ed Hardy Sunglasses is also OK. Ed hardy streak of clothing is expanded into its wholesale ED Hardy T-shirt chain so that a large number of fans and users can enjoy the cheap ED Hardy Clothing range easily with the help of numerous secured websites, actually, our ED Hardy Outlet. As we all know, in fact discount ED Hardy, is based on the creations of the world renowned tattoo artist Don Ed Hardy. Well, this question is bound to strike the minds of all individuals. Many people may say Prada shoes is a joke, but we can give you Prada Sunglasses, because we have Prada handbags. Almost everyone will agree that Prada Purses are some of the most beautiful designer handbags marketed today. Now we have one new product: Prada totes. The reason is simple: fashion prohibited by ugg boots, in other words, we can say it as Cheap ugg boots. Would you like to wear Discount ugg boots. We have two kinds of fashionable boots: classic ugg boots and ugg classic tall boots. Ankh Royalty--the Cultural Revolution. Straightens out the collar, the epaulette epaulet, the Ankh Royalty Clothing two-row buckle. Now welcome to our Ankh Royalty Outlet. And these are different products that bear the most famous names in the world of fashion, like Ankh Royalty T shirt by the way -Prada, Spyder, Moncler(Moncler jackets，or you can say Moncler coats, Moncler T-shirt, Moncler vest，and you can buy them from our discount Moncler outlet), GHD, ED Hardy, Ankh Royalty, Twisted Heart.
May 3, 2010 at 8:33 PM
2010 the best wholesale market
1. About Ed Hardy t shirt wholesale Why have wholesale edhardy t shirt become so popular among the rich and famous? The idea of clothing designs all coming from tattoos are in and of itself, a great idea. This is taking unique artwork and placing it on great, fashionable clothing.
2. How to Curl Hair With GHD Straighteners
After buying your ghd in the GHD hair straightener wholesale, you no longer need a separate curling iron. Style your hair sleek and straight or create fabulous curls whenever you feel like it.
3. Air Jordans are Not Just an Ordinary Shoe
Every kid wants to be like Michael Jordan,it is not a problem. The wholesale air jordan shoes priced at over $100, only the spoiled kids could parade these exclusive shoes around school.
4. Have you ever had any Louis Vuitton products?
If no, you will find the answer when you try on them! It doesn't matter; you can solve this problem by searching our b2cwholesale, discount LV handbag is just hot selling!
5.Vibram Five Fingers Running
Here cheap five finger shoes now, you can find the cheap professional shoes from our site, and we believe that you will satisfied with the high quality shoes and the lower price!
May 3, 2010 at 8:46 PM
We are a large online shop which sells
true religion boot cut jeans
Ricky True Religion Jeans
true religion joey flare
Coogi mens jeans
Cheap evisu jeans for sale
ed hardy jeans women
Christian Audigier Mens Jeans
Mens g star raw jeans
RED MONKEY Jeans
Christian Audigier Jeans
Ed-hardy Jeans
Evisu Jeans
May 4, 2010 at 2:22 AM
cheap cocktail dresses
Cheap Wedding Dresses
China Wholesale
May 5, 2010 at 12:01 AM
Deeply impressed and inspired! This article really made much sense to me! I want to recommend you a really great website where you can have a pair of vintage nike shoes Tn Requin with no more than 40Eu! Hurry up and go to www.nike-max-tn.com to get your favorite shoes now! Tn Requin provide various styles of nike tn, nike max,nike chaussure , tn chaussures, requin tn, we also retail Nike Chaussures Femme,Nike chaussures homme ,nike shox nz online in whosale prices. NIKE AIR MAX 360 will make you run faster and jump higher,Texture and comfortable shoes is your choice,Preferential price make you satisfied. We have Alife chaussures homme of supreme quality. Our nike shoes contains not only nike chaussures enfant, but nike chaussures femme as well as nike chaussures homme, which are all our best sellers. As to puma chaussures femme and puma chaussures homme we supply in stock contains all sizes and models. All nike shoes are being retailed in wholesale prices. Wholesale are supported, too. Express shipping!
May 5, 2010 at 7:06 AM
stylish design
Ed Hardy Wholesale
fashion excellent quality
wholesale Ed Hardy
ED Hardy clothing bring you a super surprise!
ed hardy wholesale clothing
May 5, 2010 at 11:13 PM
Deeply impressed and inspired! This article really made much sense to me! I want to recommend you a really great website where you can have a pair of vintage nike shoes Tn Requin with no more than 40Eu! Hurry up and go to www.nike-max-tn.com to get your favorite shoes now! Tn Requin provide various styles of nike tn, nike max,nike chaussure , tn chaussures, requin tn, we also retail Nike Chaussures Femme,Nike chaussures homme ,nike shox nz online in whosale prices. NIKE AIR MAX 360 will make you run faster and jump higher,Texture and comfortable shoes is your choice,Preferential price make you satisfied. We have Alife chaussures homme of supreme quality. Our nike shoes contains not only nike chaussures enfant, but nike chaussures femme as well as nike chaussures homme, which are all our best sellers. As to puma chaussures femme and puma chaussures homme we supply in stock contains all sizes and models. All nike shoes are being retailed in wholesale prices. Wholesale are supported, too. Express shipping!
May 6, 2010 at 6:27 AM
Edhardy-uk/au
ed hardy clothing|
cheap ed hardy|
ed hardy online store|
ed hardy clothes|
discount ed hardy|
cheap ed hardy|
ed hardy uk|
discount ed hardy shirts|
cheap ed hardy shirts|
ed hardy t shirts|
ed hardy long sleeve|
ed hardy mens|
ed hardy womens|
ed hardy outerwear|
ed hardy shoes|
ed hardy boots|
ed hardy snow boots|
ed hardy kid’s|
ed hardy hoodies|
ed hardy jackets|
ed hardy sunglasses|
ed hardy handbags|
ed hardy wallets|
ed hardy swimwear|
cheap ed hardy shirts|
cheap ed hardy hoodies|
ed hardy long sleeve|
ed hardy men|
ed hardy women|
ed hardy jackets|
ed hardy men’s shoes|
ed hardy boots|
ed hardy snow boots|
ed hardy winter boots|
ed hardy kid|
ed hardy hoodies|
ed hardy sunglasses|
ed hardy handbags|
ed hardy wallets|
ed hardy swimwear|
May 7, 2010 at 12:19 AM
Ed-hardy-shirts
cheap ed hardy clothing |
ed hardy t shirts|
ed hardy shirts|
ed hardy online store|
ed hardy womens|
ed hardy mens|
ed hardy bags|
ed hardy boots|
ed hardy shoes|
ed hardy men’s jeans|
ed hardy women’s jackets|
ed hardy men’s shirts|
ed hardy women’s handbags|
christian audiger long sleeve|
ed hardy wholesale|
ed hardy christian audigier|
christian audigier shirts|
ed hardy women’s long sleeve|
ed hardy outerwear|
ed hardy jacket|
ed hardy swimwear|
christian audigier hoodies |
wholesale ed hardy shirts|
ed hardy shirts for men|
ed hardy shirts cheap |
ed hardy t shirts for women|
Nike Chaussures
Remise Chaussures Sport|
Chaussure de Sports|
Marques Chaussures Sport|
2010 Air Max Chaussures|
Nike Shox R4|
nike tn requin|
nike max tn|
nike tn |
tn requin |
nikemaxtn|
Chaussures Sport|
Nike chaussures hommes|
Nike chaussures femmes|
Nike chaussures enfants|
Nike chaussures sport|
nike femmes chaussures|
chaussure nike tn requin|
chaussure nike air max|
chaussure nike tn|
nike tn|
May 7, 2010 at 12:19 AM
Ed-hardy-clothing
ed hardy clothing us|
ed hardy clothing|
ed hardy clothes|
ed hardy ugg boots|
ed hardy wear|
ed hardy polo shirts|
ed hardy love kills slowly|
ed hardy love kills slowly boots|
ed hardy love kills slowly shoes|
ed hardy love kills slowly shirts|
ed hardy trousers|
ed hardy jackets|
ed hardy women’s t shirts|
ed hardy boots|
ed hardy womens shirts|
ed hardy outerwear|
christian audigier jackets |
ed hardy womens jeans|
ed hardy handbags|
ed hardy winter boots|
ed hardy t shirts|
christian audigier hoodies|
ed hardy mens jeans|
ed hardy mens shoes|
ed hardy womens hoodies|
Ed-hardy-us
cheap ed hardy|
cheap ed hardy clothing|
ed hardy clothes|
ed hardy wholesale|
ed hardy clothing|
ed hardy outerwear|
ed hardy t shirts|
ed hardy long sleeve|
ed hardy hoodies|
ed hardy men’s|
ed hardy women’s|
ed hardy shoes|
ed hardy kid|
ed hardy jeans|
ed hardy handbags|
ed hardy pants|
christian audiger t shirts|
christian audigier hoodies|
christian audigier shirts|
May 7, 2010 at 12:19 AM
Honour to be here!
Truly a nice work! Greatest post I have ever seen! Thank you for sharing it! Have a good day!
sport shoes shop|
cheap nike air|
cheap nike shox|
cheap nike max|
cheap puma shoes|
cheap nike shoes|
puma shoes|
discount puma shoes|
nike sports shoes|
nike air force ones|
cheap sport shoes|
nike shox torch|
puma cat|
puma future cat|
nike shox r4|
nike shox nz|
nike air max ltd|
nike running shoes|
nike tn dollar|
ferrari shoes|
air jordan shoes|
nike air max tn|
jordan mens shoes|
nike air max|
nike women shoes|
cheap puma ferrari shoes|
nike shox rival|
nike mens shoes|
nike air max 360|
nike air max 2010|
nike air max 2009|
nike air max 90|
May 7, 2010 at 12:20 AM
There are cheap shoes to choose
jordan 6
jordan 7
Good quality with low price.
air jordan 2010
Air Jordan 2009
You can have a look at it.
jordan shoes
jordan ajf shoes
We offer different styles.
jordan 1
jordan 2
If you like,you can contact us.
jordan 3
jordan 4
Thanks.
jordan 5
jumpman
May 7, 2010 at 1:18 AM
http://cellphones.blog.sohu.com
http://dressweddingstores.wordpress.com
http://wedding-dress-online.blog.anous.fr
http://wedding.blogge.rs
http://cellphones.blog131.fc2.com
May 7, 2010 at 1:40 AM
There are cheap jordan shoes to choose
jordan 8
so cheap,
jordan 9
i can't believe it.
jordan 10
it is wonderful.
jordan 11
really big surprise
jordan 12
You can have a look at it.
jordan 13
May 7, 2010 at 5:41 PM
stylish design
Ed Hardy Wholesale
fashion excellent quality
wholesale Ed Hardy
ED Hardy clothing bring you a super surprise!
ed hardy wholesale clothing
May 8, 2010 at 12:10 AM
2010 the best wholesale market
1. About Ed Hardy t shirt wholesale Why have wholesale edhardy t shirt become so popular among the rich and famous? The idea of clothing designs all coming from tattoos are in and of itself, a great idea. This is taking unique artwork and placing it on great, fashionable clothing.
2. How to Curl Hair With GHD Straighteners
After buying your ghd in the GHD hair straightener wholesale, you no longer need a separate curling iron. Style your hair sleek and straight or create fabulous curls whenever you feel like it.
3. Air Jordans are Not Just an Ordinary Shoe
Every kid wants to be like Michael Jordan,it is not a problem. The wholesale air jordan shoes priced at over $100, only the spoiled kids could parade these exclusive shoes around school.
4. Have you ever had any Louis Vuitton products?
If no, you will find the answer when you try on them! It doesn't matter; you can solve this problem by searching our b2cwholesale, discount LV handbag is just hot selling!
5.Vibram Five Fingers Running
Here cheap five finger shoes now, you can find the cheap professional shoes from our site, and we believe that you will satisfied with the high quality shoes and the lower price!
May 8, 2010 at 2:55 AM
spyder
spyder jackets
spyder ski wear
coach outlet
coach handbag
coach handbags
coach bag
coach bags
vibram five finger
Five finger shoes
vibram 5 fingers
vibram kso
vibram running shoes
MBT boots
MBT shoes in fashion
cheap mbt shoes sale
discount mbt outlet 2010
MBT Walking Shoes
spyder
spyder jackets
spyder ski wear
coach outlet
coach handbag
coach handbags
coach bag
coach bags
spyder
spyder jackets
spyder ski wear
ugg boots
ugg boots cheap
discount ugg boots
classic ugg boots
ugg classic tall boots
GHD Mini Styler
cheap ugg boots
discount ugg boots
ugg boots
classic ugg boots
ugg classic tall boots
spyder
spyder jackets
spyder ski wear
May 9, 2010 at 10:39 PM
your article is very good, but do you like the
pink ghd
babyliss
armani sunglasses
gucci sunglasses outlet
discount abercrombie and fitch clothes
abercrombie fitch outlet
abercrombie & fitch clothing
ed hardy wholesale
cheap ed hardy wholesale
discount ed hardy wholesale
wholesale ed hardy
ed hardy outlet
paul smith shirt
paul smith
paul smith 2010
cheap paul smith
discount paul smith
paul smith shoes
Burberry scarf
Burberry outlet
cheap Burberry handbags
Burberry totes
newest Burberry
moncler
Moncler jackets
Moncler coats
Moncler Vest
discount Moncler outlet
moncler polo t shirt
cheap ugg boots
discount ugg boots
ugg boots
classic ugg boots
ugg classic tall boots
AirMax BW
AirMax Huarache
AirMax LTD
AirMax Skyline
AirMax TN
May 9, 2010 at 11:39 PM
how do you think the AirMax Zength
AirMax 09
AirMax 180
AirMax 2003
AirMax 2006
AirMax 2009
AirMax 2010
AirMax 360
AirMax 87
AirMax 90
AirMax 91
AirMax 92
AirMax 93
AirMax 95
coach outlet
coach handbag
coach handbags
coach bag
coach bags
discount Oaklay sunglasses
cheap Ray Ban sunglasses
ugg boots
ugg boots discount
cheap ugg boots
classic ugg boots
ugg classic tall boots
May 9, 2010 at 11:41 PM
spyder
spyder jackets
spyder ski wear
Detroit Lions
Green Bay Packers
Houston Texans
Indianapolis Colts
Jacksonville Jaguars
Kansas City Chiefs
Miami Dolphins
Minnesota Vikings
New England Patriots
New Orleans Saints
herve leger
herve leger bandage dress
moncler
moncler jackets
moncler coats
moncler vest
moncler outlet
moncler Polo t shirt
ugg boots
polo boots
polo shoes
May 9, 2010 at 11:43 PM
spyder
spyder jackets
spyder ski wear
coach outlet
coach handbag
coach handbags
coach bag
coach bags
vibram five finger
Five finger shoes
vibram 5 fingers
vibram kso
vibram running shoes
MBT boots
MBT shoes in fashion
cheap mbt shoes sale
discount mbt outlet 2010
MBT Walking Shoes
spyder
spyder jackets
spyder ski wear
coach outlet
coach handbag
coach handbags
coach bag
coach bags
spyder
spyder jackets
spyder ski wear
spyder
spyder jackets
spyder ski wear
ugg boots
ugg boots cheap
discount ugg boots
classic ugg boots
ugg classic tall boots
GHD Mini Styler
cheap ugg boots
discount ugg boots
ugg boots
classic ugg boots
ugg classic tall boots
May 10, 2010 at 6:17 PM
cheap cocktail dresses
Cheap Wedding Dresses
China Wholesale
Cheap Evening Dresses
May 10, 2010 at 6:30 PM
http://cellphones.blog.sohu.com
http://dressweddingstores.wordpress.com
http://wedding-dress-online.blog.anous.fr
http://wedding.blogge.rs
http://cellphones.blog131.fc2.com
May 10, 2010 at 11:18 PM
We are a large online shop which sells
true religion boot cut jeans
Ricky True Religion Jeans
true religion joey flare
Coogi mens jeans
Cheap evisu jeans for sale
ed hardy jeans women
Christian Audigier Mens Jeans
Mens g star raw jeans
RED MONKEY Jeans
Christian Audigier Jeans
Ed-hardy Jeans
Evisu Jeans
May 11, 2010 at 12:09 AM
your article is very good, but do you like the
pink ghd
babyliss
armani sunglasses
gucci sunglasses outlet
discount abercrombie and fitch clothes
abercrombie fitch outlet
abercrombie & fitch clothing
ed hardy wholesale
cheap ed hardy wholesale
discount ed hardy wholesale
wholesale ed hardy
ed hardy outlet
paul smith shirt
paul smith
paul smith 2010
cheap paul smith
discount paul smith
paul smith shoes
Burberry scarf
Burberry outlet
cheap Burberry handbags
Burberry totes
newest Burberry
moncler
Moncler jackets
Moncler coats
Moncler Vest
discount Moncler outlet
moncler polo t shirt
cheap ugg boots
discount ugg boots
ugg boots
classic ugg boots
ugg classic tall boots
AirMax BW
AirMax Huarache
AirMax LTD
AirMax Skyline
AirMax TN
May 11, 2010 at 7:34 PM
how do you think the AirMax Zength
AirMax 09
AirMax 180
AirMax 2003
AirMax 2006
AirMax 2009
AirMax 2010
AirMax 360
AirMax 87
AirMax 90
AirMax 91
AirMax 92
AirMax 93
AirMax 95
coach outlet
coach handbag
coach handbags
coach bag
coach bags
discount Oaklay sunglasses
cheap Ray Ban sunglasses
ugg boots
ugg boots discount
cheap ugg boots
classic ugg boots
ugg classic tall boots
May 11, 2010 at 7:36 PM
spyder
spyder jackets
spyder ski wear
Detroit Lions
Green Bay Packers
Houston Texans
Indianapolis Colts
Jacksonville Jaguars
Kansas City Chiefs
Miami Dolphins
Minnesota Vikings
New England Patriots
New Orleans Saints
herve leger
herve leger bandage dress
moncler
moncler jackets
moncler coats
moncler vest
moncler outlet
moncler Polo t shirt
ugg boots
polo boots
polo shoes
May 11, 2010 at 7:38 PM
AirMax BW|AirMax Huarache|AirMax LTD |AirMax Skyline |AirMax TN |AirMax Zength|AirMax 09 Sneakers|AirMax 180|AirMax 2003|AirMax 2006|AirMax 2009|AirMax 2010|AirMax 360 |AirMax 87|AirMax 90|AirMax 91|AirMax 92|AirMax 93|AirMax 95|Dior sunglasses|Ray Ban sunglasses|Fendi Handbags|Hermes handbags|Miu miu Handbags|Timberland Boots|timberland outlet|Moncler Jackets|Moncler coats|discount Moncler Vest|Moncler outlet|moncler polo t-shirt
May 12, 2010 at 1:17 AM
GHD Mini Styler|Nike air jordan 2010|cheap Jordan shoes|discount jordan shoes|jordan|jumpman23|Jordan ajf shoes|NFL JERSEYS|NBA JERSEYS|NHL JERSEYS|MLB JERSEYS|FOOTBALL JERSEYS|NFL|NBA|NHL|MLB|spyder|spyder jackets|spyder ski wear|
May 12, 2010 at 1:19 AM
We are a large online shop which sells
true religion boot cut jeans
Ricky True Religion Jeans
true religion joey flare
Coogi mens jeans
Cheap evisu jeans for sale
ed hardy jeans women
Christian Audigier Mens Jeans
Mens g star raw jeans
RED MONKEY Jeans
Christian Audigier Jeans
Ed-hardy Jeans
Evisu Jeans
May 13, 2010 at 12:16 AM
stylish design
Ed Hardy Wholesale
fashion excellent quality
wholesale Ed Hardy
ED Hardy clothing bring you a super surprise!
ed hardy wholesale clothing
I can't believe it.
abercrombie outlet
You can have a look at it.
abercrombie fitch outlet
Wonderful!
coats & jackets
If one knows where to, the world will give way.
north face
World-class brands
moncler
The top fabric produced in France is 100%
duvetica
Italy's top outdoor brands
peuterey
sisley
Good quality and function, more accord with fashionable design
woolrich
High-grade, innovation, the trend of the brand
refrigiwear
The quality is so good
barbour jackets
north sails
Young and creative style
diesel jacket
May 13, 2010 at 12:23 AM