Metacritic Matters: How Review Scores Hurt Video Games

Discussion in 'NMA News and Information' started by Brother None, Apr 14, 2013.

  1. Brother None

    Brother None This ghoul has seen it all
    Staff Member Admin Orderite

    Apr 3, 2003
    Kotaku has a pretty interesting piece up about how the games industry uses Metacritic, and why it's not a good thing. This New Vegas example shouldn't be news to people though the amount cited is.<blockquote>Perhaps you've heard the story: publisher Bethesda was due to give developer Obsidian a bonus if their post-apocalyptic RPG averaged an 85 on Metacritic, the review aggregation site. It got an 84 on PC and Xbox 360, and an 82 on PS3.

    “If only it was a stable product and didn't ship with so many bugs, I would've given New Vegas a higher score,” wrote a reviewer for the website 1up, which gave New Vegas a B, or 75 on Metacritic's scale.

    “It's disappointing to see such an otherwise brilliant and polished game suffer from years-old bugs, and unfortunately our review score for the game has to reflect that,” said The Escapist's review, which gave the game an 80.

    If New Vegas had hit an 85, Obsidian would have gotten their bonus. And according to one person familiar with the situation who asked not to be named while speaking to Kotaku, that bonus was worth $1 million. For a team of 70 or so, that averages out to around $14,000 a person. Enough for a cheap car. Maybe a few mortgage payments.</blockquote>
     
  2. AtomBomb

    AtomBomb It Wandered In From the Wastes

    132
    Sep 26, 2012
    Wow...I never realized how little these people make off of making a video game. So does most of the money made go to the publisher?
     
  3. Faceless Stranger

    Faceless Stranger Board Drifter

    Aug 19, 2010
    Hmm, funny, bugs didn't stop them from giving Fallout 3 an "A" :roll:
     
  4. Gaspard

    Gaspard Kasparov

    833
    May 7, 2009
    word :|
     
  5. Bewitched

    Bewitched aka Vault_13

    459
    Jun 13, 2006
    Brian Fargo would do Wasteland 2 for 1 million bucks.
     
  6. woo1108

    woo1108 Vault Senior Citizen

    Sep 27, 2012
    scoring 100 dispite of bug, shitty system, poor writing and voice acting, disconnection between 1 and 2 etc.
    Good point? maybe visual? I don't like fo3's visual though.
    And dispite of bug is beth's fault, lots of idiots blames Obsidian.
     
  7. Crni Vuk

    Crni Vuk M4A3 Oldfag oTO Orderite

    Nov 25, 2008
    Marketing. Its all about the Marketing with games today. Reviews like Metacritic dont really review the "game" but actually the marketing campaign by the Publisher/Developer. There are exceptions of course - See Diablo 3. It had to face a lot of criticism.

    But usually the better the marketing and hype, the higher the scores.

    not only.

    A good thing to look for is music, because a lot of things are similar. I dont have all the details in head, but since many games still are released with an box and DVD a large part of the money goes well, to the production.
     
  8. Beelzebud

    Beelzebud A Smooth-Skin

    675
    Mar 6, 2008
    I think the thing that annoys me about the situation with Obsidian, is that everyone seems to give Bethesda a complete pass on the exact same issues. I found Fallout 3 to be more buggy than New Vegas, and yet everyone that reviews games acts like it was perfect.
     
  9. Walpknut

    Walpknut This ghoul has seen it all

    Dec 30, 2010
    Fallout 3 was way more buggy than New Vegas, and New Vegas had more content and less development time, ain't that funny?
     
  10. Stanislao Moulinsky

    Stanislao Moulinsky Vault Fossil

    Jul 16, 2009
    The Metascore is a flawed concept based on a flawed system used in a flawed way. :| It boggles my mind that people genuinely think that it's a good metric or see no problem in the way the industry uses it.
     
  11. Kwiatmen666

    Kwiatmen666 It Wandered In From the Wastes

    160
    Aug 28, 2006
    The Metacritic's Metascore indeed evaluates mainly the marketing and hype around the game - the examples of this are all three of latest games developed by Bethesda: Oblivion, Skyrim and Fallout 3. Their writing, and storytelling are disappointingly mediocre and such a thing as "depth" as we know it from other rpgs doesn't seem to exist for Beth's lead designers. I personally do not even think those games should be called rpgs - these are just some kind of open-world-based action-adventure games with elements of rpg. And - arguably - dialogs. Kind of... But don't take me wrong - I think Skyrim is quite a game in its own measure - the design of the world itself is great, but what's in it is entirely different matter.

    As I played all of Bethesda's latest games, I played all of Obsidian's games - Kotor II, NWN 2, Alpha Protocol, New Vegas - and let me tell those of you who didn't - the difference in writing, storytelling, character design are all COLOSSAL. Such ambition and passion with which Obsidian makes their games echoes in every aspect of their products. This can't be said about Bethesda. But the passion and ambition are also traps - in every of their games Obsidian leaves quite visible traces of unfinished quests and plot-lines. And this also can't be said about Bethesda - they seem to plan carefully and take their time to wrap everything up while Obsidian wants to add more than humanly possible (not a bad thing imo...) and so while striving for perfection, it only gets further away.

    A sad thing really - cold calculation and devious advertising campaign yealds better "numbers" than true passion and dedication I see in every game of Obsidian's.

    I was thinking about something constructive, but that long-winded rant is all I could come up with. Sorry;p
     
  12. Stevie D

    Stevie D Vault Cornwall my ansome

    183
    Jan 25, 2004
    No, I think that was well said. And it probably explains why I finished FO3 and enjoyed it more than F:NV, which I let slip as the NCR vs. Caeser's Legion thing started coming to a messy head.

    I guess I prefer a mediocre idea well executed to a good idea sloppily executed.

    That said, the graphics engine used for the two had outstayed its welcome by the end of FO3, so F:NV had that particular hand tied behind its back.
     
  13. woo1108

    woo1108 Vault Senior Citizen

    Sep 27, 2012
    I don't think beth's games are well-made for it's genre.
    They always makes game buggy, unbalanced system that has critical flaw(Daggerfall and Oblivion's level scailing is just sucks)(Fo3's Vats system, location design, skill, SPECIAL, journal which from TES4, etc). For skyrim, one of the TES's good point was faction quest. But that faction quest is ruined and fo other side quests, they use TES2's way that was just copying other quest and only different thing is what NPC saying and that saying is poor written.

    Compare with beth's flawed game, I rather choose Obsidian.

    I don't think metacritic is fair. For FO3 that has lots of flaws that other game had criticised and gain infamy for that but they got high score. and I don't think fo3 has that good point to cover bad points to gain that.
     
  14. Kwiatmen666

    Kwiatmen666 It Wandered In From the Wastes

    160
    Aug 28, 2006
    Yeah, I bet everyone has a little different take on this. I hated Fallout 3 for its shallowness and so I was a bit anxious to pick up New Vegas, but I was very pleasantly surprised.

    That goes without saying, but to be honest - I don't think Obsidian's games have ever exceeded in graphics. On the other hand rpg as a genre was never on the top of the cutting-edge graphics development, and so I don't think we - rpg fans - should give too much thought. In fact, focusing a review of an rpg on graphics is like trying to judge cookie's taste on the basis of its shape. Rather ridiculous, right?;p
     
  15. Ares

    Ares First time out of the vault

    34
    Dec 2, 2011
    TB has been raging about Metacritic as well for as long as I can remember.

    But Metacritic isn't going anywhere anytime soon so I would like to ask, how to at least control the damage?

    For one I wonder if imposing and enforcing a unified "rating" system would help (yes, yes: I know that it in itself is a dumb system to judge something like entertainment but lets be honest: it also is not going anywhere anytime soon)- for one thing it would eliminate all the supidness like converting a 5 point system into a 100 scale, which seems to cause all manner of statistical inaccuracies in the metacritic weighted average equation.

    The best place IMO would probably be by Metacritic itself- for example a contract requirement would be that in order to have your reviews represented in the metacritic equation you need to adopt our parameters. As far as I have learned anything about economical statistics and analyzsis one of the prerequisites is that the units that your data is based on has to be converted into a unified scale first.
     
  16. Arden

    Arden Still Mildly Glowing

    250
    Feb 26, 2010
    Nah, the moment you try to correct at least some things in a fundamentally flawed system you only add to its lifespan and legitimacy. Metacritic is a fundamentally flawed system used to make fundamentally false decisions. Even marketing guys know this but there are no other "numbers" to base any decisions upon for now. Sales tell you how many bought the game, but not if they enjoyed it.
     
  17. Stevie D

    Stevie D Vault Cornwall my ansome

    183
    Jan 25, 2004
    To a degree, although Fallout 1 and 2's audio and visual aesthetics were a big part of the games' draw. Pretty eye- and ear-popping stuff back in the day
     
  18. gumbarrel

    gumbarrel It Wandered In From the Wastes

    159
    Oct 18, 2010
    So all games like Fallout 1 and Arcanum need to do is market the fuck out of themselves and they will sell trillions, right? :roll:
     
  19. warsaw

    warsaw Still Mildly Glowing

    214
    Dec 1, 2008
    Hyperbole aside... uh, yeah. You market a game right, and people will buy it.
     
  20. gumbarrel

    gumbarrel It Wandered In From the Wastes

    159
    Oct 18, 2010
    And yet that's not what anyone here ever actually argues. I've never seen anyone say "Man, Title X should get the highest possible marketing budget, so it can sell boatloads!" If you really think that marketing is the end-all of game sells, then you should want companies like Obsidian to increase their marketing budget, but that's not something that anyone actually argues, ever. The opposite, actually.

    Not to mention the fact, that marketing isn't brain-control magic and you have plenty of very well marketed games that flop big time. Dante's Inferno, Lair, Brutal Legend, etc.