Splitsider

Monday, January 31st, 2011

Why Nielsen Ratings Are Inaccurate, and Why They'll Stay That Way

It's easy to understand why people are ambivalent, or even hostile, toward ratings. The thinking goes something like this:

1. People love the shows that they love, a lot
2. Some of these shows get canceled "too soon"
3. They get canceled, almost invariably, because of what some mean old bully named Nelson said. Nielsen. I mean Nielsen.

To intensify your heartburn, the validity of these ratings is seemingly taken for granted. When you find out that your favorite show is getting canceled for having a 1.1 rating, there's never any "but" — just the number. A network might keep a show around for prestige, or on the hope that it'll grow, but that's the executives' call. There is no real defense against The Number.

And it feels terrible! Comedy fans know this better than anyone else, or perhaps they just suffer with the least grace. Talk to a diehard Arrested Development or Party Down fan, and you'll hear anger; at the greater public for not watching the show, then at the network for canceling it, then, perhaps, at the people who provided the ratings in the first place.

It's natural for a TV buff or comedy junkie to feel a bit of malice toward these ratings, and more specifically, toward the company that produces them. Nearly everyone has a passing familiarity with this old company — you know, Nielsen boxes, the Nielsen metrics, the Nielsen Families. And seeing as it's an image that was first rendered many, many decades ago, it's easy to resent, or even write off. It sounds foreign, especially to web-savvy viewer who watches as much comedy on his laptop as on his TV. I mean, do you know any "Nielsen Families?" Me neither.

A broader look at the company does little to stem a forsaken fanboy's skepticism. "There are billion-dollar companies invested in preserving the Nielsen model," New York Times media reporter Brian Stelter told me. "Nielsen's clients are incumbents, protecting existing business models." NBC President of Research and Media Development Alan Wurtzel was a bit more blunt: "Listen, Nielsen is a monopoly. They're the only game in town. [Their ratings] are the only currency."

Even more discomfiting, in the context of just definitely knowing that this fantastic show has like a hidden audience, because it must, is the idea that Nielsen isn't solely concerned with measuring a show's fan base. It's not, and neither network nor advertiser seems to mind much.

"People don't want to think about whether to data is true or not," continued Stelter. "But as long as we all agree to go along with them, it's relatively comfortable for the industry." He recalled balking at the numbers for Obama's inauguration, which claimed just 37.8 million people watched — fewer than watched Reagan's three decades ago. This didn't jibe with anything, really; personal anecdotes about how everyone seemed to have seen the speech were just anecdotes, but they were uniform and universal.

Nobody I talked to in the industry — be it on the ratings side or the advertising side — would dispute this story, or others like it. Some even granted that the numbers were probably off. Unanimously, though, they agreed on one thing: to focus on events like that is to miss the point.

How Ratings Work

Nielsen's ratings, in the most basic sense, are like a poll: a sample of people, which the company claims fully represents the TV-watching public, is asked to report their viewing habits back to Nielsen. These people, of which there are about 50,000 (in 20,000 households), are approached by Nielsen to participate, and paid a token amount for their time and effort. (This broad sample differentiates Nielsen from smaller competitors like TiVo, whose subscribers tend to be wealthier than average.)

Under the title, "On Being a Nielsen Family," blogger Crone and Bear It writes:

Being a Nielsen household brought with it some nice perks. They actually paid us. We got a check several times a year for $50. Now, that’s not a lot of money, but every little bit helps.

Plus, the folks at Nielsen were just super nice. Someone would visit occasionally to check the equipment and sometimes they would bring goodies along — a cheesecake or a nice apple pie.

So there may be pie involved. Probably not. But still: maybe.

Some households are monitored passively, but the real valuable Nielsen data — the stuff the networks pay for — comes from its People Meters. These things:

Each People Meter sits atop, under, beside or behind the family set, and records whatever happens: what is watched and for how long, what is recorded, and even what's fast-forwarded. The box even knows which family member is watching, by means of a small remote control. Each family member, even the children, has his or her own little "I'm watching" button.

This data is sent back to Nielsen each night and parsed into the various ratings, including the live stream count, and the live + same day DVR numbers — the number most often quoted to the press. (Ratings are available for up to seven days after the initial air date, a number Nielsen settled on a compromise between the networks' desire for more more data, and advertisers' need for viewers to be current.) The numbers represent share of total viewers: a raw 1.0 rating, for example, indicates that one percent of the 115.9 million estimated TV-watching households, or 1,159,000 TVs, were watching a program. The data is also broken off into different demographic ratings, the most important being people ages 18-34.

Here's where things get a bit weird. The press and public is interested in knowing how many people watch a show, because it's the most obvious indicator of its success. This belief drives the way we think and talk about ratings. It also happens to be wrong.

To be sure, networks are interested in knowing how many people watch their programming, and freely tout or play down Nielsen's wider audience measurements. But the numbers that networks and advertisers actually use — to sell ads, to set prices, and to decide on the fate of a show — are commercial ratings. In other words, advertisers don't care how many people are watching a show nearly as much as they care how many people are watching their ads. Nielsen provides this number, which takes into account everything from next-day DVR viewing to fast-forwarding through commercials. If every Nielsen Family watched a show the day after it aired but skipped through all its ads, that show would probably be canceled.

In reality, a large majority of viewing still takes place live, when commercial skipping isn't possible. (Though channel flipping during commercials can have a noticeable effect on C3 ratings.) Within the small-but-not-insignificant set of people who watch recorded shows, around half of them skip through commercials.

Absent from these ratings, and most others, are types of viewing that serious TV fans — and especially comedy lovers — are well acquainted with: Netflix, Hulu, iTunes and even on-demand cable viewing don't count toward the ratings totals, either because they don't have ads at all, or because the ads shown differ from the ones shown during initial broadcast. A show with 10 million weekly Hulu viewers and one million live viewers is only slightly better than a plain one-million-viewer show, in terms of revenue. Nielsen does track online activity, and plans to increase its coverage dramatically over the next few years. But unless the ads shown during an online broadcast are the same as the ones shown during the TV broadcast, online viewers will continue to be broken off into a separate and much less important figure, and not a show's main rating.

This, more than anything else, explains why viewers feel uneasy about ratings. Networks, advertisers and Nielsen itself all know and acknowledge that the ratings reported and transacted with don't reflect a show's full audience. Back to the example of the inauguration: it strikes people as problem that the event's viewership didn't seem to be measured properly, but for the people who buy Nielsen's data, it's really not. It's tough to make much money on an one-off event with no ad breaks.

NBC's Alan Wurtzel underscored this discrepancy. "Ratings are a currency, so they're just as important now as they were ten years ago. It's how we get paid. But in this new media environment, do these numbers reflect accurately how many people are viewing this content? The answer is no." As viewing habits become more fractured, and viewers become less bound to TV schedules, he says, the current ratings system's focus on live and recently recorded TV will become a problem. "Are we happy with the way we're following technology and being able to measure it? No. We're way behind. On the other hand, are Nielsen ratings important and critical to the industry and as important to the industry as they even were? Absolutely, when you consider that if we didn't have them, we wouldn't get paid."

What Comes Next?

Nielsen, networks and advertisers are all wrestling with the problem of media fragmentation, and nobody has charted a clear path forward yet. A number of competing companies like TiVo, TRA and Rentrak, are selling supplementary data to some of Nielsen's clients, but none has a product that's competitive with Nielsen's core TV ratings. The Coalition for Innovative Media Measurement, or CIMM, hopes to glean better research from set top box data, and has the backing of major TV networks. Nielsen, which is in the process of going public, has extensive plans for increasing its measurement on online activity, and is running a pilot program that uses existing set-top boxes for audience measurement.

But all of these companies are charged not just with figuring out how to measure audience, but also with predicting the very ways people will consume TV in the future. Some point to digital set-top boxes as the future, with on-demand content and DVR usage eclipsing live broadcasts. Others, said Nielsen's senior vice president for insight and analysis Patricia McDonough, would say that the set-top box is obsolete, and that we'll all be getting our TV off the Internet in five years. "I'm not saying that I believe either one or the other, but I think our strategy is to be able to measure the stuff that's coming into the set no matter how it gets there."

It's clear that the way networks and advertisers track viewership is going to change dramatically over the next ten years, as viewing habits are transformed by technology. For now, though, little is likely to change. Americans spend more time than ever before watching basic cable and broadcast programming, at an average of 34 hours per week — an hour less than that maximum legal work week in France. Networks are happy with this. Advertisers are happy with this. Viewers, apparently, are still happy with this. When one of those three facts changes, so will the way ratings work.

Until then, take what little comfort you can from this: It probably wasn't your favorite show's terrible ratings that got it canceled.

It was your favorite show's ads' terrible ratings.

John Herrman is a lovable ginger who writes about technology for Gizmodo, amongst other places.

Sponsored Content
  • http://sorryyourheinous.tumblr.com/ sorry your heinous

    This is awesome and answered so many questions.

  • http://twitter.com/petegaines petejayhawk

    Great post. I am actually a Nielsen family; a couple times a year I get an envelope in the mail containing a one-week viewing diary and $30 cash. It's a pretty easy, painless ordeal. And nobody can blame me when awesome shows get cancelled for low ratings.

    • HernandezBobby

      Could you watch Conan so he beats Jay Leno then? Cheers

  • HerooftheBeach

    I've always wondered why Nielsen has a monopoly on this. You'd think a giant cable provider like Comcast would be able to track what their own customers are watching with much more thoroughness and accuracy than Nielsen's scattershot approach, and you know Comcast would love to have that data (and sell it on the side). There must be some technical or legal reason I can't think of as to why that isn't the case.

    Thanks for the informative post!

    • serus

      Comcast does, and they do sell it. Comcast however can only deliver information on their customers, so it's a little one sided. You aren't getting their competitor information, like DirecTV, Dish Network, Charter, Cox, Verizon, etc. Plus, you don't count those with over the air TV. Also, with Comcast, you don't know how many people are watching (you do with Nielsen's people meter.)

      As for why do they have a monopoly? Think about it, 50,000 homes, statistically accurate, all over the country, reporting in every single day what was watched the night before. It's not cheap, and you're looking at years of research, equipment, personnel, etc. All in HOPES that the networks will switch. Good luck with that, not a gamble I'm willing to take no matter how wealthy I was. Nielsen has been at this for over 50 years, they've grown with television, it's how they can afford to do this since they are already the standard.

      • http://sorryyourheinous.tumblr.com/ sorry your heinous

        Your arguments are all about it being too difficult, but that is the exact opposite claim I'd make. With digital television, just like a website, the data is all there. Did Google stop collecting traffic data when it became huge? No, because increasing data collection on a digital system scales really well. Nielsen's model is old (which they basically admit) and, in my opinion, not worth arguing for as strenuously as you are. [Which isn't to say they can't adapt, but that isn't a reason to be happy with the monopoly/status quo.]

        • serus

          My point was more of, what's the point? Say you had 100% accurate data from every cable provider, besides the cost, and time it took to collect, you would have to prove without a doubt that it was more accurate than Nielsen, AND you would have to give a reason for the entire industry (or majority of it) to switch to you instead of Nielsen. It isn't just that it's difficult to do, but that there's no 'win' for the networks. You aren't going to come up with staggeringly different results, since other companies have already been comparing and judging Nielsen's data. It's that Nielsen is still the standard, and it would take a lot to persuade all the networks to switch. Google is #1 because they set the standard, Yahoo!, Microsoft, and all the other search engines are trying to get a piece of the pie. It's similar with ratings, but a much smaller group to sell the information to.

          Also, you still aren't capturing all the people that still receive over the air television. They don't have a cable box to report back, and no way to tell who has how many TV's. I think the number is still high, upwards of 20% of all TV households.

          • http://sorryyourheinous.tumblr.com/ sorry your heinous

            Proving one has 100% accurate data is a fun logical puzzle and a much higher standard than Nielsen is being judged by in this very article.

            More than one source of data is better than a monopoly, if only to validate Nielsen's numbers. As is, Nielsen could be extremely wrong or quite accurate. It would be one thing if the companies had to recreate what Nielsen has already done, but they don't. Digital television providers (Comcast, Time Warner, DirecTV, Dish) already have boxes in houses. This may not capture over-the-air, but not only is that fraction diminishing, many advertisers value these viewers less (in general, cable/satellite correlates with higher household income). I'm not arguing against Nielsen (really!), but for competition and more accurate data (even if it isn't all-encompassing; more accurate "satellite viewers" is valuable in itself).

            Now the obvious argument is "why aren't they selling this now?" Perhaps they are (you mentioned Comcast), perhaps it is more valuable in-house (negotiations with networks) or agreements have tied their hands. I guarantee they have the information, though. It may be as simple as not wanting to spread out from their core business and enter into competition with Nielsen, which is exactly the kind of pressure a monopoly can wield.

            I'm rambling, though, and really not that invested in this argument. Mostly just curious (so thanks). In the end, my complaints are minimal. The article told me about how ratings work, as they exist now, and what some of the problems are and how the companies are looking forward.

            Just as a post-script, mostly not related to Nielsen ratings exactly, this all assumes the only reason networks make shows are to sell ads against them. This is mostly true, but ignores eventual box-set sales, streaming charges, episode downloads on Amazon/iTunes, syndication rights, even things less quantifiable like identity and potential audience growth (some shows require longer times to build an audience; if DVRers/streamers are bellwethers…?).

          • bri

            I think you guys have both missed the glaringly obvious reason why Comcast or DirectTV aren't in the ratings game. Nielsen provides an independent, third-party source of data.

            The networks and to a large extent Comcast only care about ratings in so far as advertisers care about ratings. Now what is Comcast's interest in providing accurate ratings to advertisers? None. As an example ESPN charges something like $6 per-month to Comcast, etc. in per subscriber fees. It is the highest per subscriber fee. What is Comcast's interest in providing accurate viewer numbers for ESPN. They would be much better off to under report ESPN's numbers and try to bring down the subscriber fee.

            Comcast absolutely collects data on it's subscribers, and it is probably more accurate than what neilson collects. I wouldn't be at all surprised if it comes into play when negotiating subscriber fees, but at the end of the day the networks and the advertisers don't want to have to take Comcast's word that the data is correct. They would rather have a third party do it.

            At the end of the day the problem is not logistic or technical, it is ethical.

  • serus

    I don't see why it's such a surprise shows that are watched post-live are canceled. The shows aren't there for our enjoyment, they are there to get us to watch the advertisements. If the ads weren't there, sure, we would watch it, but who would pay for it? Our cable fee's don't pay for it. Ads pay for it.

    If we skip the ads and watch the show, does the network care? No, they didn't make money off of us. If they make a show that's so good people watch live, then they can make money.

    Why is that so hard for some to grasp?

    Nielsen tracks same day, next day, 5 day, and 10 day views of a show, but the advertisers don't care – they know the ads were most likely skipped in these segments. If it's not watched live, even though it's tracked, it doesn't 'really' get counted.

    It's not Nielsen's fault, it's our own fault as TV viewers. We want to watch our shows without the ads, then we're shocked when the show is canceled because we didn't pay for it. Are you surprised when your power is cut off because you didn't pay the bill? I didn't think so.

    I would encourage everyone asked to be a Nielsen home to do it. Yes, it's a hassle. No, you aren't paid nearly enough for it. Yes, it's obtrusive and it invades your privacy. BUT, you help influence TV, no matter how small, it's still the most used media in the world and influences so many across our country. The Nielsen families have the real power over what is on TV. Where else can you represent 20,000-60,000 other people.

  • riggssm

    Wow, sorry, disagree. I tolerate ads on broadcast television, sure. I even sort of understand the bi-weekly PBS pledge drives.

    But I pay for cable (quite a bit do I pay, thanks to the legal monopoly ComcastXfinityNBCUniversal enjoys in Boston!) and the shrinkage of unique content in favor of the latest new _____ that is a carbon copy of ______ is irritating.

    Remember when cable shows had no or very few ads? Because paying for the exclusive service was what made it different than the broadcast networks?

    Yeah, I'm only 32 and remember that. Now I'm paying for a premium product but still suffering ads? Um …

    • serus

      You aren't really paying for the premium product, you're just paying for access to it. HBO for instance. Most cable providers charge what, $10 a month for access? They don't have commercials. They do however get the majority of that $10, where Comcast gets very little. ESPN? They get like $2.50 from each subscriber. But these are the big boys. Lets look at Animal Planet. They're more like .14 cents from each subscriber. They can't 'live' as a network off that, they need the advertisements especially with the cost to produce shows today. Even PBS has advertisements since they lost federal funding. The BBC doesn't show ads in the UK, but they also have a huge TV tax on top of cable fees. Imagine paying your Comcast bill, then paying it again to the US Government to pay for PBS.

      • riggssm

        Back to the original premise: remember when cable shows had no or very few ads? I do.

        Perhaps instead of Animal Planet, Animal Planet 2, Animal Planet: Underwater, Animal Planet: In the Air!, Animal Planet: Insects, and Animal Planet: MON-STARZZZZZ!, there could be just Animal Planet.

        I really don't need 300 channels. I just wanna watch the Red Sox and MSNBC and occasionally some Miss Julia Child.

        (Also: I'd be happy to pay someone for more/better PBS!)

        • serus

          And the FCC ruled what, a year or two ago that you could get ala-cart TV, only choosing the channels you wanted. I still like the idea, but if you didn't have a bundle, think of how many networks would go out of business (not that it's a bad thing). So you can get it, but the cable companies won't allow it. They want $50 a month from you, not $12.

          • riggssm

            Not to be ornery, but I pay $120 per month, not $50. I'd be happy to just pay $50 … $12 would be a miracle worthy of veneration.

        • Ryan Poehls@facebook

          @serus I work far a company that provides Cable TV services. The topic of Ala-cart TV has came up but what subscribers/customers don't understand is that to have ala-cart options, you would probably be spending more money to have access to 10-20 channels than to have a bundle of channels. If Cable companies were forced to do this, the amount you would pay for say 20 channels would be more than the price of 70 channels or even a digital service of 100+ channels. A majority of cable companies don't make any money on customers who take the lowest package of channels. For example, a basic package of 20 channels or a Expanded basic package of 70 channels. Those packages by themselves don't make a profit for companies if that's the only service that the customer pays for.

      • Ryan Poehls@facebook

        @riggssm Cable channels now days have the same commercial time as Broadcast channels. You are probably referring to Premium Cable Channels like HBO, Stars, Cinemax, and Showtime. Those channels haven't changed at all as far as commercial time. The Satellite a Cable providers help generate new networks that can be accessed for viewing but still need to operate like broadcast television networks.

  • http://www.facebook.com/people/David-Shultz/1014018715 David Shultz

    You spelled our company name wrong, it's Rentrak, not Retrak

    • http://www.facebook.com/people/David-Shultz/1014018715 David Shultz

      Thx for fixing it :)

  • SFLTVMiro

    Nielsen is the currency for ratings, every other competitor that has attempted to enter the market has crashed and burned. Arbitron being one that comes to mind.

    From what I know, TV stations are contractually prevented from using a second or third ratings service alongside Nielsen. If they wish to have access to their Nielsen ratings they can't use anyone else, if they do they're subject to fines.

    Ad rates are based on Nielsen ratings so it's a catch 22.

    One of our stations here in South Florida, WSVN is attempting to get the courts to rule Nielsen is a monopoly but have so far been unsuccessful. When the electronic people meters were turned on in South Florida the station lost 49% of its audience overnight when paper diaries were used. And they lost around $1million per month in advertising.

    Nielsen maintains its old diary system was accurate, and the new people meters are also accurate. Go figure

    • serus

      It probably wasn't the switch from the diary to people meter, but probably the switch from a non-people meter meter, to the people meter meter. Makes sense? hehe…They had a system that didn't measure how many people were watching or who was watching, it was a box that you didn't have to 'log into' but it still recorded what you were watching. They made the switch a few years ago to people meters in most of the markets, and some stations saw a huge decline in viewers. It was the way they were counted, one was passive and didn't know who was actually watching. So you could leave the room for hours with the TV on. The other (people meter) requires people to log in, and push a button every half hour. You can't just leave the TV on and walk away anymore.

      I doubt the diaries were ever very accurate. I remember seeing one that was supposed to last 2 weeks. It was filled out for two weeks. It started on like October 10th through the 24th, but was postmarked the 13th. How can you fill out what you watched if it hasn't aired yet? People just put in the shows they liked, without necessarily watching them. Besides, who the hell would actually write down that they liked Jerry Springer?!

  • Elizabeth

    The next step in the media revolution has to be throwing Nielsen out the window and then spitting on its fragmented remains. It's absolutely irrelevant.

    Also, whoever thinks TV is still free (for the price of your time watching the network's advertising) is fooling themselves. Cable companies pay each network it airs millions of dollars a year for the privilege, to the point where if there's a contract dispute, they pull their programming. Who foots that bill, ultimately? The viewer.

    The only way for television networks to survive the digital age in the black is to stop relying so heavily on advertising. There is a place for commercials, but viewers aren't watching in the same way anymore, so networks have to start doing things differently, too.

    These days, at least half the shows I watch are online. Of those, at least 15% are downloaded "illegally" because there is no legal alternative (e.g., shows from the UK). BBC, iTV, and Sky could be monetizing the small American market for their programs, but they aren't.

    If they can't keep up with technology, they will be left in the lurch. Plain and simple.

  • http://www.facebook.com/people/Damien-Otis/537310234 Damien Otis

    Yo. I think that "Arrested Development" is a bad example of a show that ended before its time. AD wrapped up SO NICELY. I'm glad they didn't drag it out. Sure, I would have loved it had it gone on longer, but it really did end when it should have.

  • Katatak

    I actually registered just to make a comment. Yes, you are right on all of these, but there's also something more sinister in the works. I once worked for a market research company called Cunningham Field and Research Services. Nielsen was one of our biggest clients, and we did movie and TV surveys for them.
    The problem is that most of the data is sadly all made up by employees. The clients set unattainable quota goals and the management tells their employees "I don't care how you do it, get the numbers." I quit because I could not falsify data, but they were having us tell 10 year olds to "pretend they are 80 and they'll get 5 bucks." When they couldn't find enough people to lie, employees would simply make it up. We called these "TBEs" which stood for "to be entered." This allowed the company to maintain it's client base (Nielsen amongst others) while still meeting those ridiculous quotas. Don't trust market research data. I personally only trust academic research findings which publish their methodology (I'm a researcher, it's what I do!) Thanks for this article, I enjoyed it.

    • Rowan

      Thanks for the first hand info.

      I used to get called for marketing focus groups all the time. Without variation, I noticed that any focus group participants didn't respond favorably to the product being marketed, they would stop getting called on after a comment or two. And there would be no follow up by the person running the focus group. Everything is bent to make their clients happy – not tell them what they need to know.

      As for the Nielson ratings, system is antiquated. They don't even report the internet downloads or streaming for several weeks after the fact (which is dumb because you can see the amount of hits and how many are unique hits immediately).

  • http://www.experiate.net Paul Flanigan

    Here's another thing to consider: out-of-home advertising. Right now there is an organization in NYC called the DPAA – Digital Place-Based Advertising Assn that is trying to build a metric for measuring impact with advertising on screens in public spaces. The metric is not related to Nielsen at all, which is surprising since, based on your post, Nielsen would be most interested in this for the networks because they have a big influence on those screens.

    Great article, John.

  • HernandezBobby

    I don't believe this article highlighted enough the important fact that ONLY 25,000 HOMES ARE BEING TESTED!! Isn't that far more important than the ratings being based on ads? 25,000 is a long shoot from 300,000,000. So only an insignificant amount of the population is being tested. So ratings have a 0.0% accuracy rating then.

    • Ryan Poehls@facebook

      @HernandezBobby I agree with you. This is a flawed system. Who knows how their demographics are distributed. Take a Sci-Fi show as an example. Of the 25,000 people being tested. Maybe a few hundred watch a particular show. In terms of science fiction fans, you jump outside that 25,000 people and multiple more millions of fans of a show aren't accounted for if they watch the program live. So a show that they rate as a 1.1 million viewers could easily be at 5.0 millions viewers or higher. So when the show gets canceled, they are upsetting more fans than they realize.

    • pizzahair

      Fill a jar with 100,000 red buttons and 100,000 blue buttons.

      Now reach into the jar and randomly pull out 1000 buttons. The ratio of blue to red buttons in the sample you just pulled will be about 1:1 or 500 blue and 500 red.

      As long as the sample size is big enough, it will accurately reflect the population.

      • Mark Asher

        That's a good example, but Neilson's sample size is a lot smaller. Nielson uses 25,000 out of 300,000,000. So instead of pulling out 1000 buttons out of 100,000, you only pulled out 8. Now how accurate is that?

      • Lisa Schmaltz

        If you read my other post, you'll see that the reflection is not exactly true. Take into consider what they can capture. I have a surround sound system that goes through the entire house but it is not connected right now, and I have wall mounted tvs and a second local surround sound system in our tv room. THat was all "too sophisticated" for them. So who are they including??

      • Yaya2012

        There is a major issue with your comment. Your illustration had an equal amount of red and blue. The Nielsen boxes aren't nearly close to being half of all TVs owned!

        • bill fozzy

          Maybe you should re-read it. I don't think you understand.

          • Yaya2012

            Thanks so much! I did misunderstand it due to my location while I was reading (room full of loud kids). I understand it now. I also did some back tracking after I posted my last comment. I went through my old Econ books (Econ major) and remembered the CLT (Central Limit Theorm).

      • Jack Dundee

        That's assuming all human-beings are identical in every way.

      • Birdie Wolf

        Except the sample size is so much smaller. Your example speaks of a 1% of the total population. The Nielsen family pool is only .025% of the their total population size. This is like pulling out 25 buttons. No way that will accurately represents the population. HernandezBobby is right. This is an insignificant number of people being tested. This only works because the higher ups pretend that it does, when it is really a sack of horseshit.

  • http://www.facebook.com/people/Amlcar-Ortega/601705386 Amílcar Ortega

    Party Down, forever in my heart. And hard drive.

  • Suzanne Marie Lanoue@facebook

    The Nielsens are not inaccurate just because they don't measure what YOU think they should measure. They measure what they are supposed to measure: who is watching the commercials. The networks are businesses. They mostly just care about making money. Commercials are what they make money from, not the TV shows. The TV shows are only there to entice you to watch the commercials, period. The networks tried their best to block DVR's and VCR's by suing, but they failed. If they had their way, we would all be watching live TV, glued to the programs as well as the commercials. Technology and lifestyle has moved beyond that, and they have no way to fix it. They ARE trying to come up with a new ratings system, but it takes years. Nielsen ratings are all they got. The technology they use and the polling methodology they use is perfectly valid and accurate. Why shouldn't they cancel a show if it is not making them money? Your watching TV on a DVR or on Hulu or iTunes without commercials does not make their sponsors money. Watching on their website does not make them nearly as much money. Perhaps one day they will be gone and we only watch pay-per-view, but for now this is all we got.

  • Ryan Poehls@facebook

    I've always felt that the Nielsen ratings were inaccurate depending or the viewers demographic that the show was targeting. For example, Science Fiction and comedy shows tend to have a younger demographic and the viewers tend to watch their shows more from the DVR, online and other sources. For example, SyFy has recently canceled 2 popular series due to "low" Nielsen ratings. SGU and Caprica. My theory is that if you take the shows rating and multiply it by 3, you will get a more accurate reading of total viewer-ship of the show. For example, Family Guy was canceled but was picked back up due to DVD sales. So if SyFy wan't to know SGU's true rating. Take the 1.1 by Nielsen, multiply by 3, and you will have a rating of more like 3.3. With that rating, SGU would be on to Season 3.

  • Joan Brigham

    Glad to know how the Nielson ratings work. For years I have suspected that they were skewed and not accurate. Many shows that nearly all the people I know like, were cancelled, and many that were not good at all continued. What Ryan Poehls said is dead on. So can we change the way it is done, already? It can save some good shows.

  • smith10vet

    I am a nielsen family member and I know for a fact that nielsen does count Netflix, Hulu, iTunes and on-demand cable. The sensor on my laptop accounts for Hulu and itunes. They ask about netflix use. and on demand is recorded. You obviously dont know what your taking about, and they cant see how many people are watching during commercials because the equipment cant do that. Not to mention they hundreds of channels with different timed commercials. Actually do some research before bashing a good American company.

    • Frank White@facebook

      @smith10vet

      A good American company. And what exactly would motivate you to say that, if you aren't working for them? Aside from the fact that they monitor your viewing habits, what do you know about them and how they do business? Probably not very much.

      And you completely overlook what should be obvious to a 12 year old: that their sample base, of which you're a member, can't possibly be truly representative of the viewing habits of the over 300 million people in this country. Fact.

    • johnsonjohnson

      youre just biased because you get like 50 bucks for doing nothing. There is no way that a sample of 50,000 "average" americans gives an accurate depiction of what the entire country is watching. With on demand, hulu, netflix, dvrs, online downloads and streaming there is no way that nielson is accurate. Networks should be compiling data from all cable and satelite companies on what is recorded and watched on demand as well as neilson data about what is being watched live. If neilson didnt have a monopoly with the networks for providing rating we would have a more accurate number for our ratings and not just know that the 50,000 people that neede 50 bucks a month are watching

  • Jennifer Valenti@facebook

    You're off on a few things (Nielsen has 220,000 people and 140K households in their metered sample), but for the most part, well done. I think you'll enjoy my blog coming up this week on just this topic. As the former executive in charge of the Nielsen Television and Online Ratings portfolio, I know a thing or two as well. ;-)

    • jmskey

      This is wrong, it's approximately 20,000 Television households. Where did you get 140,000? Though that WOULD be a more agreeable number…

  • Jennifer Valenti@facebook

    As promised… Here is the blog on this very topic. It's long, but it is well worth the read. http://blog.fourelementsmedia.com/

  • http://profile.yahoo.com/BQTBNLTQXU5QE4DQEMREWU5TRM Jaime Reyes

    I want  accuracy in ratings.
    Nielsen is a joke.I've read stories that certain networks purposely overpay Nielsen hoping to get a couple of extra percentage points that in the end will bring more advertisers that of course means more money to that network for a show or newscast its trying to boost in the all important overnight ratings.It wreaks of fraud.I would rather trust Comcast.They have 20 times more cable boxes in homes in America than Nielsen has in People Meters, DVRs,and  other ancient ways they calculate ratings.
    Arbitron is just as bad.

  • http://www.facebook.com/christina.atteridge Christina Atteridge

    The cwtv.com app, has commericals… I wish, watching episodes online, would count toward the ratings, because, the show, The L.A. Complex, is doing good online

  • Lisa Schmaltz (toll free)

    I went through the entire process right up to the day of installation. Even after previous site visits the technicians told me my "setup" was too sophisticated for their equipment. I was NOT a happy camper. I'd wasted hours of my time in personal interviews, emails, answering tech questions, etc. But this brings up a bigger question…..just who are they capturing?? My setup is that I have 7 tvs (thats ok with them), satellite service, drvs, 2 surround sound systems and wall mounted tvs…nothing extravagant! Exactly who are they looking for? Anyone who is serious about tv is going to have my setup but perhaps not the two surround systems, but at least one! This is a joke…. Also, I met another Nielsen family and they said the mess from having the equipment was unacceptable so maybe I'm better off…..

  • http://about.me/bradreason Brad Reason

    How does Neilson gauge the Internet?

  • Yaya2012

    I believe the sample size is way too small! They should do away with it, and somehow incorporate the tech into all sets. Someone on an earlier post illustrated a jar filled with 100,000 blue buttons, and red of the same amount. Wrong analogy! That would only be an accurate example if the sample size was half of the viewing population, which is clearly not the case. ( I see this article is about a year old, so sorry for this out of date comment)

  • Mike

    I work for the Nielsen TV ratings and i know for a fact it is 100% inaccurate. What's sad is we have market cornered since we are the only major company doing the ratings. Without a doubt the most unethical company around that literally doesn't care for the employees or their families and kids. I hope one day someone has the money to get the company in court and expose the fraud that the Nielsen Company really is.

    • The Company is…

      Mike – we are that company and would like to speak to you in absolute privacy – trying to piece together the bread crumbs to your argument..

      • Joey L

        Hi "The Company is…" or Mike – I would love to chat with either of you about the ratings system. I am working on a project and have some information that might be valuable.

        Thanks!

  • Asashii

    yes, every poll ever taken doesnt work, because thats all the ratings are, but polls sure help predict everything from Presidential elections to upcoming laws being voted on to …..well everything!

  • http://theadonisgoldenratio.blogspot.com/ Tracy Reynolds

    With digital television, just like a website, their data is all there. Did Google avoid collecting traffic data when that it became big? Little, mainly because increasing data collection on a digital system scales really very well. Nielsen's model is adonis ratio old (which they basically admit) and, in my opinion, not worth arguing for as strenuously since one are.

  • Can'tViolateNDA

    This piece is factually challenged. There may only be 25k ratings boxes hooked up to Nielsen families, but each rating period actually surveys somewhere in the neighborhood of 250,000 to 350,000 households across the USA, across the wide range of demographics.

  • kc

    I just want 10 channels and no commercials. USA Network, TBS, Comedy Central, CSPAN, a Local News/City Hall Station, a national/global news station, an educational TV station (History/PBS), 1 sports network (ESPN), Cartoon Network, and a movie channel (HBO). I'd like to get these 10 stations and high speed wireless internet for under $60 a month. If a company could do that for me I would absolutely stay loyal to them. Seriously these ratings are bull-sh*t. I do what-ever I can in my power to avoid watching commercials. This could be anything from changing the channel, fast-fowarding (if the option is available), or even shutting off the TV and doing something else while I wait for the show to come back on. The only thing more boring, mind numbing, and horrible than watching commercials is listening to them on the radio. I literally get nauseous and want to puke when I hear commercials on the radio. I'm sure there are a million other people just like me. There needs to be a better way to gage the quality of a TV show and its followers than the number of people that stay on the channel to watch sh*tty commercials. BTW… (for you old folk that means By The Way).. BTW i don't even like to watch commercials that seem relevant to me. If I want information on a product or service I will research it online. I will not be inspired by some TV commercial. Also, please, for the love of god, stop airing commercials for food when no restaurants in the area are open, this it torturous. Just for your demographic information: I am a 22 year old college graduate living on the east coast. I am a part of the demographics Nielsen claims are most important to them.

  • Stephen Lemelin

    I remember being in college in 1998 and being in class hearing these same arguments. This is nothing new. Heck we even had a person in class who used the old paper documents. So trust me it was much much worse.

    2 Things were agreed on as facts even back then.
    1. The system sucked. And the hope was technology would make it better, which compared to the paper diaries of 15 years ago it is better by what I am reading in this article.
    2. No one wanted to pay the money to fix it, because it is just "Entertainment", not say defense, medicine, etc.

    Last we have to understand that society in a state of rapid change and has been since the internet. Before that it was relatively easy, heck before cable really got going in the 80's there were generally 3-10 TV channels in a market. So from the 50's to the 80's it was easy. Cable made it harder, but the model was still in there just with more choices.

    The issue now is so many disruptive technologies. Meaning Amazon, Youtube, DVR technology, Netflix, torrents, DVDs, etc. Plus how many people are not really "watching" TV but more of having it on for noise. So what I am saying is be glad we have something, even if it is not that good.

    Last statistics 101. A sample of 25,000 is not bad even for a "possible" viewing audience of 300,000,000. First I say possible because: not everyone watches TV and some people like babies don't count. But for simplicity's sake lets use the number. If you read the attached article you will see that 25k is an amazing large sample size and the margin of error is probably less than 1% so in real terms mathematically Nielsen is doing great work. http://www.skepdic.com/refuge/ctlessons/ch7.pdf

    Just look at polling data from the last election, certain parties chose not to believe Nate Silver a famous mathematician who used complex polling, big data and modeling data in the last election http://mashable.com/2012/11/07/nate-silver-wins/. My point is that with better models and big data (better computer power/data crunching) Nielsen data could be better going forward, but given cost/benefit analysis at this time I don't believe they will invest in it yet. That being said if their data is less useful to both networks and advertisers it will change, when that will happen I don't pretend to know.

  • stratomartin

    A great article and I think it's hysterical that a ragtag bunch of statisticians try to perpetuate this mad consensual mass delusion and apply business metrics, and fantasy, to a impossibly complicated calculus. Nevermind that this whole absurd process applies to 1955 television and not at all 2014 media consumption patterns.

    Really funny part is that nobody in the history of television, or especially in the history of Internet, has ever watched a commercial from stem to stern. Or more accurately I should say, listened to a commercial from beginning to end.

    The mute button is the most powerful button on any remote control or desktop keyboard and the torture of embedded advertising stops instantly when that small button is tapped. All of that unwelcome advertising is 100% ineffective when there is no soundtrack.

    Advertising destroys content. Everyone knows this.

  • Lonestarr022

    Here's something that's a better gauge than nielsen ratings, tweeting about what you watch. Is it possible for a show to have crap neilsen ratings, yet trend on twitter? and unlike nielsen, it's not a private party.

  • Arbitron family

    I found your post looking for an idea of how big an influence my ratings have on the number networks get.

    But this article is very out of date now, two years later. Now you carry a small pager-like device which hears everything you hear (it even has a headphone attachment) and has a motion sensor. I'd love to read an up to date analysis of how it works now, because as a 100% ipad TV viewer I can't fast forward through ads. And their ratings suffer if the connection is so buggy I give up, or they don't allow me to watch their content (coughhulucough).

  • lakawak

    Here is the thing…no one cares what YOU think is good TV. You are an adult with the mentality of a child who thinks that everyone must think the way you do. That is sad and pathetic. Why do people still use Neilsen? Because they are not children who think that the field of statistics is wrong just because they don't like the results. You see…advertisers use Neilsen..and when they do, they also back up Neilsen's data with that of their own. So they know that when they advertise during programs that Nielsen says has the highest ratings, they see a bigger increase in sales than when they advertise on programs with smaller ratings. Everyone involved in marketing is far smarter and better than you in EVERY way imaginable at life. And they are using actual data to back up their beliefs that the Nielsens are accurate to a very low margin of error. Despite what whining children think when they find out that the show they like actually sucks,

  • lakawak

    As for Neilsen being a monopoly…that is simply not true. There ARE others. And there is nothing stopping even more from popping up except for the fact that the marketing agencies know how accurate Nielsen is so there is no need to use someone else.

  • CaptPanda

    I think the ratings system needs to change now. too many great shows are getting canceled because their audience chooses to watch it on their computer than the live broadcast.