Unbiased and Accurate Mattress Research:
Sources & Methodology

Updated January 2019

Sources

Message Boards / Forums

About 50% of our research data comes from actual owner experiences collected from over 900 online message boards or forums.

The boards / forums are usually focused on a specific subject, such as gardening, politics, mountain climbing, astrophysics or just about any other subject you can think of.

But occasionally an off-topic discussion takes place, such as one about mattresses. It is from such discussions that we collect our data.

The boards / forums generally work by someone posting a question and then others reply with answers. In this case, someone typically asks others what mattress they have and whether they recommend it, and then the others respond.

Message boards / forums are generally the best source for getting quality and unbiased mattress reviews from actual owners. An advantage of the boards / forums is that many people who post about their mattress have owned it for a significant amount of time and can comment on the critical issue of mattress durability and lifespan.

In addition, the boards / forums have the important advantage of often existing in near obscurity which means that most do not rank high in the search engines for searches related to mattresses or product reviews. This obscurity tends to ensure unskewed, accurate findings because it largely prevents people from posting who are merely looking for a place to vent about buying a mattress they don't like (or to gush about buying a mattress they do like).

Existing in obscurity also tends to keep out dishonest people from the mattress industry who seek to give fictional mattress reviews or biased mattress buying advice. In addition, dishonest reviewers are often discouraged by the fact that the boards / forums are usually made up of a tight-knit group of people who keep out suspicious contributors and posts.

Retailer Websites

About 30% of our research data comes from online retailers such as Amazon, Overstock, Wal-Mart, Sam's, and Costco. These retailers provide visitors with the ability to rate and review products, including mattresses. We have mostly found the reviews on these sites to be credible as well as non-selected by the retailer.

The main problem with reviews on these sites is that most of them are written by people who have recently purchased their mattress – as opposed to people who have owned their mattress for a long enough period of time to be able to comment on durability and longevity. When possible, we gather data only from the latter.

Mattress Manufacturer Websites

Mattress manufacturers often provide sections on their websites which allow customers to review mattresses purchased from them. While the number of reviews on such sites is often high, these websites – in our opinion – are often at least somewhat selective or manipulative about which reviews are actually published. Therefore, we usually select only a small sampling of reviews from these sites to be included in our research data.

Product Review Websites

A small amount of our data may come from established product review websites. These websites allow consumers to post product reviews. Generally, we regard these websites to have only fair credibility mainly because they are so visible and easily found (see "message boards / forums" above for the advantages of being obscure) and because oftentimes anyone can post a product review. Consequently, in our opinion, these websites' aggregate ratings for a particular mattress can differ significantly from the average owner's experience.

Sources Do Not Include

Reviews From Visitors To Our Website

Sleep Like The Dead does not seek or accept mattress reviews from visitors to our website mainly because we cannot verify their credibility. We instead rely on what we regard as more credible sources for getting information from actual owners, namely the above sources.

Personal Testing

Sleep Like The Dead does not personally or mechanically test mattresses because we regard this to be a flawed methodology. The central problem of this method is that it is based on a small sample. The experience of one person or small group is often a poor indicator of the average owner's experience. Factors such as materials / manufacturing inconsistency, break in, temperature, person size / body type, and personal bias / preference are all exaggerated in a small sample but are diminished in a large sample.

Other mattress-Review websites

Such sites are almost always biased as they often have direct financial relationships with mattress companies. Learn more.

Safeguards to Ensure Accurate Findings

Sample Size

Sample size refers to the number of consumer experiences that our ratings are based on. A sample size of 50 or more gathered using our methodology is for most mattresses large enough to provide accurate findings with a maximum margin of sampling error of ±4 percentage points. We believe this because no matter how much the sample size of a particular mattress model, mattress brand or mattress type may increase beyond 50, our findings rarely extend beyond the range of a few percentage points.

Source Diversity

Unless otherwise indicated, no one source makes up more than 60% of the sample size for any particular mattress. This ensures source diversity and prevents an over-reliance on any one particular source.

Time Standards

Data is seldom collected from mattress owners who indicate that they have slept on a particular bed for less than one week since this is often not enough time to form a solid opinion one way or the other. In addition, outdated owner experience data (older than five years) is usually not collected or is removed from our data set.

Owners only

In cases where it is not clear if the reviewer owns or owned the bed, the comments / review are not included in the data. Owners are identified when they openly say they own or owned a particular bed, refer to the bed as “my bed” or identify themselves as an actual owner in some other obvious, non-ambiguous way.

Review Quality

When gathering owner experience data from a retailer's website, such as Amazon, "verified reviews" are given precedence or non-verified reviews. If non-verified reviews are included in the data set, they and the reviewers are examined closely for credibility.

Owner Satisfaction Rating

The owner satisfaction rating for a mattress is determined by dividing the number of reviews / comments showing approval for the mattress by the total number of reviews / comments for the mattress (sample size).

Mattress Satisfaction Criteria

A mattress is considered satisfactory if any or all of the following conditions are met: an owner unambiguously states that he or she likes or approves of the mattress; the mattress is described by an owner as comfortable and or pain relieving; an owner claims that the mattress improves or at least does not harm sleep quality and or quantity; the mattress is recommended by an owner to others.

Mattress Dissatisfaction Criteria

A mattress is considered unsatisfactory if any or all of the following conditions are met: an owner unambiguously states that he or she dislikes the mattress; the mattress is described by an owner as uncomfortable and or pain causing; an owner claims that the mattress undermines sleep quality and or quantity; an owner urges others to not buy the mattress.

Only Mattresses Rated

The owner satisfaction rating refers to owner evaluations of the mattress only. In other words, the rating does not refer to the buying experience, return process, customer service or anything else other than the mattress itself. (Evaluations not directly related to the mattress itself are usually included under the Company Rating.)

Entire Span of Ownership Sampled

Mattress owner satisfaction is almost always highest early in the ownership process. This is because 1) the mattress is new and has had little time to develop durability problems, and 2) owner enthusiasm is high given that the new mattress is likely a clear improvement over the at least partially worn mattress that was replaced. Due to this fact, we collected owner experiences from across the entire span of ownership when possible to allow for a more accurate assessment of satisfaction. When not possible, we adjust owner satisfaction rates to a degree determined by our best judgment.

Correcting for Newness To Market

New-to-the-market mattresses often have fewer complaints for poor durability / longevity than more established mattresses because new-to-the-market mattresses have been in use by their owners for a shorter amount of time. As a result, they often have at least somewhat higher owner satisfaction than more established mattresses.

Sleep Like The Dead believes that it is both helpful to consumers and fair to mattress companies to consider how long a mattress has been on the market when determining owner satisfaction rates. Therefore, we adjust owner satisfaction rates (to a degree determined by our best judgment) to compensate for mattress newness. This helps to provide shoppers with an undistorted "apples to apples" owner satisfaction comparison of their mattress choices.

Correcting for Negative-Review "Scrubbing"

Some mattress companies appear to actively work to remove or "scrub" negative reviews of their mattress(es) from the web, including our sources, by a variety of means. If Sleep Like The Dead believes that this is occurring, we adjust owner satisfaction rates (to a degree determined by our best judgment) to compensate for this review removal.

Correcting for Return Policy Generosity

A number of mattress companies now offer a very generous return policy (no-fee, minimal hassle). Such a policy appears to (unintentionally) encourage some consumers to buy the mattress without giving as much consideration as they normally would to their purchase given that they have little or nothing to lose. Due to this less-than-normal amount of consideration, consumers are more likely to find the mattress poorly suited for them and to therefore be dissatisfied with it.

Sleep Like The Dead believes that it is both helpful to consumers and fair to mattress companies to consider return policies when determining owner satisfaction rates. Therefore, we adjust owner satisfaction rates (to a degree determined by our best judgment) to compensate for return policy generosity. This helps to provide shoppers with an undistorted "apples to apples" owner satisfaction comparison of their mattress choices.

Topper Use

If a mattress owner reports needing to place a topper on their mattress (beyond the break-in period) to make it comfortable, then this instance is regarded by Sleep Like The Dead as a negative evaluation of the mattress. However, if a topper is provided by the mattress manufacturer to the customer at no or discounted cost or if the manufacturer at least partially compensates the customer for purchasing a topper and the topper creates acceptable mattress comfort, then the instance is regarded as neither a negative nor a positive evaluation of the mattress.

Exchanges

If a mattress is uncomfortable and the customer is able to exchange it for a different mattress offered by the brand and the customer is satisfied with this change, then this instance is regarded as neither a negative nor a positive evaluation of the mattress brand overall.

Two Ratings For Some Mattresses

If more than 30% of owners of a particular mattress use it temporarily, occasionally or lightly, then two separate owner satisfaction ratings are often provided. The first rating is for everyday adult use, and the second is for limited adult use or everyday child use.

Color Codes / Letter Grades

The color-coded / letter-grade ratings show how a mattress performs against all other mattresses. "A" rating (dark green) or "B" rating (light green) means that a mattress performs above average; "C" rating (yellow) means average; "D" or "F" rating (red) means below average. Know that "D" or "F" rating does not necessarily mean that the mattress performs poor for most or all owners. It only means that better-performing mattresses are available.

Mattresses That Are Not Rated

Not all mattresses on the market are evaluated by Sleep Like The Dead (including those that might advertise on Sleep Like The Dead). A mattress is not rated on our site if we have been unable to locate a significant amount of quality owner experience data for that particular mattress from our research sources.

Below are the three main reasons why quality owner experience data might be lacking for a particular mattress:

Mattress Is New To Market

One to two years may be required to pass before enough quality owner experience data becomes available for a new mattress and mattress brand / company.

Mattress Is Low-volume Seller

For various reasons, a number of mattresses do not sell in a particularly high volume. As a result, available owner experience data is often inadequate for such mattresses.

Mattress is available only from Manufacturer

If a mattress is sold only by the manufacturer – that is, if it is not (or has never been) sold also by an independent retailer such as Amazon – then the availability of quality owner experience data may be inadequate.

Up-To-Date Ratings and Product Information

Most of our mattress ratings are current. The date near the top of most pages indicates when the page was fully updated with the latest ratings and product information including prices and specifications. The date has a "fresh" designation next to it when the page has been updated within the past ten months. Keep in mind, however, that even the most recently updated pages are not guaranteed to provide real-time information such as current prices and model selection as these can change at any time.

Truthful, Unbiased Content With Transparency and Disclosure

Our mission is to help people make informed sleep-product buying decisions, and this can only be achieved by providing unbiased and truthful information.

No Direct Relationship With Mattress Companies or Retailers

Sleep Like The Dead does not have (and has never had) any direct financial relationship with mattress companies. This means three things:

1. No part of Sleep Like The Dead is owned (or has ever been owned) by a mattress company.
2. We do not receive (and have never received) compensation from a mattress company to influence a review(s).
3. We do not receive (and have never received) commission from mattress companies – unlike nearly all other mattress review websites. In other words, we do not receive from a mattress company a percentage of referral mattress sales.

Sleep Like The Dead could almost certainly make more revenue by having direct financial relationships with mattress companies. But our mission is to tell the truth and to help the consumer the best we can, and being financially independent from mattress companies is absolutely critical to achieve this.

Advertising

To generate revenue to fund our research, we have advertising on our site. These ads are auto-generated and auto-selected by Google Ads based on several factors. Sleep Like The Dead does not select the advertisers nor the ads that appear. In addition, revenue generated is paid to Sleep Like The Dead by Google, not mattress companies.

Verification

Given that our sleep-product ratings are almost entirely based on multiple online public sources, the truthfulness of our findings can be independently verified by anyone with a computer and internet connection, given that they are willing to invest enough time and effort.