SkepticblogSkepticblog logo banner

top navigation:

Health Information on the Internet

by Steven Novella, Aug 06 2012

Most Americans have used the internet to look for health information (a recent survey reports that 59% of adults have searched for health information on the net). Yet there are serious concerns about the accuracy and reliability of that information. There have therefore been many studies looking at the accuracy of health information, and not surprisingly the results are concerning.

Most of these studies pick a specific topic and then have one or more experts on that topic review websites obtained through specific search terms. For example, a British study looking at the treatment of fever in children concluded:

Only a few web sites provided complete and accurate information for this common and widely discussed condition. This suggests an urgent need to check public oriented healthcare information on the internet for accuracy, completeness, and consistency.

Only 4 of 41 websites they reviewed gave complete and accurate information. Another study looking at information for carpal tunnel syndrome found:

Thirty-three percent of the sites sold commercial products for the evaluation or treatment of carpal tunnel syndrome. An additional 30 percent were commercial web sites that did not sell products. Only 23 percent of the sites were authored by a physician or an academic organization. Fewer than half of the sites offered conventional information. Twenty-three percent of the sites offered unconventional or misleading information. The mean informational value of the web sites was 28.4 of a possible 100 points.

There has also been a systematic review of studies that review health information on the internet. This review found:

Most frequently used quality criteria used include accuracy, completeness, readability, design, disclosures, and references provided. Fifty-five studies (70%) concluded that quality is a problem on the Web, 17 (22%) remained neutral, and 7 studies (9%) came to a positive conclusion. Positive studies scored significantly lower in search (P =.02) and evaluation (P =.04) methods.

Therefore most studies concluded that the quality of health information on the internet is poor, and those that had a positive evaluation were less rigorous. From existing research it seems fairly clear that the quality of health information on the internet is variable and generally poor. This is not surprising given the “wild west” nature of the internet. Anyone can buy a domain name, set up a reasonably impressive website, and then start spewing whatever nonsense they wish. Other sites may intend to provide accurate information, but it takes a lot of work to provide accurate and up to date health information, so therefore high volume sites, especially those that are monetized, likely take many shortcuts that compromise quality.

There are many efforts to improve the quality of health information on the net, or at least provide consumers a way to judge the likely accuracy of the information they find. Several studies have looked at predictors of accuracy of information. A recent study of information about infant sleeping, for example, found:

Government and organizational websites had the highest percentage of accurate information (80.1% and 72.5%, respectively).  Blogs, retail product reviews, and individuals’ websites had the highest percentage of inaccurate information regarding infant sleep safety (30.9%, 36.2%, and 45.5%, respectively).  News websites were accurate only one-half of the time.

This is in line with previous studies that gave similar results. The most reliable sites are institutional and education sites (whether government, university, or patient-advocacy). These are sites that likely have the resources and access to dedicated professionals to ensure high quality information. They also have another thing in common – their professional reputation is their most treasured asset, and so they care about their public face on the internet. Having wrong or misleading information branded by their institution for the world to see is something to be avoided at all costs.

Individual websites are highly variable – they run the full gamut from an individual professional blogger (journalist, scientist, or other expert) to cranks or misguided crusaders of all stripes. Even the best individual sites, however, still represent the knowledge and opinions of an individual. This by necessity (and yes, of course this includes my own humble blog) is limited to an individual’s perspective and biases. It’s always a good idea, therefore, to balance that with the opinions of others, especially institutions that can represent a consensus of expert opinion.

Commercial sites generally should be looked at with skepticism. The bottom line is that they are trying to sell you something, so their motivation is very different from an academic institution. They may be directly trying to sell you a product, and all of the information they are providing is essentially a commercial (perhaps disguised deliberately as news or an information resource). Or they may simply be driving traffic to their website, and providing provocative opinions and information is a good way to do that.

Another study also looked at indicators of accuracy and found slightly different results:

Three indicators correlated with accuracy: displaying the HONcode logo, having an organization domain, and displaying a copyright. Many proposed indicators taken from published guidelines did not correlate with accuracy (e.g., the author being identified and the author having medical credentials) or inaccuracy (e.g., lack of currency and advertising).

The HON code stands for “Health on the Net” and is a seal of approval, meaning that the HON organization has looked at the material and found it to be reliable. It makes sense that this should be a predictor, because this review can weed out blatantly inaccurate, pseudoscientific, or commercial sites. Displaying a copyright is interesting – I wonder if that is just a marker for having institutional resources.

It is interesting but not surprising that having a medical degree did not predict accuracy. In my experience many crank or commercial websites use degrees and credential as a mark of legitimacy – it’s a marketing strategy. It apparently is very easy to find someone with an MD or PhD to endorse your product, and often times they are part owners of the company. Also, having a degree is no guarantee that one is not a crank.

Also interesting is the fact that having advertising on the website is not a negative predictor. At times I have run Google ads on this site, for example, and I often get comments from people that use this fact as an indicator that the site is somehow less legitimate. Placing a strip of ads on a website with decent traffic, however, is an easy way to support the site. In my case it generates enough money to pay for bandwidth, and that is about it, but it’s nice for a site to at least pay for itself.

Having generic ads on a site to cover costs, however, is different from a highly monetized or commercial site – making millions off of selling products, and gearing their editorial policy toward maximizing sales.

I also found a number of studies looking at the process of evaluating websites itself. The internet (despite the fact that many of us take it for granted now) is a relatively new phenomenon in our culture. Methods for evaluating information on the net are still evolving. One study looked at inter-rater reliability among the experts that are evaluating the information for accuracy. They found:

The medical experts showed a low agreement when rating the postings from the newsgroup. Hence, it is important to test inter-rater reliability in research assessing the accuracy and quality of health-related information on the Internet. A discussion of the different measures of agreement that could be used reveals that the choice of statistic can be problematic. It is therefore important to consider the assumptions underlying a measure of reliability before using it. Often, more than one measure will be needed for “triangulation” purposes.

In other words, when more than one expert is used to evaluate information, they often do not agree with each other about the accuracy of the information. Further, there are various statistical methods that can be used to measure how much they agree, and these produce different results. The authors therefore conclude that multiple experts should be used, and multiple statistical methods should be used to measure their agreement.

The same study also found that research that uses a single expert to evaluate health information may be unreliable itself. The knowledge and opinions of an individual, even an expert, are quirky.  In general it is more reliable to use a consensus of multiple experts than any individual.

Conclusion

We are still living in the wild west of the internet. The net is becoming a critical resource, and health information is near the top of the list. It is difficult, however, to create and maintain accurate resources of health information. The topics are complex and the information is constantly changing. In my experience the more controversial the topic also the more misinformation there is out there clogging up Google searches.

There is no single guarantee of information accuracy. The features I discussed above, however, are a useful guide, at least to a first approximation of which information is likely to be accurate. There are some additional rules you can follow also.

- Don’t rely on any one site. If the information is important, verify it on multiple reliable sites.

- Always seek out the counter opinion for anything that seems controversial. Until you have heard both sides (or multiple sides) to a controversy, do not think you understand it or which side is likely to be correct.

- Don’t be afraid to get help from those with more knowledge and expertise than you (including your personal doctor). Even when information on the net is accurate it may be difficult for the non-expert to understand and put into proper context. I see patients every day with information they got from the internet which is not inaccurate, they are just unable to put the information into a proper context and apply it properly to their situation. In many cases I see patients or family members very confused by accurate information, but they don’t know how to plug that information into the specific situation they are dealing with.

And – as always – be skeptical.

GD Star Rating
a WordPress rating system
Health Information on the Internet, 5.0 out of 5 based on 12 ratings

Recommended Reading

29 Responses to “Health Information on the Internet”

  1. Max says:

    Tim Farley created a custom Google search that searches only skeptical sites. Try searching for homeopathy.
    http://www.google.com/cse/home?cx=000281116881046286326%3Audsjvfvuqvs

    I think there was a custom search for health information, but I don’t remember where.

  2. Max says:

    How do these studies know what’s accurate and reliable?

    • Max says:

      Here’s one.
      http://www.ncbi.nlm.nih.gov/pmc/articles/PMC349389/

      “The accuracy and completeness of these Web pages was determined by comparing their content with that of an instrument developed from authoritative sources on treating fever in children.
      Fever in Pediatric Practice, by El-Radhi and Carroll and the Merck Manual of Medical Information were used to develop this instrument. This instrument was checked by a physician.”

      So that would be 100% accurate by their definition. Then go straight to the Merck Manual of Medical Information.
      http://www.merckmanuals.com/professional/index.html

      • Max says:

        Does the Merck Manual count as a commercial site?

      • Old Rockin' Dave says:

        In my past career as a physician assistant, I had recourse to the Merck Manual on occasion, and I never saw anything that pointed readers to any Merck product. Drug companies produce several books that are commonly used as pocket medical references and usually the company name on the cover is the only advertising. Others have all the ads on a couple of pages together at one end or in the middle, and they are easily (and usually) ignored. Sometimes a drug company will distribute a standard pocket reference and have the company name printed on it. The Sanford guide to current antibiotic usage is one that I used to see often. I knew Dr. Burt Meyers, the author, and he is not a shill for any drug company.

      • Max says:

        Well it doesn’t help when Merck creates a fake medical journal to advertise its products.
        http://scienceblogs.com/insolence/2009/05/04/when-big-pharma-pays-a-publisher-to-publ/

      • MadScientist says:

        It certainly is. The company is so damned huge (one of the largest chemical companies on earth) that you’d expect the usual mix of honest folks and woo peddlers. I call the Merck Manual “The Hypochondriac’s Bible”; I have an aging copy of it somewhere …

    • Daniel Clements says:

      Read the article: “Most of these studies pick a specific topic and then have one or more experts on that topic review websites obtained through specific search terms.”

    • tmac57 says:

      I think Max’s question is a reasonable one.By analogy,if a study came out stating that 70% of all tape measures sold by hardware stores were inaccurate,you might want to ask,”compared to what?”

  3. I will immediately snatch your rss as I can not in finding
    your e-mail subscription link or newsletter service.
    Do you have any? Kindly let me recognize in order
    that I may just subscribe. Thanks.

  4. Bob Marshall says:

    The simple solution is always use NHS Direct.
    Try it.

  5. CountryGirl says:

    You are correct about the internet but it is also true everywhere. The doctors who have a regular show on TV often misinform or make mistakes. Doctors offices and hospitals do as well. Even the “experts” who publish studies often get it wrong. Once coffee caused bladder cancer then it didn’t now it is supposed to protect you from cancer, who knows what the next study will say. The worst cause of misinformation is a result of lawyers suing doctors or pharmaceutical companies because they have a vested interest in creating paranoia and spreading disinformation. I take coumadin and once when I was diagnosed with multiple pulmonary embolisms I was admitted to the hospital for treatment. One of my nurses confided that she wouldn’t take coumadin but would seek homeopathic and natural remedies instead. I regret that I didn’t turn her into the hospital administration for that.

    • MadScientist says:

      Did the nurse say why? Was it because the drug is also used as rat poison? Atropine is reputed to have been popular with some families for eliminating political adversaries, but atropine has many uses in medicine. Nitroglycerine is still a very common industrial explosive and is also used in medicine. Antibiotics poison bacteria. From a chemist’s perspective, the majority of drugs are poisons.

      • CountryGirl says:

        Nurses don’t “prescribe” medicine. I have had numerous operations and spent days at a time in ICU. I loved my health care professionals especially the nurses. They are the cornerstone of our health system. But, they don’t prescribe medicine. That is what the nurse was doing that was inappropriate.

      • Max says:

        Nurse Practitioners can prescribe medicine, but it doesn’t sound like your nurse prescribed anything.

      • CountryGirl says:

        Not accurate. A nurse practitioner works under a doctor and the doctor is the prescriber.

        Indeed my nurse prescribed: 1: she contradicted the doctors prescription (which is prescribing) and said I should not take coumadin. 2: She told me to use specific plants/herb instead, which is also prescribing.

        My complaint is not that she has an opinion or even stepped outside her job. My concern is she doesn’t know her job. That is to suggest that I, with a life threatening condition, ignore the doctors advice and choose highly questionable alternatives, is criminal and contrary to her training.

      • Max says:

        Not accurate. Physician Assistants work under a doctor. Nurse Practitioners can open their own practice. A suggestion like “I’d use herbs” is not a prescription.

      • CountryGirl says:

        Max you are intentionally being obtuse and incorrect. If I suggest you try herbs it is indeed a suggestion. If a nurse in a hospital setting suggests you don’t take your prescribed medicine but instead take herbs to accomplish your medicianl goals it is a prescription.

        A nurse practitioner requires a collaborating physician in order to be able to prescribe medicine.

      • Max says:

        “A nurse practitioner requires a collaborating physician in order to be able to prescribe medicine.”

        Not in all states. In your case, I don’t even know if the nurse was a nurse practitioner, and whether the coumadin was prescribed by a doctor in the same hospital. You said you took it before you were hospitalized, but whatever.

    • Medina says:

      I do not understand your conclusion that lawyers are the worst cause of misinformation. Unlike commercial ventures, lawyers have to prove their case in court. If the evidence is not there, few lawyers will take the case. Any links to studies that support your assertion that lawyers cause this misinformation?

      • CountryGirl says:

        Much/most misinformation about health care and medicine is the result of and used by lawyers in the pursuit of money. I disagree with your idealistic view of lawyers. They take cases where the person being sued has money not because the evidence is there. In fact a common ploy when there is no wrong committed and the person/company being sued is totally innocent is to settle for less then it would cost the innocent person to defend themselves in court. Your chances of being sued increases with your wealth and has little or nothing to do with any act or omission on your part. I cite the currect crop of ads by lawyers to inform you that the asbestos producers have set aside $20 billion to settle claims against them and the lawyer will assist you in making your claim. When that $20 billion is gone do you think the lawyers will continue to “assist” you in making your claim??? No, they will be off to chase another ambulance or follow another wild claim of HFCS causing cancer or whatever.

  6. MadScientist says:

    There are a few excellent sites of course (if they have the information you’re looking for) such as the CDC website. Another thing that concerns me is self-diagnosis. Even when information is accurate it tends to be written for physicians; most people don’t understand that most symptoms are common to numerous ailments and you do need quite a bit of experience to diagnose the less common illnesses. It would be nice if there were a boilerplate disclaimer stating that correct diagnosis involves a lot more than simply looking up some symptoms and that medical professionals should be consulted if someone believes they may have an ailment.

  7. Beelzebud says:

    This is a symptom of a broken healthcare system.

  8. d brown says:

    Talk about “Paranoia runs deep”

  9. RoboSapien says:

    I had to have a tooth extracted yesterday that was damaged beyond repair due to abscess (still high on vicodin yay), but I had been in pain for several days prior. In my exhaustive search for home remedies, the most common treatment to turn up was colloidal silver. Ugh.

    So I gave up and relied on common sense, aspirin, acetominophen, and saline rinses until I could get in to see the dentist.

  10. Ribozyme says:

    What, no specific site recommendations?

    Come on, Dr Novella, I’m sure you know of more than a few websites that specialize on medical information that you can recommend.

  11. Ann says:

    They didn’t suggest a root canal instead of of extraction?

Optimization WordPress Plugins & Solutions by W3 EDGE