A preview of the results of the homeopathy survey

Water dropYou may remember that last month some homeopaths posted a somewhat mysterious survey up on surveymonkey.com, possibly in an attempt to get some sympathetic data. Of course, skeptics picked up on this and crashed the survey. I had a feeling that this would cause the homeopaths to abandon the survey (which is now closed), so I decided to register for surveymonkey.com myself and copy it. I promised to release the results this month.

Before I continue, I should say that surveymonkey.com are pretty cheeky monkeys! You can look at the first 100 responses of a survey for free, but anything more than that costs money. Not a prohibitively large amount of money mind, but enough. Also, as I type this there are currently 864 responses, I’d love to see it hit 1,000 soon! So, in the meantime I thought I’d look at data from the first 100 responses.

Not surprisingly, the data from these first 100 responses has a distinctively skeptical flavour. Bear in mind that this is a survey, so the most you can take from it is a measure of public opinion and not scientific conclusions! Let’s take a look at the questions and answers.

1. Do you know what Homeopathy is?

An overwhelming 96% of respondents answered this question with “yes”. This at least shows that people answering the questions think they know what they are talking about!

2. If you had a health concern, would you consider supplementing conventional medicine with alternative medicine such as Homeopathy?

I suppose this is the question that tells you what people really think of homeopathy and 93.9% answered this with “no” (one person skipped the question). So, at the early stages of the survey, I think it’s pretty save to say that the majority of respondents were not homeopathy sympathisers!

3. Have you ever taken a Homeopathic Remedy?

Now is where it gets a little interesting. 47.4% of people responded “yes”. Perhaps these people took homeopathy at a 10:23 event? Maybe they tried homeopathy and found it to be ineffective?

4. Qualified Homeopaths are no longer permitted to explain how Homeopathy works or offer any evidence on their websites because of a ruling by the Advertising Standards Agency. Do you think Homeopaths should be allowed to explain how Homeopathy works?

This is a fairly bizarre question, because the overwhelming scientific consensus is that homeopathy does not work, so how can you explain a phenomenon that isn’t present in the first place? Surprisingly (and bear in mind how many people said that they would not consider taking homeopathy as a complementary medicine) 43% of respondents answered “yes”. Fortunately, this is the first question where people are allowed to expand on their answers, and these answers reveal some interesting logic.

I think they should be required to explain how it is supposed to work and how stupid it is.

Actually, I thoroughly enjoy watching homeopaths try to explain how homeopathy works. You can’t invent that kind of comedy.

Yes because once you know you realise you should just take a sugar pill but make sure that someone else gives it to you telling you it will make you better. Will work just as well.

These answers seem to be suggesting that homeopaths should be allowed to try and explain how homeopathy works so that their patients can judge how silly it is. I’m not sure that I agree with that! Others argued on a freedom of speech level:

If homeopathy is going to exist, its practitioners should be able to explain it. Censoring information isn’t a way to effect worthwhile change — and as it is in every health food store where vitamins can’t list the conditions they treat, people will step in and tell you. This is a really terrible idea if the point is to help consumers understand their options.

5. Qualified Homeopaths are no longer allowed to state which medical conditions they treat. If you visited a Homeopaths website, would you find it useful or not useful to know which conditions they can treat?

This is in a similar vein to the last question. The science says that homeopaths can’t treat anything (unless you count the placebo effect), yet 33% of respondents said that it would be useful to know which conditions homeopaths can treat. A few people wanted the homeopaths to be honest:

They can treat the diseases which can be treated with placebos. If they said that, at least pacients would know they are not supposed to try to get rid of their cancer with Homeopathy, for example

I would find it very useful, if homeopaths would have to state, what they really can treat. According to a lot of scientific research it’s just nothing, and that has to be stated!

Since the answer is nothing, they should state that.

It would be nice to see a homeopath’s website where you click on “Conditions which can be treated with homeopathy” and get a blank page!

6. Qualified Homeopaths are no longer allowed to give testimonials from genuine patients if those patients want to state that their health has improved as a result of homeopathy. (Testimonials means comments only from verifiable, genuine patients) Do you think testimonials giving details of improvement from genuine patients should be not allowed or allowed?

The premise of this question goes against the mantra of “the plural of anecdote is not data”. Personal anecdotes are not evidence of efficacy, and the results so far reflect this position, with 82.5% of respondents saying that homeopaths should not be allowed to give patient testimonials.

7. Why do you think Homeopaths are being treated in this way?

The final question, and most of the responses are along skeptical lines:

Because they are charlatans and the public needs to be protected

Because they are quacks. At best making money from the gullible, at worst preventing effective medical treatment of illness.

Homeopaths are treated this way because homeopathy is a pseudoscience, which should not be depicted as an alternative to actual medicine.

However, towards the end of the responses a few answers from homeopathy sympathisers crept in:

Big Pharma wants to eliminate competition.

And my favourite logical fallacy-laden answer:

In a nutshell, homeopathy is a gentle, effective and increasingly popular healing model, that poses a threat to the profits of pharmaceutical companies, who have gained the support of the media (James Murdoch on GSK’s board of governors) to push their attack on homeopathy in the hope of undermining competition. Why do the skeptics not get up in arms about the number of people treated on the NHS for negative reactions to conventional drug treatments, or the fact that meta-analysis of anti-depressants show they are no better than placebo?

As these two answers were towards the end of the first 100 responses, I have an inkling that news of the survey reached homeopathy sympathisers and that the numbers will be very different once all the responses are taken into consideration. Like I say, I’m hoping to get 1,000 responses before closing the survey, and I will make the results available on request. Please spread the word!

Take the survey

9 Comments

  1. David Waldock

    One of my problems with the original survey was the quality of the questions. This is obviously a commentary on those original questions, not your implementation thereof!

    Question one asks “do you know what homeopathy is?” doesn’t actually identify what they person answering the question thinks it is, it’s just a measure of confidence in their knowledge. So the groups I’d expect to answer “yes” to that are the people who are enthusiastic proponents and the people who are enthusiastic opponents of homeopathy. Since the remainder of the population are people who don’t really care much either way, they probably wouldn’t have bothered looking into it at all. So this question provides a measure of the degree of dogmatism held by the respondents. In short, we can conclude that people who hold their opinions strongly and are certain of their knowledge are the people most likely to be engaged in what is, in essence, a battle of ideologies.

    Question two asks if people would take a CAM therapy in addition to ‘conventional medicine’. ‘Conventional medicine’ is interesting turn of phrase which isn’t defined; I’d guess it means ‘Western, evidence-based’, but if you take convention to be representative of long traditions, then homeopathy could be considered to be part of the conventional medical armoury. It also qualifies CAM with ‘like homeopathy’: did the questioner mean “like” as in “doesn’t have supporting evidence” or like as in “homeopathy as a single example” or like as in “using ultra dilute preparations”? I don’t know. I do know that auricular acupuncture, for example, is popular in chemotherapy settings where it’s said to be effective [note: not efficacious, that’s a separate claim] again chemotherapy associated nausea, probably down to the breathing exercises. Is acupuncture included or excluded from what the questioner intended? If people said ‘yes’, it would have been interesting to see which therapies they would use…

    Question three is weak because, as you note, it doesn’t note WHY the person took the remedy, which is at least as important, and I would argue more important.

    Question four, depending on one’s view of how law develops, one could argue that homeopaths were NEVER permitted to explain how homeopathy works, but were doing it in defiance of the regulations which exist to protect consumers. However, that philosophical issue aside, the question contains the assumption that homeopathy works, and also fails to define “work” in the context of the question. For example, like many CAMs, homeopathic remedies have effects (ie. an effect can be seen after many people take homeopathic remedies), but it is not efficacious (the effect is related to the taking of the remedy, not the remedy itself). Given this, the intent of the answers to the questions is not revealed, unless people add comments. It’s clear from the quotes that most people intended this as “homeopaths should explain it’s a placebo”; I would also expect answers from proponents saying “it does work, that the ASA doesn’t accept the evidence we present just shows its a conspiracy” and answers saying “anything that works should be explicable”. I’m not surprised to see freedom of speech come up (although speech has always been subject to limitations regardless of claims to the opposite).

    Question five has the same epistemic problems as question four: because the terms are not clearly defined, their meaning could be interpreted differently by different groups. I’m going to go into this in more depth in the next question, but the question really comes down to a fundamental disagreement about the boundaries of acceptable evidence in the context. If a group accepts phenomena A as evidence then they will use this to justify their claims; if another group doesn’t, then they will ignore the evidence or seek to label is as “non-evidence”.

    Question six is the fundamental epistemological problem between proponents and opponents: the nature of what counts as evidence. The mantra “the plural of anecdote is not data” is quite true (it’s anecdotes). However, as a statement it comes from a very positivist epistemology, where everything is reducible to numbers. So, for example, I wouldn’t accept two physicists tale of having seen a Higgs-Boson and then having lost the data without reproducing the experiment. This is quite justified in an environment where all of the variables can be controlled and the observations show a relationship between the dependent and independent variables.

    However, in more complex, volatile disciplines, not everything can be reduced to quantitative data, and yet it still has value for providing insights into phenomena. At the extreme, anthropology is often considered a non-science (mainly by hard scientists) because, essentially, it illustrates by explication, by exploration of qualitative responses, rather than reducing it to numbers. And yet, some of the insights of qualitative sciences are invaluable in, for example, explaining social movements and beliefs. For a social scientist, the intended meaning of “the plural of anecdote is not data” is simply not true.

    The question is whether or not the efficacy of things like homeopathy can be reduced to numbers. Students of health sciences will tend towards the positivist end of the spectrum as say yes, we can reduce the effects of therapies to something which we can interpret using our senses. For example, we can use pharmacology and molecular biology to do experiments to explore the physiological effects of a treatment on biological processes. We can use epidemiology to statistically analyse groups of patients to compare homeopathic analgesia with paracetamol with placebo, and show greater effect where paracetamol is used.

    However, not everybody believes everything is reducible to numbers, and they would say that lots of anecdotes demonstrate a clear association between homeopathy and people recovering from their illness. Indeed, I think there’s a strong argument that this association (or even perceived association) should be explored as it would tell us a lot about how to make people feel better and healthier. There are, for example, a lot of common patterns amongst homeopathic anecdotes which suggest that certain groups are more likely to a) seek and b) report effects from homeopathic therapies.

    Then a further question is whether or not people should be able to report these sincerely held beliefs (in most cases; I think we’d all concede there are people for whom CAM is truly a scam, although how we differentiate the two is a somewhat difficult problem). For example, ASA doesn’t prevent people from claiming miraculous healing by god/s by churches (although it has now moved to prevent these claims being used to persuade people not to use mainstream medical practices). I’m not sure what the difference is between someone claiming they were cured of their alcoholism/broken bone/arthritis after prayer and someone claiming they were cured of their alcoholism/broken bone/arthritis after taking a homeopathic remedy. The ASA clearly sees a difference though; I would argue that until the rules are consistent across all claims of this pattern (which one might argue also includes claims such as “I was fat and ugly until I ate this yogurt” or “I had pimple until I used this facial scrub” or “I had wrinkles until I used this incredibly expensive combination of paraffin and mashed up vegetable matter”) that perhaps homeopaths should be able to make outrageous claims, purely on the basis of fairness (not because I’d endorse such statements).

    Question seven is interesting because it suggests at conspiracy. However, as previously noted, these conspiratorial concerns are not entirely illegitimate: they are being singled out, and whilst activists may seek for cosmetic products and churches to be subject to the same rules, at the moment, it is just homeopaths who are being picked on. Of course, the conspiracy theory goes on to promote the idea that Big Pharma is behind this (after all, Big Pharma doesn’t have any vested interests in the CAM markets, oh no!).

    Anyway, interesting to see the full range of responses, but I suspect it will merely endorse the idea that we have two entrenched factions debating an issue in ways which are completely incommensurable, and so the argument won’t be resolved any time soon 😉

    D

  2. I happened to save my responses to the “extended” questions on the original survey:

    Yes, so the potential victims can see what quality bullshit homeopathy really is.

    Helps me fill out my Bullshit Bingo card more efficiently.

    Homeopathic victims have been hurt enough.

    Reality is breaking out at last!

  3. A. Basic Fundamental Research
    B. High Dilution Research
    C. Clinical Research
    1. Double-blind Randomised Placebo-Controlled Trial
    2. Double-Blind Studies
    3. Cohort/Observational/Pilot Studies
    4. Systematic Reviews & Meta Analysis
    D.
    5. Homeopathy as a Genetic Medicine
    6. Evidence for specific disease conditions
    7. Homeopathy superior to conventional medicine
    8. Homeopathy cost-effective than conventional
    9. Homeopathy equals conventional
    10. Homeopathy superior to placebo
    11. Homeopathy improving quality of life
    E.
    12. Evidence-based Medicine
    13. To distinguish one homeopathy medicine from another
    14. To distinguish homeopathy medicine from water

    G.
    15. Animal Studies
    16. Plant Studies

    Papers related to the above areas are available. Which of them you would like to see?

    • Tom

      Nancy, if you give a link to a paper and state how it supports the use of homeopathy, I will read it and write a blog post about it. How does that sound?

Leave a Reply

Your email address will not be published. Required fields are marked *