If I tell you that I've been having gastrointestinal trouble, since I started consuming Splenda itself and products that use Splenda, you would rightly say that the evidence of a connection is just anecdotal. That term is used to dismiss a lot of data gathered by amateurs, Actually, when accurately reported, I prefer anecdotal evidence to statistical evidence, just as I prefer in-depth interviews with likely voters to telephone surveys. The conclusions of a statistical study carried out at a university or health organization often turns out to be misleading or just plain wrong, because of the risks inherent in the design of population studies.
Continuing on with my "Slenda" story; I went to the web to see what I could find. Lo and behold, there was a forum which contained many first hand accounts of gastric distress accompanying the use of Splenda and a few that said complainers were all crazy. The first defect in such forums is that it automatically selects people that are in distress. If I started a forum for people who have experienced sneezing fits after turning on their computers less than an hour after eating, there would immediately be 50 contributions from people who had that experience. It's just the statistical nature of the huge number of people browsing the internet. However, I'm not quick to dismiss honest data. Outliers can contain important information. Every experiment deserves to be explained.
My question for this blog is this. Astronomers have used thousands of amateurs to scan the skies that are too vast for professionals to monitor. Can the anecdotes of amateurs be filtered and combined in such a way as to produce valid scientific evidence? If one hundred people who report stomach distress stopping after quitting Splenda, are asked to restart in order to see if the distress starts again, can we draw any conclusions. How many times must we reproduce the start-stop cycle with each of these amateurs before we can have some confidence in the results?
Thanks Joel for this great new Topic.
ReplyDeleteAnecdotal evidence may be from many different types of sources, listed here in order of perceived credibility:
1) It happened to me personally and appears credible. (I.e, it does not violate Laws of Nature or probability and there is some sort of physical evidence it is not something that I misinterpreted or imagined, etc.)
2) It is said to have happened by someone I know personally and trust and it appears credible.
3) It is said to have happened to people I don't know personally, but is reported by a source that has been credible in the past.
4) It is said to have happened to people I don't know personally, but is reported by a number of sources I don't know but have no reason to distrust.
(This list could be extended, but you get the idea.)
Anecdotal evidence may be about things with different levels of probability.
Extraordinary claims require extraordinary evidence.
An example of amateur data gathering is the Surface Stations project I mentioned in a Comment on this Blog. Hundreds of amateur volunteers reportedly surveyed 1003 of the 1221 official temperature monitoring stations in the US. The project was organized by a known Global Warming skeptic and most of the volunteers were certainly skeptics who might have biased their observations accordingly. However, each station report was accompanied by a photograph taken by the volunteer and a location that could be checked via Google Earth, where many areas have high resolution imagery where the stations can be seen.
I believe this survey because anyone on the non-skeptic side could go to the given location and take his or her own survey and post a photo to put the lie to the original report - weather stations are always outside and visible!
So, the above amateur data gathering, though anecdotal is, IMHO, quite credible.
On the other hand, I am skeptical of anecdotal evidence (and even statistically correct scientific studies) that depends upon the qualitative reports of even large numbers of people. For example, I recently read of a scientific study that gave people ineffective placebo pills. They told one group the pills were very expensive and the other group they were cheap. The expensive pills were perceived to be quite a bit more effective.
There are three types of illness: 1) Things like a broken bone that will almost certainly heal given standard care, 2) Things that will heal themselves given time, with or without professional medical care, and 3) Things medical science cannot reliably heal. Placebos work like a charm on (2) and (3). Indeed, that may be the best way to reduce health care costs.
So, in you case, just stay away from Splenda and reading about Splenda and I guarantee your gastric distress will go away, whether it is actually caused by Splenda or caused by reading anecdotal reports about Splenda. (Unless, of course, your gastric problems ara caused by something else).
Ira Glickstein
In the case of Splenda the whole point is that it's supposed to substitute for sugar. I can't detect the difference, so it's easy to detect a placebo effect. All I have to do is get Joanna to bake me some goodies with and without. Maybe I can con her into baking two chocolate pies with Splenda and with sugar. We'll see what happens. Today, in a non-blind study, I'm off Splenda, but I'm eating grapes which I've been suspicious might be the cause rather than Splenda.
ReplyDeleteHere are a couple of sites that discuss amateur-professional cooperation in astronomy. I wonder if a properly trained group of amateur health researchers might not be able to accomplish what statistical studies cannot.
Anecdotal data are very important in biology because there are so many individual genetic or epigenetic differences that do not follow any normal distribution. Allergies are a common example. The gross statistics of penicillin and peanuts show them to be safe, but they kill certain individuals. (My wife nearly died of anaphylactic penicillin shock.)
ReplyDeleteWarnings on possible side effects of most drugs are so numerous that it is ridiculous. For legal protection many even list “sudden death.” That is a serious "side effect"!
Howard
I wonder whether we could write some advice (a knol?) or a set of general rules for people who want to experiment on themselves. My own experience with allergists and other diagnosticians has not been encouraging. Many people resort to the web in order to figure out what their doctors cannot fathom. It seems to me this is the opposite of statistical studies where there is one postulate and a large number of subjects. Here we have one subject and a large number of postulated causes or cures. Especially when people are getting on in years and are prone to crypto-injuries, it may be valuable to understand how to experiment efficiently.
ReplyDeleteGreat idea Joel, write a Google Knol! You have a personal involvement and your academic background and writing ability qualifies you to do some excellent research, so go for it!
ReplyDeleteMy experience with writing and publishing Knols has been very good. While Knols, IMHO, are not nearly as extensively reviewed by the "cloud" of volunteers that makes key topics on Wikipedia so reliable, publishing a Knol is much easier. If and when you publish one on this subject, please announce it via a new Topic on this Blog. advTHANKSance.
Ira Glickstein