There are two main mistakes made about medical evidence:
The first is the idea that evidence has validity in the abstract, rather than being dependent on the context in which it is seen.
The second is that the only valid ‘objective’ medical evidence is what can be observed by someone other than the patient, and that the patient’s experience is ‘anecdotal’ and invalid.
The need for context
As Karl Popper has pointed out, evidence is not absolute, and
… the belief that we can start with pure observation alone, without anything in the nature of a theory, is absurd … Observation is always selective. It needs a chosen object, a definite task, an interest, a point of view, a problem. 
Some expectation of outcome is needed in order for a test to be constructed in the first place, and what determines this expectation is the ideas of those conducting the test. As a result the evidence produced by a test will be interpreted on the basis of preconceptions. Thomas Kuhn demonstrates this with reference to John Dalton's work on the compounds CO and CO2. John Dalton was a meteorologist who was interested in the action and composition of gases at the beginning of the nineteenth century. His contemporaries had noted that
… the two oxides of … carbon contained 56 per cent and 72 per cent of oxygen by weight … (2)
This did not reveal any obvious relationship between the two oxides, but when Dalton suggested that
… one weight of carbon would combine either with 1.3 or with 2.6 weights of oxygen … a 2:1 ratio leaped to the eye. 
As a result
… the same chemical manipulations assumed a relationship to chemical generalization very different from the one they had had before. 
In other words the same evidence had a different meaning depending on the way it was seen.
When it comes to testing medical treatments, the same principle holds true. The currently prevailing approach is based not on comparing results with general theoretical predictions (the scientific method), but on simply comparing bodies of evidence with each other. Thus the responses of those in the control arm of a randomised controlled trial (RCT) are compared with the responses in the treatment arm, and the overall outcome is compared with clinical experience. But, as with the oxides of carbon, no matter how accurate the information in this evidence, in the absence of a suitable theoretical framework, its true significance cannot be recognised.
‘Objective’ and ‘anecdotal’ evidence
This problem is made worse by claims that only ‘objective’ evidence is important, and that the reports of patients are merely ‘anecdotal’ evidence and therefore an invalid basis for assessing treatments. Such an view depends on a fundamental misrepresentation of illness. Both orthodox medicine and homeopathy recognise that illnesses involve both signs (conditions observable by others, such as a bruise) and symptoms (conditions of which only the patient is aware, such as a pain). What is more there is no absolute relationship between them, since signs can be present without symptoms and symptoms without signs. Thus an athlete may die from a heart attack without there having been any conventional symptoms of heart problems preceding the fatal attack. On the other hand a person may constantly report symptoms of heart problems without there being any observable cause for these symptoms.
Perhaps the most commonly occurring symptom which is treated without any attempt at identifying associated signs is a headache. In this case it is the patient who identifies both the existence of the illness and its disappearance, but no sensible medical practitioner would claim that it is merely an anecdotal illness. Only if headaches are frequent or very severe would a practitioner expect to make more detailed investigations to identify other possible problems associated with the symptoms.
In short, it is unscientific to claim that medical effectiveness or efficacy can be determined by evidence alone, and particularly if this is allied to an insistence on narrowly defined ‘objective’ evidence.
K R Popper, Conjectures and Refutations, 2nd edn (London: Routledge and Kegan Paul, 1965), p. 46, quoted in Henrik R Wulff, Stig Andur Pedersen and Raben Rosenberg, Philosophy of Medicine: An introduction (Oxford: Blackwell Scientific Publications, 1986), p. 22.
Thomas Kuhn, The Structure of Scientific Revolutions, 3rd edn (Chicago and London: University of Chicago Press, 1996), p. 134.
Thomas Kuhn, The Structure of Scientific Revolutions, 3rd edn (Chicago and London: University of Chicago Press, 1996), p. 133.
Why it works