The Observer Effect!

The Observer newspaper today leads on an article reporting the "scandal" (their term, not mine) of patients waiting in ambulances outside hospital accident and emergency (A & E) departments, in an effort to meet a government directive that all patients should be treated within four hours of admission. This is yet another example of the observer effect where observing or measuring something changes what is being observed. Failure to take into account the observer effect seems to be a common fault of organisations (and the British Government in particular) and is, I suspect, due to an unthinking adherence to the aphorism: "If you can't measure it, you can't manage it".

There is a similar situation in British schools where children now have a regular regime of testing at various ages. All schools know that they are judged on their test scores so will change the way they teach to try and improve their test scores. Anything that is not tested is likely to be given low priority or dropped altogether, and the net effect of this is that we are training kids to pass tests rather than educating them. The main measure for GCSEs (the school exam for 16-year olds in the UK) is the number of passes at grades A to C. One strategy to improve this is to target children predicted to get C or D grades, because the children in the top streams will get C or better without any extra help, and those in the bottom streams won't get a C whatever you do. This is a classic case of the observer effect and it does not lead to good education.

In the case of the A & E patients the four hour response pledge seems to have made the situation worse as the measures taken are just to produce better figures not a better service, and tying up ambulances as waiting rooms stops them from attending new emergency calls.

Managing and planning is clearly more effective if you have reliable data but unless the observer effect is taken into account the collection or checking of the data can have seriously undesirable consequences.