When a consumer decides to buy a used car, there are hundreds of websites that provide comparative data on price, safety, fuel efficiency, body, engine, etc. However, when a patient is planning an elective surgery for example, the sources of information comparing hospitals on the web are much more limited.

There are four main websites provide publicly reported data on hospital quality: Hospital Compare, Health Grades, Leapfrog Group and U.S News and World Report. Some sites use “structure” measures, such as the presence of a newborn ICU (Intensive Care Unit) or whether the hospital has a computerized physician order entry system that allows doctors to prescribe drugs to patients electronically. Other sites focus on “process” measures such as generally agreed-on safe practices or the number of procedures performed at the hospital per year. But all of them include “outcome” measures such as complication and mortality rates.

How reliable is that information and does it help patients make better decisions about what hospital to go to? To start with, the information available on these sites is at least two to four years old. Most importantly, there is little overall agreement between the sites about whether a certain hospital provides high quality care for a certain procedure or not.

A study that examined hospitals in Boston found that these sites failed to consistently agree on which hospital were top-performers and which were bottom-performers. A hospital that ranked first or second on one website often ranked last or second to last on another site. The main reasons for these variations are that the sites use different methods in their ratings, measure different processes and outcomes, focus on different time periods and include different patient populations.

However, even if the sites agreed on the rankings, most patients are not knowledgeable enough to make an informed decision based on the available data. For a heart patient for example, is it better to go with the hospital that performs most procedures, or the one with the electronic health record? For an expecting mother, is it the hospital with the newborn ICU or the one with the lowest C-Section rates.

New technology has made publicly reported hospital quality data easily accessible for patients. However, as long as this data is inconsistent and hard to understand, it is unlikely to result in better patient decisions.