October 3, 2011

Fielding a patient satisfaction survey is becoming de rigueur for medical practices, hospitals, ambulatory surgery centers (ASCs) and other providers.  One of the early versions of an accountable care organization (ACO), California’s Integrated Healthcare Association, ten years ago launched a pay-for-performance program in which patient satisfaction accounted for thirty percent of the overall quality score. Patient-centricity and consumerism are as important to healthcare reform as quality and efficiency.

A substantial number of anesthesia and pain medicine practices have patient satisfaction surveys in place, judging from a quick search of the Internet.  Other groups are still wondering whether and how to implement surveys.  This is not surprising, since leading models have not yet emerged.  Indeed, although it is widely accepted that “[p]atient satisfaction is an important indicator of health care outcome and evaluation of the quality of services in anesthesiology … [t]he importance and lack of standardized, valid, and reliable questionnaires to assess patient satisfaction in anesthetic care have been emphasized in many reviews.” (Mui WC, Chang CM, Cheng KF, et al: Development and Validation of the Questionnaire of Satisfaction with Perioperative Anesthetic Care for General and Regional Anesthesia in Taiwanese Patients. Anesthesiology 2011; 114:1064-75.)

Both those practices that already have patient surveys on line (or on paper) and those thinking about them may benefit from considering questions asked on third-party instruments in order to determine the best instrument for their own use.  With that in mind, we offer a structure, a sampling of questions, and a caveat about patient surveys below.

Conceptual Basis and Structure for a Patient Satisfaction Survey

The first and fundamental issue is the purpose or purposes for which the survey is intended.  There are different objectives that a survey may further, and these will determine the questions to be asked.  Some purposes of patient satisfaction surveys are as follows:

1. Marketing
• To patients
• To the public
• To the hospital
• To the managed care plan
2. Satisfaction of requirements
• Hospital/ASC requires (or appreciates) physicians’ tracking patient satisfaction data
• Integrated Physician Association (IPA) or multi-specialty network
3. Preparation for participation in a future ACO or other value-based network
4. Learning what processes and outcomes truly matter to patients and improving one’s own performance

The fourth of these objectives is undoubtedly the most difficult and takes the greatest amount of resources.  In most instances, the survey needs to be designed with expert knowledge of psychometrics and statistical sampling methodology.  The technology – whether online, administered by interview (including by telephone) or mailed to the patient -- deployed must be both user-friendly and reliable. 

The information developed from such “heavy-duty” surveys may be extremely valuable.  For example, in a national survey of 250 randomly-selected, representative postoperative pain management patients, the authors “were surprised to find that more than half of the patients surveyed were concerned about experiencing pain after surgery and that this caused some of them even to postpone surgery.” (Apfelbaum J, Chen C, Mehta S, Gan T.  Postoperative Pain Experience:  Results from a National Survey Suggest Postoperative Pain Continues to be Undermanaged.   Anesth Analg 2003;97:534-40.)

In general, physicians who perform procedures can structure patient satisfaction surveys around the pre-operative, intra-operative and post-operative phases, with questions about the overall experience and space for narrative responses at the end.  This is the case for both surgical anesthesia and pain medicine.

The majority of the surveys reviewed for this Alert used a five-point Likert scale for answers.  According to Wikipedia, a Likert scale is a psychometric instrument that “is the most widely used approach to scaling responses in survey research.” Typically it consists of a numeric scale with a brief description of each number’s corresponding meaning, such as:

1. Worst
2. Less than acceptable
3. Acceptable
4. More than acceptable
5. Best

A variety of label types can be used (acceptability, proficiency, frequency, quality, quantity), and scale point definitions can be of various lengths. The common thread is that all Likert scales range from low to high, bad to good, less to more, and so forth, in a linear fashion.  Three-point, four-point and bipolar (yes/no) scales were also seen in the instruments we reviewed.

Anonymous surveys tend to generate greater response rates.  All of the instruments reviewed offered a choice between anonymity and identification of the respondent.

Sample Questions

A notable exception to the dominance of Likert scale ratings are the scales used in the Consumer Assessment of Healthcare Providers and Systems (CAHPS) program.  CAHPS (www.cahps.ahrq.gov) is funded and administered by the U.S. Agency for Healthcare Research and Quality.  Its Hospital (HCAHPS) and Ambulatory (Primary) Care surveys are familiar to many, having been in use for years.  In 2010, the CAHPS Consortium adopted the Surgical Care Survey sponsored and submitted by the American College of Surgeons and the Surgical Quality Alliance, of which the American Society of Anesthesiologists has been a key member.

The CAHPS Surgical Care Survey contains 45 items, eight of which are about anesthesiology:

The Surgical CAHPS questions about anesthesiology are very basic.  Because they are part of a perioperative multi-specialty survey that is likely to be adopted in OR suites relatively quickly, though, you might consider at least including them in any survey of your own.  At the other end of the complexity spectrum are the 30 questions developed and validated in the Miu study referenced above.  The Miu survey has a strong cultural bias, having been designed for use in Taiwan, but some of its questions would be relevant in U.S. surveys, e.g.:

• To what degree were you satisfied with the opportunities for you to ask the questions about anesthesia?  [Pre-operative domain]
• To what degree were you satisfied with the anesthesia service that the [anesthesiologists] were respectful? [Overall experience]
• To what degree were you feeling discomforted by nausea and vomiting perioperatively?  [Post-operative domain]
• To what degree were you satisfied with the waiting time in the whole process of the anesthesia service? [Pre- and post-operative domains]

Let us now look at some of the questions that appear in patient satisfaction surveys on anesthesia and pain medicine practice websites.  You may choose which, if any, to include in your own survey; repetitiveness here is due to the fact that this is a compilation.

Try reading these real-life survey items with this question in the back of your mind:  will the answer here give me information I can use to improve patient satisfaction or the quality of my care?  Or, as in many of these examples, will the answer be just a positive or negative response to an overly broad or overly inclusive question?

Pre-operative Domain

Anesthesia

1. Pre-operative phone call from nurse (i.e. attitude, manner, etc.)
2. If you received a pre-operative telephone call from the anesthesiologist, how would you rate the personal manner (courtesy, respect, sensitivity, friendliness) of the person with whom you spoke?
3. Did your anesthesiologist clearly communicate the anesthesia options?
4. Pre-operative instructions easy to understand 
5. Explanation of anesthetic and concerns addressed regarding anesthesia
6. How satisfied are you with the pre anesthesia clinic?
7. How satisfied are you with the explanations of the risks and benefits by the anesthesia staff? 

Pain Medicine

1. Telephone demeanor:  Were the staff polite and courteous on the phone?
2. Convenience of appointment:  Did we schedule you promptly?
3. Please rate the ease of scheduling your appointment
4. Please rate your visit with the doctor, medical assistant or nurse practitioner
5. Was the staff professional and courteous during every aspect of your visit?
6. Were all your questions/concerns addressed thoroughly and to your satisfaction?

Intra-operative Domain

Anesthesia

1. Professionalism of anesthesiologist/CRNA
2. Courteousness
3. Responded to my concerns
4. Skills/abilities
5. Explained anesthesia to my satisfaction
6. Performed anesthetic to my satisfaction

Pain Medicine

1. Was the staff professional and courteous during every aspect of your visit?
2. Were all your questions/concerns addressed thoroughly and to your satisfaction?
3. How would you rate the sensitivity and attentiveness of the doctor?
4. Please rate your visit with the doctor, medical assistant or nurse practitioner.

Post-operative Domain

Anesthesia

1. How satisfied are you with the postoperative pain control? 
2. How satisfied are you with postoperative nausea and vomiting?
3. My immediate post-procedure problems were addressed and treated effectively (i.e. pain, nausea, and vomiting).
4. Discharge instructions were clear and easy to understand.
5. Post-anesthesia phone call adequately addressed my problems/concerns.
6. Have you had a surgical procedure requiring general anesthesia in the past?
   • If yes, how would you compare the care you received from us to that you received from the previous practitioner?

Pain Medicine

No post-operative survey questions were encountered.  In particular, no pain medicine surveys asked whether the patient was experiencing less pain post-procedure.  Recognizing that we are dealing with chronic pain, patients’ subjective, explicit evaluation of the effectiveness of their treatment may provide important data.  Survey responses obtained within a week or two of the procedure should provide quicker feedback than a failure to return, or a subsequent visit with a demand for an excessive quantity of narcotics.

Overall/General Questions

Anesthesia

1. How satisfied are you with the overall care of anesthesia care team before, during, and after your anesthesia?
2. How would you rate your anesthetic experience in terms of:
• Convenience
• Comfort
• Privacy
• Anxiety
• Overall experience
3. Do you have any suggestions of ways we could better serve you or your family members?
4. I felt all aspects of care received were well coordinated and flowed smoothly.

Pain Medicine

1. Please rate our communication with you (i.e., phone calls, explanation of procedures, test results, medication, etc.)
2. How would you rate your overall experience with Pain Management Associates?
3. Please rate the level of Customer Service your experienced with Pain Management Associates
4. Do you feel positive enough about our service to refer family and friends? (Yes/No)
• If no, would you allow us to contact you?
5. What did you like about our services? (Text)
6. Is there anything we might change to make future patient experience more positive?  (Text)
7. If there is any way that Pain Management Associates can improve our service to you, please let us know. (Text)

Formulating the labels for responses is easier, by far, than formulating good questions.  Below are some of the scales encountered.  Some start at the lowest rating, which is typical of Likert scales, but where the first possible response listed is positive (e.g., “Excellent”), the instrument may be more appealing.  We assume that there is published research on the issue of whether starting with the most or the least positive answer biases the results.

SAMPLE RATING SCALES

&nbsp

A Caveat about Patient Satisfaction Surveys

Only a minority of the sample survey questions above are likely to yield useful information.  To allow quantification of a variable that can then be used to identify potential service improvements, the question must be specific and must address a single issue.  If, for example, 30 percent of responding patients rated the timely scheduling of their appointment “poor,” the practice would know what it needed to do.  In contrast, a 70 percent rate of “below average” answers to a question about the “convenience” of the “anesthetic experience” would not give a clue as to the nature of the problem to correct.  As for compound questions such as “Were the staff professional and courteous during every aspect of your visit?” a rating at the midpoint of the scale might mean that the staff as a whole was barely acceptable, albeit consistent, or that they were exceptionally responsive in all but one or two aspects of the patient’s experience.  For that matter, “professional” and “courteous” are not synonymous.

Some of the actionable questions, or questions that at least point toward the right follow-up questions, are as follows:

• Did this anesthesiologist encourage you to ask questions?
• How satisfied are you with the postoperative pain control? 
• Did your anesthesiologist clearly communicate the anesthesia options?
• Telephone demeanor:  Was the staff polite and courteous on the phone?

It is perhaps surprisingly difficult to write good survey questions for purposes other than “Atta boys.”  One way to reduce ambiguities is to beta-test the instrument on a variety of people.  Another way would be to ask patients what processes or outcomes matter to them, and then to survey those variables. 

That is not to say that the questions regarding the patient’s overall experience or the extent to which s/he “felt all aspects of care received were well coordinated and flowed smoothly” are without their uses.  They may be valuable in marketing, or in complying with third-party requirements – as long as the aggregate results are positive.  By way of a mini-caveat, if more than one individual has access to survey results, you should realize that their secrecy cannot be guaranteed.

As Alan J. Schwarz, MD, wrote on the Pg. 2 Commentary blog in his article Patient Satisfaction Eludes Us!  (May 2, 2011), “It is insufficient for physicians to pay attention to patient satisfaction when their motivations lie in the areas of decreasing the potential for lawsuits or increasing their compensation.”  Meaningful, not pro forma, measurement of aspects of care that matter to patients is indispensable in a value-driven and competitive healthcare environment.  Unfortunately tools that are adequate for the task are very difficult to locate, if they exist at all.

We at ABC believe that it is both necessary and possible to develop survey instruments that will identify ways to drive upward the value of anesthesia and pain medicine services to patients and to other customers -- surgeons, integrated delivery systems, hospitals and ASCs.  We are planning further study of this area.  As always, we welcome your questions and other contributions to the effort.

With best wishes,

Tony Mira
President and CEO