The effect of visual and numeric risk information on surgeon behavior: a randomized, clinical vignette experiment

Abstract

INTRODUCTION AND OBJECTIVE: The treatment patients receive depends more on their surgeon than their disease. Risk prediction tools aim to address this quandary by providing quantitative risk information, but their clinical impact remains limited. Because data visualization can improve information processing, we designed a visual risk display and tested its impact on surgical decision-making. METHODS: We engaged a panel of urologists to develop a multidimensional, multifaceted visual risk display through user-centered design. The panel further helped us develop clinical vignettes for favorable intermediate risk prostate cancer (A), high risk prostate cancer (B), small renal mass (C), advanced kidney cancer (D), BCG unresponsive, noninvasive bladder cancer (E), and muscle-invasive bladder cancer (F) where surgical equipoise exist. From May–June 2022, we administered these vignettes to SUO and AUA members via Qualtrics, randomizing them to the visual risk display, numeric risk information, and control. Risk estimates were generated from existing risk prediction tools. For each case, respondents rated how likely they would recommend surgery (5-point Likert: strongly against to strongly for surgery) and selected health services they would obtain prior to surgery. Responses were compared using Fisher’s MC exact test with significance level set at 0.05. RESULTS: In total, 333 urologists participated: 104 in the visual arm, 111 in the numeric arm, and 118 in the control arm. Respondents were mostly fellowship-trained (71.2%) and in academic practices (60.1%). A strong or somewhat for surgery recommendation varied by case (A: 19.2%, B: 65.2%, C: 51.9%, D: 51.3%, E: 52.3%, F: 68.8%) but did not differ statistically across arms (p=0.210–0.967). In contrast, the selection of healthcare services such as preoperative care, cardiac clearance, and pulmonary clearance did differ statistically across arms depending on the case (Figure). CONCLUSIONS: Risk information—whether visual or numeric—did not affect a surgeon’s recommendation for surgery, highlighting the strong intuitive nature of surgical decision-making. However, the selection of health services appears more movable, indicating a role for risk prediction tools in optimizing surgical readiness and clinical outcomes.

Publication
Journal of Urology (Volume 209, Issue Supplement 4)