UX Experts Misjudge Cost-Benefit from Broad AI Deployment Across the Economy
Summary: Most UX professionals underestimate the measured productivity gains from current AI tools, whereas they overestimate the growth of the economy in recent decades. Combining these two mistakes means that UXers don’t realize how much AI will boost living standards if we embrace it.
Permalink for this article: https://www.uxtigers.com/post/misjudge-cost-benefit-ai
The verdict is in from my Quiz on how AI usability influences productivity and the economy: the average score stands at 65%. Despite one participant's claim of the test being a breeze, the results beg to differ.
The most challenging question revolved around the average labor factor productivity growth in the United States, with only 39% answering correctly. It appears the stagnation of productivity growth in recent years has left many in disbelief. This stark contrast with past growth rates fuels my enthusiasm for the research findings that underscore the transformative potential of current AI tools.
And imagine the possibilities if future AI tools are crafted by UX professionals — we could witness an unprecedented surge in productivity growth.
For a comprehensive understanding of the current research and the quiz's context, read my article: https://www.uxtigers.com/post/ai-productivity
79% of the quiz respondents self-identified as UX professionals. Thus, even though there was a smattering of AI experts, business executives, and other groups among respondents, the data overwhelmingly reflects the UX pros, so that’s how I will discuss the findings.
A common misconception that leads people to overestimate the economy's growth rate is the conflation of their personal economic progression with the nation's overall productivity growth. This misinterpretation stems from the natural human tendency to over-emphasize their personal experience as representative of the general public (which is why we need the UX slogan You ≠ User). As business professionals mature from their early twenties to their forties, they experience a significant enhancement in their work proficiency, fueled by the accumulation of crystallized intelligence. This personal growth trajectory is often mirrored by a corresponding salary increase. Very little of that extra money comes from general productivity improvements, but as an individual living within the stream of time, you can’t tell the difference between you getting better and all workers getting better.
The question that stumped the second-highest number of participants addressed the productivity surge experienced by programmers utilizing the GitHub Copilot. 42% got this one right. Intriguingly, a majority of respondents significantly undervalued the profound enhancement in programmers' performance when bolstered by AI assistance.
Even more intriguingly, underestimating the gains from AI support for this highly-cognitively-demanding job was shared by the programmers themselves. Even the people in the study couldn’t believe how good the AI had been at helping them develop code, when asked after the experiment to estimate their own productivity gains. (Since this was a between-subjects study, each programmer only developed the code once, with or without AI help, so there was no way for each individual to know how fast he or she would have been in the other experimental condition.)
Respondents made the most mistakes in the two money-related questions (“Coins” by Leonardo.AI).
Combine people’s tendency to overestimate the productivity growth without AI and to underestimate the measured productivity boost from an AI deployment. This striking dichotomy could explain why many UX professionals remain hesitant to fully integrate AI into their professional repertoire, see https://www.uxtigers.com/post/ux-urgency-ai
If you (falsely) believe that business-as-usual is progressing nicely and that the gains from embracing AI are small, then I understand why people are hesitant to make the scary move to a revolutionary new technology. However, the data shows the opposite: the economy has been fairly stagnant in recent decades and AI is an incredibly powerful accelerator that’ll boost our living standards to an extent not seen since the 1950s. (Likely by more! And this time it won’t just be Americans and Europeans who enjoy better living standards: AI will cause a worldwide boom, probably more so outside those old-economy regions.)
AI UX Article Series
This article is part of a more extensive series I’m writing about the user experience of modern AI tools. Suggested reading order:
AI Vastly Improves Productivity for Business Users and Reduces Skill Gaps
ChatGPT Does Almost as Well as Human UX Researchers in a Case Study of Thematic Analysis
“Prompt Engineering” Showcases Poor Usability of Current Generative AI
UX Experts Misjudge Cost-Benefit from Broad AI Deployment Across the Economy (this article)
(more next week: follow me on LinkedIn to see in your feed)
About the Author
Jakob Nielsen, Ph.D., is a usability pioneer with 40 years of experience in UX. He founded the discount usability movement for fast and cheap iterative design, including heuristic evaluation and the 10 usability heuristics. He formulated the eponymous Jakob’s Law of the Internet User Experience. Named “the king of usability” by Internet Magazine, “the guru of Web page usability" by The New York Times, and “the next best thing to a true time machine” by USA Today. Before starting NN/g, Dr. Nielsen was a Sun Microsystems Distinguished Engineer and a Member of Research Staff at Bell Communications Research, the branch of Bell Labs owned by the Regional Bell Operating Companies. He is the author of 8 books, including Designing Web Usability: The Practice of Simplicity, Usability Engineering, and Multimedia and Hypertext: The Internet and Beyond. Dr. Nielsen holds 79 United States patents, mainly on making the Internet easier to use. He received the Lifetime Achievement Award for Human–Computer Interaction Practice from ACM SIGCHI. Follow Jakob on LinkedIn to see future articles.
Jakob Nielsen can speak at your company or event. More info: speaker bureau BrightSight Speakers or email tom@brightsightspeakers.com
Presentation topics:
10 Foundational UX Insights
AI & UX
UX: What Why How (and Who When Where)
UX: Past, Present, Future
UX Fireside Chat with Jakob Nielsen (AKA AMA)