UX Roundup: Product Reviews | New Website | Wizard of Oz | Learning AI | AI in Education | Eyetracking
Many short pieces that don't warrant mailing out in separate newsletters. Collected here for your reading enjoyment.
Reviews Warn Against Bad Product Usability
It used to be "buyer beware" for the usability of products (like the Ford T), but now consumer reviews often warn customers of bad design.
It was probably difficult to use this old-time product, but you wouldn't know until after you bought one. ("Ford Model T" generated by Midjourney.)
Permalink for this piece: https://www.uxtigers.com/post/product-reviews
Product reviews are becoming increasingly focused on usability and UX, which is good news for customers. Companies can no longer impose awful design on customers as reviews provide a clear warning to potential buyers. Without reviews, customers may only discover unpleasant user experiences after purchasing the product because one doesn’t get to use it in depth without owning it. But when reviews cover usability, consider yourself warned.
Here’s an example of a car review from a recent edition of Consumer Reports. (A magazine in the United States specializing in product testing and comparisons.) An expensive new car is assessed as having “taken a sharp turn away from user-friendliness.” I’m not paying $52,000 for a vehicle where adjusting the climate controls is difficult (and thus dangerous).
Follow me on LinkedIn for more UX updates like this https://lnkd.in/gfffgNcr or subscribe to my newsletter to receive the full text of all my new articles by email as soon as they are published:
UX Tigers: Jakob’s New Website
To have a permanent home for my new articles, I started a new website, UX Tigers. It’s “your tiger team for user experience insights” aiming to serve the UX community world-wide.
You will still get new articles directly by email if you subscribe to my newsletter, but permanent links will live at UX Tigers.
Wizard of Oz
Experience the magic of the "Wizard of Oz" user research technique in this captivating video. When using this method in a study, the illusion of a user interface prototype is masterfully crafted by a human maestro behind the scenes orchestrating the "computer's" responses to user interactions.
This ingenious method is a powerful tool for emulating prospective artificial intelligence user interfaces before the AI infrastructure is operational.
However, it's worth noting that this approach may falter when simulating products with superhuman intelligence, as the human-generated responses are inherently limited to the "wizard's" intellectual capacity. While this constraint is relatively inconsequential in today's AI landscape, it may pose a challenge in the not-so-distant future.
“Pay no attention to the man behind the curtain,” was a famous quote from the Wizard of Oz movie. This is the reason the user-research technique has this strange name. (Image by Midjourney.)
Learning AI
Ethan Mollick wrote a very useful overview of the current top AI tools, with recommendations for which to use when. My experience agrees with his recommendations, so this list is a good starting point when planning your own education in things AI.
UX professionals need a sense of urgency about AI, which will change almost everything about users’ work situation and the way UX work itself is done. If you don’t feel urgency, read my article and I can hopefully change your mind.
My #1 advice is to start now getting personal experience with AI, just as a user yourself. (Then, of course, even better to conduct some user research. Please notify me when/if you publish any empirical findings.)
By the way, after you gain hands-on experience (as opposed to relying on biased press reports), your analysis of the future impact of AI on UX may change. If so, please post as a comment. There is already one interesting study that shows that business professionals with personal experience using AI are more optimistic about its potential in business than are colleagues who don’t have such hands-on experience.
AI can Boost Education
An illuminating case study by an ingenious business school professor showcases the utility of AI-generated critiques of student presentations.
Students presented normally and their audio was fed into the ChatGPT mobile app’s speech recognizer. ChatGPT4 then analyzed the transcribed speeches and furnished customized feedback for each presentation.
“Student presentation” by Midjourney.
90% of students rated the AI feedback as somewhat or very useful, with a whopping 55% branding it “very useful.” (The professor doesn’t specify how students may have appraised his critiques, but in my experience being a professor *many* years ago, if over half of the student cohort is exceptionally satisfied, one is performing admirably.)
All of the students (100%) rated the AI feedback as being somewhat or very realistic and actionable, an astonishing outcome given that it doesn’t know anything that wasn’t on the internet in 2021.
The sole metric where GhatGPT4 garnered less-than-laudatory student scores was for insightfulness. Here, the majority found the AI only “somewhat insightful” in giving them ideas they hadn’t thought of, and only 25% of students rated it very insightful.
But just wait for ChatGPT5. Judging by ChatGPT4’s leap in aptitude for passing university exams over ChatGPT3.5, the next iteration will bring a marked increase in perceptiveness.
This modest experiment from a single professor and class provides hard evidence, however preliminary, of AI’s boons for education. More real-world applications, and their evaluations, will hopefully emerge swiftly.
The immense need for better education, especially outside elite institutions, represents one of the greatest benefits of AI: delivering consistent excellence to all, even minor colleges in developing countries. As I’ve said before, AI scales, humans don’t.
Apple Vision Pro a Potential Eyetracker
With a mere $3,500 price tag, the Vision Pro is an extraordinarily affordable eyetracking option compared to its $15K counterparts. Moreover, it boasts enhanced tracking precision, pinpointing the user’s gaze within 4K resolution. It comfortably fits the user, facilitating in-store studies of consumer scanning behaviors across supermarket aisles. The device's weight becomes a non-issue during conventional one-hour usability test sessions of websites, mobile devices, and enterprise software. Participants receiving $100 for an hour-long study won't bat an eyelid wearing a slightly weighty device they wouldn't sport casually all day.
However, we need a software solution to capture, analyze the eyetracking data, and project it onto the test facilitator’s monitor in real-time. Apple, most likely, has already implemented this as an internal development tool. They could make a considerable contribution to the #userexperience community by launching an eyetracking package. If not, we can expect a third-party solution within months of the product's debut.
Consider this a gratis business proposal for those envisioning kickstarting a software company on the Vision Pro platform. If anyone capitalizes on this idea, feel free to send me a review copy — I'll provide an honest evaluation. But remember, any such product's usability must be top-notch for user researchers, or I won't hesitate to critique it.
The most talented user researcher I have met in my 40-year UX career rightly posted a scathing comment on the usability of most current user-research tools. Let’s do better this time!
Until now, eyetracking has been too expensive and painful for most user research. Whenever I suggest employing eyetracking on a project, I can see the researchers visibly recoil from the idea. If I’m right, this hate of eyetracking will vanish once Vision Pro ships. If you do any form of user research or consumer research, put $7K into your budget for next year to buy two of these headsets and eye-track away!
“Eye” by Midjourney
About the Author
Jakob Nielsen, Ph.D., is a usability pioneer with 40 years experience in UX. He founded the discount usability movement for fast and cheap iterative design, including heuristic evaluation and the 10 usability heuristics. He formulated the eponymous Jakob’s Law of the Internet User Experience. Named “the king of usability” by Internet Magazine, “the guru of Web page usability" by The New York Times, and “the next best thing to a true time machine” by USA Today. Before starting NN/g, Dr. Nielsen was a Sun Microsystems Distinguished Engineer and a Member of Research Staff at Bell Communications Research, the branch of Bell Labs owned by the Regional Bell Operating Companies. He is the author of 8 books, including Designing Web Usability: The Practice of Simplicity, Usability Engineering, and Multimedia and Hypertext: The Internet and Beyond. Dr. Nielsen holds 79 United States patents, mainly on making the Internet easier to use. He received the Lifetime Achievement Award for Human–Computer Interaction Practice from ACM SIGCHI.
Subscribe to Jakob’s newsletter to get the full text of new articles emailed to you as soon as they are published.
More on AI UX
Suggested reading order:
AI Vastly Improves Productivity for Business Users and Reduces Skill Gaps
The Articulation Barrier: Prompt-Driven AI UX Hurts Usability
ChatGPT Does Almost as Well as Human UX Researchers in a Case Study of Thematic Analysis
“Prompt Engineering” Showcases Poor Usability of Current Generative AI
UX Experts Misjudge Cost-Benefit from Broad AI Deployment Across the Economy