UX Portfolio Reviews and Hiring Exercises in the Age of Generative AI

Aug 16, 2023

Summary: Embrace AI or become a dinosaur. Job candidates must showcase AI skills in their portfolios. Hiring managers must change recruiting processes to select applicants with combined UX and AI abilities or be stuck with employees of little use next year.

Leveraging AI in creating your UX portfolio or crafting solutions for hiring exercises is unequivocally NOT cheating. Given that UX professionals are expected to harness myriad AI tools in their daily work for optimal productivity, it's not just acceptable but preferred, for job applicants to do the same. Utilizing AI tools during your job search signifies that you are an innovative professional aligned with the digital era and committed to efficiency through appropriate means.

Permalink for this article: https://www.uxtigers.com/post/ux-portfolios-hiring-exercises

Would you dismiss a visual-design candidate for utilizing Photoshop in his or her portfolio creation? Encountering a visual-design portfolio solely composed of hand-drawn images would raise doubts about the candidate’s proficiency with contemporary tools.

By no means am I implying that mastery of Photoshop, Figma, or other current design tools is the hallmark of a great designer. However, the inability to utilize software design tools will hinder your productivity as a designer unless you secure a niche role sketching initial storyboards for Pixar by hand.

Just as you wouldn’t reject a visual designer for having Photoshop expertise or a portfolio containing visuals produced in Photoshop, UX candidates shouldn't be penalized for employing AI tools. For instance, it’s expected that ChatGPT or equivalent tools would handle the summarization of user interview transcripts — the key is to articulate how you utilized the tool and interpreted its output to inform your design project.

Similarly, since AI boosts creativity — especially in the divergence stage of the UX design process — we should expect all UX job candidates to showcase how they (a) use AI to generate ideas, and (b) winnow down those ideas, using their judgment.

Do you create all the visuals for your UX portfolio like this? If not, then it’s acceptable and expected to employ AI tools in creating your portfolio. (“Drawing a design” by Midjourney.)

Presenting Your AI Use in the Portfolio

Honesty above all. Don’t pretend you did everything manually if you used AI tools. Though some Jurassic-era managers may decry AI, don't fall into the trap of deception. While you probably don’t want to work for dinosaurs, all reasonable managers will reject candidates who unethically pretend to have produced deliverables differently than what they did.

If you want to work at a company mired in antiquated paradigms, brace for a test of archaic skills: proving that you can produce deliverables without AI. If so, either step away from that job opportunity or be honest in producing the requested work product without AI assistance.

Hopefully, you’ll mostly apply for positions at enlightened companies that embrace the new world and that like to save money by allowing you to use modern tools. (The smartest UX managers will demand that you use AI because they won’t want to pay for you to produce at half speed without AI.)

The crucial point is not if you used AI but how you did it. Show examples of early AI-produced drafts and explain why you rejected those ideas. Explain how you teased the AI into producing something better for the second round. Make the world see how you selected and transformed AI’s best ideas into a sublime final product.

Don’t lazily state, “We tested 5 AI-generated homepage drafts with users and picked the best one to refine through human design.” Explain how you culled the 50+ drafts to identify the golden 5. (You did start with a vast number, didn’t you? After all, ideation is free once you employ AI in the design process.) And tell the stories about what happened in that user test and how it made you realize what elements of which design ideas worked and didn’t work. The bottom line of how you combined the best elements into a single design is undoubtedly the punch line to your story, but without the setup, it’ll fall flat. (You’ll exhibit deadly portfolio homogeneity because all final designs chosen for a portfolio will look good and have reasonable interaction design that won’t stand out unless you tell the story of why specific elements are not just nice, but brilliant.)

Particularly in these early days, be honest about any challenges you faced when using AI tools and how you overcame them. This shows problem-solving skills and resilience. On the topic of tools, also mention which tools you used and why. (Tools don’t make the designer, and somebody who goes on too much about Photoshop skills on a resume is likely not a good UX designer. But as long as AI is new in UX, familiarity with the latest technology and your ability to adapt to the newest AI tools and techniques is a strength worth showing.)

Combatting Portfolio Monotony

I must have seen 2,000 UX portfolios in my recent hiring rounds, about half from designers and half from researchers. At least for entry-level and junior staff, all portfolios seem the same, though admittedly, the designers’ portfolios usually look a little better than the researchers’. UX portfolios are so homogeneous that it’s drudgery for any hiring manager to review them.

At least during 2023, showcasing the strong use of AI in your portfolio is a way to break free from this bland homogeneity. By 2024, less so: it’ll be assumed that all UXers use AI, and all portfolios will have an AI element. But just because everybody will be using AI doesn’t mean they will know how to describe and explain how they did so convincingly. This is a skill to cultivate now.

Advanced tip: Use AI to summarize the job description and available information about the hiring company’s business. Then use these insights to customize your portfolio to highlight work of interest to that employer.

In general, because AI probably doubles design productivity, you can create compelling, differentiated portfolios using AI tools during the initial design and when updating your portfolio. Since AI supports more divergence in the early parts of the design process, you have a better chance of avoiding the homogeneous rut that has plagued UX portfolios until now.

Hiring Managers: Probe the How and Why, Shun the What

Face it, either you are hiring AI-Human symbiants, or you are a meatspace bigot. In the latter case, you will likely join your dinosaur friends and become fossil fuel. The future of UX belongs to people to have mastered human-computer symbiosis and know how to leverage AI to achieve superhuman feats.

If you hire UX staff who don’t use AI, you’re hiring dinosaurs at a time when the AI-fueled meteor has already struck the planet. (Dinosaur by Leonardo.AI.)

Assuming you want to live in the future and not at the bottom of an oil well, you must learn how to identify the best symbiants during the hiring process.

I am an abysmal illustrator who would never be hired to draw Mickey Mouse, even if Disney Character Animation scraped the barrel’s absolute bottom to fuel extreme expansion. Yet, I created an excellent mouse character: Rasmus Mouse. He’s a modern mouse who works as a journalist and publishes several Internet newsletters.

“Rasmus Mouse” image generated by Midjourney.

If I included Rasmus Mouse in my portfolio, there would be no benefit to delving into my drawing skills in an interview. AI drew every pixel. It’s much more important to question the character design and backstory. Why is this mouse a journalist and not, say, a psychologist like his mother mouse? Why is he publishing Internet newsletters instead of being a TV mouse? Why did I pick this look for the character, unlike many others suggested by Midjourney? And why did I use Midjourney and not Leonardo.AI’s “Cute Animal Characters” finetune model? (Answer to this last question: because none of the Leonardo mice looked like strong writers — I would show some of these failed mice in a portfolio and explain why I judged them to be inferior character designs.)

These questions still include some related to technique, but they probe human mastery of the AI tools and the decision-making that we add as a creativity layer on top of the AI. But many questions go beyond technique to address how we drive and guide the AI toward human-defined goals that vary in quality and business value.

Not all human-defined goals are equally profitable, and you want to hire somebody who understands and can discuss the difference. Maybe you push back on the job applicant and point out that tie-in plush animals won’t sell so well if the character is a journalist because very few kids can relate to that profession. You would sell more toys by featuring a farmer mouse as the protagonist. Whether this is factual matters less than how the candidate reacts to the question. Has he or she even considered the extreme profits of tie-in merchandise? After all, company profits are the goal of all design and what funds your UX department.

Hiring Exercises in the Age of AI

I am a firm believer in controlled experiments in the form of assigning predetermined exercises to job candidates. Portfolio reviews serve as an initial screener to separate the incompetent from the acceptable candidates. But you can’t identify UX excellence from a portfolio. Among other reasons, most projects are collaborative, and you never know what contributions came from other design team members.

Once you have the pool of not-terrible candidates, use exercises to identify the top 1% (or however selective you want to be). You learn to tell who produces the best solutions by giving everybody the same exercise. I have exercises where I have reviewed more than a thousand solutions: I know what aspects most good candidates get right and what details only the top talents get right. (Sometimes, a medium-quality candidate will get one of the most challenging points right by sheer luck, which is why you can’t hire purely from scoring the exercise solutions.) Scoring solutions to identical exercises across many applicants is a better way to filter UX job candidates than interview questions.

Unfortunately, AI has put a stop to many of my favorite exercises. I used to like asking people to write up a plan for something in the UX process under a specified set of constraints. (“You design operator interfaces for strip-mining equipment. All your customers are in war zones. How to observe users when it’s too dangerous to travel on-site.”) Now they’ll all feed the problem statement to ChatGPT, meaning everybody will have approximately the same solution. The resulting document will no longer be a test of UX skills, nor will the writing quality serve as a test of those communication skills that are so important for workplace success.

The difference in the quality of deliverables from top and medium candidates will not be as pronounced as before because AI narrows skill gaps.

There was always some uncertainty in exercise scores from (a) the candidates’ varying luck at hitting the bullseye in a short time-limited exercise, and (b) your imprecise scoring, even if guided by the experience from hundreds of previous exercise reviews. This uncertainty is now wider at the same time as the underlying variability in solution quality has narrowed. We need something new besides solution scoring, which has become a less precise tool for identifying brilliance.

Should we abandon exercises when hiring? No, but now the goal of the exercise is no longer to produce an answer. The exercise becomes how the candidate produced the answer and, most importantly, how well each candidate managed the AI tools in this process. You should still request the solution to the problem because different candidates will have different skills in refining the AI-generated draft into a final deliverable. But you should also request a rationale for the solution and any rejected drafts, together with a detailed explanation of the candidate’s process in producing the solution.

Conclusion: The Future Belongs to Symbiants

“The future belongs to Symbiants,” generated by Midjourney.

Empirical data on business use of AI shows:

Substantial productivity gains, on the order of 66% more work produced per hour on the average
Improved work quality
Narrowing skill gaps because AI lifts bad workers more than it lifts the best
Increased employee happiness when using AI, mainly from being more in a state of flow
High creativity of AI when generating ideas
Better performance of AI-Human symbiants, compared with AI alone or unaided humans

The last bullet is essential: AI is not here to replace us, but to augment us, finally bringing Doug Engelbart’s vision to life. This is likely true in all knowledge-worker fields, but it’s undoubtedly true in user experience.

The available measurement data was usually collected a few months ago, which means that the AI in those studies tended to be GPT 3.5 and not GPT 4, which is much better and probably has already led to even bigger improvements.

AI excels at:

Generating lots of varied ideas quickly
Summarizing large amounts of qualitative data
Creating pretty pictures
Crafting well-written copy at any desired length and readability level (though the initial reading levels of AI-generated prose are too high for mainstream readers, so you need to retain editorial review)
Quickly producing initial drafts of complex documents, such as research plans, questionnaires, consent forms, and the like

That’s what AI can currently do. More will be expected from future versions beyond ChatGPT 4. Yes, AI also has weaknesses, so all of the above points require human review before implementation. As I said, high-performing UX work will require Human-AI symbiants: UX professionals proficient at using AI and skilled at reviewing and improving its draft deliverables.

If you’re a company, these symbiants are whom you want to hire.

If you’re a UX professional, you should urgently embrace AI in your UX processes. Become a symbiant, or remain a dinosaur. The choice is yours, but the AI meteor has already struck the planet, so the dinosaurs’ fate isn’t pleasant.

Finally, students should demand AI-First courses. UX courses should not only teach you how to employ AI to do better and more productive UX work but should assume, as a given, that this will be the primary way you’ll work after graduation. UX courses that are not AI-First teach you how to work like a dinosaur.

Current UX courses are educating students to live out this scenario when they graduate. (Dinosaur and meteor strike by Leonardo.AI.)

More on AI UX

This article is part of a more extensive series I’m writing about the user experience of modern AI tools. Suggested reading order:

About the Author

Jakob Nielsen, Ph.D., is a usability pioneer with 40 years experience in UX. He founded the discount usability movement for fast and cheap iterative design, including heuristic evaluation and the 10 usability heuristics. He formulated the eponymous Jakob’s Law of the Internet User Experience. Named “the king of usability” by Internet Magazine, “the guru of Web page usability" by The New York Times, and “the next best thing to a true time machine” by USA Today. Before starting NN/g, Dr. Nielsen was a Sun Microsystems Distinguished Engineer and a Member of Research Staff at Bell Communications Research, the branch of Bell Labs owned by the Regional Bell Operating Companies. He is the author of 8 books, including Designing Web Usability: The Practice of Simplicity, Usability Engineering, and Multimedia and Hypertext: The Internet and Beyond. Dr. Nielsen holds 79 United States patents, mainly on making the Internet easier to use. He received the Lifetime Achievement Award for Human–Computer Interaction Practice from ACM SIGCHI.

Follow Jakob on LinkedIn.
Subscribe to Jakob’s newsletter to get the full text of new articles emailed to you as soon as they are published.

Cal Thompson

Aug 23, 2023

Jakob, Is this your real viewpoint or did AI write most of this? This doesn't sound like you: "After all, company profits are the goal of all design and what funds your UX department."?

I'm surprised to hear you use words like "meatspace bigot"-- that also doesn't sound like you. Interesting!

Expand full comment

Alan Biggs

Jul 12

Based on the article, I'm SURE that AI must have helped in the generation of it, or Jakob's not practicing what he's preaching.

But in so doing, is Jakob losing his voice? Food for thought.

AI: "Proceed with caution."

2 more comments...

Jakob Nielsen on UX

Discussion about this post