AI in the UX Design Process: What the Research Actually Says

AI in the UX Design Process: What the Research Actually Says

A vibe summarizing Stige, Mikalef, Zamani, and Zhu, "Artificial Intelligence (AI) and User Experience (UX) design: A systematic literature review and future research agenda," Information Technology & People. What this paper is, and why it e

A vibe summarizing Stige, Mikalef, Zamani, and Zhu, "Artificial Intelligence (AI) and User Experience (UX) design: A systematic literature review and future research agenda," Information Technology & People. What this paper is, and why it exists Designers have always carried a double burden. They have to figure out what to build, which the authors call problem setting, and then figure out how to build it well, which they call problem solving. Both halves eat time, expertise, and money, and both depend on data about how people will actually use a thing, data that is frequently missing or thin. Into that gap walks artificial intelligence, with the promise of faster cycles, lower cost, and more creative range. The paper sets out to map that intersection in a disciplined way. Rather than guessing at where AI is headed in design or cherry-picking impressive demos, the authors run a systematic literature review. They gather the academic record, filter it down to the studies that genuinely speak to AI inside the design process, and then organize what they find against a familiar scaffold, the user-centred design process. The result is less a prediction and more a map of the territory as it stood when the review was conducted, plus a set of signposts pointing at where the unexplored ground lies. It matters because most conversations about AI and design are loud and anecdotal. This one is structured. The authors anchor everything to a process model, which forces a useful question for every claim. Not just "can AI do design," but "where in the act of designing does AI actually show up, and what does it do once it gets there. " What the researchers were trying to do The work is built around two questions. The first is descriptive. How is AI currently used to support or enable the UX design process. The second is generative. What gaps remain, and what are the research avenues most worth chasing to make human and AI collaboration in design better. The framing choice underneath both questions is deliberate. The authors take the user-centred design process as their organizing lens, drawn from the ISO human-centred design tradition. That process runs through five recognizable stages. Understand and specify the context of use. Specify the user requirements. Produce design solutions that meet those requirements. Evaluate the designs against requirements. Then development, where the solution gets built. It is iterative by nature, so any stage can loop back when the result misses the mark. By mapping the literature onto those five stages, the authors can do something a flat survey cannot. They can show not only what AI does in design but exactly which part of a designer's job it touches, and just as importantly, which parts it has barely touched at all. That second insight turns out to be one of the most interesting things in the paper. How they did it The method follows a recognized systematic review protocol. The authors built a keyword set spanning two buckets, the AI side (artificial intelligence, machine learning, deep learning, artificial design intelligence) and the design side (user experience, UI, usability, user-centred design, prototyping, human-computer interaction, co-creation, and so on). They searched Google Scholar plus Scopus, ACM, IEEE Xplore, AIS, and several publisher engines, and ran the search across late October to late November 2022. That first sweep returned 1,647 papers. After removing duplicates, screening on titles and abstracts, and applying quality and relevance filters, the pool narrowed sharply. A forward and backward citation search then recovered eight more papers that the keyword net had missed, often because those studies used specialist terms like feedforward neural networks rather than the broad vocabulary the search relied on. The final corpus was 46 papers. Two co-authors coded everything independently into a concept matrix, then reconciled with five research assistants to reach consensus. It is a careful, conventional, and transparent process, and the authors are honest that the design itself, keyword-driven and retrospective, has built-in blind spots. A descriptive note worth keeping in mind. The literature clusters in 2020 and 2021, and skews heavily toward conference papers, especially from venues like CHI and ACM SIGCHI. This is a young, fast-moving field whose written record lives in proceedings more than in journals. The central finding: AI is bunched at the back of the process When the 46 studies are sorted into the five stages, the distribution is lopsided, and the lopsidedness is the story. The bulk of the work sits at the end of the pipeline. Roughly a third of the studies focus on producing the design solution, stage three. Just under a third focus on development, stage five, turning a finished design into actual code. A smaller set addresses design evaluation and understanding the context of use. And then the gap. The authors found no studies at all that automate the specification of user requirements, stage two. That absence is not random, and the authors offer a credible reason. Requirements specification leans on inherently human, conversational activities. Role-playing, focus groups, in-depth interviews, the slow work of teasing out what people actually need versus what they say they want. These resist clean automation because they are about interpretation and tacit understanding, not pattern matching over structured data. The most AI can do here, per the studies the authors cite, is act as a sounding board, a system a designer talks requirements through, which then proposes similar projects or inferred information and refines its understanding from feedback. Useful, but a long way from automation. So the high-level picture is this. AI today is strongest where the work is concrete, repetitive, and data-rich. It is weakest, almost absent, where the work is ambiguous, social, and interpretive. The closer a task sits to the messy front end of figuring out what to make, the less AI has penetrated. Stage by stage, what AI is actually doing Understanding the context of use Here the work concentrates on two jobs. The first is generating user personas automatically. Building personas by hand is slow and demands skills many designers lack, so researchers have trained machine learning on analytics and think-aloud data to produce personas fast. The honest result is mixed. Auto-generated personas are quicker and more current, and they can be behaviourally accurate, but they inherit the same underlying weaknesses as hand-built ones around credibility and how the information lands. Speed went up. The deeper problem did not get solved. The second job is moving between fidelity levels of prototypes. Jumping from a rough sketch to a high-fidelity, code-backed prototype is painful, because low-fidelity sketches leave room for interpretation that high-fidelity versions tend to flatten, which forces expensive iteration. One studied approach uses deep neural networks to bridge those levels automatically, smoothing what is normally a stop-start grind. Specifying requirements Nothing. As covered above, this is the empty quadrant, and the authors flag it as a meaningful gap rather than an oversight in their search. Designing the solution This is the busiest and most philosophically interesting stage. Two patterns dominate. Some tools help a designer move from low to high fidelity. Others optimize an existing design, automatically adjusting layouts based on feedback, task performance, completion time, and error rates. What is striking is how consistently the research lands on the same verdict about control. Designers do not want the machine to take the wheel. In study after study, fully automatic adaptation gets pushback because designers feel they are losing authorship. They prefer being offered a range of alternatives to choose from rather than handed a single answer. One layout-suggestion tool found that when the final outputs were judged, a clear majority preferred the AI-adapted design over the baseline, yet users still disliked the adaptive feature in use because of the loss-of-control feeling. The numbers said the AI helped. The experience said the AI overstepped. The recurring recommendation across this stage is partnership, not substitution. AI as a tool that augments creativity and surfaces options the designer would not have reached alone, while the human keeps judgment, refinement, and the aesthetic call. One creative AI system based on genetic algorithms is described as helping designers make something neither party could make alone, while plainly lacking human skills like empathy and emotion. Supervised learning can flag design patterns that test better for usability. Deep learning can sharpen iteration. None of it is positioned as a replacement for the designer. Evaluating the design Evaluation is thinly studied with AI, and for a revealing reason. Good design evaluation often means putting a real person in front of a real prototype and watching, which produces rich qualitative insight that quantitative models struggle to replace. A few studies model how a user might react to a design, for instance predicting how tappable an interface element looks to a human, and some argue this is good enough to cut the cost and time of user studies. But the more careful voices in the corpus push back hard. Usability testing as a method is treated as irreplaceable for surfacing the barriers and frustrations that only show up when a human actually struggles with something. The settled position is a hybrid. Let AI gather the quantitative signal cheaply and at scale, keep humans for the qualitative depth, and treat the two as complementary rather than competing. Developing the solution This is the most mature corner of the whole field, and the most technically convergent. The core task is turning a design into working code, closing the chronic gap between what the designer drew and what the developer built. Nearly every studied tool uses machine learning, and most lean on computer vision to detect interface components in an image and then generate the corresponding code. The tools have names and respectable accuracy figures. Some reverse-engineer a mobile UI from screenshots. Some convert hand-drawn mockups to HTML. Some classify components with accuracy in the high eighties to low nineties percent. But the weaknesses cluster tightly and the authors lay them out plainly. The tools tend to recognize only a small set of components. They merge elements that sit too close together, and merge text in similar fonts. They misclassify small elements, progress bars, and toggles when multiple styles are in play. They miss elements that are partly hidden. And critically, many of these studies tested accuracy of the algorithm rather than testing with real designers and users, which risks flattering results. The consistent conclusion is that automatic code generation belongs as a time-saving supplement, a fast first pass that the human then improves and finalizes, not as an end-to-end replacement for building. The three patterns that cut across everything Stepping back from the stage-by-stage view, the authors extract three themes that run through the whole corpus. First, it is all weak AI, and it is all machine learning. Every single study uses some form of ML rather than anything resembling general intelligence. The likely reason is pragmatic. Weak AI is easier to build, cheaper to run, and more accessible to designers who are not AI specialists, so it is a reasonable on-ramp. But ML brings the black box problem with it. Even when an AI-driven design decision is accurate, neither the designer nor the end user can readily see the causal reasoning behind it, and that opacity is a genuine departure from how design normally works, through visible collaboration between designer and user. Second, the field cannot agree on the designer's role. One camp imagines AI eventually doing the designer's job. The other argues for a symbiotic human-AI partnership. The authors lean toward the partnership view, and their reasoning ties back to the empty requirements quadrant. The early, human-heavy stages of design resist automation precisely because they depend on tacit knowledge and direct contact with users, the things ML handles worst. The realistic future is collaboration where AI augments the human, which in turn demands that designers pick up enough AI and ML literacy to know what these tools can and cannot do. Third, automation should target the repetitive, not the creative. The clean dividing line the authors borrow is between getting the right thing, the exploratory work of deciding what to build, and getting the thing right, the execution work of building it well. AI is well suited to the second and poorly suited to the first. The repetitive conversions and adjustments are exactly what should be handed off, freeing designers to spend their reclaimed time on creativity, sense-making, and their users. The whole process should not be treated as one big automatable block. The forward look at generative AI The review was conducted in late 2022, right as generative AI was breaking into the mainstream, and the authors are candid that the academic record had not caught up. They found very few scientific publications on generative AI inside the UX design process, so this section reads more as informed anticipation than evidence. Their bet is democratization. Generative tools let people without deep specialization produce outcomes that used to require experts, which the authors compare to the early web, when only a small priesthood could build websites and then suddenly many more people could. They expect generative AI to help collect and analyze data, generate insight, translate requirements into artifacts, play the role of a tester, and even simulate different user personas to catch issues human designers might miss. The framing they reach for is conjoined agency, designers and generative AI working in tandem, with many design tasks shifting toward the machine and the door opening to less-skilled people who want to make digital things. Notably, this points generative AI straight at the front-end stages that earlier AI never reached, the understanding and requirements work, which would partially fill the gap the rest of the paper identifies. The research agenda, and where the real questions live The paper closes its analytical work by consolidating everything into three research themes, each with open questions meant to seed future work. Theme one is automating the design process. The honest admission here is that we do not yet know which parts of design should be automated, how the designer's job changes when parts of it are, where AI genuinely beats humans and where it does not, or how data should flow through an AI-assisted process. The guiding principle the authors borrow is sharp. Do not automate something just because you can. Automate the high-overhead, manual, tedious work, and leave the creative work alone. Theme two is co-creating with AI, and it is really about people and skills. What competencies will UX and UI designers and developers actually need. How do organizations have to restructure their workflows and value chains to fit humans and AI together. And, more hopefully, how can AI widen participation in design, bringing in children, people with disabilities, and other users who are usually left out because doing inclusive research is slow and expensive. AI could make design more accessible and more inclusive, but the unintended consequences of that need watching too. Theme three is the negative effects, the part most surveys skip. The worries are concrete. AI use may add stress and erode designers' sense of responsibility for the final product. Data-driven tools may pressure clients to hand over quantitative or sensitive data they would rather not. New AI tooling may raise the total cost of design work and quietly hurt project outcomes. And the deepest concern, raised through the black box thread, is that designers reduced to operators of automatic tools may start overriding their own judgment in favor of whatever the AI proposes, without understanding why. The proposed antidote is explainability, building AI tools that can show their reasoning so designers stay reflective rather than deferential. What it all adds up to A few honest takeaways come out of this. The field is real but immature and uneven. Forty-six papers is a thin corpus, weighted toward conference proceedings, clustered in just two years, and lopsided toward the back end of the design process. AI in design is genuinely happening, but the written knowledge about it is early. The strongest signal in the whole review is the empty requirements quadrant paired with the loud, repeated demand for designer control. Together they say something clear. The human parts of design, deciding what to make and judging whether it is any good, are not just unautomated today, they are the parts the research community keeps insisting should stay human. The machine is welcome to handle conversion, optimization, and the tedious middle. It is not welcome to take authorship. The consistent verdict, across persona generation, layout optimization, evaluation, and code generation alike, is augmentation over replacement, and supplement over substitute. Almost every tool that tried to take over got pushed back toward being an assistant. That is not a limitation the authors lament so much as a working principle they keep finding confirmed. How it lands for the future, and for anyone building now For practitioners, the paper reads as quiet permission and a warning at once. Permission, because it documents that real parts of UX work can be automated under the right conditions, and points at the technologies that do it. Warning, because it keeps showing that the tools fail in predictable ways, that accuracy benchmarks can flatter solutions that have never met a real user, and that handing over creative judgment tends to backfire. The most durable idea here is the get-the-right-thing versus get-the-thing-right split. It is a clean test for any team deciding where to point AI. If a task is about execution, conversion, repetition, or measurement, that is fair game and the cost savings are real. If a task is about exploration, interpretation, empathy, or judgment, automating it tends to subtract more than it adds. The skill that gains value in this world is not pixel-pushing or hand-coding interfaces, both of which AI erodes. It is the framing work, knowing what problem is worth solving, reading users, and exercising taste over a sea of machine-generated options. The generative AI section, written just as the wave was cresting, has aged into the live question. The authors anticipated democratization and conjoined agency, and they pointed generative tools at exactly the front-end stages that the rest of the field had ignored. If that bet holds, the empty requirements quadrant is the next frontier, the place where the action moves. The thing to watch is whether explainability keeps pace, because the entire partnership model rests on designers being able to understand and override the machine rather than quietly defer to it. The paper does not claim to settle anything. Its own closing limitation is the most important one. Systematic reviews are retrospective by nature, and in a field moving this fast, the seminal work may well be published after the cutoff and may overturn the picture entirely. Read that way, this is best treated not as a verdict on AI in design but as a careful map of where things stood at one moment, with the most valuable contribution being the clear marking of the unexplored ground.