Aug 2024 - May 2025
Andrew Twigg
2025 kynamatrix Research Grants
Scholarly Project Funding (formerly called GuSH Research Grants)
Design Researcher and product designer
As design challenges grow more complex, it becomes crucial for designers to clearly articulate the reasoning behind their decisions during collaboration with designers and non-designers alike. Recognizing the fundamental barriers that prevent designers from effectively articulating design evidence in teamwork, my thesis explores the integration of AI into the design process to enhance designers' ability to effectively articulate their design rationale.
This thesis explores how artificial intelligence (AI) can be integrated into the design process to support the real-time capture, contextualization, and communication of design rationale. Through interdisciplinary research spanning design thinking, human-computer interaction, and AI, the project culminates in the design of an intent-based assistant tool embedded within existing design workflows. By reducing the cognitive burden of documentation and making implicit reasoning explicit, the tool aims to empower designers to articulate their thinking more effectively, foster stronger collaboration across teams, and enhance the strategic value of UX design in product development.
The final deliverable comprises a comprehensive 46-page thesis documenting the research and design process, accompanied by a presentation demo delivered to a diverse audience of faculty and students at Carnegie Mellon University.
During the exploratory phase, I deconstructed the main research questions into sub-questions that held potential significance for future design proposals. By employing a combination of research methods, including literature reviews, interview studies, and artifact reviews, I delved into various research topics and synthesized the findings. This exploratory research equipped me with the insights necessary to define the problem space, establish a framework, and develop proposed design solutions. Here, I want to share three pivotal research questions that guided me to the final product design.
Rather than simply describing the final artifact through specifications, Design Rationale emphasizes the underlying reasoning and context that shaped its development.
A design rationale is the explicit listing of decisions made during a design process, and the reasons why those decisions were made.
Its primary goal is to support designers by providing a means of record and communicate the argumentation and reasoning behind the design process.
âď¸ However, design rationale was initially introduced in the scientific domain and has not been specifically defined within the design field.
I took a look into how designer make design decisions. Designers navigate decision-making by moving from a 'Known Space'âa realm of existing solutionsâto a 'Consideration Space', where they narrow down their options, eventually selecting the most suitable and desirable solution. This selection process involves weighing numerous design aspects to support designers reaching the final decision. Through interviews with 12 designers across various disciplines, and literature reviews, there are mainly four fundamental barriers for documentations
To explore this question, I also compiled a comprehensive list of design evidence types, drawing from industry resources such as design websites, books, and blogs. To categorize the types of evidence designers use to support their decision-making and articulate their rationale in team settings, I began by considering the source of each type of evidence, whether it originates from a designerâs own expertise or from external sources like usability testing results. Design evidence can take many forms and serve different functions in collaborative environments, ranging from early ideation to final decision justification. Some types of evidence emphasize internal reasoning and reflection, while others carry more persuasive power due to their objectivity or validation through data.
To better understand this dynamic, I classified design evidence along two axes: soft vs. hard and internal vs. external (see Figure 7&8) . This framework offers a more nuanced way to think about how evidence operates in team collaboration.
By asking 12 designers and educators with diverse backgrounds to place different evidence types on this matrix, I sought to understand how they perceive and prioritize various forms of justification in collaborative settings.
Shared understanding of foundational evidence types
Across participants, there was strong agreement on a core set of design evidence regularly used in their practice. Evidence such as usability testing results, design principles, user feedback, and accessibility guidelines were widely considered hard and external, often used to justify decisions or push back in discussions with constituents. These sources are valued for their credibility and their alignment with both user needs and industry standards.
Shifting perceptions depending on role and context
There was noticeable variation in how evidence was positioned, shaped by a designerâs seniority, organizational context, and team dynamics.
Design stage and audience matter
Participants noted that the value and classification of evidence often shift depending on when and how it is used. For example, analytics data may function as soft evidence during early ideation but becomes a compelling hard evidence post-launch. Similarly, the perceived strength of evidence changes when speaking to peers in critique sessions versus presenting to external constituents or executives.
Internal Hard evidence are often undocumented and rely on tacit knowledge
While external hard evidenceâsuch as usability testing results, analytics, or accessibility standardsâis often well-documented, it is typically stored in tools that are distant from the design file itself (e.g., research repositories, Notion, Jira). In contrast, internal evidence, especially when considered hard (such as a designerâs domain expertise or repeated patterns of design judgment), is rarely captured in any formal or structured way. These insights often live in the minds of designers and are communicated informally during internal collaboration, such as design critiques or whiteboard sessions. As a result, much of the reasoning that drives critical design decisionsâespecially those that feel âobviousâ or intuitive to experienced designersâremains undocumented and inaccessible to others outside the immediate design team. This gap suggests a need for better methods to surface and externalize tacit but influential forms of design knowledge.
In software engineering, DR often supports post-hoc analysis, where decisions are recorded after implementation to aid in maintenance, team handovers, or long-term collaboration. DR systems in this context often emphasize structured representations, completeness, and documentation standards to support analytical and communicative goals. The reflection process is frequently formalized and retrospective, often aligned with retrospectives or code reviews.
In contrast, UX design is highly iterative and fluid, with reflection integrated continuously through activities such as prototyping, usability testing, and critique. UX designers often engage in real-time decision-making and iteration based on user feedback, requiring DR tools to support lightweight, non-intrusive, and flexible documentation methods. Here, reflection is not only retrospective but occurs during the act of design, embedded within collaborative sessions, design critiques, or field research.
For UX design in particular, where reflection is continuous and embedded within iteration, DR systems must put a greater emphasis on supporting design processes.
Leveraging the aforementioned discoveries in exploratory research, I redefined my design approach with the research question:
How can we design a tool that captures the evidence behind design decisions bridging the gap between Design Rationale and design articulation, thus enabling designers to justify their decisions and enhance team collaboration.
Defining design goals for supporting design rationale
Extend design understanding: Help designers externalize tacit knowledge, spot overlooked issues, manage collective insights, and focus on critical decisions.
Ease decision capture: Seamlessly document design choices within the workflow, including those often missed due to cognitive load, timing, or context.
Improve retrieval and reuse: Make rationale searchable and context-aware, supporting both structured and fl exible inputs to ensure easy access and future relevance.
Support reflection and communication: Enable real-time, nonintrusive documentation that facilitates both ongoing refl ection and clear team communication.
To put these goals into action, I moved forward with redesigning Figmaâs comment toolâa medium through which Design Rationale (DR) can be captured, represented, and leveraged in collaborative settings. The aim is to make not just the design artifact visible, but also the thinking and decisions behind itâenabling design to be read and understood as a process, not merely seen as a final product (see Figure 9). Below is a summary of the where, what, and when of the proposed design intervention:
Where: Figma, where designers iterate and collaborate most actively.
What: UI Design, Basic Prototyping, and Design Systemsâthe main spaces where decisions are made and rationale can be embedded.
When: During key moments of the design workflow such as early ideation, mid-iteration feedback, and fi nal decision checkpoints when rationale is formed, discussed, and often lost if not captured.
I reviewed Figmaâs commenting system and cross-product analysis of Design Rationale support. I analyzed the Figma Comment Tool in terms of its usability and interaction techniques, and conducted a comparative analysis with similar features in tools like Adobe Acrobat and Google Docs to identify patterns, gaps, and opportunities for improvement (see Figure 10).
To evaluate its usability, I conducted rapid usability testing with 6 designers, focusing on how they interact with comments during their design workflow. Hereâs what I observed and heard:
đ "If there are multiple issues, they tend to start new comment threads."
đ "During the design iteration stage, people frequently use commenting tools to manage feedback and revisions."
â
đ "The current comment log isnât helpful for retrieving past comments."
âđ " The comment log becomes unmanageable when comments scale."
Based on my research and usability findings, I generated ideas to redesign the Figma Comment Tool, expanding its functionality to better support the capture and management of DR. Below are a few key features I propose, along with the design considerations that informed them.
Capturing ideas quickly and spontaneously, the tool enables designers to select a design frame and write quick notes on it, without caring about grammar or full sentences; it captures instant thoughts at lower stakes.
I have considered how to make the tool seamlessly integrated into the design context and process: to not make additional work for the designers to capture the rationale, and make the DR relevant to the design. I decided to utilize the current comment tool on Figma as a foundation for designing new features and functionalities. I designed a Comment tool that allows the users to hover on a frame, either nested frame or a main frame, select the frame and start typing. This is similar to the current comment tool which allows designers to type single or multiple comments, and make a thread by themselves, or with collaborators. The designed tool should support adding mentions, adding emoji, and inserting images in editing mode. Functionalities should include resolve, delete, mark as read/unread, delete thread, and copy link.
Makes narrative that is easier for others to understand.Ensuring that all parts of the design reasoning are connected and presented logically.
However, whether this transformation is necessary should ultimately be left to the discretion of the designer. According to interviewees, long and complex projects can accumulate over 500 comments, making it impractical to convert every comment into a full narrative. Instead, comments may qualify for transformation based on criteria such as:
(1) The thread includes multiple collaborators or checkpoints, where details may have been lost or are diffi cult to track.
(2) The content is relevant to key design decisions that should be documented for future reference or team alignment.
The advent of generative AI, particularly Large Language Models (LLMs), has significantly enhanced the ability to manage defined content curation tasks such as text summarization, thereby supporting the technical aspects of design.
AI-generated design aids can assist designers in identifying gaps in their reasoning and uncovering areas that require further exploration. These tools help designers consider alternative solutions, enabling them to select options that are not just feasible but satisfactory. This functionality aligns with the divergent stage of the design thinking process, where the goal is to explore âwhat was not consideredâ and surface issues that may have otherwise been overlooked.
It aids in identifying gaps in your reasoning or areas that need more exploration.
An effective AI design aid should support designers in two key ways:
Identifying gaps in the rationale that can strengthen design decisions.
Offering alternative design options and sources of inspiration.
To guide the design of AI-powered aids, I have organized them into four categories based on their utility in supporting the two goals outlined above (see Figure 13):
Best practices: Provide established standards and benchmarks that help validate decisions and highlight missing justifications. Also serve as inspirational references, especially when drawn from diverse contexts.
Design considerations: Prompt deeper thinking about constraints, context, and goals, helping designers identify overlooked factors.
Tradeoffs: Present alternative paths and clarify the consequences of each, helping designers explore different directions.
View design examples: Offer real-world references and inspiration for alternative solutions or approaches to similar problems.
Together, these categories form a foundational framework for AI design aids that support both reflective reasoning and creative exploration in the design process. The sequence in which these categories appear, visually distinguished through color coding from top to bottom, should be dynamically determined based on the content and intent of the comment.
For instance, if a comment reflects a designerâs self-recording of a decision they made, the system may prioritize and display the "Best Practices" category first, recognizing the designerâs intent to validate or align their choices with established standards. In contrast, if the comment captures a collaborative exchange where two designers are unsure about a design request or are exploring alternative directions, the system would prioritize "Tradeoffs" or "View Design Examples" as the top categories, surfacing relevant aids to support divergent thinking and comparison.
This adaptive sequencing ensures that the AI suggestions are contextually relevant and cognitively aligned with the designerâs workflow, enabling smoother integration of rationale capture and decision support into everyday design activities. (see Figure 14)
DR outputs should support retrieval and usage across various collaborative design scenarios. To facilitate this, three types of outputs are proposed as deliverables of the DR system.
On-Frame: This type of representation places DR directly within or near the design frame. It is especially useful in collaborative settings such as stand-up meetings or design critiques, during which team members can click on DR elements closely linked to the design component in question. This proximity enhances contextual understanding and supports real-time discussion.
Next-to-frame: Here, DR is positioned adjacent to, but not embedded within, the main design frame. This approach minimizes visual interference with the design content while keeping rationale easily accessible. It is suitable for asynchronous reviews or when designers need to reference rationale without cluttering the workspace. Additional affordances may include toggling visibility or linking rationale to grouped elements for more structured navigation.
In a Design Notehub: The Design NoteHub serves as a centralized repository for all collected DR and comments across multiple frames. Evolving from the current comment log systems, this feature enables designers to organize, manage, and search rationale more efficiently. It also supports the generation of formal documentation for design handoffs, project reports, or constituent presentations.
To further refine the design and evaluate the usefulness and intuitiveness of the proposed system, I conducted a 30-minute usability testing session following a structured protocol. During this session, participants were guided through a simulated design scenario involving the creation of a color filter for a fashion website. And here are some results I collected.
Parts 2 and 3 of the prototype were particularly impactful. Five out of six junior designers, with background in UIUX, product design and visual design found Part 3, Post-editing Paragraph with AI-Generated Design Aid, especially effective in helping them identify gaps in DR and use AI as a source of inspiration (see Figure 17). In this section, participants are referred to using a number code: âP1â represents âParticipant 1,â âP2â represents âParticipant 2,â and so on.
PART 3 | Most effective: Rated as the most helpful feature by 5 out of 6 participants. Designers appreciated its ability to prompt reflection and support design tradeoff discussions.
â
đ "This tool can be helpful. It gives me inspiration and makes me consider edge cases." â P3
đ "AI works well as inspiration. I can choose which tradeoffs to highlight and discuss with the team." â P4
âđ "It depends on context. Part 3 broadens possibilities, which may not be helpful once weâve aligned on a solution with the team." â P1
âđ "The accordion design is confusing when scrolling long text threads. Could this be redesigned as tabs?" â P5
PART 2 | Most intuitive: Participants appreciated the way AI helped summarize complex, long comment threads, reducing cognitive load and simplifying communication.
â
đ "It simplifies the conversation, and this process can be automated. I like that." â P1
â
âđ "Too many words. The AI-generated content can lose important context and become frustrating." â P1
đ "I donât trust AI writing. I need a way to edit it, and I want to see the original contextâI worry it will be lost." â P3
âPART 1&4 | The entry point felt intuitive to most participants due to its similarity to existing comment tools in Figma. This made the learning curve minimal.
â
The representation of DR on the frame was found to be potentially useful across various design stages and team settings.
â
đ "This can be useful for stand-up meetings. It might help people anticipate and answer questions." â P4
đ "After handoff, this could help examine the design. The rationale hub might also support formal documentation." â P6
đ "The hub might be useful for final documentation, though it's not something I currently need." â P3
â
âđ "Too many words. The AI-generated content can lose important context and become frustrating." â P1
đ "I donât trust AI writing. I need a way to edit it, and I want to see the original contextâI worry it will be lost." â P3
The testing results show that all participants found AI summarization useful for reducing cognitive load and appreciated the post-editing feature for prompting refl ection and surfacing tradeoffs. However, concerns emerged regarding trust in AI-generated text, potential loss of context, and interface clarity. Additionally, insights from a design leader highlighted the importance of aligning design rationale with product goals and supporting communication across cross-functional teams. Overall, the evaluation demonstrated the promise of integrating DR tools into existing design workfl ows, while also surfacing key areas for refi nement.
A limitation of this study is that the use caseâa fashion website fi lter designâfocused on a relatively standardized UI component, which may not refl ect the full complexity and variability of UX design workfl ows. Additionally, the screen-based walkthrough format may not accurately represent how designers would interact with the tool in real-world, time-sensitive settings. As testing with AI-integrated products remains a challenge, I plan to implement a subset of the proposed functionalities in a working prototype to enable more realistic evaluation in future iterations.
Balancing AI-generated content with original context: Designers expressed concerns about losing visibility into the original comment threads. Future iterations will explore ways to toggle between or layer the original and summarized content.
Audience and credibility: Questions were raised about how AI-generated rationale might be perceived by collaborators and whether it could affect a designerâs credibility. This opens up further inquiry into how AI-generated content should be attributed or presented in collaborative settings.
Aligning rationale with stakeholder expectations: It will be critical to explore how AI-supported rationale can help bridge gaps between designers, PMs, and engineers, especially in contexts involving negotiation or justifi cation of tradeoffs.
Scalability in complex threads: Real-world threads often include 20â30 comments from multiple collaborators. Future versions should support organizing and summarizing dense threads while preserving traceability.