Increasing the UX maturity level of clients: A study of best practices in an agile environment - ScienceDirect

Outline

1. Abstract
2. Keywords
3. References

8e14801925ba4db2b2691622e01e3924

Abstract

Context

While multiple studies have attempted to define and measure User Experience (UX) Maturity — i.e., how familiar organizations are with UX concepts or strategies— more practice-based insight is needed to examine how UX practitioners maneuver in their relationships with low UX Maturity organizations and help these clients become more ‘UX Mature’.

Objective

This study evaluates how UX practitioners work with low UX Maturity clients, what obstacles they face, and how they cope with these obstacles. From these insights, a set of best practices are identified for UX practitioners who work with low UX Maturity clients and wish to increase their clients’ UX maturity in an agile environment.

Method

These best practices were collected in the form of case studies, involving a total of 20 case studies based on interviews with 22 UX practitioners. The case studies reflect on past projects that were conducted for clients with a low UX Maturity level. Data was obtained through semi-structured interviews and analyzed using a grounded theory approach combined with elements of a thematic analysis.

Results

The results help to identify frequently experienced obstacles in working with low UX Maturity organizations, as well as six best practices for increasing the UX Maturity of these clients.

Conclusions

The study results demonstrate that UX practitioners indeed fulfill a significant role in overcoming organizational UX boundaries. A Low UX Maturity Best Practice model was developed, which summarizes how UX practitioners can optimize their impact in working with low UX Maturity clients, while simultaneously contributing to a more user-centered focus on the part of their clients.

Keywords

Client-UX relationship

Case study

UX maturity

Agile UX

UX practitioners

UX strategy

Best practices

1. Introduction

To create a successful digital product, offering a wide variety of functionalities is no longer solely sufficient; several quality aspects should also be implemented [19]. One of the most important aspects is User Experience (UX), relating to the experience that a user has with the product. Reported benefits of UX design entail: products that are better suited to the target audience [21], and motivating the future use of products [14], thereby also contributing to higher levels of user satisfaction [2]. Providing a positive UX in products is crucial for agile product development organizations to grow, increase efficiency, and gain market shares [82639], with a growing number of organizations reaching out to UX practitioners to aid them in optimizing their products [3738].

Although clients would generally like to increase the impact of UX practitioners’ digital designs, they are often unable to allocate resources (e.g., time and access to user data) to provide the optimal level of support [287]. This inability is typically caused by misunderstandings or a lack of awareness on the part of the client as to what UX design as a process entails [3943], and how it could best be implemented in their existing agile environment [917]. Therefore, many clients struggle to prioritize integrating the concept of UX design into their agile development processes and corporate culture [1240]. Research suggests that UX practitioners are typically seen as creators of visually aesthetic user interfaces (UIs) by organizations, indicating that the role of such designers in specifying crucial product requirements may be overlooked [16]. As a result, several organizations merely adopt UX design as a strategy to differentiate their products’ visual appeal from those of their competitors [17].

Ideally, clients should operate with a user-centered mindset, i.e., be intrinsically motivated to provide positive user experiences [33]. As mentioned by Sauro, Johnson and Meenan [39], organizations need to understand and commit to improving the UX on all levels, since without users, products and services lack any worth. Briefly considering UX design as an isolated component of the agile product development process is, therefore, not solely sufficient [9]. To achieve lasting impact, a product's UX should be considered early, continuously, and thoroughly throughout the entire design process and supporting organization [93339]. Seeing UX design as part of the agile development process, where UX designers and developers align their working strategies, could not only enable a smoother workflow, but is also a key indicator for optimally meeting user needs [9]. A constitutional shift in corporate identity and processes is often necessary to commit entirely to the user-centered mindset that defines organizations with a positive UX reputation [528].

For UX practitioners who begin working with a client who is still in the process of adopting a user-centered mindset, some challenges and growing pains could arise [28]. Examples of these pains found in related work include the execution of UX activities at too late a stage to inform essential design decisions [29], as well as curtailed access to user needs and preferences [24]. This means that UX practitioners and their clients can take decidedly different stances in the design process [24].

In order to align mutual expectations, UX practitioners have to decide how to approach the client, with these early interactions serving as key determinants in the client's shift to becoming more user-centered [28]. Thus, it is largely up to UX practitioners to evangelize the importance of UX design with their clients as an integrated part of the agile methodology. In this process, a lack of resources allocated to UX design processes [27] and feeling the pressure to ‘sell’ UX design as a method to clients to obtain resources [4] are frequently experienced challenges reported by UX practitioners. While multiple researchers have built theories and models to support UX practitioners in implementing UX design correctly [e.g., 28383631], the current study focuses on best practices for supporting a client's growth in adopting UX design (i.e., fostering the client's UX Maturity).

Chapman and Plewes [5] suggest that identifying a client's familiarity with UX concepts and strategies – hereafter referred to as the client's UX Maturity level – could be key in tailoring UX approaches and realizing good design outcomes. While past studies have focused on finding methods to determine a client's UX Maturity level [38536], or finding correlations between UX Maturity and product success [10], more light has yet to be shed on the definition of best practices that UX practitioners use to adapt, create impact, and help their client grow more ‘UX Mature’. Even though some studies have proposed solutions for isolated challenges, e.g., using tailored lean or agile design processes to cope with scarce resources [11] or knowledge-building sessions with the client to increase UX acceptance [28], there is little research on how practitioners maneuver around low UX Maturity obstacles in practice.

To gain insight in how UX practitioners accommodate to low client UX Maturity levels for yielding good outcomes for the project and client relationship and UX Maturity, this study will build on the UX Maturity model of Chapman and Plewes [5]. This model will be discussed in the next section. Subsequently, new insights into the complex relation of low maturity clients with UX design will be provided by examining a set of case studies of projects carried out at different organizations. The challenges and potential solutions of working at these organizations will be discussed in Section 3 (Results). In Section 4 (Discussion), a set of best practices is presented, which emerged from a synthesis of this study's results. These insights will help test the validity of the UX Maturity model of Chapman and Plewes [5]. It is hoped that the findings of this study will contribute to a wider understanding of the UX Maturity landscape, and that this collection of best practices may support UX practitioners as they work with clients to appreciate the added value of UX design as an integrated part of the agile development process, potentially leading to clients becoming more accepting of – and familiar with – UX design.

2. Method

2.1. Design

For this research, 20 case studies were collected and examined, involving design projects performed by agile UX practitioners for low UX Maturity clients. Case studies were chosen as a method for the data collection because they provide a holistic perspective on real-life events and the process leading to specific decisions and results [45], thus helping to capture best practices and their effects on the overall design process. Another reason for a multiple case study approach lies in the method's appropriateness for exploring theories and models within an authentic, real-life context [3042]. This characteristic is fundamental in studying knowledge fields that have not attracted much attention, as in the case of the current study [1]. Case study guidelines were adopted based on [45] to ensure the reliability of this approach.

Data was collected using semi-structured interviews with UX practitioners working with agile strategies and methods. Semi-structured interviews were chosen because of their appropriateness for capturing new ideas that are brought up during the interview, the possibility to explore topics relevant to the interviewee more deeply and thus also their ability to allow probing follow-up questions [43]. A recent design project at a low UX Maturity client was discussed in these interviews. The case study proposition can be divided into two areas: (1) how the project went (problem statement, process, and project outcome) and (2) how the UX practitioners accommodated to the client's UX Maturity level to still yield optimal outcomes for both the agile product development project and the client relationship.

2.2. UX maturity model

Several models offer a basis for assessing which UX Maturity stage a client is currently at [53139]. What unites these models is the assumption that a client can only implement UX adoption changes that are appropriate for its current UX Maturity level. However, the model by Chapman and Plewes’ (see Fig. 1) gives concrete key indicators that enhance the ability to define an organization's UX Maturity [5]. In the first stage of this UX Maturity model, there is a complete absence of UX design in the organization, whereas in the fifth and final stage, UX design can be seen as a part of the organization's identity and corporate objectives.

Fig. 1. The UX maturity model [5].

Download : Download high-res image (992KB) Download : Download full-size image

Chapman and Plewes further define six key indicators (see Table 2) that help pinpoint an organization's Maturity Stage [5]. These indicators – which were found relevant in a series of case studies by Fraser and Plewes [10] – provide the basis for using this Maturity model in the current study. Therefore, an interview protocol for gathering case study data was developed based on the six key indicators of the UX model by Chapman and Plewes [5]. This list consisted of questions about the client, challenge, design process, key indicators and how they were experienced in the project, client relationship, and project outcome

It is important to note that this UX Maturity model is a generalization; in practice, clients often either present a combination of characteristics from different stages, or are in the process of transitioning from one stage to the next. Chapman and Plewes, therefore, characterize their model as one that provides indicators to UX practitioners for applying to their organization, instead of measuring UX Maturity in absolute terms [5].

2.3. Participants

A total of 22 UX practitioners participated in this study. All practitioners worked in the Netherlands and used agile methods and strategies, such as scrum and lean software development. Three UX practitioners worked on the same project together (and were part of the same case study), while the remaining participants discussed separate projects. All practitioners were recruited through professional networking sites. Table 1 shows their roles, years of experience, size of the client organization, the client's industry, and the nature of the design project. Practitioners all had an educational background in communication, design, or media, with 20 practitioners (90.9%) having obtained a higher degree. The sample of UX practitioners consisted of those working freelance (n = 4), those who acted as consultants for UX and other design firms (n = 7), and those whose employment was fixed (n = 11). At the end of the study, all pseudomynized insights and findings were shared with the UX practitioners.

The client organizations ranged in size from a minimum of 50 employees to clients with more than 20,000 employees. Most clients presented key indicators from two stages of the UX Maturity model. In such cases, the UX practitioners assessed their client's UX Maturity level as being between stages two and three (n = 15) at the start of the project, whereupon these same participants reported witnessing a noticeable increase in UX Maturity on the part of their client over the course of the projects discussed herein. Low UX maturity manifested itself primarily in the presence of few resources, a lack of understanding about the process, and a hesitation to fully commit to UX design from the start of the project.

2.4. Data collection

This study received ethical clearance for data collection in November 2020 by the Ethics Committee of the Tilburg School of Humanities and Digital Sciences. Before taking part, all UX practitioners were asked to fill in an online informed consent form. The participants were also made aware of their rights and were ensured that all data could not be traced back to specific UX practitioners or clients.

Case study data were obtained through semi-structured interviews, which lasted 45 – 60 min and were audio-recorded. Data collection happened online via video conferences due to Coronavirus measures. At the start of the interview, the practitioners were asked to provide information about the client organization, their corporate culture, industry, and the design challenge. Secondly, the design method that the UX practitioners used was discussed. Specific attention was paid to the six key indicators of UX Maturity, how they were perceived, and how UX practitioners ensured that their impact was optimal. The participants were probed to give best practices that they used to accommodate to challenges by naming examples of the literature (e.g., ‘in coping with limited resources, the literature suggests using secondary sources, did this also occur in this process?’).

After the discussion of the key indicators, the project's outcome was examined and specifically if the UX practitioners were satisfied with the outcome. Subsequently, the client's relation with UX design before, during, and after the project was discussed. The UX practitioners were asked if they could pinpoint in which stage the client was before and after the project, and the improvement of which of the six indicators would be most crucial to move to an ascending stage. To probe the participants’ recollections and aid in recalling the different thoughts and emotions that arose during the project, interviewees were asked to share and discuss artifacts from the design process (prototypes, the final product, a presentation given to management, etc.). In the case of the three UX practitioners who collaborated on the same project, thus reflecting one case study, additional questions were asked to explore possible differences in opinion and perception with respect to the project.

2.5. Data analyses

In the analysis of the pseudonymized data, the Grounded Theory (GT) approach by Strauss and Corbin [40] was used to study the events that showcased best practices used by UX practitioners to adapt to their clients and help them grow more ‘UX Mature’. Data gathering and analysis were carried out simultaneously until theoretical saturation was attained. Interviews were partially transcribed and supplemented with notes taken during the interviews. Furthermore, infographics were made per case study (see Fig. 2 for an example) to provide an overview of each UX project, to synthesize results while keeping individual differences in mind.

Fig. 2. Process outlook of one of the case study projects.

Download : Download high-res image (1MB) Download : Download full-size image

The infographics showed the aforementioned case study proposition and provided a quick outlook of the discussed project, highlighting the challenge, process, UX Maturity challenges and used best practices, the outcome, and the client relationship. All infographics were reviewed by the associated UX practitioner as a final check on information presented.

In line with the GT approach by Strauss and Corbin [39], data analysis followed a process that started with coding, in which meaningful excerpts from the interview transcripts were deductively coded into UX Maturity key indicators. During the coding phase, a representative quote – e.g., “I was just trying to constantly prioritize activities to be as effective as possible’ (UL_16) – was associated with each code (in this case, ‘time pressure’). Next, a set of themes was identified describing the observed codes. These included: ‘time pressure’, ‘creating positive PR for UX’, and ‘scoping and prioritizing work activities’. The next step in the GT-approach was to apply axial coding. Here, the key indicators from the UX Maturity model (e.g., ‘initial UX involvement’ and ‘leadership and culture’) were matched to the coded data. Here, ‘time pressure’ was, for example, organized under the axial code ’available resources’.

The final phase of our GT approach used selective coding to find overarching themes in the axial coding data. The code ‘available resources’ was placed under the main theme ‘Early and continuous commitment to UX’. The three themes that emerged from the selective coding phase of the data analysis were: ‘Comprehension of the meaning and value of UX’, ‘Early and continuous commitment to UX’, and ‘Openness to user-centered design processes’. These themes and related data will be discussed in depth in the results section.

Finally, information that was coded during the GT approach as being a potential solution for the challenges described by participating practitioners was also thematically organized using the key indicators of the UX Maturity model. The approach of Xu and Zammit [46] for thematically mapping multiple data sources in a practitioner analysis was adopted. The aforementioned challenges per key indicator from the UX Maturity model [5] were bundled with successful approaches that the participating practitioners said employed to improve their impact. Affinity diagramming was used to cluster all of the themes that emerged per key indicator and find patterns. An example of a solution based comment identified as pertaining to ‘leadership and culture’ was: ‘In this project, it felt like there were three captains on one ship! I felt like I had to give a lot of trainings to define a clear UX strategy’ (UD_4). An example of a solution-based comment, identified as being related to ‘initial UX involvement’ was: ‘At one point I was trying to save time by planning feedback “clean-up UI” sprints together with developers, and thus simultaneously educating the developers and doing my work’ (UD_9). The six key indicators of the UX Maturity model thus served as a base for six emerging best practices. For example, the accompanying best practice for the indicator ‘leadership and culture’ is ‘Lead by example’. These best practices and the related Low UX Maturity Best Practice Model are outlined in detail in the Discussion section.

3. Results

3.1. Comprehension of the meaning and value of UX

The UX practitioners stated that clients often underestimated the return-on-investment of UX design and misinterpreted its ultimate cost-saving benefits. It was apparent from the perspective of multiple UX practitioners that clients’ initial motivation to improve their product's UX design was often more related to staying ahead of the competition than genuinely wanting to provide a more positive experience for users. This conviction resulted in a resistance on the part of clients to fully commit to the user-centered mindset associated with UX Maturity. This was evident among clients favoring visual design over the user interaction, and letting developers make decisions about essential UX elements.

‘UX design is still seen too much as just a way of saving money and time, and this reflects the corporate goals.’ — UD_15

Fifteen UX practitioners (68.1%) experienced challenges with integrating UX design in corporate processes. These practitioners stated that UX goals are often clear for a product, but not at the corporate level. They furthermore indicated that clients’ key performance indicators and mission statements typically did not reflect user-centered goals. This lack of integration also had to do with clients being hesitant about innovation and making substantial changes, as was apparent across multiple case studies. However, some organizations were excited about the innovation, even if they do not fully comprehend it yet. One UX practitioner stated: ‘Design thinking is a 'marketing' term that helped to sell UX to management, even if they only understand half of it's meaning’ (UD_21). Another UX practitioner stated that she wished that she had ‘given the client a quick UX course for a tangible feel of what UX is all about’ (UD_1).

An indicator that is strongly interlinked with the implementation of UX design in corporate goals is the presence of UX leadership and culture. Thirteen UX practitioners (59.1%) felt that the UX leadership and culture were not yet sufficiently in place in the client organization, which was characterized by a lack of clarity regarding project ownership, or excessive micromanagement. One UX practitioner stated that they were required to ’heavily structure the sessions and re-explain the design thinking cycle’ [SD_8]. UX practitioners indicated that clients regularly either had an unclear grasp of the meaning of UX design, or supplied concepts according to their own insights. This misunderstanding resulted in almost all UX practitioners spending an additional investment of their time educating clients and lobbying for UX approaches and resources.

‘My colleagues and I spent 50 – 70% of our time lobbying for UX design.’ — UD_21

The UX practitioners who experienced these implementation challenges stated that defining a clear strategy in combination with educating the client could be helpful. Almost all UX practitioners offered their client workshops, e.g., presenting UX concepts in general, design sprints, persona making, and ideation hackathons. Practitioners also discussed the importance of educating management on the risks of launching a product with bad UX design, e.g., revealing how much money could be wasted if the product is not sufficiently user-tested. Examples from previous projects helped in providing evidence of such phenomena (UD_7). One UX practitioner stated that ‘quantifying the value of good UX design with usability metrics goes a long way towards ‘selling’ the client on a more user-centered approach’ (UL_16).

Furthermore, UX practitioners stressed the importance of including multiple employees from throughout the organization to enlarge the supporting base for UX design. In situations that involved unclear product ownership, some UX practitioners suggested keeping the product owner role within the UX team. However, this solution is contingent upon first gaining the client's trust.

3.2. Early and continuous commitment to UX

Most UX practitioners in this study mentioned that UX design did not have a high priority within their client's organization. This lack of commitment translated to UX practitioners often being: involved too late in the development process of new products, unable to influence the establishment of product requirements, and delayed in the execution of key UX research activities for informing essential design decisions. Consequently, assumptions about users’ needs tend to have already been made, leaving UX practitioners with little maneuver room to create impact. The case studies showed that clients typically demand tangible results quickly, thereby neglecting the need to plan carefully, form expectations, and evaluate findings in an established manner. One UX practitioner stated: ‘the client required visual outputs of the idea, since the client's initial vision of UX had to do more with the 'look and feel' of products than really seeing it as a process (UR_12)’. Another UX practitioner who operated alone was tasked to ‘design the Apple among the agricultural computers’ with very limited means (UD_9). The need to make benefits of UX tangible was also felt strongly by one UX practitioner (UST_18). Operating in this manner risks suboptimal design outcomes and late (and expensive) design changes, as was apparent from several case studies. Six UX practitioners (27.3%) stated that they were involved too late in the design process.

‘We were hired ‘during coding’, and immediately we pointed out some UX flaws. If they had continued, it would have cost them a lot of money.’ — UD_3

One UX practitioner stated that when management saw the added value of UX, the project was given an ‘financial impulse that helped to accelerate a positive outcome of the project’ (UD_4). Insufficiently prioritizing UX design also relates to limited resources being allocated to the UX research and design effort. These resources include time, existing user data, budget, and experienced staff (e.g., experienced UX developers). Eighteen UX practitioners (81.2%) stated that they often experienced time and budget limitations during the projects in focus, resulting in an increased workload. Being appointed too few resources can also result in UX practitioners having to make difficult decisions about which activities they can and cannot fulfill.

‘Because of time pressure, I was constantly trying to estimate the risk of my attention not going somewhere.’ — UL_16

All interviewed UX practitioners used some form of a lean or agile design process. These methods helped them to save time and bring structure to the design process. The UX practitioners who experienced time constraints as a challenge also used parallel design processes and design sprints to save time. For example, one UX researcher reported starting one week in advance and delivering the findings directly to the development team, thereby enabling the UX researcher to investigate a new topic in the following week (UD_1). In this manner, work was ongoing and did not come to a standstill. Another practitioner described planned feedback sprints and direct cooperation with developers near the end of the design process to clean up the UI (UD_9)]. Multiple UX practitioners also stressed the importance of professional development activities and obtaining new knowledge and skills, thus avoiding outsourcing certain tasks. One stated; ‘I just tried to do as much myself as possible, by being present in every meeting and being aware of all relevant information’ (UST_20). To cope with minimal resources, two UX practitioners further described the use of secondary resources, such as competitor designs (UR_12) and personas from past projects (UD_19), to gain a better perspective on the users’ needs.

3.3. Openness to user-centered design processes

According to the interview findings, many clients experience difficulties in adopting a user-centered mindset, in which user involvement and design thinking hold a central place. Ten UX practitioners (45.6%) stated that the involvement of users as stakeholders throughout the design process was insufficient. The main reason for inadequate user involvement was the assumption that clients already know their users, resulting in a reluctance to grant UX practitioners direct access to their customers. One UX practitioner stated that ‘it was necessary to create a support base for UX, before the client admitted to maybe not fully knowing the client’ (UD_13). Another one stated, ‘users should be central to the elements of the design process, not the other way around, which happened with my client’ (UD_17).

In the case of B2B clients, gaining access to users was sometimes even more difficult, as several UX practitioners reported having to obtain the approval of two organizations to gain this access. One UX practitioner stated: ‘I wish I had been able to do one more user test before transitioning to the final product, we could have been finished so much sooner’ (UD_15). Furthermore, in the experience of 10 practitioners, clients demonstrated a strong preference for user testing over user research. All UX practitioners stated that they would prefer the combination of user research and testing to sufficiently map user needs.

‘Continuous access to users would have really helped to create a more suitable design.’ — UD_1

Another UX practitioner stated: ‘I had to educate the client on UX, they asked a lot of perspective throughout the process and the outcomes’ (UR_12). Additionally, 17 UX practitioners (77.3%) stated that design thinking was not being sufficiently implemented. In multiple case studies, design thinking was present within the design team but missing from the corporate perspective. Practitioners stated that while some clients were familiar with the term, the perception is often too solution-focused. One UX practitioner highlighted the underlying reason that causes this insufficient application as: ‘Thinking’ doesn't translate to ‘doing’ it yet’ (UD_1). In another interview, it was reported that the client organization went so far as to completely banish design thinking, as the management came to associate this design approach with instability.

‘Design Thinking was a forbidden word that we were not allowed to use.’ — UD_19

To promote a user-centered mindset, six UX practitioners stressed the importance of being actively present in the client organization. One UX practitioner stated that this helped to ‘make the client see that a UX-centered mindset can also contribute to improved business processes for the entire organization’ (UD_7). They achieved this by being approachable, giving workshops, and including people from different departments in the design process when possible. Training staff to implement design thinking in their respective departments was also described as a helpful practice (UD_13). Some UX practitioners spoke about emphasizing the relative ease of including users in the design process. For example, one UX practitioner held a phone interview with a user in front of management to show what kind of questions they would ask (UD_1). Finally, in the case of inadequate user data, three UX practitioners applied a more context-based approach to derive user insights throughout the development of personas (UR_12; UD_9; UD_19). These UX practitioners placed a greater emphasis on the context in which the product was used.

4. Discussion

This paper focused on the use of the key indicators of the UX Maturity model from Chapman and Plewes [5] to define best practices for UX practitioners working with Low UX Maturity clients in agile environments. These best practices are presented in the following sections so that UX practitioners can use them as a resource for adapting to projects that involve low UX Maturity clients, while contributing to the establishment of a user-centered organization.

The best practices and accompanying recommendations are also summarized in the Low UX Maturity Best Practice model (see Fig. 3). The left side of this model focuses on the UX practitioner's optimal impact in a constrained environment: ‘Adapt to optimize impact in every stage of the design process’ (paragraph 4.1). The right side of this model shows additional best practices that UX practitioners might use to sufficiently guide their client in achieving a more user-centered mindset: ‘Contribute to the establishment of a user-centered organization’ (paragraph 4.2).

Fig. 3. The low UX maturity best practice model.

Download : Download high-res image (571KB) Download : Download full-size image

4.1. Adapt to optimize impact in every stage of the design process

Most UX practitioners in this study experienced a late investment of UX strategy, inadequate resources, or limited user involvement throughout the design process. The resulting challenges are similar to those stated by Chapman and Plewes, for example, concerning limited resources, no standard design process, and involving users in the design process either too late or not at all [5]. This practice is also described by [13]. In the experience of multiple UX practitioners, clients often wanted results quickly and rushed into coding due to the high cost of keeping developers idle. Interview data showed that, in practice, user testing is typically preferred over user research, a finding echoed by [17]. Even though, according to some practitioners, this could potentially lead to faulty personas and ensuing design flaws. Some UX practitioners argued that this preference for user testing stemmed from the client's assumption that they already know the user's needs at the start of the design process.

4.1.1. Use lean/ agile design processes together with parallel processes

UX practitioners in the case studies often used elements of lean or agile design processes, as well as design sprints and parallel processes, for greater efficiency. Several UX practitioners had arrangements with development teams, in which they performed their work activities ahead of time and subsequently provided the results to the development team. The use of these methods is advocated in design processes proposed in [3925] and [15] for their time and resource-saving qualities. Furthermore, as multiple UX practitioners reported, appropriate design is also about making difficult choices. Sometimes a practitioner had to choose between design elements, thus having to try to correctly estimate the risk of UX design not being fully implemented. In such cases, some UX practitioners suggested the scoping of work activities and the use of a decision matrix to assess risks.

4.1.2. Invest in skills and secondary resources

Multiple case studies show that acquiring new knowledge and skills can help UX practitioners improve their response to certain situations. Some UX practitioners took coding courses to improve communication with the development team and promote a shared understanding. The interview data showed that this type of knowledge could also be obtained by working closely with other departments and observing what they do and how they operate. Several UX practitioners in the case studies, for example, joined marketing meetings to see how they viewed the user's needs and how they communicated with users.

Several UX practitioners highlighted the need to understand how a client operates and how work processes flow, especially if a client operates in a very specialized work field. Findings in [35] and [12] also underscore the importance of understanding the organizational culture and each stakeholder's role within this culture. Interestingly, none of the UX practitioners interviewed made use of the assumption-based proto-personas presented by [11], instead preferring research-based personas, or, in cases with no access to users, personas that place a user in a context. In the latter case, practitioners gave special prominence to the context of use. A PACT-analysis (People, Activities, Contexts, and Technologies) aided such practitioners in creating these context-based personas.

Another UX practitioner utilized the competition's resources (e.g., website articles) and approach. Other secondary resources used in the case studies included secondary statistics (from the Dutch central bureau of statistics), existing market research with the target user group, and personas from previous projects. These findings align with [3] and [15]. These researchers also advocate the use of secondary resources to gain a better understanding of users in the case of limited resources.

4.1.3. Lower the client's investment threshold

Another best practice to optimize working with low UX Maturity clients is to lower their investment threshold. Multiple UX practitioners did this by concretizing work activities. The use of feasibility and impact/effort matrices can provide a solution here. These matrices helped several UX practitioners to concretize their work activities and the ensuing design possibilities for the client.

Some UX practitioners stated that since matrices show the result-driven characteristics of UX design, the concretization of options helps to reach a compromise with the client when negotiating strategy. Moreover, UX practitioners used MVPs and low-fidelity prototypes to lobby for more resources. Several practitioners presented a visual example of the product in its most minimal form to persuade the client to choose alternative designs or additional features.

To lower the client's investment threshold, showing how easily user involvement can be accessed can also be a helpful practice. One UX practitioner accomplished this by holding a phone interview with a user in front of management, a best practice also found by [40]. Another UX practitioner showed that feedback from three users could already make a difference and help avoid design mistakes. Diminishing the reluctance to involve users can also help to shatter the illusion that management ‘already knows their users best’. One UX practitioner used the most critical quotes from a user evaluation in a management presentation to show that the product was not experienced in the manner that had been assumed.

4.2. Contribute to the establishment of a user-centered organization

Challenges with UX leadership and culture, the application of design thinking, and the implementation of UX design in corporate processes in the case studies could often be traced back to clients misunderstanding the added value of user-centered design. These findings echo those of [39] and [44]. Due to this misunderstanding, several UX practitioners stated that clients often allocated few resources towards certain aspects of a design process, such as user testing. In the case studies, clients also struggled with UX product ownership during the design process, as well as integrating design thinking in existing processes. Work by [5] and [28] shows that low UX Maturity clients often focus mainly on the visual aspects of UX, a finding that was also evident in the results of this study. The results show that UX practitioners who built trust and guided the client well enabled clients to grow on the UX Maturity scale. For example, one UX practitioner was only hired for three months to improve a digital product's UI. However, after two years, the entire application had been rebuilt through a user-centered design process, and the client had hired two additional UX practitioners.

4.2.1. Define a clear UX strategy

Ideally, most UX practitioners in this study wanted a UX strategist to be added to the project, but in reality, it was mostly up to the practitioners to structure the process, due to limited resources. The significance of a concrete and structured strategy when working with low UX Maturity clients was apparent from multiple case studies. Several UX practitioners described the practice of starting with a kick-off workshop to gauge the client's attitude towards UX concepts and approaches. In this way, an indication of the client's familiarity and openness to UX processes can be established, which can help tailor the UX approach. Two UX practitioners even described using a Design Maturity scale to tailor their UX approach; this scale was very similar to the UX Maturity model used in this study. These UX practitioners used the kick-off workshop to assess this Design Maturity scale and tailor the education and strategy accordingly.

The findings show that demonstrating how UX approaches align with existing corporate processes can be essential. In defining a UX design strategy, mapping out the organization's processes to find bottlenecks could show where and how UX practitioners are of particular value. Multiple UX practitioners used this practice to define growth and improvement opportunities, for example, to create greater efficiency in the production process. A UX process should always be transparent to the client, which some UX practitioners achieved by visualizing the design process steps and project roles in a visual roadmap, or by mapping all design components in a design system. A design system combines all design components, such as style guides and rules, and thus serves as a guide on how an organization conducts its design processes.

4.2.2. Lead by example

To help clients grow more ‘UX Mature’, it is necessary to structurally guide them along the way. The findings of the case studies indicated that sessions should be structured and concepts should be made concrete. Concepts such as design sprints and MVPs are often very new and vague for low UX Maturity clients, as was apparent from both the literature [5] and the case studies. Therefore, it is relatively easy for clients to form their own interpretations of concepts and for the practitioner and the client to then communicate at cross purposes. Creating common ground and a shared understanding of each other's expectations and roles is crucial to create a broad UX support base.

Furthermore, multiple UX practitioners stressed the importance of education to create this common ground. As shown in the study by MacDonald [28], education can help foster awareness of correct ways to implement UX design. A client who is very new to UX could initially be educated in UX concepts and techniques. Simultaneously, the UX practitioner could inform the client of other possibilities along the way, such as the accessibility and consistency of features. Almost all UX practitioners interviewed for this study use workshops to concretize UX concepts and techniques for the client.

Additionally, several UX practitioners highlighted the importance of being present and approachable in the client organization. Being approachable enables employees to experience a low threshold to collaborate, learn more about UX, or ask for a UX practitioner's perspective on their work. To make UX design widely accessible, UX practitioners could be present in relevant meetings and give themes for design sprints or feedback sessions. The suggestion to be approachable is also in line with the recommendation by Ede and Dworman [7] to start with an open and fun UX project, in which anyone from within the organization can participate. One UX practitioner mentioned various low-fidelity prototypes for innovation platforms that employees could design themselves. In these projects, employees can directly apply design thinking and learn the correct techniques. Involving employees in such activities, as well as the UX project itself, could contribute to a shared design thinking mindset, as it offers first-hand experience to employees working with this method.

Another way to create a widely supported user-centered mindset in large organizations is to give training in design thinking to heads of departments, who then will further educate their staff and thus implement design thinking in multiple departments. One UX practitioner who worked for a large organization had experienced this practice as being very helpful. This suggestion of phased design thinking implementation is in line with the results of [6] and [34], who stress the importance of making every stakeholder aware of the usefulness of design thinking.

4.2.3. Quantify the added value of good UX design

A frequently heard challenge from the case studies was that UX practitioners felt the need to ‘sell UX’, a finding that is in line with [4]. Quantifying the value of UX design could help mitigate management distrust. Some UX practitioners explained that qualitatively describing user experiences often felt subjective to clients and enabled them to dismiss the data. However, when metrics are added, the perceived value of UX becomes more concrete and evidence-based. These findings are in line with Lachner et al. [23], who argue that metrics help demonstrate how UX methods are tied to positive user satisfaction levels.

Applying usability metrics or the results of A/B tests are examples of quantifying the added value of UX design. With longer projects, some UX practitioners suggested starting with sharing usability metrics and quantitative data reports with management. In these projects, the initial focus was on providing quantitative evidence, which can then motivate the client to determine the ‘why’ behind the numbers. Therefore, employing metrics can pave the way for user research in the experience of these UX practitioners.

Furthermore, results show that metrics can help UX practitioners to convey authority on the subject. Multiple UX practitioners frequently provided this evidence throughout the design process, e.g., giving short demos or using before/after scenarios. This evidence helped to establish valuable PR and a support base for subsequent UX approaches. Last but not least, UX practitioners stated that ‘evangelism’ for UX design should be shared wider than just with management; it should also be shared with marketing and developer divisions to render user needs more apparent and tangible for these departments.

4.3. Implications and limitations

The study results demonstrated that UX practitioners indeed fulfill a significant role in overcoming organizational UX boundaries. The results appear to support the accuracy of the key indicators of the UX Maturity model of Chapman and Plewes [5]. Subsequently, the data helped to define frequently occurring obstacles and practical solutions that UX practitioners can use to overcome these obstacles. The resulting Low UX Maturity Best Practice model can aid UX practitioners in implementing the corporate and cultural changes necessary to increase agile UX design's overall impact, while simultaneously contributing to the establishment of a user-centered organization. With this study, we demonstrated that while agile practices are often an integral part of project development within organizations, their use within projects can also be essential in increasing a client's UX Maturity. For example, agile UX practices were strategically applied by the participating UX practitioners to save time and increase the client's awareness of the benefits of applying agile methods to increase efficiencies.

By collecting best practices that UX practitioners use in low UX Maturity environments, a new and practical insight into the relationship between UX practitioners and their clients was given. In a previous study regarding good UX implementation in organizations, MacDonald [28] observed that UX practitioners are often responsible for guiding their clients in implementing UX design. This responsibility was also evident in the current study; it appears that the UX practitioners’ guiding role is even larger than may have initially been assumed in the research literature. As one UX practitioner stated, 50 – 70% of their time is spent lobbying for the value of UX to create a support base. Maintaining positive relationships with low UX Maturity clients primarily revolves around UX practitioners building trust and guiding clients in the organizational change necessary to become more user-centered.

This study also gave witness to various areas of friction that UX practitioners experienced during these projects. These problems can be attributed primarily to UX design being a relatively new area of expertise, especially for clients unfamiliar with its methods and concepts. This novelty naturally results in a hesitance on the part of such clients to embrace UX design and its concomitant constitutional shift in the early stages of UX adoption. As multiple UX practitioners indicated, education and trust-building are a logical and integral part of this process and should be considered as such. As one UX practitioner stated: ‘You have to make the client fall in love with the problem, not with the solution’ (UR_12). UX practitioners could employ their UX and design thinking skills to meet the challenge of creating an appropriate UX adoption strategy for these clients. The Low UX Maturity Best Practice model can aid UX practitioners in making this process more efficient.

The results also contribute to a clearer understanding of the UX Maturity domain. The key indicators of the UX Maturity model of Chapman and Plewes [5] were found valid. Obstacles per indicator were recognized, as well as the acknowledgment of Stage five as the ideal scenario for organizations. Moreover, it was relatively easy for UX practitioners to pinpoint the client's UX Maturity level within the model using the key indicators. One could argue that since all clients indicated signs of multiple UX Maturity stages, the model's scaling should be adjusted to make these stages more distinct. However, as several UX practitioners argued, most clients were in the process of going from one stage to another. It is important to emphasize that Chapman and Plewes [5] did not intend for their model to be used as a way of measuring UX Maturity in absolute terms, but rather as key indicators for enabling UX practitioners to assess their clients’ maturity levels.

Most clients showed signs of maturity at Stages 2 and 3. Maturity Stage 3 was the most frequently discussed stage, which is interesting, as Chapman and Plewes [5] state that this is the most crucial stage to overcome to become ‘UX Mature’. The significance of this stage can be attributed to its now-or-never elements; organizations have to decide if they want to fully commit to UX design or if they want to revert to their previous mode of achieving designs. Therefore, appropriate UX designing guidance is especially crucial at this stage.

Another implication of the findings is that the organization's size does not necessarily correlate with a different UX implementation challenge. UX practitioners who worked for clients with either >10,000 employees or <50 employees often faced the same challenges in, for example, establishing a UX culture. Often, these UX practitioners also provided similar best practices, which endorses the Low UX Maturity Best Practice model's appropriateness for organizations of varying sizes.

The generalizability of this study's results is somewhat limited in scope, since only Dutch organizations were sampled. While some of these organizations were multinationals, there could still be a cultural bias. As one interviewee (a UX practitioner working for a multinational firm) stated: ‘There can be many differences in how employees approach management. In the Netherlands, there is a lot less organizational hierarchy than in some other cultures’ (UD_4). An approach of UX practitioners educating management, and leading the way, could then be less appropriate due to management being less receptive to shifts in the established hierarchy.

The UX practitioners in this study were encouraged to pinpoint the UX Maturity of their clients themselves using the UX Maturity model as a guide. This could have biased the results, since all observations were aligned with a single theoretical model. Participating designers may have been more invested in their client's UX Maturity growth, since this reflects on their success within the project. As a result, this assessment may suffer from confirmation bias, while failing to account for project challenges that were not as successfully resolved. However, due to privacy concerns, verifying each UX practitioner's assessment of their clients’ UX Maturity level was not possible in the current study. Instead of asking the UX practitioners to specify their client's UX Maturity level at the end of the interview, it might have been more beneficial to do so at the beginning to encourage a more critical assessment. Shifting the order of the interview questions in this way might have helped participants to better reflect on how the organization performed at the beginning of the project, when the potential challenges emerged.

Furthermore, it should be noted that the reliability of the data may have been impacted due to confidentiality concerns. Several UX practitioners had signed non-disclosure agreements and were bound by contracts that prevented them from providing specific details about the projects that were discussed. While this did not apply to the sharing of experienced obstacles and best practices, it could cause some data to appear as idiosyncratic due to the need for some UX practitioners to withhold key project details. Nonetheless, the results appear to coincide with earlier studies and reflect an overall willingness on the part of the UX practitioners to contribute essential insights.

5. Conclusion and suggestions for future research

The ultimate goal of this research was to facilitate the work of UX practitioners and help organizations grow towards achieving a more user-centered mindset. While multiple studies have attempted to measure UX Maturity, more practice-based insight was needed to examine how UX practitioners maneuver in their relationships with low UX Maturity organizations and help these clients become more ‘UX Mature’. With a growing understanding of the value of UX design, more organizations are expressing an interest in improving the UX design of their products. However, to achieve this goal, organizations need to commit to the accompanying user-centered mindset.

The results give direction to three possible forms of follow-up research that can potentially contribute to a growing number of organizations becoming more user-centered. Firstly, where this study has focused on finding best practices for UX implementation from the UX practitioner's perspective, future research could focus on the organizational side to create a more balanced model. While UX practitioners often feel responsible for guiding their clients in the UX implementation process [28], this does not necessarily mean there are no possible improvements on the organizational level to encourage openness to UX design. Collecting and defining best practices from the client's perspective could, therefore, be interesting to further study good integration of UX design and optimal cooperation with UX practitioners.

A second suggestion for future research stems from the finding that quantifying the added value of UX may be a crucial tactic in organizations becoming more receptive to UX design, as argued in [23]. However, further research is needed to elaborate upon this quantification strategy. Important questions include: ‘Which metrics are most useful at what (maturity) stage?’ and ‘What metrics are most convincing for which stakeholder group?’. Consequently, it can be worthwhile to explore how an internal PR division might be best deployed to successfully implement UX design within an organization.

Finally, it is recommended that future researchers also sample from a broader spectrum of UX projects to ensure that the solutions identified from Low Maturity organizations in fact lead to success in high UX Maturity organizations. A future study could also take a more critical look at the UX Maturity model that was used in this study and seek ways to refine it.

CRediT authorship contribution statement

Elaine, E.G. Buis: Conceptualization, Methodology, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization, Project administration. Simone, S.R. Ashby: Supervision, Writing – review & editing. Kristel, K.P.A. Kouwenberg: Validation, Resources, Writing – review & editing, Data curation, Writing – review & editing.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

We would like to thank all UX practitioners who took part in this study for their valuable insights and enthusiasm. Furthermore, we would like to extend our gratitude to Tim Scholten, who gave the first practical insights into the topic of UX Maturity, and whose feedback proved pivotal in shaping this study's problem statement.

References

[1]F. Almeida, J.A. MonteiroApproaches and principles for UX web experiences: a case study approachInt. J. Inf. Technol. Web Eng. (IJITWE), 12 (2) (2017), pp. 49-65View Record in ScopusGoogle Scholar[2]R. Alves, P. Valente, N.J. NunesThe state of user experience evaluation practiceProceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational, New York, NY, ACM (2014), pp. 93-102CrossRefView Record in ScopusGoogle Scholar[3]J. Blindheim, A. Wulvik, M. SteinertUsing secondary video material for user observation in the needfinding process for new product development and designDS 84: Proceedings of the DESIGN 2016 14th International Design Conference, Dubrovnik, Croatia, The Design Society (2016), pp. 1845-1854View Record in ScopusGoogle Scholar[4]A. Bruun, M.K. Larusdottir, L. Nielsen, P.A. Nielsen, J.S. PerssonThe role of UX professionals in agile development: a case study from industryProceedings of the 10th Nordic Conference on Human-Computer Interaction, New York, NY, ACM (2018), pp. 352-363CrossRefView Record in ScopusGoogle Scholar[5]L. Chapman, S. PlewesA UX maturity model: effective introduction of UX into organizationsInternational Conference of Design, User Experience, and Usability, Cham, Switzerland, Springer Int. Publishing (2014), pp. 12-22CrossRefView Record in ScopusGoogle Scholar[6]D. DunneImplementing design thinking in organizations: an exploratory studyJ. Organ. Des., 7 (1) (2018), pp. 1-16CrossRefView Record in ScopusGoogle Scholar[7]M. Ede, G. DwormanWhy designers might want to redesign company processes to get to better UX design: a case studyProceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, New York, NY, ACM (2016), pp. 840-848CrossRefView Record in ScopusGoogle Scholar[8]F. ErdősEconomical aspects of UX design and development10th IEEE International Conference on Cognitive Infocommunications, Piscataway, NJ, IEEE (2019), pp. 211-214CrossRefView Record in ScopusGoogle Scholar[9]J. Ferreira, H. Sharp, H. RobinsonUser experience design and agile development: managing cooperation through articulation workSoftware: Pract. Exp., 41 (9) (2011), pp. 963-974CrossRefView Record in ScopusGoogle Scholar[10]J. Fraser, S. PlewesApplications of a UX maturity model to influencing HF best practices in technology centric companies–lessons from EdisonProcedia Manuf., 3 (2015), pp. 626-631ArticleDownload PDFView Record in ScopusGoogle Scholar[11]J. GothelfLean UX: Applying Lean Principles to Improve User ExperienceO'Reilly Media, Inc, Beijing, China (2013)Google Scholar[12]C.M. Gray, A.L. Toombs, S. GrossFlow of competence in UX design practiceProceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, New York, NY, ACM (2015), pp. 3285-3294CrossRefView Record in ScopusGoogle Scholar[13]M. GualtieriBest practices in user experience (UX) designForrester Res., 1 (2009), pp. 1-17View Record in ScopusGoogle Scholar[14]M. HassenzahlExperience design: technology for all the right reasonsSynth. Lect. Hum.-centered Inform., 3 (1) (2010), pp. 1-95View Record in ScopusGoogle Scholar[15]L. Hokkanen, K. Väänänen-Vainio-MattilaUX work in startups: current practices and future needsInternational Conference on Agile Software Development, New York, NY, ACM (2015), pp. 81-92CrossRefView Record in ScopusGoogle Scholar[16]K. Holtzblatt, J. Barr, L. HoltzblattDriving user-centered design into IT organizations: is it possible?In CHI'09 Extended Abstracts on Human Factors in Computing Systems, New York, NY, ACM (2009), pp. 2727-2730CrossRefGoogle Scholar[17]M. Isomursu, A. Sirotkin, P. Voltti, M. HalonenUser experience design goes agile in lean transformation-a case study2012 Agile Conference, Cham, Switzerland, Springer Int. Publishing (2012), pp. 1-10CrossRefView Record in ScopusGoogle Scholar[18]P. Jain, S. Djamasbi, J. WyattCreating value with proto-research persona developmentInternational Conference on Human-Computer Interaction, Cham, Switzerland, Springer Int. Publishing (2019), pp. 72-82CrossRefGoogle Scholar[19]P. Kashfi, R. Feldt, A. NilssonIntegrating UX principles and practices into software development organizations: a case study of influencing eventsJ. Syst. Softw., 154 (2019), pp. 37-58ArticleDownload PDFView Record in ScopusGoogle Scholar[20]W.J. Keijzer-Broers, M. de ReuverApplying agile design sprint methods in action design research: prototyping a health and wellbeing platformInternational Conference on Design Science Research in Information System and Technology, Cham, Switzerland, Springer Int. Publishing (2016), pp. 68-80CrossRefView Record in ScopusGoogle Scholar[21]A.E. Krueger, K. Pollmann, N. Fronemann, B. FoucaultGuided user research methods for experience design—a new approach to focus groups and cultural probesMultimodal Technol. Interact., 4 (3) (2020), p. 43CrossRefGoogle Scholar[22]K. Kuusinen, K. Väänänen-Vainio-MattilaHow to make agile UX work more efficient: management and sales perspectivesProceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design, New York, NY, ACM (2012), pp. 139-148CrossRefView Record in ScopusGoogle Scholar[23]F. Lachner, P. Naegelein, R. Kowalski, M. Spann, A. ButzQuantified UX: towards a common organizational understanding of user experienceProceedings of the 9th Nordic Conference on Human-computer Interaction, New York, NY, ACM (2016), pp. 1-10CrossRefView Record in ScopusGoogle Scholar[24]E.L.-C. Law, P. van Schaik, V RotoAttitudes towards user experience (UX) measurementInt. J. Hum.-Comput. Stud., 72 (6) (2014), pp. 526-541ArticleDownload PDFView Record in ScopusGoogle Scholar[25]L.A. Liikkanen, H. Kilpiö, L. Svan, M. HiltunenLean UX: the next generation of user-centered agile development?Proceedings of the 8th Nordic Conference on Human-computer Interaction: Fun, Fast, Foundational, New York, NY, ACM (2014), pp. 1095-1100CrossRefView Record in ScopusGoogle Scholar[26]K.Y. Lin, C.F. Chien, R. KerhUNISON framework of data-driven innovation for extracting user experience of product design of wearable devicesComput. Ind. Eng., 99 (2016), pp. 487-502ArticleDownload PDFView Record in ScopusGoogle Scholar[27]C.M. MacDonaldIt takes a village”: on UX librarianship and building UX capacity in librariesJ. Library Adm., 57 (2) (2017), pp. 194-214CrossRefView Record in ScopusGoogle Scholar[28]C.M. MacDonaldUser experience (UX) capacity-building: a conceptual model and research agendaProceedings of the 2019 on Designing Interactive Systems Conference, New York, NY, ACM (2019), pp. 187-200CrossRefView Record in ScopusGoogle Scholar[29]V.K. de Meerendré, L. Rukonić, S. KiefferOvercoming organizational barriers to the integration of UX methods in software development: a case studyInternational Conference on Human-Computer Interaction, Cham, Switzerland, Springer Int. Publishing (2019), pp. 263-276Google Scholar[30]C. MeyerA case in case study methodologyField Methods, 13 (4) (2001), pp. 329-352View Record in ScopusGoogle Scholar[31]J. NielsenCorporate Usability Maturity: Stages 1-4Nielsen Norman Group (2006)Google Scholar[32]G. NudelmanLean UX communication strategies for success in large organizationsInteractions, 25 (5) (2018), pp. 80-82CrossRefView Record in ScopusGoogle Scholar[33]T. Øvad, L.B. LarsenThe prevalence of UX design in agile development processes in industry2015 Agile Conference, Piscataway, NJ, IEEE (2015), pp. 40-49CrossRefView Record in ScopusGoogle Scholar[34]I. Rauth, L. Carlgren, M. ElmquistMaking it happen: legitimizing design thinking in large organizationsDes. Manag. J., 9 (1) (2014), pp. 47-60CrossRefView Record in ScopusGoogle Scholar[35]J.A. RohnHow to organizationally embed UX in your companyInteractions, 14 (3) (2007), pp. 25-28CrossRefGoogle Scholar[36]L. Rukonić, V.K. de Meerendré, S. KiefferMeasuring UX capability and maturity in organizationsInternational Conference on Human-Computer Interaction, Cham, Switzerland, Springer Int. Publishing (2019), pp. 346-365CrossRefView Record in ScopusGoogle Scholar[37]D. Satterfield, M. FabriUser participatory methods for inclusive design and research in autism: a case study in teaching UX designInternational Conference of Design, User Experience, and Usability, Cham, Switzerland, Springer Int. Publishing (2017), pp. 186-197CrossRefView Record in ScopusGoogle Scholar[38]J. Sauro, K. Johnson, C. MeenanFrom snake-oil to science: measuring UX maturityProceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, New York, NY, ACM (2017), pp. 1084-1091CrossRefView Record in ScopusGoogle Scholar[39]M. Stone, F. Bentley, B. White, M. ShebanekEmbedding user understanding in the corporate culture: UX research and accessibility at YahooProceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, New York, NY, ACM (2016), pp. 823-832CrossRefView Record in ScopusGoogle Scholar[40]A. Strauss, J. CorbinBasics of Qualitative Research TechniquesSage publications, Thousand Oaks, CA (1998)Google Scholar[41]D. SyAdapting usability investigations for agile user-centered designJ. Usabil. Stud., 2 (3) (2007), pp. 112-132View Record in ScopusGoogle Scholar[42]T. VissakRecommendations for using the case study method in international business researchQual. Rep., 15 (2) (2010), pp. 370-388View Record in ScopusGoogle Scholar[43]C. Westby, A. Burda, Z. MehtaAsking the right questions in the right ways: strategies for ethnographic interviewingASHA Leader, 8 (8) (2003), pp. 4-17CrossRefView Record in ScopusGoogle Scholar[44]J. Winter, K. Rönkkö, M. RissanenIdentifying organizational barriers—a case study of usability work when developing software in the automation industryJ. Syst. Softw., 88 (2014), pp. 54-73ArticleDownload PDFView Record in ScopusGoogle Scholar[45]R.K. YinCase Study Research: Design and Methods (3rd ed.), Sage Publications, Inc, Newbury Park, CA (2003)Google Scholar[46]W. Xu, K. ZammitApplying thematic analysis to education: a hybrid approach to interpreting data in practitioner researchInt. J. Qual. Methods, 19 (2020), Article 1609406920918810Google Scholar