In a previous post, we analyzed the demographic Bias in Artificially Generated Facial Pictures that raised a concern that the generated images might not fairly represent all demographic groups.
In this post, we discuss if these artificially generated pictures are good enough for use in personas profiles for real-world systems and applications, which are highly dependent on images for the personas. One of the key aspects of generating personas using a data-driven approach is to be able to represent the persona profile with a matching picture.
Artificial image generation (AIG) provides novel opportunities for a wide range of applications. For example, the Verge has published a tech article about one of the studies that use AI to generate endless fake faces using Generative Adversarial Network (GAN). However, despite the far-reaching interest in AIG among academia and across industries, there is scant research on evaluating the suitability of the generated images for practical use in deployed systems.
This means that the quality and impact of the artificial images on user perceptions are often neglected and lacking user studies of their deployment in real systems. This area of evaluation is an overlooked but critical area of research, as it is the ‘final step’ of deployment that actually determines if the quality of the AIG is good enough for real systems. Therefore, the impact of AIG on user experience (UX) and design applications is a largely unaddressed field of study.
The AIG impact can add an enhancement and improves the user experience for the highest matching persona picture, which can be useful for various applications. Such applications included, for example, designing a chatbot, where the idea is to make the bot more engaging and close to the human. Therefore, using the best persona image that describes the bot is of great value. For example, which one of the pictures below do you feel most comfortable talking with when dealing with a chatbot? The chatbot in PictureA or B? For many folks, it’s much more realistic to speak with a chatbot who looks like the character in Picture B.
There are other domains that can benefit from the best matching picture with the APG, including Marketing and Advertising, Ecommerce sites, Retail brochures, and virtual agents.
Most importantly, the use of AIG is generally free of copyright restrictions and can allow for a wide range of demographic diversity (age, gender, ethnicity). The challenge is how to reach a certain level of agreement that the generated pictures are ‘good enough’ for real-life applications.
In this research, the goal is to evaluate artificially generated pictures across multiple dimensions for deployment in data-driven personas. To achieve this evaluation task, we apply a pre-existing method for generating the persona profiles and then evaluate the results for user perceptions. This evaluation can be considered as a practical design and implementation of AIG.
Overall in this study, we measure the:
- Perceived quality of artificial pictures among crowd workers
- User perceptions when implementing the pictures for data-driven personas (DDPs).
To achieve the research objective we conduct two studies to evaluate the suitability of artificially generated facial pictures for use in a customer-facing system using data-driven personas. STUDY 1 investigates the quality of a sample of 1,000 artificially generated facial pictures. Obtaining 6,812 crowd judgments, we find that 90% of the images are rated medium quality or better. STUDY 2 examines the application of artificially generated facial pictures in data-driven personas using an experimental setting where the high-quality pictures are implemented in persona profiles. Based on 496 participants using 4 persona treatments (2 × 2 research design), findings of Bayesian analysis show that using the artificial pictures in persona profiles did not decrease the scores for Authenticity, Clarity, Empathy, and Willingness to Use of the data-driven personas.
Picture C shows a snippet of the experiment performed in STUDY 2 for the use of an artificial image in the persona profile. The other elements of the persona profiles are identical between the two treatments. For this, we manipulated the base personas by introducing either (a) a real photograph of a person or (b) a demographically matching artificial picture.
The results of this evaluation research shows at least two important aspects:
- Most of the pictures are satisfactory: The crowd evaluation suggests that more than half of the artificial pictures are considered as either perfect or high quality. The ratio of ‘perfect and high-quality’ pictures to the rest is around 1.5, implying that most of the pictures are satisfactory according to the guidelines we provided.
- Artificial pictures are as good as real pictures: The persona perception analysis shows that the use of artificial pictures vs. real pictures in persona profiles does not reduce the authenticity of the persona or people’s willingness to use the persona, two crucial concerns of persona applicability. Therefore, we find the state-of-the-art of AIG satisfactory for a persona and most likely for other systems requiring the substantial use of facial images. So, it is possible to replace the need for manually retrieving pictures from online photo banks with a process of automatically generated pictures.
Read more about this evaluation study in the published research article.
Salminen, J., Jung, S.G., Kamel, A. M., Santos, J. M., Kwak, H., An, J., and Jansen, B. J. (2020) Using Artificially Generated Pictures in Customer-facing Systems: An Evaluation Study with Data-Driven Personas. Behaviour & Information Technology. DOI:10.1080/0144929X.2020.1838610 https://www.tandfonline.com/doi/full/10.1080/0144929X.2020.1838610