Intro

AI safety is a big concern for a lot of people, and for good reasons. AI systems are becoming increasingly powerful, and if they’re not designed safely, they could pose a serious threat to humans and society.

As UX designers, our role goes beyond just crafting intuitive interfaces; we are equipped and trained to ensure user trust and safety to the maximum level possible, in collaboration with data scientists, developers, architects, product managers and other contributors. But regardless of your discipline, we all play a big role in ensuring that technologies like generative AI improve things for people without causing harmful outcomes. It’s also important to ensure that there is no personally identifiable information (PII) in the training data sets, and that we do our due diligence when it comes to GDPR compliance.

Best Practices for Safe AI

We recommend incorporating the following best practices into the design process:

• Make systems transparent and accountable.
  Generative AI systems should be able to track and record their output, so that people can see what the system has created and how it was created.
• Design for reliability.
  The system should gracefully handle unexpected or ambiguous user inputs and provide consistent, reliable responses, minimizing user confusion or potential harm. (This is similar to designing resilient user flows that can handle varied user behaviors without breaking or causing frustration.) Building robust generative AI systems is crucial for safety, but it also fosters trust when users see that the AI can handle a variety of scenarios reliably.
• Design systems that are robust and explainable.
  We must make sure that the AI system can handle unexpected inputs and situations, and that humans can understand how the AI works and why it makes the decisions it makes. Generative AI systems should be able to detect and filter out harmful content, and they should be able to explain their decisions in a way that is clear and understandable to a wide range of people.
• Consider the potential risks of generative AI systems.
  What could go wrong if AI is used to create harmful content, such as amplifying bias or hallucinations? How could it be misused? What are the ethical implications? One approach taken by teams externally at companies like Google is the concept of “red teaming” AI applications before they are launched. This critical capability requires teams to leverage “attacker tactics” to mitigate risk and harm. Some of the exercises they use include prompt attacks, training data extraction, backdooring the model, adversarial examples, data poisoning, and exfiltration. These types of activities can really help to mitigate risk, reduce harm, and improve outcomes.

Technical Aspects of AI Safety

Explainability

Generative AI decisions and actions must be transparent and understandable to the user. When the system suggests something or takes an action, users should have access to an easily digestible rationale, enhancing trust and comprehension (similar to tooltips, help sections, or user-friendly error messages that explain why something happened in a user interface).

Example:

Consider an AI for financial advice built into a banking application. The AI recommends that the user apply for a particular loan. Instead of just showing the recommendation, it explains the rationale:

“Looking at your consistent income growth, your commendable saving habits, and your steadily improving credit score over the past six months, we believe this loan would be a great fit for you.”

This isn’t an arbitrary suggestion; there’s a clear line of reasoning behind it. By explaining its decisions, the AI offers transparency, helping end-users understand why a particular piece of advice was given. Such explanations can increase trust because users can see that recommendations aren’t random but are based on their personal financial patterns and data.

Constraints

Clearly define what the system can and cannot do. For instance, if a user interacts with an AI health assistant, it should be clear that the AI provides suggestions, not definitive medical diagnoses.

Avoid Negative Reinforcements

The system shouldn’t learn or perpetuate harmful behaviors or biases from its interactions. If an AI is frequently confronted with misleading user behaviors, it should be designed not to adopt or reinforce those behaviors.

The user interface can act as a gatekeeper or filter for the kind of data that a generative AI system interacts with. If users often input biased or misleading information because of the design of the interface, the AI may learn from that skewed data. To prevent this:

• Ensure the UIs are designed to collect unbiased and representative data.
• Implement checks and balances, like validation mechanisms, to prevent the input of misleading data.
• Design interfaces that educate users on the implications of their inputs, especially in systems where user interaction heavily influences AI learning.

In short, the design of user interfaces and interactions can play a pivotal role in ensuring that AI systems receive accurate, unbiased data, shaping the AI’s learning and behavior.

Failsafes

Design mechanisms must allow users or administrators to intervene, correct, or pause the AI if it produces unintended results. This could be a user-accessible “stop” feature or a prompt for human verification in uncertain scenarios.

This is akin to designing controls that allow for human oversight. Consider an AI that curates news articles for users. If the AI shows increasingly biased or inappropriate content, users or administrators should have an easy-to-access method to stop its operations or guide it back on track.

Examples might be:

• A “Report Issue” button for the user to provide feedback when the system doesn’t behave as expected.
• A “Pause Recommendations” feature to temporarily stop AI-driven suggestions.

The idea is that while generative AI can automate and enhance many processes, it’s essential to embed layers of human oversight within its operations. Just as we design UI/UX with user errors in mind, generative AI design should account for system errors, with interfaces that allow humans to correct or intervene as needed. Check out our article on explainable AI once you dive deeper into the actual design.

Put Humans First

Beyond technical considerations, we must focus on the human-centered aspects of AI safety. Here are some places to start:

• Gain a deep understanding of the people that use our products. How will people interact with the system?
  What are their needs and concerns?
• Make it easy to use and understand.
  People should be able to use the system safely and effectively, without having to be experts in AI. Experiences powered by generative AI should use clear and concise language, and indicate what content is produced – to ensure people are aware.
• Build trust through design!
  People should feel confident that the system is safe and reliable. This can be done by being transparent about the system’s capabilities and limitations, and by proactively seeking feedback from the people that the system is designed for. You can also check out our Gen AI Design Guidelines around Building Trust.
• Be aware of the potential for harmful bias. Generative AI systems are trained on data, and that data can reflect the biases of the people who created it. It’s important to be aware of the fact that left unchecked, those biases could be amplified by LLMs and result in harmful outcomes for customers and the people who use our products. There are many examples where biases around gender, race, and ethnicity in algorithms and data sets have resulted in unfair practices and exclusionary product experiences. A couple that come to mind include: Soap dispensers not recognizing darker skin tones and the security code built into TSA checks that flags transgender people as higher security risks solely because of their gender expression and bodies. There’s a recent example an algorithm designed to make the recruiting process easier for hiring managers only surfaced male candidates because of gender bias in the training data. The truth is, we must pressure test our data sources, be very intentional about who is represented in the data – and who we are including in our user research from the start.
• Encourage user feedback. As generative AI powered applications become more sophisticated, it is important to be open to feedback from people and experts. This feedback can help to ensure that these systems are used safely and ethically. We should also make it easy for people to report problems and issues they identify.
• Give people control over the system.
  People should always have control of what’s being created by generative AI applications, including the ability to turn it off or to override its decisions.

Responsible AI Is a Team Sport

By incorporating safety and ethics into your design process, we can help to ensure that we improve customer outcomes and create experiences that ALL people love. However, what we design is only as good as what we ship.

Remember: Cross-Functional Collaboration is Key!

We must work actively with the entire team – which includes product managers, data scientists, and engineers – to raise awareness of the risks and benefits of generative AI. It’s important to keep in mind and apply both the Guiding Principles for AI Ethics and Design Principles for Generative AI throughout the entire process – from idea to delivery.

Related Links

Design Guideline on Explainable AI