Redefining Game Mechanics through Voice, Memory & LLM-Backed NPCs

Context & Research Question

Game accessibility has come a long way, yet audio-only games for visually impaired players remain limited in complexity, often focused on simplistic or child-oriented experiences. In parallel, the rise of large language models (LLMs) presents an opportunity to reimagine how players interact with non-player characters (NPCs) not just through pre-scripted dialogue trees, but via natural, real-time conversation.

Research Question:
Can the use of LLM-backed NPCs improve player experience and agency in narrative-based games?

Solution

The proposed prototype was an audio-only, mystery game where players take on the role of a blind Oracle navigating a politically unstable medieval court. Instead of visuals or fast-paced controls, the gameplay relies entirely on:

• Voice commands for natural interaction with LLM-backed NPCs
• Spatial audio cues for navigation and discovery
• Player's memory and deduction as the main mechanics for progression

Each NPC is equipped with emotional profiles, memory stacks, and narrative roles, allowing them to respond fluidly to the player's tone, choices, and investigation history making every conversation feel dynamic and personal.

Role & Responsabilities

As the lead and sole UX researcher on this MSc thesis project, I was responsible for the full research and design lifecycle:

Research Planning & Literature Review

• Identified key accessibility and interaction gaps in both academic literature and game industry practices
• Defined the research question and framed the UX problem around LLM-backed NPC interaction in non-visual gameplay

Prototype Ideation & Narrative Design

• Conceptualised the core gameplay around memory, voice, and sound instead of visuals
• Designed character roles, emotional profiles, and narrative arcs for smart NPCs

Technical implementation & AI Integration

• Implemented the ConvAI plugin in Unreal Engine 5
• Structured prompts and memory stacks for LLM-based NPC behavior
• Designed spatial audio elements to support immersive, navigable environments

User Testing & Methodology

• Conducted pilot and formal testing with 22 participants
• Created SUS and GUESS-based test protocols and interview scripts
• Captured behavioral data using screen recordings of gameplay for post-analysis
• Applied Braun & Clarke’s (2006) thematic analysis for qualitative data

Iteration & Insight Synthesis

• Analysed user feedback to identify pain points in comprehension, trust, and immersion
• Proposed improvements for voice interaction, NPC logic, and audio layering

Communication & Reporting

• Documented and visualized findings for academic thesis submission
• Prepared research outputs for conference presentation