Co-authored IEEE research (2022) · Human-centered AI
Emotion-aware music, designed for human understanding.
Facial expression is an uncertain signal. I designed the experience that turns it into a clear, supportive listening choice: synthesizing research into a four-stage journey, designing feedback and explanation states, and helping integrate them into a working research prototype documented in a co-authored IEEE paper.
- My role
- UX research · Experience design · Prototyping · Implementation · Co-author
- Year
- 2022
- Output
- Research prototype + IEEE MysuruCon publication
Problem
Music recommendations rarely reflect a listener's immediate context, and an expression signal alone can be wrong.
My role
UX research synthesis, journey architecture, feedback states and prototype integration support.
Deliverable · 2022
One working research prototype and a co-authored 2022 IEEE MysuruCon paper.
Design challenge
An uncertain signal.
A clear listening choice.
How might we turn an uncertain expression signal into a clear, supportive and understandable listening decision?
A facial expression is not a feeling. The experience had to present the system's reading as an interpretation, keep each step legible, and leave the listener with a choice, without implying clinical value or that emotions were definitively inferred.
My contribution
My focus within the team:
Research-to-UX synthesis: translated emotion and music-therapy literature into listener needs and design principles.
Mapped the capture → recognize → curate → recommend journey and its system states.
Designed feedback and explanation states so a prediction reads as an interpretation, not a verdict.
Supported integration of recognition output into the interactive prototype.
Documented design decisions in the jointly published paper.
Listening journey
From emotion cues to an intuitive listening experience.
01Capture
Expression input
Input: facial expression
Webcam expression input, requiring clear user expectations.
02Recognize
Interpret the input
Output: expression prediction
An expression prediction provides the input for playlist curation.
03Curate
Playlist mapping
Mapping: emotion to playlist
Research-informed playlists connect detected expressions with listening options.
04Recommend
Music recommendation
Result: listening option
Present a clear music recommendation and understandable next action.
Design decisions & engineering handoff
Three design decisions
Feedback clarity, intentional curation and a focused interaction path shaped the experience.
01
Make feedback and system states clear.
Connect expression prediction to playlist mapping so the listener can understand why a recommendation appears.
Trade-off
Keep explanations concise and frame the detected expression as an interpretation.
02
Curate with a listening intention.
Music-therapy literature informed playlist choices beyond simply matching music to a detected mood.
Trade-off
The curation rationale is literature-informed; listening benefit would need a separate user study.
03
Guide one interaction at a time.
Separate capture, interpretation, curation and recommendation into a focused path toward a listening option.
Trade-off
Prioritize the core journey in the research prototype, with consent controls and manual choice as next design priorities.
Prototype implementation
From research to functioning prototype.
Research & definition
Emotion-recognition and music-therapy literature helped frame the problem: translate a technical prediction into a useful listening experience.
Flow, UI & prototyping
The four-stage journey connected expression capture, system feedback and playlist delivery in an interactive flow.
Integration, evaluation & publication
Recognition output was connected to recommendations in a working prototype. The team evaluated the recognition system and documented the work in a co-authored IEEE paper.
Outcomes · what the evidence actually means
From research to interactive implementation
Literature-informed playlist mapping and clear input states connected in a functioning research prototype.
- Experience architecture
- 4stages
- Capture, recognize, curate, recommend: a connected input-to-feedback journey.
- Recognition model evaluation
- 89%
- CNN expression-classification accuracy vs 86% SVM baseline (+3 percentage points). This is a model measure, not user-task success, satisfaction or a shipped product metric.
- Working research prototype
- 1
- Expression input, playlist mapping and recommendation feedback connected through interaction design.
- IEEE publication
- 2022
- MysuruCon conference paper documenting the implementation and recognition evaluation.
Limits & what I'd improve
Keep the listener in control.
The 2022 evaluation measured recognition-model performance only. Usability, trust and listening benefit were not tested, so the priorities on the right are what I would design and validate next—not features of the original prototype.
Next steps · if redesigned today
Priorities for a future iteration.
- Consent and privacy: explicit opt-in before webcam capture, with visible capture state.
- User control: a manual override when the prediction doesn't match how the listener feels.
- Trust feedback and usability research with diverse listeners, measured separately from model accuracy.
Co-authored research · 2022
Music Recommendation using Emotion Recognition. IEEE MysuruCon 2022. The publication documents the research methods and full team attribution.
Official IEEE Xplore record