2 service planners
5 designers
6 developers
Project info
Timeline
Role
Tools used
Sponsored project by Korea Creative Content Agency
User flow design
UX/UI design
User testing
Prototyping
3D modeling
Sep 2022-Feb 2024
Unity
Blender
Shapes XR
Figma
Just Jam is a social VR platform for musical collaboration and interaction, featuring instruments specifically designed for VR ensembles. Users can collaborate and create music together, regardless of their skill level, while socializing with others.
VR social ensemble platform
Relevant link
Featured in
New contents academy is a program held by Korea Creative Content Agency (KOCCA) which is an official government agency that supports production, creation, business growth, R&D, including broadcasting, video game, IP, new technology convergence content. Out team was selected as a participant of this program with 7500 USD financial aid to develop the project.
New Contents Academy - Interactive VR contents
The challenge
The goal
How can we enhance engagement and user retention of VR games?
Many existing VR games fail to motivate users to come back to the experience.
Create an ensemble game where players can socialize and interact musically, using VR-specialized instruments.
Gameplay demo
VR-friendly instruments
Socialize with other players
There are three instruments in the game. EDI, marimba, and percussion. Users can ensemble with other players with these instruments which are ergonomic and VR-friendly.
The note mode allows people who can't play instruments to perform songs and be part of the gang.
Make an ensemble with a friend. If the users are not into playing music, they can just watch and hang out with people around them. The virtual space is consisted of the stage, gathering area, and a bar. Share ideas, comment on the music, and play!
Service goal
Engage players in musical interactions and exchanges through ensemble performances using VR-specialized instruments.
• Develop musically complementary VR instruments
(melody, chords, percussion) that can be intuitively played using 6DoF input in VR.
• Implement fast synchronization capable of real-time ensemble performance and listening on Quest 2.
• Provide sheet music for each instrument so that players without musical experience can also enjoy ensemble performances.
‘Just Jam’ as a Social VR platform
Increasing Engagement
Ensemble
Content creation
Community
Just Jam aims to be a community where users can comfortably interact with others, an ensemble space where they can perform with a fantastic combo, and, in the long term, a boundless content space where user-created songs can be infinitely generated.
Flow chart (Vertical slice)
Regarding the virtual space and movement path of the user in VR, I designed a flow chart of user activity from when they enter to when they play music.
Virtual space design
Using Shapes XR, spatial awareness could be properly adjusted. The mood of the space was aimed to be cozy and trendy. After scale checking, scene model was migrated into Unity get the lighting and texturing done.
Concept
VR instrument EDI
Our goal was to create a new instrument that has new but controller-friendly interaction for VR environment. Since VR environment has XYZ axis, we thought bowing interaction would allow easy and fun controlling. Setting bowing interaction as our main interaction, the length and control range was determined based on the ergonomics.
Instrument setting controller
Scale indicator
Scale point
Scale panel
High pitch
Low pitch
EDI design process
Form sketch
Basic texturing
Final version
User testing
XMC spring conference, Feb 17-18
What
When
Who
Why
How
To test usability of EDI
To check user reaction to bowing interaction
Feb 17-18 2024
12 people
To find out key difficulties users might face
User testing, Interview
“Playing it was very fun and intuitive compared to the other VR music games. I can play long notes!”
“Since it is a new instrument optimized to controllers, the tutorial for playing it should be very clear.”
“Since it is used with both hands, it would be better if the view or focus was fixed to prevent accidental control.”
Key findings
• Users find bowing interaction interesting and fun. 11/12
• It is hard to control the power of trigger button subtly.
• More detailed tutorial is needed.
• Due to hand tremor, the reference point is unclear, which can cause confusion for users.
• Visual feedback should be added as well as auditory feedback to emphasize the effects.
The things we found out were...
EDI iteration
After the user test, we improved the areas that were lacking in the initial design. We added a scale point to help play the descending notes and incorporated visual effects to represent the playing state. Additionally, I suggested to fix the aiming of the right controller so that it only targets the area above the scale indicator to prevent missed notes.
EDI demo video
3D Interface & UI development
User flow
In the process of selecting and playing songs, we used a 3D interface with a jukebox and records. The hands-on interaction of inserting records into the jukebox enhances immersion in the virtual world when playing a song. While UI elements that require many options, like play settings, remain in 2D, we incorporated 3D interaction for starting the song to make the experience of playing the song more enjoyable.
Instrument, album selection panel
(In a distance that has good readability for text and contents)
Juke box to insert record
(placed near to the user’s waist)
If user selects song, the record comes near to the user with a visual effect.
Insert record to juke box.
(When the trigger is released near the insertion point, it snaps into place)
The instrument is generated and user can start playing.
Scale & distance check
Design process
IA & Sketch
The design
Based on the planning specification, we visualized the UI needed for the ensemble feature. We ensured a clear distinction between the default and activated states, using diagonal designs to create a funky feel that matches the mood of the game.
Initial version in Unity
After organizing the pixel values for the design and passing them to the developer, we verified the sense of distance and usability within Unity. The UI was positioned so that players would view it from about 1.5 meters away, ensuring readability was maintained. Due to time constraints, the record disc was temporarily replaced with a 2D placeholder.
Over the course of a year working with my team to complete the project, I learned practical methods of aligning and communicating ideas. Additionally, thanks to the mentor provided by KOCCA, I was able to grow significantly through professional advice. Due to time constraints, we were only able to implement part of the planned design, but if we were to further develop the project, I would...
• Measure the metrics again with updated EDI and compare the results.
• Implement record inserting 3D interface and interaction including haptic response and snap interaction.
• Observe social behavior among the users and supplement socializing features in virtual space.
Learnings
Next steps
• Collaborating with more than 10 people for a year taught me how I can effectively manage assets, schedule meetings, and communicate with multidisciplinary team.
• Prototyping in a 3D environment enhances spatial awareness, enabling the creation of optimal movement paths and • appropriately scaled UI elements relative to the space.
• By showcasing it to many potential users, I was able to gain real-time feed back and observe their vivid response.