Characterizing midair handwriting in virtual reality

Abstract

Midair handwriting poses challenges due to the lack of a physical plane to press against while writing, making it difficult to determine when ink should be placed. In this thesis, we gathered midair handwriting data from 24 participants in an environment that allowed them to write freely. We compared writing with a pen-like object and writing using a finger across two writing methods (writing freely versus on a virtual whiteboard). Using our data, we trained a neural network to detect when ink should be placed during midair handwriting, achieving an overall 85% accuracy. We developed a data-viewing application to recreate sentences for visual analysis. Participant feedback favoured the pen-like object as a writing utensil, with equal preference for both writing methods. Our contributions include a midair handwriting Virtual Reality (VR) application for data collection, a dataset containing 480 sentences of frame-by-frame midair handwriting data, and 20 unique prompts used in participant trials.