ePad Device
Hardware
The ePad device takes the form of a near-vertical multi-touch display, as shown in the image on the right. A person can interact with this display by directly touching it with their hands and fingers, as well as physical objects (e.g., a paintbrush). There has been a recent surge in research that enables this type of multitouch interaction (e.g. ThinSight, Microsoft Surface), and each of these techniques has in common that the hands, fingers, and/or objects are illuminated by infrared light, and a camera sensitive to that light can therefore detect their movement on the surface of the screen.
In the next iteration of ePad project, we will will use a combination of the results from our previous work and consultation with art therapists to determine the appropriate hardware setup. The device will be made so it can be easily dismantled and transported, or can be used in other configurations (e.g. on a tabletop).
Interface
The application running on ePad device provides art therapists with an opportunity to create an interface suitable for a particular user. There is a number of "widgets" provided by default (e.g. paintbrush tool, pencil tool, colour palette, etc.). In the future will will provide an online service from which new types of widgets can be downloaded (i.e., an "App Store").
The device saves the created paintings and can be attached to a printer for saving hard copies of the artwork, it also maintains logs of each session and produce summary reports for therapist assessments.
Artificial Intelligence
The system uses an Artificial Intelligence (AI) backend based on a partially observable Markov decision process (POMDP) that can estimate a user’s engagement. The POMDP can learn how a person’s engagement is a function of their activity, and can then adapt its choices of interface presentation and art activities to improve the level of engagement, as explored in our previous work (see Publications for more information).
The touch screen and speakers are used to display audio-visual prompts to the user for motivation or guidance. An art therapist has full control over the usage of the prompts and may create new ones. Some of the examples of available prompts are: playing a recording of the therapist's voice, placing a background image on the drawing canvas, flashing a button. Finally, a digital video camera is used for tracking the user’s face to monitor the level of engagement.