TalkBox Project

The goals of the TalkBox Project are as follows:

  • Empowerment: to provide a low-cost open-source, open-hardware Speech Generating Device (aka “SGD”) to the community, so that those who need such devices can more easily obtain them.  To empower more individuals who have disabilities be makers themselves.  To create an ecosystem of making that provides employment opportunities for high school students who have developmental disabilities.
  • Community: to build a community of knowledge and practice around maker approaches to assistive technologies.
  • Innovation: to make use of new modes of academic-community collaboration

This project is a collaboration with Ray Feraday and Tetra Society of North America. For more information, please visit the project page.

Fabrication Lab-in-a-Kit (FLiK): DIY Assistive Technology for Children with
Disabilities in Kenya

Approximately 10% of the Kenyan population is living with a disability; they face infrastructure barriers, (e.g., accessibility of transportation and buildings), barriers to economic participation and employment, and negative attitudes and beliefs of the community. At the same time, Kenya is fast becoming a technological innovation hub with opportunities for providing technological resources. In this project, we use a Community-Based Research (CBR) approach (Horowitz et al., 2009) to understand the socio-economic, cultural, psychological and technological dynamics that determine the impact of designing, developing and deploying do it yourself (DIY) digital assistive technology (DAT) for Kenyan children with disabilities.

Specifically, this project examines 1) how the stakeholders’ cultural attitudes and norms about disability affect the process of development and use of DAT for this population and 2), how collaborative Participatory Design (PD) processes shift the understanding of technology and disability in this cultural context.

This project is being conducted by a transnational multidisciplinary team of researchers in human-computer interaction, community development, social and cultural psychology, social work and refugee studies. One of our members is a founding member of a Kenyan NGO with several years of community-based experience working with children with disabilities. We have been working with local partners including Maseno University, 3 special education schools, and national disability advocacy groups including the Association of the Physically Disabled in Kenya (APDK), and the National Council for People with Disabilities.

As a prototyping and fabrication platform, we use TalkBox, a low-cost DIY open-source communication and information access system that we have developed and used in collaborative workshops with teachers, parents and children with disabilities who themselves created variations to fit their needs (Hamidi et al., 2015; Baljko et al., 2016).

The project outcomes include 1) understanding the impact of participatory practices on attitudes towards disabilities, on self-efficacy, and on the technical knowledge of stakeholders; 2) increased access to DATs both through access to the developed technologies retained by stakeholders and the participants’ increased technical knowledge; 3) KM products in the form of supporting documentation for the maintenance and re-creation of DIY DATs in Kenya; and 4) an increased understanding of the stakeholders’ socioeconomic and technological dynamics in the design and deployment of assistive technology with potential implications for policy and guidelines for others in the international network seeking the meaningful social inclusion of persons with disabilities through genuinely collaborative communities that bridge ability, age, income, and expertise domain.

ENAbling MEdia for braille Literacy (ENAMEL)

The objective of the ‘ENAbling MEdia for Literacy’ (ENAMEL) research project is to develop and to evaluate the digital technologies that are required by stakeholders to support vibrant ecosystems for literacy development. The ENAMEL project is currently supporting two projects: one focused on supporting learning of early-level Braille (in North America) and the other focused on capacity building in Special Education contexts in an East Africa country. The project focuses on low-cost, DIY technologies (such as 3D printing and low-cost single-board computers, such as Raspberry Pi and Arduino) and open source software. These technologies provide innovative and new approaches to supporting the development of functional knowledge in learners (primarily children) concerning printed and written materials.  This project espouses user-centered, rather than technology-centered, design approaches, and seeks to critically reflect on the hidden assumptions, ideologies and values underlying the design of such technologies.

Fabrication Lab-in-a-Kit (FLiK)

In this multidisciplinary project, a Community-Based Research (CBR) approach is used to understand the socio-economic, cultural, psychological and technological dynamics that determine the impact of designing, developing and deploying Do-It-Yourself (DIY) digital assistive technology (DAT) for Kenyan children with disabilities. Specifically, the project sets to examine 1) how the stakeholders’ cultural attitudes and norms about disability affect the process of development and use of DAT for this population and 2), how collaborative Participatory Design (PD) processes shift the understanding of technology and disability in this cultural context

Serious Games for Child-Focused Speech Therapy

This project entails the development of novel serious games to provide speech therapy interventions that are focused on childhood speech disorders. The current focus is on childhood Apraxia of Speech (AoS). This project is a collaboration between PiET lab and the iDAPT (Intelligent Design for Adaptation, Participation and Technology) Centre for Rehabilitation at UHN-TRI.

Real-Time Visualizations of the Vocal Tract for Clinical Interventions (VTV)

Speech impairments arise often due to difficulties controlling and coordinating the tongue, the most important speech organ. In the VTV project, we are devising novel modes of visualization for use in a clinical setting for interventions focused on tongue movements. Our goal is to develop a suite of visualization techniques to provide salient feedback both to clients and to clinicians, for use among the client-clinician therapeutic dyad. The feasibility of this approach is predicated on recently completed work that provides a sensor-rich alternative to acoustic-to-articulatory inversion, an approach that is based on the Wave system (Norther Digital, Inc.), which is a state-of-the-art enabling technology that can track a patient’s tongue with sub-millimeter accuracy. This work is funded in part by the Centre for Information Visualization and Data-Driven Design (CIV-DDD). These research activities are being undertaken by a collaborative team consisting of members from York, University of Toronto and OCADU (M. Baljko, Y. Yunosova, P. Faloutsos, N. Tenhaaf and F. Lebouthillier).

The Tattoo Project

The Tattoo Project is an interdisciplinary project that has several objectives:

  • To provide a repository for commemorative tattoos, for the public to upload their commemorative tattoos and contextualizing narratives, empowering users to make the project a social tool of their own;
  • To serve as a cultural heritage site, acknowledging important memories and sharing them publicly;
  • To provide scholars with a digital database of commemorative tattoos and narratives for analysis;
  • To develop relationships between academics and the public.

The project website can be found here.

The PiET lab’s involvement in this project focuses on aspects of human-computer interaction in technologically-mediated commemorative tattoo practices.


This research project concerns the design and evaluation of an embedded tangible system that uses a living media interface in the form of a living mushroom colony as part of its display, where the success of the mushroom colony is driven by the therapeutic and/or learning activities that are performed by children, who are its target users.

Synchrum: A Tangible Interface for Rhythmic Collaboration

Syncrhum is a tangible tool for rhythmic collaboration, inspired by Tibetan prayer wheel. Traditionally, turning a prayer wheel is used to send prayers to the universe, in fact, interfacing with it. Synchrum calculates rotations per minute and compares it with a target rate, deciding whether a user is in synch and for how long. Thus, it affords audience participation via sustained in-synch rhythmic movement, using a tangible physical object.

Syncrhum was originally developed as a student design challenge project and was presented at several conferences and events. For more information please see the publications page.


Traditional cultural artifacts hold great potential as sources of inspiration for the design of novel digital interfaces. While travelling in India and China, I observed the use of a fascinating cultural object, the Tibetan prayer wheel. This object exists in different sizes and colors and its use is prevalent among Tibetan populations in the aforementioned countries. A Tibetan prayer wheel consists of a circular cylinder, mounted on top of a graspable handle that can be used to rotate the cylinder rhythmically. A written prayer roll is inserted in the cylinder and the traditional belief is that each turn of the cylinder corresponds to a recitation of the prayer. This idea inspired the development of Synchrum, a computational object for use by members of the audience in a digital performance.

Aspects of Synchrum’s design

  • Initial prototypes involved hacking everyday objects (e.g., potato smasher and chair wheel) which later evolved into an original design.
  • Deliberate effortful interaction engaging the user in continuous rotation of a physical weight is used.
  • A distributed architecture, allows scalability where multiple units are monitored and controlled from an intuitive central interface.
  • The skill level difficulty of being in-synch is adjustable in real time.

Synchrum’s Applications

  • Engaging the audience in a digital performance that opens the possibilities for collaboration in narration and extends the range of expression.
  • Collaborative children activities, such as music controlling and synching.
  • Possible future applications include music education and physical therapy.


This project was supported by the Department of Computer Science and Engineering and the Faculty of Fine Arts at York University. The design and development team consisted of Foad Hamidi, Alexander Moakler and Assaf Gadot. It was done under the supervision of Professor Melanie Baljko. We would like to thank Professor Nell Tenhaaf and Sahand Amiri for their help and support.

Parametric Kinematic Speech Project

The Parametric Kinematic Speech Project is focused on the digital modelling of kinematic speech.  We are developing a data-driven biomechanical statistical anatomical model of the motion of the speech articulators. We are coordinating efforts with the Parametric Human Project.

Interactive Systems Project