Early concepts
A few years ago, I was exploring ideas for simple, experimental tangible user interfaces (TUI).
These concepts were intended for use with musical instruments. Since then, I've become interested in developing this further. What I wanted to build upon was minimal and portable (pocket-sized) TUIs for creative expression.
There are many options for MIDI controllers of varying levels of portability and minimalism. The original intention was to develop something that would be a fun DIY kit. Such a kit could be the basis for creating novel musical toys. This could leverage a smartphone or raspberry pi.
What I have in my mind is something like a fidget toy.
When I started exploring fidget toys, I began to reason that because these toys are meant to be easily taken out and used whenever. Therefore, this controller idea would need to be similarly portable. In order to begin prototyping, I explored what people were doing with DIY fidget toys. It occured to me that these homemade tactile-focused toys would be great as TUIs.
However, it is beyond me as to how to cheaply couple them with electronics and sensors and make it portable.
Now, I am considering other approaches. It seems like a reasonable time to consder computer vision (CV).
CV is now distributed across 2 paradigms. The original paradigm is sometimes referred to as "classic CV". Classic CV uses image processing and carefully tuned parameters to very quickly detect specific shapes.
In recent years, due to the development of deep learning and GPUs, there are now many neural net (NN) CV techniques. This area is proving to be promising and has broadened the space of detectable objects. The trade off here is the necessity for much more computing power.
What I intend to do is explore the potential of CV in both of these paradigms. Such a system involving software and crafting would take advantage of technology people are likely to already have at home.
