Hi, all. Happy new year. Here’s something I’ve been working on which I reckon you might like.
A while back, Roo Reynolds posted about his experiments with Reactivision. Reactivision is a really neat way of building what’re called tangible user interfaces; it’s a bit of software which reads in video being captured by a webcam and tracks the position (and rotation) of special blobs called “fiducial markers”. In other words, it can work out where, in the camera’s field of vision, these objects are, and what way they’re facing.
I’ve mentioned this thing in passing before, actually, but I didn’t think anything more of it at the time. Roo’s post mentioned that the core vision library was open source, though, and there’s a Processing library, so one night on my way home from work, I decided to buy a cheap £20 webcam from Tesco. The following day:
I had this set up. (That’s my desk at work, by the way. Tidier than normal.)
Anyway, back then, I wrote:
Imagine a visualization program where you could build molecules like that…
so I took a couple of days and did pretty much that (and here’s the source code. It runs in Processing, as long as you’ve installed the TUIO library from the Reactable website.). What you do is you lay atoms (represented by different designs of blob) on the table: they appear on the computer screen. Move them closer to each other and, if they can, they form a bond to each other. It feels kind of magical, really!
I was well chuffed.
Anyway, having the markers face up with the camera above isn’t great for a couple of reasons; the markers might be easy to tell apart for computers, but they certainly aren’t for humans, and even more importantly, your hands keep getting in the way. So the next stage was making a glass-topped table:
(You’ve got to love the improvised bounce light, if nothing else.)
This worked a lot better; if nothing else, I could scrawl element symbols on the back of the “atoms” so I could remember which was which! That’s ethene on the table, by the way. If you lay out something else - say, carbonyl sulphide - you get
on screen. Cool, isn’t it? Click on that window and it sends you off to the PubChem substructure search page with the structure already loaded in for you. Start the search and this is what you get back:
So, to sum up, that’s performing a chemical search by just putting real objects down on a real surface, made really cheaply. Result! You don’t even need the webcam to try it out: install Processing and the TUIO library, download the TUIO simulator and unzip it, replace resources/config.xml with this one here, start up Processing, load in ChemReact.pde, and run TuioSimulator.jar; that gives you a virtual table, and you’re home - move the boxes around in the simulator to add or move atoms in ChemReact. Here’s what the simulator looks like;
Anyway, this little program’s (as you’d expect) rough around the edges, there’s no documentation other than the source code (one hint - to see what marker corresponds to what atom, have a look in moieties.txt), and if it breaks chuck me a comment and I’ll try and fix it - but it’s a usable prototype, and building it cost me under £30. That’s kind of awesome really; playing with this kind of thing really doesn’t have to be hard or expensive any more. It doesn’t even need any really specialist kit. Brilliant. Maybe, in 2008, the future’s finally catching up with us.
I still want my jetpack and my flying car, though.
Just want to say: this is really cool! Go you!
That is really cool.
Andrew, simply amazing. A blog post follows :)
Brilliant, I’ve sent the tip to Phil Torrone at Makezine.com
Great stuff Andrew!
As I did with the MIDI experiment, I like sticking the printed tags to the bottom of physical objects. Even if they’re just different wooden blocks of different colours (or shapes), it would look seriously cool to drag identifiably different things around, and would be easier to pick a selection of them out to combine in interesting ways.
[...] of them is Andrew Walkingshaw, whom I met in person at Scifoo. Turns out Andrew has developed an absolutely killer concept UI called Sentinel that allows you to use physical markers with different shapes, each representing an [...]
[...] I’ve mentioned this thing in passing before, actually, but I didn’t think anything more of it at the time. Roo’s post mentioned that the core vision library was open source, though, and there’s a Processing library, so one night on my way home from work, I decided to buy a cheap £20 webcam from Tesco.DIY Tangible user interface for performing a chemical search by just putting real objects down on a real surface, thanks Attila! - Link. [...]
Lovely idea (as I have said in person I know) … just one thing - that surely can’t be ethane you have on the table … ethene quite possibly.
Joe: naw, it’s always said ethene there! (Ahem. Erm. Thank you.)
[...] Sentinel (progetto pazzesco!!!) Sentinel [...]
Duly sent along to some cow orkers at Microsoft with smug comments about MS Surface ;)
To be fair, Surface does rather a lot more than this does!
It does, especially with the display tech side of it. That said, could you use frosted glass and coaxially mount a projector? Either that or polarised glass, I guess…
Mark: that’s pretty much exactly what the guys behind the original software do. They sent one on tour with Bjork - check Youtube for the videos…
(That’s how I saw Reactable initially - it was on Music Thing or Create Digital Music. This is kind of a sweet-spot, all my different skeins of dorkiness knotting together!)
Woooohoho! Amazing home-made stuff!
[...] well, scientists. You know I’ve got a horse, or two in these races, so I’m biased. Even so, maybe that’s a pretty academic question; [...]
[...] any sense at all. This is a cool tangible software interaction demo based on Reactable. Here’s another example that enables physical molecular modeling by arranging peices of paper on glass. This is similar to [...]