Flush Touch Screen Working Prototype, Version 8

So obvious question is; whats different from version 7. The answer is not much in that it uses the design articulated in verion 7. But this is not a test, it is a real working prototype, with all the machining done to specifications. It has a finished rectangular LED Guide, inset into a plywood version of the MDF table material that will be used in the real INVIVIA conference table version. And, I might add, done on a DIY CNC milling machine by a guy with a walker…(’cuz the idiot broke his hip skiing!)

There is one important difference with version 7 test. Here I have only had to use one line of LEDs running along the top of the LED Guide. That this would be enough was a pleasant surprise, and it bodes well for using it in an environment where there is more ambient IR light that would force me to use LEDs on all sides to compete.

Here’s video of demo shown above and video from camera’s view

[The blob tracker is still jittery and sometimes looses the blob position, even though the video image is solid. I worked for about an hour and found the place in the python code where this is happening, but couldn't find a way to keep python from forgetting what it should do. This will take some work to fix, but is very fixable.]

[The melamine screen edge in this prototype version was cut into squares and taped to the plywood guide holder as a way of conserving the one sheet of melamine I plan to use for the real table. In the real table there will be a perfect hole that the screen sits flush into.]

Next Steps:

Lenseless Projector, no mirrors approach:

The big question, as I see it, is what kind of projector to use which determines how the table will look. I think there are two options. The simplest and initially most expensive one, and the one I believe will produce the best looking table uses the NECWT610e lenseless projector that was demo’d to us at INVIVIA.

This option is different from all the alternatives to follow in that there is no additional large mirror to mount under the table. The folding of the optical path is all done in the one projector. The big downside, beside the cost of the projector, is that the projector needs to be oriented vertically which will decrease the bulb life (but maybe not by much). This is an unknown we will need to deal with if we choose this options. (and sorry for the crude representation of the projector, but I think the massing and position is correct)

Conventional Projector, one or more mirrors approach:

The other approach uses a conventional projector, 1/3 the cost, but since the throw distance is so much longer must use two mirrors mounted under the table. This version, really just a klugy repurposing of an earlier design, folds the optics so that we get the shortest table, but what that does is put the mirrors down near the floor… maybe not the best design for the conference room table. Biggest downside to this design is that it would be very difficult to sit on either side of this table.

Here’s another two mirror conventional projector mirror arrangement. It is much less deep but not really as simple as it can be. We would choose this version if people had to sit on all sides of the table, as it gives a reasonable amount of leg room all around.

This final version turns the projector upside down, mounts it to the underside of the table at the very end, where these is just enough room to project an image large enough to fill the one, medium sized mirror, right below the screen. The mirror is held in place under the table by two clear polycarb supports that connect cross members with mirror position adjusting screws. Only downside I can see is that the projector is at the end of the table, making that side impossible to sit at. Otherwise this is the second simplest to build, with the NECWT projector version being the simplest.

here it is seen from slightly below. The mirror rests against the 3 angle adjusting screws.

Currently, I am working out the best way to cut a perfect (+/-.003″) rectangular hole into the top melamine sheet.

Further Next Steps Thoughts:

6/7/08->

Having become more informed about MS Surface, INVIVIA would be crazy not to differentiate what we are doing as much as possible from the current Surface direction. To this end I have found another projector type, an ultra short throw conventionally lensed 2000 lumen DLP projector that allows us to fill nearly the whole conference table surface with bright projected image using one mirror. The projector is the Toshiba TDP-EW25U Conference Room Projector.

Here we see the table from the side showing the light fan (imagine the room filled with smoke) which shows how the projected image approaches and bounces off the mirror. This arrangement has a projector throw distance of just 26 inches, projecting an image 26 by 44 inches (50 inch diagonal). The projector has a native resolution of 1280 by 800 pixels.

The projector and mirror need to be protected from the sides and this is done with long pieces of frosted plexi attached to legs.

This size and shape would allow 4-6 people to work comfortably around the table, each with enough real estate to not feel squeezed. Given the 2000 lumens and the fact that the projector uses DLP the image should be bright, sharp and nice contrast. We need to see whether 1280 by 800 is enough resolution for that size.

Dual Use Version:

7/3/’08

Following up on a suggestion by Daniel Spann that it might be really nice to use the touch table in a (near) vertical orientation, I reworked the optical path, added a mirror and projector cowl and added a hinge so that the table can now be used in either the horizontal or (near) vertical orientations.

Lifting the back of the table swings it to the near vertical orientation.

Of course there are many specific details to be worked out, but I think this concept sketch gives an idea of one way to make a very flexible prototype for exploring more uses of the touch table approach.

The projection surface shown here is ~ 22 by 35in (41in diagonal), set in a 28 by 41in table top. This is the smallest image the projector is made to deliver without modification. I think it may be possible to demount the lens and add a short lens extender tube to make projecting a smaller image possible.

Glowing Breath

While doing tests to understand the last subtleties (I hope) of the shape of the touch screen LED guide, I noticed that when I held my hand close to the screen, but not quite touching it, the moisture from my fingers condensed on the screen leaving a glowing area even after I moved my hand away. Then I breathed onto the screen….

(video)

It’s not a stunning revelation, but I thought it was a neat, ephemeral phenomena worthy of sharing.. something that might spark an idea for an artwork or interaction piece.

And while you’re at it.. have a great New Year!

cheers, ronmac

Flush Touch Screen, Version 7

At our last meeting we determined that big plastic touch screen frame bezels were out, that inserting the touch screen directly into the melamine-MDF table surface was the way to go. In further comparison testing of the different screen material options, I found that the thinner the touch screen material the better the FTIR effect. I now have a design that shows the glowing finger with the minimum of pressure across the entire surface, as well as eliminate all “bezelness”. I also had the good fortune to find a supplier of the same “Cherry Veneer” melamine surface used on the conference tables.

The challenge was to get the maximum amount of light into the edge of the touch screen. The approach I took with version 5, with the 1/4″ touch screen, was to put the LED on axis with the center of the screen. But in the case of this .107″ screen, there simply is not the room to do this. The LED must be at an angle to the axis of the screen, but the angle must be as shallow as possible to keep the light inside the screen, rather than spilling out onto the projection surface.

In this version we expose the LED as much as possible and cover the channel with reflecting mylar tape to direct the light into the screen edge.

here’s edge view from below the screen

and finally a little video of what the IR camera sees. The small dots are where the finger is lightly touching the screen, bigger dots show more pressure.

Z-Depth Touch Table Proposal

Recent INVIVIA brainstorms and discussions have pointed to the desirability of adding Z-Depth to the Touch Table idea. The least obtrusive way to do this would be to illuminate the hand from below the screen and use stereo cameras situated there as well. I’ve made numerous tests of this approach using different screen materials and illuminator positions.

What I’ve learned is that the ideal screen material would be a great diffuser of the white projector light and yet be transparent to the IR light illuminating the hand and provide a clear view for the IR camera. I have not found such a material. What I find instead is that a terrific screen material is almost opaque to the IR source and IR camera, and a material that functions well for the IR camera is a terrible projector screen.

To illustrate the problem I used frosted mylar, the material that is the closest I could find to fulfilling both criteria: allowing the cameras to see through somewhat and still provide an adequate projector screen.

So here’s what the camera saw, with the hand touching, looking at the underside of the frosted mylar illuminated by a couple of rows of IR LEDs (MS Surface works something like this, I assume)

When the hand is slowly pulled away from the mylar (here at 1inch increments) the diffusion of the image (very good for the projection) blurs the hand so that at 4inches away it is no longer recognizable, not by human eyes and certainly not by a video camera.

 

If all we needed was 2inches of hand travel we could probably make a frosted mylar screen work.

Looking back at the ZDepth stereo demo from last year is instructive. It shows that you need at least 8-10 inches of depth of movement in order not to severely constrain the user. (the cameras have the blue lights and are located above the screen)

 

Here’s what the stereo cameras see:

 

The Proposal:

 

Until we find the magical screen material which diffuses in white light and is transparent in IR, I propose that we adapt the approach I used successfully in the Hand Follower demo above to the Touch Table environment.

My idea is that we build (or at least think hard about building) a Z-Depth module that is simply a small wide angle USB video camera, a stacked and angled fan of IR LEDs, and a housing to protect them (and in this case hold the connector to get the signals into the table.)

Here are other housing variations:

If we can’t find a suitable small USB camera, this housing is big enough to cover the existing Logitech version

And, of course, there is the obligatory Egg variation:

…and this final (not very well crafted) variation where the parts are partially hidden in the 1inch table depth and clunky plastic parts are traded away for lots more veneer…

….obviously, ideas for other variations (or changes to these variations) are welcome….

 

Solving the stray window light problem:

When you look again at the vertical ZDepth demo pictures you realize that they were made in a pretty controlled environment where stray light was easily controlled. The INVIVIA conference room is another matter entirely.

My living room has windows big enough to offer a similar experimental environment for testing. The problem that has to be dealt with is that at least one of the cameras on the table corner will likely be looking across the table to a well lit window. Blacking out the window entirely is not an option. What is needed is a way to modulate the IR light coming through the window without drastically changing the white light amount or color.

What I found was a family of heat absorbing commercial window films, intended to keep heat out during the summer and in during the winter that do the job pretty well.

 

Here you see the video camera (blue light to the right of the screen) looking at my hand and the window, to the left of the screen is the IR Illuminator. In the video capture window on the screen the only things visible are my hand and the source, even though the bright window in the background is in view…

Apparently there are non reflective versions of this film that would decrease the surface crinkly effect on the film.. That version was not available in Home Depot…

 

 

 

 

 

Flush Touch Screen, version 5

The version 3 Touch Screen’s geometry was greatly determined by the thickness of UV coated polycarbonate sheet I had found at HomeDepot that seemed to do the best job of aiding the FTIR effect as well as providing smooth finger glide across the screen. Since the sheet was .095″ thick and the LEDs are .2″ in diameter LED guide had to have a substantial bevel in order to align the LED and the screen.

I have since found thicker UV coated plastic after a short web search and determined after a couple of quick prototypes using existing simple LED guides that the thicker sheet did in fact exhibit satisfactory FTIR qualities. It seemed right to design a flush touch table LED guide that would allow us to inset a touch screen in an existing table, mount the projector and camera underneath and begin in-house touch table experiments.

Here is the AutoCAD side view of the LED guide:

Since this is just a (quick and dirty) proof of concept prototype, I only machined enough of an LED guide to hold 18 LEDs or about 5.1″ worth of LED, (twice, one for each side)…

A closer view of the flush edge LED guide:

The side view, seen if screen extends to edge of table…

All this assuming that we might end up with a table that looks something like this:

Anyway, I will bring both the beveled and flush prototypes in to evaluate…

Inside the LED Guide

Just to make whats happening with the LED guide a little clearer, here’s the underside of the guide with the LED PCB board in the background. Note that the LEDs have to turn the corner in this version.

Here the LED PCB has been put in place (with some difficulty, ’cause there’s not much room in there)

Finally, a shot looking from the side the touch screen and video screen see. I discovered after this version 4 proto was made that the holes need to be deeper in order to mitigate the strong side illumination lobes from the LEDs. (Sadly I didn’t get a picture of the version 5 deep holes until the whole thing was taped together)..

Version 3 of TouchScreen…

Oct 16, 2007

I thought you’d all like to see the first integrated screen and LED Guide working…

Whats different from previous versions, you might ask?
- LED Guide, touch surface, video screen and backing sheet are now in one strong, reasonably nice looking frame
- LED orientation gives even light distribution across screen using only side arrays
- Heat dissipation into frame is reduced and now manageable.

what needs to be done next?
- Narrow beam LED circuits need to be made and tested. As it stands I am 2 circuits short of filling the screen with light and
found some narrow beam, very bright LEDs that should be tried… they will reduce spillover and may improve performance.

- decide on housing shape (table, kiosk…convertible?) and build it (or whatever variations seem to make sense) so you guys can have a new toy to play with and we can begin to get some applications developed and running on it…

here are some development process pix:

the AutoCAD cross section

Milling the slot to hold the touch surface…

just finished cutting the shelf that makes connection to next piece

the four pieces before cutting off clamp ends

partially assembled showing LED circuits..

……more as it happens…..

cheers, ronmac

Draft of Touch Screen Instruments

We’ve decided that an instrument is the most interesting way to go re: the touchscreen interface.

Here are a few xylophones. Their shape is designed to look like a xylophone but they aren’t too specific or refined. Any feedback would be appreciated, e.g. if you think they need to comply to certain specifications, etc. that would be helpful. After I hear back from you, I’ll draw up another prototype and we’ll go from there. Thanks!

Invivia’s Research Blog Launches!

Welcome to Invivia’s informal blog for their ongoing research projects!

Edith’s Notes from 7/12

INVIVIA multi-touch displays: what could they be? beyond surface! We imagined different shapes and textures of displays, at different scales and orientations. We though of possible activities—or experiences—that, more than others, scream for X-touch displays (ex: things people like to do using 2 hands, ten fingers, or working together, coordinating touch-points and traces among themselves; transactions that become “magical” when mediated through touch; objects that unveil themselves in unusual ways when placed on touch-screen… In this note, I started to list a few projects / ideas under different categories,

Categories:

1. form - scale up / scale down — horizontal/vertical/slanted
2. form - bend / wrap / fold – from flexible to rect-verso to3D
3. Experience - playing music, data visualization/manipulation
4. other areas to explore – art installations, science,

___________________________________________

Short video [13MB] (pretty old)
http://cs.nyu.edu/~jhan/ftirsense/
http://www.perceptivepixel.com/
http://www.ted.com/index.php/talks/view/id/65
Han, J. Y. 2005. Low-Cost Multi-Touch Sensing through Frustrated Total Internal
Reflection. In Proceedings of the 18th Annual ACM Symposium on User Interface
Software and Technology

Scale up / scale down – Horizontal / vertical / slanted
UP- multi-touch walls / boards (vertical), tables / floors (horizontal)
http://www.engadget.com/2007/06/24/prometheans-interactive-whiteboard-gets-multiinput-upgrade
http://technabob.com/blog/2007/06/09/worlds-largest-multi-touch-screen/
when horizontal, boards and walls become tables, pools, and floors. You can use feet (stepping), wheels (rolling), or skates (gliding) to navigate. You can dip fingers in ewater and see circles on the surface. You can put objects on surface, play around with them as if they were real (photos, cameras, albums, maps). You can manipulate traces.

You can transfer contents between objects put on table: ex. Digital camera, download pics (cf. surface). The bigger the floor-display, the more options you have to immerse yourself, with your props, like an actor on a stage.

DOWN - multi-touch-tiles / mini-screens (portable devices)
EX: Nintendo http://www.engadget.com/2007/07/11/nintendo-wii-fit-gets-you-stepping/
Ex: Wii Fit. Wii Balance Board with dual sensors detects your weight and balance on each side as you exercise. The board is wireless, and holds the potential of full-body games involving the Wiimote and the Wii Balance Board simultaneously. If you’ve had enough gaming, you can also have the Balance Board diagnose your body mass index. imagine multi-player scenario if attached on ceiling, tile could be engineered to sense pulls instead of pushes

Bend - Wrap – Fold (1)
Flexible large-scale: From walls to drapes and curtains
Khronos - Alvaro Cassinelli. NextFest 2006
http://www.technovelgy.com/ct/Science-Fiction-News.asp?NewsNum=912
Khronos Projector uses a flexible sheet screen; the projector itself stores a video of a single scene over time. When you press on the flexible screen, the projected scene varies in time. (cf. video on website)

[related] the energy curtain – Touched by the sun… The energy curtain captures energy from sunlight, returns light at night
http://www.technovelgy.com/ct/Science-Fiction-News.asp?NewsNum=798
It is the product of a research project funded by the Swedish Energy Agency. It was demonstrated at the Wired NextFest 2006 festival of technology at the Javits Center in NYC about a month ago Flexible small scale: From tiles to patches – from portables to wearables I imagine multi-touch / multi-sensor patches, bandaids, jewelery, and gloves…But then, again, I may be way out there given the technology we work with.
http://www.technovelgy.com/ct/Science-Fiction-News.asp?NewsNum=1093

——————————————————————————-

Cool website; technovelgy – where science meets fiction
http://www.technovelgy.com/ct/Science-Fiction-News.asp?NewsNum=797

Bend – Wrap – Fold (2)
Two-sided multi-touch screen — Ex: Under the table interaction (UIST 2006) Two-sided touch-table allows input from both bottom and top
http://www.youtube.com/watch?v=fDDhbHrLSnk
It would be great to have a large-scale two-sided multi-touch screen Screen could be placed vertically or horizontally (front/back or over/under) It could be “see-through” or not. All kinds of activities can be imagined. 3D multi-touch screens (DOME, column, cylinder) this is a huge area to explore. Could be worth further exploring

Activities
Playing music
Digital DJing –
http://module-records.blogspot.com/2006/01/hawtin-wants-multi-touch-screen.html
Jazz editor
http://www.jazzmutant.com/lemur_overview.php
drawing music…
here goes a cool pre-multi-touch-sensor version……
http://www.technovelgy.com/ct/science-fiction-news.asp?NewsNum=754
creation of Japanese musician and artist Miyako Dub Squall The musician/artist draws with different colors on a standard LP record-sized blank piece of paper. The sensors in the arm of the turntable can distinguish between five different colors (black, red, yellow, blue and green). As the turntable spins the ‘platter’ the music plays

Information visualization and manipulation
Multi-touch sensing on LED matrix displays
http://cs.nyu.edu/~jhan/ledtouch/index.html

Art Installations
Media Mirror
An interactive video installation, in which 200 channels of live cable television are continuously rearranged in real-time to form a mosaic representation of the person that stands in front of it. (in progress)
http://cs.nyu.edu/~jhan/mediamirror/index.html

TraceEncounters
A minimalist social network tracking pin one thousand pins distributed at Ars Electronica 2004 (with Brad Paley, Peter Kennard)
http://www.traceencounters.org

Z-Depth Touchscreen // Ron

Here is a very crude, very quickly made partial demo of stereo on the touch screen, which will enable the hand to be tracked near the screen…
( I didn’t have time to get the math right for this camera orientation so am not generating the righ z value which we can map to size, or whatever…)
but what you see here is that putting stereo cameras off axis, behind the screen can generate hand position…