Reposting some camera data I sent to NUIGroup today

I’ve been meaning to post this for a while. I’ll just outline in brief some of the camera tests I’ve been running in the last month:

Prosumer webcams:
Logitech C910
Logitech C600
Agama V-2025

Consumer:
PS3Eye
GearHead WCF2600HD

Professional:
Firefly FMVU-13S2C-CS
Firefly FMVU-03MTM

Of the cameras listed above the Firefly 03MTM and the Logitech C910 are certainly the frame rate and field of view kings. Both the C910 and the Firefly cameras (color and BW) have larger sensors then the PS3eye. This means a wider field of view is possible with a smaller distortion. Even with this, I suggest a low to no distortion lens. I’m currently arranging to evaluate a handful of low/no distortion lenses with the above cameras. When that is done I’ll post a write up on all of the cameras and the CS/M12 mount lenses tested on each.

However, to remain true to why I’m posting in this thread. All of the cameras, including the Firefly suffer significant frame rate drops when used with CCV versus reacTIVision and TouchLib. I’m not sure why this is loss is so high in CCV.

With the C910 I was able to manage 1280x960 @ 15fps via settings. This translated to roughly 3 - 7 fps in CCV. I tested with 1.2, 1.3, and 1.4. In the other two processing packages I was getting 15 - 30 fps with the C910.

With the Firefly BW (still debugging some driver issues with the color cam) I was able to get a very solid 30 fps at 640x480. This has little advantage over the PS3eye other then a larger sensor so wider FOV and the image was significantly crisper.

The Agama and GearHead cameras were underwhelming.

Again, it is worth stating the current the Color High Res Firefly MV is not responding in CCV. I’m working with the Point Grey dev team to fix this issue.

Quick note about the new CCV 1.4 release!

The new version is available for download. The key update points are separate slider configs for Fingers versus Fiducials. Read more about it on the NUIGroup blog.

More fiducial tracking tests with MT4j

The following three videos show fiducial objects from the reactivision default library being read by CCV 1.4 from nuigroup. The demo application is MT4j with a couple of non-standard demo's running in the shell interface. The application I run for the fid-test is the fiducial tracker demo application from the advanced examples section. Enjoy!

(note: I did not trim the ambient audio out of these videos like the previous videos)

PAX Prime 2010 (Photos)

Below are a handful of photos from PAX Prime.  This shows the table build we demoed as well as the lighting we were under.

The new table design does fantastic under bright, indirect sunlight.  The only time we encountered a real issue was when the sun was setting on Friday and we had these lovely golden rays casting patterns across my demo surface.  Interestingly, after a lot of tweaking on Friday night, we did not have a single moment on Saturday or Sunday where the performance changed due to lighting.  I did make minor adjustments to the CCV threshold value around noon, but literally, the change was from a value of 54 to 64 -- already high.

We were on Sky Bridge which meant that we were on a large bridge platform under an arch of glass and steel.  This meant great lighting for showing off hardwood furniture, and what should have been horrible lighting for an IR multitouch table.  However, the most recent version of CCV (1.4a; thanks Amit!) meant a reconfiguration of my settings.  The new settings worked so ridiculously well that I only mentioned light sensitivity as something to watch out for if you took one home.

This was our (Geek Chic) very well lit booth

Here is where the Locus sat.  Well lit, with some filtering.

Look Ma, no flash.  The natural light had me nervous at first.

On Friday, we moved the blind around to block the sunset.  We did not need do this on Saturday or Sunday after I reconfigured the sensitivity.

Unfortunately the customer facing side got a lot of glare from the Atrium overhead.  As possible I had the customers pop around for a look at what a normal view would be like.

PAX Prime 2010 (Videos)

Wow and wow.  PAX Prime was very different then PAX East.  I was an attendee at PAX East and this time an exhibitor attached to Geek Chic, but even with the obvious differences in role, PAX Prime was vast and spacious and had the level of good feeling that I'd hoped to have found at PAX East.

I meet a lot of great people and showed the table to as many of them as I could.  The technology and the over all design were very well received with the (quickly becoming) usual question of alternate form factor was asked many a time.

I was surprised to see how many people didn't touch the system.  Even after learning that the system was a multitouch display, they were awed and asked lots of questions which I explained or demoed, but few felt that they should just reach in and touch the table.

This was not true of the other exhibitors.  They were not shy at all and the following videos are of the Runic Games folk playing their own game on the table.

And our second! A full blown interview no less.

Okay, so the ball has been rolling and here's the skinny.  My wife, a close friend of ours, and I have started a company to surround the work on the table.  We are called Mesa Mundi Inc. (website to come), and are working toward selling completed MT builds using both projectors and/or LCD screens.  Currently we are finishing our dev cycle for the first iteration of the product as well as resolving final pricing.

The video below took place before the formation of Mesa Mundi.  At the time (last week) I was planning to simply place these screen systems into Geek Chic products and bill them as such.  Now this is mostly still the plan.  Geek Chic makes awesome heirloom quality, hand crafted wooden furniture.  The Locus is a housing for the MT stuff I've been working on and together we have an amazing gaming product.  So, without further adieu (or caveats), I present to you  the video by Third Orange:

And the first of the videos has come in...

 

I'll admit to getting a little punchy after several days of demoing.  This is an excellent example.  Of course I'm not actually showing multi-touch here, just single touch events to show lighting effects and movement based in Map Tools.

However, this is a good example of the lighting quality of the screen after I realized the brightness on the projector was turned -all- the way down for the first 3 days of the Con.  Ugh!

Images from Gencon

I was so busy at Gencon demoing that I forgot to take video and barely took any photos.  Below are my only samples from the table as presented.   I'll finish my Gencon narrative as soon as possible.

Steam Punk and Multi-Touch Gaming Tables --- Oh! My!

 

The kids loved playing with the screen saver on the table.  I was running MS Surface Lagoon.

This was taken quickly while the crowd was still at bay talking to a Felicia Day cos-player.

As expected, someone in the (Do you want to date my) Avatar costume drew more of a crowd then a multi-touch gaming surface!  Her staff actually had a plasma ball at the top, it was awesome!

Return from Gencon 2010

I've just gotten back from Gencon 2010 and all I can really say is wow!  The response from the community to my multi-touch table was amazing.

I presented my multi-touch screen and software pack with Geek Chic from their booth.  They built a lovely enclosure (all of Geek Chic's tables are super awesome) and we put the whole thing together on Wednesday before the show.

I'll admit to severe nervousness regarding the table actually fitting the enclosure prior to the show.  The owner of Geek Chic and I had been talking on and off for a couple of months about build and specs, but I was concerned that I might have gotten something wrong or sent the wrong info.  And when I saw the box the Locus (the awesome name they came up with for the combined table) shipped in, I was even more concerned.  It was thin.  To thin.

Fortunately, Geek Chic knows exactly what they're doing and we took out the parts from the box, assembled the table and had roughly 1/2 and inch of extra room for guts of the table.  This lead to some confusion at first because the projection was too large.  After determining the source of the problem I grabbed some scrap and we propped up the projector resulting in much success.

The next hurdle was getting the darn thing to aligned right.  There was some damage to my mirror assembly by AirTran's baggage personnel.  This led to a twist in the video which I had to untwist by adjusting the mirror placement.

Finally, with all the initial kinks worked out, I was able to present the table at in all it's 42" of multi-touch goodness.

Then the convention center turned up the lights in the Dealer Room.  Oh Noes!

Actually the table did wonderfully under the super bright lights.  We had some heating problems but a kind fellow from another booth (Thanks Adrian and thank you to your lovely wife who's name I have forgotten, as well) lent us the use of a fan!

The single fan worked perfectly.  We were able to keep the whole setup cool with only the occasional overheat here or there afterward.   Sunday, I moved the fan into a better position and survived the whole day without incident.

-- More to come tomorrow --

Another long pause.

I'm linking to a design document which I put together for a project I'm working on with GeekChic, a really cool game table designer and fabricator.


Please feel free to check it out and comment.

Table improvements

Recently I purchased a new projector as part of the plan to make a clean, professional and ultimately marketable product. The new projector has a ridiculously short throw distance for a forward facing projector (traditional lenses in front type).

The purchase of the projector is exciting because, in the end, I'm a technophile. But in all project related seriousness. This new kit is vastly reducing the required height of my table design. While not the slim line concept we'll be seeing in the not so distant future. it is small enough in depth from surface that the large screen can be placed easily inside of most table top type setups; i.e. coffee table, work desk, formal table.

Anyway, as I finalize my design, as always, I'll post more data!

Images!

To go along with my recent video post. Here are some diagrams I whipped up in Lightwave.  The only thing not listed in these diagrams are mirror angles (these are mock-ups) and camera placements.  I'm currently running two PS3Eye cams on either side of the bottom mirror.  This configuration is pending more work on the object recognition side of the table.

 Enjoy.

Videos!

Some new developments

I've been quite for a while due to spending my time working on running the game this table was conceived of to support.  Now that we're moving along and I'm planned far enough ahead that I have some down time, I've returned to the table design work and began the process of looking for a housing.

Recently, I attended PAXE.  While at this wondrous if not line ridden event, I encountered a company which specializes in making furniture for gamers.  I've since begun a dialog with them to create a housing for my large format MT table.  Due to this recent development I'm going to be mum for a bit as I work out the legal details of what might be a business deal for making more of these toys.

So enough about that for now.  Here's the tech developments from the last 24 hours of work. 

1. Reconfigured my mirror array using a front surface mirror acquired from an old projection TV.  This new configuration resolves a fair amount of the screen "blur" (ghosting) which was there with the plexi-rear faced mirror.
2. Adjusted placement of the mirror and projector slightly to maximize the total display dimension.  I've not got a solid 46" diagonal 16:9 screen showing 100% horizontally and 98% vertically.  This means there is some cut off of the desktop environment in the vertical direction.
3. Tested fiducial with CCV 1.2 (fiducial) using two PS3 Eye cams and 3.6 mm lenses.  This worked out quite well.  I was able to track positive images (white background with black lines) but not negative images.  Interesting.  The downside to this test was the loss of a fair amount of screen real-estate.  The 3.6m lenses lead to loss of approx. 4" from the perimeter of the screen.  For a taller build this wouldn't be a problem as the cameras could be lower, but with my coffee table and projector choose it is not possible to diminish the screen size any more then I have.  
4. Swapping the 3.6mm lenses for my 2.1mm lenses without moving the camera bodies actually resolved my stitching problem!  With the new placement I now have full screen touch from corner to shiny corner!
5. Shot some video of RPTools and Fiducials in action.  I'll be posting this video as soon as I get a chance to compress it into a reasonable size for sharing (from an HD camera).

Fiducials work -- sort of.

The good news is that fiducials work.  Every method of CCV with fiducial support that I tried worked like a champ.  With some caveats.

The bad news is that the dimensions of the table I've built and the surface area to camera resolution are not going to support fiducials.  This isn't to say that I won't get it working for the table properly, but it does mean that I'll have to run a second instance of CCV or reactivision to handle fiducials using a second set of cameras.

Fortunately I have a couple of HD cams sitting around from an earlier venture into the realm of moding auto focus cameras.  I'll do more testing with these this weekend and hopefully take some photos for those visually minded readers (out of the three of you).

Back in action!

I recently recovered from taking an evening class.  Grabbed the new cameras and dug into the table again.

Success if pretty much how I'd describe it.  Very quickly I was able to get nearly full screen width coverage of touch which was previously unattainable with the lesser camera.  I have two ps3Eye cams now, but only one of them is in use for touch.  My plan is to try to use the second camera for object and shape recognition. 

Using CCV 1.2 with the fiducial tracker I was able to see the shape but the software failed to detect it. 

Will have to tweak this more when I get a moment.  For now?  Sleep!

Camera's abound and vacation

Last month I acquired a second camera and purchased some housings from Peau.  Good stuff but I also picked up about 3 different horrid illnesses (the last of which was the Noro virus) and have done little to advance the project.

I did build up the table surface by roughly an inch and half.  This has resolved some of my focus issues as I have a bit more play room for the InFocus XS1.  Additionally, I started crafting the actual box.  Currently the surface support, i.e. Top, is made of Poplar arranged using lap joints.

I'm still trying to find just the right mirror arrangement to maximize the view on the screen while still maintaining good blobs.  Currently my blobs are quite weak and I'm just not satisfied with that.

I'll post some pictures when I've got the time to scale them for posting.

Continued Refinement

Below is a collection of images of the table running at the improved scale.
The operating resolution is set to 1280x720.  With a max resolution of 1400x1080 or some such.  I've run it at max res and it looks good.  The InFocus XS1 does a good job of scaling resolutions over it's native 1024x768.  It is also worth nothing that the machine I'm using to drive this table is a Toshiba M200 which does not have digital output.  Using DVI I could probably get higher fidelity of image.  Still, I'm not about to start complaining!

Worth noting, the image projected is still blurred due to the double reflection of rear faced mirrors compounded by using two of them.  I'm waiting till I'm completely done testing to attempt the task of removing the protective coating from the rear face of the mirror due to the delicate nature of the then unprotected mirrors.

When in use, I added rigid cardboard along the long edges to minimize the camera seeing a passer by.

Apparently Rosco Grey suffers from hot spots fairly significantly.  Unless viewed straight on, the image is very muted into the far distance.  The photos below actually do not show the level of muting which is visible to the naked eye.

  

  

 

Happy Accidents

As I've stated numerous times in previous posts, mirror placement has been a complete bear.  Well, for various reasons I found myself needing to move the test table temporarily.  This process was slow and involved as the parts are not fastened down in a solid way.

The table moved, cleaning of the space commenced and I hear that there was an accident involving a vacuum cleaner being shoved into the projector side of the case repeatedly.  Egads!

When I got home the signs were there that tampering with the delicate parts of the table had occurred.  The IR Ribbon power cable which I'd reinforced was protruding at a very unattractive, freightening angle.  Fearing the worse I rolled out the support cables and powered the bad boy on.

My first shock was in discovering that the projector wasn't displaying.  Opening the case I was greeted with the sight of the projector resting on it's lense pressed heavily on one of the mirrors! Oh noes!

I gently tilted the projector back into place cursing my not having crafted an upper latch yet, and inspected the mirror for damage.  Since the projector hadn't been on for too long there were no signs of heat damage and there were no new scuffs in the already scuffed section of mirror I've been using for testing.  Whew.

Next I plugged my laptop/control computer back into system and fired it up for real.  The first thing I noticed as an even blur across the image.  Ugh, the projector mishap had messed up my focus.  Grumbling I reopened the case and adjusted the focus.  Close, check, open, adjust, close, check, good.

The second thing of note wasn't apparent until after I'd logged into windows.  The focus was nearly completely even.  I could easily read text in both the near and far height of the screen.  Sweet!

This test was with a different laptop, as I'd left my normal machine at work, I quickly jumped in to the display properties and set it for my normal 16:9 1280x720.  This produced the third most significant discovery.  The screen now contained 98% of the actual display being presented.  Roughly 5 pixels along the bottom edge (start bar) and maybe 2 pixels across the top edge (ui cruft) were clipped by the screen edges!

Tonight I will be spending some, very delicate, time measuring the mirror placements and recording angles.

Finally, I feel confident that I can begin planning and working on the final hardwood coffee table/case.

Here's a pre-party photo of my son playing on the table... literally.

Party Time!

This past weekend we had a party in which the table mock-up was setup.  Fun times!  It survived a number of under 10-year old's as well as their parents.  Thanks to the fire flash demo (the party was fire themed), it was a big success and the various mirror placement problems were not evident.

Multi-Touch Success at last!

Crude but working!  Now it's refinement time, then on to final build.  Decided on Maple for the primary wood for the final box/coffee table last night as well.  Maple for the framing and major portions with Birch ply for the paneling.

 

Screen layering as follows:

=======  10mm Endlighten
-------  Rosco Grey (saggy but working)

 

The camera for this setup is still the GigaWare 640x480 cam with a 160 degree fish-eye lense.  It certainly looses quality at the edges of my screen.  


I've gotten two 720P camera's for use but need to desolder the autofocus elements and remove the ir-filters.  Then solder the focus mechanism back into place -- ugh.

Gaming on the Mock Up

Over last weekend my gaming group actually got a chance to use the mock up in non-touch mode.  At the time I was still waiting on my camera and lense for the setup.  The camera arrived today, so hopefully I'll have something to show with full touch in the next day or so.  

But back to the gaming!  Even in low tech, no special reveal during exploration the group loved using the table.  I was able to quickly illustrate large section of world maps and tactical areas without the players having to guess at what my hasty symbology was going to mean.  The down side is that when running games in the future, I might actually have to prep my maps in advance.  Still, this is absolutely the bed I'm planning to make for myself I better get used to sleeping in it (or not sleeping as the case may be).

The images I've included in this post do not show the actual setup we used.  Silly, tired self didn't take any photos of the game in session.  In the images attached I'm just demoing the screen with paper for a projection surface.  For the session I used a cut of my Rosco grey stapled to the wooden frame.  


For stretching the Rosco I've actually found stapling to be quite effective.  Using a large gauge staple gun staple similarly to stretching canvas.  I left the Rosco on for a 4 days of constant touch and test usage without any snags or distortion in the material.  

Important note:  Rosco grey blocks IR.  Yes this is information that is posted in the nuigroup's forums.  No, I hadn't found it in the forums when I switched from an FTIR to DSI setup.  As to the FTIR effect using the Endlighten + Rosco Grey -- excellent but required much pressure, so much in fact that I was concerned for the safty of the Endlighten.

The Framing

Initially, I designed the table for a minimal number of relfections.  Using one mirror I'd planned on having the measure roughly 36" across.  After creating the mock up inner frame, I decided that while 36" would be ideal for a game table, it was simply too wide for my space.

The final design I've settled on is 26" - 27" wide and 45" long.  This version uses 2 mirrors and has the projector at a jaunty angle.  Currently my mirrors are acrylic rear face jobs which I plan on using a DIY method I found on-line to remove the protective coating.  There is an interesting link in the comments to the article about an alternate product.  I'm probably going to try both depending on what I can get my hands on tonight.  Additionally, nuigroup has a post linking to lumenlab with a procedure very similar to the instructable.

[diagram]

In both cases the total height to projection surface is 19".

This is excluding the convertible I was considering.  A table top which rotated and maintained aspect ratio through magic.  Reality finally caught hold and I had to ditch the idea for its geometric reality was greater then I.

Building the Screen

This first image is of the endlighten, L brackets, Foil tape and IR Ribbon -- exciting!

Here we have the first length of ribbon added to an L bracket.  The depth of the shot really gives you an idea of how long this bar really is.

And put together, no tape yet:

Before taping, I tested the IR with my phone's camera...

I tacked the frame to the screen with small sections of foil tape and reinforced the power connector area from the IR reel with foil tape as well.  The flexibility of the connectors contacts made me rather nervous.  I insulated the connector with small pieces of electrical tape first.  Tacking the frame on allowed me to make sure that all of my edges lined up as I'd hoped.

Once I was satisified with my arrangement I applied a layer of foil tape around the entirty of the frame.

And then I apparently over wrote the image files for the remaining steps I took to create the the prototype I've been using.  How sad...

The story so far

A year ago I started the process of building a DI rig roughly 18"x12".  Nothing every came of this setup.  I tried to build my own illuminators and found that soldering is less fun after the 100th LED.  Especially when it's the placement of that 100th LED which burns out the previous 99.  Rather then invest in a better soldering gun and setting up sort of staging holding rig for the LED's I started the stalling/planning stage of project two.

My next build was a 19"x24" free standing screen using IR Pens crafted loving from old stinky Expo I markers.  This build was great for testing as well as for use as a drafting table in my shop.  I still use the pens as well.

This screen is made up of the following layers:

=======  1/8" Acrylic

-------  Rosco Grey

=======  3/4" Acrylic (buffed and polished for future FTIR use)

Testing the screen without projection met with much success!

Percission was somewhat lacking due to the cheap webcam I was using.  This wasn't the end of the world however as it was a proof of concept.

Then I added projection.  At the time I had (have) an old Sony LCD projector with a _huge_ (read Long) throw distance.  This was nice for projecting 100"+ art installations.  However, it required most of my shop and a large spinning mirror to vaporize... err.. wait, wrong project.  The throw distance requirement was extremely disheartening.  A quick search for front face mirrors led me to the purchase of some mylar-like mirror stuffs.  Very nice, have used in other projects since.  However, I've yet to use it as an actual mirror due to the difficult in stretching the material in such a way as to get a truly flat mirror.  To the credit of the material, I have not tried the clamp method the vendor suggested simply because it would add unacceptable additional size to the projects foot print.  Certainly not usable for a coffee table design.

As a result this table has become a fixture of the basement/shop's space. It's fate is still undetermined.

The difficulties with the Mirrors led me to a bit of a slump.  I still talked about the project and still puttered with it yet I made no real progress.

Finally, this past December (2009), my wife who has been very supportive of the project (read expensive hobby) thus far and perhaps was tired of seeing me moping around, presented me with a flexible budget and a reel of IR ribbon.  Thus was born the coffee table gaming surface which I've begun to create in earnest.

 Price       Product
------------------------------------------------------------------

$240.00  --  1 Spool of 850nm IR Ribbon + powersupply from
               Environmental Lighting
$230.00  --  1 pc. 43"x24" Endlighten Acrylic
$840.00  --  1 Projector, InFocus XS1 Moderate-Short-Throw
               projector 2'2" for a 49" diag according to
               projectorcentral.com
 $20.00  --  1 Gigaware (Radio Shack) WebCam
 $50.00  --  1 Roscoe Grey (oops, I'll get back to that)
  $2.50  --  1 roll of foil tape
  $8.50  --  2 L shaped aluminum 'bars' from Home Depot
-------  --  1 Pile of Scrap wood

Through all of this the nuigroup has been immensely helpful!