The Back Room

There is a trackball at each station that is seen by the computer running the Map application as four mice

When I first started working for Boston Productions, we were just starting development on the interactive exhibits for The Hershey Story.  The exhibit called Explore Hershey is a town model of Hershey, PA that features four user stations that have to be able to simultaneously interact with the map of the town.  This means that somehow we had to allow four mice to simultaneously interact with one Flash application.

After a large survey of how people were handling multi-mouse applications in all areas of programming, I found out academia actually calls using multiple-mice “SDG” or Single Display Groupware.  A decent amount of work is indirectly going into using multiple mice right now due to the proliferation of multiple touch interactive displays.

Flash cannot do multiple mouse interaction on its own.  It needs help from another program that has more direct ties to the underlying hardware of the system you are using.
Based on the survey of currently available SDG applications, the most usable starting point with the best documentation I came across was the University of Calgary’s SDGToolkit.  The SDGToolkit can be integrated into many of the Microsoft programming languages as a COM Object which basically allows you to have multiple mouse applications running in a Windows environment in a matter of minutes.  I chose to use Microsoft Visual C# to write the SDG application that is in use at Hershey.

So how do we use multiple mice in flash then?

There are four stations and four projection screens. In total five computers run this interactive.

The secret is that you need to have the SDGToolkit based application handle all interaction with the mice and send that to the Flash application.  Another option is that you can actually embed a flash application inside a .NET based forms application using another COM Component called the “Macromedia Flash Factory Object”.  For our situation, we decided this would not be a good option because we had to run the Flash Application full screen.

The way chosen to send commands from the .NET application to Flash was through sockets.  I crafted the .NET application in such a way that it would use threads to easily handle multiple connects and disconnects from the Flash application.  A great starting point for multi-threaded socket programming in .NET is the Multithreaded Chat Server at the CodeProject by member Sidzone. The commands I send over the socket connection to flash several times a second as a string look like this: <*M0,337,-142,1,*M1,209,198,0,*M2,0,0,0,*M3,0,0,0,*>.  They report the screen x,y coordinates of each mouse.  The third number is a 0 for mouse not pressed and 1 for mouse pressed.

In the Flash Application, I used the Socket class in order to connect to and receive commands from the .NET application as a comma delimited string.  I decided not to use the XMLSocket class because the extra steps needed by both sides in setup seemed onerous for the utilitarian string I would be sending to flash.  Once the command string is received by flash, I parse it, then use events to pass it through a series of classes which allow the commands to be crafted into targets moving around on the stage and interacting with buildings.