The Back Room

A panel of lights being controlled via ethernet

In relation to the line of work we are in, our Flash applications don’t normally fit into the mainstream web application paradigm.  Often we interface with special hardware, touch screens, and systems with more than one monitor in non-standard orientations.  We also have to ensure that our applications can run non-stop for a minimum of a full day and potentially seven days a week.

One of the Many Test Setups

In order to try and simulate the final production environment, when developing software; Bruce and I like to test applications on the hardware it will be run on in situ for several days at a time.  On one corner of the desk that I work at, we have a big three touch screen configuration that we often hook machines into for testing.

Monitoring the Hardware Using a Webcam

Recently while testing large touch screen monitors, the company pool table was occupied for several days with 40-inch vertical monitors and their associated computers.

For situations requiring interaction with hardware such as a telegraph, we will often isolate the piece of hardware in our studio and run it for days while controlling it via the local network.  A cool way of monitoring the hardware being tested in the studio from our office in the backroom is to use a webcam and Red5.  In the images I am using Red5s sample broadcast and subscriber applications.

There is a trackball at each station that is seen by the computer running the Map application as four mice

When I first started working for Boston Productions, we were just starting development on the interactive exhibits for The Hershey Story.  The exhibit called Explore Hershey is a town model of Hershey, PA that features four user stations that have to be able to simultaneously interact with the map of the town.  This means that somehow we had to allow four mice to simultaneously interact with one Flash application.

After a large survey of how people were handling multi-mouse applications in all areas of programming, I found out academia actually calls using multiple-mice “SDG” or Single Display Groupware.  A decent amount of work is indirectly going into using multiple mice right now due to the proliferation of multiple touch interactive displays.

Flash cannot do multiple mouse interaction on its own.  It needs help from another program that has more direct ties to the underlying hardware of the system you are using.
Based on the survey of currently available SDG applications, the most usable starting point with the best documentation I came across was the University of Calgary’s SDGToolkit.  The SDGToolkit can be integrated into many of the Microsoft programming languages as a COM Object which basically allows you to have multiple mouse applications running in a Windows environment in a matter of minutes.  I chose to use Microsoft Visual C# to write the SDG application that is in use at Hershey.

So how do we use multiple mice in flash then?

There are four stations and four projection screens. In total five computers run this interactive.

The secret is that you need to have the SDGToolkit based application handle all interaction with the mice and send that to the Flash application.  Another option is that you can actually embed a flash application inside a .NET based forms application using another COM Component called the “Macromedia Flash Factory Object”.  For our situation, we decided this would not be a good option because we had to run the Flash Application full screen.

The way chosen to send commands from the .NET application to Flash was through sockets.  I crafted the .NET application in such a way that it would use threads to easily handle multiple connects and disconnects from the Flash application.  A great starting point for multi-threaded socket programming in .NET is the Multithreaded Chat Server at the CodeProject by member Sidzone. The commands I send over the socket connection to flash several times a second as a string look like this: <*M0,337,-142,1,*M1,209,198,0,*M2,0,0,0,*M3,0,0,0,*>.  They report the screen x,y coordinates of each mouse.  The third number is a 0 for mouse not pressed and 1 for mouse pressed.

In the Flash Application, I used the Socket class in order to connect to and receive commands from the .NET application as a comma delimited string.  I decided not to use the XMLSocket class because the extra steps needed by both sides in setup seemed onerous for the utilitarian string I would be sending to flash.  Once the command string is received by flash, I parse it, then use events to pass it through a series of classes which allow the commands to be crafted into targets moving around on the stage and interacting with buildings.

When I started developing interactive exhibits for museums and other institutions, everything was done in Macromedia Director. Flash was there, but it only could do some animations and only very light interaction. Director could do it all, connect to printers, interact with databases, play video, etc. So, for years it was all Director, all the time. But, a few years ago, we came to the realization that Flash had grown enough to start using it for some exhibits, and it actually became necessary for others, where the program might need to be put online as well. Sure, Director can publish to Shockwave, but the installed base is much lower than Flash, and it came with a few bumps, like a mandatory form to fill out on Macromedia’s site with contact information. Flash had a seamless installation and an installed base of over 98% of all web users. Director had failed to see an update in years, and was starting to lose the developers who helped make it the powerhouse it was. So, when I got a project that needed database integration that might need to also live online, I made the switch, and I have really never gone back.

Sure, Director is better at some stuff than Flash. Without making this all about a side by side comparison, it integrates with the operating systems better, allowing easy file writing, connections to external devices (something we do often) and more options on how it displays itself on a system. We solved that here by using small pieces of software to allow Flash to connect to what we need it to. For instance, a recent installation included multiple mice, a serial connection to the museum control system, and a connection to a network server to allow communication among 5 systems. Director was not involved at all, and the system works very well. The overall performance and graphics rendering is better than what I believe we would have gotten with Director.

Director has just come out with its latest version, 11.5. It does add some new features, but unfortunately does not deal with many of the things that cause us to move away from it in the first place. First of all, it has no support for ActionScript 3.0. This is what we write all of our Flash projects in, and is a necessity if we wanted to integrate some of those components into a project. It also maintains a layering system that when using multiple layers of buttons and graphics, becomes unwieldy. There is nominal improvement in graphics playback, but many of the Xtra developers have stopped developing plug-ins because of the general lack of interest in the software. It is good for some 3-D game development, as well as supporting direct use of a number of different media types like QuickTime, WAV files, Mpegs, etc. Flash is fairly limited in this regard to a few media types that it can handle and many file types must be converted to be used. Flash recently introduced support for .h264 encoded video, which we use for most of our encoding now, so that has been a big plus.

Using Flash for the bulk of our programs allows us to post interactive exhibits for easy previewing in the browser, to post online versions of exhibits much quicker and to a broader audience, and to develop, if we need to, completely open source with Flex and other available tools. I have been happy with what we’ve produced for the industry with Flash, and particularly happy with how we’ve been able to integrate with a number of pieces of software and hardware. The introduction of Adobe AIR development has added a lot of tools that Director brought to the table. You can write files, have embedded web browsers, control some interaction with the operating system, etc.

Overall, we still use Director occasionally, but it looks like the development of it as a software platform is gravitating more towards gaming and 3D. For the foreseeable future, we’ll be using Flash as the front end for the bulk of our programs, but it’s all about the user experience, so if there is something that comes along that helps us improve that, we’ll take a look at it.

Side picture of Bunnell telegraph in the Boston Productions studio.

Recently I have had the pleasure of getting to work with an authentic working telegraph from the early 1900s. From what I understand this is a Bunnell railroad telegraph key and sounder. We purchased it for a project that is underway in New York.

Testing the sounder around the office we have simply been using a 9v battery. However, eventually this will need to be driven by a microcontroller. Until I was able to experiment with the apparatus, I had been trying to guess how much amperage it takes to cause the coils to generate the magnetism needed for the sounder to work. The actual wood that the key and sounder are mounted on states “4 ohms”. After testing and poking around a little bit, I think this model requires about 350mA to create the electromagnetic effect and produce a sound.

Vertical view of a Bunnell Railroad Telegraph.

Currently we are using the MakingThings Make controller coupled with a high watt rated 10ohm resistor in order to do some initial stress testing. The Make Controller’s output drivers can source a maximum of 1 amp. I currently have the outputs set to 5v. If you combine the resistance written on the wood of the telegraph with the resistor (10 + 4 = 14), you get 14ohms total resistance. Use Ohms law 14ohms/5v and you get .357A or about 350mA.

The ultimate goal is to have the telegraph activate upon the sense of motion. For this purpose we are planning on using the popular Arduino, but I am going to have to make a more powerful driver shield for it in order to source the current needed to run the telegraph.