The Back Room

We have a Flash application that takes an input from a bar code scanner via serial keys in windows. Each time a ticket is read by the scanner, it’s translated into a keyboard input, and that is sent to the Flash application. The application can then analyze the string of characters, validate them through php with a Microsoft SQL database, and return whatever info we need. It’s a good example of integrating Flash with an external input device – the scanner. Although the scanner we’re using requires a few pieces of software to send keyboard command (because of the way it works with the touchscreen), most just emulate a keyboard input when an item is scanned.

Because what we’re making has to work hundreds of times a day, every day, we needed a way to do some solid stress testing. It wasn’t practical to have one of us sit in front of a computer with a ticket and scan it every 5 seconds to make sure that our application and the hardware were functioning properly. So, Charles came up with the idea to trigger a small servo motor to scan a ticket every XX number of seconds. In about two hours, he had the device you see below. It is pretty utilitarian, but  the main function is to just make sure the ticket scans properly, so it doesn’t have to be too sexy. The program is actually a modified version of the servo motor example that ships with the Make Controller that we have used in a few exhibits. The controller is pretty handy for connecting Flash to all sorts of external devices.

We’re not showing the actual program we’re testing, but instead you can see the input from the scanner appear on notepad. It helped us assure that our software could take the continuous input of thousands of scans a day and still be functioning well.

Interactive Developer Job Opening

About Boston Productions, Inc
Boston Productions is a multimedia production company that produces videos, interactive programs, and immersive theater experiences for museums across the United States. We are looking for an experienced interactive developer to join our staff. Recent clients include: Connecticut Science Center, Hershey Chocolate Company, Smithsonian Institution, College Basketball Hall of Fame, and the International Spy Museum.

Description of the Position
Our work combines Flash, video, content management, and occasionally giant chocolate machines. We’re looking for someone to join our team with a broad range of technical abilities. This is someone who is comfortable with PHP and SQL programming, but also can get their hands dirty with Actionscript and Flash development.

Here are some examples of some of the challenging projects we’ve done:

–a theatrical experience that let’s up to 32 people use touch screens to interact with a 3 screen immersive video projection with lighting and atmospheric effects

–a multi-user map table that allows up to four users to explore the history of the town of Hershey through photos, video and other rich media.

–an interactive presidential desk that teaches visitors about Teddy Roosevelt’s policy decisions and allows them to photograph and email an image of themselves as president.

As an interactive developer, you will program touch-screen applications, websites, and other multimedia programs.  You’ll also be key in building custom applications to allow institutions to manage their own content and allow the programs to grow over time. As a valued member of the production team you will participate in creative development, programming, prototyping software and hardware, and ensuring proper QC of deliverables.

Qualifications
The interactive developer should have a broad knowledge of interactive technologies and in-depth software knowledge in Flash, Dreamweaver, and Photoshop and in technologies like XML, ActionScript, PHP, SQL, CSS, and HTML.

Helpful skills include familiarity with video encoding for the web and stand-alone applications, the ability to diagnose and solve computer problems, and the poise to deal with problems you may have never heard of before. We realize there are a lot of Flash animators and designers out there, as well as pure PHP coders, but this position is for a person who can work with both, and integrate them to create seamless experiences.

Excellent communication skills and the ability to collaborate are extremely important as a project team may include a producer, video editor, graphic designer, scriptwriter, and the interactive developer.

We don’t need you to know it all, but an adventurous spirit is important.  We take on a lot of challenging projects that are different than anything we’ve done before, so if you like sticking to just web development, or don’t feel comfortable with integrating graphics and Actionscript, then this probably isn’t the job for you.

Please email your resume, cover letter explaining why you’d be a great addition to our team, and your favorite candy bar to Bruce Spero at bspero @t bostonproductions d0t com. No phone calls about this job please. We’re NOT looking to contract out work to other companies or development firms. We’re looking for an individual to work out of our Norwood, MA office, so please make sure this is a possibility before applying.

One of the ever increasing trends for visitor centers and museums, especially in the public sector, has been the inclusion of open source software and hardware into their exhibits, user stations, and infrastructure.  At face value, it seems like open hardware and software systems should be considered over closed hands-down.  There are a few high level considerations to take into account when making the decision to utilize a certain open source project.

What is open source software?
Open source software means that the original source code for a software product or tool is made available.  An open source software project may or may not have a corporate entity backing it.  Traditionally open source software is volunteer based unless there is a corporate entity actively developing portions of a project.

An example of a corporate backed open source software project is Jive Software’s Openfire Server for XMPP (an instant messaging protocol).

An example of a volunteer based project is the multimedia encoder/decoder/transcoder ffmpeg.

Open source software usually does not provide typical avenues for support.  At a bare minimum there will be a developer’s mailing list that may or may not have active participants who can provide helpful support.  My recommendation is that the amount of documentation for a project should be carefully examined prior to becoming involved with it.

What is open source hardware?
In this context I am referring to open source hardware as circuit boards.   So this may include special electrical converters that allow an exhibit to do external communication, control something remotely, turn on and off devices and lighting, and several other operations dealing with management of electricity.

Open source hardware means that electrical schematics, part lists, gerber files(standard circuit board fabrication CAD file format) and potentially low level code are made available with hardware.  Basically all of the necessary items are provided in open source hardware so that anyone with the ability to fabricate circuit boards could produce and potentially sell or use the piece of hardware.  A very popular piece of open source hardware is a little circuit board for hobbyists, students, and professionals known as the Arduino.

The Sparkfun Electronics product page for Arduino shows the power of open hardware as there are several variants and accessories of the popular kit.

Important Questions You Should Ask When Considering Open Software and Hardware
1.  Will the open source software or hardware work for you as-is?
If you decide that you would like to expand or modify an open source project, then make sure you have the manpower and expertise to do so.

2. If modifying an open source project; would it be faster to do your own implementation?
If an engineer is not familiar with an already implemented solution, then there is an additional cost of time for the engineer to learn how to modify the open source project.  Many implementations in software are built on strict templating systems that an engineer will have to figure out how to work with.

3. Does the licensing for the project fit your intended use case?
There are many different open source licenses that all come with their own restrictions.  Some allow you to do whatever you want.  Some place strict guidelines on what is and isn’t allowed.  Some require you to release your source code/schematic changes if they are requested.

Recently, while creating a content editor I had a problem setting the visible property of components. Since much of the content editor’s functionality is dependent on previously entered content, I had the need to hide many of these components at certain stages of the application. I needed my editor to start by only showing a few options, while hiding most of the other components on the stage. I added a function in the constructor of my document class that would set the .visible property of these components to false. This did not work.

After a few minutes of scratching my head I came to the conclusion that the .visible = false commands I was giving were being overwritten by something else. I then remembered a workaround I have used in the past when I had needed to determine the size of the stage upon loading the application. The trick was to wait for the document class to fully instantiate itself before the stage properties become available. The same was the case with the components. Flash was rendering my .visible = false in the constructor, but since the document class hadn’t fully instantiated itself, the components would then reset to their default settings, as seen in the component inspector, upon loading complete.

The Solution:

Listen for the loaderInfo Complete event in the constructor and then have it call an initialization function.

public function MyDocClassConstructor():void
{
// listen for initial load to insure settings are active to set into global vars
loaderInfo.addEventListener(Event.COMPLETE, init);
}

private function init(e:Event):void
{
// remove the listener
loaderInfo.addEventListener(Event.COMPLETE, init);

// set your components visibility to false
myComponentBtn.visible = false;
}

By doing this you allow the application to complete loading before you start modifying attributes of your display objects.

My name is Dave Nelson, and I am “the new guy” here at Boston Productions filling the role of interactive developer. Over the past few months, I have begun to adjust my methodology to be more in tune with developing compiled .exe files used in standalone kiosks. I thought in my first entry for the Back Room, I could share some of the simple differences between scripting for the Web versus a kiosk.

LOADING THE APPLICATION

One of the first and most obvious differences between the two is that many of our interactive programs are not meant to be viewed in an Internet browser. This instantaneously removes the need to detect bandwidth, create loading segments and develop for a full gamut of hardware and software capabilities. I now have the ability to develop on the machine that will be running the program, so I can observe the same experience that any end-user will encounter.

On the subject of loading, it’s possible to include more graphical assets in the Flash library, with less of a need to load graphics “on the fly.” Typically, in a Web application, I would try to get the .swf to load as quickly as possible and then load further assets as needed. This begets multiple loading screens. Oftentimes when developing for the web, it is better to have multiple short-loading segments, as opposed to a slow screen on the initial load. Many of today’s Web users will not wait very long and may get frustrated, ultimately navigating away from the site.

Typically, museum kiosks are started up once a day in the morning, before the museum opens. This means that despite a longer loading time, the compiled .exe file has to load only once a day, when no one is waiting for immediate information. During this initial load, the major graphical assets are loaded and will be available throughout the day; however, it is important to ensure your program is efficient and can run for a full day without excess scripts building on top of one another.

ADJUSTING TO TOUCHSCREENS

Another large difference is the use of touch-screens versus a computer mouse and keyboard. Of course, touch-screens have no “hover state,” and any graphical change or animation that occurs on a mouse event is typically associated with a “MOUSE_DOWN” listener. The “MOUSE_DOWN” event will typically trigger a highlighted state, while a “MOUSE_CLICK” listener will trigger the action derived from clicking on the interactive interface.

Before starting here, I didn’t imagine how programming for kiosks would be different from a Web application. As I continue developing interactive programming with Boston Productions, I look forward to contributing more to the Back Room.
Dave

 

In a recent project we worked on; the client was very adamant about being able to display a plethora of different types of unstructured data inside of a kiosk.  This included information that contained tables and images inside of the same display at the same time.  The content also included a plethora of font variations.  The length of content was greatly variable and really didn’t have any defined or repeatable structure.

If you are familiar with the flash text fields htmlText capabilities, you will see that it can render basic html.  However, this just wasn’t enough for this project so we decided to implement the Adobe AIR HtmlLoader which; from my understanding runs an instance of webkit inside of AIR.  It’s awesome being able to load web content directly into a project; but there is one huge problem with the HtmlLoader.  You can’t smooth the HtmlLoader!  If you rotate or resize the loader, you will quickly find yourself with an ugly, poorly rendered mess.

After thinking about the issue for a while, I came up with a solution.  The trick is to load the web content inside of the HtmlLoader, then use the BitmapData, and Bitmap classes to render out a bitmap of the HtmlLoader.  You will then be able to apply smoothing, size, and rotate the newly created bitmap without much distortion.

When using this technique you lose the ability to scroll and navigate web content.  Our whole point was to not let people know there was a web page being displayed inside the kiosk; so this methodology worked very well for us.
Below I have provided the class that I use in order to implement the smooth html loader functionality.  Be careful because my class is using some external references.  Namely Tweener as well as GenericNotifyEvent.

package com.bostonproductions.web {

import flash.display.MovieClip;
import flash.display.Sprite;
import flash.display.Bitmap;
import flash.display.BitmapData;
import flash.events.Event;
import flash.html.HTMLLoader;
import flash.events.HTMLUncaughtScriptExceptionEvent;
import flash.geom.Matrix;

import com.bostonproductions.events.GenericNotifyEvent;
import com.caurina.transitions.Tweener;

public class smoothHtmlLoader extends MovieClip {

public static var HTMLREADY:String = “smoothHtmlLoader_HTMLREADY”;
private var theBrowser:HTMLLoader;
private var htmlToRender:String;
private var smoothCap:Sprite;
private var rawSmoothCap:Bitmap;
private var scrollPosition:int = 0;
private var tryCenter:Boolean = false;

private var targetHeight:Number = 650;
private var targetWidth:Number = 720;

// Alter the native height based on the resize ratio
private var HeightResizeRatio:Number = 1;

public function smoothHtmlLoader(_HeightResizeRatio:Number = 1, _targetWidth:Number = 0, _targetHeight:Number = 0):void {
HeightResizeRatio = _HeightResizeRatio;

if (_targetWidth != 0) {
targetWidth = _targetWidth;
}

if (_targetHeight != 0) {
targetHeight = _targetHeight;
}

createHtmlViewer();
}

/*
* DATE: 10/14/2009
* AUTHOR: Charles Palen
* NAME: displayHtml
* DESCRIPTION: Tries to fit all the html into the instance of
* the HTMLLoader.  If it cant fit it all, it MUST return the
* non-fitting html.
*/
public function displayHtml(_HtmlToDisplay:String):String {
var nonFittingHtml:String = “”;

htmlToRender = _HtmlToDisplay;

loadTargetLocation();

return nonFittingHtml;
}

/*
* DATE: 10/15/2009
* UPDATE: 10/28/2009
* The html parser now splits html up into chunks in instances where there are tables and images
* If this occurs – especially in the case of a table – we want to be able to CENTER it.
*
* Added optional boolean to specify we want to examine the content height, and resize/center the browser
* based on it!
*
* AUTHOR: Charles Palen
* NAME: setScrollPosition
* DESCRIPTION: Stores what should be the vertical scroll for this.  Once the html content loads it can be set.
*/
public function setScrollPosition(_VerticalScroll:int = 0, _TryCenterBrowser:Boolean = false):void {
scrollPosition = _VerticalScroll;
tryCenter = _TryCenterBrowser;
}

private function createHtmlViewer():void {
theBrowser = new HTMLLoader();
theBrowser.width = targetWidth;
//theBrowser.height = targetHeight;

theBrowser.placeLoadStringContentInApplicationSandbox = true;

theBrowser.addEventListener(Event.COMPLETE, htmlLoaded, false, 0, true);
theBrowser.addEventListener(HTMLUncaughtScriptExceptionEvent.UNCAUGHT_SCRIPT_EXCEPTION, scriptException, false, 0, true);
theBrowser.addEventListener(Event.LOCATION_CHANGE, htmlClicked, false, 0, true);

theBrowser.paintsDefaultBackground = false;
}

private function destroyHtmlViewer():void {
if (theBrowser != null) {
theBrowser.removeEventListener(Event.COMPLETE, htmlLoaded);
theBrowser.removeEventListener(HTMLUncaughtScriptExceptionEvent.UNCAUGHT_SCRIPT_EXCEPTION, scriptException);
theBrowser.removeEventListener(Event.LOCATION_CHANGE, htmlClicked);

theBrowser.cancelLoad();

if (contains(theBrowser) ) {
removeChild(theBrowser);
}

theBrowser = null;
}
}

private function loadTargetLocation():void {
if( theBrowser != null ) {
theBrowser.loadString(htmlToRender);

}
}

private function htmlLoaded(e:Event) {
// Remove the event
e.currentTarget.removeEventListener(Event.COMPLETE, htmlLoaded);

// Set the vertical scroll
theBrowser.scrollV = scrollPosition;
// If the try resize boolean is set – lets try it!
// Dont center – just shrink it if possible
if (tryCenter == true) {
var resizedHeight:Number = theBrowser.contentHeight;
var resizedWidth:Number = theBrowser.contentWidth;

if (resizedHeight > targetHeight) {
resizedHeight = targetHeight;
}
if (resizedWidth > targetWidth) {
resizedWidth = targetWidth;
}

theBrowser.width = resizedWidth;
theBrowser.height = resizedHeight;
targetWidth = resizedWidth;
targetHeight = resizedHeight;
}

theBrowser.width = targetWidth;
theBrowser.height = targetHeight * HeightResizeRatio;
//var bitmapAsset:Bitmap = e.target.content as Bitmap;
//bitmapAsset.smoothing = true;
Tweener.addTween(this, { alpha:1, time:.1, onComplete:generateSmoothVersion } );
}

private function generateSmoothVersion():void {
// Create the smoothed version
var tmpImage:BitmapData = new BitmapData(targetWidth, targetHeight, true,0×000000);
// Draw it – the last parameter set to true is SMOOTHING – tmpImage will be smoothed
var transformMatrix:Matrix = new Matrix();
//transformMatrix.translate(0, 500);
tmpImage.draw(theBrowser, null, null, null, null, true);
//tmpImage.draw(theBrowser,
// Create the bitmap
rawSmoothCap = new Bitmap(tmpImage);
rawSmoothCap.smoothing = true;
smoothCap = new Sprite();
smoothCap.addChild(rawSmoothCap);
//rawSmoothCap.x = 50;
//rawSmoothCap.y = 50;
addChild(smoothCap);
// Destroy the heavy html components to free up some memory
destroyHtmlViewer();
//swapChildren(theBrowser, smoothCap);
//theBrowser.alpha = 1;
//smoothCap.x = 300;
//smoothCap.y = 100;
// Broadcast so that we know its ready
if(tryCenter == true) {
dispatchEvent(new GenericNotifyEvent(smoothHtmlLoader.HTMLREADY, false, true, true));
}
}

private function destroySmoothVersion():void {
if (smoothCap != null) {
if (rawSmoothCap != null) {
if (smoothCap.contains(rawSmoothCap) ) {
smoothCap.removeChild(rawSmoothCap);
}

rawSmoothCap = null;
}

if ( contains(smoothCap) ) {
removeChild(smoothCap);
}

smoothCap = null;
}
}

private function htmlClicked(e:Event):void {
// Auto go back to wherever we were
//theBrowser.reload();
//loadTargetLocation();
}

private function scriptException(e:HTMLUncaughtScriptExceptionEvent):void {
trace(”ScriptException:” + e);
}

public function destroyInternals():void {
destroyHtmlViewer();
destroySmoothVersion();
}
}
}

For roughly the past two years there has been an assortment of articles flowing through the technical gadget pipelines regarding innovative and consumer grade touch screen technology.  Traditionally, touch technology hasn’t been ready for mainstream despite being used heavily in museum and kiosk applications.

Although we could all probably argue the value of a touch screen on a traditional home or office PC; the HP Touchsmart line of multi touch consumer grade machines is something to watch out for from a museum and visitor center perspective.  Will this line of PC’s be the freight truck that delivers touch screen technology to the masses?

My belief is that if this line of consumer grade products doesn’t make touch technology ready for mainstream then something else will.  There has already been a plethora of multi touch mobile devices making the rounds lately.

The bottom line is that we all need to get more creative in the museum and visitor center industry when it comes to interfaces.  The touch screen has been a work horse and novelty over the past several years that we have relied on to give kiosks a feeling of freshness and uniqueness.  This paradigm only has at most a few years left before the novelty effect will be gone.  Without the touch screen; where does that leave us?

Unique Interfaces

There are several sources of information available that you can find about unique interfaces scattered throughout popular and technology publishing outlets.  The New York Times has an article about possibilities called: “Coming Soon: Nothing Between You and Your Machine” by John Markoff.  Engadget has a recent article on a muscle interface in “Microsoft researchers build muscle-computer interface, play air Guitar Hero.  Tech crunch has the “CeBIT Highlight: University of Bremen’s Brain-Computer Interface”.  These are all very cool technologies, but are they applicable for museums and visitor centers?  Probably not.

The real challenge in interface design for museum and visitor center use is the immense age group and user diversity that must be supported.  Unlike many successful technology products which can target users in a specific demographic or age group; we must do our best to support all users!

One of the strategies we are employing at the National Museum of the Pacific War is a server/client content synchronization method.  The battle kiosks have so much information that it is unrealistic should we, or the institution need to update them; to perform the updates on each individual machine.

In order to remedy this, we are using Microsoft SyncToy along with scheduled tasks on each kiosk machine to synchronize their content.  Due to the fact that each time a page is loaded on the exhibit, it is parsed from HTML, you can update the html templates in the background and changes are seen instantly on the kiosks on the museum floor.  It is really a neat solution that works well by simply having each client machine able to see the master network folder on the server.

Around the office we have also been using Microsoft SyncToy to backup our development machines to a Drobo that we have on our office network.  Sync Toy is a free utility for people using Windows XP and higher.  You need at least the .NET 2.0 framework in order to use Microsoft Sync Toy.

A large percentage of Boston Productions has been working on the National Museum of the Pacific War.  The facility opened to the public on December 7^th, 2009 with a ribbon cutting ceremony by former president George H.W. Bush.  There was a special preview event for veterans the Saturday prior to the official opening.  Many of them had a difficult time completing the tour due to the emotional impact of some of the media.

There are roughly eight kiosks in the institution that Bruce, myself, and our production staff worked on.  Boston Productions also worked on a large portion of the video and audio media for the facility.  The kiosks have been added to the Boston Productions Interactive Exhibit key.

The over-all theme that I heard from military personal, veterans, museum staff, and donors while on-site; is that the facility will help convey the severe cost of war and sacrifice of our previous generations.

If you’ve never developed a touch screen application before, there’s a lot to think about that makes it different from a browser based or native application. It’s impossible to go through the full breadth of things to consider, but I figured posting a few helpful tips might be good.

Well Spaced and Big Button Sensing Areas
So, this does not necessarily mean giant buttons. Often that’s a good solution, since museum visitors are not just those familiar with computers but include many folks who don’t interact daily with computers. Really though, it’s about making sure that the sensing areas of buttons are forgiving enough to work with clumsy fingers, and to account for the different alignment people will get from looking at the screen from different angles. If you touch a screen from a low angle on what you see as the button area, and then change your perspective, that button area can be drastically different from above. A mouse is absolute. The button is the button. But with a fingers on a large screen, it can actually chance. So, large sensing areas for the buttons, and don’t put them too close together to account for different perspectives and clumsy fingers.

Consider Hand Position
When using a touch screen, a person uses their hand, thus creating a natural block for some of the screen area. With a mouse pointer, it’s small enough that unless an interface is very tiny, it won’t block anything when using it to select menu items, etc. But, with a finger, you’ve got a whole hand attached. So, imagine you have a screen where you have 5 buttons. Each button makes a change to a large image in the center of the screen. Although many web pages have navigation and other items at the top, this makes little sense in our scenario, because you want to see how your selection is changing the screen. If the buttons are at the top, you’ll have to move your hand away each time you press one to see the results. It may make more sense to place buttons along the bottom, or the right side. Although it may seem biased, you have to consider that most people are right handed, and if you have to favor one or the other, then choose right. The main thing to get from this one is just to think about where people’s hands are going to be and if they’ll be blocking items you need them to see when operating the screen.

Choose the Best Viewing Angle
This has more to do with fabrication than the software programming. You should make sure you consider who the audience is when choosing what angle and height to put the screen at. It seems like a basic enough idea, but there are many times when we haven’t been involved in the hardware or installation decisions of an institution, and they’ve installed a touch screen for children flat on a table that’s 36 inches high. How can a kid see that properly without standing on something? Or, a kiosk is supposed to take a photo of a person to put on postcard and email home, yet the camera is pointing almost at the floor, so if you’re over 3 feet tall, you’re bending awkwardly and contorting to get in the picture. Little things like this really affect the usability of a screen.

Hide the Mouse
I really don’t understand museum exhibits that have mice showing. I think people leave it there so that you see where you’re clicking and navigating. The thing is, you don’t want people to walk up to a touchscreen and see it as just a computer program running. You want them to walk up and see it as an experience that is something they can’t get at home. You can click on buttons at home, but you can’t touch a screen and move elements around the same way you can on a touch screen. Having the mouse pointer on screen breaks up your interface and reminds people they’re just looking at a computer screen.

Give Clear Direction
An interface must give users clear direction about what to do. Children will walk up to an exhibit and start banging away on the screen and they’ll figure it out. It’s the visitors that are new to touch screens and computer interactive programs that will just stand and stare at the screen, not sure what to do next. Things need to be clearly defined, and text must be readable. Sure, many people won’t read the directions, not matter how much you highlight them, but they need to be able to understand what to do by visual clues. There is a lot that can be written about creating touch screen interfaces, but my overall comment here is make sure people understand what to do.

I’m open to suggestions here, and I am not saying my comments are the be-all and end-all, but we’ve certainly learned a lot from working in this area for years. Some other miscellaneous items to consider:

- There are no rollOver or rollOut commands. Tooltips, highlights, etc. won’t work since it’s just about single touches.
- It’s good to use high contrast colors to make sure it’s visible to as many people as possible
- Choose a type of touch screen technology that works for what you need. (Capacitive, Resistive, SAW, etc.)

Anyone else have any tips?