next up previous
Next: 4.2 Software Structure Up: 4. WEBeye: A Case Study Previous: 4. WEBeye: A Case Study

Subsections


  
4.1 What WEBeye Is

The purpose of WEBeye is to allow browser users to aim and zoom a videocamera and simultaneously view the continuous stream of images being captured. The user can control the camera by clicking or dragging a pointer on either the image itself or small widgets adjacent to it.

The zoom widget is a slider with a red bar that follows the user's pointing actions and a blue bar that shows the amount of zoom actually achieved by the camera. The blue bar therefore tends to follow the red one at a rate constrained by the speed of the zoom servo-motor. For the Canon VC-C3 [35], the zoom factor ranges from 1 to 10.

The pan/tilt widget is the two-dimensional analogue of the one-dimensional zoom slider, and for the VC-C3 hardware has a range of -90 to +90 degrees in azimuth (pan) and -30 to +25 degrees in elevation (tilt), depicted on a flat grid. A red rectangle shows the requested position in pan/tilt space, and can be dragged or moved quickly by clicking, while a blue rectangle shows the progress of the hardware in moving towards the current goal. Additionally, the size of the current field of view is reflected in the size of the red and blue rectangles. This size is an inverse function of the zoom factor. More precisely, the width and height of the rectangles show directly on the grid how many degrees of azimuth and elevation respectively are subtended by the camera's current view, so that the user has an immediate indication of what part of pan/tilt space is being viewed.

The widgets for the real-time streaming image, the zoomer, and the pan/tilt control are simple Java applets which receive events from the pointing device (a mouse or equivalent) and from the WEBeye Box. Each applet maintains a rudimentary real-time graphical display and also sends events derived from user actions back to the Box.

Image production and camera control are managed by the Box, which is the ``server side'' of WEBeye. It notifies all clients of changes in the hardware parameters, and accepts requests to change those parameters. The Box also supplies clients with the processed image stream from the camera, which is independent of the camera control and motion event streams.

The Box services are provided through TCP streams. Some are designed for the direct convenience of Java clients, while others are more general in intent. Every service, except for those which supply compressed image data embedded in MIME documents, can be tested and learned about using the standard text-based telnet client. For example, the most comprehensive camera control service, which supports many commands, includes a help directive.

Besides Java clients, the Box supports browsers such as Netscape in a more primitive way by delivering JPEG images in either a streaming (``server-push'') or snapshot mode. It also accepts camera control commands implied by ``imagemap'' clicks.

The httpd service provided by the vc-httpd.setl program listed in Section A.19 [vc-httpd.setl] responds to HTTP requests as they might be generated by a browser, and instantiates a template document with port numbers and other parameters, such that the resulting document presents an imagemap with the live video playing in it. The hyperlink associated with the imagemap is a reference back to the httpd service, so that a ``click'' within the map causes camera motion and a new document instantiation. This is the only service in the Box that has both a video and a camera (motor) control aspect.

  
4.1.1 Video Services

From the client's point of view, the simplest video service is the ``snapshot'' service, snap. It sends a MIME-encapsulated JPEG image in response to any HTTP-protocol GET or POST request. A browser will always translate any URL beginning with the server's host name and port number into such a request, and display the resulting JPEG image as it is received. A non-browser client such as wget can use snap to take pictures periodically for archival, and Java clients use it to fetch images based on URLs in the same fashion as browsers.

Closely related to the snapshot service is the image-stream service, push. This is intended for browsers which support Netscape's ``server-push'' method of playing a continuous sequence of JPEG or GIF images contained in an indefinitely long multi-part MIME document, and the httpd service just mentioned generates a reference to this service in the document it instantiates. For the sake of clients whose slow network connections could cause the buildup of images all along the route from server to client, the URL carrying this reference supports an optional rate parameter, measured (perversely) in milliseconds, which sets a minimum time between image transmissions. This rate can be included in a URL as rate=integer after a question mark, in the manner of parameters supplied by browser-based ``forms'' to Web servers.

Unfortunately, the only industry-wide standard for image streaming in browsers appears to be the one proposed for version 1.2 of the Java API, and most popular browsers at the time of this writing have only recently caught up with version 1.1 in terms of their built-in support and bundled Java classes.

However, a third ``video'' service in the WEBeye Box, giver, is offered as a transitional workaround to deal with this problem, which afflicts Java applets in both major browsers. A natural interface for Java clients is to have a simple command/response handshake over a sustained network connection. Each time the client requests another image by sending the command JPEG, the server replies with the latest image as soon as it has one that is different from the last one it sent that client. The server side of this has already been implemented, because just such a protocol is used by a server within the Box. The client side, if Java is used, is easy to implement in the 1.2 API version but practically infeasible with earlier versions. So, to make it as convenient as possible to adopt the new API when it becomes available and still provide a working (if suboptimal) service meanwhile, the giver service accepts JPEG commands, but replies to each one not with an image but with a URL that the client can then use (even in the 1.0 API) to fetch the JPEG image itself. Each URL that this rather trivial service generates is a reference to the snapshot service, decorated with a sequence number to help defeat browser caching (which cannot even be fully turned off in Internet Explorer).

Finally, there is the image service within the Box which is used by snap and by httpd. It employs a command/reply handshaking protocol over a TCP connection, where the client receives a JPEG image in response to each JPEG command and the response is delayed until the latest image picked up by the server is different from the last image sent to that client, if any.

  
4.1.2 Camera Control Services

The pan/tilt/zoom camera control hardware accepts commands and delivers acknowledgements and event notifications over a serial line. The Box provides a bridge between this device and any number of Internet clients.

The most comprehensive of the camera control services, camera, supports a protocol that is designed to be convenient for programs and at the same time mnemonic for people, in the best tradition of Internet servers. Programmers intending to write code which communicates with the camera service can obtain all the information they need by using the standard telnet client to connect to the appropriate host and port number, which should begin a session with the server sending something like this:

>Welcome to the Canon VC-C3 pan/tilt/zoom camera control server.
>Type Help for help.  Cavil and whine to dB (bacon@cs.nyu.edu).
.
This will be displayed in the interactive telnet session window. If help is entered as suggested, the response will be something like this:
>
>Commands are:
>
>Help [command-name]
>Mode {Host | RC}
>Notify {On | Off}
>Zoom {[To] factor | By factor | In | Out} [At speed]
>Move {[To] pan tilt | By pan tilt} [[In] ms | At speed]
>{Up | Down | Left | Right} deg [[In] ms | At speed]
>Ramp ms
>Show {Mode | Notify | Zoom | Move | Position | Ramp}
>Clear
>Reload
>Setup
>Reset
>Quit
>
>A null command (empty line) repeats the previous command.
.
If the user then playfully types help help, the serious reply is as follows:
>
>Help
>  -Gives a compact synopsis of all commands, with optional
>     words shown in brackets [ ], grouping indicated by braces {
>     }, and alternatives separated by bars |.
>  -All command names and arguments are case-insensitive,
>     though for clarity they are shown here as literal names
>     starting with an uppercase letter.  Substitute a value for
>     any (possibly hyphenated) name that begins with a lowercase
>     letter.  Numbers may include signs and decimal points.
>  -Help is the only command besides Show which produces output
>     back to you, the client, when asynchronous notification is
>     off (see the Notify command).  You can tell where a piece
>     of help ends by where the ">" lines leave off and the final
>     "." on a line by itself occurs.  Server usage errors (your
>     protocol mistakes) are also reported in this "help" format.
>     Output from Show always consists of a single line, as does
>     each asynchronous notification (event message), so their
>     ends are also easy to recognize.
>
>Help command-name
>  -Tells you all about a specific command.
.
This describes the difference between help/diagnostic output and show/notify output. Here are the results of help notify and help show:
>
>Notify On
>  -Turns on asynchronous notification.  You (the client) will
>     get an event message, formatted as a command recognized by
>     this server for convenience in playback, whenever there is
>     a change in the mode, zoom, pan/tilt, or ramp, and whenever
>     a zoom or pan/tilt limit is reached.  (Other messages,
>     with no corresponding command but formatted similarly,
>     will later be added.  For now, there is a catch-all message
>     "Canon", showing things the hardware is saying.)
>
>Notify Off
>  -Turns off asynchronous notification.  You can still get
>     information synchronously by using the Show command.
.

> >All Show commands produce their output in the form of a >command that could later be fed back in to the server to >re-establish the state reported by the Show. > >Show Mode > -Yields Mode Host or Mode RC. >Show Notify > -Yields Notify On or Notify Off. > -Each asynchronous notification (event message) and > Show result is sent to you, the client, on a single, > newline-terminated line. >Show Zoom > -Yields the current zoom factor as a Zoom [To] command. >Show {Position | Move} > -Yields the current pan and tilt angles as a Move [To] > command. >Show Ramp > -Yields a Ramp command for the current ramp period. .

Finally, here is the output from help move:
>
>Move [To] pan tilt [[In] ms] | At speed]
>  -Points the camera at pan degrees azimuth, tilt degrees
>     elevation, and stores these as the current values.
>  -Positive means right for pan, up for tilt.
>  -Range is -90 to 90 for pan, -30 to 25 for tilt.
>  -Resolution is 0.115 deg.
>  -The angular trajectory is shaped at each end by the
>     parabola suggested by the Ramp period.  If the angular
>     distance to move is large enough, maximum speed will be
>     sustained in the interval between the acceleration and
>     deceleration ramps unless constrained by the optional In
>     or At specification.
>  -If "[In] ms" is specified, the server will try to plan a
>     camera motion trajectory that takes ms milliseconds.
>  -If instead "At speed" is specified, the trajectory
>     speed will be limited to the given maximum during
>     the constant-speed interval between acceleration and
>     deceleration ramps.
>  -The units of speed in "At speed" are deg/sec, with a
>     resolution of 1 deg/sec and a range of 1 to 70 deg/sec.
>Move By pan tilt [[In] ms] | At speed]
>  -Adds pan degrees azimuth and tilt degrees elevation to
>     the current pan and tilt values, and calls Move [To].
.
Now, suppose the user enters the command notify on, requesting event notification. Any updates to hardware settings resulting from, e.g., move and zoom commands will cause command-like messages to be sent to all such clients interested in events. For example, if this telnet user or any other client typed the command zoom out, a message such as
Zoom 6.180
would appear in the telnet user's display. Clients can thus easily record camera control activity for later playback. Current settings can always be sensed with the Show command, so for example show move or equivalently show position might produce
Move To -76.671 -19.774

The camera service itself is little more than a command-processing front end for the do and notice services provided by the vc-do.setl program listed in Section A.11 [vc-do.setl], which is responsible for maintaining a high-level, state-bearing model of the camera as an acceptor of commands and producer of events.

In principle, all clients could use the general camera service to issue commands and receive notifications, but it works out better if things are made even simpler for Java clients, which tend to be multi-threaded and therefore favor specialized services relating to their particular responsibilities. Accordingly, the jumper, mover, and zoomer services accept client input consisting purely of one or two numbers on each line, and build the appropriate command to send to the do service for each such line. Also in this category is the mouse service, which is so tailored to the needs of Java clients that they essentially just have to pass pointing device gestures through ``in the raw'', and mouse maps these into camera control commands for transmission to the do service. This is effectively a generalization of the imagemap handling performed by the httpd service previously mentioned.

Similarly, there are Java-friendly services to provide an interface to the notice service, so that events can be delivered to Java threads in the simplest possible way. These are called evjump and evzoom.

The vc-do.setl program, in its event-producing capacity (the notice service), relies upon a pair of lower-level services, notify and event, which are linked such that every event received by notify is broadcast to all clients of event.

4.1.3 Administrative Services

WEBeye is designed with round-the-clock, unattended service in mind, but its administrative interface also makes it easy for people who are not computer experts to start, stop, and check up on the system, and to manage the log files.

WEBeye will normally run continuously unless there are problems severe enough to cause a critical component failure. For example, certain forms of resource exhaustion under heavy load conditions can trigger failures. The system is large and complex enough that it is also appropriate to ``expect the unexpected'' from hardware and software (see Section 5.3.3 [On the Unexpected]). When the unexpected happens, WEBeye does its best to bring itself down cleanly and completely, in the hope of clearing the condition which caused the failure. Then, when it is restarted, it has the best possible chance of survival.

In order to run unattended, WEBeye has to be able to be restarted automatically. This means that there has to be some program external to WEBeye that can observe when it has crashed or is not functioning correctly, and attempt to restart it. To this end, the vc-cron.setl program listed in Section A.9 [vc-cron.setl], which is intended to be run every minute from the administrative user's crontab file, steps through a series of checks on the presence and correct functioning of the Box. If the Box appears not to be running, or any of its principal servers fail to give satisfactory responses, vc-cron.setl tries to shut it down cleanly and restart it. However, in the event of recurring failures unrelieved by any indications of complete success, the frequency of restart attempts is decreased by powers of two until only one attempt is being made every 64 minutes. (This is in part an effort to avoid the Sorcerer's Apprentice syndrome, where an attempt to expunge one problem only leads to more problems, and in part an effort to avoid sending the WEBeye administrator too much e-mail, since a failure report is sent to that party every time vc-cron.setl tries to restart the Box.)

Restarting WEBeye involves shutting it down cleanly and then starting it again, as vc-restart.setl (Section A.37 [vc-restart.setl]) does by calling vc-quit.setl (Section A.35 [vc-quit.setl]) and then vc-go.setl (Section A.18 [vc-go.setl]). Another program, vc-check.setl (Section A.5 [vc-check.setl]) is also provided, and all these programs can be invoked via trivial wrapper commands such as restart for a more manual style of administration.


next up previous
Next: 4.2 Software Structure Up: 4. WEBeye: A Case Study Previous: 4. WEBeye: A Case Study
David Bacon
1999-12-10