next up previous
Next: 4.3 Summary Up: 4. WEBeye: A Case Study Previous: 4.1 What WEBeye Is


4.2 Software Structure

The primary program in the WEBeye Box is vc-toplev.setl, as listed in Section A.42 [vc-toplev.setl]. It is responsible for starting, monitoring, and stopping all the programs which provide one or more TCP services. These programs and their descendants can be clients of services in the Box, so vc-toplev.setl is careful to start and stop them in an order which respects client-server dependencies.

Following is a snapshot of a process tree taken a few minutes after the initialization of a WEBeye Box that is currently in operation [18] at Lehigh University. Each line begins with the number of minutes and seconds of CPU time consumed so far by its corresponding process. The indentation structure indicates parent-child relationships:

  0:00 setl vc-toplev.setl 
  0:03  \_ setlrun -5 
  0:00      \_ setl vc-event.setl 
  0:00      |   \_ setlrun -8 
  0:00      \_ setl vc-giver.setl 
  0:00      |   \_ setlrun -8 
  0:00      \_ setl vc-image.setl 
  0:44      |   \_ setlrun -8 
 12:56      |       \_ image-pump 
  0:00      \_ setl vc-push.setl 
  0:00      |   \_ setlrun -8 
  0:11      |       \_ setlrun -8 
  0:00      |       \_ setlrun -8 
  0:00      \_ setl vc-snap.setl 
  0:00      |   \_ setlrun -8 
  0:00      \_ setl vc-do.setl 
  0:01      |   \_ setlrun -8 
  0:00      |       \_ setl vc-model.setl 
  0:01      |           \_ setlrun -9 
  0:00      |               \_ setl vc-seq.setl 
  0:00      |                   \_ setlrun -10 
  0:00      |                       \_ setl vc-send.setl 
  0:00      |                       |   \_ setlrun -11 
  0:00      |                       \_ setl vc-recv.setl 
  0:01      |                           \_ setlrun -11 
  0:00      |                               \_ setl vc-input.setl
  0:00      |                                   \_ setlrun -12 
  0:00      \_ setl vc-httpd.setl 
  0:00      |   \_ setlrun -8 
  0:00      \_ setl vc-mouse.setl 
  0:00      |   \_ setlrun -8 
  0:00      \_ setl vc-mover.setl 
  0:00      |   \_ setlrun -8 
  0:00      \_ setl vc-jumper.setl 
  0:00      |   \_ setlrun -8 
  0:00      \_ setl vc-zoomer.setl 
  0:00      |   \_ setlrun -8 
  0:00      \_ setl vc-camera.setl 
  0:00      |   \_ setlrun -8 
  0:00      |       \_ setl vc-ptz.setl -- 10 
  0:01      |       |   \_ setlrun -11 -- 10 
  0:00      |       \_ setl vc-ptz.setl -- 10 
  0:01      |       |   \_ setlrun -11 -- 10 
  0:00      |       \_ setl vc-ptz.setl -- 10 
  0:01      |           \_ setltran 
  0:00      |           \_ setlrun -11 -- 10 
  0:00      \_ setl vc-evjump.setl 
  0:00      |   \_ setlrun -8 
  0:00      \_ setl vc-evzoom.setl 
  0:00          \_ setlrun -8
At the time this snapshot was taken, there were two clients connected to the push image-streaming service (Section 4.1.1 [Video Services]) and three to the camera control service (Section 4.1.2 [Camera Control Services]), though one of the latter had just connected--the instance of setltran indicates that we caught vc-camera.setl at a moment when it had just started to instantiate vc-ptz.setl as a child process for the new client, and the SETL translator was still active (setltran disappears immediately after passing the result of its translation to setlrun; occasionally, one can even catch an instance of setlcpp, the SETL preprocessor, in one of these displays).

Each negative number appearing in this process display is actually a command-line parameter passed from setl to setlrun identifying the file descriptor on which setlrun is to read the translated form of a SETL program. User-level command-line arguments to setl and hence setlrun are placed after an argument of the form ``--'', as mentioned in Section 2.1 [Invocation Environment], and we see instances of this in the process display where vc-camera.setl passes the file descriptor for each newly accepted client to vc-ptz.setl. The fact that it is always the same number (10) here reflects the fact that vc-camera.setl always closes its own copy of that file descriptor after it has created the child process (vc-ptz.setl) which preserves the reference to the underlying kernel data structure, so the number is available to be used again in vc-camera.setl when the next client seeks a connection.

When vc-toplev.setl is started, it first attempts to make sure there are no other running instances that could conflict with it. The mutual exclusion mechanism is conservative. It is based on the existence of a lock file that is atomically created and destroyed. A clean shutdown of the Box ends with removal of the lock. If this does not happen, due to some catastrophic failure (such as loss of system power or an uncaught error in vc-toplev.setl itself), or if the lock exists because there is another running instance, the administrator of the Box is cautioned to perform certain checks before separately removing the lock and then restarting the Box. (See the commence procedure in vc-admin.setl, listed in Section A.1 [vc-admin.setl].)

Because of the dependencies between servers and clients through the services they respectively provide and use, including the dependencies transitively created by parent-child relationships, the next major job of vc-toplev.setl is to analyze all the source texts comprising the Box, so that it will be able to start servers in an order that ensures no client is started before a service it depends on is available.

This is a matter of preprocessing the SETL source programs, scanning for the recognizable patterns indicating the relationships of interest, and constructing the appropriate maps. Provided that consistent idioms are used to create child processes and to request TCP services, this is more reliable than trying to maintain a separate database of dependency information (a hazard familiar to Makefile users).

Dependencies on services effectively disappear once they have been satisfied by services becoming available, so just before starting to spawn the server processes, the top-level program sets up a little registry service to allow servers to ``publish'' their services, and a corresponding service for clients to look up information about services, particularly their dynamically assigned TCP port numbers.

With the registry in place, vc-toplev.setl proceeds to start servers which have no (outstanding) dependencies, waiting for all the services of the Box to be published, and shrinking the dependency maps as services appear. It also issues warnings if some services seem to be taking unreasonably long to publish themselves.

If and when all the services do come up, vc-toplev.setl announces to the world that the Box is ready for use by external clients. It does this by instantiating a template HTML document with an embedded URL that refers to a key port number in the Box. It then stores that document into a file and redirects a certain link to point to it. The link is what is called a symbolic link in Unix, which is a special kind of file that has no content of its own, but merely refers to another file, such that when input or output is performed on the link file, it really happens on the referred-to file. The link that is manipulated by vc-toplev.setl has a name that is known to a nearby Web server. This well-known link has four stages in its life cycle:

When the Box is cleanly down, it points to an HTML document which says so.
When the Box is in the initial process of analyzing dependencies and starting servers, the link is made to point to a static document indicating that the Box is in its ``initializing'' transition state.
When the Box is fully operational, the link is redirected to the HTML document that is instantiated with the main WEBeye port number as mentioned in the previous paragraph.
When the Box is about to attempt a clean shutdown, the link is again redirected to a static document identifying a transition state, this one called ``closing''. Under normal circumstances, the Box will not spend long in the latter state, and the link will quickly be redirected to the ``cleanly down'' document.

There is actually one deliberate oversimplification in the above description, however. Because the principal interface to WEBeye is via the imagemap provided by the httpd service implemented by vc-httpd.setl, and this service already instantiates a template HTML document in response to each imagemap click, it is logical for the httpd implementation to serve the initial document as well. Redirection of a symbolic link is not an atomic operation in most versions of Unix, either--the link has to be destroyed and then re-created--so there is the slim possibility that it might not exist just at the moment a Web server attempts to read from it. For these reasons, the link is not read directly by the Web server itself, but by a CGI (Common Gateway Interface) script which waits a short while for the link to exist if necessary. This script then reads the file referred to by the link, and inspects it for a special prefix character sequence. If this sequence is not found, the file is presumed to contain a static document, which the script serves verbatim. If the special prefix is found, however, it is assumed to be followed by a properly delimited ``host:port'' designation of the lookup service. The script then uses lookup to find the host and port location of the httpd service, and opens a client connection to the latter. Web servers supply CGI scripts with any parameters that were received on the original request, typically arising from information in a URL after the script name, and transmit these parameters to CGI scripts through the PATH_INFO and QUERY_STRING environment variables. The CGI script that is generated at system configuration time from vc-master.cgi (see Section A.26 [vc-master.cgi]) passes the URL-originated parameters through to the httpd service just as a browser would, and serves whatever httpd returns. Similarly, the CGI script that is generated from vc-jmaster.cgi (Section A.24 [vc-jmaster.cgi]) uses the lookup service to obtain the host and port locations of the services needed by Java-based clients, and substitutes these into the template HTML document it serves, which becomes the primary ``page'' for the continuous-motion browser interface featuring the control applets described in Section 4.1 [What WEBeye Is].

The template document instantiated by httpd refers back to the httpd service, so after serving the initial document, the Web server is not involved. Indeed, the Web server could have been bypassed entirely by a user who already knew the port number on which the WEBeye httpd service was listening, perhaps from previous contact or through having sufficient access to WEBeye's private files. In the case of the Java interface, the Web server is also involved only long enough to invoke the CGI script which instantiates the page referring to the applets, and to serve the byte code of those applets.

4.2.1 Video Services

The program vc-snap.setl, listed in Section A.41 [vc-snap.setl], implements the public snapshot service. It checks but ignores the details of the required HTTP GET or POST request from the client, and replies with a MIME-wrapped JPEG image, which it in turn obtains from the core video server.

Being a defensive public server, it deals with each client through a pump stream connected to a child process, and keeps a map from pump stream file descriptors to client information records. It is a small, multiplexing data processing module.

The program which implements the server-push service is vc-push.setl, listed in Section A.34 [vc-push.setl]. Like the snapshot server, it is not fussy about the details of the HTTP request. It does, however, support an optional rate parameter as described in Section 4.1.1 [Video Services]. It is another small, multiplexing data processing module that deals with clients through child processes.

So is the vc-giver.setl program listed in Section A.17 [vc-giver.setl], whose giver service gives out URLs for a sequence of JPEG images as a temporary measure pending the widespread availability of the Java API that supports direct image streaming. Each URL it creates contains a reference to the snap (snapshot) service.

Finally, there is the image service provided by the vc-image.setl program listed in Section A.20 [vc-image.setl]. It is used by vc-snap.setl and vc-push.setl. It interfaces with an external program, image-pump, which is written in C and, when active, captures images and converts them to JPEG format as quickly as it can, telling the parent SETL server (vc-image.setl) about each image as it is ready.

This server, vc-image.setl, is intended for use strictly within the Box. It does a rudimentary check to authenticate each client (see the listing of vc-allowed.setl in Section A.2 [vc-allowed.setl]). Once satisfied, it dispenses with the usual need for an intervening child process and communicates directly with the client. It is not a complex program, but does keep a little state associated with each client so that each client can start receiving an image when it says it is ready to receive it and when a JPEG image that is new to that client has arrived from image-pump.

4.2.2 Camera Control Services

The camera service is provided by the SETL program vc-camera.setl listed in Section A.4 [vc-camera.setl]. This program is really nothing more than a front end for a much larger program, vc-ptz.setl (Section A.33 [vc-ptz.setl]), which implements the protocol introduced in Section 4.1.2 [Camera Control Services]. The front end performs the important function of making the service available to an arbitrary number of clients simultaneously, which it does by instantiating vc-ptz.setl once for each client, as a child process connected through a pump stream. The front end server also logs some information, such as the beginning and end of each client session, and distributes TERM (``terminate'') signals when it receives a TERM signal, so that all the child processes will close their connections and exit rather than continuing after the camera service itself has (presumably) gone down.

When vc-camera.setl accepts a new client connection, it passes the new file descriptor to vc-ptz.setl on the command line. This child process then deals with the client directly through this file descriptor. In the interests of modularity and what Dijkstra called a ``clear separation of concerns'' [58], vc-ptz.setl does little more than parse and check protocol commands, and pass them to the local do service (see vc-do.setl) in the form of SETL maps. If the client of vc-ptz.setl has requested continuous asynchronous notification of events by issuing the command notify on, then vc-ptz.setl also maintains a connection to the local notice service, which (like the do service) is provided by vc-do.setl.

Once again, by the same structuring principles, vc-do.setl is a small program whose chief concern is the multiplexing of client sessions, and it hands the ``real'' work off to another program, vc-model.setl, which maintains a stateful, high-level model of the pan/tilt/zoom camera controller. Normally, vc-do.setl is instantiated just once, and so is vc-model.setl, for a given serial port. This model maintainer is strictly sequential, and completes each command (as passed down from vc-ptz.setl through vc-do.setl) before it begins the next one.

Thus vc-do.setl actually has to do some minor arbitration of requests from the multiple clients of its do service when they cannot all be satisfied at once. If it is busy (meaning that a command has been sent to vc-model.setl but not yet been replied to) when a command is received from a client, it will enter the request on a FIFO queue. However, each client is only allowed up to one pending request--vc-do.setl will not even read a further command from a client that already has one in the queue. Clients at this level get an explicit indication of command completion from the do service.

The jumper, mover, and zoomer services designed for Java clients, whose programs vc-jumper.setl, vc-mover.setl, and vc-zoomer.setl are listed in Sections A.25 [vc-jumper.setl], A.29 [vc-mover.setl], and A.43 [vc-zoomer.setl] respectively, all use the do service in such a similar way that they are implemented by using the SETL preprocessor to define some symbols and then textually include vc-simpler.setl (Section A.40 [vc-simpler.setl]).

The gesture-interpreting mouse service, provided by the program vc-mouse.setl listed in Section A.28 [vc-mouse.setl], also uses do.

The Java-friendly evjump and evzoom services, which are provided by the programs vc-evjump.setl and vc-evzoom.setl listed in Sections A.13 [vc-evjump.setl] and A.14 [vc-evzoom.setl] respectively, use the notice service through the textually included vc-javent.setl listed in Section A.23 [vc-javent.setl].

The vc-do.setl program is the logical place to provide the notice service as well as the do service, because it is in direct contact with both the model maintainer and with a low-level event service, event, provided by vc-event.setl.

The vc-model.setl program, in its turn, implements high-level commands as sequences of lower-level commands, but it hands the actual timing, handshaking, and retrying responsibilities pertaining to command sequences off to yet another program, vc-seq.setl, which also happens to be in the best position to send low-level event notifications to a service called notify. The notify service is provided by the vc-event.setl program just mentioned, and works as follows: when a client apprises notify of an event, vc-event.setl distributes that event to all clients of its event service.

next up previous
Next: 4.3 Summary Up: 4. WEBeye: A Case Study Previous: 4.1 What WEBeye Is
David Bacon