Monday, October 01, 2007

ControlTier 3.1b1 and Job Center

I'm happy to announce that ControlTier 3.1b1 with the new Job Center has been released.

ControlTier 3.1-beta-1 has bug fixes and performance enhancements throughout Antdepo and Workbench. But the biggest thing included in this release is the new Job Center application.

Job Center

Job Center is a web interface for creating, scheduling and managing jobs executed in the Antdepo environment. It is still in Alpha form (0.4-alpha), so the interface may eventually be different and bugs may pop up.

Features include:

  • Jobs can either be transient (one-off), stored for on-demand execution, or scheduled to run periodically.
  • Job execution progress can be followed live on the web
  • Browse, filter and customize reports of previously executed jobs
  • Easy access to all Jobs that are Now Running, or have recently finished executing.
  • Intuitive interface.
Technical Notes

Job Center development required converting the Antdepo Framework concept into an embeddable application. Since Antdepo executes in the Ant environment, this posed some problems because Ant was not necessarily designed to be embedded within another application. Until now, Antdepo has not had to address this problem because the Antdepo context is necessarily single-user and short-lived when running as a command line application. Converting this method of operation into a multi-user, long-lived web server process paradigm required getting over some hurdles in both the Ant and Antdepo frameworks.

One problem brought up by Ant was that of output on STDOUT and STDERR. For example, when a command is dispatched to a remote Node, an Ant SSHExec task is created to handle the SSH connection to a node and execution of a remote Antdepo command. The SSHExec task in Ant prints its output to STDOUT and STDERR. With the command-line "ad" command this is not a problem. However in Job Center the output of the command has to be captured by the executing thread in order to make it visible for the user. In addition, multiple commands may be running simultaneously in several Java Threads, and if multiple commands are executing SSHExec tasks then all of the output will end up on the output console and not in the command log where we want it to go.

The solution to this problem is to use a custom replacement for the Java System's standard-output and standard-error PrintStream objects. By using the special InheritableThreadLocal class, the STDOUT and STDERR can be replaced by a custom OutputStream for each command Thread. The OutputStream is then inherited by any child Threads spawned by the command thread. This is necessary because SSHExec specifically uses another Thread while it is executed, and other tasks may do the same thing. This custom OutputStream logs the execution output to a file for use by Job Center to view the command output. After execution, the InheritableThreadLocal is cleared of the custom OutputStream for the Thread.