Elements module library source re-structured

I've just restructured the Elements module library source on Moduleforge to establish its own separate trunk, branches and tags.

The trunk has been copied from the 1.0 branch, and in addition I've established a new 2.0 working branch as a copy of the trunk.

New development will proceed on the 2.0 branch aimed at integrating an out of the box end-to-end Java Server build and deployment solution based on:

• CVS and Subversion source code control
• Cruise Control based continuous integration
• Ant and Maven based builds
• Tomcat and JBoss application servers
• HSQLDB and Derby Java based databases

In addition, we plan to formally integrate and document the Duke's Bank sample application as a demonstration application.

By the way, I'm the only one who likes the name "elements" it seems, but there doesn't seem to be any constructive alternative "short" names out there that aren't acronyms that have already been grabbed by the Java guys (JSSL, JSL, etc)!

The "full" name of the library is "Java Server Module Library". Any ideas for a catchy moniker that could supplant "elements" in the source base, etc? Or, like me, are you satisfied with the "elemental" appropriateness of the existing name? Or, more likely, do give a toss at all?

Thanks,

Anthony.

Converting Workbench modules for use with ProjectBuilder

ProjectBuilder is one of the key new features of ControlTier 3.1. This base type allows architects and administrators to manage their projects as a set of (XML) source using development tools of choice as opposed to solely using Workbench.

The thing is, many people already have sets of modules created under Workbench and therefore need to convert them for use with ProjectBuilder. Since this process is not currently well documented on Open.ControlTier, here's a quick cheat sheet:

• Setup a minimal ProjectBuilder object for your project. By convention, the ProjectBuilder is given a library name ("mymodules" in this case) that reflects the combined intent of the set of modules it manages.
  
  $ ad -p MyProject -t ProjectBuilder -o mymodules -c Register -- -basedir \${env.CTIER_ROOT}/src/mymodules -installroot \${env.CTIER_ROOT}/target/mymodules -install
  .
  .
  .
  For more information about this object run: ad -p MyProject -t ProjectBuilder -o mymodules -c Get-Properties -- -print
  
  [command.timer.Deployment.Register: 4.123 sec]
  

• By the way, the base directory is usually put under source code control to provide version management for the module source. A good example of this is the structure of the Elements Module Library source on ModuleForge.

• Create the minimal base directory structure:
  
  $ mkdir -p $CTIER_ROOT/src/mymodules/modules

• Copy in the latest version of your module(s) from Workbench's WebDAV working directory (Note that these commands assume you are setting up a ProjectBuilder development environment on the same system where Workbench is deployed. This need not always be the case):
  
  $ cd $CTIER_ROOT/src/mymodules/modules
  $ cp -r $CATALINA_BASE/webapps/webdav/MyProject/modules/MyBuilder .
  
  
• Convert the module's RDF files to type XML format:
  
  $ ad -p MyProject -t ProjectBuilder -o mymodules -c convert-rdf -- -type MyBuilder
  .
  .
  .
  convert-rdf completed. execution time: 0.888 sec.
  

• Remove extraneous RDF and property files from the module source:
  
  $ rm MyBuilder/*.rdf MyBuilder/*.properties

• Test building the module from source and uploading it to the project:
  
  $ ad -p MyProject -t ProjectBuilder -o mymodules -c build-type -- -type MyBuilder -upload
  Building type using the buildmodule.xml via classloader
  converting type.xml for module: MyBuilder
  packaging module: MyBuilder
  Building jar: /home/anthony/ctier/target/mymodules/modules/MyBuilder-1.jar
  calling repoImport ...
  processing files in directory: /home/anthony/ctier/target/mymodules/modules
  Uploading jar: /home/anthony/ctier/target/mymodules/modules/MyBuilder-1.jar to server: localhost ...
  [command.timer.repoImport: 3.058 sec]
  repoImport completed. execution time: 3.058 sec.
  [command.timer.build-type: 4.112 sec]
  build-type completed. execution time: 4.112 sec.
  

QED.

Anthony Shortland.

Initial "Duke's Bank" JBoss demonstration application posted

One of the key ways that ControlTier's commercial activities directly benefit our Open Source community is via the Solution Libraries maintained at the Moduleforge project.

The experience gained implementing solutions for our commercial customers is made generally available via these libraries, making Moduleforge the means of kick-starting projects by supplementing the base types/modules distributed with the ControlTier framework.

One example of these libraries is "Elements" - a "grab bag" of modules largely concerned with automating the Java server application build and deployment life-cycle with modules covering continuous integration with CruiseControl, builders such as Ant and Maven, application servers such as Tomcat and JBoss, and associated services such as Apache web server and Apache MQ.

There's plenty of work left to extend the scope of Elements to cover a broader set of technologies as well as improve the standard of the solution libraries documentation.

We made a solid step in the right direction this week by releasing notes on using the library to build and deploy Sun's "Duke's Bank" sample application (from their J2EE 1.4 Tutorial) to JBoss 4.0.3SP1, thus providing a clear example to JBoss users of the benefits of using ControlTier.

The plan is to make "Duke's Bank" the primary sample application of a major revision of Elements dubbed the "Java Server Solution Library" that we're preparing ahead of next month's SaasSCon in Santa Clara.

Anthony Shortland.

anthony@controltier.com

Upgrading to ControlTier 3.1-rc2

For those of you who are upgrading an existing ControlTier installation, Anthony has posted a set of tips on the mailing list to ensure things go smoothly.

You can find them here:
http://groups.google.com/group/controltier/browse_thread/thread/40fbdf43139d2f3c?hl=en

If you aren't already subscribed to the mailing list (run by google groups), I suggest doing so. Its a timely source of information and the quickest way to communicate with project developers.

ControlTier 3.1-rc2 Released

Download it here: http://sourceforge.net/project/platformdownload.php?group_id=151079

What's New since 3.1-rc1:

A Maven 2 plugin was created for antdepo, and a Maven archetype was created to support using maven to build a ControlTier Library Jar.

Changes since 3.1-rc1:

bugfix: Change-Dependencies command used by Updaters now correctly allows multiple package files with the same type and version when allowMultiples is true, but the dependency constraints between the deployment and its packages is not exactly specified.

bugfix: a bug preventing Transform creation was fixed.

Installer:

Improve ability to do 'upgrade' installations.

Base types:

Builder repoImport command now supports a 'separator' cli option, allowing the separator between package base and version to be something other than the default of a hyphen character '-'. Also added as an attribute named 'packageSeparator' which can be set with an appropriate Setting subtype. Some typo fixes.

Builder Subversion support improved: scmModule parameter was not being used in some cases.

ProjectBuilder: The meaning of the 'basedir' commmand-line flag was changed to be the top-level directory containing the modules directory.

ATG Commerce Solution Library 1.3 update posted to ModuleForge

One of the key lines of business for ControlTier is delivering automated provisioning solutions for ATG based e-commerce sites. Over the last three years we've been building up best-practice implementation skills that we've distilled into a freely available integrated solution library. 

I'm happy to announce that we've just posted a significant update to version 1.3 of our ATG Commerce Platform Solution Library to Moduleforge. This release consolidates the results of a good six months work focussing on JBoss based ATG build and deployment management with a number of key customers including ATG On-Demand Hosting Services themselves.

The release includes detailed documentation on how to build a demonstration code and data provisioning solution based on the Pioneer Cycling sample application running on JBoss, Solid and Linux. This demonstration is a solid starting point for anyone interested reviewing the functionality of our Solution Library; in fact, our consultants build on the Solution Library as the foundation for building custom solutions to meet our clients' site-specific requirements.

Please go ahead and post feedback on the release here or at our main discussion group.

Thanks,

Anthony Shortland,

ControlTier Software, Inc.

ControlTier Staff working with Open Managememt Consortium on Design Patterns

Alex Honor (ControlTier's open source project lead and principle architect) has begun working with Open Management Consortium on establishing a design pattern repository for operations software.

Read more about the genesis of this exciting project on the dev2ops.org blog:
Where are the design patterns for software operations?

You can also read more about the Open Management Consortium's establishment of the project on the Open Management Consortium Blog.

Things are really early, but we are committed to seeing this become a vibrant and fruitful project.

New Installation and Library Documentation

As part of our ongoing efforts to improve documentation...


1. We've posted updated ControlTier installation instructions on Open.ControlTier.

The key installer enhancements for 3.1 are:

* Out-of-the-box compliance with the self-contained directory convention largely avoiding the need to edit "default.properties".

*Jobcenter installation support has been completed including start/stop scripts.

The instructions have been tested on both Windows and Linux using the latest 3.1 release candidate.

2. New ModuleForge documentation has been posted. This includes new documentation for the ATG, Content Deployment, and RDB Deployment libraries.

As always, feedback on the mailing list is greatly appreciated.

ControlTier 3.1rc1 relesased

We are happy to announce that 3.1rc1 has been released. 3.1rc1 includes a number of bug, performance, and usability enhancements. We are now on the home stretch towards a general release for 3.1!

• Download 3.1rc1 here.

Below is an extensive list of the fixes that have gone in the last beta and this first release candidate. Changes are group by whether they are in the ControlTier project or and the underlying AntDepo framework (with tracker ID, if available):

ControlTier (http://sourceforge.net/tracker/?group_id=151079)
* #1740053 – dispatchCmd support for remote user name
* #1807666 – Add ThreadCount to Mediator’s dispatchCmd
* #1620501 – Workbench/Activity Report: URL query-able activity report
* #1776641 – Show command usage for Deployment objects
* #1769627 – Ant task to check if Workbench project exists
* #1766323 – creating a project from the command line
* #1720429 – library import support for ProjectBuilder
* ProjectBuilder: XML based object definition format (projectxml)
* make logging messages consistent format
* report task to echo report text to console
* Mediator dispatchCmd option initialization supported by full set of attributes
* Mediator: add support for passing buildstamp
* ProjectBuilder: For command generate-objects, add support for projectxml format
* ProjectBuilder: add -load option to build-library to load built library to server
* Package: ensure all the package metadata field values are passeed into the “install” command.
* Deployment: move runChangeDependencies from Updater to Deployment
* Deployment: new “Deploy” command to supersede “Update”
* Builder: allow non propfile method to repoImport
* Builder: Automatic multi-package (Builder) repoImport
* Installer: Follow CTIER_ROOT self-contained directory convention
* Workbench: Initial integration with AntDepo to streamline antdepo object installation
* Java code interdependencies among the ControlTier components have been separated into a common library
* the change-package-dependencies task/action has been altered on the server side to remove the use of rules-based reasoning with the Jena engine, bringing performance improvements and adding resilience to model inconsistencies.
* change-package-dependencies now behaves as expected when the package Type specified is actually a sub-type of one of the allowed-types for the dependencies.


AntDepo (http://sourceforge.net/tracker/?group_id=147344)
* #1803918 – lazy framework initialization. Loads lazily and now dynamic.
* #1765616 – Support get-opts “defaultproperty” attribute
* #1835431 – depot-setup option:—name <>
* #1835432 – depot-setup action name change
* Antdepo now has a ‘framework.ssh.user’ property, used as the username when invoking SSH executions.
* Read “framework.ssh.user” from command execution context. Will default to value in framework.properties file.prefix logged messages with command context info
* depot-setup now must use -p option
* Allow Framework object to be embedded in long running server processes.
* Add java min/max heap settings to cmdline client
* Add ignormalformed and propertydefault attributes

ControlTier 3.1b1 and Job Center

I'm happy to announce that ControlTier 3.1b1 with the new Job Center has been released.

• Download it here.

ControlTier 3.1-beta-1 has bug fixes and performance enhancements throughout Antdepo and Workbench. But the biggest thing included in this release is the new Job Center application.

Job Center

Job Center is a web interface for creating, scheduling and managing jobs executed in the Antdepo environment. It is still in Alpha form (0.4-alpha), so the interface may eventually be different and bugs may pop up.




Features include:

• Jobs can either be transient (one-off), stored for on-demand execution, or scheduled to run periodically.
• Job execution progress can be followed live on the web
• Browse, filter and customize reports of previously executed jobs
• Easy access to all Jobs that are Now Running, or have recently finished executing.
• Intuitive interface.

Technical Notes

Job Center development required converting the Antdepo Framework concept into an embeddable application. Since Antdepo executes in the Ant environment, this posed some problems because Ant was not necessarily designed to be embedded within another application. Until now, Antdepo has not had to address this problem because the Antdepo context is necessarily single-user and short-lived when running as a command line application. Converting this method of operation into a multi-user, long-lived web server process paradigm required getting over some hurdles in both the Ant and Antdepo frameworks.

One problem brought up by Ant was that of output on STDOUT and STDERR. For example, when a command is dispatched to a remote Node, an Ant SSHExec task is created to handle the SSH connection to a node and execution of a remote Antdepo command. The SSHExec task in Ant prints its output to STDOUT and STDERR. With the command-line "ad" command this is not a problem. However in Job Center the output of the command has to be captured by the executing thread in order to make it visible for the user. In addition, multiple commands may be running simultaneously in several Java Threads, and if multiple commands are executing SSHExec tasks then all of the output will end up on the output console and not in the command log where we want it to go.

The solution to this problem is to use a custom replacement for the Java System's standard-output and standard-error PrintStream objects. By using the special InheritableThreadLocal class, the STDOUT and STDERR can be replaced by a custom OutputStream for each command Thread. The OutputStream is then inherited by any child Threads spawned by the command thread. This is necessary because SSHExec specifically uses another Thread while it is executed, and other tasks may do the same thing. This custom OutputStream logs the execution output to a file for use by Job Center to view the command output. After execution, the InheritableThreadLocal is cleared of the custom OutputStream for the Thread.