README for clusterlib

Large-scale distributed applications are difficult and time-consuming to develop since a great deal of effort is spent solving the same challenges (consistency, fault-tolerance, naming problems, etc.). Additionally, coordinating these applications is typically ad-hoc and hard to maintain. Clusterlib fills the gap by providing distributed application developers with an object-oriented data model, asynchronous event handling system, well-defined consistency semantics, and methods for making coordination easy across cooperating applications.

Package Contents:

Clusterlib library

• Infrastructure for building and managing distributed applications

ActiveNode

• A process that is designed to be run on a physical node and manages processes started through clusterlib's process management system. It will create a clusterlib Node in the clusterlib object hierarchy and register a health checker for reporting health.

Zookeeper user interface (ZKUI)

• An http server that can display and manipulate Zookeeper znodes or clusterlib objects

Command line interface (CLI)

• Can make changes to clusterlib objects from the command line with this tool. It also can be used as an interactive shell for manipulating clusterlib objects

Unittests

• Can be run in single or multiple process node and tests the correctness of many APIs. It is only compiled when MPICH can be found.

Source Directory Structure:

docs Doxygen generated content and notes src/activenode ActiveNode implmentation src/cli CLI implementation src/core Internal clusterlib objects implementation src/core/md5 MD5 implementation from RSA Data Security, Inc. src/example Several example programs that use clusterlib object src/gui Base directory for ZKUI src/gui/conf Configuration files for ZKUI process src/gui/regressions ZKUI regression tests src/gui/src ZKUI imlplementation src/gui/unittest ZKUI unittests src/gui/webroot Directory that is served by the ZKUI server src/include Include files for application developer consumption misc Miscellaneous scripts or configuration files regressions Clusterlib regression tests unittests Clusterlib unittests

Generated directory structure:

zookeeperServerDir Directory for storing Zookeeper data for testing install Generated from 'make package': the essential headers, library, configuration files, and executables

Required Dependencies:

boost regex (http://www.boost.org) zookeeper (http://hadoop.apache.org/zookeeper/) logcxx (http://logging.apache.org/log4cxx/)

Optional Dependencies:

cppunit - Core library unit tests mpich2 - clusterlib unittests readline, ncurses - CLI tab completion libmicrohttpd - ZKUI apr-1, aprutil-1 - ZKUI XML parsing boost auto_unit_test - ZKUI unittests doxygen - Doxygen documentation

Building:

autoreconf -if Set the CXX, CXXFLAGS, CPPFLAGS, and LDFLAGS appropriately. ./configure make

Running Core Unittests:

make check

More details and options on how to run unittests are in the directory unittests.

Extending clusterlib:

Add new Notifyable objects (subclass Notifyable)

• Create an interface (i.e. class Application)
• Create an implementation (i.e. class ApplicationImpl)
• (optional) Create cached interfaces (i.e. class CachedKeyValues)
• (optional) Create cached implementation (i.e. class CachedKeyValuesImpl)
• Register your implementation in FactoryOps (i.e. registerNotifyable())

=====================

Clusterlib used several 3rd party open source libraries and tools. The following portions of this file summarize the tools used, their purpose, and the licenses under which they're released.

Except as specifically stated below, the 3rd party software packages are not distributed as part of this project, but instead are separately downloaded from the respective provider and built on the developer's machine as a pre-build step.

boost -Source: http://www.boost.org/ -License: http://www.boost.org/LICENSE_1_0.txt

log4cxx -Source: http://logging.apache.org/log4cxx/download.html -License: Apache License, Version 2.0

zookeeper -Source: http://hadoop.apache.org/zookeeper/releases.html -License: Apache License, Version 2.0

libmicrohttpd -Source: http://gnunet.org/libmicrohttpd/ -License: LGPL

The following 3rd Party code is included as part of this Project:

md5 test driver -Souce: RSA Data Security, Inc. -License: none

bluff.js 0.3.4 -Source: http://bluff.jcoglan.com/ -License: MIT

excanvas.js 0.4.0 (optional) -Source: http://code.google.com/p/explorercanvas/ -License: Apache License, Version 2.0

query.zend.jsonrpc.js 1.0 -Source: http://blog.jtclark.ca/tag/javascript/ -License: MIT

jQuery UI 1.8.5 -Source: http://jqueryui.com/ -License: MIT

jQuery JavaScript Library 1.4.2 -Source: http://jquery.com/ -License: MIT

TableSorter 2.0.3 -Source: http://tablesorter.com/docs/ -License: MIT and GPL

jQuery File Tree Plugin 1.01 -Source: http://abeautifulsite.net/2008/03/jquery-file-tree/ -License: Creative Commons 3.0

js-class.js -Source: http://jsclass.jcoglan.com/ -License: MIT

parseUri 1.2.1 -Source: http://blog.stevenlevithan.com/archives/parseuri -License: MIT