Description ------------------------------------------------------------------------------- This is a benchmark suite for different implementations of the SAGA Advert Service (http://www.ogf.org/OGF27/materials/1767/saga_adverts.pdf) and their backend implementations. It takes three parameters: (1) Number of components: This describes the number of individual (potentially distributed) processes that access the advert service concurrently. (2) Number of attributes: The number of attributes that "belong" to each component. The attributes are stored in the component's advert server entry and are accessible by other components. (3) Number of iterations: Number of iterations the main benchmark loop is executed and measured. More iterations usually mean a more accurate average. The Main Benchmark Loop The main benchmark loop is controlled by the parameters above. The control flow is quite simple: For each Iteration (1) A component CX opens the advert workspace of another, randomly assigned component CY. (2) CX lists all of CY's attributes (3) CX reads each of CY's attributes (# 'read' calls = Number of attributes) (4) CX writes its own attributes into CY's "inbox" (# 'write' calls = Number of attributes) (5) CX records the duration that it took to execute the above steps Example Usage ------------------------------------------------------------------------------- (1) Using the default saga advert service with 10 components (workers) 10 attributes per component and 5 iterations. The components are running locally (using the default job adaptor): $> python advert-benchmark.py \ advert://SAGA:SAGA_client@advert.cct.lsu.edu:8080/benchmark \ -p -c 10 -a 10 -i5 The results are human-readable and accessible in the advert database: $> saga-advert list_attributes \ advert://SAGA:SAGA_client@advert.cct.lsu.edu:8080/benchmark/jobstats An entry for an indivdual component may look like this: (vector) comp0 : STATUS:DONE, STIME:Wed, 07 Sep 2011 14:59:56 +0000, ETIME:Wed, 07 Sep 2011 15:00:13 +0000, ITERTIME0:1.00112605095, ITERTIME1:1.00112605095, ITERTIME2:1.0011241436 The keys have the following meaning: STATUS: Can be OFFLINE (not yet started), ONLINE, DONE or FAILED STIME: The component startup time ETIME: The component shutdown time ITERTIME(X) The duration of the Xth iteration The log-files of the individual components can be found in /tmp/. The location can be changed with the --tmpfile option. (2) The same works also with the new 'node.js/mongoDB' advert service, just by changing the advert url: $> python advert-benchmark.py \ sqlasyncadvert://faust.cct.lsu.edu:8080/benchmark \ -p -c 10 -a 10 -i5 $> saga-advert list_attributes \ sqlasyncadvert://faust.cct.lsu.edu:8080/benchmark/jobstats