Monitoring In Grid

2007 IEEE Asia-Pacific Services Computing Conference

Implementation of Monitoring and Information Service Using Ganglia and NWS for Grid Resource Brokers* Chao-Tung Yang† Tsui-Ting Chen Sung-Yi Chen High-Performance Computing Laboratory Department of Computer Science and Information Engineering Tunghai University, Taichung, 40704, Taiwan [email protected] due to the considerable diversity, large numbers, dynamic behavior, and geographical distribution of the entities. Hence, information services are a vital part of any Grid software infrastructure. Typical monitoring and discovery use cases include providing data so that resource brokers can locate computing elements appropriate for a job, and streaming data to an application [7, 12, 19]. MDS of Globus Toolkit provides a nice information management tool, but it is not capable of providing the rich set of all the requisite information by itself. For this paper, we will concentrate on building monitoring and information services focused on providing comprehensive details about the resource information on a grid [8, 9, 10, 13, 14, 15]. It can provide us with execution details of any grid job or grid CPU (load, architecture, etc.,) of interest. As we known, Ganglia [16] can monitor cluster resources and enhance MDS information with comprehensive cluster-level status information. Ganglia is flexible open source, coexists easily with MDS and other information providers, and it is well tested and widely used. For the collection of historic information we chose Ganglia [16]. The Ganglia monitoring system collects a set of system properties at regular intervals and stores them in a round-robin database. It is also possible to monitor additional properties by providing a script to extract these properties to Ganglia. Ganglia provide a PHP Web front-end for administrator to view cluster or Grid status information in real time. The default information includes some metrics, such as processor load, memory usage, network (bytes input/output), and disk utilization. For more information related to network of a grid, The

Abstract Recently, Grid computing is increasingly used by organizations to achieve high performance computing and heterogeneous resources sharing. These Grids may span several domain administrations via internet. As a result of this, it may be difficult to monitor, control and manage those machines and resources. This paper aims at providing a multi-platform Grid monitoring service which can monitor resources such as CPU speed and utilization, memory usage, disk usage, and network bandwidth in a real-time manner. Monitoring data is extracted form Ganglia and NWS tools then stored and transmitted in XML form and then used for displaying. All the information is displayed using real-time graphs.

1. Introduction Grid computing can be defined as coordinated re source sharing and problem solving in dynamic, multi institutional collaborations [1, 2, 3, 5, 7, 11, 12, 19, 21]. Grid computing involves sharing heterogeneous resources, based on different platforms, hardware/software, computer architecture, and computer languages, which located in different places belonging to different administrative domains over a network using open standards. As more Grids are deployed worldwide, the number of multi-institutional collaborations is rapidly growing. However, for Grid computing to realize its full potential, it is expected that Grid participants are able to use resource of one another. In the Grid the characterization and monitoring of resources [1, 7, 12, 19], services, and computations are very challenging *

This work is supported in part by National Science Council, Taiwan (ROC), under grant no. NSC 96-2221-E-029-019-MY3 and NSC 95-2218-E-007-025. † Corresponding author.

0-7695-3051-6/07 $25.00 © 2007 IEEE DOI 10.1109/APSCC.2007.74

356

Network Weather Service (NWS) [18], is used to obtain monitoring information. As we know, NWS though not targeted at Beowulf clusters, is a distributed system that periodically monitors and dynamically forecasts the performance various network and computational resources can deliver over a given time interval. However, the administrator needs more flexible and variable operation and network bandwidth related information provided by Web front-end for all practical purposes. As a result of this, it may be difficult to monitor, control and manage those machines and resources. This paper aims at providing a multi-platform Grid monitoring service which can monitor resources such as CPU speed and utilization, memory usage, disk usage, and network bandwidth in a real-time manner. Monitoring data is extracted form Ganglia and NWS tools then stored and transmitted in XML form and then used for displaying. All the information is displayed using real-time graphs.

however, allows new internal sensors to be configured into the system.

3. System design 3.1. Resource broker The previous work has implemented a Resource Broker for Computational Grid. Resource broker discovers and evaluates Grid resources, and makes job submission decisions by comparing the requirements of a job with Grid resources. The system architecture of Resource Broker and the relation of each component were shown in Figure 1. Each rectangular represents a unique component of our system. Users could easily make use of our Resource Broker through a common Grid portal [9, 13, 15, 19, 21]. The primary task of Resource Broker is to compare requests of users and resource information provided by Information Service. After choosing the appropriate job assignment scheme, Grid resources are assigned and the Scheduler is responsible to submit the job. The results are collected and returned to Resource Broker. Then, Resource Broker records results of execution in the database of Information Center through the Agent of Information Service. The user can query the results from Grid portal.

2. Background 2.1. Machine information provider The Ganglia [16] is an open source project grew out of the University of California, Berkeley’s Millennium initiative. The Ganglia is a scalable distributed system for monitoring status of nodes (processor collections) in wide-area systems based on Grid or clusters. It adopts a hierarchical; tree-like communication structure among its components in order to accommodate information from large arbitrary collections of multiple Grid or clusters. The information collected by the Ganglia monitor includes hardware and system information, such as processor type, CPU load, memory usage, disk usage, operating system information, and other static/dynamic scheduler-specific details.

2.2. Network information provider The NWS (Network Weather Service) [18] is a distributed system that detects network status by periodically monitoring and dynamically forecasting over a given time interval. The service operates a distributed set of performance sensors (network monitors, CPU monitors, etc.) from which it gathers system condition information. It then uses numerical models to generate forecasts of what the conditions will be for a given time period. The system includes sensors for end-to-end TCP/IP performance (bandwidth and latency), available CPU percentage, and available non-paged memory. The sensor interface,

Figure 1. System architecture of resource broker

3.2. Software stack diagram

357

The software stack diagram of the system includes three layers constructed of bottom up methodology, such as bottom layer, middle layer, and top layer, the sense of each layer are described in the following. Bottom Layer: The principal part of this layer is composed of Nodes, i.e., the node in Grid should be constructed by software stack which is shown in Figure 2. This layer contains two main blocks, first is Information Provider, which gathers machine information of Nodes, such as the number of processor/core, the load of processor, the free/total size of memory, and the usage of disk, for the abovementioned purposes the Ganglia serves as the Machine Information Provider in this system. The part of essence of Grid is connecting Nodes in Grid with Internet, hence the network information among Nodes such as the bandwidth and latency is essential, and for above purposes the NWS takes on the Network Information Provider. The second block is Grid Middleware, used to join Grid Nodes together, and the MPICH-g2 [4] that compatibles with GT is required for running parallel applications on the Grid.

Figure 3. The software stack of all Sites and the Service

4. System implementation 4.1. Information Aggregator The main phases of Resource Broker are Resource Discovery, Application Modeling, Information Gathering, System Selection, and Job Execution [14]. The subject matter of Information Gathering phase is aggregating machine and network information for Resource Broker making a suitable match of job and resources. For above purposes, this work devises two services called Information Service and Monitoring Service, Information Service plays the role of gathering the machine and network information and store up into database, and Monitoring Service provides a Web front-end page for users to observe the variation during the process of jobs execution. Figure 4 depicts architectures of Information Service and Monitoring Service, and their relation between Resource Broker. The primary purpose of Information Service is to collect related resource information (processors, memory, disk, and network bandwidth) of all machines in the Grid and provide the analyzed information. These components and their relations are described as follows: z Agent: It is the primary component of Information Service, and is the contact window of Information Service. Either Scheduler of Resource Broker or Controller of Monitoring Service needs real-time information of machines or estimated information. For example, assume that Resource Broker is requested by users for the list of machines with low CPU loading. First, Resource Broker sends a Request to Agent. After Agent receives the Request, it uses Getter and Setter to get required information, and returns it to Agent. Then, Agent sends it to Resource Broker. After the task is finished, Resource Broker delivers the related data during execution to Agent, including number of used CPUs,

Figure 2. The software stack of all Nodes Middle Layer: The main composition of this layer is Site. The software stack diagram is shown in Figure 3. Each Site consists of several Nodes, which are located in the same place or connected with same switch/hub, each Node in a Site should connect to each other by Internet. Moreover, each Site usually is built up as a cluster and each Node has a real IP, and the first Node of this Site is called the head Node in this Site. The construction of this layer is related to the domain-based network information model that will be described later. Top Layer: The core component of this layer consists of two blocks, Resource Broker and Monitoring Service, as shown in Figure 3. Moreover, the Monitoring Service provides a web front-end for users to observe the variation during the progress of jobs. Besides, users can specify the duration of particular Nodes or several particular links in a domain which was developed based on Ganglia and NWS tools.

358

execution time, disk space usage, memory usage, task requirement, etc. These historical data are stored in Message Center. Predictor will be able to analyze these data, and then report a suggested machine list to Agent. z Gatherer and Setter: This component responds information collecting and data accessing could occur at any time, so events of database operations would be frequent. In order to unify information access and reduce redundant program development, Getter and Setter are designed and placed at the front end of Message Center, to control the access of Message Center. z Message Center: This component is mainly used to store native information from the Grid, including CPU Load, Memory Free, Disk Usage, Network Information, etc. In addition, observation data of Job Execution and Prediction data analyzed by Predictor are included. z Gather: MDS service of Globus could collect resource information such as CPU speed, number of CPUs, CPU loading, memory size, available memory, disk space usage, and network interface information. The NWS tool is used to collect the network bandwidth currently. Then, the Getter and Setter component stores that information to Message Center for future usages. z Predictor: This component has two functions. One is to periodically get native information from Message Center. By Modular design, different Type of native information is adopted by different prediction model. Then, they are stored in Message Center for future use. The other is to accept Request of Agent to predict and get required results, increasing system flexibility and more applications. The goal of Monitoring Service is to acquire the information maintained by Information Service, and present it in graphical form. The tasks and relations of these components are described as follows. z Controller: It is the main component of Monitoring Service, and its primary task is to control the behavior of Monitoring Service, including Grid Nodes configuration and parameter setting. Controller needs periodically to get native information of Nodes from Agent of Information Service. Then, it sends parameters and data to Drawer for illustration. z Drawer: It receives parameters and data from Controller and draws these figures. Then, Displayer presents the figures. The functions of drawing need to be flexible. It has to draw appropriate figures according to information types.

z

Displayer: This component is to provide a query mechanism for users to observe historical data of Grid Nodes. Therefore, a web interface must be provided for convenient use. This component will be integrated into Portal for users to query conveniently.

Figure 4. Architecture of Information Service and Monitoring Service The Ganglia is a scalable distributed monitoring system for monitoring status of host in cluster or Grids. It provides a PHP Web front-end for administrator to view cluster or Grid status information in real time. The default information includes some metrics, such as processor load, memory usage, network (bytes input/output), and disk utilization. For all practical purposes, the administrator needs more flexible and variable operation provided by Web front-end. For this purpose, this work developed a system that can satisfy above needs and compatible with Ganglia. The main steps are listed in the following: 1. Dump the contents of a RRD file [17, 20] to XML format: The following shell script is used to dump the contents of a RRD file to XML format. #!/bin/sh for i in `*.rrd` do rrdtool dump ${i} > ${i}.xml done

2. Convert the XML output of an RRD file to JRobin RrdDB format - RrdDb is a class of JRobin, and it provides a constructer used to create new RRD from XML dump. This class is listed as belows. public static void xml2JRrd(String name) { String xml = name + ".xml"; String jrrd = name + ".jrrd"; RrdDb db = new RrdDb(jrrd, xml); db.close(); }

3. Render the graph of JRobin RrdDB by RrdGraphDef: The following codes show an example of rendering an image from JRobin

359

/* do render graph */ RrdGraph rrdGraph = new RrdGraph(def);

RrdDB that contains processor load information and the output graph of CPU loading is shown in Figure 5.

}

public static void cpuReport(String rrd_dir, String hostname, long start_time, long end_time, String img_file) { String rd = rrd_dir; // rrd file dir /* start of RrdGraphDef */ RrdGraphDef def = new RrdGraphDef(); /* definition of graph */ def.setMaxValue(100); def.setMinValue(0); def.setRigid(true); def.setVerticalLabel("Percent"); def.setTimeSpan(start_time, end_time); def.setTitle(hostname + " CPU last " + getRange(end_time - start_time)); def.setAntiAliasing(true); def.setFilename(img_file); def.setWidth(WIDTH); def.setHeight(HEIGHT); def.setLazy(LAZY);

Figure 5. A CPU load visual graph of Node gamma2 Furthermore, this work developed a system that can satisfy above needs and compatible with NWS for extracting network bandwidth. The main steps of Rendering Network Information Graph with JRobin are listed in the following: 1. Create a JRobin RrdDB for a domain: Each JRobin RrdDB file responses for a domain and each JRobin RrdDB file using constructor to create new RRD object from the definition.

/* definition of datasource */ def.datasource("cpu_user", rd + "/cpu_user.rrd.jrrd", "sum", "AVERAGE"); def.datasource("cpu_nice", rd + "/cpu_nice.rrd.jrrd", "sum", "AVERAGE"); def.datasource("cpu_system", rd + "/cpu_system.rrd.jrrd", "sum", "AVERAGE"); def.datasource("cpu_wio", rd + "/cpu_wio.rrd.jrrd", "sum", "AVERAGE"); def.datasource("cpu_idle", rd + "/cpu_idle.rrd.jrrd", "sum", "AVERAGE");

public void addRrd(String domainname, String[] links) { String jrrd = domainname + ".jrrd"; String head = null; String tail = null; RrdDef def = new RrdDef(this.nwsrrds_root + "/" + jrrd); def.setStep(this.step); for (int i = 0; i < links.length; i++) { head = links[i].split(" ")[0]; tail = links[i].split(" ")[1]; def.addDatasource( head + "." + tail + "_b", "GAUGE", 600, 0.0, Double.NaN); def.addDatasource( head + "." + tail + "_l", "GAUGE", 600, 0.0, Double.NaN); } def.addArchive("MIN", 0.5, 1, 603); def.addArchive("MIN", 0.5, 6, 603); def.addArchive("MIN", 0.5, 24, 603); def.addArchive("MIN", 0.5, 288, 800);

def.area("cpu_user", CPU_USER, "User CPU"); def.gprint("cpu_user", "AVERAGE", " avg: %6.2f\\l"); def.stack("cpu_nice", CPU_NICE, "Nice CPU"); def.gprint("cpu_nice", "AVERAGE", " avg: %6.2f\\l"); def.stack("cpu_system", CPU_SYSTEM, "System CPU"); def.gprint("cpu_system", "AVERAGE", "avg: %6.2f\\l"); def.stack("cpu_wio", CPU_WIO, "Wait CPU"); def.gprint("cpu_wio", "AVERAGE", " avg: %6.2f\\l"); def.stack("cpu_idle", CPU_IDLE, "Idle CPU"); def.gprint("cpu_idle", "AVERAGE", " avg: %6.2f\\l"); def.comment("- " + new Date() + " \\r");

def.addArchive("AVERAGE", def.addArchive("AVERAGE", def.addArchive("AVERAGE", def.addArchive("AVERAGE", 800); def.addArchive("MAX", def.addArchive("MAX", def.addArchive("MAX", def.addArchive("MAX",

0.5, 0.5, 0.5, 0.5,

RrdDb db = new RrdDb(def);

360

0.5, 0.5, 0.5, 0.5,

1, 603); 6, 603); 24, 603); 288,

1, 603); 6, 603); 24, 603); 288, 800);

}

2. Query measurement from NWS and update JRobin RrdDB file: The detail codes are listed as follows.

public static void rrd_file, String domain, long start_time, long img_file) {

public void updateDB(String[] members, NwsJRrd jrrd) { // links in a domain String[] links = this.getLinks(members); // valid members in a domain String[] mbrs = this.updateMembers(members); // query string String[] q = this.getFilename(mbrs); String[] result = null; String[] ss = null; long now = Util.getTime(); long step = 0L; long stamp = 0L; double value = 0L; Vector rec = new Vector();

netReport(String end_time,

String

/* start of RrdGraphDef */ RrdGraphDef def = new RrdGraphDef(); /* definition of graph */ def.setMinValue(0); def.setRigid(true); def.setVerticalLabel("(Mbit/second)"); def.setTimeSpan(start_time, end_time); def.setTitle(domain.replace(".", "-") + " Network last " + getRange(end_time - start_time)); def.setAntiAliasing(true); def.setFilename(img_file); def.setWidth(WIDTH); def.setHeight(HEIGHT); /* definition of datasource */ RrdDb db = new RrdDb(rrd_file, true); int maxLength = -1; for (int i = 0; i < db.getDsCount(); i++) { Datasource ds = db.getDatasource(i); String dsn = ds.getDsName(); if (dsn.endsWith("_b")) { if (dsn.length() > maxLength) { maxLength = dsn.length(); } } } for (int i = 0; i < db.getDsCount(); i++) { Datasource ds = db.getDatasource(i); String dsn = ds.getDsName();

/* Query Bandwidth Info. from NWS */ for (int i = 0; i < q.length; i++) { if (!q[i].contains("null")) { result = memory.retrieve(q[i], 1); if (result.length != 0 && result != null) { ss = result[0].trim().split("\\s+"); stamp = Long.parseLong(ss[0]); step = now - stamp; value = (step < this.period) ? new Double(ss[1]) : 0L; } else { value = 0L; } } else { value = 0L; } rec.add(value); }

if (dsn.endsWith("_b")) { String space = ""; for (int j = 0; j < (maxLength dsn.length()); j++) { space += " "; } String sn = dsn.replace("_b", "").trim(); def.datasource(sn, rrd_file, dsn, "AVERAGE"); def.line(sn, getRandomColor(), sn.replace(".", "~")); def.gprint(sn, "MIN", space + "min: %6.2f"); def.gprint(sn, "AVERAGE", "avg: %6.2f"); def.gprint(sn, "MAX", "max: %6.2f\\l"); }

/* Update RrdDB */ RrdDb db = jrrd.getDb(); Sample sample = db.createSample(now); int discount = db.getRrdDef().getDsCount(); for (int i = 0; i < dscount; i++) { sample.setValue(i, rec.get(i)); } sample.update(); db.close(); }

3. Render the graph of JRobin RrdDB by RrdGraphDef: This is an example of rendering graph from JRobin RrdDB that contains network information about a domain. A diagram example of network bandwidth is shown in Figure 6.

} db.close(); def.comment("- " + new Date() + " -\\r"); /* do rendering graph */ RrdGraph rrdGraph = new RrdGraph(def); }

361

Figure 6. A monthly network graph of betadomain

Figure 7. The enhanced design of previous work

5. Experimental results

This subsection describes experimental results of network information model (NIM) and dynamic network information model (DNIM). Two clusters, eta and beta, are used in this experiment. We transfer a 5GB file from eta1 to beta1 during time period between the 20th until 30th timestamp, and the bandwidth between eta2 and beta1 are observed in every 60 seconds. Figure 8 depicts that the bandwidth of the connection from eta2 to beta1 obtained by NIM is a smooth curve, which cannot reflect the actual situation. But DNIM can present the variation of the link. Figure 9 depicts an unstable fluctuation of the error rate of NIM, providing broker an unstable information reference and causing broker to make wrong decisions.

The previous work reduced the number of bandwidth measurement between all Grid Nodes, but it lacks network information between two Nodes other than the head Node located in two different Sites other than the head Node. For example, the bandwidth measurement between Nodes A2 and B3 not performed in the previous model. For solving the above need, this work enhanced the previous model by increasing a switching mechanism. We call it the dynamic domain-based network information model which is shown in Figure 7. The principal improvement is switching the head Node to the next free Node of a Site. For example, when Node A1 is busy, the head Node of Site A will be the next free Node A2, which will conduct the bandwidth measurement between itself and Nodes B3, C2, and D4, if they are the free Nodes of their own Site respectively. There are three obvious advantages in using this model. z First, the number of bandwidth measurement can be still reduced in the same as the previous static model. z Second, the bandwidth measurement between two arbitrary Nodes in two different Sites can obtain easily. z Finally, the bandwidth measurement obtains real values instead of estimation values of a network. That is, the Resource Broker is useful in scheduling jobs with multi-site condition.

6. Conclusions This paper is presented to help the user make better use of the grid resources available. This paper will look at the use of information services in a grid and discuss the monitoring use of the Ganglia toolkit to enhance the information services already present in the Globus environment. Our grid resource brokerage system discover and evaluate grid resources, and make informed job submission decisions by matching a job’s requirements with an appropriate grid resource to meet budget and deadline requirements.

362

eta2 -> beta1

DNIM v.s. NIM

[6] V. Laszewski, I. Foster, J. Gawor, and P. Lane, “A Java commodity grid kit,” Concurrency and Computation: Practice and Experience, 2001, vol. 13, pp. 645-662. [7] H. Le, P. Coddington, and A.L. Wendelborn, “A DataAware Resource Broker for Data Grids,” IFIP International Conference on Network and Parallel Computing (NPC’2004), LNCS, 3222 Springer-Verlag, Oct. 2004. [8] C.T. Yang, C.L. Lai, K.C. Li, C.H. Hsu, and W.C. Chu, “On Utilization of the Grid Computing Technology for Video Conversion and 3D Rendering,” Parallel and Distributed Processing and Applications: Third International Symposium, ISPA 2005, Lecture Notes in Computer Science, vol. 3758, pp. 442-453, SpringerVerlag, Nov. 2005. [9] C.T. Yang, P.C. Shih, and K.C. Li, “A High-Performance Computational Resource Broker for Grid Computing Environments,” Proceedings of the International Conference on AINA’05, vol. 2, pp. 333-336, Taipei, Taiwan, March 2005. [10] C.T. Yang, K.C. Li, W.C. Chiang, and P.C. Shih, “Design and Implementation of TIGER Grid: an Integrated Metropolitan-Scale Grid Environment,” th Proceedings of the 6 IEEE International Conference on PDCAT’05, pp. 518-520, Dec. 2005. [11] J. Nabrzyski, J.M. Schopf, and J. Weglarz, Grid Rrsource Management, Kluwer Academic Publishers, 2005. [12] S.M. Park and J.H. Kim, “Chameleon: A Resource Scheduler in a Data Grid Environment,” Proceedings of rd the 3 IEEE/ACM International Symposium on Cluster Computing and the Grid, pp. 258-265, May 2003. [13] C.T. Yang, C.L. Lai, P.C. Shih, and K.C. Li, “A Resource Broker for Computing Nodes Selection in Grid Environments,” Grid and Cooperative Computing - GCC rd 2004: 3 International Conference,, Lecture Notes in Computer Science, Springer-Verlag, vol. 3251, pp. 931934, Oct. 2004. [14] C.T. Yang, P.C Shih, S.Y. Chen, and W.C. Shih, “An Efficient Network Information Modeling using NWS for Grid Computing Environments,” Grid and Cooperative th Computing - GCC 2005: 4 International Conference, Lecture Notes in Computer Science, vol. 3795, pp. 287299, Springer-Verlag, Nov. 2005. [15] C.T. Yang, C.F. Lin, and S.Y. Chen, “A Workflowbased Computational Resource Broker with Information th Monitoring in Grids,” Proceedings of the 5 International Conference on Grid and Cooperative Computing (GCC 2006), IEEE CS Press, pp. 199-206, China, Oct. 2006. [16] Ganglia, http://ganglia.sourceforge.net/ [17] JRobin, http://www.jrobin.org/ [18] Network Weather Service, http://nws.cs.ucsb.edu/ewiki/ [19] TIGER, http://gamma2.hpc.csie.thu.edu.tw/ganglia/ [20] Tomcat, http://tomcat.apache.org/ [21] UniGrid, http://140.114.91.31/ganglia/

eta2 -> beta1 (NIM)

80 70 60

Mb/s

50 40 30 20 10 0 1

2

3

4

5

6 7 8 9 10 Time (1 minute/unit)

11

12

13

14

15

Figure 8. DNIM shows a better performance than NIM Error Rate of NIM (eta2 -> beta1)

Error Rate

450 400 350

%

300 250 200 150 100 50 0 1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

Time (minute/unit)

Figure 9. Error rate of NIM worse to 428.37%

References [1] K. Czajkowski, S. Fitzgerald, I. Foster, and C. Kesselman, “Grid Information Services for Distributed Resource Sharing,” Proceedings of the Tenth IEEE International Symposium on High-Performance Distributed Computing, IEEE press, 2001. [2] I. Foster and C. Kesselman, “The Grid 2: Blueprint for a nd New Computing Infrastructure,” Morgan Kaufmann, 2 edition, 2003. [3] I. Foster, “The Grid: A New Infrastructure for 21st Century Science,” Physics Today, 2002, vol. 55, no. 2, pp. 42-47. [4] I. Foster and N. Karonis, “A Grid-Enabled MPI: Message Passing in Heterogeneous Distributed Computing Systems,” Proceedings of 1998 Supercomputing Conference, 1998. [5] I. Foster and C. Kesselman, “Globus: A Metacomputing Infrastructure Toolkit,” International Journal of Supercomputer Applications, 1997, vol. 11, no. 2, pp. 115-128.

363