International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XXXV, Part B3. Istanbul 2004
MMAüDaa | BawÜGenicacei Osa | ADS Proc Datz | Las Status |
ADS 200307038: 1
RIHRREREERIAL 1055 1 o conces deut
BEL roo cos Dana! 42280
E oe Save Messages. Scart: Pun May 9 16:55:03 2004 4
ADSZ00307038: | CCM. : =}
BIBHEEBEREH 100% +2 0 42 image: conclerse Fare: a VUE EN
NURANNNNNNEET 102 cnn Lone gi an e
e: i sui Time 30 wing 36 4ucı
Oe | SavaMessages || at | af
ero
Figure 2. GPro run in a Condor cluster
4. PRACTICAL RESULTS
Two recent small projects were selected to show the
timesavings in the computationally intensives processes. These
projects were done using a portion of the cluster setup at NWG.
The production cluster at NWG comprises of dedicated rack
mounted systems in conjunction with operator’s machines
during off hours. All the single runs were performed on a dual
Xeon 3.06 GHz with 3GB RAM running Windows XP. A
RAIDO disk array with 10 disks of 146GB SCSI drives was
used for all data input/output. Analysis of the average 1/0 queue
during runs showed that disk contention was not a serious issue
for a single machine run. The distributed runs were done on six
nodes of same hardware specification as the single run.
Interconnect between the nodes and file server was done using
gigabit Ethernet. Disk I/O analysis for the distributed runs
showed some disk contention.
The first dataset used is a small block from a project that was
part of the Florida statewide program undertaken in 2003/2004
by North West Geomatics and Earth Data. This block is roughly
one degree by one degree in size. It comprises of 256 USGS
Digital Ortho Quarter Quads (DOQQs), delivered in both color
and FCIR format. The numbers contained here are for color
data only, but one could simply double the Ortho times for both
datasets. This block was flown at 24,000 AGL with a 5-minute
line spacing at 340 knots. Seven control points were used iri the
block to provide a datum shift. The DEM was the USGS
format DEM converted into a 50m grid (flat terrain so the 50m
grid is more than sufficient).
The second dataset is a small test area over Zolloffo springs in
Florida, which is of environmental concern. It was flown on
850m line spacing to capture 0.13m data for a final product of
0.15m. Flight speed was 100 knots. Five control points were
available in this area. Both color and FCIR ortho photos were
also generated for this area. Table 1 summarizes the
characteristics of the projects and Table 2 lists the processing
time for the most time consuming processes of the workflow.
Florida Zolloffo
Capture GSD (m) 0.8 0.13
Output GSD (m) 1.0 0.15
Lateral Overlap (90) 30 30
Raw Data Size (GB) 110 21
Number of Lines 14 4
Average Line Length (km) 120 5
Project Area (km 5 11500 4.5
Table 1: Dataset Summary
Process Florida Zollofo
1 Node 6 Nodes ] Node | 6 Node
Rectification 11.27 1.92 2.34 0.60
Automatic Point 14.80 4.66 3.14 1.50
Matching
Ortho 14.65 2.54 3.02 0.78
Generation
Table 2: Processing Times in Hours
As could be seen in the table, even though the processing time
seem to decrease linearly, the cluster efficiency numbers drop
for Zollofo processing due to only four nodes being utilized
even though there were six available.
5. CONCLUSIONS
Distributed computing has enabled us to process large amounts
of data in a reasonable time. It has empowered large mapping
projects to be done in unprecedented turnaround times. Condor
with it high throughput performance has enabled us to deliver
products even in adverse condition when there have been node
failures without human intervention. We plan to extend our
usage of the Condor distribution model to other non-interactive
applications so that they could benefit from all the advantages
discussed above. However, in order to provide the scalability
required we would have to increase our granularity and
parallalize our applications.
6. REFERENCE
Ashton, D., Gropp, W., Lusk, E., 2002. Installation and User's
Guide to MPICH, a Portable Implementation of MPI Version
1.2.5. The ch nt device for workstations and clusters of
Microsoft Windows machines, http://www-
unix.mcs.anl.gov/mpi/mpich/ (accessed Feb. 2003).
Condor Team, 2003. Condor Manual,
http://www.cs.wisc.edu/condor/manual/v6.4/, pp. 7-8 (accessed
Feb. 2003).
Geist, A., et al., 1994. PVM: Parallel Virtual Machine A Users'
Guide and Tutorial for Networked Parallel Computing, MIT
Press, pp. 10-11.
Microsoft, 2001. Application Center 2000 Evaluation Guide,
http://www.microsoft.com/applicationcenter/evaluation/product
guide.asp, pp. 3-5 (accessed Feb. 2003).
Interna
Sandau
ADS4(
Archiv
Inform:
Part B1
Sterling
MIT Pr
Temple
Micros:
Tempel
the LH
Internat
and Sp
Vol. X]
