Due to the flexibility for extracting 3D line features from 
different data sources and the advantage of data fusion, the 
authors proposed, in this study, a scheme comprising 
Construct-Shape-Refine (CSR) three steps and developed a 
line feature-based geometric inference algorithm by fusing 3D 
line features with 2D image lines for building roof 
reconstruction. Figure 1 illustrates the flowchart of the proposed 
CSR method for the generation of 3D building roof models. 
2. 2D AND 3D LINE FEATURE EXTRACTION ENGINE 
At current stage where the accuracy of line features is not the 
main concern, we consider simple but reliable extraction tools 
for acquiring 3D line features from imagery and laser range 
data, respectively. The algorithm starts by extracting 3D line 
features from laser range data by using the semi-automatic 3D 
line feature extraction engine, which contains three major 
procedures: (1) interpolating the dispersed laser point clouds 
into regular grid data, then forming the range images; (2) 
automatically extracting 3D line features by using Canny 
operator with Hough transform; (3) considering the reliability 
for extracting these 3D line features, a user interface which 
fuses the human intervention was implemented to select the best 
3D line features obtained in last step and to make up for the 
deficient parts that resulted from automatic extraction, if any. 
The effectiveness of the above design can be illustrated through 
the demonstrations of Figures 2 and 3, where the blues lines 
present the 3D line features extracted by image processing and 
the red one is the feature selected by a user. For those line 
features of building roof that miss in the detection, the operator 
would undertake manual measurements, as the green line shown 
in Figure 3. By way of the semi-automatic extraction, complete 
3D line features of building roofs out of LIDAR point clouds 
can be obtained, as those black lines in Figure 4 illustrate. 
As for 2D line feature extraction from aerial imagery, the (2) 
and (3) steps for 3D line features extraction are utilized. 
CSR METHOLOGY 
After 3D line features are extracted and regarded as input data 
for the CSR algorithm, then the procedures of building roof 
reconstruction are performed through the following geometric 
inferences: (1) constructing the topological relationship of 3D 
line features that belong to the same building roof by using the 
special intersecting property of 3D line features when projected 
onto a plane; (2) shaping the initial building roof by means of 
adjusting the 3D line features, and compensating missing parts 
by the shortest path algorithm, if any, and reporting whether or 
not the investigated building roof is completed; (3) refining the 
building roof automatically or semi-automatically by 
integrating 2D line features observed from the images through 
2D and 3D geometric inference processes. The function of the 
automatic mode is not only to find the building boundaries from 
images by using the distance, angle and topological checks in 
images and object space, but also adjust the 3D line features 
obtained either from laser range data or aerial images; on the 
other hand, in the semi-automatic refined mode, there are three 
modules that are separated by the feature types of 
measurements, namely point-based, line-based and hybrid mode. 
The point-based model is the traditional photogrammetric 
approach for building reconstruction where the first step is to 
measure the conjugate points in the comer point order, either 
clockwise or counter-clockwise, then execute the point-based 
intersection followed by connecting the roof comers through a 
CAD design. The line-based mode is nevertheless based on line