N A 
ed on a 
and the 
view for 
ients the 
imber of 
deo see- 
analyse 
own that 
le in the 
jological 
R-work" 
order to 
from the 
'mented- 
york will 
led with 
ncluding 
bile duct 
struction. 
during 
ients and 
atomical 
dividual 
1ding the 
ed in the 
rocedure 
r, tumor, 
with the 
andatory 
e. Here 
surgical 
enhanced 
'eoscopic 
ost AR- 
d not for 
ating the 
h, 1998; 
2. NAVIGATION IN LIVER SURGERY WITH ARION™ 
Deutsches Krebsforschungszentrum and University Clinics 
Heidelberg have developed a prototype of an IGSS for 
application in oncological liver surgery. This IGSS will enable 
the surgeon to see her/his instrument in relation to important 
structures inside the liver. For the successful resection of tumors 
in oncological liver surgery (RO resection) the exact knowledge 
of the localization of the tumorous tissue and the surrounding 
security margin is necessary [Hassenpflug, 2001; Hassenpflug, 
2001b; Vetter, 2001]. The transfer of the preoperatively planned 
resection margins to the current situs is of interest. The complex 
structure of the intrahepatic vessels like the liver veins and the 
portal veins assist the surgeon to orientate inside the liver. 
Damaging a vessel that has to be preserved is life-threatening 
and a major intraoperative risk. The transfer of the preoperative 
anatomy and planning results to the intraoperative situs involves 
intraoperative image acquisition, registration with the 
preoperative data, deformation tracking and modeling, and the 
adequate presentation to the surgeon (cf. Fig. 1). 
  
  
  
  
  
  
  
Figure 1. Screenshot of ARION with view on a surface 
visualisation of a clipped liver with intrahepatic 
vessels, tumor, and a depiction of the virtual 
instrument 
We are building a prototype named ARION (Augmented 
Reality for Intra-Operative Navigation) to demonstrate the 
feasibility of image-guided liver surgery (Meinzer, 2002). It 
consists of five modules to realize the aforementioned visions: 
Module 1: 
Contrast agent enhanced images are acquired preoperatively and 
postprocessed by LENA, DKFZ's already clinically established 
computer-assisted surgical planning system. This module 
provides the portal and venous vessel tree in a mathematical 
graph representation, the tumor with ^ surrounding security 
margin and calculated resection planes. 
Module 2: 
Intraoperative vessel trees are reconstructed from three- 
dimensional freehand Doppler ultrasound-scans and represented 
in graph representation. The vessel graphs provide all the 
features necessary for registration. For Module 2 to 4 the liver is 
kept in position by jet ventilation and stabilised within the 
surrounding space with sterile cloth. (Glombitza, 2001) 
Module 3: 
Registration of the vessel trees by rigid pre- and elastic post- 
registration. The resulting deformation vector field is used to 
infer the deformation of the parenchyma. Now, the localisation 
of the virtual planning structures is known via the world 
coordinate system of the transmitter of the applied electro- 
magnetic tracking-system. (Vetter, 2001b) 
Module 4: 
Localisable navigation aids (NSA), consisting of a needle, a 
tracking sensor, and an anchor, are brought into the liver to keep 
its registered state after it has been released for parenchyma 
resection. Therefore, an adaptive transformation correction 
parameterised by the navigation aids' sensor values are used to 
update the deformation vector field in a volume of interest in 
order to sustain the registered state. (Vetter, 2002) 
Module 5: 
The tracked surgical instruments are visualised in respect to the 
transparent intrahepatic structures on an auto-stereoscopic flat- 
panel to achieve an adequate depth perception. 
3. AUGMENTED-REALITY TECHNIQUES 
The introduction of an augmented-reality technique like see- 
trough is an important component to find out the best 
visualisation technology for liver surgery. 
3.1 Display technologies 
The combination of real and virtual images into a single image 
could be realised with the following display technologies: 
-  Head-Mounted Display 
o Optical-See-Trough 
o Video-See-Trough 
o Microscope 
- Image Overlay Systems 
- Virtual Retinal Systems 
-  Monitor-AR-Systems 
- Direct Projection 
In our work we focus on wearable systems that means on head- 
mounted displays and virtual retinal displays. 
Head-mounted displays (HMD) have been widely used in 
virtual-reality systems. Augmented-reality researchers have 
been working with two types of HMD. These are called video 
see-through and optical see-through. The term see-through 
comes from the need for the user to be able to see the real world 
view that is immediately in front of him even when wearing the 
HMD. The HMD approach consists of viewing the outside 
world via a video camera fixed on a HMD. The video image is 
combined digitally with the computer generated image and 
displayed within the HMD. 
The virtual retinal display (VRD) is a new technology for 
creating visual images. It was developed at the Human Interface 
Technology Laboratory (HIT Lab) by Dr. Thomas A. Furness 
III. The VRD creates images by scanning low power laser light 
directly onto the retina (Viirre, 1998). 
The other systems came not into the questions because they are 
not able to serve the requirements of our application (Sect. 3.3). 
3.1.4 Head-Mounted Displays 
A. see-trough HMD is a device used to combine reality and 
virtuality. Standard closed HMDs do not allow any direct view 
of the real world. In contrast, a see-trough HMD lets the user 
see the real world with virtual objects superimposed by optical 
or video technologies. 
—275— 
  
We m 
AN" PIT M MESES