Research in Computer Vision

The objective is to perform scientific contributions in these different topics in order to build fast, robust and reliable vision-based medical and industrial applications.

Contact Person and Group Coordination

Slobodan Ilic	Nassir Navab

Research Projects in Computer Vision

Activity Recognition and Pose Estimation using Wearable Inertial Sensors We propose a method for human full-body pose tracking from measurements of wearable inertial sensors. Since the data provided by such sensors is sparse, noisy and often ambiguous, we use a compound prior model of feasible human poses to constrain the tracking problem. Our model consists of several low-dimensional, activity-specific motion models and an efficient, sampling-based activity switching mechanism. We restrict the search space for pose tracking by means of manifold learning. Together with the portability of wearable sensors, our method allows us to track human full-body motion in unconstrained environments. In fact, we are able to simultaneously classify the activity a person is performing and estimate the full-body pose. Experiments on movement sequences containing different activities show that our method can seamlessly detect activity switches and precisely reconstruct full-body pose from the data of only six wearable inertial sensors.

Intensity-based Tracking In this project, we are interested in the fundamental problem of simultaneously tracking a piecewise-planar scene and accurately estimating the 3D displacement of the camera. The objective is to have an algorithm with a higher convergence frequency and a higher convergence rate than standard optimization methods while having an equivalent computational complexity. Since the tracking is aimed to be used in industrial real-time augmented reality applications, in addition to the robustness and the speed performance, the algorithm should be very precise. For some applications, we are investigating alternatives where an off-line learning step improves the efficiency and the convergence properties of the tracking algorithm.

Human Motion Analysis using Time-of-Flight (ToF) Cameras Time-of-Flight cameras provide a novel means of acquiring depth information in a scene in real-time. In this project, we focus on analyzing human motion using this imaging device. We develop methods for capturing the full-body pose in 3D form ToF? camera data and for recognizing activities performed by a person. In particular, we investigate learning-based approaches that allow us to predict full-body poses and classify performed activities based on prior knowledge learned in a training phase. We also explore methods for direct, discriminative pose estimation without relying on a training phase.

Stereo Time-of-Flight This paper describes a novel method to acquire depth images based on combining a pair of ToF? (Time of Flight) cameras. As opposed to approaches that register point clouds posterior to the image acquisition, we propose to combine the measurements of the two cameras at the acquisition level. To do so, we actively control the infrared lighting of the scene such that the two cameras emit and measure the reflected infrared light first one after the other, and then simultaneously. Assuming the scene is static during these three stages and gathering the obtained depth measurements in each state, we derive a set of constraints that allow us to optimize the two depth images. In terms of hardware requirements, the cameras need to have access to the control of the infrared lights and should work with the same infrared wavelength and with exactly the same modulation frequency. A quantitative evaluation of the performance of the proposed method for different objects and setups is provided based on a simulation of the ToF? cameras. Results on real images are also provided. In both simulation and real images the stereo-ToF acquisition is able to produce more accurate depth measurements.

Catheter Extraction in Abdominal Fluoroscopic Image Sequences This work's focus lies in catheter and guide wire extraction from abdominal fluoroscopic sequences. Due to the fact that fluoroscopic X-ray images are of low quality and suffer from a lot of background clutter in the abdominal area the task we are working on is very difficult and not yet solved by the community. The detection process is very important since a properly detected catheter or guide wire is required by many applications that have been proposed in the last few years. One of the major goals is the enhancement of the navigation during abdominal cathterizations in order to reduce the time of interventions and thus the radiation exposure for the patient and especially the physician.

Deformable Guide Wire Tracking These days, X-ray fluoroscopic imaging is the modality used most widely to guide physicians in angiographic interventions. Current guidance procedures are based on a roadmap, e.g. a digital subtracted angiography (DSA) acquired from a fixed viewpoint and the injection of contrast agent to visualize the current vessel anatomy. Such roadmaps cannot directly be blended with the fluoroscopic sequence due to misalignment caused by respiratory motion. A fundamental step toward a successful integration of any navigation application into clinical routine is the estimation and compensation of respiratory motion. We propose a novel method for deformable tracking of multiple ridge segments based on a MAP-MRF formulation and efficient discrete optimization. Ridges are modeled as B-Spline curves which allows us to apply smooth deformations based on a minimal number of parameters. Our tracking framework is well adapted to the noisy domain of fluoroscopic imaging and the handling of multiple segments.

Real-Time 3D Reconstruction We are working on a real-time 3D reconstruction system aimed at recovering the 3D shape of objects inside a working area using only camera images. The working area is observed by 16 cameras mounted on the ceiling. Using these images we reconstruct an occupancy map and extract individual objects. The long-term goal of the project is to use this occupancy map to detect possible collisions between a robot placed in the working area and other objects in its path.

Hybrid Tracking In this project a complete real-time model-based tracking system for piecewise-planar objects which combines template-based and feature-based approaches is being developed. The main contributions are an extension to the ESM algorithm used for template-based tracking and the formulation of a feature-based tracking approach, which is specifically tailored for use in a real-time setting. In order to cope with highly dynamic scenarios, such as illumination changes, partial occlusions and fast object movement, the system adaptively switches between template-based tracking, feature-based tracking and a global initialization phase. The tracking system achieves real-time performance by applying a coarse-to-fine optimization approach and includes means to detect a loss of track.

Deformable Shape Matching This project aims a real-time detection of two dimensional deformable patterns for industrial inspection. The fast, robust and accurate localization of a given 2D object template in images has been a research topic since many decades. The results of this research efforts has enabled numerous different industrial applications, as the detection of the objects pose is the natural prerequisite for any senseful operation like counting, defect detection or pick and place operations. In case that the object to be localized is rigid or articulated, different successful approaches have been proposed and are applied in everyday industrial practice. Various applications domains, which necessitate the detection of deformable objects, like medical, packaging, textile or paper industry, can still not be comprehensively solved. The goal of current research is the development of a method which is fast, robust to illumination changes, occlusion and clutter plus allows the objects shape to deform.

Discrepancy Check Using Augmented Reality Construction companies employ CAD software during the planning phase but what is finally built often does not match the original plan. The procedure of validating the model is called 'discrepancy check'. The system proposed here allows the user to easily obtain an augmentation in order to find differences between the planned 3D model and the built items. The main difference to previous body of work in this field is the emphasis on usability and acceptance of the solution. While standard image-based solutions use markers or rely on a 'perfect' 3D model to find the pose of the camera; our software uses Anchor-Plates. Anchor-Plates are rectangular structures installed on walls and ceiling in the majority of industrial edifices. We are using them as landmarks because they are the most reliable components often used as reference coordinates by constructors. Furthermore; for real industrial applications; they are the most suitable solutions in terms of general applicability. For this project we try to bring a innovative solution to the discrepancy check procedure by applying Augmented Reality.

Vision Targeted CAD Models This project investigates vision features and their intelligent application on known CAD models for pose estimation. The advantage of combining CAD models and vision features lies in the known 3D geometry of the CAD model. Exemplary vision features are e.g. SIFT features, Randomized Trees, SURF features and other well known methods. The main goal is to use such an extended CAD model for pose estimation and for tracking initialization.

Endoscopic Video Processing The diagnosis and surveillance of several gastrointestinal (GI) diseases such as colateral or oesaphageal cancer are performed under endoscopic guidance. However, currently there exists no computer aided navigation tool to support these procedures. The goal of this project is to provide an advanced visualisation and automatic recognition tool in order to support the navigation and targeting in GI endoscopic procedures. The aim of the project is twofold: reconstructing the organ surface from endoscopic videos using monoSLAM and automatically recognising the biopsy locations to support the navigation and targeting in endoscopy.

Workflow Analysis Using 4D Reconstruction Data This project targets the worklow analysis of an interventional room equipped with 16 cameras fixed on the ceiling. It uses real-time 3D reconstruction data and information from other available sensors to recognize objects, persons and actions. This provides complementary information to specific procedure analysis for the development of intelligent and context-aware support systems in surgical environments.

Related Publications

2012
	A. Ladikos, W. Wein Geometric calibration using bundle adjustment for cone-beam computed tomography devices SPIE Medical Imaging, February 2012, San Diego, USA. (bib)
2011
	S. Hinterstoisser, C. Cagniart, S. Ilic, P. Sturm, N. Navab, P. Fua, V. Lepetit Gradient Response Maps for Real-Time Detection of Texture-Less Objects IEEE Transactions on Pattern Analysis and Maschine Intelligence (TPAMI). (bib)
	J. Bohren, R. B. Rusu, E. G. Jones, E. Marder-Eppstein, C. Pantofaru, M. Wise, L. Mösenlechner, W. Meeussen, S. Holzer Towards Autonomous Robotic Butlers: Lessons Learned with the PR2 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, May 2011. (bib)
	S. Hinterstoisser, S. Holzer , C. Cagniart, S. Ilic, K. Konolige, N. Navab, V. Lepetit Multimodal Templates for Real-Time Detection of Texture-less Objects in Heavily Cluttered Scenes (Oral) IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, November 2011. (bib)
	S. Hinterstoisser, V. Lepetit, S. Benhimane, P. Fua, N. Navab Learning Real-Time Perspective Patch Rectification International Journal of Computer Vision (IJCV), Springer Verlag, The final publication is available at www.springerlink.com (bib)
	, V. Haltakov , Fast and Robust Object Detection in Household Environments Using Vocabulary Trees with SIFT Descriptors In IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Workshop on Active Semantic Perception and Object Search in the Real World, 2011 (bib)
	C. Unger, E. Wahl, S. Ilic Efficient Stereo Matching for Moving Cameras and Decalibrated Rigs Intelligent Vehicles (IV), 6 - 8 June, Baden-Baden, Germany 2011 (bib)
	C. Unger, E. Wahl, S. Ilic Efficient Stereo and Optical Flow with Robust Similarity Measures 33rd Annual Symposium of the German Association for Pattern Recognition, Frankfurt am Main, Germany, August 30th - September 2nd 2011 (bib)
	S. Lieberknecht, S. Benhimane, S. Ilic Simultaneous Reconstruction and Tracking of non-planar Templates 33rd Annual Symposium of the German Association for Pattern Recognition, Frankfurt am Main, Germany, August 30th - September 2nd 2011 (bib)
	P. Schroeder, A. Bartoli, P. Georgel, N. Navab Closed-Form Solutions to Multiple-View Homography Estimation IEEE Workshop on Motion and Video Computing (WMVC), Kona, HI, USA, January 2011 (bib)
2010
	C. Cagniart, E. Boyer, S. Ilic Probabilistic Deformable Surface Tracking From Multiple Videos 11th European Conference on Computer Vision (ECCV), Crete, Greece, September 2010. (bib)
	A. Ladikos, E. Boyer, N. Navab, S. Ilic Region Graphs for Organizing Image Collections ECCV Workshop on Reconstruction and Modeling of Large-Scale 3D Virtual Environments Workshop, September 2010 , Crete, Greece. (bib)
	A. Ladikos, C. Cagniart, R. Gothbi, M. Reiser, N. Navab Estimating Radiation Exposure in Interventional Environments Medical Image Computing and Computer-Assisted Intervention (MICCAI), September 2010 , Beijing, China. (bib)
	S. Holzer , S. Ilic, N. Navab Adaptive Linear Predictors for Real-Time Tracking IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California (USA), June 2010. (bib)
	N. Alt, S. Hinterstoisser, N. Navab Rapid Selection of Reliable Templates for Visual Tracking IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California (USA), June 2010. (bib)
	S. Hinterstoisser, V. Lepetit, S. Ilic, P. Fua, N. Navab Dominant Orientation Templates for Real-Time Detection of Texture-Less Objects IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California (USA), June 2010. (bib)
	C. Cagniart, E. Boyer, S. Ilic Free-From Mesh Tracking : a Patch-Based Approach IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), San Francisco, California (USA), June 2010. (bib)
	E. Wahl, C. Unger, A. Zeller, D. Rossberg 3D-Environment Modeling as an Enabler for Autonomous Vehicles ATZ Automobiltechnische Zeitschrift, Ausgabe 02/2010. (bib)
2009
	S. Hinterstoisser, O. Kutter, N. Navab, P. Fua, V. Lepetit Real-Time Learning of Accurate Patch Rectification (Oral Presentation) IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Miami, Florida (USA), June 2009. (bib)
	P. Georgel, A. Bartoli, N. Navab Simultaneous In-Plane Motion Estimation and Point Matching Using Geometric Cues Only IEEE Workshop on Motion and Video Computing (WMVC), Snowbird, Utah, USA, December 7-10 2009 (bib)
	P. Georgel, S. Benhimane, J. Sotke, N. Navab Recovering the Full Pose from a Single Keyframe IEEE Workshop on Applications of Computer Vision (WACV) , Snowbird, Utah, USA, December 7-8 2009 (bib)
	F. Schweiger, B. Zeisl, P. Georgel, G. Schroth, E. Steinbach, N. Navab Maximum Detector Response Markers for SIFT and SURF In Vision, Modeling and Visualization Workshop (VMV), Braunschweig (bib)
	P. Georgel, P. Schroeder, N. Navab Navigation Tools for Augmented CAD Viewing IEEE Computer Graphic and Application (CGA), special issue on 3D User Interfaces (bib)
	P. Georgel, S. Benhimane, J. Sotke, N. Navab Photo-based Industrial Augmented Reality Application Using a Single Keyframe Registration Procedure The 8th IEEE and ACM International Symposium on Mixed and Augmented Reality, Orlando, US, Oct. 19 - 22, 2009 (bib)
	B. Zeisl, P. Georgel, F. Schweiger, E. Steinbach, N. Navab Estimation of Location Uncertainty for Scale Invariant Feature Points British Machine Vision Conference (BMVC), London (UK), September 7-10, 2009 (bib)
	C. Unger, S. Benhimane, E. Wahl, N. Navab Efficient Disparity Computation without Maximum Disparity for Real-Time Stereo Vision British Machine Vision Conference (BMVC), London (UK), September 7-10, 2009 (bib)
	A. Bigdelou, A. Ladikos, N. Navab Incremental Visual Hull Reconstruction British Machine Vision Conference (BMVC), London (UK), September 7-10, 2009 (bib)
	S. Holzer , S. Hinterstoisser, S. Ilic, N. Navab Distance Transform Templates for Object Detection and Pose Estimation IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Miami, Florida (USA), June 2009. (bib)
	A. Hofhauser, Carsten Steger, N. Navab Perspective planar shape matching IS&T/SPIE Electronic Imaging, San Jose, California, USA, January 2009. (bib)
2008
	P. Georgel, P. Schroeder, S. Benhimane, M. Appel, N. Navab How to Augment the Second Image? Recovery of the Translation Scale in Image to Image Registration The 7th IEEE and ACM International Symposium on Mixed and Augmented Reality, Cambridge, UK, Sept. 15 - 18, 2008. (bib)
	S. Hinterstoisser, S. Benhimane, V. Lepetit, P. Fua, N. Navab Simultaneous Recognition and Homography Extraction of Local Patches with a Simple Linear Classifier British Machine Vision Conference (BMVC), Leeds (UK), September 1-4, 2008 (bib)
	D. Zikic, B. Glocker, M. S. Hansen, A. Khamene, N. Navab Construction of Statistical Shape Models from Minimal Deformations Workshop "Manifolds in Medical Imaging: Metrics, Learning and Beyond" in conjunction with Medical Image Computing and Computer-Assisted Intervention (MICCAI 2008), New York City, NY, USA, September 2008 (bib)
	D. Zikic, M. S. Hansen, B. Glocker, A. Khamene, R. Larsen, N. Navab Computing Minimal Deformations: Application to Construction of Statistical Shape Models. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska (USA), June 24-26, 2008 (bib)
	M. S. Hansen, B. Glocker, N. Navab, R. Larsen Adaptive Parametrization of Multivariate B-splines for Image Registration IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska (USA), June 24-26, 2008 (bib)
	B. Glocker, N. Paragios, N. Komodakis, G. Tziritas, N. Navab Optical Flow Estimation with Uncertainties through Dynamic MRFs IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska (USA), June 24-26, 2008 (bib)
	A. Hofhauser, Carsten Steger, N. Navab Edge-based Template Matching and Tracking for Perspectively Distorted Planar Objects 4th International Symposium on Visual Computing, Las Vegas, Nevada, USA, December 2008. (bib)
	A. Ladikos, S. Benhimane, N. Navab High Performance Model-Based Object Detection and Tracking Computer Vision and Computer Graphics. Theory and Applications, Communications in Computer and Information Science, Vol. 21, Springer, 2008 (bib)
	A. Ladikos, S. Benhimane, N. Navab Multi-View Reconstruction using Narrow-Band Graph-Cuts and Surface Normal Optimization British Machine Vision Conference (BMVC), Leeds (UK), September 1-4, 2008 (bib)
	A. Ladikos, S. Benhimane, N. Navab Real-time 3D Reconstruction for Collision Avoidance in Interventional Environments Medical Image Computing and Computer-Assisted Intervention, MICCAI, 2008, New York, USA, September 6-10 2008 (bib)
	A. Bigdelou, A. Ladikos, N. Navab Shape from Silhouette in Dynamic Scenes Technical Report at Technical University of Munich (TUM), Munich (DE), August 15, 2008 (bib)
	A. Ladikos, S. Benhimane, N. Navab Efficient Visual Hull Computation for Real-Time 3D Reconstruction using CUDA IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (USA), June 2008. Workshop on Visual Computer Vision on GPUs (CVGPU) (bib)
	S. Hinterstoisser, S. Benhimane, N. Navab, P. Fua, V. Lepetit Online Learning of Patch Perspective Rectification for Efficient Object Detection IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (USA), June 2008. (bib)
	C. Unger, M. Groher, N. Navab Image Based Rendering for Motion Compensation in Angiographic Roadmapping IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Anchorage, Alaska (USA), June 24-26, 2008 (bib)
	A. Ladikos, S. Benhimane, M. Appel, N. Navab Model-Free Markerless Tracking for Remote Support in Unknown Environments International Conference on Computer Vision Theory and Applications, Funchal, Portugal, January 2008. (bib)
	A. Hofhauser, Carsten Steger, N. Navab Harmonic deformation model for edge based template matching International Conference on Computer Vision Theory and Applications, Funchal, Portugal, January 2008. (bib)
	S. Benhimane, H. Najafi, M. Grundmann , E. Malis, Y. Genc, N. Navab Real-time object detection and tracking for industrial applications International Conference on Computer Vision Theory and Applications, Funchal, Portugal, January 2008. (bib)
	A. Zaharescu, C. Cagniart, S. Ilic, E. Boyer, R. Horaud Camera Clustering for Multi-Resolution 3-D Surface Reconstruction ECCV 2008 Workshop on Multi Camera and Multi-modal Sensor Fusion Algorithms and Applications - 2008 (bib)
2007
	S. Hinterstoisser, S. Benhimane, N. Navab N3M: Natural 3D Markers for Real-Time Object Detection and Pose Estimation IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil, October 14-20, 2007 (bib)
	S. Benhimane, E. Malis Homography-based 2D Visual Tracking and Servoing Special Joint Issue IJCV/IJRR on Robot and Vision. Published in The International Journal of Robotics Research, Volume 26, No. 7, 661-676, July 2007. (bib)
	S. Benhimane, A. Ladikos, V. Lepetit, N. Navab Linear and Quadratic Subsets for Template-Based Tracking IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Minneapolis, Minnesota, June 2007 (bib)
	A. Ladikos, S. Benhimane, N. Navab A Real-Time Tracking System Combining Template-Based and Feature-Based Approaches International Conference on Computer Vision Theory and Applications, Barcelona, Spain, March 2007. (bib)
2006
	G. Panin, A. Ladikos, A. Knoll An Efficient and Robust Real-Time Contour Tracking System IEEE International Conference on Computer Vision Systems, New York, USA, January 2006. (bib)
2005
	F. Althoff, R. Lindl, L. Walchshäusl Robust Multimodal Hand- and Head Gesture Recognition for controlling Automotive Infotainment Systems VDI-Tagung - Der Fahrer im 21. Jahrhundert, Braunschweig, Germany, 21.11.2005 (bib)
1999
	G. Klinker, D. Stricker, D. Reiners An Optically Based Direct Manipulation Interface for Human-Computer Interaction in an Augmented World Computers and Graphics 23(6), 1999. (Also in Proc. Workshop EGVE'99, Austria, 1999.) (bib)
	G. Klinker Augmented Reality: A problem in need of many computer vision-based solutions NATO Advanced Research Workshop at the 8. International Conference on the Computer Analysis of Images and Patterns (CAIP-99), org. by A. Leonardis and R. Bajcsy. Ljubljana Slowenia, Aug. 29-31, 1999. (bib)
	G. Klinker, D. Stricker, D. Reiners Augmented Reality: A Balance Act between High Quality and Real-Time Constraints Proc. 1rst International Symposium on Mixed Reality (ISMR'99): Y. Ohta and H. Tamura (eds.), "Mixed Reality - Merging Real and Virtual Worlds", 1999. (bib)
1998
	D. Stricker, G. Klinker, D. Reiners A Fast and Robust Line-based Optical Tracker for Augmented Reality Applications Proc. 1rst International Workshop on Augmented Reality (IWAR'98), San Francisco, Nov. 1998, pp. 31-46. (bib)
	D. Reiners, D. Stricker, G. Klinker, S. Mueller Augmented Reality for Construction Tasks: Doorlock Assembly Proc. 1rst International Workshop on Augmented Reality (IWAR'98), San Francisco, Nov. 1998, pp. 31-46. (bib)
	G. Klinker, D. Stricker, D. Reiners The Use of Reality Models in Augmented Reality Applications Proc. European Workshop on 3D Structure from Multiple Images of Large-scale Environments (SMILE), in combination with ECCV'98, 1998. (bib)
1997
	D. Koller, G. Klinker, E. Rose, D. Breen, R. Whitaker, M. Tuceryan Real-time vision-based camera tracking for augmented reality applications Proc. ACM Symp on Virtual Reality, Software and Technology (VRST'97), 1997, pp. 87-94. (bib)
	D. Koller, G. Klinker, E. Rose, D. Breen, R. Whitaker, M. Tuceryan Automated camera calibration and 3D egomotion estimation for augmented reality applications Proc. 7th Int. Conf. on Computer Analysis of Images and Patterns (CAIP'97), 1997. (bib)
	G. Klinker, K. Ahlers, D. Breen, P.-Y. Chevalier, C. Crampton, D. Greer, A. Kramer, E. Rose, M. Tuceryan, R. Whitaker Confluence of Computer Vision and Interactive Graphics for Augmented Reality PRESENCE - Teleoperators and Virtual Environments, Special Issue on Augmented Reality 6(4), 1997, pp. 433-451. (bib)
1993
	G. Klinker A Physical Approach to Color Image Understanding Book published by AK Peters, Ltd., Wellesley MA, March 1993. (Based on Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, May, 1988) (bib)
1988
	G. Klinker A Physical Approach to Color Image Understanding Ph.D. thesis, Computer Science Department, Carnegie-Mellon University, May, 1988 (bib)

Working Group

Vasileios Belagiannis

Ali Bigdelou

Nicolas Brieu

Stefan Hinterstoisser

Stefan Holzer

Slobodan Ilic

Diana C. Mateus

Loic Peter

Loren Arthur Schwarz

Juergen Sotke