|
Computational Methods for Dynamic Scene Reconstruction
Navy STTR 2016.A - Topic N16A-T017 ONR - Ms. Dusty Lang - [email protected] Opens: January 11, 2016 - Closes: February 17, 2016 N16A-T017 TITLE: Computational Methods for Dynamic Scene Reconstruction TECHNOLOGY AREA(S): Battlespace, Information Systems ACQUISITION PROGRAM: Data Focused Naval Tactical Clouds (DFNTC) FNC; Also relevant to DCGS-N OBJECTIVE: Develop and demonstrate efficient and robust computational methods for 4D space-time reconstruction of dynamic scenes by integrating data from multiple imaging sensors and ancillary information when available. Also, develop the capability to browse the reconstructed scene from different viewpoints and at different levels of detail. DESCRIPTION: Proliferation of imaging sensors provides the opportunity for integrating image data to reconstruct a dynamic scene, namely, reconstructing both the static background and the actors (people, vehicles, animals) moving in the scene. The imaging sensors may be stationary such as webcams and security cameras installed on buildings, and mobile such as cameras mounted on ground and air vehicles, body-worn, and hand-held smart phones. While there have been substantial advances in 3D spatial reconstruction of static scenes from multiple viewpoints, especially scenes with distinctive landmarks, 4D space-time reconstruction of moving objects has lagged behind. The three main technical challenges for reconstructing 4D space-time scenes include (i) determining correspondences for dynamic features in multiple cameras and images, (ii) reconstructing moving 3D features which may be sparse and have gaps, and (iii) space-time alignment of moving cameras with respect to the static scene. These challenges are compounded by the fact that images from cameras are taken from vastly different and changing viewpoints and have different resolutions and qualities due to variations in distance, intrinsic camera parameters, motion blur, illumination, and occlusion. We want to develop automated methods for 4D space-time reconstruction of dynamic scenes, in particular for scenes that are extended in space and events that have long durations. We also want to develop appropriate data structures and visualization methods to (iv) enable interactive browsing of the reconstructed 4D scene. The scenario is that there is a central processing place, where it receives and processes imagery from all the cameras. PHASE I: Develop robust computational methods/algorithms for reconstruction of the 3D stationary background and the 4D space-time of moving entities. Demonstrate the feasibility of the algorithms using data from a small number of cameras (at least one of which is hand-held or body-worn) in a relatively benign urban scene with few moving entities. Estimate the scalability of the reconstruction methods to crowded scenes with many cameras and many actors moving in extended spaces over longer time periods. Also estimate the computing and storage requirements as a function of complexity of scenarios and processing time. PHASE II: Based on Phase I effort, further develop algorithms for 4D space-time scene reconstruction by integrating images and video taken by many stationary and moving cameras. Demonstrate the performance of the algorithms applied to crowded scenes with many moving actors, in large spaces over long durations. Develop simple models of actor�s behaviors and reasoning about their motion to fill potential gaps in the data coverage. Develop metrics for assessing the quality of images and whether their use would enhance or degrade the 4D scene reconstructions. Develop methods for integrating ancillary data, such as existing imagery and maps or reports that may also be available to improve the reconstructions. Develop the data-structure and visualization methods for interactive browsing of the 4D reconstructed scenes from arbitrary viewpoints and at different spatial and temporal levels of detail. The prototype system should be evaluated with publicly available data from urban street scenes. PHASE III DUAL USE APPLICATIONS: Refine the 4D space-time reconstruction algorithms and interactive browsing methods into a final product that can be used by the Navy. Develop plans to transition a fully functional system to defense, security or law enforcement agencies for applications in after-action reviews and forensic investigations, and real-time surveillance, monitoring, and mission planning. The system should be further developed and refined according to the computational platform specifications of the intended agencies, and evaluated with publicly available data from crowded scenes and events such as fairs and sporting events. Potential applications of this topic are in defense, security agencies both government and private, and law enforcement. This technology will primarily support forensic investigations, after-action analyses, and real-time planning of actions and monitoring of events and activities. REFERENCES: 1. H. Joo, H.S. Park, Y. Sheikh, "Optimal Visibility Estimation for Large-Scale Dynamic 3D Reconstruction," in CVPR 2014. 2. R.A. Newcombe, D. Fox, S.M. Seitz, "DynamicFusion: Reconstruction and Tracking of Non-rigid Scenes in Real-Time," in CVPR 2015. 3. M. Pollefeys, L. Van Gool, et al., "Visual modeling with a hand-held camera," in International Journal of Computer Vision, 59(3): 207-232, 2004. 4. N. Snavely, S. Seitz, R. Szeliski, "Photo tourism: Exploring Photo Collections in 3D," in ACM Transactions on Graphics, 25(3): 835-846, 2006. 5. Y. Tian, S.G. Narasimhan, "Globally Optimal Estimation of Nonrigid Image Distortion," in International Journal of Computer Vision, 98(3): 279-302, 2012. KEYWORDS: Scene reconstruction; images and video; ad hoc network of cameras; 4D space-time reconstruction; dynamic scene; interactive browsing TPOC-1: Behzad Kamgarparsi Email: [email protected] TPOC-2: Predrag Neskovic Email: [email protected] Questions may also be submitted through DoD SBIR/STTR SITIS website.
|