Project : 3D Reconstruction for Mobile Visual Search Application
Semester : Fall 2011 - Spring 2012
Course : Directed Research
Instructor : Prof. Gerard Medioni (http://iris.usc.edu/people/medioni/index.html)
I worked as a Student Worker for two semesters with Prof. Gerard Medioni, on 3D Reconstruction for Mobile Visual Search Application.
Given two or three input images, the system extracted planar surfaces of input object, rectified these planes to have synthesized frontal views, and performed recognition using both 2-D descriptors extracted from the rectified views and the 3-D spatial relationship of the 2-D descriptors. We used Homography relationships to extract dominant planes from multiple input images and RANSAC+SBA (RANdom Sample Consensus and Sparse Bundle Adjustment) to reconstruct a sparse 3-D model of matching points in the presence of outliers.
My major work involved :
- Creating and using synthetic data to test the performance of the current system
- Comparing and explaining the results of 3D-3D matching with 3D-2D matching and 2D-2D matching
- Trying methods to improve performance of the current system
- Tried Patch-based reconstruction
- Optical flow for determining change in orientation
Following image shows pairs of input images and the rectified front views extracted from them :
 |
| Rectified planes generated from 16 pairs of input images |
 |
| Closer examples of generated rectified planes |
Plots generated to show experimental results :
For Pair 1 as reference :
 |
| Plot showing number of matching points between the reference image and the images at different angles from the reference image |
 |
| Plot showing matching ratio between the reference image and images at different angles from the reference image |
For Pair 9 as reference :
 |
| Plot showing number of matching points between the reference image and the images at different angles from the reference image |
 |
| Plot showing matching ratio between the reference image and the images at different angles from the reference image |
Optical flow results (blue to red represents minimum to maximum displacement) :
 |
| Input Images |
 |
| Optical Flow result |
