A Semi-Automatic Method for Resolving Occlusions in Augmented Reality
Constructing the 3D occluding boundary
Lets ignore the uncertainty on the viewpoints for the moment. Let C1
and C2 be the 2D boundaries of an occluding object outlined in two
consecutive key views. We build the 3D occluding boundary using stereo
triangulation : Let m1 be a point on C1. Its corresponding
point m2 in the other view is the intersection of the epipolar line
with C2. As usual, the order constraint is used to solve the ambiguity
of the correspondence problem.
Taking into account the uncertainty on the viewpoints while constructing
the 3D occluding boundary
Since we have curve correspondences, the point m2 actually depends on the
viewpoints in the two key views. To take into account the uncertainty on
these viewpoints, we resort to an exhaustive approach, and consider the
extremal
viewpoints, that are the vertices of the 6-dimensionnal indifference
ellipsoid. Let
(resp ) the extremal
viewpoints in the two key views. Let m1 be a point on C1.
Given an extremal viewpoint p1, we can compute the 12 possible reconstructions
of m1 with the 12 extremal views in key view 2:
Using the 12 extremal viewpoints in key view 1, we then obtain 12^2
reconstructions of m1. The convex hull of these 144 points is a
good approximation of the 3D reconstruction error of m1.
Taking into account the uncertainty on the viewpoint while reprojecting
the 3D occluding boundary
To estimate the 2D uncertainty on the projected boundary C, we have
to consider the 3D reconstruction error and the uncertainty on the consider
viewpoint. We again resort to an exhaustive method : for each point m
of C1, the 12^2 extremal reconstructions are projected onto the
current frame using the 12 extremal viewpoints of this frame. We define
the uncertainty on the predicted occluding boundary to m as the
convex hull of these 12^3 image points. This area is noted
in the following.
Real case : the point m1, the associated epipolar lines and the
reprojections.