Reprojection error

The reprojection error is a geometric error corresponding to the image distance between a projected point and a measured one. It is used to quantify how closely an estimate of a 3D point ${\displaystyle {\hat {\mathbf {X} }}}$ recreates the point's true projection ${\displaystyle \mathbf {x} }$. More precisely, let ${\displaystyle \mathbf {P} }$ be the projection matrix of a camera and ${\displaystyle {\hat {\mathbf {x} }}}$ be the image projection of ${\displaystyle {\hat {\mathbf {X} }}}$, i.e. ${\displaystyle {\hat {\mathbf {x} }}=\mathbf {P} \,{\hat {\mathbf {X} }}}$. The reprojection error of ${\displaystyle {\hat {\mathbf {X} }}}$ is given by ${\displaystyle d(\mathbf {x} ,\,{\hat {\mathbf {x} }})}$, where ${\displaystyle d(\mathbf {x} ,\,{\hat {\mathbf {x} }})}$ denotes the Euclidean distance between the image points represented by vectors ${\displaystyle \mathbf {x} }$ and ${\displaystyle {\hat {\mathbf {x} }}}$.

Minimizing the reprojection error can be used for estimating the error from point correspondences between two images. Suppose we are given 2D to 2D point imperfect correspondences ${\displaystyle \{\mathbf {x_{i}} \leftrightarrow \mathbf {x_{i}} '\}}$. We wish to find a homography ${\displaystyle {\hat {\mathbf {H} }}}$ and pairs of perfectly matched points ${\displaystyle {\hat {\mathbf {x_{i}} }}}$ and ${\displaystyle {\hat {\mathbf {x} }}_{i}'}$, i.e. points that satisfy ${\displaystyle {\hat {\mathbf {x_{i}} }}'={\hat {H}}\mathbf {{\hat {x}}_{i}} }$ that minimize the reprojection error function given by

${\displaystyle \sum _{i}d(\mathbf {x_{i}} ,{\hat {\mathbf {x_{i}} }})^{2}+d(\mathbf {x_{i}} ',{\hat {\mathbf {x_{i}} }}')^{2}}$

So the correspondences can be interpreted as imperfect images of a world point and the reprojection error quantifies their deviation from the true image projections ${\displaystyle {\hat {\mathbf {x_{i}} }},{\hat {\mathbf {x_{i}} }}'}$