Correctly implementing Lens::do_project()


I’ve been trying to implement a non-linear lens (see previous post here — simplified my problem since then).

Things are not working the way I’m expecting them to and that makes me think I misunderstand what the method do_project() in class Lens is supposed to do. The code I have for that method is this:

bool PhisheyeLens::
do_project(const Lens::CData *lens_cdata, const LPoint3 &point3d, LPoint3 &point2d) const {
  // First, account for any rotations, etc. on the lens.
  LVector3 v2 = point3d * do_get_lens_mat_inv(lens_cdata) * do_get_projection_mat(lens_cdata);

  PN_stdfloat dist = v2.length();
  PN_stdfloat hangle = catan2(v2[0],v2[2]);
  PN_stdfloat vangle = catan2(v2[1],v2[2]);

    (transform_angle_to_film_horizontal(hangle, vangle),
     transform_angle_to_film_vertical(hangle, vangle),
     // Z is the distance scaled into the range (1, -1).
     (do_get_near(lens_cdata) - dist) / (do_get_far(lens_cdata) - do_get_near(lens_cdata))

  // Now we have to transform the point according to the film
  // adjustments.
  point2d = point2d * do_get_film_mat(lens_cdata);

  bool retval =
    point2d[0] >= -1.0f && point2d[0] <= 1.0f &&
    point2d[1] >= -1.0f && point2d[1] <= 1.0f;

  return retval;

This is based on FisheyeLens. transform_angle_to_film_horizontal and transform_angle_to_film_vertical are functions which take an angle around the y-axis and an angle around the x-axis (in radians), and return the x and y coordinate this is projected to on a film whose lower left coordinate is (-1,-1) and whose upper right coordinate is (1,1).

What I’m getting is seemingly random distorted patches of the scene on a gray background. I’m relatively sure that the transform_angle… functions do what I want them to. Now I’m wondering if that is actually the right thing.

As I said, I’ve adapted the code from FisheyeLens. Some details about that confuse me:

  • Why do I even set the third coordinate of point2d if I want to specify a coordinate on a 2D film? The method comment says some lenses do that, but what does it mean and does it matter?
  • I’m not sure I understand what the first line of the method (multiplying point3d by mat_inv and projection_mat) and the line where I multiply point2d with film_mat do. I’m especially confused about whether I should really first apply film_mat and then check whether the coordinates are in [-1,1].