BEV / IPM transformation of rendered scene

Scripting Issues
1

Hello everyone,

I’m trying to achieve the following and wasn’t able to make any progress so far :

  • Given a panda3d-rendered road image with lanes from a vehicles perspective (dash cam) I want to transform the image to a birds-eye view via the OpenCV cv2.warpPerspective function, so that, i.e., straight road lanes are parallel in the transformed image
  • To achieve this, I need to calculate the Inverse Perspective Mapping (IPM) matrix, which is defined as the inverse of IPM = (P * M) whereas P is the Projection Matrix = K[R|t] and M is a transformation mapping (naming should be equivalent to this slide deck)
    • basically, I want to be able to use a BEV transformation like implemented in the GitHub project Cam2BEV → ipm.py, but with automatic calibration with the information I can get out of panda3d

Unfortunately, I wasn’t able to understand how to retrieve the needed matrices and calculate the correct IPM matrix for OpenCV with the existing questions and docs: - getting the intrinsic parameters **K**: https://discourse.panda3d.org/t/camera - Pastebin.com

Is there a way to get the intrinsics K in the format

[f_x, s , x_0;
0 , f_y, y_0;
0 , 0 , 1 ]

?

Currently, I’m not even sure anymore if this is possible without manually defining the 4 edge points that should reflect the new image.

Alternatively I thought of this approach: select 4 points from the image and calculate 2D coordinates. Then move the camera up the z-axis and set the yaw to -90 degrees (which is basically the Birds Eye View, but directly rendered) and calculate the new 2D coordinates. Use these points to calculate the transformation matrix with cv2.findHomography.

Any tips or thoughts on how this could be done differently would be very appreciated! :slight_smile:

2

First of all, welcome to the forum! I hope that your time here proves positive! :slight_smile:

Hmm… I’m not sure that I fully understand what you’re looking for, but would one or more of the matrix-fetching methods of the Lens class be of use?

Specifically, if you have the Panda3D camera that generated the image, then you should be able to access its lens and retrieve the relevant matrices, it seems to me.

See the documentation on the “Lens” class, below:
https://docs.panda3d.org/1.10/python/reference/panda3d.core.Lens#panda3d.core.Lens.project

3

Hey,
Sorry for the late response. The lens method was the right approach to get the intrinsic :grinning:

 lens = cam.get_lens()
 fov_angle = lens.getFov()
 x_0 = window_x / 2
 y_0 = window_y  / 2
 fx = window_x / (2 * np.tan(fov_angle[0] * np.pi / 360))
 fy = window_y  / (2 * np.tan(fov_angle[1] * np.pi / 360))
1 Like