Which K is the best K?
After working for a while with the Caltech and other code that uses homogenous coordinates (like P, K, etc) an issue has come up that needs a solution. I think I have one, but let's describe the problem and then how to fix it.
Quick review: what is K
K is the 3x3 matrix of intrinsic camera parameters. It contains horizontal and vertical focal lengths, image center, and skew. K(1,1) is the horizontal focal length, and is converted to the FOV by:
fov = 2*(atan(w/2/K(1,1)))
Where do we get K?
K is an output of the Caltech camera calibration toolbox ("calib_gui").
What's wrong with K from Caltech?
Inherently, nothing. It should be a good representation of the intrinsics with consideration of the associated distortion parameters. I.e., it takes the radial distortion parameters into account, where the first order radial distortion has a direct relation to the calculated FOV values. However, the Caltech output solves for both horizontal and vertical focal lengths and often (always?) returns different numbers.
It would seem reasonable to assume that modern manufacturing processes are conducted to much higher tolerances than appear in the calibrated focal lengths. It is hard to believe that any lens would be as non-symmetrical, or sensor out of tolerances, as the focal length difference would indicate. There should be one number, or at least, two that are a lot closer to each other. Now, it may be that the difference may come from target printing errors. (A potential experiment is to perform the calibration with the target "horizontal" for one set of images, and then "vertical" for another, and compare the K(1,1) and K(2,2) values.)
However, this leaves us with the quandry, what do we put in K? The average?
But there's more...
There is another issue, which is actually more important. The initial K has to come from the camera calibration, and it is considered a feature of the camera itself. It is, after all, intrinsic to the camera. Therefore, in Argus, K has been a field in the camera table. The camera table contains entries for each camera and is assumed to be fixed, or rarely changed.
A newer version of the geometry solver ('cilgeomNew') which is being implemented at CIL and will be distributed soon actually includes the ability to solve for K. This is logical since the previous version was almost always used to solve for the FOV, and the FOV and K values are inter-related. Thus the newest problem: how do we deal with a solved K?
Why bother?
We have the K in the camera table, why bother with the updated K? Unfortunately, one of the reasons the new geometry solution code was written is because using the calibrated K didn't always result in good geometry results. Also, the 'm' vector and angles in the geometry solution are dependent on the K that is used. If you use a geometry and camera struct together to generate P matrices, you will be mixing Ks.
In other words, the geometry now can depend highly on the K that is calculated from the geometry solution code and not on the calibrated K.
The "simple" solution, of course, seems to be just to update the K in the camera struct. Si? Au contraire.
- The geometry and K must match, and simply updating the camera struct means that it will no longer correspond to any other geometry solution.
- Creating a new camera struct for each new geometry will be a huge waste of space.
- There is currently no mechanism in the Argus database code to update the K in a camera struct anyway.
How do we solve this?
The proposed solution is to add the K matrix to the geometry solution.
Advantage
Your geometry solution is now completely internally consistent. You have the same K used in creating 'm' and the four angles in the same struct.
Disadvantage
You now have two (or hundreds) of potential K matrices to pick from. If you have not yet created a geometry, have only the K from the calibration in the camera struct. If you have a geometry created by the new solver, you have one in the camera struct and one in the geometry. You will need to keep track of which K you need to use.
You will probably always want to use the K from the geometry. Current Argus code uses the K from the camera. This will need to be changed.
Comments?
Which K is the best K?
After working for a while with the Caltech and other code that uses homogenous coordinates (like P, K, etc) an issue has come up that needs a solution. I think I have one, but let's describe the problem and then how to fix it.
Quick review: what is K
K is the 3x3 matrix of intrinsic camera parameters. It contains horizontal and vertical focal lengths, image center, and skew. K(1,1) is the horizontal focal length, and is converted to the FOV by:
fov = 2*(atan(w/2/K(1,1)))
Where do we get K?
K is an output of the Caltech camera calibration toolbox ("calib_gui").
What's wrong with K from Caltech?
Inherently, nothing. It should be a good representation of the intrinsics with consideration of the associated distortion parameters. I.e., it takes the radial distortion parameters into account, where the first order radial distortion has a direct relation to the calculated FOV values. However, the Caltech output solves for both horizontal and vertical focal lengths and often (always?) returns different numbers.
It would seem reasonable to assume that modern manufacturing processes are conducted to much higher tolerances than appear in the calibrated focal lengths. It is hard to believe that any lens would be as non-symmetrical, or sensor out of tolerances, as the focal length difference would indicate. There should be one number, or at least, two that are a lot closer to each other. Now, it may be that the difference may come from target printing errors. (A potential experiment is to perform the calibration with the target "horizontal" for one set of images, and then "vertical" for another, and compare the K(1,1) and K(2,2) values.)
However, this leaves us with the quandry, what do we put in K? The average?
But there's more...
There is another issue, which is actually more important. The initial K has to come from the camera calibration, and it is considered a feature of the camera itself. It is, after all, intrinsic to the camera. Therefore, in Argus, K has been a field in the camera table. The camera table contains entries for each camera and is assumed to be fixed, or rarely changed.
A newer version of the geometry solver ('cilgeomNew') which is being implemented at CIL and will be distributed soon actually includes the ability to solve for K. This is logical since the previous version was almost always used to solve for the FOV, and the FOV and K values are inter-related. Thus the newest problem: how do we deal with a solved K?
Why bother?
We have the K in the camera table, why bother with the updated K? Unfortunately, one of the reasons the new geometry solution code was written is because using the calibrated K didn't always result in good geometry results. Also, the 'm' vector and angles in the geometry solution are dependent on the K that is used. If you use a geometry and camera struct together to generate P matrices, you will be mixing Ks.
In other words, the geometry now can depend highly on the K that is calculated from the geometry solution code and not on the calibrated K.
The "simple" solution, of course, seems to be just to update the K in the camera struct. Si? Au contraire.
How do we solve this?
The proposed solution is to add the K matrix to the geometry solution.
Advantage
Your geometry solution is now completely internally consistent. You have the same K used in creating 'm' and the four angles in the same struct.
Disadvantage
You now have two (or hundreds) of potential K matrices to pick from. If you have not yet created a geometry, have only the K from the calibration in the camera struct. If you have a geometry created by the new solver, you have one in the camera struct and one in the geometry. You will need to keep track of which K you need to use.
You will probably always want to use the K from the geometry. Current Argus code uses the K from the camera. This will need to be changed.
Comments?