Hello, can you tell me your running environment

[wsz1998]

Hello, can you tell me your running environment and the order in which these codes are run. If I have a video, how can I import it with your code. I just started learning about camera calibration. I hope your reply

[kocurvik]

Hello,

I lost the environment for this project so I am not sure which versions it used. But you will need an environment with tensorflow, numpy, opencv, matplotlib and scipy.

If you want to run this on your own data you should first download the model from Google Drive. You should then follow the instructions in the Model evaluation section of the readme.

[wsz1998]

Thank for your reply. I have another problem now. I download the BrnoCompSpeed datasets and 'model.030.h5 '. But there are some errors when I run 'detect_bcs.py'. For example, the expected input_size of the model is [None, 128,128,3],but the size of every frame of the video is [1,1080,1920,3]. I don't know if there is something wrong with my operation.

[kocurvik]

The detect_bcs.py script should download an object detection model from tensorflow hub and then simply run it while saving the output bounding boxes. You do not need to use the pretrained model from the Google drive for this step.

[wsz1998]

I'm so sorry to bother you again. I have a new problem at your code. When I run eval/extract_calib.py for model evaluation. There are some error. The focal length is calculated in line 196, a negative number appears under the radical sign. The denominator appears to be 0 in line 208. I don't know if it's a problem with my operation, or a problem with the detector. How can I handle this situation?

[kocurvik]

Hi, there could be few problems.

First the horizon line could actually be vertical leading to the error. Though this would not make much sense in a traffic camera calibration setting as that would mean that the y-z plane of the camera is parallel to the road plane. So I would assume it should be something else.

What is more likely is that there is some problem with the data which originates way earlier. Could you try to print out some elements so I might get a better idea of what is going on? https://github.com/kocurvik/deep_vp/blob/349cf304446070d73683e2c665eaa4279fe14eee/eval/extract_calib.py#L198-L201

[wsz1998]

Hi, I'm sorry for the late reply, there are something happend at my family.

First, the dataset I use is session0_center in BrnoCompSpeed. I ran detect_bcp.py and got the detections.json file. Then, I ran extract_vp_bcs_heatmap.py and get the VP.json file. Finally, I ran extract_calib.py and get an error on line 196.

https://github.com/kocurvik/deep_vp/blob/349cf304446070d73683e2c665eaa4279fe14eee/eval/extract_calib.py#L196

-np.sum((vp1 - pp[np.newaxis, :]) * (vp2 - pp[np.newaxis, :]), axis=1) There are more than 14,000 elements in it, of which 10,000 are negative numbers. So using the sqrt function will report an error.

Then, I added the abs function to the above code to make it always positive. f = np.sqrt(abs(-np.sum((vp1 - pp[np.newaxis, :]) * (vp2 - pp[np.newaxis, :]), axis=1))) But an error occurred on line 208.

https://github.com/kocurvik/deep_vp/blob/349cf304446070d73683e2c665eaa4279fe14eee/eval/extract_calib.py#L208

Because there are 120+ zero elements in (vp1[:, 0] - vp2[:, 0])

I don't know how to modify the code to keep running or is there something wrong with the way I am running.

If you want to see those two json files, you can view them at the following link. https://github.com/wsz1998/Camera-Calibration/tree/master

I am sorry to reply lately.

[kocurvik]

The square root of negative numbers should return NumPy NaNs. This should then be taken care of in the following lines:

https://github.com/kocurvik/deep_vp/blob/349cf304446070d73683e2c665eaa4279fe14eee/eval/extract_calib.py#L198-L201

Negative numbers as squares of focal lengths are a common occurrence in many algorithms related to camera geometry. Usually such solutions are simply discarded and this is done in the referenced lines.

The second issue with zeros in denominator is handled in the same way:

https://github.com/kocurvik/deep_vp/blob/349cf304446070d73683e2c665eaa4279fe14eee/eval/extract_calib.py#L212-L213

Meaning that the median is not counted for nan values which are the result of 0 in the denominator.

The code should therefore work without any modification. You can ignore the warnings.

[wsz1998]

Thank you for your reply. I understand what you mean now. I'm a beginner, please forgive me

[wsz1998]

I hope I didn't bother you, I have another question. If I want to get some parameters in the real world through video, such as in the image, the displacement of the front and rear frames of the car, the speed of the car. What real values ​​do I need to add, what do I need to add or modify in your code? I hope you can give me an idea. Thank you very much.

[kocurvik]

You can use the projector which is defined as:

https://github.com/kocurvik/deep_vp/blob/349cf304446070d73683e2c665eaa4279fe14eee/eval/eval_calib.py#L24-L53

For example to measure distance between two points in pixel coordinates (p1 and p2) given already calculated calibration data (a dict which contains a 'calibration_data' key with 'vp1', 'vp2' and 'pp') you would do something like this:

projector, scale = get_system_projector(calib_data)
P1 = projector(p1)
P2 = projector(p2)
distance = scale * np.linalg.norm(P1 - P2)

[wsz1998]

So I need not measure value in real world if I use my own datasets. I just need to use your model and enter the coordinates of the two points in my image. Can I understand it this way?

[kocurvik]

The issue is that this method does not compute the scale. So you would have to set the scale parameter by first measuring a distance of two points for which you know the real world distance. Also note that these two points have to be on the road plane and the scale will only be valid for measurements on the given plane. You could also perform this for any other plane which is parallel to the road plane, but then all the relevant points have to be on that same plane.

You can get the scale parameter from a single known measurements like this:

projector, _ = get_system_projector(calib_data)
P1 = projector(p1)
P2 = projector(p2)
scale = known_distance_from_p1_to_p2 / np.linalg.norm(P1 - P2)