Optical Flow Odometer
Update Whilst inital tests where promising in more realistic situations any significant change of distance from the camera to the surface between frames lead to fairly large errors. Also the "moving" shadows, due to moving lighting confuse the optical flow algorithm.
This project kicked off with the desire to have an easy way to measure distance traveled in a cave. But it could be applied to pretty much any situation, robots, vehicles, walking, anywhere where a distance estimate is wanted and there is the opportunity to video the route.
The inspiration came from the optical mouse and it works in a similar way, however the optical mouse has the advantage of being a fixed distance from the surface on which it operates. This approach gets around that limitation by using a laser pointer.
The principal is pretty simple, attach a laser pointer to the camera so that it is parallel with the axis of the lens. The laser dot on the image is then always a fixed distance from the centre of the image (on my camera it turned out to be 81mm). So the number of pixels from the centre of the image to the laser dot represents that distance - 81mm for me. From this you can work out how many pixels represent 1 metre and so by tracking the movement in pixels from one video frame to the next the distance traveled can be calculated.
My initial tests were very encouraging. After some tweaking I got the error down to less than 100mm over 5.7m - much better then I ever expected. This was under fairly ideal conditions and there is still a lot more work to do but these inital results are encouraging.
The inital work was done on the paving slabs of my patio - flat and level - and whilst a good easy testing ground, not very realistic for caving. Tests on real situations revealed that the real problem comes when there is a significant variation in distance from the camera to the surface between frames. So on smooth surfaces the technique works, on uneven surfaces (stony ground or rough walls) the errors become large.
Here is the code and a windows executable. The code is a bit messy because it cobbles together a couple of OpenCV examples to get the job done. You will need cygwin.dll in your path to get this to run.
- Strap a laser pointer to your camera and video a walk or other journey.
- Convert the video to an AVI and then you may need to use mencoder (download MPlayer and you get mencoder) to change the format to i420 - at the command prompt run
mencoder in.avi -ovc raw -vf format=i420 -o out.avi(add -nosound if your camera records sound)
- Ensure cygwin.dll is in your path (best to download cygwin if you do not already have it, just getting the dll should work).
- At the command prompt run LOdometer.exe video_file.avi laser-lens-center-mm points_to_track
- You get two windows, in the TrackDot window drag a rectangle over the laser dot on the video and the second window will start showing the Optical Flow, the console will show distance in metres and angle.
There needs to be a reasonable contrast between the laser dot and the background. I found that filming at dusk or indoors gave the right amount of contrast.
I found the system pretty sensitive to the points_to_track parameter; points to track sets how many points will be tracked between video frames. The code takes an average of all vectors generated for these points. Too many and small movements of the camera, such as wobbles and rotations seem to gain more weight, too few and the same rotations and wobbles are ignored giving too much smoothing.
When recording the video try to avoid filming non-stationary objects (e.g. your feet). These will get tracked and if you watch them in the optical flow window you will see that they create motion vectors which detract from the accuracy of the result.
If you want to build this you need a C++ compiler (I used Cygwin and Eclipse CDT) and you need OpenCV installed. It is important to note that OpenCV needs building, even though it appears to extract the binaries on install.
There are only a source and header files because OpenCV does all the complex stuff out of the box so these are just concerned with settting up the tracking, running the optical flow and calculating the result.
OpenCV includes the ability to capture directly from a WebCam - so with a few simple mods to the code this could be run live rather than from a recorded video.
This project is far from complete, at the moment it is a proof-of-concept. For the original application of measuing cave distance I believe either a pair of cameras to give a stereo vision approach or an IMU (initerial measurement unit) is needed to allow camera rotations to be measured and compenstated for.
It would also be possible to use the principals set out on this page to track distance from the camera to the surface. In caving terms this would potentiall allow for passage sections to be measured.