Project Video Scanner

[FlyPT]

Hi,

First time posting here.
Since some members here have shown some interest for my work, I decided to put here a link to my program: Project Video Scanner

What is it? Well, all started at PCars with the idea of getting a cheap alternative to laser scanning, so instead of lasers I decided to make something with two cameras:

So we can get some point clouds on the cheap.
Please visit my site where you can download the program and make some tests.
Would love to have some feedback. Is it of any interest, practical?
At first the idea was to use the point cloud has reference, now I'm starting to think if we could get some surface from them.

Anyway, that's just some leisure. If it's useful, good if not at least I learned something.

 

[FlyPT]

@ Mr Whippy,

Been out for some days.
I'm looking at SFM. It's not really hard to implement. But associating with odometry might be harder.
Since we have a sequential movie where frames don't repeat and we don't see the same objects twice (unless we make multiple passes...) we might be able to implement the SFM correction during reconstruction. We could avoid the final correction this way.
I will probably add a parameter to specify frame interval to use SFM.
I'm going to try this.

Thank's for your ideas.

 

[FlyPT]

@ Mr Whippy,

Instead of SFM I'm going to implement (try) this article:
https://www.ri.cmu.edu/pub_files/2013/12/badino_cvad13.pdf

Right now I'm computing odometry between two sequential images. That's the reason for the error. On the above paper Badino's works with multiple frames, as long as the feature is present in the movie.
It's almost like SFM, but adapted to a sequential movie.
Take a look at the results on the paper, they are really good and currently it's the best ranked method for stereo cameras. It's on third place right next to velodyne (laser) based ones.
The method used currently in PVS is on 11th place.....

 

[QuadCoreMax]

Hi,
What are your system specs?
The program is 32 bits so it should work. I have tried in Win7 32 and it works. For Vista, I was unable to make it work, but I can try...

I'm still on XP SP3 32 bits

 

[Mr Whippy]

Hi FlyPT,

That sounds good. 65% improved drift performance with only 3.8% cost in time!

Hopefully it's something that can have coefficients tweaked somewhat to improve the drift performance more, at a higher cost in time.

I'd prefer to wait a whole day for reconstruction if I knew it would have near zero drift, vs getting a fast reconstruction that needed a days worth of manual user input corrections.


I'm excited to see what you do next.

Like I said if you want/need any example data in certain formats let the community know, I'm sure people will be happy to help you with data to help improve the data sets.

 

[Alex Forbin]

@ Mr Whippy,

Instead of SFM I'm going to implement (try) this article:
https://www.ri.cmu.edu/pub_files/2013/12/badino_cvad13.pdf

Right now I'm computing odometry between two sequential images. That's the reason for the error. On the above paper Badino's works with multiple frames, as long as the feature is present in the movie.
It's almost like SFM, but adapted to a sequential movie.
Take a look at the results on the paper, they are really good and currently it's the best ranked method for stereo cameras. It's on third place right next to velodyne (laser) based ones.
The method used currently in PVS is on 11th place.....

Would your scanner prg work better with cameras with a FOV of 140 degrees, or worse?

Alex Forbin

 

[FlyPT]

Would your scanner prg work better with cameras with a FOV of 140 degrees, or worse?

Alex Forbin

140º his to much for the calibration method used in PVS.
There are other available tools to calibrate those lenses but until now I have not obtained good results (didn't try to much).

That's on the list of things to try, but for now that's not a priority. By the way, bigger FOV could enhance odometry.
I'm more interested in stereo 360º lenses. Could give really good odometry and a 360º reconstruction, but at the cost of resolution.

Lenses like this: https://www.kickstarter.com/projects/95342015/360o-for-gopro-dslr-video-and-compact-cameras-ocul

 

[Mr Whippy]

I think you'd always be better to run more stereo pairs, or more cameras arranged in a circle, to get high resolution but wider coverage.

360 devices are all really low quality when you zoom in, even say 4k wide would be quite low resolution per degree of field of view at 360deg!


Time to try find a local road that loops around so I can do some testing with drift and correction

Dave

 

[Alex Forbin]

140º his to much for the calibration method used in PVS.
There are other available tools to calibrate those lenses but until now I have not obtained good results (didn't try to much).

That's on the list of things to try, but for now that's not a priority. By the way, bigger FOV could enhance odometry.
I'm more interested in stereo 360º lenses. Could give really good odometry and a 360º reconstruction, but at the cost of resolution.

Lenses like this: https://www.kickstarter.com/projects/95342015/360o-for-gopro-dslr-video-and-compact-cameras-ocul

That looks like a must buy item for sure. I can see how even having 2 stereo
pairs (one forward and one back) would help odometry. Based on what I read in the pdf you posted, it looks like you could have the drift correction without back-tracking any frames, plus you could cloudmap both sides of an object as you pass.

Alex Forbin

 

[Mr Whippy]

Yeah a backward facing pair seems the best approach to me too.

You'll always want to drive back to get the 'other side' of objects, so doubling up on cameras and capturing backwards at the same time makes oodles of sense vs driving both ways (maybe not even possible on one way roads/race tracks etc)

My only reservation still at this stage is the range that we can capture at. Having higher res captures (1080p) and mounting the cameras higher would seem the obvious ways to fix this?


Hmmm

Dave

 

[Osk1]

Hi flyPT,

this very good application and I ran the examples that you have and the cloud of points does it very well, I am very interested in this topic because I'm working on my research degree, I tried to do something like this with opencv but not been so easy and the results are not good. I want to know if you can guide me with your experience, through email.

appreciate your help,

 

[FlyPT]

Hi,

Contact me by PM. I will help you where I can.