voelker

Papilio + ov7670 camera = spartcam !

Recommended Posts

Hi,

just wanted to share som progress i made with using the papilio to read the ov7670 sensor.

I have connected an ov7670 camera module, i found on ebay for cheap, to the papilio one with the goal of reading an image from it and being able to apply some treatment. My goal is to do have a cmucam like platform for teaching purpose.

At this point i have designed a small pipeline that grab the pixels, takes the Y (luminance) component  performs a downscale of the image (640x480 -> 80x60) and send the picture on serial at 3Mbaud. Frame grabbing and sending is done at 30hz. There is no soft core involved, everything is performed using homemade modules (i2c, pixel grabbing, downscaling ...) and i'am only using some BRAM for configuration storing, one 80 pixel line storing for dwonscaling, and a 128 byte FIFO for the serial communication.

I made a little java app to display the picture and test evrything. The picture are fine but it seems that i sometime get transmission errors.

The next step is to add an edge detector to the pipeline and try to dectect a line an build a line following robot in the future !

som pictures:

capt_ov7670.png

the interface

120302-022556.jpg

and the board, the reset button is directly pugged into the female header ...

The sensor connected straight into the papilio headers, is just had to add a little stripboard to get access to the vcc and gnd of the papilio. There is still room to connect a second camera ... stereovision anyone ?

Device Utilization :

Selected Device : 3s250evq100-4

Number of Slices:                      252  out of  2448    10% 

Number of Slice Flip Flops:            215  out of  4896    4% 

Number of 4 input LUTs:                483  out of  4896    9% 

    Number used as logic:              474

    Number used as Shift registers:      9

Number of IOs:                          22

Number of bonded IOBs:                  22  out of    66    33% 

Number of BRAMs:                        3  out of    12    25% 

Number of GCLKs:                        4  out of    24    16% 

Number of DCMs:                          2  out of      4    50% 

EDIT: The project can be checkout on

http://code.google.com/p/fpga-cam/ 

The VHDL and SystemC code are not very clean but i plan to do some refactoring.

The project also contains a small SystemC to VHDL application i designed to help with this project but not a complete work. It only translate RTL like SystemC.

This post has been promoted to an article

Share this post


Link to post
Share on other sites

Hi,

this is the kind of module i'am using:

http://cgi.ebay.fr/VGA-OV7670-Camera-Module-640X480-SCCB-Compatible-with-I2C-Interface-/180732693353?pt=BI_Electrical_Equipment_Tools&hash=item2a14820b69#ht_6450wt_1031 

10usd seems really cheap to me. The quality of the board is not very good, but it does the job. There is other kind of module, but they will require a 2.8v power suppply, while this one takes 3.3v.

In the picture of the interface i joineed to the post, the picture looks grainy but its because i'am doing a dirty upscaling (copying line and pixe twice) to have a larger image.

Share this post


Link to post
Share on other sites

Hey this is great! This is like Christmas, I happen to have an ov7670 camera module and a stack of PCB's for a Wing for this camera sitting on my desk. :)

Great work on getting this camera working! This is a shiny new toy to play with thanks to your great work. :)

I've been sitting on a Wing design for this module, I just didn't get the time to make it work. Since there is code to play with now I just committed the Wing design to Github (Girish designed it) and I have 9 PCB's that I can drop in mailing envelope's and ship out. Anyone that wants one drop me a PM with your shipping address. :)

The Wing was designed for a slightly different module that requires 2.8V. I'm trying to see if the pinouts are the same for the module that voelker is using. Ahh, ok, I see the pinout on the PCB and it matches so this Wing will work with both modules. However, you will probably have to jumper 3.3V to the pin 1 of the module that voelker used, or provide the 2.8V voltage regulator...

I'm going to try and see if I can get this working today. :) This is real exciting! I might have to wait until I can order a 2.8V voltage regulator though. But I'm willing to try and power it with 2.5V or 3.3V. They are cheap enough that I don't mind if I blow one up. :)

The other thing to consider, Alvie just told me yesterday that he has added I2C support to the ZPUino. So for anyone who wants to interface this thing with a sketch should have a much easier time of it now.

Thanks again and I will schedule a blog post about this.

Jack.

post-3-13431627491823.png

Share this post


Link to post
Share on other sites

I also happen to have the 2.8v version of the module but didn't want to have to design a wing for it. I'm gonna take some more picture of the setup to post and shoot video asap. @Jack PM me if you have any trouble to get the thing working.

Share this post


Link to post
Share on other sites

I have added the viewer programm to the project. This is really messy java code but my only goal was to see the images, no time to waste on programming a fancy UI.

I'am also gonna do a kind of VHDL template that one could modify to integrate a new function into the image pipeline in the FPGA. I also started working on a sobel filter to perform edge detection in the image.

For the moment i'am trying to use only the internal BRAM by storing only 2 lines + 3 pixel at a time (for using 3x3 convolution kernels), but at some point i might consider using an external SRAM.

Share this post


Link to post
Share on other sites
Guest gmpundlik

Hi, great work voelker! This was one of the few wings we are meaning to start working on. As Jack said, its been quite a while since we had made this wing but could not start working on it. This wing also has a companion 3.2" LCD wing for this module:

http://www.ebay.com/itm/3-2-TFT-LCD-Module-Display-Touch-Panel-PCB-adapter-/200475566068?pt=LH_DefaultDomain_0&hash=item2ead465ff4

I have attached a image of intended setup for the combination, that would make a CCTV type display, or replacing the LCD wing with a high-speed USB wing can also make a PC Cam from Papilio One and the camera!

Now I have some time to fiddle with FIFO core and hi-speed USB wing, hopefully we can put together a viable system for surveillance or machine vision real soon.

The idea of stereo cameras is also interesting, we can try that after we get reading the camera in hardware, with a camera core.

Girish

post-0-13431627491945_thumb.gif

post-0-13431627492022.jpg

Share this post


Link to post
Share on other sites
Guest gmpundlik

BTW, the setup in the photograph above won't work, both the LCD and camera have pins soldered on the underside of the board, so the pinout on the connector is mirrored. So either you solder wing end headers on the underside as well, or just make a special wing with mirrored headers.

I am making another set of wings with headers on the underside of the board, I can then test voelker's code with suitable changes to the UCF file.

Girish

Share this post


Link to post
Share on other sites

The setup with the LCD seems promising. I saw a guy on ebay that had the same setup but with a cyclone FPGA and he was using a NIOS softcore. The high speed usb wing could also help, i'am currently transferring a 80x60 preview image but with a high speed USB FIFO i might be able able to transfer the full frame.

The seterovision project seems ambitious, but i'am sure that with the help of some SRAM we could generate a depth map or compute some obstacle detection algorithm.

I will try to add comment to my code this week-end to make it easier to read.

Share this post


Link to post
Share on other sites

One more thing. I'am getting YUV out of the camera, it should be pretty easy to get the RGB frame just a modification to the camera configuration register set. RGB is better if you want to drive an LCD ...

Share this post


Link to post
Share on other sites

I made some progress tonight. I have cleaned up the code a little, added comment to the register configuration and added a new file to store the rgb565 configuration of the sensor.

Thanks to the OpenWorkbench Logic Sniffer i could see that my I2C configuration wasn't working. I have fixed it, and now the image comming out of the sensor is much cleaner and the 50hz coming from the lights is filtered out ! One of my mistake was that the sensor needs 1ms to be ready after reset and i was sending i2c commands straight after reset. I also did a mistake on the sensor address ...

The OLS is a very handy tool to debug such things.

The first picture is grabbed with the YUV configuration while the second one was grabbed using the RGB565 configuration (grabbed in a YUV mode which explain why the sensor seems to be on drugs ...)

ols.png

rgb565.png

Share this post


Link to post
Share on other sites

Nice! I love to see the OLS in action, its probably the most used tool on my desk. :)

Sadly, I did not get a chance to try and get the camera working. Hopefully very soon I will get to. :)

On the topic of SRAM, I just built a batch of Papilio Plus boards which use a Spartan 6 LX9 and have 4Mb of SRAM. I'm updating the memory test right now so I can verify this batch of boards using the newly finished test plan. :) So, I should have some prototype boards for sale in the store in the next couple of days.

Jack.

Share this post


Link to post
Share on other sites

I eager to test the spartan6 board, the SRAM will for sure help with image processing. I just hope you've sticked with the 32mhz oscillator because its easier to generate the 24mhz pixel clock.

Does the spartan6 increases the price of the board a lot ?

Share this post


Link to post
Share on other sites

Thanks for the blog post Jack !

It seems that i was too slow to grab one of your spartan6 board, can you keep one for me in the next batch ?

The sobel filter is close to be complete i hope to post some edge detection pcitures very soon.

Share this post


Link to post
Share on other sites

Voelker,

I'm ordering parts to manufacture 10 more Papilio Plus boards, hopefully they can be ready in about a week... I am sticking with the 32Mhz oscillator by the way.

The funny thing is that the reason I selected a 32Mhz oscillator in the first place was because I used to do all the Papilio manufacturing myself and I was able to pick up several reels of 32Mhz oscillators off ebay. That got the price of each oscillator down to .05 cents instead of the $1.20 I was paying from Digikey. :) So I still have tons of those oscillators around and it doesn't make much sense to change them. As an added bonus several people have told me that 32Mhz is much easier to work with then 50Mhz. So in the end it all comes out for the best. :)

Jack.

Share this post


Link to post
Share on other sites

32MHz is wonderful. For serial interfaces.

Look:

  3 Mbit/s

  8x oversampling: 32Mhz :)

  2 Mbit/s

  8x oversampling: 16Mhz (32/2)

  1 Mbit/s

  8x oversampling: 8 MHz (32/4)

Always perfect serial interfaces. :)

Alvie

Share this post


Link to post
Share on other sites

Hi,

just a quick update. I got the sobel filter running ! I'am now using a little more BRAM but i still have some room. I have designed a generic 3x3 block concolution component and uses it with the sobel filter x and y matrices. Right now i'am using two embedded multipliers but i have the option to use shift register instead, as sobel filter uses only 2 and 1 element.

sobel_tv.png

This is my old TV in the sobel world !

Share this post


Link to post
Share on other sites
Guest Jax_Kobi

Hi Voelker,

I'm very impressed by what you've done, great job!  :)

I've had a project like this in mind for a long time. What I want to do is connect one of those high-res miniature camera modules (leopardimaging.com/1.html) and record video at full resolution. I'm good with analog circuits, and I have good background in C, python and little in programming arduinos but never did VHDL or the likes. What would you say is the FASTEST way to achieve my goal? Do you recommend specific books or methods to learn all I need to do what I want? I have lots of free time and a budget of about $1000 for hardware (fast FPGA?) if that helps.

Any info is greatly appreciated!!

Thanks,

Share this post


Link to post
Share on other sites

Hi,

@

Jax_Kobi, if you want to start developing hardware in VHDL i would recommend buying a true development board such as :

http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400,897&Prod=NEXYS3 

or

http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400,789&Prod=NEXYS2 

or (if you prefer Altera)

http://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=139&No=593 

or

http://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=53&No=226 

this kind of board offer peripherals that are usefull in the learning process (leds, memories ...) .

You can find lot of good documents on internet such as "the VHDL cookbook", Altera also has some free vhdl course (http://www.altera.com/education/training/courses/OHDL1110).

My method to learn, is :

1) learn the basic language construction and theory

2) Find a project you want to work on even if it might seem too hard

3) Divide the project into small chunck. This way you 'll see you project progress.

4) Start having fun working on your project and share with people onf forum, blog ...

@hechao: Nice to have someone testing my code. I just hope your FPGA board has a 32mhz oscillator, or you 'll have hard time generating the 24Mhz pixel clock ...

Share this post


Link to post
Share on other sites

Voelker,

Looking like great progress, I just remembered that Dan Strother released some OpenCV compatible vision code here:

http://danstrother.com/2011/06/10/fpga-stereo-vision-core-released/

As a bonus it might even have been tested directly on the Papilio, Dan has a Papilio. :) Might help you out in some way.

Jack.

Share this post


Link to post
Share on other sites

Ok, I admit, this is cool :)

 

  I'm too lazy to take a look at your code, so a few questions:

* what's your I2C frequency ?

* bit depth ? (I assume <16bit)

Why are you considering not to use DSP48A for the job ? They are smaller than general purpose shifters.

Reason I'm asking this is because eventually I can have ZPUino processing the image with some MACC extensions. That would make things easier for, as example, streaming the video using some codec over ethernet or some other high speed interface.

Alvie

Share this post


Link to post
Share on other sites

Hi,

the i2C frquency is 200khz, and i'am not using the DSP48A because the Spartan3E does not have any ...

The pixels bit depth is

My sobel filter can make use of the embedded multipliers, but i'am trying to save them for convolution that would necessitate mulitplication by factor other than 2,4,8.

Right now, i'am not considering compressing the data, because compression is usually a memory consuming task and i want to use only BRAMs. I will have a look at the zpuino and see how i can use a softcore to perform other kind of computation.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now