voelker Posted March 2, 2012 Report Share Posted March 2, 2012 Hi,just wanted to share som progress i made with using the papilio to read the ov7670 sensor. I have connected an ov7670 camera module, i found on ebay for cheap, to the papilio one with the goal of reading an image from it and being able to apply some treatment. My goal is to do have a cmucam like platform for teaching purpose. At this point i have designed a small pipeline that grab the pixels, takes the Y (luminance) component performs a downscale of the image (640x480 -> 80x60) and send the picture on serial at 3Mbaud. Frame grabbing and sending is done at 30hz. There is no soft core involved, everything is performed using homemade modules (i2c, pixel grabbing, downscaling ...) and i'am only using some BRAM for configuration storing, one 80 pixel line storing for dwonscaling, and a 128 byte FIFO for the serial communication.I made a little java app to display the picture and test evrything. The picture are fine but it seems that i sometime get transmission errors.The next step is to add an edge detector to the pipeline and try to dectect a line an build a line following robot in the future !som pictures:the interface and the board, the reset button is directly pugged into the female header ...The sensor connected straight into the papilio headers, is just had to add a little stripboard to get access to the vcc and gnd of the papilio. There is still room to connect a second camera ... stereovision anyone ?Device Utilization :Selected Device : 3s250evq100-4 Number of Slices: 252 out of 2448 10% Number of Slice Flip Flops: 215 out of 4896 4% Number of 4 input LUTs: 483 out of 4896 9% Number used as logic: 474 Number used as Shift registers: 9 Number of IOs: 22 Number of bonded IOBs: 22 out of 66 33% Number of BRAMs: 3 out of 12 25% Number of GCLKs: 4 out of 24 16% Number of DCMs: 2 out of 4 50% EDIT: The project can be checkout onhttp://code.google.com/p/fpga-cam/ The VHDL and SystemC code are not very clean but i plan to do some refactoring. The project also contains a small SystemC to VHDL application i designed to help with this project but not a complete work. It only translate RTL like SystemC.This post has been promoted to an article Link to comment Share on other sites More sharing options...
hamster Posted March 2, 2012 Report Share Posted March 2, 2012 Very impressed... How cheap was "cheap on Ebay"? :-) Link to comment Share on other sites More sharing options...
voelker Posted March 2, 2012 Author Report Share Posted March 2, 2012 Hi, this is the kind of module i'am using: http://cgi.ebay.fr/VGA-OV7670-Camera-Module-640X480-SCCB-Compatible-with-I2C-Interface-/180732693353?pt=BI_Electrical_Equipment_Tools&hash=item2a14820b69#ht_6450wt_1031 10usd seems really cheap to me. The quality of the board is not very good, but it does the job. There is other kind of module, but they will require a 2.8v power suppply, while this one takes 3.3v. In the picture of the interface i joineed to the post, the picture looks grainy but its because i'am doing a dirty upscaling (copying line and pixe twice) to have a larger image. Link to comment Share on other sites More sharing options...
Jack Gassett Posted March 2, 2012 Report Share Posted March 2, 2012 Hey this is great! This is like Christmas, I happen to have an ov7670 camera module and a stack of PCB's for a Wing for this camera sitting on my desk. Great work on getting this camera working! This is a shiny new toy to play with thanks to your great work. I've been sitting on a Wing design for this module, I just didn't get the time to make it work. Since there is code to play with now I just committed the Wing design to Github (Girish designed it) and I have 9 PCB's that I can drop in mailing envelope's and ship out. Anyone that wants one drop me a PM with your shipping address. The Wing was designed for a slightly different module that requires 2.8V. I'm trying to see if the pinouts are the same for the module that voelker is using. Ahh, ok, I see the pinout on the PCB and it matches so this Wing will work with both modules. However, you will probably have to jumper 3.3V to the pin 1 of the module that voelker used, or provide the 2.8V voltage regulator... I'm going to try and see if I can get this working today. This is real exciting! I might have to wait until I can order a 2.8V voltage regulator though. But I'm willing to try and power it with 2.5V or 3.3V. They are cheap enough that I don't mind if I blow one up. The other thing to consider, Alvie just told me yesterday that he has added I2C support to the ZPUino. So for anyone who wants to interface this thing with a sketch should have a much easier time of it now. Thanks again and I will schedule a blog post about this. Jack. Link to comment Share on other sites More sharing options...
Guest evil_andy Posted March 2, 2012 Report Share Posted March 2, 2012 That looks like an awesome project! I look forward to being able to do some nifty stuff with that. I've actually used the CMUcam before (http://cs.gmu.edu/~eclab/projects/robots/flockbots/pmwiki.php) though it was a while ago, it would be interesting to see what I can come up with that's similar. Link to comment Share on other sites More sharing options...
voelker Posted March 2, 2012 Author Report Share Posted March 2, 2012 I also happen to have the 2.8v version of the module but didn't want to have to design a wing for it. I'm gonna take some more picture of the setup to post and shoot video asap. @Jack PM me if you have any trouble to get the thing working. Link to comment Share on other sites More sharing options...
voelker Posted March 2, 2012 Author Report Share Posted March 2, 2012 I have added the viewer programm to the project. This is really messy java code but my only goal was to see the images, no time to waste on programming a fancy UI. I'am also gonna do a kind of VHDL template that one could modify to integrate a new function into the image pipeline in the FPGA. I also started working on a sobel filter to perform edge detection in the image. For the moment i'am trying to use only the internal BRAM by storing only 2 lines + 3 pixel at a time (for using 3x3 convolution kernels), but at some point i might consider using an external SRAM. Link to comment Share on other sites More sharing options...
Guest gmpundlik Posted March 3, 2012 Report Share Posted March 3, 2012 Hi, great work voelker! This was one of the few wings we are meaning to start working on. As Jack said, its been quite a while since we had made this wing but could not start working on it. This wing also has a companion 3.2" LCD wing for this module: http://www.ebay.com/itm/3-2-TFT-LCD-Module-Display-Touch-Panel-PCB-adapter-/200475566068?pt=LH_DefaultDomain_0&hash=item2ead465ff4 I have attached a image of intended setup for the combination, that would make a CCTV type display, or replacing the LCD wing with a high-speed USB wing can also make a PC Cam from Papilio One and the camera! Now I have some time to fiddle with FIFO core and hi-speed USB wing, hopefully we can put together a viable system for surveillance or machine vision real soon. The idea of stereo cameras is also interesting, we can try that after we get reading the camera in hardware, with a camera core. Girish Link to comment Share on other sites More sharing options...
Guest gmpundlik Posted March 3, 2012 Report Share Posted March 3, 2012 BTW, the setup in the photograph above won't work, both the LCD and camera have pins soldered on the underside of the board, so the pinout on the connector is mirrored. So either you solder wing end headers on the underside as well, or just make a special wing with mirrored headers. I am making another set of wings with headers on the underside of the board, I can then test voelker's code with suitable changes to the UCF file. Girish Link to comment Share on other sites More sharing options...
voelker Posted March 3, 2012 Author Report Share Posted March 3, 2012 The setup with the LCD seems promising. I saw a guy on ebay that had the same setup but with a cyclone FPGA and he was using a NIOS softcore. The high speed usb wing could also help, i'am currently transferring a 80x60 preview image but with a high speed USB FIFO i might be able able to transfer the full frame. The seterovision project seems ambitious, but i'am sure that with the help of some SRAM we could generate a depth map or compute some obstacle detection algorithm. I will try to add comment to my code this week-end to make it easier to read. Link to comment Share on other sites More sharing options...
voelker Posted March 3, 2012 Author Report Share Posted March 3, 2012 One more thing. I'am getting YUV out of the camera, it should be pretty easy to get the RGB frame just a modification to the camera configuration register set. RGB is better if you want to drive an LCD ... Link to comment Share on other sites More sharing options...
voelker Posted March 4, 2012 Author Report Share Posted March 4, 2012 I made some progress tonight. I have cleaned up the code a little, added comment to the register configuration and added a new file to store the rgb565 configuration of the sensor. Thanks to the OpenWorkbench Logic Sniffer i could see that my I2C configuration wasn't working. I have fixed it, and now the image comming out of the sensor is much cleaner and the 50hz coming from the lights is filtered out ! One of my mistake was that the sensor needs 1ms to be ready after reset and i was sending i2c commands straight after reset. I also did a mistake on the sensor address ... The OLS is a very handy tool to debug such things. The first picture is grabbed with the YUV configuration while the second one was grabbed using the RGB565 configuration (grabbed in a YUV mode which explain why the sensor seems to be on drugs ...) Link to comment Share on other sites More sharing options...
Jack Gassett Posted March 5, 2012 Report Share Posted March 5, 2012 Nice! I love to see the OLS in action, its probably the most used tool on my desk. Sadly, I did not get a chance to try and get the camera working. Hopefully very soon I will get to. On the topic of SRAM, I just built a batch of Papilio Plus boards which use a Spartan 6 LX9 and have 4Mb of SRAM. I'm updating the memory test right now so I can verify this batch of boards using the newly finished test plan. So, I should have some prototype boards for sale in the store in the next couple of days. Jack. Link to comment Share on other sites More sharing options...
voelker Posted March 5, 2012 Author Report Share Posted March 5, 2012 I eager to test the spartan6 board, the SRAM will for sure help with image processing. I just hope you've sticked with the 32mhz oscillator because its easier to generate the 24mhz pixel clock. Does the spartan6 increases the price of the board a lot ? Link to comment Share on other sites More sharing options...
Jack Gassett Posted March 7, 2012 Report Share Posted March 7, 2012 Voelker, Congratulations! You are on the front page over at Hackaday! http://hackaday.com/2012/03/07/10-camera-module-for-your-next-fpga-project/ Awesome work! Jack. Link to comment Share on other sites More sharing options...
voelker Posted March 8, 2012 Author Report Share Posted March 8, 2012 Thanks for the blog post Jack ! It seems that i was too slow to grab one of your spartan6 board, can you keep one for me in the next batch ? The sobel filter is close to be complete i hope to post some edge detection pcitures very soon. Link to comment Share on other sites More sharing options...
Jack Gassett Posted March 8, 2012 Report Share Posted March 8, 2012 Voelker, I'm ordering parts to manufacture 10 more Papilio Plus boards, hopefully they can be ready in about a week... I am sticking with the 32Mhz oscillator by the way. The funny thing is that the reason I selected a 32Mhz oscillator in the first place was because I used to do all the Papilio manufacturing myself and I was able to pick up several reels of 32Mhz oscillators off ebay. That got the price of each oscillator down to .05 cents instead of the $1.20 I was paying from Digikey. So I still have tons of those oscillators around and it doesn't make much sense to change them. As an added bonus several people have told me that 32Mhz is much easier to work with then 50Mhz. So in the end it all comes out for the best. Jack. Link to comment Share on other sites More sharing options...
alvieboy Posted March 8, 2012 Report Share Posted March 8, 2012 32MHz is wonderful. For serial interfaces. Look: 3 Mbit/s 8x oversampling: 32Mhz 2 Mbit/s 8x oversampling: 16Mhz (32/2) 1 Mbit/s 8x oversampling: 8 MHz (32/4) Always perfect serial interfaces. Alvie Link to comment Share on other sites More sharing options...
voelker Posted March 9, 2012 Author Report Share Posted March 9, 2012 Hi, just a quick update. I got the sobel filter running ! I'am now using a little more BRAM but i still have some room. I have designed a generic 3x3 block concolution component and uses it with the sobel filter x and y matrices. Right now i'am using two embedded multipliers but i have the option to use shift register instead, as sobel filter uses only 2 and 1 element. This is my old TV in the sobel world ! Link to comment Share on other sites More sharing options...
Guest Jax_Kobi Posted March 9, 2012 Report Share Posted March 9, 2012 Hi Voelker, I'm very impressed by what you've done, great job! I've had a project like this in mind for a long time. What I want to do is connect one of those high-res miniature camera modules (leopardimaging.com/1.html) and record video at full resolution. I'm good with analog circuits, and I have good background in C, python and little in programming arduinos but never did VHDL or the likes. What would you say is the FASTEST way to achieve my goal? Do you recommend specific books or methods to learn all I need to do what I want? I have lots of free time and a budget of about $1000 for hardware (fast FPGA?) if that helps. Any info is greatly appreciated!! Thanks, Link to comment Share on other sites More sharing options...
Guest hechao Posted March 10, 2012 Report Share Posted March 10, 2012 I bought this module here, and also a FPGA minimum board to do the same test. Link to comment Share on other sites More sharing options...
voelker Posted March 11, 2012 Author Report Share Posted March 11, 2012 Hi, @Jax_Kobi, if you want to start developing hardware in VHDL i would recommend buying a true development board such as : http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400,897&Prod=NEXYS3 or http://www.digilentinc.com/Products/Detail.cfm?NavPath=2,400,789&Prod=NEXYS2 or (if you prefer Altera) http://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=139&No=593 or http://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=53&No=226 this kind of board offer peripherals that are usefull in the learning process (leds, memories ...) . You can find lot of good documents on internet such as "the VHDL cookbook", Altera also has some free vhdl course (http://www.altera.com/education/training/courses/OHDL1110). My method to learn, is : 1) learn the basic language construction and theory 2) Find a project you want to work on even if it might seem too hard 3) Divide the project into small chunck. This way you 'll see you project progress. 4) Start having fun working on your project and share with people onf forum, blog ... @hechao: Nice to have someone testing my code. I just hope your FPGA board has a 32mhz oscillator, or you 'll have hard time generating the 24Mhz pixel clock ... Link to comment Share on other sites More sharing options...
Jack Gassett Posted March 11, 2012 Report Share Posted March 11, 2012 Voelker, Looking like great progress, I just remembered that Dan Strother released some OpenCV compatible vision code here: http://danstrother.com/2011/06/10/fpga-stereo-vision-core-released/ As a bonus it might even have been tested directly on the Papilio, Dan has a Papilio. Might help you out in some way. Jack. Link to comment Share on other sites More sharing options...
alvieboy Posted March 12, 2012 Report Share Posted March 12, 2012 Ok, I admit, this is cool I'm too lazy to take a look at your code, so a few questions: * what's your I2C frequency ? * bit depth ? (I assume <16bit) Why are you considering not to use DSP48A for the job ? They are smaller than general purpose shifters. Reason I'm asking this is because eventually I can have ZPUino processing the image with some MACC extensions. That would make things easier for, as example, streaming the video using some codec over ethernet or some other high speed interface. Alvie Link to comment Share on other sites More sharing options...
voelker Posted March 12, 2012 Author Report Share Posted March 12, 2012 Hi, the i2C frquency is 200khz, and i'am not using the DSP48A because the Spartan3E does not have any ... The pixels bit depth is My sobel filter can make use of the embedded multipliers, but i'am trying to save them for convolution that would necessitate mulitplication by factor other than 2,4,8. Right now, i'am not considering compressing the data, because compression is usually a memory consuming task and i want to use only BRAMs. I will have a look at the zpuino and see how i can use a softcore to perform other kind of computation. Link to comment Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.