Serial port issues

[mindrunner]

Hi!

 

I have a weird problem with my papilio pro board. I am sending large amounts of data over the uart port to an attached computer. Sometimes, the computer reads a nullbyte (0x00), which was not sent by me. I tried ZPUino and Misoc's LM32 on the papilio side and a couple of Java serial libraries as well as pyserial on the computer's side for reading. All with the same problem. Some guys in IRC told me that this might be an hardware related issue an I need to stitch the ground plane together with a bunch of short bridges across signal traces.

 

Is this a well known problem?

 

I am not a hardware expert, so I might need some advice or maybe a picture from a fixed board to exactly know what I need to do. 

 

 

Thank you!

[Jack Gassett]

Mindrunner,

 

No, you should not have to stitch the ground plane. You might have a board with an issue, if you would like, send us an email at support@gadgetfactory.net and we will send you a replacement. Just include your shipping address and a link to this forum thread.

 

Thanks,
Jack.

[alvieboy]

We sort of traced this back to the PC application, although it's still not clear why app fails to read stream properly.

 

Alvie

[mindrunner]

At the moment, I am able to reliably read with a c program from the serial port. Reading from a ZPUino SoC gives me full 3MBit. Misoc is slower. Unfortunately, I made most of my tests with python which seems to have problems to read the Data correctly.

 

Maybe I am using it wrong, I have no idea...

 

FPGA side:

http://pastebin.com/yS2zReDr

 

 

PC side:

 

serialread.py

http://pastebin.com/qD5WMNGv

 

called with: 

./serialread.py /dev/ttyUSB1 3000000

 

output:

http://pastebin.com/fkDTXsck

  

 

I would be really interested, what I am doing wrong.

[Felix]

Try using crlf (\r\n) instead of just lf (\n) to see if that does anything? (I wouldn't expect it to but you never know)

\00 is the Python way to represent 0x0 but it's not immediately obvious where the null byte is coming from

[mindrunner]

I already tried the different linebreak variations. Also this null-byte is kind of mysterious. Where does it come from? I am definitely never sending it...

[Jaxartes]

Hmm.  Miscellaneous thoughts, mostly things you could try in debugging:

 

1. You say your C program doesn't have the problem.  Would it let you know if it got a 0x00 byte?  Depending on how the C program is written, the 0x00 byte might be invisible.

 

2. What CPU utilization is shown by your Python program?  Is there any chance it's failing to keep up with the 3Mbps data rate?  (I have no idea what effect it would have, if it failed.)

 

3. What happens with a different baud rate setting?  This could make a difference a number of ways.  If something is failing to keep up with the speed, of course it would show here.  And I had problems with one of my designs, where the host (computer) side was using a different baud rate than I told it to.  (I didn't realize 28.8kbaud was nonstandard.)  Then the problem was far more severe than this (all bytes were corrupted); I wonder if a small mismatch could cause subtler problems, though I don't think it should.

 

4. Just a random note, another program that can read/write the serial port on Linux is "screen /dev/ttyUSB1 [baudrate]"; exit with control-A backslash, or it'll leave a background process talking to the serial port.

[alvieboy]

Hmm.  Miscellaneous thoughts, mostly things you could try in debugging:

 

1. You say your C program doesn't have the problem.  Would it let you know if it got a 0x00 byte?  Depending on how the C program is written, the 0x00 byte might be invisible.

 

I wrote the application for hin to test. If a 0x00 is received it will be written to a file. 

 

Alvie

[mindrunner]

Hmm.  Miscellaneous thoughts, mostly things you could try in debugging:

 

1. You say your C program doesn't have the problem.  Would it let you know if it got a 0x00 byte?  Depending on how the C program is written, the 0x00 byte might be invisible.

 

2. What CPU utilization is shown by your Python program?  Is there any chance it's failing to keep up with the 3Mbps data rate?  (I have no idea what effect it would have, if it failed.)

 

3. What happens with a different baud rate setting?  This could make a difference a number of ways.  If something is failing to keep up with the speed, of course it would show here.  And I had problems with one of my designs, where the host (computer) side was using a different baud rate than I told it to.  (I didn't realize 28.8kbaud was nonstandard.)  Then the problem was far more severe than this (all bytes were corrupted); I wonder if a small mismatch could cause subtler problems, though I don't think it should.

 

4. Just a random note, another program that can read/write the serial port on Linux is "screen /dev/ttyUSB1 [baudrate]"; exit with control-A backslash, or it'll leave a background process talking to the serial port.

 

1. Like Alvie said, every read byte is written to a file.

 

2. Will check that tomorrow, but I doubt that 3MBit of data burns my CPU

 

3. I dont get the question. FPGA and PC uses different baud rates, I am not getting any stable communication, which is the expected.

 

4. I dont like screen for reading serial port, because it kind of messes around with newlines. I rather use picocom instead, which unfortunately cannot handle more than 115200 baud. But might give it a go tomowwow as well.

 

 

@Alvie:

After running the simple string communication tests with your program a ported my java code to c. I am now reading my data with a c program. The serial i/o routines are exactly the ones from you. However, I can get a stable connection at 250000 baud, transferring 120kb/s of data. 3 MBaud gives me lots of errors. Its kind of strange. It is still Misoc/LM32 on the FPGA side. I would like to port it to ZPUino, but I do not know if I have got enough time left for that.

 

 

Cheers

Lukas

[alvieboy]

Lukas: you're losing data, because you don't read from serial port fast enough, and usually serial port buffers/FIFOs are small. LM32 is not to blame here.

 

I suggest:

* use a thread, with a simple read from serial port, which posts to a memory queue.

* another thread reads from there, processes, and outputs.

 

I can help you with that, if you need.

Objective is to let read() get data from serial port as much as possible, and defer processing of data.

[Jaxartes]

My question #3 about different baud rates -- was about different from 3Mbps, not from each other.  A lower baud rate on both sides, might not be as demanding about timing.  It's also possible that one side is not accepting the baud rate setting it was given.  It's happened:  I set my computer, and my FPGA design, to do 28,800 baud, but the computer did 38,400 baud instead.  In that case the failure was blatant, though; not like this.

 

I agree with Alvie's comment about reading fast enough.  While our computers are able to process 3Mbps easily on average, doing it reliably, continuously, is another story.  I'm not sure what-all buffers are involved in reading this serial port, but the FTDI chip's buffers are 384 bytes in one direction and 128 bytes in the other direction.  At 3Mbps, a buffer-full would pass in about a millisecond.  If the operating system schedules another task it may let it run for longer than that.  (Depending on settings; I know our Linux platform at work is set to one millisecond, and many are set up for longer.)  And random access to a spinning disk can take a few milliseconds.

 

Although, that would only explain the loss of data (lines 3482108 through 3482112 in the output you posted); I don't see how it would insert the zero byte.  And I've seen similar lossage at only 115.2kbps.

[hamster]

Make sure you have software flow control turned off on the pprt , or you will lose all you XON/XOFF characters. (\x11 and \x13 IIRC)

Mike

[mindrunner]

Lukas: you're losing data, because you don't read from serial port fast enough, and usually serial port buffers/FIFOs are small. LM32 is not to blame here.

 

I suggest:

* use a thread, with a simple read from serial port, which posts to a memory queue.

* another thread reads from there, processes, and outputs.

 

I can help you with that, if you need.

Objective is to let read() get data from serial port as much as possible, and defer processing of data.

Thats what I assumed first as well. I already implemented threading. However, since the communication with the FPGA is totally synchronous, it actually does not make any difference. Anyway, I could figure out the following facts:

 

1) Downstreaming data from fpga is possible with 3MBit

2) When sending data, I am losing bytes. Probably because the FPGA is not able to read fast enough. Since there is no threading on that side, there is no "easy" way around. I already got an idea... Will try that out later.

3) Python is really CPU intensive when reading data. Reading with 3MBits stresses one of my cores to 100%, which might be the reason for losing data here. (Thanks Jaxartes)

 

Cheers

Lukas

[alvieboy]

 

2) When sending data, I am losing bytes. Probably because the FPGA is not able to read fast enough. Since there is no threading on that side, there is no "easy" way around. I already got an idea... Will try that out later.

 

As I told you, I am able to stream raw audio @3MBit/s to ZPUino, play it, and no drops/errors. Full stream is 44100Hz*16bit*2 = 1.411200Mbit/s, and with some overhead (I use HDLC framing). ZPUino has a 2MByte receive buffer, as long as you do not exceed that, you're OK.

 

I can share that code with you, but note that it uses my SerPro library, and for PC only QT and GLIB versions are available (not Java nor Python).

 

Alvie

[mindrunner]

Cool, sounds good for me! I am gonna try that out.

[alvieboy]

Please, send me an email so I won't forget this.

 

I've most of those libs/apps scattered around my repos.