Tony Ivanov

Lockups when trying to write to Wishbone register.

17 posts in this topic

Hi! I'm trying to create a library based on the "Wishbone VHDL" sample i think.

I'm currently stuck trying to write to a wishbone register, the ZPUIno just locks up while the vhdl-module continues to happily tick without any noticeable changes.

I've cross-referenced my code with the simple writeLeds example but I really can't see anything that should cause this lockup, please help!. ( btw I'm on an Papilio Pro board )

 

Here's my code:

I can see the print of line 13 just fine, line 14 initiates the write, and then it gets stuck and I never seems to reach line 15

https://github.com/telamon/papilio-dspwing/blob/865e322b7eb3d761216595fead3c85fb294811ed/examples/simple_tremolo/simple_tremolo.ino#L15

which goes to:

https://github.com/telamon/papilio-dspwing/blob/865e322b7eb3d761216595fead3c85fb294811ed/DSP_Wing.cpp#L33  <- how i perform the actual register write

https://github.com/telamon/papilio-dspwing/blob/865e322b7eb3d761216595fead3c85fb294811ed/DSP_Wing.vhd#L98 <-- Here i try to receive/read.

and finally..

https://github.com/telamon/papilio-dspwing/blob/865e322b7eb3d761216595fead3c85fb294811ed/tremolo.vhd#L68 <-- unpack and act upon the command. but that never happens either, so i can only assume that something went wrong with the register write inside the zpuino and that the changes to the register were never applied.

 

Best Regards

/Tony

Share this post


Link to post
Share on other sites

Hello Tony,

Sorry for the slow response to this message, I had the blinders on while working on a new product.

I'm trying to understand your setup here first, from what I've read of your code so far it looks like you have created a ADC wing and everything is being output to the ADC? Is that correct?

Have you had any luck with simpler things, such as outputting a tone or something like that?

I'm going to keep looking at the code, but wanted to get a reply out there.

Jack.

Share this post


Link to post
Share on other sites

Ok, so tracing through your code:

It looks like this is the last line that is called before ZPUino hangs:

dsp.enableFX(FX_TREMOLO,true);

This is what enableFX looks like:

void DSP_Wing::enableFX(unsigned char fx,boolean enabled){
  this->setFXCTRL(fx,PARAM_ENABLED,enabled?1:0);
}

Defines are:

#define REG_FX_CTRL 0

// Effects definition
#define FX_TREMOLO 1

// Parameters definition
#define PARAM_ENABLED 0 // param enabled is 0 for all effects

seFXCTRL is:

void DSP_Wing::setFXCTRL(unsigned char fx,unsigned char param, unsigned char value){
  REGISTER(IO_SLOT(wishboneSlot),REG_FX_CTRL) = ((value & 0xff) << 9) | ( (param & 0xf) << 5 ) | ( (fx & 0xf) << 1) | 1;
  //REGISTER(IO_SLOT(wishboneSlot),REG_FX_CTRL) = ((value & 0xff) << 9) | ( (param & 0xf) << 5 ) | ( (fx & 0xf) << 1) | 0;
}

setFXCTRL is meant to set the following control bus defined in the VHDL code:

			-- fx controlbus 0  0000 0000 00000000
			--               en fxid prop value

So this line is the most likely culprit:

REGISTER(IO_SLOT(wishboneSlot),REG_FX_CTRL) = ((value & 0xff) << 9) | ( (param & 0xf) << 5 ) | ( (fx & 0xf) << 1) | 1;

This is a known working example that writes to a control register:

void del1::writeLEDs(unsigned long value)
{
	REGISTER(IO_SLOT(wishboneSlot),0) = value;
}

Everything looks good as far as the code is concerned...

I talked to Alvie and he says that the only thing that will halt the ZPUino from the Wishbone bus is if 'ACK' is not handled correctly. So I'm going to look over the Wishbone_to_Registers you use and see if anything is wrong there:

https://github.com/GadgetFactory/DesignLab_Examples/blob/master/libraries/ZPUino_Wishbone_Peripherals/Wishbone_to_Registers.vhd

Alvie asks if you can post the *.ncd, *.ngd, and *.syr files.

Jack.

Share this post


Link to post
Share on other sites

I just compared how the Wishbone array is packed and unpacked:

In Wishbone_to_Registers line 68-80:

-- Unpack the wishbone array into signals so the modules code is not confusing. - Don't touch.
  wb_clk_i <= wishbone_in(61);
  wb_rst_i <= wishbone_in(60);
  wb_dat_i <= wishbone_in(59 downto 28);
  wb_adr_i <= wishbone_in(27 downto 3);
  wb_we_i <= wishbone_in(2);
  wb_cyc_i <= wishbone_in(1);
  wb_stb_i <= wishbone_in(0); 
  
  wishbone_out(33 downto 2) <= wb_dat_o;
  wishbone_out(1) <= wb_ack_o;
  wishbone_out(0) <= wb_inta_o;
-- End unpacking Wishbone signals

And in the top level file for the ZPUino lines 450-460:

  wishbone_slot_6_in(61) <= wishbone_slot_6_in_record.wb_clk_i;
  wishbone_slot_6_in(60) <= wishbone_slot_6_in_record.wb_rst_i;
  wishbone_slot_6_in(59 downto 28) <= wishbone_slot_6_in_record.wb_dat_i;
  wishbone_slot_6_in(27 downto 3) <= wishbone_slot_6_in_record.wb_adr_i;
  wishbone_slot_6_in(2) <= wishbone_slot_6_in_record.wb_we_i;
  wishbone_slot_6_in(1) <= wishbone_slot_6_in_record.wb_cyc_i;
  wishbone_slot_6_in(0) <= wishbone_slot_6_in_record.wb_stb_i; 
  wishbone_slot_6_out_record.wb_id_o <= wishbone_slot_6_out(49 downto 34);
  wishbone_slot_6_out_record.wb_dat_o <= wishbone_slot_6_out(33 downto 2);
  wishbone_slot_6_out_record.wb_ack_o <= wishbone_slot_6_out(1);
  wishbone_slot_6_out_record.wb_inta_o <= wishbone_slot_6_out(0); 

Everything matches up there and looks good.

Also the ACK in Wishbone_to_Registers line 84 looks good:

  -- Asynchronous acknowledge

  wb_ack_o <= '1' when wb_cyc_i='1' and wb_stb_i='1' else '0';

Jack.

Share this post


Link to post
Share on other sites

Wow big thanks for your time Jack! Yeah I'm really looking forward to the new product, It's a must have and great job! :D

Yep that's the line that freezes the ZPUino.

REGISTER(IO_SLOT(wishboneSlot),REG_FX_CTRL) = ((value & 0xff) << 9) | ( (param & 0xf) << 5 ) | ( (fx & 0xf) << 1) | 1;

, the sound from the ADC to the AudioWing continues happily as if nothing ever happened, I can try and just write a shorter or simpler value to the register to see If i get different results, or maybe try to write to a different register,

it shouldn't matter if i use Register0 or 1/2 right? ( Also when i resynthesize with the tremolo effect hardcoded to enabled it's heard loud and clear, so I kindof almost ruled out that the issue is in the VHDL code. )  

Let me get back a little bit later with the .ncd/ngd/syr files, I don't have access to my workdesk right now.

I don't think I recognize those files but I assume they're artifacts produced during the synthesis process? I'll take a look in my project folder as soon as i can.

Share this post


Link to post
Share on other sites

Tony, can you send me these files that are generated by the ISE build process:

.ncd

.ngd

.syr

.mrp (make sure you enable detailed map report, if possible -see options for Map generation)

Zip all of them and send to "alvieboy@alvie.com". I'll try to see what is happening.
The only "normal" lockout from the CPU point of view is when it tries to read/write to a IO location and ACK never comes in...

 

Alvie

Share this post


Link to post
Share on other sites
Quote

the sound from the ADC to the AudioWing continues happily as if nothing ever happened, I can try and just write a shorter or simpler value to the register to see If i get different results, or maybe try to write to a different register,

Yes, VHDL is synchronous to the clock, and completely independent of what CPU does.

 

Quote

it shouldn't matter if i use Register0 or 1/2 right? ( Also when i resynthesize with the tremolo effect hardcoded to enabled it's heard loud and clear, so I kindof almost ruled out that the issue is in the VHDL code. )  

No, all registers (if address is correct) shall at least ACK the transaction (although it may happen that read/writes do not read anything useful or have any side effects).

Can you also share the code (.elf) generated ? It may happen that the IO is being directed to a wrong or invalid address.

Share this post


Link to post
Share on other sites
2 hours ago, alvieboy said:

Tony, can you send me these files that are generated by the ISE build process:

.ncd

.ngd

.syr

.mrp (make sure you enable detailed map report, if possible -see options for Map generation)

I gathered up all i could find and just sent it to you, but i'm afraid i forgot to enable detailed map report, will have to send it separately.

Thank you 

EDIT: Activated detailed map-report, regenerated and resent.

This is the first time in my life that someone who wanted to help me debug a problem asked for the compiled binaries instead of the source, hehehe.

Speaking of debugging, there isn't a way to attach a debugger to a running FPGA in order to inspect its internal state right? Or something along those lines.. I haven't quite yet figured out how to troubleshoot my own designs.

Share this post


Link to post
Share on other sites

Some thoughts on debugging:  I'm not aware of any general FPGA "debuggers" but I wouldn't necessarily know.  (And I'm not a big fan of "debugger" tools anyway -- I often don't use them when I debug software.)

Debugging is a real important thing when working with FPGAs.  I recommend developing a wide variety of techniques, and patiently applying those techniques to the problem.

One thing you can do is add output signals derived from whatever you want to monitor.  So if you have a pair of LEDs unused, you could light one up when in a particular bus state, and light the other up when not in that particular bus state.  When you get it running, if the "yes" LED is lit, and the "no" LED is not, then it's perhaps stuck in that bus state.  If the "no" LED is lit, and the "yes" is not, then it's probably stuck in some other bus state.  If both are lit, it's not stuck in any one bus state (but may be stuck in a cycle of them).  Each time you do this, you might (or might not) get some information.  Repeated several times, monitoring for different states, might get you somewhere.  (Like: what's it doing? writing to an address.  is it the device address? no.  what address is it?  the wrong address.)

Make sure you got the right wishbone slot.  If the one in your C++ code (6) doesn't match the one in your circuit, then you might be writing to some other device's registers, which could have unpredictable and dire consequences, such as causing a lockup (or causing a loss of communication).

A "logic analyzer" tool exists in DesignLab, though I don't know much about it.

Share this post


Link to post
Share on other sites

Thank you Jaxartes!

Stupid as it may seem i did't actually consider connecting lots of leds for debugging purpose, since duplicating signal output to an alternative led is quite easy. (Soldering a small 8led wing might be the next project) 

Well yes basically I wanted to know if there was any way to read the current state of registers using the JTAG bus or something, getting the power of a logic-analyzer but without connecting any external hardware.

But nevermind that now, I guess papilio debugging is a completley different topic.

 

As for the lockups, yeah I've quadruple checked that the wing is connected to wishbone-slot 6 and that the c/c++ on the zpuino tries to use the same slot.

But I suppose that even if I wrote to the wrong wishbone slot then It shouldn't cause a complete ZPUino lockup?

Share this post


Link to post
Share on other sites

It may be possible to have a logic analyzer on the FPGA, and connect it only internally.

Regarding what a wrong wishbone slot might do:  No, there are ways it could turn out badly:

  • If the register it's writing to controls the serial port:  Then you might lose connectivity.  And it might appear like a lockup, even though your code is running ok, it can't communicate.
  • If there is no device responding to that address, not even enough to do an ACK:  Then it would wait forever.  I think ZPUino has something to prevent this, but I don't really know.

A few more debugging ideas:

  • Put short temporary debug prints in the sections of code you want to watch.  You can see with more precision where exactly it's getting stuck.  It might not be where you think it is.  I'd use single characters.
  • In place of the LEDs suggested above, use audio output as an indicator:  Superimpose a particular sound on top, to indicate a particular state.  It has its own difficulties -- it's not the simple on-off of LEDs, and you can't hear a 48MHz square wave (and probably wouldn't want to - ouch!).  But if audio's what interests you, and what you have, you can use it.

Share this post


Link to post
Share on other sites
1 hour ago, Jaxartes said:
  • In place of the LEDs suggested above, use audio output as an indicator:  Superimpose a particular sound on top, to indicate a particular state.  It has its own difficulties -- it's not the simple on-off of LEDs, and you can't hear a 48MHz square wave (and probably wouldn't want to - ouch!).  But if audio's what interests you, and what you have, you can use it.

Yeah that's basically the current state of the sourcecode, If that bit just gets flipped to a 1 then the audible effect should kick in.

 

1 hour ago, Jaxartes said:

It may be possible to have a logic analyzer on the FPGA, and connect it only internally.

Funny! That thought hit me an hour ago,  I guess it should be possible with some adaption to rewire the logic analyzer wing in the Designlab-examples to listen on internal signals instead of external pins. 

Share this post


Link to post
Share on other sites

So here's a summary of what Alvie found by looking at the generated files.

 

He traced it to the schematic where it's clearly visible that the wishbone bus of the wing is not connected to anything,

0.png

 

But in my schematic it looks like this: 

Screenshot from 2016-09-18 11-50-58.png

 

Super wierd.. o_o

Anyways, Alvie asked me to forward my current state of project-folder to you @Jack Gassett

I uploaded the whole wing folder here, if you have time:

https://www.dropbox.com/s/l18lyfrvpmfs11m/DSP_Wing.zip?dl=0 ~ 7mb. 

Meanwhile I'm gonna try and delete the wishbone connections and reconnect them, maybe it's just some graphical glitch that makes it look as if they're connected.. (hopefully not, that's a little bit scary)

Regards

/Tony

 

Share this post


Link to post
Share on other sites

... I solved it!! Damn i feel so stupid. I had accidentally connected the Output->Output and Input->Input... Damn 

Flipped the connections and now everything works as expected.

Big thanks to you Alvie! 

I'd later wish to learn how you Loaded up the generated-schematic or what file you opened to produce that image that showed the final schematic.

Huge thanks for the help!

Share this post


Link to post
Share on other sites
12 minutes ago, Tony Ivanov said:

... I solved it!! Damn i feel so stupid. I had accidentally connected the Output->Output and Input->Input... Damn 

Flipped the connections and now everything works as expected.

Big thanks to you Alvie! 

I'd later wish to learn how you Loaded up the generated-schematic or what file you opened to produce that image that showed the final schematic.

Huge thanks for the help!

That's easy. :)

"Design" tab, expand "Synthesize - XST" and choose "View RTL Schematic". Alternatively you can open the .ngr (RTL) or .ngc (Technology) file inside ISE, after synthesis.

The RTL view shows you the design prior synthesizing the low-level elements (LUTs, Flip-Flops, so on). The technology shows you how they are mapped into LUTs and FF and so on. This is still prior to mapping, so things can change a bit in the process.

Note that due to optimizations sometimes what you see may be a bit "mangled". In that case, enable "Keep Hierarchy" in Synthesis options (right-click on "Synthesize - XST", choose "Process properties").

Glad your problem is fixed now.

Alvie

Share this post


Link to post
Share on other sites

As far as debugging, think of it as debugging a circuit board. I find that I end up using the same techniques to debug a new circuit board as I do an FPGA circuit. You can also implement the Logic Analyzer internally in DesignLab. The only difficult part is that the Serial port is in use by ZPUino so the debugging has to go over the JTAG port. There is an example included in DesignLab of doing this.

Jack.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now