An automated Papilio build environment with Cloud9 IDE in Docker


Recommended Posts

I've been working on making a build environment that has everything needed to do Papilio development already installed and ready to go, including the Cloud9 IDE.

This is a rough start and still needs a lot of work but might be worth checking out. Do the following to get it up and running on your local machine.

You need to have Docker installed and working first, then:

docker run -v $(pwd):/workspace -p 8181:8181 gassettj/papilio-environment-cloud9 --auth username:password

You can get the ip address of the cloud9 instance by doing:

docker-machine ls

This will give you an ip address under the URL column. 

You can then open up a web browser at http://<the docker machine ip>:8181 and use username=username and password=password

Once you are in cloud9 you will have to upload a Xilinx.lic file to the ~/.Xilinx directory (There is a upload tool built into the file menu).

In the Cloud9 IDE you will see a terminal window, in that window type:

git clone https://gitlab.com/Papilio-FPGA/papilio-quickstart-vhdl.git
cd papilio-quickstart-vhdl/
fusesoc --cores-root=. build webpack_quickstart

You should see the project build in the terminal window and you can navigate to the project source files on the left. Once the build is done you can right click on the bit file that is generated, download it, and run it on the papilio pro board.

Still needs a lot more work, but is a pretty cool start!

Papilio-Cloud9.png

Link to comment
Share on other sites

Sounds promising. I will hopefully try it out soon. I'm not familiar with Cloud 9, does it have an editor with VHDL support?

I think the final target of your effort a cloud build environments or "casual users". For a local development workflow there is not really an alternative to have access to ISE and the isim gui to analyze and troubleshoot the code.

On my local machine currently I use a combination of Geany as a lightweight editor with VHDL and build support, FuseSOC and ISE/isim. It works quite well. Maybe one day I will learn enough tcl do write a script which updates a FuseSoc core description with the changes from ISE (e.g. adding a source file).


Thomas

 

 

Link to comment
Share on other sites

After taking a look at the Coud9 website I'm still a bit confused how things are setup, e.g. what runs in the cloud and what locally. I have seen that you can create a SSL connection between the build/tools machine in the IDE. Does this require a unique public IP address? Maybe you can outline the deployment setup you aim for.

 

Link to comment
Share on other sites

Hey Thomas,

I put this together because the number one complaint that people had in the back to basics thread was frustration with downloading and installing the tools and setting up the environment. This solves that problem...

Think of it as a lightweight virtual machine, it has all the tools needed pre-installed but without the need to include the OS. Xilinx ISE, fusesoc, zpuino toolchain, a local version of cloud9, and could also include your bonfire toolchain and the arduino toolchain. It has everything pre-installed and configured to build bit files for the papilio board. 

If I had done this with virtualbox then I would have to include the OS inside virtualbox, we would easily be pushing a 30GB download really quick. With Docker we can setup a VM that does not require the OS and has the benefit of a centralized repository and compresses the download. If I remember correctly you only have to download 4GB or something like that for this setup.

If you are already on Linux its super easy to get up and running. Just do a "apt-get install docker.io" and then the docker run command that I posted earlier. Docker does all the rest, it will download the virtual machine (called images in docker) and then run it in a container and bind the port 8181 from the container (think virtual machine) to your local machine port 8181. Then you just fire up a browser, connect to port 8181 and you are in a versio of cloud9 IDE that is running in the container (virtual machine) with a terminal window that has everything setup to make a bit file inside the container. The other thing docker will do is that it will map the workspace inside the container to the directory that you ran the docker run command in so you can work with the source files with cloud9 or any local editor you prefer. You just need to supply a Xilinx.lic file and you are up and running in a couple of minutes...

Try it out, its magical. :)

Jack.

Link to comment
Share on other sites

Ok, I was not aware that there is a local version of cloud9, my understanding was always that you need to connect to their cloud.  I have some basic knowledge about docker, but not really experience with it. What you describe is hopefully a good way for people who want to "consume" reproduced  FPGA IP, maybe reconfigure it a bit. When you need serious debugging you will at the end require at least the isim gui. I think a good IDE together with FuseSoC can easily replace ISE, but not  the simulator/debugger. 

I hope I have a chance on the weekend to try it out.

 

Link to comment
Share on other sites

Hi Jack,

I was able to start the container now. So on a first glance it works. But I'm still fighting with Docker, it runs only with sudo (as root), which also means that all worksapce files are owned by root. Most commands in the docker documentation didn't work, neither "docker-machine  ls". The different docker versions/editions seem to have different commands. Can you tell me what docker version and eidition you installed?

Thomas
 

Link to comment
Share on other sites

Hi Jack,

I tried out the docker image and extended it with a "bonfire version":

  • Added the riscv-gcc toolchain to it
  • Added the lua 5.1 and some lua libs to it (is required for the eLua build system)
  • Replaced fusesoc with "my" version ;)

Works really well. I think I will also add ghdl to it, than I'm finished.

The only "drawback" I see with the current image, is that it runs only as root (I have tried the --user option of docker, but than it does not work correctly). Running with root means that using the image will clutter the users home directory (which is mapped to /workspace as volume) with lot of files owned by root.Have you build your image with a Dockerfile or "interactivly" with docker commit?

Another question: Are you sure that Xilinx has no problem with providing a Docker image with ISE installed? Is releasing it without a license key "enough"?

 

Thomas
 

 

Link to comment
Share on other sites

9 hours ago, Thomas Hornschuh said:

Hi Jack,

I tried out the docker image and extended it with a "bonfire version":

  • Added the riscv-gcc toolchain to it
  • Added the lua 5.1 and some lua libs to it (is required for the eLua build system)
  • Replaced fusesoc with "my" version ;)

Works really well. I think I will also add ghdl to it, than I'm finished.

The only "drawback" I see with the current image, is that it runs only as root (I have tried the --user option of docker, but than it does not work correctly). Running with root means that using the image will clutter the users home directory (which is mapped to /workspace as volume) with lot of files owned by root.Have you build your image with a Dockerfile or "interactivly" with docker commit?

I'm not sure, one of the downsides that I'm worried about is that maybe docker is too complicated for some people... I was looking at other options and Eclipse Che might be easier for end users. Eclipse Che is hosted as a cloud service at Codenvy.io and I was able to create what is called a Factory which is just a URL that people click on and it will automatically start up the docker environment, any example projects we want, and the eclipse che IDE. Here is an example:

https://codenvy.io/f?id=factoryyd01z6lgdawlk52f (fuseSOC is not working correctly in this example but gives the idea of how easy this is for an end user.)

Then if a user wants to be local there is a docker command they can give to pull their workspace down locally...

The IDE is not as slick as the cloud9 IDE and there is no default support for VHDL and Verilog files. But, it looks like it uses the same standard as the VS Code so we can pull those modules into Eclipse Che and have nice support for not only VHDL/Verilog, but also code snippets, templates, and builders/wizards for ZPUino and Bonfire. I think it looks straightforward to get a graphical wave viewer supported too for simulations.

 

Quote

Another question: Are you sure that Xilinx has no problem with providing a Docker image with ISE installed? Is releasing it without a license key "enough"?

 

Thomas
 

 

I really don't know. :) I started down this path because I noticed that Xilinx has released an Amazon AMI (virtual machine) that has Vivado pre-installed and direct people to run the licensing tool after they start the AMI. I figured that if they were doing this then they must realize the benefits of this type of approach for their customers... Maybe they will shut it down at some point, but I have a backup plan. I'm thinking to develop an "agent" that will run on your workstation and that will be called from the cloud instance. When you invoke it, it will download all project source files, synthesize on your local machine, and upload the results back to the cloud server.

Ultimately though, if Xilinx will not allow this type of use case then I will be migrating Papilio boards away from Xilinx and to other vendors that do allow it. 

Link to comment
Share on other sites

On 31.10.2017 at 1:19 AM, Jack Gassett said:

I'm not sure, one of the downsides that I'm worried about is that maybe docker is too complicated for some people.

Docker is indeed not intended for "End Users",  it is a product for DevOps professionals. Do use it you need also understand basically how it works. For example it is not a got idea to always execute the docker run command you mentioned above every time, because you will fill up your machine with containers. It is better to use only docker start/stop after the initialization.

It helps of course to bundle independent software packages (e.g. ISE, fusesoc and the gcc toolchain for your favorite soft processor) into a reproducible environment. But it has also a lot of drawbacks. The question is what you want to reach. 

The cloud9/Docker approach would make senses when combined really with the cloud version of cloud9, running also the builder in the cloud. Combined with maybe some "extended" version of fusesoc to configure e.g. the slots of zpuino. For people that want to use the  FPGA as a "super-flexible" Microcontroller instead of diving deep into FPGA design. 

In the meantime I have switched the workflow for Bonfire to FuseSoC. For editing and debugging I use ISE, but only on the project generated by FuseSoc (with the --no-export option). So i can use the comfort of ISE and isim, but have always a project structure which I can easily reproduce on a new computer/VM.

I'm always trying to have an environment which I can recreate in a few hours. In the past I had often a development machine, maintained over years where I simply didn't know anymore how it was setup and was therefore also not able to break down the prerequisites of a project. Nowadays my development VMs are usually only a few weeks or months old and I frequently create new ones. So I will not update my Vivado 17.02 VM to 17.03, I will create a new one from scratch and move all my work over. I never install anything besides Office and a Web Browser on the host system (which usually is Windows).

fusesoc helps a lot, because it is a tool chain agnostic method to organize and package HDL projects. 

 

BTW: I have started with the zpuino UART to isolate it and extract it to a FuseSoC project, I will post details soon. 

 

Link to comment
Share on other sites

  • 9 months later...

I like the idea of having a VM to avoid all the install hassle. I finally setup a kubuntu 14.04 VM with DesignLab and ISE. I can offer that VM in case anyone wants it - however the license would have to be removed first.

I ran your docker container as explained here and got cloud9 running. How can I run DesignLab and ISE? Is that possible?

Link to comment
Share on other sites

  • 4 weeks later...

Ok, I figured out how to run ISE from this docker image (tested on ubuntu 14.04):

 

  1. Install docker:
    wget -qO- https://get.docker.com/ | sh

     

  2. Set user and group permissions (might need logout and new login to take effect):
    sudo usermod -aG docker osboxes

     

  3. (Re)start docker service:
    sudo service docker restart

    Now you can run the command given by Jack before:

     

  4. Enable access to X server:
    xhost local:root

    or may better since a bit more restrictive:

    xhost local:docker

     

  5. Finally run ISE:
    docker run --entrypoint "ise" -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $(pwd):/workspace gassettj/papilio-environment-cloud9

    as you can see you have to modify the entrypoint, set the DISPLAY env variable and bind /tmp/.X11-unix - by the way it also works with gassettj/papilio_environment:

    docker run --entrypoint "ise" -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $(pwd):/workspace gassettj/papilio_environment

     

This will start ISE in a window and ask for the license file.


[1] https://docs.docker.com/engine/reference/run/#foreground (-it param)
[2] https://forums.docker.com/t/docker-run-cannot-be-killed-with-ctrl-c/13108 (docker stop)
[3] https://medium.com/@oprearocks/how-to-properly-override-the-entrypoint-using-docker-run-2e081e5feb9d (entrypoint)
[4] https://github.com/jessfraz/dockerfiles/issues/6 (xhost)

 

You can also run a bash shell in the docker container for testing and debugging with:

docker run -it --entrypoint "/bin/bash" -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $(pwd):/workspace gassettj/papilio-environment-cloud9

in order to stop the docker containers (especially with the original entrypoint as Jack set it) use ps to get the container id and then stop it:

docker ps
docker stop <container>

In order to get more info on how the docker image was created, run:

docker history gassettj/papilio-environment-cloud9 --no-trunc

 

Link to comment
Share on other sites

In order to provide your license file the License Configuration Manager (xlcm) has to be run. To make this work the package libqt4-network and a symlink are needed. I created a docker image providing these modifications. This enables to run ISE with correct license.

To run ISE you need to have Docker installed and working first, then:

xhost local:root
docker run -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $(pwd):/workspace drtrigon/papilio-environment-ise bash -c "/opt/Xilinx/14.7/ISE_DS/common/bin/lin64/xlcm; ise;"

This will first start the License Configuration Manager allowing you to add your Xilinx.lic license file (load it from /workspace directory). After you close that dialog it finally starts ISE in a window.

Further info can be found in: https://github.com/drtrigon/docker-papilio-environment-ise or https://hub.docker.com/r/drtrigon/papilio-environment-ise/

Hope that helps someone else too. Incredible to me is the fact that the whole build environment suits into a 2 GB docker image (5.88GB on the local machine) while the download of ISE is around 6 GB and the installations needs something around 20 GB. That's very cool IMHO! Thanks Jack for starting this! What are your future plans for this docker environment?

Link to comment
Share on other sites

Dockerfile including DesignLab: https://github.com/drtrigon/docker-papilio-environment-ise/blob/master/Dockerfile.DesignLab

Build it using:

docker build -f Dockerfile.DesignLab -t test:latest .

Run it using:

docker run -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $(pwd):/workspace test:latest bash -c "/opt/Xilinx/14.7/ISE_DS/common/bin/lin64/xlcm; DesignLab-1.0.8/DesignLab;"

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.