riscv
Build fails in endless loop, downloading more than a million files
1. Problem
I attempted to build this RISC-V OpenOCD, but I had to stop the build after two hours because the harddrive of my virtual machine got full. I'll explain in more details.
2. How I build OpenOCD
I'll explain precisely how I build OpenOCD. First I install an Ubuntu virtual machine on my Windows 10 computer in Oracle VirtualBox. My current virtual machine is Ubuntu 20.04.1 LTS. Then I follow these steps:
2.1 Install docker
I issue these commands to install docker:
$ sudo apt-get clean
$ sudo apt-get update
$ sudo apt-get install -y apt-transport-https ca-certificates curl software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ version_name=$(lsb_release -cs)
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu ${version_name} stable"
$ sudo apt-get update
$ sudo apt-get -y install docker-ce
To check if docker is properly installed, I run:
$ sudo docker run hello-world
To configure docker as regular user:
$ sudo groupadd docker
$ sudo gpasswd -a ${USER} docker
$ sudo service docker restart
$ shutdown -r now
The last command reboots Linux. Now Docker should work without sudo command. To test, run:
$ docker run hello-world
2.2 Download build scripts
First I cleanup anything left from a previous build:
$ sudo rm -rf ~/Downloads/openocd-xpack.git
$ sudo rm -rf ~/Work
Now I download the build scripts from Liviu Ionescu:
$ git clone --recurse-submodules https://github.com/xpack-dev-tools/openocd-xpack.git ~/Downloads/openocd-xpack.git
His build scripts need a slight edit, such that the RISC-V OpenOCD fork is pulled instead of the regular OpenOCD:
$ cd ~/Downloads/openocd-xpack.git/scripts/
$ gedit defs-source.sh
The last three lines in defs-source.sh should be uncommented and changed into:
OPENOCD_GIT_URL=https://github.com/riscv/riscv-openocd.git
OPENOCD_GIT_BRANCH=master
OPENOCD_GIT_COMMIT=HEAD
The build scripts are ready now.
2.3 Start build
To do the build, just issue the command:
$ bash ~/Downloads/openocd-xpack.git/scripts/build.sh --all
Eventually you should find the build output in ~/Work/openocd-0.10.0-14/deploy. The beauty of this procedure is that it generates builds for Windows 32b, Windows 64b, Linux 32b and Linux 64b!
3. What goes wrong
The build scripts from Liviu work perfectly for building the official OpenOCD repo. I've also applied them for the Nuvoton fork (see https://github.com/OpenNuvoton/OpenOCD-Nuvoton), and I can confirm it works on that repo as well. So I believe the build scripts are good.
Applying them on this RISC-V repo results in a problem though. The build lasted for more than two hours. I had to stop it because the ~/Work/ folder filled up with more than a million files and 13.5 GB of data.
Please help.
Kind regards, Kristof Mulier
Have you please tried the standard build procedure (bootstrap - configure - make - make install), as described in the readme? Does it work for you? Do you have any special requirements why simple native build would not suffice?
This is perhaps not not the right place where to ask about Liviu's build scripts. Not sure if these scripts are intended to work with riscv-openocd out of the box or if anybody over here uses them regularly for their builds. You'll perhaps have a better chance of getting help directly in the repository where the scripts are hosted: https://github.com/xpack-dev-tools/openocd-xpack/
Quoting the author from https://github.com/xpack-dev-tools/openocd-xpack/blob/xpack/README-BUILD.md:
(...) The entire process is quite complex, and an attempt to explain its functionality in a few words would not be realistic. Thus, the authoritative source of details remains the source code.
In another words: It is complex, and if it does not work as expected, you may need to dive into the code the scripts yourself.
Hi Jan, Thank you for your reply.
" Do you have any special requirements why simple native build would not suffice?"
Yes. The build scripts from Liviu produce build output for Linux 32b, Linux 64b, Windows 32b and Windows 64b. That's super nice. It's exactly what we need for our microcontroller IDE development (Embeetle IDE).
Unfortunately, I'm not familiar with the OpenOCD source code or the inner mechanisms of Liviu's build scripts.
Kind regards, Kristof Mulier
Understood. If you'd like to use Liviu Ionescu's build infrastructure as part of a product you're working on, then I suggest spending bit more time on studying the internals of the scripts (perhaps troubleshoot your issue & learn at the same time as you go).
Kind regards, Jan
I have used Liviu's (@ilg-ul ) xPack docker scripts to build the riscv-openocd fork successfully many times for many years so you must be doing something wrong,
E.g. use them as is but just edit these lines
https://github.com/xpack-dev-tools/openocd-xpack/blob/xpack/scripts/defs-source.sh#L41
to read:
OPENOCD_GIT_URL=https://github.com/riscv/riscv-openocd
OPENOCD_GIT_BRANCH=riscv
OPENOCD_GIT_COMMIT=HEAD
You don't need to understand anything more than that to use Liviu's scripts.
Hi @TommyMurphyTM1234 ,
E.g. use them as is but just edit these lines https://github.com/xpack-dev-tools/openocd-xpack/blob/xpack/scripts/defs-source.sh#L41
That's exactly what I did. I uncommented the last three lines from defs-source.sh and filled in the RISC-V repo URL for OPENOCD_GIT_URL. However, I see a small difference in the lines you proposed vs what I did.
Here is what I did:
OPENOCD_GIT_URL=https://github.com/riscv/riscv-openocd.git
OPENOCD_GIT_BRANCH=master
OPENOCD_GIT_COMMIT=HEAD
Here is what you propose:
OPENOCD_GIT_URL=https://github.com/riscv/riscv-openocd
OPENOCD_GIT_BRANCH=riscv
OPENOCD_GIT_COMMIT=HEAD
Your value for OPENOCD_GIT_URL doesn't end in ".git" - although that would probably not make much of a difference. Your value for OPENOCD_GIT_BRANCH is riscv instead of master. I didn't try that yet. I'll do that right now and get back here when done :-)
Huray,
The build worked! I run the executable with the --version flag:
$ ./openocd --version
Open On-Chip Debugger 0.10.0+dev-01250-g1712dc2c5 (2020-09-17-13:05)
Licensed under GNU GPL v2
For bug reports, read
http://openocd.org/doc/doxygen/bugs.html
So apparently the build must happen with OPENOCD_GIT_BRANCH given the value riscv instead of master. Thanks for your help @TommyMurphyTM1234
Kind regards, Kristof
I think if you specify the master rather than riscv branch then it tries to pull (and build?) lots more than just openocd - e.g. a full riscv toolchain, qemu etc.
Hi @TommyMurphyTM1234 ,
It looks like you're right. Although it's still weird, because building the master branch for the standard OpenOCD repo doesn't invoke all those extra things. It's strange that the master branch of the RISC-V fork behaves so differently.
Note: to build the standard OpenOCD repo, I put the following 3 lines at the end of defs-source.sh:
OPENOCD_GIT_URL=git://git.code.sf.net/p/openocd/code
OPENOCD_GIT_BRANCH=master
OPENOCD_GIT_COMMIT=HEAD
Kind regards, Kristof
This is simply down to how sifive are naming/arranging their branches. I don't think that you can infer what master should do here from what it does elsewhere (e.g. in the official openocd repo). If you want to build openocd from this forked openocd repo then just use the riscv branch (or another experimental branch if relevant) and not master.
Thank you @TommyMurphyTM1234 . Once again, you saved my day! I'll close this issue as it is solved now.
Kind regards, Kristof
