OrangeC
OrangeC copied to clipboard
LADSoft
travis-ci build
now that github can trigger travis-ci builds again, write a build-script for the travis-ci build. this is ONLY to build once with an alternate compiler and is not intended to compile the compiler with itself. Challenge is it has to run on linux and should probably use GCC. It is assumed something similar to the current scripts could be used as omake is somewhat compatible with gnu make.
You may want to not only define but also add two internal variables to the Makefile and sub-Makefiles:
$(COPY) (copy or cp) and $(PATHSEP) (\ or /) and additional in sub-Makefiles: $(MKDIR) (mkdir or mkdir -p)
$(PATHSEP) should fix the current issue.
$(MKDIR) should fix the now current issue - the normal unix mkdir doesn't create the missing child directories, but mkdir -p will; for portability $(PATHSEP) would still be reasonable.
And maybe investigate why $(NULDEV) evaluates to NUL instead of /dev/null
wish i had read your comments before working on it... could have used mkdir -p. that $(PATHSEP) idea is looking good about now too lol! Maybe I will rework that so it is cleaner, after I get further along.
right now my main problem is with goofy things like random inclusions of windows headers and #include file names not having the right case.
Yhea, that's a lot of why I wanted things such as c++11 full threading support: it makes supporting linux SO much easier because you're removing windows header dependencies
... actually I was a bit surprised that you do all this in master branch. That is no issue, I just was expecting you'd do this in a ci-branch, do the complete cleanup and then a single commit to master. But after all those commits it lost its minor importance completely...
wish i had read your comments before working on it... could have used mkdir -p. that $(PATHSEP) idea is looking good about now too lol! Maybe I will rework that so it is cleaner, after I get further along.
That was the single reason I've posted them... And yes: doing a cleanup in the Makefiles is surely useful as this would lower the "special" cases to a minimum and make them easier to maintain.
right now my main problem is with goofy things like random inclusions of windows headers and #include file names not having the right case.
In any case it is good that you fix windows-dependencies where they exist by using the correct casing in includes and remove them where they aren't necessary like the change to system().
This definitely is worth the work, even in the parts that aren't ci-related!
the change to system() is necessary on windows, because some of the command lines omake generates while building the project are so long the built-in system() can't handle them (because of line length limitations on cmd.exe). Don't know if linux has the same problem... but really my goal for travis-ci is just to get it to compile so I can proceed with coverity. At least for now!
A good way to fix the issue with the omake command lines is to allow it to link it in parts, e.g. 1 part gets linked together to form a bigger blob that then gets linked together, done conditionally if it detects the whole thing would be longer than the command line length (8091 characters on anything XP and after), this would allow for a simple system to build it with the command line path limitations, however, this would be an interesting challenge to fix anyways.
A good way to fix the issue with the omake command lines is to allow it to link it in parts, e.g. 1 part gets linked together to form a bigger blob that then gets linked together, done conditionally if it detects the whole thing would be longer than the command line length
Do you think about this part as something omake could fix? If it is only about linking and omake would use an inbuilt rule for this, this may could work. Otherwise it looks like those long command lines should be splitted (also because of needed time to rebuild if only a part was changed) into groups within the offending Makefiles.
I was trying to think and it seems there is another reason for the SYSTEM() rewrite... I still support DOS (I keep trying to get rid of it but then get a lot of FLAK) and unfortunately FREEDOS was broken last time I looked and there was no prospects of them fixing it. Basically, I had to go through the HXDOS spawning rather than try to use command.com, because any attempt to use command.com with a standard version of system() just didn't work, with the borland compiler. Other DOS variants worked properly though... I pinned it down and told them what to fix but they didn't seem interested...
but as far as the fact of the 'huge' links omake is generating those in the first place. The original plan was to use a file-oriented thing sorta like what borland make does (you do @&&| and then end it with | and it puts everything in between in a temporary file and then puts the temporary file name on the command line). The tools all support huge files for that. But gnu make doesn't support it so I had to quit with that approach last spring when we were doing all the omake updates. The main solution would have to be, rewrite the make files to glob things together in smaller chunks and then do partial links (the linker will support it). But that will add specificity to the make file that i didn't really want to add... and there is no ability to make syntax changes to support a more general approach because of the gnu make compatibility problem. And the only other thing is to have omake detect the specific linkers that are likely to be involved and split lines accordingly (if it is even possible with all the linkers involved).
If anyone has any other ideas I'm open to them...
Congratulation for getting GCC and Travis working! I suggest to handle the Makefile cleanup in this ticket and then close it as it works now "build passing :-)".
Coverity has its own issue #63 so "travis with coverity" may also be handled there.
i did most of the cleanup the other night... just a min as there is one left...
note that I did get pretty much everything compiling, with the notable exception that I had to exclude OCIDE... too much direct dependency on windows.
unfortunately the changes for the travis build broke the appveyor build, hope to have it working again tomorrow...
at this point the continuous integration seems to work on both servers. I'm closing this.
We may open a new ticket for this but I'd suggest to reopen this one instead...
Since October 2018 Travis CI added Win32 environment, along with Visual Studio 2017 build tools, so it should be possible to use this one to also compile and run the existing regression tests on Win32 by only adjusting the travis configuration file.
I tried to do win32 as that was preferred to having to get it working on linux, but couldn't get it to work. I'll try again if you think they added it just last month...
I'll try again if you think they added it just last month...
Following the notes in the given Travis-CI blog entry should work. Please use a multi-os variant, keeping the parts that are currently working under linux host. Extending the commands done in win32 (up to what we do on AppVeyor) will need a check for the OS, but should be doable.
ok as it stands I've got what I think might work for the travis CI on a combined windows/linux, however, there are bugs in travis ci that don't allow me to complete it. Specifically, chocolatey won't work properly in the presence of secure environment variables. This seems to be emblemantic of a deeper problem with secrets in the windows environment in general.... I'll keep an eye on it and continue with this once they've got it resolved.
as far as the appveyor builds, I finally got all my ducks in order but then ran out of disk space on the appveyor server... trying to resolve...
I actually checked this a couple of weeks back, I couldn't find any updates that seemed to mean they might be working on it and left it at that.
So today I've tried the windows build again, it fails in the same place. The build tag still says windows support is in the very early stages as well...
guess we have to continue to table it for now.
guess we have to continue to table it for now.
OK, I'll drop a note here if I read something "new". Converning CI we also still have #255 and as it is said to be much faster than Appveyor it is likely useful to implement it, too.
Update: Windows on Travis should work relative well now. See documentation and open issues (none of these seem to be important for this repo). If anything is not clear afterwards then there's the invitation to ask at the linked discussion board.
i think i like travis a little better. than appveyor... if I can ever get the buidls to work lol!
well this was a nice idea, but it turns out the build time-out on travis-ci.org is 50 minutes. The individual builds definitely take longer than 50 minutes on that platform, so, we cannot do it. I will remove the logic to invoke the builds on windows...
note that the posix-seh rev of mingw64-mak built executables that had problems compiling the library, with omake -j:4. We are elsewhere using the w64 rev.
clang also had problems; I use clang rev 9 with success but excutables built on clang rev 10 may hang when building the library. I will probably open a separate issue to investigate further.
note that the posix-seh rev of mingw64-mak built executables that had problems compiling the library, with omake -j:4. We are elsewhere using the w64 rev.
So I guess there's an issue for this?
The individual builds definitely take longer than 50 minutes on that platform, so, we cannot do it. I will remove the logic to invoke the builds on windows...
Hm, so Travis now builds on linux with GCC as one build, correct? Can you please revert 8f374316962a0b5fc8d54dc7f6b9c4686a778af1 and comment the lines out instead of removing them? That would be useful to be able to re-investigate if things change in the future and also would provide a template for other environments like the git workflow.
And then - Why does the build take so long at all?
yeah travis now builds with GCC.
I readded the llines in .travis.yml but commented them out this time. Will push it later...
As far as why the builds take so long I dunno. I haven't done measuring but it seems like they complete more quickly on my home computer. I also have been making various fixes this year that seem to have the qualitative effect of speeding up builds on my home computer, but the builds on appveyor never change in speed no matter what I do at home. Except for minor fluctuations based I assume on appveyor load. I don't know why; assume it has something to do with the virtual machines but just don't know.
just for some closure I timed a build at home... took 20-25 minutes with omake -j:8. I did some experimenting and appveyor doesn't improve after -j:2, but -j:2 is an improvement over -j:1. Appveryor reports as having two processors, so...
It crosses my mind that the latest builds on travis were with -j:2, maybe I will round this out with an experiment with a higher number to see if the timing improves 😄
I tried Travis with -j:8. It didn't change anything. Travis gets into running the tests and then aborts on a timeout... I also got a report on the number of processors; Travis also reports two processors. At this point the obvious way to get Travis working is to cut out the third compile (the time is currently for three separate compiles of the suite plus the test run) but I don't know if it is really worth running the tests that way.
Then maybe remove the test part for those long-running environments? This would at least run one test - the compilation of itself.