libxml2-win-binaries icon indicating copy to clipboard operation
libxml2-win-binaries copied to clipboard

Enable building arm64 libraries and upgrade from VS 2015 to 2019

Open niyas-sait opened this issue 2 years ago • 9 comments

The change will add support for building arm64 version of all of the libraries.

As the ARM64 toolchain is only available from VS 2019, I've upgraded all of the compilation except VS 2008 targets to VS 2019.

Unfortunately, the version of libiconv used does not yet support newer toolchains so I had to create a local copy of the VS project supporting MSVC 16 for compiling libiconv. There is a PR in progress for libiconv to add arm64 targets but that would be only merged to master and since we are using an older release I don't see any option other than a local project for the building. Other options are using a separate branch or a repository for iconv but I wasn't sure if that was okay.

The build scripts have been tested locally and with the appveyor and all of the targets can be successfully compiled.

https://ci.appveyor.com/project/nsait-linaro/libxml2-win-binaries/builds/41057902

The generated libraries are also tested with lxml package for arm64 and x86 packages and have passed all tests

TESTED VERSION: 4.6.3
    Python:           sys.version_info(major=3, minor=10, micro=0, releaselevel='candidate', serial=1)
    lxml.etree:       (4, 6, 3, 0)
    libxml used:      (2, 9, 5)
    libxml compiled:  (2, 9, 5)
    libxslt used:     (1, 1, 30)
    libxslt compiled: (1, 1, 30)
    FS encoding:      utf-8
    Default encoding: utf-8
    Max Unicode:      1114111

----------------------------------------------------------------------
Ran 1937 tests in 8.066s

OK

I've also created a release on my local repository with the generated packages deployed by the appveyor which can be found here

niyas-sait avatar Oct 07 '21 11:10 niyas-sait

Hello @mhils and @TBBle, Would you be able to review the changes, please?

niyas-sait avatar Oct 07 '21 11:10 niyas-sait

Thanks for the work here. As mentioned in https://github.com/mhils/libxml2-win-binaries/issues/20#issuecomment-875309934, I don't have the capacity to maintain this at the moment. I'm happy to hand over everything to @scoder or whomever he considers suitable. :)

mhils avatar Oct 07 '21 11:10 mhils

Thanks, @mhils for the quick response. I will wait to hear from @scoder for proposals.

niyas-sait avatar Oct 07 '21 11:10 niyas-sait

@scoder , any thoughts ?

niyas-sait avatar Oct 19 '21 20:10 niyas-sait

Thanks a lot for going through this, @nsait-linaro.

libxml used: (2, 9, 5)

While we're at it, we should upgrade to libxml2 2.9.10 and libxslt 1.1.34. That's what the other wheels currently use. I never restricted the versions of the other libs, so they should also just use the latest available.

The need to copy the libiconv setup is very unfortunate. Also, it doesn't look like they picked up your PR yet, so it might take a whole while until there's an actual release. Once that's out, we could probably just use it. I never had compatibility problems with libiconv.

I think having an appveyor setup that does the job is fine. Github Actions would be nicer to integrate into the lxml build, but having the libraries built once, in whatever way, is ok. We can stick to that.

Let me see how I can best integrate your repo.

scoder avatar Nov 01 '21 13:11 scoder

So, you said that you made a copy of libiconv. Could you maybe use a patch against the original sources instead? That would simplify upgrades (such as the imminent one).

scoder avatar Nov 03 '21 08:11 scoder

So, you said that you made a copy of libiconv. Could you maybe use a patch against the original sources instead? That would simplify upgrades (such as the imminent one).

libiconv is a sub-module to this repository. I can prepare a patch that we can keep in this repository that can be applied to extend the submodule libiconv for win/arm64. Is that what do you meant ?

niyas-sait avatar Nov 03 '21 18:11 niyas-sait

I can prepare a patch that we can keep in this repository that can be applied to extend the submodule libiconv for win/arm64. Is that what do you meant ?

Yes, exactly. Note that I forked this repo over to the lxml organisation and updated the submodules. I'll have to take another look, because I think some of the source URLs might be wrong, libxml2 maybe?

In and case, would be cool if you could add a patch over there, so that we only need to take care of the submodules themselves.

scoder avatar Nov 03 '21 21:11 scoder

I can prepare a patch that we can keep in this repository that can be applied to extend the submodule libiconv for win/arm64. Is that what do you meant ? Yes, exactly. Note that I forked this repo over to the lxml organisation and updated the submodules. I'll have to take another look, because I think some of the source URLs might be wrong, libxml2 maybe? In and case, would be cool if you could add a patch over there, so that we only need to take care of the submodules themselves.

@scoder I've created a PR to add the patch. I also had to revert few submodules as they were causing some issues. Details in PR

niyas-sait avatar Nov 05 '21 15:11 niyas-sait