aarch64 windows

[zhmushan]

Could we build an installer for aarch64-pc-windows-msvc ? (I'm working on Surface Pro X)

[kinnison]

In theory we could. std and host are built for this target. We need to know by what means (cross from Windows x86_64? cross from Linux?) and for that I shall request a hint from @pietroalbini on how we build that target's rustc/cargo.

[pietroalbini]

For rustc/cargo we're cross-compiling from Windows x86_64.

[kinnison]

Cool, would you expect GH Actions to just work if I rustup target install aarch64-pc-windows-msvc and then cargo build --target=aarch64-pc-windows-msvc ? If so I should be able to add the action pretty easily.

[pietroalbini]

I'm not sure exactly, I was not involved in setting that CI up. I guess trying won't hurt?

[kinnison]

Probably wouldn't hurt, no. I'll see if I can knock up a PR to check

[kinnison]

A brief test suggests this will be a significantly bigger job than thought.

Hyper doesn't support tokio 0.3 yet. tokio 0.2 relies on mio 0.6 - mio 0.6 relies on winapi 0.2.

I believe the fixes for aarch64 didn't make it into winapi 0.2 so we'd need to fix hyper to work on tokio 0.3 (and redo our test suite with that, which is less hassle) in order to eliminate incompatible winapi deps.

[kinnison]

tokio v0.2.24
├── h2 v0.2.7
│   └── hyper v0.13.9
│       ├── hyper-rustls v0.21.0
│       │   └── reqwest v0.10.10
│       │       └── download v0.6.8 (/home/dsilvers/dev-git/rustup/download)
│       │           └── rustup v1.23.1 (/home/dsilvers/dev-git/rustup)
│       ├── hyper-tls v0.4.3
│       │   └── reqwest v0.10.10 (*)
│       └── reqwest v0.10.10 (*)
├── hyper v0.13.9 (*)
├── hyper-rustls v0.21.0 (*)
├── hyper-tls v0.4.3 (*)
├── reqwest v0.10.10 (*)
├── tokio-rustls v0.14.1
│   ├── hyper-rustls v0.21.0 (*)
│   └── reqwest v0.10.10 (*)
├── tokio-socks v0.3.0
│   └── reqwest v0.10.10 (*)
├── tokio-tls v0.3.1
│   ├── hyper-tls v0.4.3 (*)
│   └── reqwest v0.10.10 (*)
└── tokio-util v0.3.1
    └── h2 v0.2.7 (*)

[kinnison]

When tokio 1.0 releases, hyper intends to do a release: https://github.com/hyperium/hyper/issues/2352 - In particular, https://github.com/hyperium/hyper/issues/2352#issuecomment-748382861 suggests that we should wait for Tokio 1.0 to come out, then hyper will be updated for that, then presumably reqwest, and finally we can update. That might pave the way for this target.

[Alovchin91]

It should already be possible to compile it by switching the backend from reqwest to curl. I managed to compile v1.22.1 some time ago. Didn't test it though as I'd prefer to do this kind of tests in a virtual machine and I don't run Insiders build, but maybe I'm just being too cautious 🤔

[kinnison]

Given that Tokio 1.0 came out today I'm more inclined to wait and then update as described above.

[Alovchin91]

I'm not suggesting to completely switch to curl. What I'm saying is — it can already be used to test as there's probably more to it (e.g. I know for sure MSVC detection in Cargo doesn't work on Windows arm64).

[kinnison]

If you want to test it and submit patches for anything broken, then you can try building on/for aarch64 windows yourself. However if you think Cargo is broken currently, then it seems counterproductive for me to fight to sort CI builds out for a platform I cannot test at all and that you think won't work anyway, rather than to work on other things.

If you want help setting up a build then I can guide you, though it should be fairly easy to try:

$ cargo build --no-default-features --features vendored-openssl,curl-backend --target aarch64-pc-windows-msvc

at a guess.

[Alovchin91]

Umm... Let me try to explain.

First of all, I never fought for anything. You're totally right that there is no reason to sort out CI builds if it doesn't build in the default configuration. But I never even mentioned CI, so please don't put words into my mouth.

Second, and most important, I just pointed out the fact that it can be done. I did build it but didn't test it because I was too cautious running an installer software on a real hardware as non-Insiders builds don't have Hyper-V on arm64 yet. The original question was "could we build"? I said "yes, it can be built, but...".

Third, I never said Cargo doesn't work. Again, please don't put words into my mouth. It works and I use it every day. I said there is a (known) issue with determining MSVC on an arm64 Windows. I can even tell you why is that — because Visual Studio Installer doesn't register an instance on arm64. I created an issue in Visual Studio Community but didn't create one for Cargo yet.

Honestly, now I really hope you're not part of rust-lang team, because your responses are rude and don't really motivate to help. It's not the attitude I feel comfortable with. Please reconsider.

[kinnison]

I apologise for the delay responding, and also for how my previous comment came across. The 'fighting' was about me and CI, since the only way I have to test builds is to push them and wait for Windows CI to run on Github, and was not intended to suggest you were in any sense being combative about this.

As for 'cargo being broken', perhaps I misinterpreted what you said about Cargo and MSVC detection and for that I apologise for my wording again.

I will endeavour to improve my language. While I was very tired and stressed when I came to this issue before Christmas it is no excuse for how I came across in that comment.

If you have any suggestions for how to get things going once our dependency chain has updated to Tokio 1.0 then I would appreciate the help, I'm not a Windows-based developer by default and so I do not know the intricacies of things like the visual studio registration you mentioned.

[Alovchin91]

Thank you for your response! We were all tired and stressed for the last few weeks I think. Hopefully we now have some time to recharge our batteries 🙂

For rustup, I think I could suggest the following plan:

1. I could still build rustup with curl locally and test it on my Windows arm64 machine (Surface Pro X). Since it's an installer, I expect there might be some issues with paths if for any reason anything is being written to Program Files or registry.

2. I remember there was an issue with the sys-info crate because it uses Intel asm to determine CPU clock on Windows. I know a way to solve this using WinNT API, so I'll prepare a Pull Request to that crate. But if anything, this code in rustup can be cfg-ed.

3. I don't know if it makes sense but I'll open an issue in Cargo about MSVC support. While it's not directly related to rustup, it's still nice to report it.

Unfortunately, there is still no CI/CD for Windows arm64, so the only thing that can be done is cross-compilation. Somebody will have to make sure it runs on Windows arm64, but I don't mind volunteering for now.

[kinnison]

Thank you for being prepared to help with this. I agree that step one is to see if you can get a build up and running at all.

In point 1 you talk about issues for program files and registry, what in particular differs for aarch64 windows here? AIUI we use the user's home directory for the most part, but we do adjust the PATH environment variable (don't know if that touches the registry?)

In point 2 - Our use of sys-info is via the effective-limits crate where we care only about memory consumption. If your PR to sys-info gets caught in a non-semver-compatible version change then we'll need to PR effective-limits to get the updated sys-info too.

For point 3, yes, I agree it's important to report these things. If Cargo (or rustc?) can help with that situation then it'll improve users' lives.

For rustup, we'll likely end up cross-building for that target anyway, so here's hoping there's no significant behavioural differences which will sit "untested" per-se.

Thanks again, and I look forward to the results of your attempts to cross-build. In the meantime I'll sit tight for all our deps to update to tokio 1.0 and after that I'll start that update to rustup.

[Alovchin91]

In point 1 you talk about issues for program files and registry, what in particular differs for aarch64 windows here? AIUI we use the user's home directory for the most part, but we do adjust the PATH environment variable (don't know if that touches the registry?)

I can't tell yet, I will need to check on that. Windows arm64 has 2 WoW subsystems (ARM32 on ARM64 and x86 on ARM64) so there might be some redirection when working with registry if this is how the PATH variable is changed. But for now this is just from the top of my head 🙂

[Alovchin91]

reqwest v0.14 has been released today: https://github.com/seanmonstar/reqwest/releases/tag/v0.11.0

Unfortunately, rustls feature does not compile on Windows arm64: https://github.com/briansmith/ring/issues/1167

Hope that this can be resolved without much trouble.

[kinnison]

Sounds like progress is being made indeed. I will probably get a chance to work on updating rustup to the tokio 1.0 ecosystem at the weekend.

Thanks for keeping an eye out.

[kinnison]

@Alovchin91 I've prepared a PR which updates us to Tokio 1.0 which I believe solves at least some of the winapi problems.

[Alovchin91]

@kinnison Great, thanks!

I didn't try to compile it yet but given that I couldn't compile reqwest with the same feature flags because of ring dependency, I don't expect it to compile just yet. We'll probably have to wait for @briansmith to either figure out ASM support on Windows arm64, or to merge a non-ASM version 🙂

I've been too slow to make a PR in sys-info crate so its master already supports Windows arm64 😅 There's still a room for improvement though so I've also opened a new PR to fix behavior on arm64: https://github.com/FillZpp/sys-info-rs/pull/82

[kinnison]

Fantastic news, we edge closer :D

[kinnison]

I am going to be removing this from the 1.24.0 milestone (and pushing it to 1.25.0) as I am not confident we'll get to the point of building for this platform in this cycle. If you happen to have an update on how the dependency landscape lies, it'd be handy to have it here. I believe Ring is still not quite there.

[Alovchin91]

Yes, didn't hear anything from Ring author ever since :\

[briansmith]

Yes, didn't hear anything from Ring author ever since :\

I do not know anything about your schedule but I would be happy to help. It would be useful to know the cut-off dates for the milestones you have.

[kinnison]

@briansmith Hi! Thank you for coming along.

Right now there's no particular timeline on getting aarch64-windows support into Rustup, though obviously we'd prefer to do it as soon as we can. One? of the blockers to this is ensuring we can build rustls which AIUI relies on ring. I'm sure @Alovchin91 can explain more since he's really the person with the context here, but I'm fairly sure the linked isue (https://github.com/briansmith/ring/issues/1167) is the best place to start.

Thanks again, and here's hoping we can make some forward progress together.

[briansmith]

Thanks. I understand there's no deadline for this particular task, but it would be useful to know the change freeze dates for the next one or two milestones so I can prioritize this work against other work.

[kinnison]

In rustup we don't operate our milestones on a strict timescale because we're a pretty small group of developers as you can appreciate that doesn't always lend itself to fixed release dates. In general I try to get 2 to 4 releases out per year, and I'm hoping to release 1.24 pretty soon (within a couple of weeks) so that'd put the 1.25 release (unless something unusual happens) no later than around early-to-mid June.

If there're things non-specialist devs can do to help with your work which would subsequently help with ours, please feel free to use the point that we're asking for this in any call-for-help type tweets etc. you might make. Ditto if there's stuff you think that utter-layment crypto-code-wise can do, then linking to issues related to this here might get you an audience from those watching this repo/issue. We could even do a joint call for help in TWIR if that might work.

[Alovchin91]

@briansmith Thank you for the response! Just FYI, at this point in time I can still build rustup with curl backend instead of reqwest, so Ring is not really blocking other work on making rustup arm64-compatible. Obviously though, it should indeed happen before we can proclaim win-arm64 version ready for release.

[Alovchin91]

Status update:

• When integrating the latest sys-info into effective-limits, I faced an issue with windows-gnu builds: https://github.com/rbtcollins/effective-limits.rs/pull/14 The fix for sys-info is in the pipeline: https://github.com/FillZpp/sys-info-rs/pull/92

• vendored-openssl doesn't seem to work; not sure how important it is, I can build without it; but it's probably a good low hanging fruit for openssl-sys crate since openssl itself does support Windows arm64

• ~cli::self_update::tests::default_toolchain_is_stable test is failing; investigating...~ Found, fixed (TargetTriple::from_host())

• cargo build --no-default-features --features reqwest-backend,reqwest-default-tls works, given the patch to effective-limits