openhaystack
openhaystack copied to clipboard
seemoo-lab
Tracking on non-Macs
Problem The problem: it's limiting to not be able to track things on non-Mac computers.
Solution Reverse-engineer either the local Find My API or the Find My web app and allow for tracking on non-Macs
Alternatives
- Host a macOS server to track stuff. If not self hosted it allows whoever hosts it to access private information. I believe this was discussed earlier
- Upload the keys to Apple somehow and use the web Find My App to do this. May require temporarily disabling SIP, or possibly it's possible to use the Mail-app entitlement route to upload it.
Additional context I understand that this is an long term goal and that either way it's easier said then done. It may be worthwhile to try seeing how auth is done for the Find My web app and then MITM'ing the connection to see how it requests the location.
The web API (icloud.com) won't show your offline devices as far as I know. Because that would mean that Apple had access to the private advertisement keys to decrypt the location – which they claim they don't.
The web API (icloud.com) won't show your offline devices as far as I know. Because that would mean that Apple had access to the private advertisement keys to decrypt the location – which they claim they don't.
@schmittner Well, I guess we have to reverse engineer the local Find My app then. My guess is that it uses HTTP(S) like location services but it wouldn't be too farfetched for it to use a proprietary protocol.
I assume you are aware of https://github.com/picklepete/pyicloud ? This is a python framework to access Apple device location. The issue is that it does not yet support "objects" like AirTags. For this to be implemented, the iCloud API endpoint would need to be known, which does not seem to be the case.
I assumed I could find the API endpoint by using a man-in-the-middle-proxy (mitmproxy) with my iPhone, but of course Apple is using certificate pinning and the Find My app simply refused to work through the proxy. Other apps and websites worked.
So we'll probably have to wait until someone with a jailbroken iPhone and https://frida.re finds the time to dig this up.
Was not aware of that Python framework, thanks.
So we'll probably have to wait until someone with a jailbroken iPhone and https://frida.re finds the time to dig this up.
@hhf42 Is there any particular reason we have to use a jailbroken iPhone and not a Mac?
Oh, right. Of course you could also do this with a Mac and it might even be easier. Maybe as easy as running strings against the Find My app?
Apple has not yet implemented Find My for objects in the iCloud.com website, once that is available, that may also be an option.
Oh, right. Of course you could also do this with a Mac and it might even be easier. Maybe as easy as running strings against the Find My app?
Sorry, my RE knowledge is pretty lackluster, so I have no idea how to do that. Could you explain further?
Apple has not yet implemented Find My for objects in the iCloud.com website, once that is available, that may also be an option.
Apparently this is impossible for reasons explained above.
Hi, thank your for the recommendations. @hhf42 we actually tried using exactly this framework to access the location reports. So we can guarantee that at the moment it won't be easily possible.
@AwesomestCode you don't need to reverse find my. We already did that and all work you can find here is largely based on the reverse engineering we did. All the necessary http requests are implemented in OpenHaystack.
It has already been discussed if we can port this to windows. And the answer here is maybe. Windows has an iCloud app that can be downloaded and that is able to produce similar access tokens as the Mac. You can refer to our paper to see what access tokens are necessary to download location reports: https://arxiv.org/abs/2103.02282
We need:
-
Apple Anisette Data
-
A search party token, which is stored in the Keychain.
-
Is highly obfuscated code by Apple and we have not been able to reverse engineer it. But generally it produces tokens that are used to access iCloud services that are protected by 2FA. Those tokens are generated on Windows as well and AltStore already accesses them. https://github.com/rileytestut/AltStore
-
I don't know if this is actually available on Windows
So the actually best option to port this to the web or any other system is to have one macOS server running in the background that downloads location reports and forwards them. This is basically not a big security issue (for now), because all the reports are protected by end-to-end encryption.
And I have already implemented a very simple Mac server that actually does just this. I just don't want to host it somewhere publicly.
It's available on my private fork: https://github.com/Sn0wfreezeDev/openhaystack
Windows has an iCloud app that can be downloaded and that is able to produce similar access tokens as the Mac.
Was hoping for Linux support, not Windows. This way, it'll be easy to self-host finding/forwarding servers. While I have a Mac, I don't want to use it as a server that's on 24/7. I do sort of hope that it's possible to install OpenHaystack's finder app on all devices eventually.
Another, albeit silly-sounding option: could we possibly run the Windows iCloud app in a sandbox just for token generation? This sounds inefficient and silly, but it could be a viable path forward.
@Sn0wfreezeDev
we actually tried using exactly this framework to access the location reports. So we can guarantee that at the moment it won't be easily possible.
Thanks for the answer, can you elaborate further? The pyicloud framework works for devices, so should it not work for objects once the API endpoint is known? Or do you know it already and there are other issues?
@hhf42 the "old" Find My (to find iPhones, Macs) and the "new" Find My (for finding offline devices via Bluetooth incl AirTags) use two completely different protocols. The Find My app integrates them both.
The "old" one works via iCloud. The "new" one does not work via iCloud by design, so this will not come as a future update (read our paper to understand why this is the case, essentially this provides stronger privacy for the user).
Oh, I am reading the linked paper and I get it:
the owner device fetches location reports from Apple’s servers by sending a list of the most recent public advertisement keys of the lost device.
So the python framework would have to have access to those keys, they will not be in iCloud. So this would not work on a Linux or Windows device. Maybe on a Mac if the python framework could have access to keychain, but maybe only if jailbroken or with frida instrumentation.
This is disappointing :- ) I was very much looking forward to nice tracking beyond automated screenshots of Find My app.
@schmittner @Sn0wfreezeDev Just thought of something: if the keys are never uploaded to iCloud, how do other devices track them? For example, if I add an AirTag on my iPhone, I can still track it on my Mac... this seems to imply that the keys are uploaded to a server somewhere.
The keys that are needed for fetching location data are uploaded to iCloud (Drive).
The keys that are needed for fetching location data are uploaded to iCloud (Drive).
@Sn0wfreezeDev Wouldn't that mean that Apple does indeed have the keys and therefore it could theoretically be accessible using the Find My web app? Also, could this exploit theoretically be possible: upload the keys of an OpenHaystack tag to iCloud, have the other devices download the key from iCloud and then it'll appear on the Find My app on other devices.
I’m aware it’s not easy. I’m not suggesting that anything is easy; in fact, I do realise that implementing this entire thing is difficult. However, I do wonder if it’s possible in theory.
On Mon, Jun 7, 2021 at 9:37 AM, Alexander Heinrich @.***> wrote:
No it's not that easy
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/openhaystack/issues/63#issuecomment-855936216, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMVK4MW2PHBKWBVMPIUL733TRTDQPANCNFSM45VELK4Q .
Hello! I've made an example how to create a Grafana dashboard so you can view current and past locations on any browser-enabled device. It allows you to use any Linux server to host Grafana, InfluxDB, and the reports gathering daemon itself by running OpenHaystack simple-server on any macOS (or Hackintosh) machine to download requests from Apple servers. It also includes the AirTagCrypto.py library that you can use in your projects to decrypt Airtag reports.
@hatomist Love it! To make sure I'm following, you're able to programmatically pull airtag device location? I've been using pyicloud but it doesn't support airtags.
Not quite, I just use OpenHaystack simple-server to pull the data, it can be hosted on any macOS system, will it be Hackintosh or a genuine device. But then it is fully processed in Python on any server you want, will it be a Raspberry pie, VPS or a home server etc.
Actually, you can use somebody else’s OpenHaystack server to download your tags’ reports, as my daemon doesn’t require any private keys to be transmitted to the server. I don’t know anything about rate limits Apple have, but it would be cool if a dozen of people who don’t have a macOS machine use a public proxy server for their air tags. Or if someone would implement some kind of centralised proxy server, which will proxy requests to other user’s servers and take care of load balancing and rate limiting. But it will only make sense if we won’t have any other way to fetch reports directly (as it is now).
On 28 Jun 2021, at 22:06, zefoo @.***> wrote:
@hatomist Love it! To make sure I'm following, you're able to programmatically pull airtag device location? I've been using pyicloud but it doesn't support airtags.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
Now that I think about it, I wonder if MacStadium would be willing to loan us a server as an open source project that we can use for Find My querying.
On Mon, Jun 28, 2021 at 18:05, Bezmenov Denys @.***> wrote:
Not quite, I just use OpenHaystack simple-server to pull the data, it can be hosted on any macOS system, will it be Hackintosh or a genuine device. But then it is fully processed in Python on any server you want, will it be a Raspberry pie, VPS or a home server etc.
Actually, you can use somebody else’s OpenHaystack server to download your tags’ reports, as my daemon doesn’t require any private keys to be transmitted to the server. I don’t know anything about rate limits Apple have, but it would be cool if a dozen of people who don’t have a macOS machine use a public proxy server for their air tags. Or if someone would implement some kind of centralised proxy server, which will proxy requests to other user’s servers and take care of load balancing and rate limiting. But it will only make sense if we won’t have any other way to fetch reports directly (as it is now).
On 28 Jun 2021, at 22:06, zefoo @.***> wrote:
@hatomist Love it! To make sure I'm following, you're able to programmatically pull airtag device location? I've been using pyicloud but it doesn't support airtags.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/openhaystack/issues/63#issuecomment-870076365, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMVK4MUMU5VLKDNJEBVFQC3TVDW4JANCNFSM45VELK4Q .
To be fair it shouldn’t be a genuine MacOS server either, I’m running a Hackintosh Big Sur on a 4gb ram 100gb hdd 4 core virtual machine. It should be possible to run it in QEMU on any Linux VPS as well, as far as it supports (nested) virtualisation.
On 29 Jun 2021, at 01:08, AwesomestCode @.***> wrote:
Now that I think about it, I wonder if MacStadium would be willing to loan us a server as an open source project that we can use for Find My querying.
On Mon, Jun 28, 2021 at 18:05, Bezmenov Denys @.***> wrote:
Not quite, I just use OpenHaystack simple-server to pull the data, it can be hosted on any macOS system, will it be Hackintosh or a genuine device. But then it is fully processed in Python on any server you want, will it be a Raspberry pie, VPS or a home server etc.
Actually, you can use somebody else’s OpenHaystack server to download your tags’ reports, as my daemon doesn’t require any private keys to be transmitted to the server. I don’t know anything about rate limits Apple have, but it would be cool if a dozen of people who don’t have a macOS machine use a public proxy server for their air tags. Or if someone would implement some kind of centralised proxy server, which will proxy requests to other user’s servers and take care of load balancing and rate limiting. But it will only make sense if we won’t have any other way to fetch reports directly (as it is now).
On 28 Jun 2021, at 22:06, zefoo @.***> wrote:
@hatomist Love it! To make sure I'm following, you're able to programmatically pull airtag device location? I've been using pyicloud but it doesn't support airtags.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/seemoo-lab/openhaystack/issues/63#issuecomment-870076365, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMVK4MUMU5VLKDNJEBVFQC3TVDW4JANCNFSM45VELK4Q .
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
MacStadium is offering free hosting for open source software. So we could give it a try. I should merge the simple server into the main project at some point. For now feel free to refer to my fork for this.
EDIT: I just sent in the application form to MacStadium. Maybe this solves the issue for us 😊
Glad to hear that, looking forward to their response 🙃
On 29 Jun 2021, at 10:13, Alexander Heinrich @.***> wrote:
MacStadium is offering free hosting for open source software. So we could give it a try. I should merge the simple server into the main project at some point. For now feel free to refer to my fork for this.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.
Hello @Sn0wfreezeDev, can you please also add an ability to specify a time period in the request? Right now it's hard-coded in simple-server like so:
let duration: Double = (24 * 60 * 60) * 7
It would be better for an exporter like mine if I could get only the last 30 minutes of reports every 15 minutes, rather than the whole week, in terms of performance and bandwidth consumption.
And it would be nice if a user could specify port and host server will run in the command line, without re-compilation of the server. Right now I'm using lines like these:
app.http.server.configuration.hostname = "0.0.0.0"
app.http.server.configuration.port = 8090
I can't see an issues tab in your fork so I thought it would be appropriate to mention you here.
Oh, and I am also interested if there were any updates on MacStadium :thinking:
We are checking if we can host a server. If this does not impose license issues, we will be able to do this 😊
And it would be nice if a user could specify port and host server will run in the command line, without re-compilation of the server. Right now I'm using lines like these:
app.http.server.configuration.hostname = "0.0.0.0"app.http.server.configuration.port = 8090
@hatomist, you can already specify hostname and port on the command line:
./OpenHaystackServer serve --hostname 0.0.0.0 --port 8090
These are fed directly to the Vapor server that the project is using.
@hhf42 @Sn0wfreezeDev If I understood the paper correctly, one would need to know how to generate a search-party-token and have a real AppleID as well as some real device metadata (Anisette Data) to download reports.
The device metadata is easy, just take it from any Mac. AppleID login has also been done before (pyicloud). So we would just need to reverse engineer where the search-party-token comes from and then make the request as described in table 4 of the paper on a non-MacOS system...?
I may be overlooking some details, but I'd definitely be interested in a collaborative effort to reverse engineer where the request headers for downloading reports come from... :) Even if that means having to reverse everything statically, since the iCloud client software doesn't seem to like being passed through a proxy.
The only problem I could think of would be if the search-party-token was generated using device-specific keys from the Secure Enclave, but I haven't looked into the Find-My app and protocol yet, so I don't know whether that's the case.
It can’t be from the SEP though because it also works on pre-T2 macs, right?On Mon, Nov 8, 2021 at 08:39, lennyerik @.***> wrote:
@hhf42 @Sn0wfreezeDev If I understood the paper correctly, one would need to know how to generate a search-party-token and have a real AppleID as well as some real device metadata (Anisette Data) to download reports.
The device metadata is easy, just take it from any Mac. AppleID login has also been done before (pyicloud). So we would just need to reverse engineer where the search-party-token comes from and then make the request as described in table 4 of the paper on a non-MacOS system...?
I may be overlooking some details, but I'd definitely be interested in a collaborative effort to reverse engineer where the request headers for downloading reports come from... :) Even if that means having to reverse everything statically, since the iCloud client software doesn't seem to like being passed through a proxy.
The only problem I could think of would be if the search-party-token was generated using device-specific keys from the Secure Enclave, but I haven't looked into the Find-My app and protocol yet, so I don't know whether that's the case.
—You are receiving this because you were mentioned.Reply to this email directly, view it on GitHub, or unsubscribe.
So yes as @AwesomestCode and @lennyerik said we need the search-party-token and the anisette data. Unfortunately, the iCloud tokens from pyicloud are not enough to get location reports from Apple. Those are extra secured with the anisette data (special iCloud tokens). We are currently working on a lot of things in this direction. At first, one of my colleagues is looking in the Anisette data generation. Then we are now evaluating a Mac mini Server that accesses the location reports for an OpenHaystack client.
For this we are also working on a Flutter based implementation of the OpenHaystack tracker app. I have to say that all these projects are student projects that are set for a 6 months period.
After one to two days of researching I found out that:
- The anisette data does contain regularly changing tokens that no one has managed to generate without the use of Apple software yet: namely
X-Apple-I-MD(the one-time password, changes every 30 seconds) andX-Apple-I-MD-M(the machineID, also changes regularly) - These tokens are cryptographically generated by the AuthKit framework, which no one has fully understood or dared to completely reverse yet, due to its complexity and legal / copyright concerns
- It seems to be fairly easy to extract these tokens from the Windows versions of iCloud / iTunes (AltServer for Windows does it, here is my own minimal example)
- AltServer's anisette extraction method even has the ability to get similar anisette data as on a physical Mac (see this file,
SPOOF_MAC#define directive), should we need that in order to download reports - The search-party-token seems to be harder to get: I'm fairly certain the Windows apps don't have one and on MacOS it seems to be generated and added to the keychain by
searchpartyuseragent/searchpartyditself, since the MacOS console log messages suggest that the Apple Account preference pane invokessearchpartydto set the token when logging into an Apple Account in the Preferences app
So, as far as Windows support goes: definitely feasible if we figure out how search-party-tokens are generated. As for Linux support: Maybe just run iCloud / iTunes in Wine? Or reverse the AOSKit.dll (which is where the anisette data comes from on Windows)? Even I don't dare touch AuthKit on MacOS / iOS... :)
But in any case, it would be great if you were able to share some technical details of what you mean when you say you're working on anisette data generation. Would that mostly be the search-party-token or does it literally mean reversing AuthKit? I totally understand if you are unwilling or unable to share these sorts of details, since it sounds like a lot of research is internal student projects for now, but it'd be interesting to know what is and isn't worth looking at myself.
Yes all that you found out is 100% correct. Actually AltServer is the option we adapted for the Apple Mail PlugIn. Since it's open source we uses the AltServer code as a starting point. On Windows we did not dare to touch that for now. For cross-platform compatibility we are planning to go for a way that uses a macOS based server to fetch the keys.
Legal issues
I'm not sure if forwarding location reports through a macOS server will lead to some legal problems. Basically it just accesses an open API endpoint and forwards the results, but that's something we have to investigate before we make this publicly available.
For reversing AuthKit. I'm not sure if that's actually a legal issue. We reverse Apple software all the time and report security issues to Apple. As far as the reversing is done for a security purpose at first place, which it has to be since it generate sensitive tokens, we are allowed to do so. Even with Apple's licensing terms.
So AuthKit and especially akd is what we are now looking into. But as you mentioned with the linked tweet it's highly obfuscated and very difficult / close to impossible to understand. That's why we start on checking input and output to see if we can trace what the actual input is that we need for this computation.
So far, we have not looked into the search party token. That one is still a complete black box to me. So I guess it gets generated somewhere or is fetched from Apple.
hi , any update on being able to use openhaystack on a non apple device ?
(pc/android/linux)?
thanks
Would building an opt-in p2p network be better for avoiding legal issues? Since anyone could participate and there's no central endpoint to take down, we get free load balancing/scalability and (to my knowledge) less legal friction as there's no "official central point" to go after.
@jessestowe what is this OpenHaystackServer you are talking about? How to launch it / where to find it? Is it a configuration in this openhaystack repo, or some clone?
It's currently only available in my private fork. https://github.com/seemoo-lab/openhaystack A p2p network would be nice, but this may expose security risks to the computers hosting it. It would probably be good to get a lawyer who knows if proxying and API is actually a legal issue or not.
Would it be an option to find such an API on the tor network? That should give the hoster enough anonymity...
I'm trying to access the reports using curl and only internal macOS tools so I can start research on the anisette data and the search-party-token. I've followed the paper and particularly @lennyerik 's hints, but I'm getting a 401 response from the server. I've added my shell script to this repo: https://github.com/biemster/FindMy
Can someone here give an indication how long the Authentication: header should be? The one my script generates is particulary big..
I've fixed my bug, I can now fetch reports using the shell script in the repo from my previous post. I had indeed a mistake in the generation of the Authorization header.
The paper is not clear on how the ids should be formatted in the request json, I suspect it's base64 but I can't be sure. And is it the hashed key or the advertised key? @Sn0wfreezeDev can you confirm / comment? EDIT: figured it out, the id in the request is base64 of the hashed advertisement.
The paper is not clear on how the ids should be formatted in the request json, I suspect it's base64 but I can't be sure. And is it the hashed key or the advertised key? @Sn0wfreezeDev can you confirm / comment? EDIT: figured it out, the id in the request is base64 of the hashed advertisement.
In your repo the method you use to gather the party token doesnt seem to work (https://github.com/biemster/FindMy/blob/cb2c31d5c1f484f4ac462801675767a956dace5c/request_reports.py#L40) im not able to access the iCloud Keychain using the internal security tool, any clue?, after some research all i found is that its not possible to access the iCloud keychain that way. Nice work there.
@xliee what version of macOS are you on? On my Catalina it works fine. It also depends if you're on a logged in Terminal or SSH login. (of you want to discuss further, please open an issue in the repo there as not to clutter this one)
We could get Anissette data on Linux using provision with the Apple Music ipa
https://github.com/Dadoum/Provisio
Hope this helps
https://github.com/Dadoum/Provisio
Looks like your link got cut-off: https://github.com/Dadoum/Provision
You would still need a macos device to deploy the openhaystack airtag though?
I don’t know anything about rate limits Apple have, but it would be cool if a dozen of people who don’t have a macOS machine use a public proxy server for their air tags. Or if someone would implement some kind of centralised proxy server, which will proxy requests to other user’s servers and take care of load balancing and rate limiting. But it will only make sense if we won’t have any other way to fetch reports directly (as it is now).
@hatomist
According to Mayberry et al., “Note that there is no throttling or limiting on the number of keys that a client can request from Apple’s servers; it is possible to request hundreds or thousands of them in a few seconds.”
You would still need a macos device to deploy the openhaystack airtag though?
Only the advertisement key is needed in the request, no deployment necessary. So if you can make those requests on a non-Mac, no macos device is needed anymore.
The Provision project looks really cool! We will definitely have a look into that.
Has anyone gotten Dadoum/Provision to work? When trying their 'X-Apple-I-MD' and 'X-Apple-I-MD-M' I always get a 401 Unauthorized on the https://gateway.icloud.com/acsnservice/fetch endpoint.
This is interesting guys. I'm following th openhaystack project for a while now. but i don't own a mac. I guess my only way to integrate is to use proxmox right? I have a low end mini-pc i could use for this but i am not sure performance wise it willl work.
This is interesting guys. I’m following th openhaystack project for a while now. but i don’t own a mac. I guess my only way to integrate is to use proxmox right? I have a low end mini-pc i could use for this but i am not sure performance wise it willl work.
Yea that’s why non-mac is something a lot of people want, you can’t exactly run macos on a raspberry pi 😅
For those landing here. I solved my issue by installing monterey on a proxmox installation of mine and set up this. Then I adapted the mobile app to run on windows and linux which is my main OSes.