microsoft
Foundry Offline Installation Bugs
Context: Working on Edge deployments - no connectivity, including access to internet for install on target PCs.
Previous v0.6 worked well for disconnected installation, manual installer, manual copy of models into cache, run cache list and models appeared 0- able to run models and connect services
Recent 0.07 updates has included an internet dependency after install (to pull back the model catalogue?) - this means it cannot progress unless there's a connection to fetch the model catalogue. Work around is to connect once to internet for initial model list, and then seems to restore functionality but still throws up alot of errors when trying to check the cache - but does not meet the requirement of disconnected capability.
Request - Can you consider offline installation and running support? Either a manual catalogue transfer or if internet is not detected, it can still retrieve whats in cache?
Why its important - for some edge use cases in high security domains there's no ability to connect hardware to internet even once, and particularly not for regular updates etc. So being able to offline install, run, update would be a massive enabler.
Or if something I'm missing - grateful for advice!
Hi @Liam-dev, Can you please share your machine specs (Windows version, if your machine has an NPU/GPU)? Also, can you please share the full version of FL that you are using (the output of foundry --version)?
Absolutely 😀
Windows 11 Pro, surface laptop 7, NPU - Qualcomm snapdragon, FL version v0.7.120.
Previous functionality worked for offline install and manual cache transfer on FL v0.6.87.
Keep up the awesome work on FL!
This is some of the errors when running offline - in case useful:
🟢 Service is Started on http://127.0.0.1:64711/, PID 32148!
[06:29:10 ERR] Exception fetching models from Azure Foundry catalog $System.Net.Http.HttpRequestException: No such host is known. (ai.azure.com:443)
---> System.Net.Sockets.SocketException (11001): No such host is known.
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError, CancellationToken) + 0x40
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16) + 0x58
at System.Net.Sockets.Socket.<<ConnectAsync>g__WaitForConnectWithCancellation|285_0>d.MoveNext() + 0x1b8
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnectionPool.<ConnectToTcpHostAsync>d__52.MoveNext() + 0x734
--- End of inner exception stack trace ---
at System.Net.Http.HttpConnectionPool.<ConnectToTcpHostAsync>d__52.MoveNext() + 0x7a8
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnectionPool.<ConnectAsync>d__51.MoveNext() + 0x3a4
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnectionPool.<CreateHttp11ConnectionAsync>d__80.MoveNext() + 0x234
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnectionPool.<InjectNewHttp11ConnectionAsync>d__79.MoveNext() + 0x358
--- End of stack trace from previous location ---
at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.<WaitWithCancellationAsync>d__1.MoveNext() + 0x100
--- End of stack trace from previous location ---
at System.Net.Http.HttpConnectionPool.<SendWithVersionDetectionAndRetryAsync>d__50.MoveNext() + 0x920
--- End of stack trace from previous location ---
at System.Net.Http.DiagnosticsHandler.<SendAsyncCore>d__10.MoveNext() + 0x5dc
--- End of stack trace from previous location ---
at System.Net.Http.RedirectHandler.<SendAsync>d__4.MoveNext() + 0x1e0
--- End of stack trace from previous location ---
at System.Net.Http.HttpClient.<<SendAsync>g__Core|83_0>d.MoveNext() + 0x3f8
--- End of stack trace from previous location ---
at Microsoft.Neutron.AzureFoundry.AzureFoundryApiService.<FetchModelPageFromApiAsync>d__13.MoveNext() + 0x1d4
--- End of stack trace from previous location ---
at Microsoft.Neutron.AzureFoundry.AzureFoundryApiService.<FetchAllLocalModelsAsyncImpl>d__15.MoveNext() + 0x100
--- End of stack trace from previous location ---
at Microsoft.Neutron.AzureFoundry.AzureFoundryApiService.<FetchAllLocalModelsAsync>d__14.MoveNext() + 0x7c
Model Phi-4-generic-cpu:1 was found in the local cache.
🕔 Loading model...
🟢 Model Phi-4-generic-cpu:1 loaded successfully
Hi @Liam-dev ,
I just saw your previous message. For some context, the update in 0.7 that relied on Internet connectivity is for downloading pluggable execution providers, and in the initial 0.7.117 release, we would attempt to download EPs for 10 minutes before timing out. If you update to use the latest version of Windows for your computer and use the latest version of Foundry Local, there should now be improvements that should allow the download portion to time out much quicker if no Internet connection is available.
An Internet connection should not be required to fetch models, assuming that the model you want is already downloaded locally. While we attempt to download the list of our latest models from the Azure catalog, we will always default to using a local cached copy if no Internet connection is available. However, a cache must be available, either through running foundry model list or foundry model load/download/run when an Internet connection is available, or through adding it manually, as you are doing now. So, I think that fetching the cached catalog should not cause any issues for you, given how you are setting up the cache manually.
Can you please update both Windows and Foundry Local and confirm if you still experience any issues with being offline? Please let me know what versions you are using. Thanks.