[✨] Add bundling options for SPA and offline PWA

[lbensaad]

Is your feature request related to a problem?

I am creating an SPA where all 15 pages are SSG and private, now qwik generates nearly 300 js files. I need the app to work offline, so all js has to be pre-fetched. It is not easy now to download all those 300 js files. also i dont know how to get the list of those files in the service worker.

Also for an SPA and private pages and in offline PWA, mostly the user will need all functionalities beforehand, so no need to slice the js code in small chunks. it is better to deliver the whole of it in one shot, while the user is filling the sign up or login page for example.

Describe the solution you'd like

I want to make it possible to configure the optimizer to put all js in one bundle app.js for example, or configure it to bundle one js file per route. This will be easy for prefetching. I also need an option to bundle all code that I write in one js file and all 3rd party libraries in another file, to make cache configuration easier. because my code changes more frequently than other libs.

Describe alternatives you've considered

i tries to configure the service worker to download the generated files but i failed to do it. I do not know how to do it and how to get the list of all generated bundles.

Additional context

No response

[mhevery]

Yes, this is something that should be covered by our upcoming work on insights prefetching...

[lbensaad]

I think some configurations such as these proposed here shouldn't depend only on insights, there should be an option for the developer to decide and enable/disable it.

[mhevery]

I think some configurations such as these proposed here shouldn't depend only on insights, there should be an option for the developer to decide and enable/disable it.

yes. Insights needs to tell bundler what these are, so the work on insight, will create API which insights uses, and which can be used by the developer directly to tell which chunks to load when.

[maiieul]

@lbensaad, I think for the moment you could use workbox in service-worker.ts with something like:

import {
  cleanupOutdatedCaches,
  createHandlerBoundToURL,
  precacheAndRoute,
} from "workbox-precaching";
import { NavigationRoute, registerRoute } from "workbox-routing";

const revision = import.meta.env.BUILD_NUMBER;

precacheAndRoute([
  { url: "/", revision },
  { url: "/page1/", revision },
  { url: "/page2/", revision },
  { url: "/page3/", revision },
]);
cleanupOutdatedCaches();
registerRoute(new NavigationRoute(createHandlerBoundToURL("/")));

And then to keep your app working offline you could use cacheControl in your layout file:

cacheControl({
   private: true, // if your pages are static but different for every user; this tells CDNs to not cache the response
   maxAge: 60 * 60 * 24, // you can set any value you're comfortable with if it's static pages
   staleWhileRevalidate: 60 * 60 *  24 * 365
})

Or you could use workbox to re-cache your routes with staleWhileRevalidate strategy, but using cacheControl is probably simpler.

[lbensaad]

Thank you @maiieul for this solution but I have a question please. Will this code eagerly download all needed js files or it will only cache the files that have been at least opened once. To clarify more, consider this scenario: let's say I have /page1/ that references file1.js and file2.js and let's say that the user interacted with the page and the code of the action is in file1.js, at this time if qwik has prefetched file1.js then it will be used otherwise it will be downloaded and cached. let's say that the user now is offline and opens the catched page and interacts with the page using another button for example that has its code in file2.js. In the case if qwik did not prefetch the file when the user was connected then the action will not work. So will the code you proposed prefetch every needed file even before the user interact with it?

[maiieul]

I think that the service worker code will prefetch all the files for each page in precacheAndRoute and cache them in the Cache API. Then when the user navigates to the page, cacheControl headers will tell the browser how to cache the resource in the HTTP cache (which is different from the Cache API). I'm not an expert on those two caching mechanisms (just been exploring them for the past week), but I managed to make my app work offline with static pages with those two. So, that might require more testing on your end :see_no_evil:

[lbensaad]

Thank you @maiieul, I have succeeded to make workbox download the whole build folder, but the problem is that it makes 332 requests because qwik generates so many js files. That is why I asked @mhevery to make it configurable. 332 is too much for a site with only 15 routes. But I had to use the plugin vite-plugin-pwa, and used precacheAndRoute(self.__WB_MANIFEST) and the plugin will traverse the dist folder and add all files to __WB_MANIFEST. I am having a problem where this plugin is run before Qwik SSG and therefor html files and routes are not added to __WB_MANIFEST, so I had to add them manually to precacheAndRoute as you mentioned above . Do you know how to make the plugin run after Qwik SSG static adapter?

[gioboa]

Hi did you try entryStrategy single to produce less files?

[lbensaad]

Many thanks @gioboa, That is exactly what I needed. Now I have 82 requests instead of 332, and it takes 9 seconds to download all files instead of 35 s.

[lbensaad]

I think Qwik needs Workbox integration. Maybe @eric-burel can help with that.

[Aslemammad]

Hey folks, here's an example I've been working on, it has complete PWA/offline support with workbox integration. Most of the changes are in the service-worker.ts and vite.config.ts files.

https://github.com/Aslemammad/qwik-pwa-example

Solution: 1- cache all q-.js files using the qwik service worker (a hack using self.dispatchEvent(an event containing all the q-*.js files) 2- cache assets (public/ + all the imported ones) with workbox 3- cache routes with navigation routes in workbox

It's just a prototype, and if the Qwik team is open to it, we can work on an official integration for PWA apps in Qwik (adapters? vite plugins? adding it to qwik/service-worker?

[thejackshelton]

Hey folks, here's an example I've been working on, it has complete PWA/offline support with workbox integration. Most of the changes are in the service-worker.ts and vite.config.ts files.

https://github.com/Aslemammad/qwik-pwa-example

Solution: 1- cache all q-.js files using the qwik service worker (a hack using self.dispatchEvent(an event containing all the q-*.js files) 2- cache assets (public/ + all the imported ones) with workbox 3- cache routes with navigation routes in workbox

It's just a prototype, and if the Qwik team is open to it, we can work on an official integration for PWA apps in Qwik (adapters? vite plugins? adding it to qwik/service-worker?

Would love this! I think this is something that Qwik really needs some exploring in.

[Aslemammad]

Would love this! I think this is something that Qwik really needs some exploring in.

Thank you, I'm also excited, as I mentioned to Misko, that the next steps (if this goes right) can be the native export to IOS/Android, using capacitor for instance.

https://twitter.com/asleMammadam/status/1724271418873933970

[maiieul]

Hey @Aslemammad, I'm excited to try your prototype and work going forward!

Question:

Have you tried Qwik Insights without your solution? If yes, what's the difference between your solution and Qwik Insights in terms of offline capabilities? (I haven't tried Insights yet personally.)

Remark:

I tried using Capacitor with Qwik about 4 months ago, and it either doesn't work with SSR or is a nightmare to use with things like qwik-auth. There are many benefits to using qwik with SSR on the web, so I believe it's far easier to use PWAs and port them to Android (with TWA) and to ios (with a webview), using PWABuilder for instance.

I don't think Capacitor will ever support SSR, but perhaps Tauri or Socket-Supply will. The 𝕏 stack is currently deployed on the web and on android. I discuss these questions further here: https://github.com/x-ploration-of-mars/x-stack#cross-platform-solutions.

That being said, I will definitely explore your service-worker solution for the 𝕏 stack as I think it should work with PWAs :+1:. Going to try and deploy on ios as soon as I can with PWABuilder. You're more than welcome to join the discord if you'd like to give it a try (https://discord.com/invite/W4e8ReQWv2).

[Aslemammad]

@maiieul Thank you so much. I'm excited to talk to you on this topic.

Have you tried Qwik Insights without your solution? If yes, what's the difference between your solution and Qwik Insights in terms of offline capabilities? (I haven't tried Insights yet personally.)

I don't know about offline capabilities in Qwik Insights, do you have any links?

And yes, my first plan is to try PWABuilder first and It's fairly easy to get it working. After that, I'll try Capacitor someway and if it does not work, we'll stick to PWABuilder.

I just joined the Discord server and we can discuss things.

[lbensaad]

Thanks a lot @Aslemammad, it is working and clean. I had to add this setDefaultHandler(new NetworkFirst()); in the service worker so that calls to /api/ are also cached for offline.

[Aslemammad]

Happy to hear that, @lbensaad. If possible send a PR to the example, so I also add it to the integration I'm working on.

[lbensaad]

@Aslemammad this is what I added in the service worker:

import { NetworkFirst } from 'workbox-strategies';
import {  setDefaultHandler } from "workbox-routing";

setDefaultHandler(new NetworkFirst());

The NetworkFirst strategy is explained here

[Aslemammad]

Thank you so much @lbensaad.

[lbensaad]

@Aslemammad I have tried this code and also worked

registerRoute(
    ({ url }) => url.pathname.startsWith('/api/'),
    new NetworkFirst()
);

[lbensaad]

Because we are prefething every thing, I think it is better also to change the bundling strategy of qwikVite from default 'smart' to 'single', that will produce one js file per route. It can be changed in vite.config.ts

qwikVite({ entryStrategy: { type: "single" } }),

[lbensaad]

@Aslemammad for each route, qwik generates a q-data.json file next to index.html. Those files need to be pre-fetched also. qwik can not route to those routes offline if you did not visit that page when you were online. I had to change your assets array in the service worker like this:

const assets = [...publicDirAssets, ...emittedAssets, ...routes.map((r) => `${r.pathname}q-data.json`)];

[Aslemammad]

I didn't realise that, thank you for the info. For the single js file per route, I prefer to have the smart strategy since I think the priority is to keep Qwik smart as it is. But that's great since anyone can change this from their config.

And yes, I'll make sure to add this line two (I may send the changes as is and let you know so you can send a PR so you can have your name part of the competitors). Here's my discord if you're willing to talk about that (@aslemammad).

[kauderk]

🙌 Hi Everyone I tried to load qwik (without qwik-city) into an existing page because I want to create a chrome extension https://discord.com/channels/842438759945601056/842438761287254019/1116118813000351754

@mhevery recommended to look at https://github.com/BuilderIO/qwik/pull/4719 but I lack the knowledge to execute the idea

Can anyone point out how to create chrome extensions using qwik? I'm stuck here https://github.com/kauderk/qwik-chrome-extension-template

[gioboa]

I can help you. You can contact me on discord. 👍

[kauderk]

Now I have working templates for creating chrome extensions with qwik - thank you @gioboa

• General Purpose - using the @crxjs/vite-plugin
• Options Page - simple configuration
  • I didn't know csr: true was enough to build an spa >.<

[wmertens]

@kauderk you can also use SSG to generate an SPA instead of doing CSR. That way the skeleton is already present and JS is minimal.

See https://github.com/krausest/js-framework-benchmark/pull/1447/files for an example that builds a single page.

[kepi]

@kauderk you can also use SSG to generate an SPA instead of doing CSR. That way the skeleton is already present and JS is minimal.

See https://github.com/krausest/js-framework-benchmark/pull/1447/files for an example that builds a single page.

I'm not sure if we are not highjacking this thread with whole Extension bussiness, but this and @kauderk repository are only resources about it. Using SSG for Chrome Extension was my first pick, but I can't find more documentation about it. Problem is, that with default settings, JavaScript is inlined, which isn't possible in Chrome Extensions:

Inline JavaScript will not be executed. This restriction bans both inline

Is there some option which will enable this? I'm trying to wrap my head around whole Vite/Qwik build thing and find myself in too many dead ends. Btw. I would like to use QwikCity too in the extension.

[wmertens]

@kepi the only inline JS is the Qwik loader and you can probably fish that out of the package somewhere to import.

It should also be possible to make a loader mode where it is in an external file.

After that it's just a SPA, right?

[maiieul]

@gioboa I think this can be closed too. Same as https://github.com/QwikDev/qwik/issues/5438.