angular icon indicating copy to clipboard operation
angular copied to clipboard

feat(DomRenderer): allow partial DOM hydration from pre-rendered content

Open jeffbcross opened this issue 7 years ago • 169 comments

I'm recapping a discussion I just had with @alxhub and @tbosch. This is mostly Tobias' design.

I'm submitting a ... (check one with "x")

[x] feature request

Current behavior

Typically when a page is pre-rendered, such as with Universal, the Angular application bootstraps in the browser, then blows away the pre-rendered content, and replaces it with newly created nodes. The DOM tree that is pre-rendered is often very similar, or identical to the DOM tree created on the front end, but there are issues that make it difficult for Angular to take over the existing DOM tree (otherwise referred to as DOM hydration).

The actual handoff of destroying the old DOM and showing the new DOM is pretty fast and seamless, so it's not necessarily a major UX issue in and of itself. Where it becomes problematic is in cases like ads that load in iframes (which is pretty much all display ads). If these ad iframes are pre-rendered -- which is a business requirement for many publishers -- and the iframe gets moved in the DOM, the iframe will refresh. This causes some ad networks to suspect abuse, as if publishers are trying to sneak more ad views.

Why Not Use the Already-Rendered DOM?

One issue is that with asynchronicity+conditional DOM (i.e. *ngIf="data | async"), the tree in the client may be rendered before the condition is truthy, whereas the pre-rendered version may have the full tree with async data resolved.

Another challenge is that text nodes are not able to be selected by CSS selectors, which would mean the renderer would have to rely on child ordering in order to associate pre-rendered nodes with client-rendered nodes (which is not always correct). Similar challenge goes for elements in an *ngFor, the order must be assumed to be identical.

The renderer would also be responsible for cleaning up pre-rendered orphan nodes. i.e. if 30 items in an *ngFor were pre-rendered, but only 20 were rendered in the client, the additional 10 nodes would need to be removed to prevent unexpected behavior.

Proposal: Optional, explicit, partial DOM Hydration

Allow setting a user-specified attribute on elements to associate the pre-rendered version with client-rendered version. If the renderer comes to a node that it can't associate with an existing node, it will blow away the node and re-create it. The developer would be responsible for setting these ids on the elements they care about. Example:

import { HydrateDomConfig, NgModule } from '@angular/core';

@NgModule({
  providers: [
    { 
      provide: HydrateDomConfig, 
      useValue: {
        hydrate: true, // default false for backwards compat
        attribute: 'pid', // default 'id'
      } 
    }
  ]
})

Component:

@Component({
  template: `
    <div pid="header">
      <header-ad pid="header-ad"></header-ad>
      <div>
        <!-- this will get blown away and re-created since it lacks attribute -->
      </div>
    </div>
  `
})

This design allows the DomRenderer to traverse the DOM tree and match elements for each branch starting at the root until it can't go any deeper, at which point it would blow away the descendants and re-create them.

Text nodes would all be destroyed and re-created with this design, as well as any node that doesn't have the set attribute, pid.

I don't expect that the rendering would be user-perceivable, other than if there are discrepancies between pre-rendered and client-rendered DOM, but that's a concern even without this feature.

CC @gdi2290 @pxwise

@tbosch & @alxhub please feel free to add anything I missed (or misrepresented).

jeffbcross avatar Dec 14 '16 01:12 jeffbcross

Ad wipeout on client bootstrap is a real world issue for us using universal and this hydration proposal should solve it. Our current workaround gets us partway there, moving server DOM into the same position in client rendered DOM but does not outsmart ad verification services that watch for DOM mutations, hiding the ad upon client bootstrap.

Big thumbs up for opt-in hydration.

pxwise avatar Dec 14 '16 02:12 pxwise

LGTM, Keep in mind this is a huge problem with all js frameworks so for Angular to have a solution is a great win for when comparing different SSR solutions. The real solution is having Ads actually work together with js frameworks but that will never happen, other than AMP. I can see how it would be implemented by rewriting selecting root element and providing a different render path.

For the |async you definitely have that problem with Promises due to the microtask while Observables do return synchronously. On the client, we can assume there will be batched data sent from the server to the client which gives us all of the results immediately. For Universal, we need to have the data available to the client synchronously anyways to reuse the server Http responses

var res = {data: 'ok'};
var api = Rx.Observable.of(res);
var api2 = Promise.resolve(res);

var vm = {};
var vm2 = {};
api.subscribe(function(data) { vm = data; });
api2.then(function(data) { vm2 = data; });
console.log(vm); // {data: 'ok'}
console.log(vm2); // {}

PatrickJS avatar Dec 14 '16 19:12 PatrickJS

An alternative design would be to provide an alternate renderer that would extend DomRenderer, rather than modifying DomRenderer to behave differently depending on the hydrate value.

@NgModule({
  providers: [{
    provide: Renderer, useClass: DomHydrationRenderer
  }]
})

jeffbcross avatar Dec 14 '16 22:12 jeffbcross

+1 for DomHydrationRenderer

PatrickJS avatar Dec 15 '16 01:12 PatrickJS

Two things:

  • angular.io v42 might also benefit from this. I had to put in a bunch of code that prevented some flickering caused by the content being wiped out by the renderer code when initializing components
  • I wonder if there is something from the AMP solution that we could leverage to in universal, it would be good to look into that before we go too creative with our own solution/workaround.

IgorMinar avatar Dec 16 '16 20:12 IgorMinar

+1 either solution will be beneficial and vital for us to move forward.

playground avatar Dec 19 '16 14:12 playground

We can still make the work generically, when using auto generated ids based on the place in the element hierarchy (and not based on the creation order).

I think we should think this through as well before we decide.

tbosch avatar Dec 28 '16 19:12 tbosch

is HydrateDomConfig part of @angular/core? I get error when I try to compile the application.

error TS2305: Module '"/home/fahad/Workspace/siteCuriouss/node_modules/@angular/core/index"' has no exported member 'HydrateDomConfig'.

FahadMullaji avatar Jan 10 '17 05:01 FahadMullaji

@FahadMullaji it's only a proposal.

DzmitryShylovich avatar Jan 10 '17 07:01 DzmitryShylovich

This proposal is super exciting!

Has any thought been given to pre-rendered lazy routes? We can achieve pre rendered lazy routes via https://github.com/angular/universal/blob/master/modules/ng-module-map-ngfactory-loader/README.md

Is it possible for the hydration to occur after lazy route is fetched and then rendering begins?

Just curious on thoughts here.

Universal has been great and straightforward to use so far; thanks to everyone involved!

josephliccini avatar Aug 29 '17 01:08 josephliccini

What couples the ComponentRef obj to the Dom string (cmpref.changeDetectorRef.rootNodes[0] & cmpref.hostView.rootNodes[0] ect)? Is there an explicit reference(i.e. dom string selector) or does this binding live in memory? Is there a possibility of calling createComponent with an existing dom node as an argument?

mcferren avatar Sep 24 '17 20:09 mcferren

Any updates on this - i have big issue with this as everybody here :(

vytautas-pranskunas- avatar Aug 18 '18 16:08 vytautas-pranskunas-

I quit working in Angular JS a year ago. I just got tired of all the problems and false promises.

On Sat, Aug 18, 2018 at 11:37 AM Vytautas Pranskunas < [email protected]> wrote:

Any updates on this - i have big issue with this as everybody here :(

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/angular/angular/issues/13446#issuecomment-414070398, or mute the thread https://github.com/notifications/unsubscribe-auth/ABU4JqXoyDHyfxps21n2DGm0jYang4h6ks5uSEK0gaJpZM4LMcBh .

-- Regards Fahad Mullaji

FahadMullaji avatar Aug 20 '18 19:08 FahadMullaji

Is there dom hydration problem solved in ather spa libraries?

On Mon, Aug 20, 2018, 9:26 PM Fahad Mullaji [email protected] wrote:

I quit working in Angular JS a year ago. I just got tired of all the problems and false promises.

On Sat, Aug 18, 2018 at 11:37 AM Vytautas Pranskunas < [email protected]> wrote:

Any updates on this - i have big issue with this as everybody here :(

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/angular/angular/issues/13446#issuecomment-414070398 , or mute the thread < https://github.com/notifications/unsubscribe-auth/ABU4JqXoyDHyfxps21n2DGm0jYang4h6ks5uSEK0gaJpZM4LMcBh

.

-- Regards Fahad Mullaji

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/angular/angular/issues/13446#issuecomment-414433412, or mute the thread https://github.com/notifications/unsubscribe-auth/ADvMl2WKIU4PSv84mXBXAe-33WD-jbmAks5uSw1ggaJpZM4LMcBh .

vytautas-pranskunas- avatar Aug 20 '18 20:08 vytautas-pranskunas-

@vytautas-pranskunas- Every framework, other than angular, has solved this problem. With that said it might not be a problem once the new ivy renderer is released.

PatrickJS avatar Aug 20 '18 23:08 PatrickJS

gdi2290 What makes you think that ivy renderer will solve this sunce i have not found any mentions of solving this or introducing virtual doms in ivy?

I wonder why Angular team is silent about this forcing more people to be dissapointed...

On Tue, Aug 21, 2018, 1:40 AM PatrickJS [email protected] wrote:

@vytautas-pranskunas- https://github.com/vytautas-pranskunas- Every framework, other than angular, has solved this problem. With that said the problem might not be a one with the new ivy renderer

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/angular/angular/issues/13446#issuecomment-414498736, or mute the thread https://github.com/notifications/unsubscribe-auth/ADvMl-LwoyJmtroffUkIvEGkXDtsMjm3ks5uS0jjgaJpZM4LMcBh .

vytautas-pranskunas- avatar Aug 21 '18 05:08 vytautas-pranskunas-

For the ads, I think the ad code shouldn't initialize on sever, if it does I think it would be a policy issue for most of the ad networks/advertisers including Google Adsense.

For this we can check and include ad code only when in browser.

You can check if its running in browser

import { PLATFORM_ID } from '@angular/core';
import { isPlatformBrowser } from '@angular/common';

private isBrowser = isPlatformBrowser(this.platformId);

constructor(@Inject(PLATFORM_ID) private platformId) { }

Adding ad code to SPA is little tricky, if your page is making an ajax request and main content of the page depends on completion of that request you should also wait for that request to complete before you add ad code to the page. Otherwise, if ad code is initialized before ajax request completes (and contents are populated) it would also cause ad network policy issue.

naveedahmed1 avatar Aug 22 '18 04:08 naveedahmed1

yeah the issue isn't rendering ads on the server so much as trying to make sure angular doesn't remove/edit/change the ssr element. There was a solution which reattach the ad dom from the server to the new dom created by the client. Again the problem with that is the ad detecting dom manipulation. So the only reason why people want hydration is for ads since it's always faster just to replace the ssr view with csr view. There were solutions made to also keep the element in the dom and insert/remove around that element but would require some rewriting of the renderer to keep track of the elements.

Ivy render will likely solve this since you can choose which elements you want to boot angular into and ignore the ad elements.

PatrickJS avatar Aug 22 '18 07:08 PatrickJS

Adds are not only reason why people want hydration. Hydration needed for prerendering because in scenario when page content is fetched after bootstrapping SSR has no flashing but prerendered. So hydrations needed for prerendered pages also and not sure if ignoring will solve this problem or you will have to ignore whole body because content is everywhere.

I think best solution would be something simmilar like React Virtual DOM which checks what parts where changed and update only those one.

vytautas-pranskunas- avatar Aug 22 '18 07:08 vytautas-pranskunas-

Hydration is only a lib stuff, nothing tied to the view layer, one can already provide custom Renderer to achieve it.

Clear existing contents is an implementation-specific behavior of default Renderer2#selectRootElement provided in platform-browser.

trotyl avatar Aug 22 '18 13:08 trotyl

Do you have any example of custom Renderer2 implementation and how to use it after?

vytautas-pranskunas- avatar Aug 22 '18 14:08 vytautas-pranskunas-

@vytautas-pranskunas- There's an article for old V1 Renderer: https://blog.nrwl.io/experiments-with-angular-renderers-c5f647d4fd9e, should be easy to migrate:

  • Renderer -> Renderer2;
  • RootRenderer -> RendererFactory2;

trotyl avatar Aug 22 '18 14:08 trotyl

trotyl thanks for this. However instead forcing all users to write own renderers it would be benifitial for Angular renderer to update only those dom parts that has changes. Because comparing SSR or prerendered DOM with new one is not on day task if we want to do it correctly not just keep SSR content...

  • Do you know how to access SSR rendered content from custom renderer for comparison?

vytautas-pranskunas- avatar Aug 22 '18 14:08 vytautas-pranskunas-

Yes, the feature request is totally valid, but since it might still need to wait, one can made it in 3rd party library and share to public. Community inputs could also be very beneficial.

trotyl avatar Aug 22 '18 14:08 trotyl

is there any progress with this feature request?

4z5lz avatar Apr 07 '19 08:04 4z5lz

So many things were blocked behind the release of Ivy. I am so ignorant to the many wonderful things that Ivy enabled the Angular team and Angular lib authors to do, so I am going to ask: does the release of Ivy enable this ticket to be worked on?

With 2020 being the year that the Angular community got improved pre-rendering with Angular Universal and also got wonderful pre-renderer in Scully (I am biased), this would be a fantastic time for hydration to become a feature that Angular devs can count on. Perhaps not this exact flavor of hydration. But some flavor that allows developers to opt-in to a "don't delete my nodes if they still work" type of functionality.

Anything?

aaronfrost avatar Feb 19 '20 06:02 aaronfrost

Good question (I don't know the answer), but we at Nrwl have clients who would benefit from this, and might be willing to sponsor work.

jeffbcross avatar Feb 19 '20 06:02 jeffbcross

but we at Nrwl have clients who would benefit from this, and might be willing to sponsor work.

Oh yes, +1 for this 👏

juristr avatar Feb 19 '20 09:02 juristr

I can’t speak in an official capacity without official sign-off from the team, but this is one of the Universal team’s priorities post-v9. What this will end up looking like will depend almost wholly on commitment and design collaboration with the framework team, which is currently pending. We’ll post updates here as they come, but I will certainly relay this interest higher up.

CaerusKaru avatar Feb 19 '20 13:02 CaerusKaru

I hope that whatever is the finalized solution is, that is available to Scully and other pre-render solutions out there as well.

Thanks for the update @CaerusKaru

aaronfrost avatar Feb 19 '20 15:02 aaronfrost

Here is another case:

The app aiming for a max potential Lighthouse 6 Performance score gets:

image

Looking at the trace:

image

What happens is:

  • The app gets FCP at 1.4s
  • Universal re-hydration kicks in replacing all the DOM although the DOM between FCP and LCP does not change at all
  • LCP at 3.1s blows Lighthouse score..

Solution

Fix re-hydration, by keeping the same DOM (not replacing it).

Suddenly, the score starts looking like:

image

Trace:

image

What happens:

  • LCP = FCP = 1.4s (boom! 100% score)

In the example above I have commented the pollyfill zone.js pretty much stopping the app from executing JS which triggers rehydration. The DOM is fully legit and styled (with the data loaded from the DB via API), except for it does not contain any JS listeners..

--

@CaerusKaru

The https://github.com/angular/universal/blob/master/README.md#angular-50-and-beyond contains:

In Progress Static site rendering Planning Full client rehydration strategy that reuses DOM elements/CSS rendered on the server

Are there any chances this will include the fix for LCP as well? Adding that to Universal makes Angular Universal a pretty much ultimate PWA solution with all the tools for devs, SEO and clients.

h3d0 avatar May 28 '20 10:05 h3d0

One more important note. Yesterday Google has officially announced that Web Vitals become a major factor for updated page rank.

This means that FCP, LCP and FID will affect how the apps are ranked in the search results.

The bare-bone Angular Universal + PWA project will have numbers similar to this:

image

  • This is a 20ms TTFB hosting service.
  • With all possible improvements possible up to date (Brotli compression, proxy server + DB caching, and many other improvements).

Looking at the trace:

image

This is pretty-much the minimum main bundle one can possible have (without any 3rd party code):

image

And the Total Blocking Time already around ~500ms.

With this input in mind, users of Angular are not allowed to have 100% score even with the minimum setup. Now what happens when the actual app code is added?


Ivy-universal live?

There is a very interesting project by vikerman - https://github.com/vikerman/ivy-universal that could possible resolve this and many other issues. Creating a 12kb main bundle with ability to code split on the component level. Doing that + smart re-hydration (without destroying existing DOM) would be a huge leap. Are there any plans to continue working in this direction?

h3d0 avatar May 29 '20 08:05 h3d0

There's a difference between hydration and progressive hydration. The demo by Vikram implements the latter.

In general, once an app has been pre-rendered or server-side rendered the user would get a large FCP. Later on, the page would reference script files which contain the framework as well as our Angular app. Once the framework takes over, there are two scenarios:

  1. It (re-)renders the UI
  2. It tries to hydrate the UI

If, as developers, we do our job well, after the framework takes over users would see the exact same UI, no matter if we went through 1. or 2. This means that theoretically LCP and FCP for server-side/pre-rendered application done right should be the same (I'm not saying they are the same right now).

Of course, ideally we'd want to progressively hydrate the application. This way we'd not only improve LCP and FCP but also TTI.

Let me quickly share what's progressive hydration and why it's that hard to achieve. With progressive hydration we have the same SSR or pre-rendering behavior. Together with the rendered page we also ship a tiny bit of JavaScript. This tiny bit of JavaScript tries to understand when it should trigger hydration of a particular component in the component tree. Things get a little more complicated when we start talking about state, so let's ignore state for now.

Imagine the user clicks a button. The tiny bit of JavaScript would detect that interaction, it'll figure out which is the associated component with this button and will trigger a network request, downloading the component and all of its dependencies that need to be loaded synchronously.

See how we only hydrated the component the user was interested in and its synchronous dependencies, nothing else. This means that we got the smallest amount of JavaScript that can handle the user action. That's how we'd speed the TTI up.

Now, why is this hard? Imagine the component which handled the button click emits an event that needs to be handled somewhere in the component tree, outside of the synchronous dependencies that we downloaded. By event I don't mean an output or a DOM event, I mean any event, for example:

const pubSub = new EventEmitter();
pubSub.emit('foo', bar);

How can the framework understand that this event was triggered? How can it understand who's listening for it? There's no way, unless the message bus abstraction comes from the framework and the dependency graph was statically analyzable (i.e. at build-time we know who is going to listen for foo so we can add a loading instruction somewhere).

This is the hard part. We need to set constraints on how folks can use Angular so that we can enable progressive hydration. Theoretically, you can use this subset of Angular and build apps with progressive hydration today. The example app from above implements progressive hydration and also uses a subset of Angular that's compatible with this paradigm today. It's not clear whether the constraints the progressive hydration sets are viable at the moment.

Aside from that, if you notice significant differences between the timestamps for FCP and LCP for apps which do not render additional content on the screen when the framework takes over, please comment here.

mgechev avatar May 29 '20 21:05 mgechev

@mgechev thank you for an explanation.

if you notice significant differences between the timestamps for FCP and LCP for apps which do not render additional content on the screen when the framework takes over, please comment here.

Here is an example.

Steps to reproduce

  1. Run ng new --routing foo.
  2. Run ng add @nguniversal/express-engine.
  3. Clean up app.component.html all code, keep <router-outlet></router-outlet>.
  4. Run ng generate module --module=/ --routing=true bar.

At this point, this is the most minimal setup possible. There is pretty much a root app component and a bar component which is supposed to be loaded on demand (code splitting).

  • The main.js bundle contains only the core Angular code (219Kb, 83 Gzipped).
  • The 5-es2015.js contains bar code (744B).

Performance

Now have a look at the Performance trace:

image

This is the same setup used by Google Lighthouse

FCP is at 1.2s mark:

image

LCP is at 2.5s mark!

image

This already barely falls under Google's Core Web Vitals:

image

Taking into account Lighthouse 6 updated calculations (LCP (25%) and TTI (25%) both affect 50% of the rating). This already puts the users (devs) of Angular Universal in a disadvantage. Adding any meaningful 3rd party tools (Google's Angular Material, fonts, animations) will make the performance result even worther.

Potential problems

NOTE: The simulation (used by LH as well) adds ~560ms round-trip delay to all Network events.

  1. The client makes a request and the server returns a rendered HTML page. This part is completely on the dev (optimizing server TTFB via using a viable hosting service, tuning the proxy, using compressions and cache):

image

  1. The client receives the data and starts loading the resources (JS, styles, fonts, images). After this is done the FCP occures:

image

The data is used for SEO (crawlers) and to provide an early FCP. If the dev does everything right (and do not run different code on client and server side) - the FCP is also a LCP, because Universal renders the exact copy of the view.

What happens next is the client discovers the lazy-loaded chunk (for the bar component) and start loading it - resulting in another 600ms delay! (Remember LH uses 560 round-trip delay for measurements). It takes extra 600ms to load the 700B JS file:

image

Thoughts

  • LCP should occur with FCP (-600ms) because the rendered content looks exactly the same at the final result. Angular re-hydration re-creates DOM delaying the LCP.

  • The lazy loaded chunks should be discovered earlier (-600ms), not on DCL. Keeping things like this makes route-level code-splitting a joke. Moving the bar component into the main bundle reduces this 600ms. Unless the lazy-loaded chunk worth approx 200+ Kb it will be spoiling the Performance score instead of improving it.

In my opinion the root of the problem is the code that cause LCP at the very end. I have found a similar issue reported with a comment:

You've bumped into an unfortunate edge case where the heuristic used by first meaningful paint fails. Apparently the hydration step of angular universal touches/adds enough elements to fool the heuristic into selecting the later paint instead of the real first one.

The case I've shown is the "Hello world" Angular Universal app - it makes every app powered by Universal an edge case and is clearly a show stopper for falling under Google's Core Web Vitals.

h3d0 avatar May 30 '20 10:05 h3d0

I share the same concerns as pointed out by @h3d0 . Maintaining great Lighthouse score has always been very challenging with Angular Apps; with Lighthouse 6's updated calculations and Web Vitals it has now become more difficult.

naveedahmed1 avatar May 30 '20 12:05 naveedahmed1

@h3d0 thanks for providing this analysis. We'll discuss the implementation of LCP with the right folks and come back to this thread when we can.

mgechev avatar May 30 '20 14:05 mgechev

Just to give you heads up. We got in touch with Chrome. The way we hydrate the view is not yet impacting your search ranking, but may do in the next a couple of months.

We're in a process of discussing how to reduce the risk of this happening.

mgechev avatar Jun 04 '20 19:06 mgechev

Chrome is experimenting with a new definition of LCP, which would have much lower impact on Angular apps (if any). You can track the progress here.

Based on my understanding, Google Search does not yet penalize Angular apps with SSR.

mgechev avatar Jun 05 '20 18:06 mgechev

What can we do here? I've been racking my brain for 2 days now in trying to make this work and in my case, simply rendering just the header of profile page - no extra parts, still that goes way above Google's threshold

pitAlex avatar Jun 06 '20 03:06 pitAlex

Chrome is experimenting with a new definition of LCP, which would have much lower impact on Angular apps (if any).

Thank you for escalating this with the Chrome team. While been an excellent and fun to develop with, I am grateful that Angular will be treated fairly by the new performance measurement algorithms.

Based on my understanding, Google Search does not yet penalize Angular apps with SSR.

That is correct. If I understood Google's statement correctly as well:

A note on timing: We recognize many site owners are rightfully placing their focus on responding to the effects of COVID-19. The ranking changes described in this post will not happen before next year, and we will provide at least six months notice before they’re rolled out.

So the Chrome team has at least 6 months to address the issue.


@pitAlex Angular has many ways to increase the performance of the app (both start-up and run). Ivy renderer improves compiling and reduces the bundle size, Universal got new ways of debugging your app (running a production-like build, while being moderately fast to update).

I would suggest to get more knowledge at places like Web.dev or Angular InDepth. If you still struggle with your issue, start a thread and provide more details (setup, what has been done, what issues have popped).

h3d0 avatar Jun 08 '20 09:06 h3d0

I don't see how this can be fixed if Google starts using the client version of the page and this is what we see in our current search console analytics. For example, the angular bundle size of a page is 191kb and that takes 2.4s to download on a simulation of "4x slowdown" with "fast 3g". As it flushes the DOM it ends up with LCP of 6.3s when measured. But if I take the same page and using angular node rendering, without insert any js, I am below 2s with a perfect score - always. This his how we set up our pages: for search engines we server non-js version and for users, we give the one with angular. But if Google is now ignoring the search engine version, there is no point in even using frameworks like angular. If it takes 2.4s to download, I have almost nothing left for executing the respective bundle and rendering the page. And no production page will be as simple as a "hello world" template that I keep seeing in all these guides. Let me see, for example, that the angular.io can meet these requirements. That's a realistic production page.

pitAlex avatar Jun 16 '20 23:06 pitAlex

I definitely appreciate you sharing your concerns, they are a valuable input for the team.

As I mentioned above, we're in a process of resolving this issue, working in collaboration with Chrome. Fixing the problem is a top priority for us.

At the moment there are no Angular apps impacted. I'd suggest to follow the progress from the ticket I shared above. We're looking into the implementation of LCP together with the original authors. I strongly believe we're on the right track.

I'll keep you posted in this issue if there are updates.

mgechev avatar Jun 17 '20 16:06 mgechev

This his how we set up our pages: for search engines we server non-js version and for users, we give the one with angular. But if Google is now ignoring the search engine version, there is no point in even using frameworks like angular.

If you are serving two different versions of the same page, one for the search engines and one for your users, isn't it clocking? and wont the search engines consider it as spamdexing technique?

And since you mentioned:

I don't see how this can be fixed if Google starts using the client version of the page and this is what we see in our current search console analytics.

There's a possibility that Google is already ignoring search engine specific version of your page and choosing to index what an end user actually see when they visit your website.

As announced in last year's Google I/O, I think Google's crawlers Googlebot is now “evergreen,” which means the crawler will always be up-to-date on the latest version of Chromium so it will see your page exactly or at least close to as a normal users see it in browser.

Ref: https://webmasters.googleblog.com/2019/05/the-new-evergreen-googlebot.html

We're using Angular + SSR/Universal since v2 and so for we haven't seen any search engine indexing issue with our website. The only issue we noticed is the new LCP matrices introduced in Lighthouse 6, but I'm confident that the way Angular team is working closely with Chrome team, we will see a fix soon.

naveedahmed1 avatar Jun 17 '20 17:06 naveedahmed1

Hi folks 👋🏾.

Dropping some updates here (I work with the Chrome team):

  • As Minko mentioned, you can track the progress of the work here. A CL has already been merged to include a new experimental version of LCP that handles removed elements (there's still more work to be done).
  • Next steps would involve getting approvals before landing and updating the current implementation of LCP in Chrome's metrics pipeline as well as the web performance API, which can take some time.

This issue was prioritized after noticing it affects server-rendered sites that re-render DOM on the client, so thanks to all of you for flagging!

housseindjirdeh avatar Jun 18 '20 19:06 housseindjirdeh

@mgechev @housseindjirdeh what is the current status of this issue?

Chrome 84 has been officially released around 4 days ago. Lighthouse 6.1.0 still incorrectly puts LCP to the very end, while Angular SSR returns 100% unchanged DOM on FCP:

image

The DOM has no single change between FCP and LCP, Angular would only re-create the nodes and add JavaScript to it (re-hydrate).

This change already hit our commercial project Performance:

image

the Performance score dropped from 97-100 to 67 - without any changes have been made to the project code itself.

h3d0 avatar Jul 18 '20 07:07 h3d0

Lab tests

There is a spread in the metric got from different sources. Ran 5 tests for each method.

Web Dev Measure

https://web.dev/measure

image

Pagespeed Insights

https://developers.google.com/speed/pagespeed/insights/

image

Chrome Lighthouse extension (Chrome 84, latest extension)

Incognito mode

image

NPM Lighthouse 6.1.0

lighthouse <url> - https://www.npmjs.com/package/lighthouse

image

Summary

From all the tools only NPM Lighhouse 6.1.0 sometimes, on some pages correctly counts the LCP (1.5 sec in case of our proj for that page).

I would like to point out that from the developer's perspective - having 4 tools which does the same thing and getting different results is a complete mess. Secondly, it seems like only the NPM Lighthouse correctly measure the LCP.

Which tool does Google use for ranking a page? We kindly need a single verified working source of truth for measuring web-apps performance.

h3d0 avatar Jul 18 '20 11:07 h3d0

The change hasn't been reflected in Google search yet (see comments above).

There are also no changes in the metric implementation. @housseindjirdeh and I will make sure to share any updates.

mgechev avatar Jul 18 '20 12:07 mgechev

Hi folks, the Chrome team has been working hard on the new implementation of LCP and it is already available in recent versions. You can give it a try with your Angular apps using:

  1. Use a Chrome version that's at least 85.0.4182.0 (Canary or Dev right now).
  2. Make sure chrome://ukm shows ENABLED, otherwise relaunch chrome with --force-enable-metrics-reporting.
  3. Load the website (don't do any early inputs to avoid terminating the algorithm early), and then close that tab.
  4. Refresh the chrome://ukm and find PaintTiming.NavigationToExperimentalLargestContentfulPaint within the correct URL with the PageLoad event (compare with the current version: PaintTiming.NavigationToLargestContentfulPaint).

The expected behavior is NavigationToExperimentalLargestContentfulPaint being less than NavigationToLargestContentfulPaint. Keep in mind that if you have layout shift (i.e. you don't render the exact same content), you may still see both markers showing the same value.

If you see a different behavior, please share an example where we can reproduce the incorrect LCP.

mgechev avatar Jul 22 '20 04:07 mgechev

It looks good ! Thanks for the update @mgechev Leaving this in case it can help anyone ( make sure to close the tab where you loaded the site before refreshing chrome://ukm ) Screenshot 2020-07-22 at 11 04 44 Screenshot 2020-07-22 at 11 10 37

ghost avatar Jul 22 '20 10:07 ghost

@h3d0, would you also check both metrics?

mgechev avatar Jul 23 '20 17:07 mgechev

@h3d0, would you also check both metrics?

Thank you for the efforts and feedback. Was travelling for 2 days, finally back at my desk to test the new solution.

Testing

Dashboard

image

Products page

image

Individual product page

image

Summary

So far pretty noticeable positive impact! The most simple (lightweight) Individual Product page the NavigationToExperimentalLargestContentfulPaint is 1587 (with the NavigationToFirstContentfulPaint at 1387). The most complex testes Dashboard page has (NavigationToLargestContentfulPaint is 5294 vs NavigationToExperimentalLargestContentfulPaint is 2909).

I suppose, the perfect case is when the LCP is as close as possible to FCP for Angular SSR. But, so far the NavigationToExperimentalLargestContentfulPaint looks very promissing :+1:

h3d0 avatar Jul 24 '20 11:07 h3d0

I managed to get good results too. In my case, FCP and experimental LCP are the same Screen Shot 2020-07-25 at 6 26 58 AM Took a while to correctly do the angular SSR setup, I kept getting flickers. However, if I were to run the lighthouse tab I would get LCP 10 seconds... why so much difference? Shouldn't it have been at least 4s?

pitAlex avatar Jul 25 '20 03:07 pitAlex

@pitAlex it could be useful to share here what did you do to remove flickers so folks can follow same practices.

However, if I were to run the lighthouse tab I would get LCP 10 seconds...

If you have a lot of JavaScript and other blocking resources on a low-end device this could be possible. In such a case, you'd get LCP after the app's JavaScript is downloaded and executed.

mgechev avatar Jul 25 '20 15:07 mgechev

@mgechev in my case it was about properly ensuring that the module of the route, which is lazily loaded, is at the ready at the moment angular bootstraps by waiting for both to finish downloading (main + lazy). What I see is that you must avoid async code or promises as much as possible. So even data, I have it printed as json and parsed through ngOnInit and not by looking through a guard resolve data.

pitAlex avatar Jul 25 '20 21:07 pitAlex

Hi everyone, @mgechev,

It seems that the focus of this thread has shifted a bit for fixing LCP / FCP because of its potential negatives impacts for SEO of existing websites. I understand that of course. This was related to Google Search.

But what about the 1st main topic of this issue and its use case for dealing with ad programs such as Google AdSense ?

How to not have this flickering effect ? Would it be possible to have a Universal Angular prerendered app which could be accepted in Google AdSense program after that ? Is this on Angular Universal roadmap ?

Are there any best practices to fetch data "the right way" if Users would need to display Ads ? Do you have any links ?

Thanks a lot for your efforts towards this, all teams working together like that is something very valuable 👍.

johanchouquet avatar Jul 29 '20 14:07 johanchouquet

Currently, our focus is on LCP because it's a critical issue that we need to address as soon as possible. At the same time, hydration has a large scope. Depending on it to workaround the LCP problem may cause delays and have negative consequences.

This doesn't mean that hydration is not worth exploring in the future. It could definitely bring benefits on the table.

mgechev avatar Jul 29 '20 22:07 mgechev

Thanks @mgechev for your explanations.

Sure thing about LCP. About hydration, as I understand it, benefits could be that pre-rendered Angular Universal SPA could be validated for AdSense program or similar. Is that correct ? If so, it could mean a LOT to our business 🚀.

johanchouquet avatar Jul 30 '20 13:07 johanchouquet

The latest Chrome bug report page (https://bugs.chromium.org/p/chromium/issues/detail?id=1045640) states:

NextAction: 2020-09-01 We need to wait until this experimental version reaches Stable (end of August) to be able to make a decision here.

Yesterday, Chrome 85 has been released, the problem seems to persist. Does NextAction: 2020-09-01 mean the decision will be taken on September 1st?

h3d0 avatar Aug 26 '20 11:08 h3d0

Very strange results regarding LCP in Lighthouse. I made a component with this only code: <br><span>try</span> and results is 1.7 for LCP. with code: <br><div>try</div> the result is 4.1 for LCP. I compiled and tried many times, in Incognito mode, removing all caches etc. Angular version: 9.1.1 Chrome version: 85 with_span with_span_lighthouse with_div with_div_lighthouse

cantacell avatar Sep 22 '20 08:09 cantacell

@cantacell I tried to reproduce but no success I am seeing similar results for both using a div / span. Do you mind providing a way to reproduce or link to a demo ? I am curious to see if that has any impact.

Do you mind as well trying out with Google Page Speed , see if you have similar issue, this could just be lighthouse scoring.

ghost avatar Sep 22 '20 10:09 ghost

@cantacell I tried to reproduce but no success I am seeing similar results for both using a div / span. Do you mind providing a way to reproduce or link to a demo ? I am curious to see if that has any impact.

Do you mind as well trying out with Google Page Speed , see if you have similar issue, this could just be lighthouse scoring.

I tried on my (complex) project, eliminating components one at time until I reached an empty (lazy load) component. Now I try to a clean project.

cantacell avatar Sep 22 '20 10:09 cantacell

I made a lazy module (home) and tried many times with <span> and with <div>: With span the LCP is ever near FCP. With div only few times is near FCP, other times far from FCP. First 2 screenshoots are for <span>, other 2 are for <div> <span> span_good_2 span_good_1

<div> div_bad div_good

cantacell avatar Sep 22 '20 15:09 cantacell

project files https://drive.google.com/file/d/1EjYr9jmceWFiNkUM9NzKlJueRh8AxTRp/view?usp=sharing

cantacell avatar Sep 22 '20 15:09 cantacell

Another quick Chrome update:

  • The new experimental implementation of LCP that addresses these issues has landed as the default version of the metric https://chromium-review.googlesource.com/c/chromium/src/+/2480845 🎉
  • As soon as this rolls out in the next Chrome stable version, the web perf API will be updated. At a later point, it would also be updated internally.

Keep an eye out on the crbug for any newer updates :)

housseindjirdeh avatar Oct 29 '20 17:10 housseindjirdeh

One can now download chrome version 88 from the dev channel https://www.google.com/chrome/dev/?extra=devchannel. If you use lighthouse inside there, you'll see the LCP calculated with the new method.

And google announced via https://webmasters.googleblog.com/2020/11/timing-for-page-experience.html that the switch to the new ranking starts 05/2021

sod avatar Nov 16 '20 10:11 sod

Thanks @sod - results look good! Mobile score jumped from low 60s to 90+ for a Scully generated Angular site.

michielkikkert avatar Nov 16 '20 10:11 michielkikkert

It's a step in the right direction, but we still seem to be getting penalized for have an angular prerendered app.

LCP is now good after the hydration at about 600ms as opposed to 2.x seconds, but Time to Interactive is still at 2.x seconds, assuming that this is still waiting for the hydration? The page is able to be scrolled and looked at, not all buttons work, but we could put logic in to use hrefs instead of clicks, that would probably still keep us at the same Time to Interactive though I wager.

Also, still getting pinged for unused JavaScript when preloading the site :/

Should I file a new bug for the Time to Interactive?

If anyone's interested, I'm pulling these numbers from our homepage running lighthouse on Desktop

https://startbootstrap.com

And then pulled from ChromeDev.

I suppose the question I really want answered is, will this score of 89 rank us lower, even though we have under 1% bounce rate and long page views and lots of interaction? I ask because we just deployed the new site 2 weeks ago, and previously the site was just a jekyll site that had a 99 page speed score.

initplatform avatar Nov 16 '20 20:11 initplatform

@initplatform this does not seem like a hydration problem. Having a lot of JavaScript will delay time to interactive because the browser needs to be download, parse, and execute the scripts.

What you're referring to, could be improved with progressive hydration, but that's an entirely new project. See this comment for more details. We have a prototype from Google I/O 2019, but the implementation is not production ready and the APIs are not ergonomic.

mgechev avatar Nov 16 '20 23:11 mgechev

@mgechev That makes sense... the problem/advantage is we have a lot of functionality in the app so after initial load everything is near instantaneous, whereas before every page was a reload. I'm lazy loading sub-modules, but I'll have to investigate further how I can trim off more of the initial payload. Thanks!

initplatform avatar Nov 17 '20 01:11 initplatform

I have Angular+Scully site and I also have a problem with LCP because hydration happens on top of an already rendered page and it removes the text/images that are already rendering/rendered and starts it again. I could see it by screenshots in profiling when some images are already rendered from the pre-rendered HTML and they get removed by Angular and added again. Lighthouse considers the last paint as LCP. I tried profiling with the dev version of Chrome and it was still inconsistent there when LCP happens (I think depending on whether the image was rendered before angular started hydrating or not. I believe rehydration would at least partially solve this problem and I wouldn't get such poor scores with page speed insights. (On my pc lighthouse currently gives me 80-90 score, but on my laptop and page insights it's around 50 - quite inconsistent result). Feels like right now the only way to have a good score is to get rid of angular on the page or delay its initialization, but I don't want to ruin the UX on my site because it is more important.

BlindDespair avatar Dec 02 '20 13:12 BlindDespair

I'm really hoping that this will fix soon. It impacts a large amount of organization who merely relies on the pagespeed. My application when blocked JS gives a 100 score.

sahilpurav avatar Dec 21 '20 14:12 sahilpurav

@BlindDespair if there are no other problems, this shouldn't be the case. Are these text/image removals visible only in Chrome DevTools or are they also noticeable from users' perspective? How does LCP compare to FCP? Could you provide a URL where we could reproduce the problem?

mgechev avatar Dec 24 '20 00:12 mgechev

@mgechev I'm facing the same issue. Even in the Dev version of Chrome, my LCP goes way beyond FCP and the results are inconsistent. URL - https://www.editage.com/

sahilpurav avatar Dec 24 '20 07:12 sahilpurav

@sahilpurav after having a quick look, the problem does not seem related to hydration. Notice how the navigation causes the content below to jump once Angular takes over when the scripts load. This triggers another LCP. Make sure the styles at SSR are exactly the same as the CSR styles Angular applies.

mgechev avatar Dec 24 '20 11:12 mgechev

@mgechev You can check our home page: https://sakurachef.com.ua/ I am gonna attach a few screenshots here. Check how all the images in the header get rendered and then disappear for a moment on the next screenshot and then one screenshot later all the images are there again. As I user I cannot see this though, there is no flickering nor flashing. LCP in my case is way far away from FCP and all of my checks only a couple of times it randomly was basically same time. LCP in this case is considered the render of the picture in the banner, however, I tried to also remove most of my DOM, essencially leaving only text and still LCP had the same issue. :( Screenshot from 2020-12-24 15-00-54 Screenshot from 2020-12-24 15-00-55 Screenshot from 2020-12-24 15-00-56

BlindDespair avatar Dec 24 '20 14:12 BlindDespair

@BlindDespair - It looks like, once Angular kicks in, it also start the a slider kinda script that clears you large header. Also, the chat link seems to be falling within the measurement (don't think that has that much impact though). Maybe worth investigating if you can completely delay the initialization of the Image slider and see if that helps?

michielkikkert avatar Dec 24 '20 15:12 michielkikkert

@MikeOne The slider is not an external script, it's just an Angular component. I do have a delay of sliding there, but angular still rerenders it when it takes over. Here is how sliding works:

readonly activeBannerIndex$: Observable<number> = isScullyRunning()
    ? of(0)
    : this.selectedBannerIndex$.pipe(
        switchMap(index =>
          interval(this.autoSlideInterval).pipe(
            skip(1),
            scan(activeIndex => (activeIndex === this.banners.length - 1 ? 0 : activeIndex + 1), index),
            startWith(index)
          )
        ),
        shareReplay(1)
      );

And here is how chat is initialized:

  get timeout(): number {
    return isScullyRunning() ? 0 : 2000;
  }

  loadChatWidget(): void {

    this.zone.runOutsideAngular(async () => {
      const [, content] = await forkJoin([timer(this.timeout).pipe(first()), this.content$]).toPromise();
      if (isScullyRunning()) {
         return;
      }
      (window as any).intergramId = environment.chatWidget.id;
      (window as any).intergramServer = environment.chatWidget.serverUrl;
      (window as any).intergramCustomizations = { ...content, mainColor: '#ff686b' };
      await loadScript(environment.chatWidget.scriptUrl);
    });
  }

As you can see chat is already delayed by 2 seconds. But here is the thing. I tried to completely remove the slider but the LCP number didn't change at all.

BlindDespair avatar Dec 24 '20 17:12 BlindDespair

@BlindDespair, thanks for sharing the URL. There seems to be a lot of changes on the screen after FCP. I'll have to take a better look after the holidays, but at first sight, this is not an Angular issue.

Keep in mind that non-destructive hydration doesn't mean FCP == LCP. The cost of rebuilding the DOM is not so much higher than traversing the existing structure and attaching the corresponding listeners. On top of that, both operations have potential frame drops since they execute on the main thread. Other issues are related to content shifts and problems in the state transfer between server and client. They are usually on the application side, not the framework.

With the current state of the tooling, however, this is not obvious. We'll pass this feedback to the groups working on the debugging tooling for web vitals and see where we can go from there.

All this feedback is very valuable and the more examples we have, the better guidelines and tooling we'll be able to build. If anyone else faces similar issues, please share.

mgechev avatar Dec 25 '20 16:12 mgechev

All this feedback is very valuable and the more examples we have, the better guidelines and tooling we'll be able to build. If anyone else faces similar issues, please share.

@mgechev

I can share our example. https://floralle.com.ua - is an e-commerce PWA powered by Angular + Angular Universal, aims to follow all the latest recommendations and techniques (code splitting, minifications, obfuscations, Brotli/Gzip compression, WebP/AVIF, server optimizations, fastest hosting provide in the area). Yet its weak point still seems to be the re-hydration.

Google Chrome Version 87.0.4280.88 (incognito mode)

Simulated throttling off

image

LCP is again somewhere at the end, while in fact the page never really changes after FCP (except for the BG image got loaded). Even if the image get pre-loaded with rel="preload" - the output is the same, LCP is far behind (which does not really make sense?):

fcp-vs-lcp

Simulated throttling on

image

Google Chrome Version 89.0.4356.6 (Official Build) dev (64-bit) (incognito mode)

image

Same problem, LCP seem not to be fixed. If the dev-teams views the Original Trace - the behavior will be the same as for Chrome 87 (didn't the experimental bug fix from Chromium team suppose to get into 88 stable?? Then why Chrome 89 and NPM 7.0.0 seem not to have it?).

image

Note how the Trace is completely the same as for Chrome 87 - the fact that the background image is already rendered and stays for around 1.5s is completely ignored by the LIghthouse - putting the LCP timing at the very end:

chrome-89

Notice the Cumulative Layout Shift of 0 in every case.

NPM 7.0.0

And here is the result from running Lighthouse 7.0.0 from a local machine (throttling set to 2x):

image

Again we can see the LCP 2.6s vs FCP 1.1s.

h3d0 avatar Dec 28 '20 07:12 h3d0

All this feedback is very valuable and the more examples we have, the better guidelines and tooling we'll be able to build. If anyone else faces similar issues, please share.

@mgechev

I can share our example. https://floralle.com.ua - is an e-commerce PWA powered by Angular + Angular Universal, aims to follow all the latest recommendations and techniques (code splitting, minifications, obfuscations, Brotli/Gzip compression, WebP/AVIF, server optimizations, fastest hosting provide in the area). Yet its weak point still seems to be the re-hydration.

@h3d0 Is this really using Angular Universal. I checked the view source and couldn't see the DOM added inside app-root.

sahilpurav avatar Dec 28 '20 08:12 sahilpurav

@sahilpurav after having a quick look, the problem does not seem related to hydration. Notice how the navigation causes the content below to jump once Angular takes over when the scripts load. This triggers another LCP. Make sure the styles at SSR are exactly the same as the CSR styles Angular applies.

Thanks for the reply @mgechev, the CSS for SSR and CSR are exactly matching. Although the fonts are using "swap" property because of which it first fallback to the system fonts and then switches back to the original font-face. I believe that's one of the recommendations by pagespeed. Will this really cause an issue in LCP?

sahilpurav avatar Dec 28 '20 09:12 sahilpurav

@h3d0 Is this really using Angular Universal. I checked the view source and couldn't see the DOM added inside app-root.

Yes.

If one simply navigates to the url and try to view the source - it will show no DOM. One would need to use Incognito Mode and make sure to turn off Service Worker and clear the cache. The idea is that full DOM is shown only on first load. After that everything (assets, JS and even some requests) are cached and served using JS and Service Worker (and the blazing fast experience for the user). So if you see no data inside app-root it is because it has been already cached and served.

One can use a script to fetch as Google bot, e.g.:

fetch_as_google() {
  url=${1:-https://floralle.com.ua}
  tmp_path=/tmp/foo
  tmp_file=$tmp_path/foo.html
  mkdir -p $tmp_path
  echo "Fetch as Google Bot"
  curl --user-agent "Googlebot/2.1 (+http://www.google.com/bot.html)" \
    -o $tmp_file \
    -v $url
  vim $tmp_file
}

Google Search Console also shows the correct DOM. So eah - it is Universal-powered, and Google crawls, indexes and knows about the content.

h3d0 avatar Dec 28 '20 11:12 h3d0

@mgechev

Question from @sahilpurav guided me into the next finding. When loading a page, Chrome Dev Tools clearly shows that the server returns a rendered page full of DOM (Dev Tools > Network):

image

However when checking the source for the same page, the app-root is indeed empty:

image

Is it possible that Angular incorrectly destroys the full DOM returned by the server and replaces everything with the plain empty DOM (which one gets when Universal is not used)? And that is why Lighthouse does not treat the returned DOM as "true" DOM and triggers LCP at the very end, when the main.js has been loaded, compiled and executed?

h3d0 avatar Dec 28 '20 12:12 h3d0

@h3d0 I can't reproduce this. When I open your page it's always CSR.

Screen Shot 2020-12-29 at 12 13 55 AM

mgechev avatar Dec 28 '20 22:12 mgechev

@sahilpurav it looks like the loading is not consistent. The banner alternates from banner.jpg to banner-default.jpg, back to banner.jpg and the number of visible menu items changes.

mgechev avatar Dec 28 '20 22:12 mgechev

I can't reproduce this. When I open your page it's always CSR.

@mgechev

Attached GIF demonstrating how Service Worker affects the CSR vs SSR

  • Chrome > Application > Bypass for nework checked - means Service Worker does not serve cached assets, in this case SSR is returned.
  • Chrome > Application > Bypass for network unchecked - means Service Worker serves cached files, returning CSR.

sw

Both Universal and Service Worker are added into the project using standard schematics provided by angular docs:

  • SW ng add @angular/pwa --project *project-name*
  • Universal ng add @nguniversal/express-engine

All config files remain unchanged, this is the behavior one gets with the current Angular Stable by default.

h3d0 avatar Dec 29 '20 08:12 h3d0

@mgechev - There were some glitches on the site at the time you checked. Following is the example where you won't see any banner and deviation in the navigation.

Link: https://www.editage.com/help

Below screengrab is on the Chrome 89 Dev version. You can see the difference between FCP and LCP while things remain exactly the same.

image

sahilpurav avatar Dec 30 '20 15:12 sahilpurav

~The current LCP metric (as of 89.0.4356.6) seems not to handle picture elements like:~

        <picture>
            <source type="image/webp" [srcset]="image.srcset.webp" />
            <img [src]="image.fallback" [alt]="image.title" class="w-100 h-auto" />
        </picture>

~So we now give chrome the jpeg treatment - twice as big files with a single <img> tag - just to not lose 20 lighthouse points :)~

edit: This claim was wrong, see https://github.com/angular/angular/issues/13446#issuecomment-756791046

sod avatar Jan 08 '21 13:01 sod

@sod - I am interested in your last comment. Can you please elaborate more on this OR point me to a chrome bug somewhere related to 'picture' tag?

rockeynebhwani avatar Jan 08 '21 14:01 rockeynebhwani

@rockeynebhwani thx for asking so I went into the numbers and looked more thoughtfully. Seems like my claim was wrong.

I misinterpreted LCP being above TTI as LCP not working with SSR on that page. image

But if I slow down the CPU so TTI is way slower (e.g. TTI 5s), FCP stays at 1.6s. So it seems to actually work. Sorry for the confusion :/

sod avatar Jan 08 '21 14:01 sod

no problem @sod. Thanks for clarifying

rockeynebhwani avatar Jan 08 '21 14:01 rockeynebhwani

Hey folks, thank you a lot for all the reproductions! They were very helpful in determining what the impact of the current LCP implementations on Angular apps is.

Originally there were concerns that Angular might be causing delayed LCP because of the way it hydrates the view after SSR. After a couple of conversations and a collaboration with the Chrome metrics team, I can now confirm that this is not the case.

Here's the current implementation of the metric:

...Content that is removed is still considered by the algorithm. In particular, if the content removed was the largest, then a new entry is created only if larger content is ever added. The algorithm terminates whenever scroll or input events occur, since those are likely to introduce new content into the website. Source.

The paragraph above into the Angular context means that once the browser renders Universal's markup, it will find an LCP candidate. Later, when the client-side JavaScript takes over, it will render the same UI, meaning that there will be no larger LCP candidate, so Chrome will not trigger the metric again. All this will happen, assuming that there are no state inconsistencies between the client and the server. Angular provides the tools to ensure a consistent state between the client and the server.

It's essential to understand how to debug your LCP. The best guide I'm aware of is on web.dev/lcp. Notice that the post has the disclaimer:

In the future, elements removed from the DOM may still be considered as LCP candidates. Research is currently being done to assess the impact of this change. You can follow the metrics CHANGELOG to stay up-to-date.

Well, the future is here, and this is no longer the case. Keep in mind that the Chrome team did changes in the metric implementation in version 88, so to understand your LCP, I'd recommend using Canary.

mgechev avatar Jan 13 '21 23:01 mgechev

@mgechev Thank you for actively replying on this issue. Although I still don't understand how I can debug/fix my LCP, I did all the optimizations possible to this moment. Yet my LCP is around 4.8-5.1 seconds. But if I go ahead and delete all the Angular script tags from the index.html that was generated by Scully LCP goes down to 2.1-2.2 seconds which results in 99 performance score. Page still looks the same mostly just cannot be interacted with (except links, you're able to navigate anywhere). What does Angular have to do with rendering the largest image (in my case)? Why is LCP pretty much doubled when Angular is loaded? What can I do other than delaying Angular initialization by like 4 seconds? (Which I won't, because UX is more important to me than Lighthouse scores).

BlindDespair avatar Jan 14 '21 00:01 BlindDespair

I just wanted to update our product status in regards to this thread.

A 1-year optimization journey came to an end:

  • First we have updated everything related to infrastructure, compression (Gzip, Brotli), caching (Nginx) and file formats (WebP, Avif served.
  • Then we got rid of 3rd party JS tools (e.g. flex-layout) and re-done in using native tools (e.g. flex-box and grid).
  • Then we got rid of Material and re-done it using native CSS (SCSS) which turned to be solving everything an ecommerce would need (including things like scrolling, slideshows, carts and forms).

The only unresolved bottleneck left was.. Angular itself. The Angular+Universal+PWA was simply too heavy. The latest Chromium fix did not resolve the problem. But at this point there were no actual benefits to keep Angular, there was no unique problems it would solve.

And so the project was moved from Angular+MySQL+Nodejs stack to Django+Postgres. Super fast models+API out of the bat, native CSS features (via SCSS), native JavaScript was 100% enought to resolve UI interactive part (using latest JS tools + smart principles like BEM). The migration actually took around 3 weeks.

And here we go:

image

The work is still on the finishing stage, however we already see a sturdy performance increase. And most of all - we have a 100% control of what is loading, when it is loading. Every single bit of JS or CSS is loaded exactly only on its own page.

--

I must admit, it was an important experience. Do not use tool just because it is new or cool. Use the tool that solves the problem. If you need an e-commerce (which depends on SEO and Core Web Vitals) - find appropriate tool. If you need an app - use Angular + Material + other fun stuff (or competitors).

h3d0 avatar Feb 01 '21 13:02 h3d0

The Angular+Universal+PWA was simply too heavy

IMHO it's not angular, it's lighthouse. Simulated moto 4g - an over 7 year old device - is just weirdly outdated. >= 4 year old devices is just 2% of our traffic. And even on those - e.g. a samsung galaxy s7 - the page feels fast and responsive.

Desktop lighthouse (this page https://www.rebuy.de/p/iphone-11-pro/11168607): image

I'd never sacrifice angular for 5 extra points.

sod avatar Feb 01 '21 15:02 sod

Hey guys, please fix this issue with LCP.

Google is announcing Core Web Vitals and LCP is very important for us.

Here some examples on fresh angular starter app with enabled prerender and ssr: Lighthouse (7.2.0)
Device - Emulated Moto G4 Network throttling - 150 ms TCP RTT, 1,638.4 Kbps throughput (Simulated) CPU throttling - 4x slowdown (Simulated)

Google Chrome Canary Version 91.0.4444.0 (Official Build) canary (x86_64):

  1. tested with runtime, polyfills and main scripts - LCP ~6.4s with-scripts

  2. tested with commented/removed runtime, polyfills and main scripts - LCP ~0.9s removed-scripts

alexkushnarov avatar Mar 12 '21 20:03 alexkushnarov

@mgechev I'd like to second what @alexkushnarov has said.

We have 30 high traffic e-commerce websites running on Angular and we have spent 100s of hours trying to resolve this LCP issue but clearly, DOM replacement is a problematic solution.

Without high Core Web Vitals scores come May, Angular is not a viable solution for the e-commerce world.

Here is an example https://www.vashi.com/

tomandco avatar Mar 13 '21 09:03 tomandco

If I lighthouse e.g. https://www.vashi.com/loose-diamonds/marquise, then chrome identifies the first image in the product list for LCP

image

But if I open the same site with javascript disabled, thus only seeing the ssr DOM, exactly that item is missing the src

image

The element that chrome identifies for LCP should be identical in ssr & client. So this element should have a src upfront and render an image with javascript disabled.

sod avatar Mar 13 '21 11:03 sod

@sod thanks for pointing this out. This is actually a recent experiment to deal with the fact we dynamically calculate image sizes within our Angular app. This approach runs a simple inline script in the SSR source code to calculate and insert images before the app takes over.

I'm afraid having an identical image URL in the SSR vs the CSR page still results in the same LCP problems as @alexkushnarov example perfectly illustrates.

tomandco avatar Mar 13 '21 11:03 tomandco

FYI https://www.loom.com/share/35330a858cd741ba92e8be0c0496ffbb

perjerz avatar Mar 15 '21 05:03 perjerz