Detect Joomla

[jeckodevelopment]

Add Joomla to the Library Detector.

As requested in https://github.com/GoogleChrome/lighthouse-stack-packs/pull/44#issuecomment-630900983

[zero-24]

Also it might make sense to test on different frontend media / js files to detect joomla even when someone blocks access to all xml files.

[zero-24]

To be fair I'm not good with JS so please take that just as an example:

             try {
                const response_core_cms_version = await fetch('/administrator/manifest/files/joomla.xml');
                const response_4x_language_version = await fetch('/language/en-GB/joomla.xml');
                const response_3x_language_version = await fetch('/language/en-GB/en-GB.xml');
                const manifest_core_cms = await response_core_cms_version.text();        
                const manifest_4x_language = await response_4x_language_version.text();
                const manifest_3x_language = await response_3x_language_version.text();
                const parser = new DOMParser();
      
            let parsedManifest = null;
          // First check the core CMS version file
            if (manifest_core_cms) {
                    parsedManifest = parser.parseFromString(manifest_core_cms, "text/xml");
            }
          // Check for the 4.x language version file
                if (parsedManifest === NULL) {
                    parsedManifest = parser.parseFromString(manifest_4x_language, "text/xml");
            }
          // Check for the 3.x language version file
                if (parsedManifest === NULL) {
                    parsedManifest = parser.parseFromString(manifest_3x_language, "text/xml");
            }

            if (parsedManifest && !!parsedManifest.getElementsByTagName("version").length) {
                return  { version: parsedManifest.getElementsByTagName("version")[0].childNodes[0].nodeValue }
            } else if (generatorMeta || joomlaComponent) {
                return { version: UNKNOWN_VERSION };
            }
        }
        catch (err) {
            return false;
        }

Please test this against a site that blocks access to the administrator site, that blocks access to all XML files via htaccess and against a Joomla 4.x dev site to make sure the fallback checks work correctly.

cc @jeckodevelopment @housseindjirdeh

[housseindjirdeh]

Thanks a ton @zero-24, this is great :)

Simplified it a bit:

test: async function (win) {
  const generatorMeta = !!document.querySelector('meta[name="generator"][content*="Joomla"]');
  const joomlaComponent = !!document.querySelector('[href*="index.php?option=com_"]');

  // Mini polyfill for Promise.any: https://github.com/tc39/proposal-promise-any
  if (!Promise.any) {
      Promise.any = (promises) => {
          return new Promise((fulfill) => {
              promises.forEach((promise) => {
                  promise.then(fulfill, () => {});
              });
          });
      };
  }
  
  try {
      const versionFile = await Promise.any([
          fetch('/administrator/manifest/files/joomla.xml'), // core CMS version file
          fetch('/language/en-GB/joomla.xml'), // 4.x language version file
          fetch('/language/en-GB/en-GB.xml') // 3.x language version file
      ]);

      const manifest = await versionFile.text();
      const parser = new DOMParser();
      const parsedManifest = manifest ? parser.parseFromString(manifest, "text/xml") : null;

      if (parsedManifest && !!parsedManifest.getElementsByTagName("version").length) {
          return  { version: parsedManifest.getElementsByTagName("version")[0].childNodes[0].nodeValue }
      } else if (generatorMeta || joomlaComponent) {
          return { version: UNKNOWN_VERSION };
      }
  }
  catch (err) {
      return false;
  }

  return false;
}

Spot tests seem to work, but I would love if you could share some URLs for different scenarios (blocked access, etc...).

@jeckodevelopment Let me know if you're okay if I update the PR with these changes!

[zero-24]

Spot tests seem to work, but I would love if you could share some URLs for different scenarios (blocked access, etc...).

joomla.org allows you to access the frontend and backend xmls joomla.de does not allow the access to the backend xml

As for Joomla 4.x the easiest might be to setup a free site via https://launch.joomla.org (under Advanced Settings there is a 4.0.0-beta1-dev setting) and test against it as 4.x is still in dev there are no live sites i'm aware of.

PS I have to correct myself it is not joomla.xml in 4.x but it is langmetadata.xml

[jeckodevelopment]

@housseindjirdeh please feel free to update the PR! :)

[housseindjirdeh]

Apologies for leaving this open for so long folks. Things got side-tracked quite a bit, but sorry about that!

We realized that detections that make fetch requests can fail authentication in some sites (see #164). Also, it may be too costly to have run within Lighthouse 😞

@jeckodevelopment @zero-24 I hate to bring this back to the drawing board, but do you think there's a way we can change this to detect Joomla without making any requests for language or manifest files?

[zero-24]

Well technical you could try to check the html code i mean the gemerator tag but IIRC by default atleast the version is disabled.

The other way would be to check for some paths that are to my knowledge unique for joomla. But all of this will usually not get you the version.

[zero-24]

Based on the WP detection one for joomla could look like this:

    'Joomla': {
        id: 'joomla',
        icon: 'joomla',
        url: 'https://joomla.org/',
        npm: null,
        test: function (win) {
            // You can disable the generator tag as well as the version from the backend
            const generatorMeta = document.querySelector('meta[name=generator][content^="Joomla"]');
            const version = generatorMeta ? generatorMeta.getAttribute("content").replace(/^\w+\s/,'') : UNKNOWN_VERSION;
            return { version };

            // Check whether the Joomla JS is there.
            if (win.Joomla) {
                return { version: UNKNOWN_VERSION };
            }

            // This is the path to the joomla core bootstrap but sites are not required to load that file but could also load a different version
            const hasJoomlaBootstrap = !!document.querySelectorAll('script[src*="/media/jui/js/bootstrap.min.js"]').length;

            if (hasJoomlaBootstrap) return true;

            return false;
        }
    },

I hope that helps @housseindjirdeh

[zero-24]

But both tests (generator and bootstrap) are not that deep and can both be fooled. For example; joomla.de would fail that tests.

I have just extended that test to check for win.Joomla can you test that detection script with your setup on joomla.org and joomla.de @housseindjirdeh ?

[housseindjirdeh]

Genuinely can't thank you enough @zero-24, and again - sorry for all the delays on my part.

The new detection you've outlined looks great, and mostly works (even on .be and .de sites). However:

• The version Joomla! - Open Source Content Management is being picked up on some sites through the content attribute in the meta tag. Maybe we can map this to a specific version where this string is used?
• The check for generatorMeta will always return true the way it's currently set up. Updating the snippet slightly:

    'Joomla': {
        id: 'joomla',
        icon: 'joomla',
        url: 'https://www.joomla.org/',
        npm: null,
        test: function (win) {
            // You can disable the generator tag as well as the version from the backend
            const generatorMeta = document.querySelector('meta[name=generator][content^="Joomla"]');
            
            // This is the path to the joomla core bootstrap but sites are not required to load that file but could also load a different version
            const hasJoomlaBootstrap = !!document.querySelectorAll('script[src*="/media/jui/js/bootstrap.min.js"]').length;
            
            if (generatorMeta) {
                return { version: generatorMeta.getAttribute("content").replace(/^\w+\s/,'') };
            } else if (win.Joomla || hasJoomlaBootstrap) {
                return { version: UNKNOWN_VERSION };
            }
            
            return false;
        }
    }, 

Here's an updated version that cleans up a bit:

[housseindjirdeh]

@zero-24 I've merged in base-level detection using your snippet #188.

Let's work on improving it in future versions, but I want to get at least this merged in in time for Lighthouse's next release (not fair to keep you waiting much longer 😅 )

[housseindjirdeh]

Closing this since we have #188