Would it be worth adding an expanded 404 tracking system?

[chrisblakley]

Currently, Nebula bundles the Redirection plugin which does a great job of logging 404s with features for grouping them for display.

However, Nebula could either attempt to enhance this with its contextual data -or- Nebula could create its own output table if its detected 404s with this additional data.

Some data points that may be worth incorporating into 404 detection:

• Already provided by Redirection:
  • Request Path
  • User Agent
  • Timestamp
  • IP Address
  • Method
  • Domain
  • HTTP Response Code
  • Referrer
• What Nebula could enhance:
  • Device Type (mobile/desktop) (already discernible from UA, but could be more clear)
  • Operating System (already discernible from UA, but could be more clear)
  • Browser (already discernible from UA, but could be more clear)
  • Country (when enabled)
  • Region, City (when enabled)
  • Role (which includes bot label, although Nebula's 404 tracker currently does not track bot 404s, that could be changed)
  • Bot Identity (such as Googlebot, Bingbot, GPTbot, and others) (already discernible from UA, but could be more clear)
  • Could also provide Nebula's IP detection to ensure accuracy
  • Any other audience information such as notable visitors
  • Highlighting campaign traffic (more than just UTMs, use Nebula's Attribution functionaltiy) (already discernible from Source URL, but could be more clear)
  • Session LP
  • Session Referrer
  • Nebula Session ID
  • Google Analytics CID (to correlate to GA4 data)

I am currently not convinced this would be worth adding an expanded system into Nebula, but I do think it would be worth looking into adding some hooks into the Redirection plugin to see if it would be easy to add this contextual data to its output table.

[chrisblakley]

Redirection has the following filter where I may be able to inject Nebula data via the $details parameter.

Inside models/log/log-404.php

$insert = static::sanitize_create( $domain, $url, $ip, $details );
$insert = apply_filters( 'redirection_404_data', $insert );

Unfortunately, when I try to add a new key into the $insert array, it triggers an error because the column name doesn't exist in the database:

add_filter('redirection_404_data', function($insert){
	var_dump($insert);
	$insert['yolo'] = 'this is a test hello';
	return $insert;
});

WordPress database error Unknown column 'yolo' in 'field list' for query INSERT INTO `wp_redirection_404`

[chrisblakley]

If I expanded the Nebula 404 monitor, I could still use the transient method of storage and expand the array to include all of the various table contents. Then to output it, it would be as simple as reading from the transient and looping through to include data from the various rows. Pagination would be the hardest part- unless I read the entire transient and use DataTables to output... I'd prefer to use the Native WP table format and not read so much of the data at once... Like if there are thousands of 404s in a 24 hour period.