Statistics Page: URL leads to error on INT

[sosiology]

We recently introduced a link to the statistics page in the footer. The link leads to a 505 page To Reproduce Steps to reproduce the behavior:

1. Go to 'int.visualize.admin.ch'
2. Click on 'Statistics`in the footer (https://int.visualize.admin.ch/statistics)
3. 4. See error.

Expected behavior The Stats page shows up

Screenshots or video

Environment (please complete the following information):

• Visualize environment and version: INT v5.4.1 (6adc945)

Additional context

• Link works on TEST and PROD

[adintegra]

Sentry Issue: VISUALIZE-ADMIN-ES

[adintegra]

Status update:

Found and removed 48 malformed records in config table on INT.

-- Cleanup query
WITH filtered_data AS (
  SELECT id,
    jsonb_agg(
      COALESCE(
        data ->> 'dataSet',
        chart_config_obj ->> 'dataSet',
        cubes_obj ->> 'iri'
      )
    ) AS iris,
    COALESCE(
      jsonb_agg(chart_config_array ->> 'chartType') FILTER (WHERE chart_config_array ->> 'chartType' IS NOT NULL),
      jsonb_build_array(chart_config_obj ->> 'chartType')
    ) AS chart_types
  FROM config
  LEFT JOIN LATERAL jsonb_array_elements(data -> 'chartConfigs') AS chart_config_array ON true
  LEFT JOIN LATERAL jsonb_array_elements(chart_config_array -> 'cubes') AS cubes_obj ON true
  LEFT JOIN LATERAL (SELECT data -> 'chartConfig' AS chart_config_obj) AS single_config ON true
  GROUP BY id, data, chart_config_obj
)
DELETE FROM config
WHERE id in
  (SELECT id
  FROM filtered_data
  WHERE NOT EXISTS (
    SELECT 1
    FROM jsonb_array_elements_text(iris) AS iri
    WHERE (iri LIKE 'http%://%')
  )
)

[adintegra]

As a result of security testing conducted in March 2020, the database on INT was populated with ~2500 rows of malformed or dummy charts. This data either breaks the statistics page completely or skews the reporting. Some of the records have been removed manually following a conservative approach:

• Manually removed chart ids 27 through 99 and 972 through 1056 (inclusive)
• From timestamps and configuration content, it looks as though ids 24-2657 could safely be removed as well

[adintegra]

Cleanup of INT completed.

delete
from config
where id BETWEEN 25 and 2654;