dbt-utils icon indicating copy to clipboard operation
dbt-utils copied to clipboard

get_url_parameter not taking the good parameter filled in url_parameter

Open etx121 opened this issue 8 months ago • 0 comments

Describe the bug

Let's say I have this URL in a column called url: https://example.com/?sku=EXAMPLE_SKU&u=https%3A%2F%2Fexample.com%2F

{{ dbt_utils.get_url_parameter(field='url', url_parameter='sku') }} extract the first parameter being LIKE %sku= {{ dbt_utils.get_url_parameter(field='url', url_parameter='ku') }} extract the first parameter being LIKE %ku= instead of finding a proper params with ku

Steps to reproduce

WITH
table_with_url AS (
    SELECT 'https://example.com/?sku=EXAMPLE_SKU&u=https%3A%2F%2Fexample.com%2F' AS url
),

final AS (
    SELECT
        url,
        {{ dbt_utils.get_url_parameter(field='url', url_parameter='sku') }} AS params_sku,
        {{ dbt_utils.get_url_parameter(field='url', url_parameter='ku') }} AS params_ku,
        {{ dbt_utils.get_url_parameter(field='url', url_parameter='u') }} AS params_u
    FROM table_with_url
)

SELECT * FROM final

Expected results

params_sku = EXAMPLE_SKU params_ku = NULL params_u = https%3A%2F%2Fexample.com%2F

Actual results

params_sku = EXAMPLE_SKU params_ku = EXAMPLE_SKU params_u = EXAMPLE_SKU

System information

The contents of your packages.yml file:

Which database are you using dbt with?

  • [x] postgres
  • [ ] redshift
  • [ ] bigquery
  • [ ] snowflake
  • [ ] other (specify: ____________)

The output of dbt --version:

Core:
  - installed: 1.9.3
  - latest:    1.9.4 - Update available!

  Your version of dbt-core is out of date!
  You can find instructions for upgrading here:
  https://docs.getdbt.com/docs/installation

Plugins:
  - postgres: 1.9.0 - Up to date!

Are you interested in contributing the fix?

I use this custom macro in postgres, between the time you fix the bug:

{% macro get_url_parameter(field, url_parameter) -%}
    {# Create a regex pattern that matches either ?param= or &param= followed by the value #}
    {%- set regex_pattern = "'[?&]" ~ url_parameter ~ "=([^&#]*)'" -%}
    
    {# PostgreSQL regexp_match returns an array, so we need to extract the first element #}
    NULLIF((regexp_match({{ field }}, {{ regex_pattern }}))[1], '')
{%- endmacro %}

etx121 avatar Apr 13 '25 13:04 etx121