Passing userIds to Prebid Server using AMP

[ian-lr]

It is not currently possible to pass userIds (e.g. user.ext.eids) to Prebid Server via AMP. While cookie-based syncs are enabled with the amp-iframe, this limits the information available to bidders processing requests from AMP pages. Given the growing popularity of both userIds and AMP, this seems like a good opportunity to extend Prebid Server capability.

Acceptance Criteria

• Publishers should be able to pass known userIds to supported bidders via a well-defined interface in amp-ad.
• Bidders should be able to process userIds sent from AMP pages in a standard eid format.

Note, as AMP-RTC does not support eids , userIDs will likely need to be added to the query string or through some other Prebid-specific interface. A potential strawman follows.

<amp-iframe width="1" title="User Sync with eids"
  height="1"
  sandbox="allow-scripts"
  frameborder="0"
  src="https://<PBSERVER_DOMAIN>/load-cookie.html?endpoint=appnexus&max_sync_count=5&eids=%5B%7B%22source%22%3A%22liveramp.com%22%2C%22uids%22%3A%5B%7B%22id%22%3A%22AovIJXGIWHHMhHyOeQiDk0_rtTQ--fVkmWU7xftkAh9rxgUeLHBcsoUE6gdZwFFYmvAJXw%22%2C%22atype%22%3A1%7D%5D%7D%5D">
  <amp-img layout="fill" src="data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACH5BAEAAAAALAAAAAABAAEAAAICRAEAOw==" placeholder></amp-img>
</amp-iframe>

[bretg]

I'd rather not store the eids in the PBS cookie. 1) that cookie can already get big. 2) more privacy headaches

How about we just define a new macro USER_IDS in the RTC vendors:

  prebidrubicon: {
    url:
      'https://prebid-server.rubiconproject.com/openrtb2/amp?tag_id=REQUEST_ID&w=ATTR(width)&h=ATTR(height)&ow=ATTR(data-override-width)&oh=ATTR(data-override-height)&ms=ATTR(data-multi-size)&slot=ATTR(data-slot)&targeting=TGT&curl=CANONICAL_URL&timeout=TIMEOUT&adc=ADCID&purl=HREF&gdpr_consent=CONSENT_STRING&account=ACCOUNT_ID&userids=USER_IDS',
    macros: ['REQUEST_ID', 'CONSENT_STRING', 'ACCOUNT_ID', 'USER_IDS'],
    disableKeyAppend: true,
  },

Then the publisher would be responsible for passing the eids array in:

  <amp-ad width="300" height="50"
    type="doubleclick"
    data-slot="/11111/amp_test"
    data-multi-size-validation="false"
    rtc-config='{"vendors": {"prebidrubicon": {"REQUEST_ID": "14062-amp-AMP_Test-300x250"}, "ACCOUNT_ID": "1001", USER_IDS="%5B%7B%22source%22%3A%22id5-sync.com%22%2C%22uids%22%3A%5B%7B%22id%22%3A%22ID5-12345%22%7D%5D%7D%2C%7B%22source%22%3A%22netid.de%22%2C%22uids%22%3A%5B%7B%22id%22%3A%2211111111%22%7D%5D%7D%5D"}}'
    json='{ "targeting": {"site": {"tags": "autoestima","url": "/amp/familia/materias/33559-princesa-africana-da-disney-lembra-por-que-toda-crianca-precisa-se-sentir-representada"}}}' >
  </amp-ad>

We could come up with a short-hand for this JSON, but would rather not PBS have to understand it, and there are exceptions to the general rule of source+id. e.g. TDID has an ext, as will SharedId.

[ian-lr]

@bretg I like that approach better! I wasn't sure how much flexibility we have on the vendor macros, but this is more approachable and simpler from my perspective.

[SyntaxNode]

Discussed and accepted by Prebid Server Committee.

We'll just need to work out the JSON format. To keep things simple, does Base64 encoded JSON work for you? PBS would decode the Base64 was representation and parse it as eids. If there are no parse errors, it will be set as-is in the request. If there is a parse error, should this be an error or a warning?

[ian-lr]

@SyntaxNode From my perspective, that would work well. I would suggest throwing an error unless there is precedent (other macro validation?) to do things differently. In most cases, I'd expect PBS would expect a valid user ID string.

[SyntaxNode]

We need to settle on the eid format to begin implementing. Is a base64 encoded json blob alright? Do we want to go with escaped json as you originally included? Did you @bretg want to suggest a short hand encoding?

[ian-lr]

Base64 JSON blob works for me. To illustrate, something like:

let encodedEids = btoa(JSON.stringify(pbjs.getUserIdsAsEids())) will be sent through, then PBS will do the equivalent of JSON.parse(atob(encodedEids))?

[pycnvr]

Would this approach work with cached pages? It seems a customized page has to generated per user, but cache on google and bing serve the same copy.

[ian-lr]

@pycnvr Good question. Could you use something like https://amp.dev/documentation/components/amp-access/ to get user-specific data without invalidating the cache?

[pycnvr]

@ian-lr Possibly. Not sure how to link dynamic values to rtc-config, though. The list of available macros is controlled by doubleclick rtc.

[ian-lr]

@pycnvr OK, let me see if I can experiment a bit with this and see if I can get the cache working, too.

[SyntaxNode]

@pycnvr OK, let me see if I can experiment a bit with this and see if I can get the cache working, too.

Sounds good. We'll hold off implementing for now.

[bretg]

We don't need to use macros. We can evolve the First Party Data feature to support eids similar to the way proposed for the Publisher-Provided User ID feature in https://github.com/prebid/Prebid.js/issues/5690

[deleted obsolete straw example. see below for the most recent proposal]

[pycnvr]

@bretg As far as I can tell, AMP is not making this easy. When a user does a google search, and click on one of the AMP links, it's actually a page cached by google. So the publisher doesn't even have a chance to fill in the first party data. The properties in amp-ad are static, except for those that can be supplied via macros.

For example, the following are the same page but served from different domains.

1. Google Cache from cdn.ampproject.org https://www-si-com.cdn.ampproject.org/v/s/www.si.com/.amp/soccer/2020/09/10/lionel-messi-cristiano-ronaldo-lead-fifa-21-player-rankings?usqp=mq331AQFKAGwASA=&amp_js_v=0.1

2. Publisher si.com https://www.si.com/.amp/soccer/2020/09/10/lionel-messi-cristiano-ronaldo-lead-fifa-21-player-rankings?usqp=mq331AQFKAGwASA=&amp_js_v=0.1

[bretg]

@pycnvr - can arbitrary javascript run on an AMP page? Does it have DOM access to tags?

If so, could that javascript scan the tags and add/update data like either the 'rtc-config' or the 'json'?

We haven't finalized how we're going to pass eids to Prebid Server -- anyone have thoughts about whether that's easier either way?

[bretg]

Nevermind. I found the reference that <amp-script> tags can't currently create <amp-ad> tags. https://amp.dev/documentation/components/amp-script/

[ian-lr]

@bretg I found this AMP Issue tagged with INTENT TO IMPLEMENT that, depending on the implementation, could be useful: https://github.com/ampproject/amphtml/issues/28095

[bretg]

That's interesting @ian-lr , though I'm not sure how helpful it will be if it's tied to Permutive. No one seems to have pushed back on a vendor-specific tag.

[adamleslie]

@bretg non-Permutive specific implementation:

https://github.com/ampproject/amphtml/issues/30193

LMK if we can help test a PBS execution on our domains once the above is built

[SyntaxNode]

@bretg @adamleslie Is this spec complete / ready to implement?

[bretg]

AMP 30193 is still in the proposal stage. Could take a while for them to implement. My understanding is that it would allow dynamic 'json' targeting to be applied to amp-ad RTC. (corrections welcome)

In the meantime, we can discuss how to pass that data through to Prebid Server. If it's on the json field, I'd propose we come up with a way to pass an eids structure through along with data permissioning as described in https://github.com/prebid/Prebid.js/issues/5814

Here's a straw:

  <amp-ad width="300" height="50"
    type="doubleclick"
    data-slot="/11111/amp_test"
    data-multi-size-validation="false"
    rtc-config='{"vendors": {"prebidrubicon": {"REQUEST_ID": "14062-amp-AMP_Test-300x250"}, "ACCOUNT_ID": "1001"}}'
    json='{"targeting":{"eids":[{"source": "example.com", "uids": [{"id": "11111111"}]}], "eidPermissions": [{"source": "example.com", "bidders": ["bidderA"]}]}}' >
  </amp-ad>

Prebid Server would look in the targeting field for "eids", validate it, and inject into user.ext.eids. Assuming an EIDs permissioning scheme is approved, it would also handle that.

[ian-lr]

@bretg Returning back to this, do you think that we should hold off until the eid permissioning is locked in before proceeding with the suggested straw here?

[bretg]

AMP has a feature now that allows RTC to call a script. But I'll have to admit I'm not connecting the dots for how to integrate the two pieces. Here's the example on the AMP page

    <amp-ad width="320" height="50"
            type="doubleclick"
            data-slot="/4119129/mobile_ad_banner"
            rtc-config='{"urls": ["amp-script:targetingFns.getTargeting"]}'
    </amp-ad>
    <amp-script nodom data-ampdevmode id="targetingFns" script="targetingFnsScript"></amp-script>
    <script id="targetingFnsScript" type="text/plain" target="amp-script">
      exportFunction("getTargeting", () => {
        return {
          targeting: {food: ["chicken", "beans"]},
          categoryExclusions: ["sports", "food", "fun"]
        };
      });
    </script>

From this example it's not clear that we can use the standard vendors in rtc-config. (?) But this example seems like a high level "idea" because the getTargeting function doesn't actually return a URL.

I'm not particularly fond of a making pubs hardcode the PBS URLs in their pages because we change the vendor config sometimes. e.g. we recently added AMP consent fields.

Maybe one of the Prebid AMP experts can weigh in on the possibilities here?

[philipwatson]

I'm currently experimenting with this feature (though I'm not an AMP expert). You can use the standard vendors in the rtc-config: simply by adding the "vendors" field. eg:

rtc-config='{"vendors": {"prebidappnexus": {"PLACEMENT_ID": "13144370"}}, "urls": ["amp-script:targetingFns.getTargeting"], "timeoutMillis":1000}'

The script RTC callouts don't return a url. They return JSON containing the key/values. So for doubleclick's AMP-AD, we want to return the same structure as your example (he same as prebid-server's RTC response).

Not sure if you can configure a vendor in rtc-config to be a script RTC (in AMP's callout-vendors.js).

However, you can use an external script to reduce publisher maintenance: <amp-script nodom id="targetingFnsScript" src="https://example.com/script.js" data-ampdevmode></amp-script>

As far as I know, you can't use the MACROS with scripts. Instead, the script will get dynamic values from amp-analytics.

{
    "transport": {"amp-script": true},
    "requests": {
      "pageview": "amp-script:targetingFns.receiveFn"
    },
    "vars": {
      "clientId": "CLIENT_ID(ampId)",
    },
    "triggers": {
      "trackPageview": {
        "on": "visible",
        "request": "pageview"
      }
    },
    "extraUrlParams": {
      "cid": "${clientId}",
      "canonicalUrl": "${canonicalUrl}"
    },
...

And in your script:

exportFunction("receiveFn", (msgObj) => {...});

The msgObj will have the contents of extraUrlParams.

Not sure about consent unfortunately.. but you can get other bits of info. Documented here.

Not sure how to get the EIDS. The script runs in webworker (and inside another iframe) so it has no access to local storage. So might only be limited to TP cookie.

Also RTC callouts happen at the same time. Not processed/merged sequentially. So prebid-server RTC callout can't get results from another callout. Or you guys thinking about changing PBS callout to script callout?

Hope this info helps a bit.

[bretg]

Thanks @philipwatson , but I'm not following.

you guys thinking about changing PBS callout to script callout

We don't care what the AMP syntax is. Publishers that want to pass dynamic data (like user IDs) can use a different syntax if needed. The original use case can stay with the currently documented 'vendors' approach. We'll document whatever other scenarios are necessary.

Not sure how to get the EIDS

In order to be useful for Prebid Server, the requirement is straightforward: some kind of AMP setup that gathers the desired dynamic data and passes it through to the Prebid Server /amp endpoint.

script RTC callouts

What is this -- is it ["amp-script:targetingFns.getTargeting"]?

You also say "The script RTC callouts don't return a url.", but why then is ["amp-script: in the 'url' section of rtc-config?

[bretg]

Discussed in committee today. It does seem possible to thread the need here, but that may not be the most valuable thing to do here.

Assuming that a url protocol of amp-script: tricks the system into calling a local script rather than making an HTTP call, here's a general approach that might be made to work to get dynamic values from an AMP page into PBS:

    <amp-ad width="320" height="50"
            type="doubleclick"
            data-slot="/4119129/mobile_ad_banner"
            rtc-config='{"urls": ["amp-script:targetingFns.getTargeting"]}'
    </amp-ad>
    <amp-script nodom data-ampdevmode id="targetingFns" script="targetingFnsScript"></amp-script>
    <script id="targetingFnsScript" type="text/plain" target="amp-script">
      exportFunction("getTargeting", () => {
            // grab dynamic values off page, add them to a PBS URL
            // use XHR to call the PBS URL
            // return the response so the amp-ad block can add targeting values to the ad server call
      });
    </script>

The only things we'd need to do are:

• define a new parameter for IDs. I'd go for something simple like &eids=URL-ENCODED-EIDS-BLOCK. Then PBS would merge the value of this param into user.ext.eids.
• document the approach

FWIW, I couldn't get this approach to work, but didn't spend much time with it. If someone has an example they could post here, that would be great.

Alternate approach

But in the meeting, an alternate approach was proposed: use a new host company cookie to store the eids block.

1. define a new PBS ID cookie. e.g. 'eids'

2. update the /setuid endpoint to accept an new 'eid' parameter

   /setuid?eid=URL-ENCODED-EIDS-ENTRY

3. when /setuid sees this parameter, it updates the 'eids' cookie with the additional value

4. when /auction or /amp are called, the eids cookie is parsed and merged into user.data.eids

5. Customize load-cookie.html to call an ID endpoint and parse the response into the new /setuid?eid (lots of details here to work out)

Any thoughts about either of these approaches?

[philipwatson]

Hi @bretg Yes, the "script" RTC callout is ["amp-script:targetingFns.getTargeting"]. Regarding the url: I mean the function that this RTC points to (getTargeting) doesn't return a URL. Sorry, I think I misunderstood what you said.

Not sure if readers are aware, but getTargeting can return a Promise. Obviously needed because of the PBS request. Also needed if using amp-analytics to receive dynamic values - because the receiveFn (using my example) and getTargeting can be called in any order. Though this could be dependent on the trigger/event being used. Note: I found out recently that you can get consent string via amp-analytics.

Might have time to create an example later today or tomorrow.

[SyntaxNode]

@philipwatson I think an example would be great, when your time allows. Thank you.

[philipwatson]

Hi @SyntaxNode Here is my example.

But there is a problem: AMP does not support query params on the script URLs. This is needed to pass in placement-level parameters placementId, slot, width, height, timeout, etc. So this example is only good if you have one placement.

I added a feature request for this on the amphtml project: https://togithub.com/ampproject/amphtml/issues/35097

<!doctype html>
<html amp lang="en">
    <meta charset="utf-8">
    <script async src="https://cdn.ampproject.org/v0.js"></script>
    <script async custom-element="amp-analytics" src="https://cdn.ampproject.org/v0/amp-analytics-0.1.js"></script>
    <script async custom-element="amp-ad" src="https://cdn.ampproject.org/v0/amp-ad-0.1.js"></script>
    <script async custom-element="amp-script" src="https://cdn.ampproject.org/v0/amp-script-0.1.js"></script>
    <title>Hello AMP</title>
    <link rel="canonical" href="https://amp.dev/documentation/guides-and-tutorials/start/create/basic_markup/">
    <meta name="viewport" content="width=device-width,minimum-scale=1,initial-scale=1">
    <style amp-boilerplate>body{-webkit-animation:-amp-start 8s steps(1,end) 0s 1 normal both;-moz-animation:-amp-start 8s steps(1,end) 0s 1 normal both;-ms-animation:-amp-start 8s steps(1,end) 0s 1 normal both;animation:-amp-start 8s steps(1,end) 0s 1 normal both}@-webkit-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@-moz-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@-ms-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@-o-keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}@keyframes -amp-start{from{visibility:hidden}to{visibility:visible}}</style><noscript><style amp-boilerplate>body{-webkit-animation:none;-moz-animation:none;-ms-animation:none;animation:none}</style></noscript>
</head>
<body>
<h1>Test</h1>

<amp-analytics>
    <script type="application/json">
        {
            "transport": {
                "amp-script": true
            },
            "requests": {
                "pageview": "amp-script:targetingFns.receive"
            },
            "vars": {
                "clientId": "CLIENT_ID(foo,,foo,false)"
            },
            "triggers": {
                "trackPageview": {
                    "on": "visible",
                    "request": "pageview"
                }
            },
            "extraUrlParams": {
                "accountId": "12345",
                "consentString": "CONSENT_STRING",
                "curl": "${canonicalUrl}",
                "purl": "${ampdocUrl}",
                "gdprApplies": "CONSENT_METADATA(gdprApplies)",

                "placementId": "13144370",
                "slot": "/19968336/universal_creative",
                "timeout": "1000",
                "width": "300",
                "height": "250"
            }
        }
    </script>
</amp-analytics>


<amp-script nodom data-ampdevmode id="targetingFns" script="targetingFnsScript"></amp-script>
<script id="targetingFnsScript" type="text/plain" target="amp-script">
var receivedParams;

function getTargetingFromPBS(params) {
  var url = "https://prebid.adnxs.com/pbs/v1/openrtb2/amp?";
  url += "tag_id=" + params.placementId;
  url += "&timeout=" + params.timeout;
  url += "&w=" + params.width;
  url += "&h=" + params.height;
  url += "&slot=" + encodeURIComponent(params.slot);
  url += "&curl=" + encodeURIComponent(params.curl);
  url += "&purl=" + encodeURIComponent(params.purl);
  url += "&gdpr_consent=" + params.consentString;

  return fetch(url, {method: 'GET'})
    .then(function (response) {
      return response.json();
    })
}

exportFunction('receive', function(msg) {
  receivedParams = msg;
});

exportFunction('getTargeting', function() {
  return new Promise(function (res, rej) {
    var start = new Date().getTime();
    function tryResolve() {
      if (receivedParams) {
        getTargetingFromPBS(receivedParams)
          .then(function(json) {
            res(json);
          });
      }
      else if (Date.now() - start < 1000) {
        setTimeout(function () {
          tryResolve();
        }, 250);
      }
    }
    tryResolve();
  });
});
</script>
<amp-ad width="300" height="250"
        type="doubleclick"
        data-slot="/19968336/universal_creative"
        rtc-config='{"urls": ["amp-script:targetingFns.getTargeting"], "timeoutMillis": 1000}'>
</amp-ad>
</body>
</html>

Note: can generate the amp-analytics config on the server-side. Adds other possibilities like adding TP cookie values. From the docs:

<amp-analytics
  config="https://example.com/analytics.account.config.json"
></amp-analytics>

[SyntaxNode]

Thank you @philipwatson. To be clear, this example depends on AMP implementing this feature request?

[philipwatson]

Hi @SyntaxNode
The example works as-is. But we run into trouble if we want to support multiple ads on the page. There are "hacky" workarounds.

It only depends on it if we want a nice, harmonious solution. This is where we need the AMP implementation.

An alternative workaround, as mentioned on that feature request, is having a separate analytics config for each . This does not feel good.

Another possible workaround that came to mind recently is associating the function name with the subset of configs. Not perfect but feels better.

For example:

<amp-ad width="300" height="250"
        type="doubleclick"
        data-slot="/19968336/universal_creative"
        rtc-config='{"urls": ["amp-script:targetingFns.getTargeting1"], "timeoutMillis": 1000}'>
</amp-ad>
<amp-ad width="320" height="50"
        type="doubleclick"
        data-slot="/19968336/universal_creative"
        rtc-config='{"urls": ["amp-script:targetingFns.getTargeting2"], "timeoutMillis": 500}'>
</amp-ad>

And in analytics configs:

"extraUrlParams": {
  ...
  "placementId1": "13144370",
  "slot1": "/19968336/universal_creative",
  "timeout1": "1000",
  "width1": "300",
  "height1": "250",

  "placementId2": "98383113",
  "slot2": "/19968336/universal_creative",
  "timeout2": "500",
  "width2": "320",
  "height2": "50"
}

And in the script:

exportFunction('getTargeting1', function() {
    // know we want to use slot1, placement1, etc
});
exportFunction('getTargeting2', function() {
    // know we want to use slot2, placement2, etc
});

I tried this and it works.

We also have to keep in mind that the script runs inside a web worker. So we won't have access to FP storage (window.localStorage, document.cookie). We would always have to rely on making successful calls to the external providers on every ad request. However, if I read it right, Google may make the storage api less restrictive - by allowing 3p scripts access to the storage API provided that the page is delivered from the publisher's domain - not from cache or AMP Viewer (google.com). I2I here: https://togithub.com/ampproject/amphtml/issues/30872