prebid-server icon indicating copy to clipboard operation
prebid-server copied to clipboard

Rubicon: Update segtax logic

Open CTMBNara opened this issue 10 months ago • 10 comments

CTMBNara avatar Feb 18 '25 22:02 CTMBNara

Reverting reverted PR: https://github.com/prebid/prebid-server/pull/3915

CTMBNara avatar Feb 18 '25 22:02 CTMBNara

Code coverage summary

Note:

  • Prebid team doesn't anticipate tests covering code paths that might result in marshal and unmarshal errors
  • Coverage summary encompasses all commits leading up to the latest one, c6f652c8f2394171c9ab97434033cd98d1357ef1

rubicon

Refer here for heat map coverage report

github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:179:	appendTrackerToUrl			87.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:195:	Builder					100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:207:	updateRequestTo26			92.3%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:232:	MakeRequests				80.6%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:545:	createImpsToExtMap			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:564:	prepareImpsToExtMap			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:583:	splitMultiFormatImp			61.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:620:	resolveBidFloor				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:628:	updateImpRpTarget			77.8%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:715:	extractDfpAdUnitCode			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:725:	isNotKeyPathError			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:729:	addStringAttribute			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:733:	addStringArrayAttribute			0.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:737:	updateUserRpTargetWithFpdAttributes	70.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:767:	updateExtWithIabAndSegtaxAttribute	100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:781:	getTaxonomyIdToSegments			86.4%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:820:	pickRelevantSegments			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:853:	groupSegments				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:870:	populateFirstPartyDataAttributes	92.9%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:898:	isStringArray				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:908:	isBoolArray				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:918:	convertToStringArray			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:929:	rawJSONToMap				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:937:	mapFromRawJSON				80.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:946:	contains				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:955:	isVideo					100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:964:	isFullyPopulatedVideo			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:969:	resolveNativeObject			88.2%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1001:	setImpNative				82.4%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1030:	MakeBids				92.2%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1130:	mapImpIdToCpmOverride			90.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1148:	resolveAdm				87.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1163:	cmpOverrideFromBidRequest		100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1172:	updateBidExtWithMetaNetworkId		82.4%
total:									(statements)				84.8%

github-actions[bot] avatar Feb 18 '25 22:02 github-actions[bot]

This error again... But I found the reason: since relevantSegments is a hash-based map, there is no guarantee of the order of the keys, and because of this we return ext.rp.target.XXX entries in random order. And then we fail when comparing json (although in fact all the necessary elements are there, they are just not shown in the error trace for some reason)

CTMBNara avatar Feb 19 '25 00:02 CTMBNara

This error again... But I found the reason: since relevantSegments is a hash-based map, there is no guarantee of the order of the keys, and because of this we return ext.rp.target.XXX entries in random order. And then we fail when comparing json (although in fact all the necessary elements are there, they are just not shown in the error trace for some reason)

Looking at the failed test, isn't segmentId1 supposed to be in this block though? Screenshot 2025-02-20 at 9 54 08 AM

bsardo avatar Feb 20 '25 14:02 bsardo

@bsardo Yes, and it's there, but it's not shown in the diff for some reason. You can check it yourself if you try to log the outcoming request. It will contain all the values but in different order

CTMBNara avatar Feb 25 '25 10:02 CTMBNara

Code coverage summary

Note:

  • Prebid team doesn't anticipate tests covering code paths that might result in marshal and unmarshal errors
  • Coverage summary encompasses all commits leading up to the latest one, bc7e8a31b98adc25e48ee80f1cbd1877464f6236

rubicon

Refer here for heat map coverage report

github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:179:	appendTrackerToUrl			87.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:195:	Builder					100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:207:	updateRequestTo26			92.3%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:232:	MakeRequests				80.6%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:545:	createImpsToExtMap			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:564:	prepareImpsToExtMap			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:583:	splitMultiFormatImp			61.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:620:	resolveBidFloor				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:628:	updateImpRpTarget			77.8%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:715:	extractDfpAdUnitCode			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:725:	isNotKeyPathError			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:729:	addStringAttribute			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:733:	addStringArrayAttribute			0.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:737:	updateUserRpTargetWithFpdAttributes	70.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:767:	updateExtWithIabAndSegtaxAttribute	100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:781:	getTaxonomyIdToSegments			86.4%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:820:	pickRelevantSegments			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:853:	groupSegments				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:870:	populateFirstPartyDataAttributes	92.9%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:898:	isStringArray				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:908:	isBoolArray				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:918:	convertToStringArray			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:929:	rawJSONToMap				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:937:	mapFromRawJSON				80.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:946:	contains				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:955:	isVideo					100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:964:	isFullyPopulatedVideo			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:969:	resolveNativeObject			88.2%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1001:	setImpNative				82.4%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1030:	MakeBids				92.2%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1130:	mapImpIdToCpmOverride			90.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1148:	resolveAdm				87.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1163:	cmpOverrideFromBidRequest		100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1172:	updateBidExtWithMetaNetworkId		82.4%
total:									(statements)				84.8%

github-actions[bot] avatar Feb 26 '25 17:02 github-actions[bot]

Code coverage summary

Note:

  • Prebid team doesn't anticipate tests covering code paths that might result in marshal and unmarshal errors
  • Coverage summary encompasses all commits leading up to the latest one, 671adc79be613ec082b07be7adb4c11925c43f13

rubicon

Refer here for heat map coverage report

github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:174:	appendTrackerToUrl			87.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:190:	Builder					100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:202:	updateRequestTo26			92.3%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:227:	MakeRequests				80.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:529:	createImpsToExtMap			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:548:	prepareImpsToExtMap			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:567:	splitMultiFormatImp			61.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:604:	resolveBidFloor				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:612:	updateImpRpTarget			76.6%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:690:	extractDfpAdUnitCode			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:698:	isNotKeyPathError			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:702:	addStringAttribute			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:706:	addStringArrayAttribute			0.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:710:	updateUserRpTargetWithFpdAttributes	70.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:740:	updateExtWithIabAndSegtaxAttribute	100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:754:	getTaxonomyIdToSegments			86.4%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:793:	pickRelevantSegments			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:826:	groupSegments				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:843:	populateFirstPartyDataAttributes	92.9%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:871:	isStringArray				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:881:	isBoolArray				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:891:	convertToStringArray			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:902:	rawJSONToMap				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:910:	mapFromRawJSON				80.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:919:	contains				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:928:	isVideo					100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:937:	isFullyPopulatedVideo			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:942:	resolveNativeObject			88.2%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:974:	setImpNative				82.4%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1003:	MakeBids				92.2%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1103:	mapImpIdToCpmOverride			90.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1121:	resolveAdm				87.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1136:	cmpOverrideFromBidRequest		100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1145:	updateBidExtWithMetaNetworkId		82.4%
total:									(statements)				84.6%

github-actions[bot] avatar Feb 26 '25 17:02 github-actions[bot]

Code coverage summary

Note:

  • Prebid team doesn't anticipate tests covering code paths that might result in marshal and unmarshal errors
  • Coverage summary encompasses all commits leading up to the latest one, 922ec9ebf077759287dc16f1a61a0a6ac44c9a1a

rubicon

Refer here for heat map coverage report

github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:173:	appendTrackerToUrl			87.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:189:	Builder					100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:201:	updateRequestTo26			92.3%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:226:	MakeRequests				80.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:528:	createImpsToExtMap			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:547:	prepareImpsToExtMap			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:566:	splitMultiFormatImp			61.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:603:	resolveBidFloor				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:611:	updateImpRpTarget			76.6%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:689:	extractDfpAdUnitCode			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:697:	isNotKeyPathError			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:701:	addStringAttribute			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:705:	addStringArrayAttribute			0.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:709:	updateUserRpTargetWithFpdAttributes	70.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:739:	updateExtWithIabAndSegtaxAttribute	92.3%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:783:	resolveTaxName				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:791:	populateFirstPartyDataAttributes	92.9%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:819:	isStringArray				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:829:	isBoolArray				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:839:	convertToStringArray			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:850:	rawJSONToMap				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:858:	mapFromRawJSON				80.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:867:	contains				100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:876:	isVideo					100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:885:	isFullyPopulatedVideo			100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:890:	resolveNativeObject			88.2%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:922:	setImpNative				82.4%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:951:	MakeBids				92.2%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1051:	mapImpIdToCpmOverride			90.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1069:	resolveAdm				87.5%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1084:	cmpOverrideFromBidRequest		100.0%
github.com/prebid/prebid-server/v3/adapters/rubicon/rubicon.go:1093:	updateBidExtWithMetaNetworkId		82.4%
total:									(statements)				84.0%

github-actions[bot] avatar Feb 27 '25 17:02 github-actions[bot]

I talked to Bret and we agreed to simplify the logic. Related pr in Java: https://github.com/prebid/prebid-server-java/pull/3796

CTMBNara avatar Feb 27 '25 17:02 CTMBNara

@CTMBNara we'll give this another look this week.

bsardo avatar Mar 24 '25 03:03 bsardo

@CTMBNara sorry for the delay. We'll get to this again shortly.

bsardo avatar Apr 28 '25 02:04 bsardo