Add functional tests for aligning TCF and tests for support transmitEids activity controls #2893

osulzhenko · 2024-01-09T17:12:32Z

No description provided.

…add tests for transmitEids activities

…ing-TCF-and-Activity-Controls

marki1an · 2024-02-22T14:24:29Z

src/test/groovy/org/prebid/server/functional/model/config/PurposeEid.groovy

+@ToString(includeNames = true, ignoreNulls = true)
+@JsonNaming(PropertyNamingStrategies.SnakeCaseStrategy)
+class PurposeEid {
+


Probably missing this configuration prebid/prebid-server#2904.

link for the general page. But config is:

privacy.gdpr.purposes.p4.eid.require_consent: true/false privacy.gdpr.purposes.p4.eid.exceptions: ["pubcid.org"]

Did I miss something?

I suppose this one privacy.gdpr.purposes.p4.eid.activity_transition

@marki1an added, thanks for notes

marki1an · 2024-02-22T15:16:23Z

...org/prebid/server/functional/tests/privacy/TcfBasicTransmitUfpdAligningActivitiesSpec.groovy

+
+        then: "Generic bidder request should have data in Eid field"
+        def bidderRequest = bidder.getBidderRequest(bidRequest.id)
+        assert bidderRequest.user.eids == userEids


AFAIK we should check FPD fields, not a Eids

updated to Eids tests as the main point for this feature, let me know if ufpd test required

marki1an · 2024-02-22T15:35:31Z

src/test/groovy/org/prebid/server/functional/model/privacy/EnforcementRequirments.groovy

@@ -0,0 +1,23 @@
+package org.prebid.server.functional.model.privacy
+
+import com.iabtcf.v2.PublisherRestriction


Unused import

marki1an · 2024-02-22T15:45:16Z

src/test/groovy/org/prebid/server/functional/model/privacy/EnforcementRequirments.groovy

+    List<BidderName> vendorExceptions
+    List<BidderName> softVendorExceptions
+    Purpose purposesLITransparency
+    List<RestrictionType> restrictionType


Please use your own RestroctionType

I don't see the flexiblePurposes, legIntPurposes, publisherRestrictions, fields

good point, updated

flexiblePurposes, legIntPurposes, publisherRestrictions are completely included in the GVL files for TcfFullTransmitEidsAligningActivitiesSpec. Creating separate files for each purpose would be unnecessarily resource-intensive for software systems

marki1an · 2024-02-22T16:36:27Z

.../org/prebid/server/functional/tests/privacy/TcfFullTransmitUfpdAligningActivitiesSpec.groovy

+        def purposes = TcfUtils.getPurposeConfigsForPersonalizedAds(enforcementRequirments, true)
+        def accountGdprConfig = new AccountGdprConfig(purposes: purposes)
+        def activity = Activity.getDefaultActivity([ActivityRule.getDefaultActivityRule(Condition.baseCondition, true)])
+        def account = getAccountWithGdpr(bidRequest.accountId, accountGdprConfig).tap {
+            config.privacy.allowActivities = AllowActivities.getDefaultAllowActivities(TRANSMIT_UFPD, activity)
+        }
+        accountDao.save(account)
+
+        when: "PBS processes auction requests"
+        privacyPbsServiceWithMultipleGvl.sendAuctionRequest(bidRequest)
+
+        then: "Generic bidder request should have data in Eid field"
+        def bidderRequest = bidder.getBidderRequest(bidRequest.id)
+        assert bidderRequest.user.eids == userEids
+
+        where:
+        enforcementRequirments << getLegalEnforcementRequirments(P4) + getCompanyEnforcementRequirments(P4)


Minor, typo requierments

You try to populate as much as you can, but the method getPurposeConfigsForPersonalizedAds() doesn't take a list of enforcementRequirements

Updated typo, and we don't put a list of enforcementRequirements for getPurposeConfigsForPersonalizedAds. It simply creates a list concatenation by the + operator and then appends it to enforcement requirements by the “<<” operator. That is why we have all requirements separately

marki1an · 2024-02-22T16:38:47Z

.../org/prebid/server/functional/tests/privacy/TcfFullTransmitUfpdAligningActivitiesSpec.groovy

+class TcfFullTransmitUfpdAligningActivitiesSpec extends PrivacyBaseSpec {
+
+    @Shared
+    static final int PURPOSES_ONLY_GVL_VERSION = PBSUtils.getRandomNumber(0, 4095)


Please place private

…ing-TCF-and-Activity-Controls

marki1an · 2024-02-27T10:50:26Z

...org/prebid/server/functional/tests/privacy/TcfBasicTransmitEidsAligningActivitiesSpec.groovy

+import static org.prebid.server.functional.model.request.auction.TraceLevel.VERBOSE
+import static org.prebid.server.functional.util.privacy.TcfConsent.GENERIC_VENDOR_ID
+
+class TcfBasicTransmitEidsAligningActivitiesSpec extends PrivacyBaseSpec {


TcfBasicTransmitEidsActivitiesSpec

marki1an · 2024-02-27T10:53:15Z

...org/prebid/server/functional/tests/privacy/TcfBasicTransmitEidsAligningActivitiesSpec.groovy

+        assert bidderRequest.user.eids == userEids
+
+        where:
+        enforcementRequirments << getCompanyBasedEnforcementRequirments(P4) +


Type: getCompanyBasedEnforcementRequirements

marki1an · 2024-02-27T10:55:25Z

src/test/groovy/org/prebid/server/functional/model/privacy/EnforcementRequirements.groovy

+import org.prebid.server.functional.util.privacy.TcfConsent
+
+@ToString(includeNames = true, ignoreNulls = true)
+class EnforcementRequirements {


Better naming for me: EnforcementRequirement

...org/prebid/server/functional/tests/privacy/TcfBasicTransmitEidsAligningActivitiesSpec.groovy

marki1an · 2024-02-27T11:59:19Z

src/test/groovy/org/prebid/server/functional/util/privacy/TcfUtils.groovy

+    static Map<Purpose, PurposeConfig> getPurposeConfigsForPersonalizedAds(EnforcementRequirements enforcementRequirments, boolean requireConsent = false, List<String> eidsExceptions = []) {
+        def purpose = enforcementRequirments.purposeConsent ?: enforcementRequirments.purpose
+        // Basic Ads required for any bidder call, should be present at least as company consent
+        Map<Purpose, PurposeConfig> purposes = [(Purpose.P2): new PurposeConfig(enforcePurpose: NO, enforceVendors: false)]


Minor: please replace Map on def

marki1an · 2024-02-27T12:16:32Z

.../org/prebid/server/functional/tests/privacy/TcfFullTransmitEidsAligningActivitiesSpec.groovy

+import static org.prebid.server.functional.util.privacy.TcfConsent.RestrictionType.UNDEFINED
+import static org.prebid.server.functional.util.privacy.TcfConsent.TcfPolicyVersion.TCF_POLICY_V2
+
+class TcfFullTransmitEidsAligningActivitiesSpec extends PrivacyBaseSpec {


TcfFullTransmitEidsActivitiesSpec

marki1an · 2024-02-27T12:23:59Z

...org/prebid/server/functional/tests/privacy/TcfBasicTransmitEidsAligningActivitiesSpec.groovy

+
+    private final PrebidServerService activityPbsServiceExcludeGvl = pbsServiceFactory.getService(PBS_CONFIG)
+
+    private final PrebidServerService activityPbsServiceExcludeGvlWithElderOrtb = pbsServiceFactory.getService(PBS_CONFIG + ["adapters.generic.ortb-version": "2.5"])


In my opinion, we can omit elder ORTB.

We have tests for it, and privacy doesn't affect these fields.

This is legacy behaviour and we should not care about it.
@Net-burst What do you think about it?

I haven't reviewed the whole PR yet, but I'd leave it be. Because otherwise, we won't be able to tell for sure whether masking happens before or after the normalization. But maybe it would be worth moving those tests to a separate location.

marki1an · 2024-02-27T12:26:19Z

.../org/prebid/server/functional/tests/privacy/TcfFullTransmitEidsAligningActivitiesSpec.groovy

+    private static String getVendorListPath(Integer gvlVersion) {
+        "/app/prebid-server/data/vendorlist-v${TCF_POLICY_V2.vendorListVersion}/${gvlVersion}.json"
+    }


Better to move in PrivacyBaseSpec

marki1an · 2024-02-28T10:52:47Z

.../groovy/org/prebid/server/functional/tests/privacy/TcfBasicTransmitEidsActivitiesSpec.groovy

+import org.prebid.server.functional.model.request.auction.Eid
+import org.prebid.server.functional.service.PrebidServerService
+import org.prebid.server.functional.util.privacy.TcfUtils
+import spock.lang.IgnoreRest


used import

marki1an · 2024-02-28T11:36:15Z

.../groovy/org/prebid/server/functional/tests/privacy/TcfBasicTransmitEidsActivitiesSpec.groovy

+
+import static org.prebid.server.functional.model.bidder.BidderName.ALIAS
+import static org.prebid.server.functional.model.bidder.BidderName.GENERIC
+import static org.prebid.server.functional.model.bidder.BidderName.RUBICON


marki1an · 2024-02-28T13:44:57Z

src/test/groovy/org/prebid/server/functional/tests/privacy/GppTransmitUfpdActivitiesSpec.groovy

@@ -177,6 +181,55 @@ class GppTransmitUfpdActivitiesSpec extends PrivacyBaseSpec {
        assert metrics[DISALLOWED_COUNT_FOR_GENERIC_ADAPTER] == 1
    }

+    def "PBS auction call when transmit UFPD activities is rejecting requests with activityTransition false should remove only UFPD fields in request"() {


Please move this test to the end of the spec

marki1an · 2024-02-28T13:47:12Z

src/test/groovy/org/prebid/server/functional/testcontainers/scaffolding/VendorList.groovy

@@ -33,6 +33,11 @@ class VendorList extends NetworkScaffolding {
        request().withPath(VENDOR_LIST_ENDPOINT)
    }

+    @Override
+    void reset() {
+        TcfPolicyVersion.values().each { version -> super.reset("/v${version}/vendor-list.json") }


Here should be ${version.vendorListVersion}

marki1an · 2024-02-28T13:57:42Z

src/test/groovy/org/prebid/server/functional/tests/privacy/GppTransmitEidsActivitiesSpec.groovy

+            it.setAccountId(accountId)
+            it.ext.prebid.trace = VERBOSE


Unnecessary setup of those fields since they are already applied in givenBidRequestWithAccountAndEidsData(...) and check other occurrences.

marki1an · 2024-02-28T15:04:50Z

src/test/groovy/org/prebid/server/functional/util/privacy/TcfUtils.groovy

+        def purposesLITransparency = enforcementRequirements.getPurposesLITransparency()
+        def restrictionType = enforcementRequirements.restrictionType
+        def vendorIdGvl = enforcementRequirements.vendorIdGvl
+        def builder = new Builder()


Please use new TcfConsent.Builder() istead of new Builder()

marki1an · 2024-02-29T10:32:00Z

.../groovy/org/prebid/server/functional/tests/privacy/TcfBasicTransmitEidsActivitiesSpec.groovy

+
+        then: "Generic bidder request should have data in Eid field"
+        def bidderRequest = bidder.getBidderRequest(bidRequest.id)
+        assert bidderRequest.user.eids == userEids


What about bidderRequest.user?.ext?.eids

marki1an · 2024-02-29T10:32:39Z

.../groovy/org/prebid/server/functional/tests/privacy/TcfBasicTransmitEidsActivitiesSpec.groovy

+
+        then: "Generic bidder request should have data in Eid field"
+        def bidderRequest = bidder.getBidderRequest(bidRequest.id)
+        assert bidderRequest.user.ext.eids == userEids


bidderRequest.user?.eids?

Check other cases

marki1an · 2024-02-29T10:33:05Z

...t/groovy/org/prebid/server/functional/tests/privacy/TcfFullTransmitEidsActivitiesSpec.groovy

+
+        then: "Generic bidder request shouldn't have data in Eid field"
+        def bidderRequest = bidder.getBidderRequest(bidRequest.id)
+


Minor empty line

marki1an · 2024-02-29T11:45:01Z

src/test/groovy/org/prebid/server/functional/tests/privacy/PrivacyOrtbConverterSpec.groovy

+import static org.prebid.server.functional.model.request.auction.ActivityType.TRANSMIT_EIDS
+import static org.prebid.server.functional.model.request.auction.TraceLevel.VERBOSE
+
+class PrivacyOrtbConverterSpec extends PrivacyBaseSpec {


Probably better naming TransmitEidsOrtbConverterActivitiesSpec

marki1an · 2024-02-29T11:47:30Z

src/test/groovy/org/prebid/server/functional/tests/privacy/PrivacyOrtbConverterSpec.groovy

+
+        then: "Generic bidder request should have data in Eid field"
+        def bidderRequest = bidder.getBidderRequest(bidRequest.id)
+        assert bidderRequest?.user?.ext?.eids == userEids


Can you add an assertion for bidderRequest.user.eids?
Check other occurrences.

osulzhenko added 3 commits January 8, 2024 00:51

WIP

030675c

WIP

f9402e2

Add functional tests for aligning TCF and activity controls and also …

90a51ce

…add tests for transmitEids activities

osulzhenko requested a review from marki1an January 9, 2024 17:12

osulzhenko added the tests Functional or other tests label Jan 10, 2024

Merge branch 'add-transmit-eids-activity' into functional-tests/Align…

86355e7

…ing-TCF-and-Activity-Controls

osulzhenko changed the base branch from master to add-transmit-eids-activity January 15, 2024 08:54

osulzhenko added 2 commits January 15, 2024 19:08

Merge branch 'add-transmit-eids-activity' into functional-tests/Align…

e433aae

…ing-TCF-and-Activity-Controls

Merge branch 'add-transmit-eids-activity' into functional-tests/Align…

bbe60b1

…ing-TCF-and-Activity-Controls

osulzhenko added the work in progress Signals not finished work label Jan 16, 2024

refactor full enforcement requirments for tcf

010be34

osulzhenko force-pushed the functional-tests/Aligning-TCF-and-Activity-Controls branch from 7acc755 to 010be34 Compare January 22, 2024 22:02

refactoring functional tests for tcf

85215c7

osulzhenko removed the work in progress Signals not finished work label Jan 29, 2024

osulzhenko added 2 commits January 31, 2024 21:14

Merge branch 'add-transmit-eids-activity' into functional-tests/Align…

7e4bd4c

…ing-TCF-and-Activity-Controls

update tcf test cases

a205776

osulzhenko force-pushed the functional-tests/Aligning-TCF-and-Activity-Controls branch from 47b66c3 to c6bd6a2 Compare February 5, 2024 11:46

minor fix

f23a2be

osulzhenko force-pushed the functional-tests/Aligning-TCF-and-Activity-Controls branch from c6bd6a2 to f23a2be Compare February 5, 2024 12:02

osulzhenko added 2 commits February 14, 2024 21:54

Update TCF functional tests

6a97a7a

Minor update

673b451

marki1an requested changes Feb 23, 2024

View reviewed changes

osulzhenko added 2 commits February 27, 2024 10:40

Merge branch 'add-transmit-eids-activity' into functional-tests/Align…

34dcc0a

…ing-TCF-and-Activity-Controls

Update after code review

7ca04dd

osulzhenko requested a review from marki1an February 27, 2024 08:58

Update after code review

44e6549

osulzhenko force-pushed the functional-tests/Aligning-TCF-and-Activity-Controls branch from 894e020 to 44e6549 Compare February 27, 2024 11:46

marki1an requested changes Feb 27, 2024

View reviewed changes

Update after code review

d73655f

osulzhenko requested a review from marki1an February 27, 2024 15:29

marki1an requested changes Feb 28, 2024

View reviewed changes

Update after code review

a8f62cc

osulzhenko requested a review from marki1an February 28, 2024 16:37

marki1an requested changes Feb 29, 2024

View reviewed changes

Split tcf tests and minor fixes

19558d4

osulzhenko requested review from marki1an and Net-burst February 29, 2024 11:37

marki1an requested changes Feb 29, 2024

View reviewed changes

osulzhenko added 2 commits February 29, 2024 13:49

minor fix

4e471ca

minor update

e6f19df

osulzhenko requested a review from marki1an February 29, 2024 12:11

marki1an approved these changes Feb 29, 2024

View reviewed changes

SerhiiNahornyi merged commit 4eaf393 into add-transmit-eids-activity Feb 29, 2024

SerhiiNahornyi deleted the functional-tests/Aligning-TCF-and-Activity-Controls branch February 29, 2024 12:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add functional tests for aligning TCF and tests for support transmitEids activity controls #2893

Add functional tests for aligning TCF and tests for support transmitEids activity controls #2893

osulzhenko commented Jan 9, 2024

marki1an Feb 22, 2024

osulzhenko Feb 27, 2024

marki1an Feb 27, 2024

osulzhenko Feb 27, 2024

marki1an Feb 22, 2024

osulzhenko Feb 27, 2024

marki1an Feb 22, 2024

osulzhenko Feb 27, 2024

marki1an Feb 22, 2024

osulzhenko Feb 27, 2024

marki1an Feb 22, 2024

osulzhenko Feb 27, 2024

marki1an Feb 22, 2024

osulzhenko Feb 27, 2024

marki1an Feb 27, 2024

marki1an Feb 27, 2024

marki1an Feb 27, 2024

marki1an Feb 27, 2024

marki1an Feb 27, 2024

marki1an Feb 27, 2024

Net-burst Feb 28, 2024

marki1an Feb 27, 2024

marki1an Feb 28, 2024

marki1an Feb 28, 2024

marki1an Feb 28, 2024

marki1an Feb 28, 2024

marki1an Feb 28, 2024

marki1an Feb 28, 2024 •

edited

Loading

marki1an Feb 29, 2024

marki1an Feb 29, 2024

marki1an Feb 29, 2024

marki1an Feb 29, 2024

marki1an Feb 29, 2024

marki1an Feb 29, 2024

		@@ -0,0 +1,23 @@
		package org.prebid.server.functional.model.privacy

		import com.iabtcf.v2.PublisherRestriction


		private final PrebidServerService activityPbsServiceExcludeGvl = pbsServiceFactory.getService(PBS_CONFIG)

		private final PrebidServerService activityPbsServiceExcludeGvlWithElderOrtb = pbsServiceFactory.getService(PBS_CONFIG + ["adapters.generic.ortb-version": "2.5"])


		then: "Generic bidder request shouldn't have data in Eid field"
		def bidderRequest = bidder.getBidderRequest(bidRequest.id)

Add functional tests for aligning TCF and tests for support transmitEids activity controls #2893

Add functional tests for aligning TCF and tests for support transmitEids activity controls #2893

Conversation

osulzhenko commented Jan 9, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marki1an Feb 28, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marki1an Feb 28, 2024 •

edited

Loading