Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TransmitUserFPD and TransmitPreciseGeo activity integration #2906

Merged
merged 11 commits into from
Jul 27, 2023

Conversation

VeronikaSolovei9
Copy link
Contributor

@VeronikaSolovei9 VeronikaSolovei9 commented Jul 6, 2023

This PR contains existing GDPR related code refactoring due to activity integration.

I deviate from existing scrubber strategy implementation. New scrubber function ScrubRequest receives enforcer data, not a strategy. There are couple of reasons for this:

  1. We run 2 activities and we need to handle 2 strategies at once: 1st UFPD 2nd PreciseGeo
  2. Both UFPD and PreciseGeo should modify User and Device together, not just one of them.
  3. I think it's better and cleaner to have separate variables for activities, not mix them with existing CCPA, COPPA, etc
  4. Existing GDPR strategies are slightly different from what activity should do, like for User we need to scrub additional bidRequest.User.EIDs = nil, bidRequest.User.Data = nil and in User.Ext remove "data" instead of "eids".

With this implementation all existing unit tests pass, local testing of activities looks good.

@VeronikaSolovei9 VeronikaSolovei9 changed the base branch from activities to master July 6, 2023 22:52
@VeronikaSolovei9 VeronikaSolovei9 marked this pull request as ready for review July 7, 2023 01:51
@SyntaxNode SyntaxNode self-assigned this Jul 10, 2023
@bsardo bsardo assigned hhhjort and unassigned bsardo Jul 17, 2023
hhhjort
hhhjort previously approved these changes Jul 18, 2023
Copy link
Collaborator

@hhhjort hhhjort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

privacyEnforcement.CCPA = ccpaEnforcer.ShouldEnforce(bidderRequest.BidderName.String())
// potentially block passing IDs based on COPPA
if req.BidRequest.Regs != nil && req.BidRequest.Regs.COPPA == 1 {
privacyEnforcement.COPPA = true
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COPPA is already set at the start of the loop. Why do we need to include it again here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, removed

privacyEnforcement.COPPA = true
}
// potentially block passing geo based on LMT
privacyEnforcement.LMT = lmtEnforcer.ShouldEnforce(unknownBidder)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

COPPA and LMT are already set at the start of the loop. Why do we need to include it again here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed

}
if test.expectedScrubDeviceExecuted {
m.On("ScrubDevice", req.Device, ScrubStrategyDeviceIDAll, ScrubStrategyIPV4Lowest8, ScrubStrategyIPV6Lowest32, ScrubStrategyGeoFull).Return(replacedDevice).Once()
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice use of the mock library.

//transmitEids covers user.eids and user.ext.eids
if bidRequest.User != nil {
bidRequest.User.EIDs = nil
bidRequest.User.Ext = scrubExtIDs(bidRequest.User.Ext, "eids")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could unmarshal user.ext at the start of this method if either UFPD or Eids are true, modify as necessary throughout the function, and at the end marshal it back once more if either UFPD or Eids are true. That would bring us down to at most 1 unmarshal/marshal pair for each ext. Thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done. Please see this change in this commit bf4b27b

}
}

if userExtModified {
userExt, _ := json.Marshal(userExtParsed)
bidRequest.User.Ext = userExt
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure that User is not a shared object at this point?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The bidRequest is absolutely at least a shallow copy at this point. I'm good with adding a shallow clone here to be certain if it's not otherwise proven. We'll replace it in the near future with the RequestWrapper and the cost of cloning the User object doesn't seem too concerning.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added clones.

@VeronikaSolovei9
Copy link
Contributor Author

IMHO, we should follow the IPv6 scrubbing approach used by PBS-Java.

Will it be ok if I put a follow up PR with this change only?

@SyntaxNode
Copy link
Contributor

Will it be ok if I put a follow up PR with this change only?

Yup. Let's go with the stricter 32 bits for now.

hhhjort
hhhjort previously approved these changes Jul 25, 2023
Copy link
Collaborator

@hhhjort hhhjort left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@SyntaxNode SyntaxNode merged commit 9d72dbe into master Jul 27, 2023
@SyntaxNode SyntaxNode deleted the activity_ufpd_geo branch October 2, 2023 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants