From aaff53cfc81a4e5eda28a4b28504b75be51e4d26 Mon Sep 17 00:00:00 2001 From: Michael Gloystein Date: Fri, 14 Jun 2019 09:16:25 -0600 Subject: [PATCH] Master merge (#3) * Use the correct labels for cache performance metric (#904) * PubMatic Adslot validation (#886) * testing a few changes to validate adslot, more to follow * Changed AdSlot to optional parameter, modified validation and test cases for the same * removed imp id from adslot validation error msgs * assign banner size * remove TrimSpace where it is not needed as recommended in the review * Implementation of Categories Http fetcher (#882) * Implementation of Categories Http fetcher -Added code to fetch data using http -Added previously loaded categories to run http request just at the first time * Moved common Category struct to stored_request * Added comments * Minor refactoring * Minor refactoring -Added verification if category exist in map * Minor refactoring * Minor refactoring -strings.Replace changed to strings.TrimSuffix * Improved error handling on Beachfront adapter (#873) * Removed a redundant error message that was causing some confusion. * trying to turn '[]' into null and 200 inti 404 * not a goot path * looking at the status codes * I think I have covers all these bases. * checking for empty array response a failing sanely * removed an attempt to pointlessly reset the status code. * corrected and added test cases * Fix Rubicon bidder for multi-format impression processing (#902) * Replace Dockerfile Alpine linux with Ubuntu (#885) * Ubuntu dockerfile works, includes backups and out file * Ubuntu Dockerfile creates image successfully and prebid-server was tested with a sample request. Seems to work fine * Ubuntu 18.04 now builds, validates.sh, and curls sample request successfully * Removed sed command * [fix] broken Go 1.9 compatibility (#910) This CL addresses Go 1.9 compatibility issue along with adding back Go 1.9 to travis testing setup to prevent such bugs. Issue: #895 * Fixed "invalid BidType: " error for lifestreet adapter (#893) * Fixed "invalid BidType: " error for lifestreet adapter * After run gofmt * Used standard bid.ext type to get bid.ext.prebid.type * [Currency support] Activate multi-currencies support (#894) This CL is the last piece to activate currency conversion support in PBS. It activates currency support per default. Currency rates will be fetched once per hour (PrebidJS file is updated once a day). Issue: #280 * ImproveDigital adapter: (#887) - Add support for video - Add support for mobile/app - Add support for multibid responses - Extend optional parameters * Add a PI exemption environment variable to PBS (#916) * Refactored to official name of config item * Changes suggested by Mansi Nahar * [Sharethrough] Add new Sharethrough Adapter (#903) * wip * wip * wip * exploration for sharethrough prebid-server * WIP: updating sharethrough adapter to latest prebid version Co-authored-by: Chris Nguyen * WIP: adding butler params to the request #164291358 Co-authored-by: Josh Becker * Manage bid sizes [#164291358] Co-authored-by: Josh Becker * Manage GDPR [#164291358] Co-authored-by: Josh Becker * Populate prebid-server version if provided [#164291358] Co-authored-by: Josh Becker * Refactor gdpr data extraction from request [#164291358] Co-authored-by: Eddy Pechuzal * Add instant play capability [#164291358] Co-authored-by: Eddy Pechuzal * Split in multiple files [#164291358] Co-authored-by: Eddy Pechuzal * Add s2s-win beacon todo? replace server name by server id? [#164291358] Co-authored-by: Eddy Pechuzal * Removing `server` param in s2s-win beacon (will be added in imp req) [#164291358] * Clean up code + enable syncer (#4) [#165257793] * Proper error handling (#6) * Proper error handling [#165745574] * Address review (baby clean up) + catch error that was missed [#165745574] * Implement Unit Tests (#7) * Proper error handling [#165745574] * Address review (baby clean up) + catch error that was missed [#165745574] * Implement unit tests for utils [#165891351] * Add UT for utils + butler [#165891351] Co-authored-by: Michael Duran * Attempt for testing Bidder interface function implementations [#165891351] Co-authored-by: Michael Duran * Finalizing Unit tests [#165891351] Co-authored-by: Chris Nguyen Co-authored-by: Josh Becker * Fixing sharethrough.yaml capabilities [#165891351] Co-authored-by: Josh Becker * Send supplyId to imp req instead of hbSource (#5) [#165477915] * Finalize PR (#8) [#164911891] Co-authored-by: Josh Becker * Remove test setting * Add Sharethrough in syncer_test * Update deserializing of third party partners * Refactor/optimize UserAgent parsing (#9) following josephveach's review in prebid/prebid-server#903 * Addressing June 3rd review from prebid/prebid-server#903 Optimizations, clean up suggested by @mansinahar * Addressing June 4th review from prebid/prebid-server#903 (#10) * Addressing June 4th review from prebid/prebid-server#903 Clean up canAutoPlayVideo + hardcode bhVersion to unknown for now... * Removing hbVersion butler param since it's not accessible * Fix adMarkup error handling * [Mgid] Add new Mgid Adapter (#907) * new mgid adapter * increase coverage * remove extra imports * Cache validation fix (#911) * Cache validation fix if no bids returned - don't throw cache error, just return empty result * Cache validation fix - optimization: do not run auction logic if no bids returned * Cache validation fix: minor refactoring * Cache validation fix: minor refactoring * Cache validation fix: added unit test for no bids returned * Cache validation fix: minor refactoring * Remove hard coded targeting keys (#923) --- .travis.yml | 1 + Dockerfile | 20 +- adapters/beachfront/beachfront.go | 57 ++- .../exemplary/minimal-banner.json | 2 +- .../exemplary/simple-video.json | 1 + .../minimal-banner-empty_array-200.json | 72 +++ .../supplemental/minimal-mobile-banner.json | 2 +- .../supplemental/minimal-mobile-video.json | 1 + .../supplemental/minimal-site-banner.json | 2 +- .../supplemental/multi-banner.json | 2 +- .../supplemental/multi-mix.json | 1 + .../supplemental/multi-video.json | 1 + adapters/improvedigital/improvedigital.go | 59 ++- .../exemplary/app-multi.json | 152 +++++++ .../exemplary/simple-banner.json | 1 - .../exemplary/site-multi.json | 156 +++++++ .../params/race/banner.json | 13 +- .../improvedigitaltest/params/race/video.json | 12 + .../supplemental/bad_response.json | 2 + .../supplemental/multi-seatbid.json | 60 +++ .../supplemental/native.json | 61 +++ .../supplemental/nobid-debug.json | 48 ++ .../supplemental/nobid.json | 52 +++ .../supplemental/noseatbid.json | 52 +++ .../supplemental/status_204.json | 2 + .../supplemental/status_400.json | 2 + .../supplemental/status_418.json | 2 + .../supplemental/wrong_impid.json | 77 ++++ adapters/improvedigital/params_test.go | 10 +- adapters/lifestreet/lifestreet.go | 29 +- adapters/mgid/README.md | 1 + adapters/mgid/mgid.go | 169 ++++++++ adapters/mgid/mgid_test.go | 10 + .../mgidtest/exemplary/simple-banner.json | 135 ++++++ .../exemplary/simple-banner_no_device.json | 129 ++++++ .../simple-banner_no_device_no_site.json | 121 ++++++ .../exemplary/simple-banner_with_crtype.json | 141 ++++++ .../mgidtest/supplemental/noaccountid.json | 29 ++ .../mgidtest/supplemental/status_204.json | 68 +++ .../mgidtest/supplemental/status_not200.json | 70 +++ .../mgid/mgidtest/supplemental/video.json | 57 +++ adapters/mgid/usersync.go | 12 + adapters/mgid/usersync_test.go | 18 + adapters/pubmatic/params_test.go | 4 +- adapters/pubmatic/pubmatic.go | 103 +++-- .../supplemental/invalidparam.json | 42 +- adapters/rubicon/rubicon.go | 22 +- adapters/rubicon/rubicon_test.go | 116 +++++ adapters/sharethrough/butler.go | 191 ++++++++ adapters/sharethrough/butler_test.go | 399 +++++++++++++++++ adapters/sharethrough/params_test.go | 58 +++ adapters/sharethrough/sharethrough.go | 72 +++ adapters/sharethrough/sharethrough_test.go | 245 +++++++++++ .../sharethroughtest/params/race/banner.json | 5 + .../sharethroughtest/params/race/native.json | 5 + adapters/sharethrough/usersync.go | 11 + adapters/sharethrough/usersync_test.go | 19 + adapters/sharethrough/utils.go | 193 +++++++++ adapters/sharethrough/utils_test.go | 410 ++++++++++++++++++ config/config.go | 23 +- config/config_test.go | 19 + endpoints/auction.go | 35 +- endpoints/auction_test.go | 56 +-- endpoints/cookie_sync_test.go | 2 +- endpoints/openrtb2/video_auction.go | 18 +- endpoints/openrtb2/video_auction_test.go | 15 +- endpoints/setuid_test.go | 2 +- exchange/adapter_map.go | 4 + exchange/exchange.go | 42 +- exchange/exchange_test.go | 4 +- exchange/targeting.go | 2 +- exchange/utils.go | 11 +- exchange/utils_test.go | 2 +- gdpr/gdpr.go | 2 +- gdpr/impl.go | 9 +- gdpr/impl_test.go | 10 +- macros/macros.go | 9 +- openrtb_ext/bid.go | 2 +- openrtb_ext/bid_response_video.go | 6 +- openrtb_ext/bidders.go | 4 + openrtb_ext/imp_mgid.go | 11 + openrtb_ext/imp_pubmatic.go | 5 +- openrtb_ext/imp_sharethrough.go | 102 +++++ pbsmetrics/prometheus/prometheus.go | 18 +- pbsmetrics/prometheus/prometheus_test.go | 37 +- static/bidder-info/improvedigital.yaml | 5 + static/bidder-info/mgid.yaml | 11 + static/bidder-info/sharethrough.yaml | 11 + static/bidder-params/improvedigital.json | 20 +- static/bidder-params/mgid.json | 34 ++ static/bidder-params/pubmatic.json | 2 +- static/bidder-params/sharethrough.json | 28 ++ .../backends/db_fetcher/fetcher.go | 2 +- .../backends/empty_fetcher/fetcher.go | 2 +- .../backends/file_fetcher/fetcher.go | 13 +- .../backends/file_fetcher/fetcher_test.go | 10 +- .../backends/http_fetcher/fetcher.go | 57 ++- stored_requests/fetcher.go | 11 +- stored_requests/fetcher_test.go | 2 +- stored_requests/multifetcher.go | 4 +- usersync/usersyncers/syncer.go | 4 + usersync/usersyncers/syncer_test.go | 2 + 102 files changed, 4259 insertions(+), 216 deletions(-) create mode 100644 adapters/beachfront/beachfronttest/supplemental/minimal-banner-empty_array-200.json create mode 100644 adapters/improvedigital/improvedigitaltest/exemplary/app-multi.json create mode 100644 adapters/improvedigital/improvedigitaltest/exemplary/site-multi.json create mode 100644 adapters/improvedigital/improvedigitaltest/params/race/video.json create mode 100644 adapters/improvedigital/improvedigitaltest/supplemental/multi-seatbid.json create mode 100644 adapters/improvedigital/improvedigitaltest/supplemental/native.json create mode 100644 adapters/improvedigital/improvedigitaltest/supplemental/nobid-debug.json create mode 100644 adapters/improvedigital/improvedigitaltest/supplemental/nobid.json create mode 100644 adapters/improvedigital/improvedigitaltest/supplemental/noseatbid.json create mode 100644 adapters/improvedigital/improvedigitaltest/supplemental/wrong_impid.json create mode 100644 adapters/mgid/README.md create mode 100644 adapters/mgid/mgid.go create mode 100644 adapters/mgid/mgid_test.go create mode 100644 adapters/mgid/mgidtest/exemplary/simple-banner.json create mode 100644 adapters/mgid/mgidtest/exemplary/simple-banner_no_device.json create mode 100644 adapters/mgid/mgidtest/exemplary/simple-banner_no_device_no_site.json create mode 100644 adapters/mgid/mgidtest/exemplary/simple-banner_with_crtype.json create mode 100644 adapters/mgid/mgidtest/supplemental/noaccountid.json create mode 100644 adapters/mgid/mgidtest/supplemental/status_204.json create mode 100644 adapters/mgid/mgidtest/supplemental/status_not200.json create mode 100644 adapters/mgid/mgidtest/supplemental/video.json create mode 100644 adapters/mgid/usersync.go create mode 100644 adapters/mgid/usersync_test.go create mode 100644 adapters/sharethrough/butler.go create mode 100644 adapters/sharethrough/butler_test.go create mode 100644 adapters/sharethrough/params_test.go create mode 100644 adapters/sharethrough/sharethrough.go create mode 100644 adapters/sharethrough/sharethrough_test.go create mode 100644 adapters/sharethrough/sharethroughtest/params/race/banner.json create mode 100644 adapters/sharethrough/sharethroughtest/params/race/native.json create mode 100644 adapters/sharethrough/usersync.go create mode 100644 adapters/sharethrough/usersync_test.go create mode 100644 adapters/sharethrough/utils.go create mode 100644 adapters/sharethrough/utils_test.go create mode 100644 openrtb_ext/imp_mgid.go create mode 100644 openrtb_ext/imp_sharethrough.go create mode 100644 static/bidder-info/mgid.yaml create mode 100644 static/bidder-info/sharethrough.yaml create mode 100644 static/bidder-params/mgid.json create mode 100644 static/bidder-params/sharethrough.json diff --git a/.travis.yml b/.travis.yml index f36c2e535ae..59fe5274bd6 100644 --- a/.travis.yml +++ b/.travis.yml @@ -1,6 +1,7 @@ language: go go: + - '1.9' - '1.10' - '1.11.1' diff --git a/Dockerfile b/Dockerfile index e5d5f5e5a78..1143cd3a94d 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,20 +1,24 @@ -FROM alpine:3.8 AS build +FROM ubuntu:18.04 AS build WORKDIR /go/src/github.com/prebid/prebid-server/ -RUN apk add -U --no-cache go git dep musl-dev +RUN apt-get update && \ + apt-get -y upgrade && \ + apt-get install -y git golang go-dep && \ + apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* ENV GOPATH /go ENV CGO_ENABLED 0 COPY ./ ./ -RUN dep ensure -RUN go build . +RUN dep ensure && \ + go build . - -FROM alpine:3.8 AS release -MAINTAINER Hans Hjort +FROM ubuntu:18.04 AS release +LABEL maintainer="hans.hjort@xandr.com" WORKDIR /usr/local/bin/ COPY --from=build /go/src/github.com/prebid/prebid-server/prebid-server . COPY static static/ COPY stored_requests/data stored_requests/data -RUN apk add -U --no-cache ca-certificates mtr +RUN apt-get update && \ + apt-get install -y ca-certificates mtr && \ + apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* EXPOSE 8000 EXPOSE 8080 ENTRYPOINT ["/usr/local/bin/prebid-server"] diff --git a/adapters/beachfront/beachfront.go b/adapters/beachfront/beachfront.go index 81d7371bf69..bfcc7692d54 100644 --- a/adapters/beachfront/beachfront.go +++ b/adapters/beachfront/beachfront.go @@ -3,11 +3,13 @@ package beachfront import ( "encoding/json" "errors" + "fmt" + "github.com/prebid/prebid-server/errortypes" "net/http" + "strconv" "strings" "github.com/prebid/prebid-server/adapters" - "github.com/prebid/prebid-server/errortypes" "github.com/prebid/prebid-server/openrtb_ext" "github.com/mxmCherry/openrtb" @@ -22,7 +24,7 @@ const VideoEndpoint = "https://reachms.bfmio.com/bid.json?exchange_id=" const VideoEndpointSuffix = "&prebidserver" const beachfrontAdapterName = "BF_PREBID_S2S" -const beachfrontAdapterVersion = "0.2.2" +const beachfrontAdapterVersion = "0.3.0" type BeachfrontAdapter struct { } @@ -140,7 +142,7 @@ func (a *BeachfrontAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapte var beachfrontRequests BeachfrontRequests var reqJSON []byte var uri string - var errs = make([]error, 0) + var errs = make([]error, 0, len(request.Imp)) var err error var imps int @@ -148,7 +150,6 @@ func (a *BeachfrontAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapte beachfrontRequests, errs, imps = preprocess(request, uri) - // These are fatal errors ------------- if uri == VideoEndpoint { reqJSON, err = json.Marshal(beachfrontRequests.Video) uri = uri + beachfrontRequests.Video.AppId + VideoEndpointSuffix @@ -175,6 +176,15 @@ func (a *BeachfrontAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapte headers.Add("Content-Type", "application/json;charset=utf-8") headers.Add("Accept", "application/json") + if request.Device != nil { + addHeaderIfNonEmpty(headers, "User-Agent", request.Device.UA) + addHeaderIfNonEmpty(headers, "X-Forwarded-For", request.Device.IP) + addHeaderIfNonEmpty(headers, "Accept-Language", request.Device.Language) + if request.Device.DNT != nil { + addHeaderIfNonEmpty(headers, "DNT", strconv.Itoa(int(*request.Device.DNT))) + } + } + return []*adapters.RequestData{{ Method: "POST", Uri: uri, @@ -288,9 +298,6 @@ func getBannerRequest(req *openrtb.BidRequest) (BeachfrontBannerRequest, []error return beachfrontReq, errs, imps } -/* -Prepare the request that has been received from Prebid.js, translating it to the beachfront format -*/ func getVideoRequest(req *openrtb.BidRequest) (BeachfrontVideoRequest, []error, int) { var videoImpsIndex = 0 var beachfrontReq = NewBeachfrontVideoRequest() @@ -324,7 +331,7 @@ func getVideoRequest(req *openrtb.BidRequest) (BeachfrontVideoRequest, []error, The req could contain banner,audio,native and video imps when It arrives here. I am only interested in video - The beach front video endpoint is only capable of returning a single nurl and price, wrapped in + The beachfront video endpoint is only capable of returning a single nurl and price, wrapped in an openrtb format, so even though I'm building a request here that will include multiple video impressions, only a single URL will be returned. Hopefully the beachfront endpoint can be modified in the future to return multiple video ads @@ -394,12 +401,33 @@ func getVideoRequest(req *openrtb.BidRequest) (BeachfrontVideoRequest, []error, func (a *BeachfrontAdapter) MakeBids(internalRequest *openrtb.BidRequest, externalRequest *adapters.RequestData, response *adapters.ResponseData) (*adapters.BidderResponse, []error) { var bids []openrtb.Bid var bidtype = getBidType(internalRequest) + + /* + Beachfront is now sending an empty array and 200 as their "no results" response. This should catch that. + */ + + if response.StatusCode == http.StatusOK && len(response.Body) <= 2 { + return nil, nil + } + + if response.StatusCode == http.StatusNoContent { + return nil, nil + } + + if response.StatusCode == http.StatusBadRequest { + return nil, []error{&errortypes.BadInput{ + Message: fmt.Sprintf("Unexpected status code: %d. Run with request.debug = 1 for more info", response.StatusCode), + }} + } + + if response.StatusCode != http.StatusOK { + return nil, []error{fmt.Errorf("Unexpected status code: %d. Run with request.debug = 1 for more info", response.StatusCode)} + } + bids, errs := postprocess(response, externalRequest, internalRequest.ID, bidtype) if len(errs) != 0 { - return nil, append(errs, &errortypes.BadServerResponse{ - Message: "Failed to process the beachfront response", - }) + return nil, errs } bidResponse := adapters.NewBidderResponseWithBidsCapacity(BidCapacity) @@ -499,6 +527,13 @@ func extractVideoCrid(nurl string) string { return strings.TrimSuffix(chunky[2], ":") } +// Thank you, brightroll. +func addHeaderIfNonEmpty(headers http.Header, headerName string, headerValue string) { + if len(headerValue) > 0 { + headers.Add(headerName, headerValue) + } +} + func NewBeachfrontBidder() *BeachfrontAdapter { return &BeachfrontAdapter{} } diff --git a/adapters/beachfront/beachfronttest/exemplary/minimal-banner.json b/adapters/beachfront/beachfronttest/exemplary/minimal-banner.json index d0fae5dc3cb..e234d161ca4 100644 --- a/adapters/beachfront/beachfronttest/exemplary/minimal-banner.json +++ b/adapters/beachfront/beachfronttest/exemplary/minimal-banner.json @@ -56,7 +56,7 @@ "dnt": 0, "ua": "", "adapterName": "BF_PREBID_S2S", - "adapterVersion": "0.2.2", + "adapterVersion": "0.3.0", "user": { } } diff --git a/adapters/beachfront/beachfronttest/exemplary/simple-video.json b/adapters/beachfront/beachfronttest/exemplary/simple-video.json index f62942811bd..2d3ff44f7de 100644 --- a/adapters/beachfront/beachfronttest/exemplary/simple-video.json +++ b/adapters/beachfront/beachfronttest/exemplary/simple-video.json @@ -80,6 +80,7 @@ } }, "mockResponse": { + "status" : 200, "body": { "id":"61b87329-8790-47b7-90dd-c53ae7ce1723", "seatBid":[ diff --git a/adapters/beachfront/beachfronttest/supplemental/minimal-banner-empty_array-200.json b/adapters/beachfront/beachfronttest/supplemental/minimal-banner-empty_array-200.json new file mode 100644 index 00000000000..12baf7235d5 --- /dev/null +++ b/adapters/beachfront/beachfronttest/supplemental/minimal-banner-empty_array-200.json @@ -0,0 +1,72 @@ +{ + "mockBidRequest": { + "id": "some_test_ad", + "site": { + "page": "https://test.opposingviews.com/i/society/republican-sen-collins-may-change-vote-tax-bill?cb=1234534" + }, + "imp": [ + { + "bidfloor": 0.02, + "banner": { + "format": [ + { + "w": 300, + "h": 250 + } + ] + }, + "ext": { + "bidder": { + "bidfloor": 0.02, + "appId": "00000000-1111-2222-3333-11111111111" + } + } + } + ] + }, + + "httpCalls": [ + { + "expectedRequest": { + "uri": "https://display.bfmio.com/prebid_display", + "body": { + "slots": [ + { + "slot": "", + "id": "00000000-1111-2222-3333-11111111111", + "bidfloor": 0.02, + "sizes": [ + { + "w": 300, + "h": 250 + } + ] + } + ], + "domain": "test.opposingviews.com", + "page": "https://test.opposingviews.com/i/society/republican-sen-collins-may-change-vote-tax-bill?cb=1234534", + "referrer": "", + "search": "", + "secure": 0, + "requestId": "some_test_ad", + "isMobile": 0, + "ip": "", + "deviceModel": "", + "deviceOs": "", + "dnt": 0, + "ua": "", + "adapterName": "BF_PREBID_S2S", + "adapterVersion": "0.3.0", + "user": { + } + } + }, + "mockResponse": { + "status": 200, + "body": [] + } + } + ], + + "expectedBids": [] +} \ No newline at end of file diff --git a/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-banner.json b/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-banner.json index 1bc5cf7c37f..b73c025c277 100644 --- a/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-banner.json +++ b/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-banner.json @@ -57,7 +57,7 @@ "dnt": 0, "ua": "", "adapterName": "BF_PREBID_S2S", - "adapterVersion": "0.2.2", + "adapterVersion": "0.3.0", "user": { } } diff --git a/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-video.json b/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-video.json index ffef0972011..e752884425e 100644 --- a/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-video.json +++ b/adapters/beachfront/beachfronttest/supplemental/minimal-mobile-video.json @@ -63,6 +63,7 @@ } }, "mockResponse": { + "status": 200, "body": { "id":"61b87329-8790-47b7-90dd-c53ae7ce1723", "seatBid":[ diff --git a/adapters/beachfront/beachfronttest/supplemental/minimal-site-banner.json b/adapters/beachfront/beachfronttest/supplemental/minimal-site-banner.json index eaf4ae0b6c4..fe3982d7302 100644 --- a/adapters/beachfront/beachfronttest/supplemental/minimal-site-banner.json +++ b/adapters/beachfront/beachfronttest/supplemental/minimal-site-banner.json @@ -56,7 +56,7 @@ "dnt": 0, "ua": "", "adapterName": "BF_PREBID_S2S", - "adapterVersion": "0.2.2", + "adapterVersion": "0.3.0", "user": { } } diff --git a/adapters/beachfront/beachfronttest/supplemental/multi-banner.json b/adapters/beachfront/beachfronttest/supplemental/multi-banner.json index aab07059f7a..f18fa6b9e00 100644 --- a/adapters/beachfront/beachfronttest/supplemental/multi-banner.json +++ b/adapters/beachfront/beachfronttest/supplemental/multi-banner.json @@ -96,7 +96,7 @@ "dnt": 1, "ua": "Opera/9.80 (X11; Linux i686; Ubuntu/14.10) Presto/2.12.388 Version/12.16", "adapterName": "BF_PREBID_S2S", - "adapterVersion": "0.2.2", + "adapterVersion": "0.3.0", "user": { "buyeruid": "some-buyer", "id": "some-user" diff --git a/adapters/beachfront/beachfronttest/supplemental/multi-mix.json b/adapters/beachfront/beachfronttest/supplemental/multi-mix.json index 71f36be3e91..297b757a9a3 100644 --- a/adapters/beachfront/beachfronttest/supplemental/multi-mix.json +++ b/adapters/beachfront/beachfronttest/supplemental/multi-mix.json @@ -154,6 +154,7 @@ } }, "mockResponse": { + "status": 200, "body": { "id":"61b87329-8790-47b7-90dd-c53ae7ce1723", "seatBid":[ diff --git a/adapters/beachfront/beachfronttest/supplemental/multi-video.json b/adapters/beachfront/beachfronttest/supplemental/multi-video.json index f4ad38ad105..22f207fb5a8 100644 --- a/adapters/beachfront/beachfronttest/supplemental/multi-video.json +++ b/adapters/beachfront/beachfronttest/supplemental/multi-video.json @@ -106,6 +106,7 @@ } }, "mockResponse": { + "status": 200, "body": { "id":"61b87329-8790-47b7-90dd-c53ae7ce1723", "seatBid":[ diff --git a/adapters/improvedigital/improvedigital.go b/adapters/improvedigital/improvedigital.go index b4d86ee9df8..8cf52c56c7d 100644 --- a/adapters/improvedigital/improvedigital.go +++ b/adapters/improvedigital/improvedigital.go @@ -8,6 +8,7 @@ import ( "github.com/mxmCherry/openrtb" "github.com/prebid/prebid-server/adapters" "github.com/prebid/prebid-server/errortypes" + "github.com/prebid/prebid-server/openrtb_ext" ) type ImprovedigitalAdapter struct { @@ -58,15 +59,36 @@ func (a *ImprovedigitalAdapter) MakeBids(internalRequest *openrtb.BidRequest, ex return nil, []error{err} } - bidResponse := adapters.NewBidderResponseWithBidsCapacity(1) + if len(bidResp.SeatBid) == 0 { + return nil, nil + } + + if len(bidResp.SeatBid) > 1 { + return nil, []error{&errortypes.BadServerResponse{ + Message: fmt.Sprintf("Unexpected SeatBid! Must be only one but have: %d", len(bidResp.SeatBid)), + }} + } + + seatBid := bidResp.SeatBid[0] + + if len(seatBid.Bid) == 0 { + return nil, nil + } + + bidResponse := adapters.NewBidderResponseWithBidsCapacity(len(seatBid.Bid)) - for _, sb := range bidResp.SeatBid { - for i := range sb.Bid { - bidResponse.Bids = append(bidResponse.Bids, &adapters.TypedBid{ - Bid: &sb.Bid[i], - BidType: "banner", - }) + for i := range seatBid.Bid { + bid := seatBid.Bid[i] + + bidType, err := getMediaTypeForImp(bid.ImpID, internalRequest.Imp) + if err != nil { + return nil, []error{err} } + + bidResponse.Bids = append(bidResponse.Bids, &adapters.TypedBid{ + Bid: &bid, + BidType: bidType, + }) } return bidResponse, nil @@ -78,3 +100,26 @@ func NewImprovedigitalBidder(endpoint string) *ImprovedigitalAdapter { endpoint: endpoint, } } + +func getMediaTypeForImp(impID string, imps []openrtb.Imp) (openrtb_ext.BidType, error) { + for _, imp := range imps { + if imp.ID == impID { + if imp.Banner != nil { + return openrtb_ext.BidTypeBanner, nil + } + + if imp.Video != nil { + return openrtb_ext.BidTypeVideo, nil + } + + return "", &errortypes.BadServerResponse{ + Message: fmt.Sprintf("Unknown impression type for ID: \"%s\"", impID), + } + } + } + + // This shouldnt happen. Lets handle it just incase by returning an error. + return "", &errortypes.BadServerResponse{ + Message: fmt.Sprintf("Failed to find impression for ID: \"%s\"", impID), + } +} diff --git a/adapters/improvedigital/improvedigitaltest/exemplary/app-multi.json b/adapters/improvedigital/improvedigitaltest/exemplary/app-multi.json new file mode 100644 index 00000000000..1aee6bdf2c7 --- /dev/null +++ b/adapters/improvedigital/improvedigitaltest/exemplary/app-multi.json @@ -0,0 +1,152 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "app": { + "id": "appID", + "publisher": { + "id": "uniq_pub_id" + } + }, + "device":{ + "ua": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" + }, + "imp": [{ + "id": "test-imp-id-banner", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }, + { + "id": "test-imp-id-video", + "video": { + "mimes": ["video/mp4"], + "protocols": [2, 5], + "w": 1920, + "h": 1080 + }, + "ext": { + "bidder": { + "placementId": 13244 + } + } + } + ] + }, + + "httpCalls": [{ + "expectedRequest": { + "uri": "http://localhost/pbs", + "body": { + "id": "test-request-id", + "app": { + "id": "appID", + "publisher": { + "id": "uniq_pub_id" + } + }, + "device":{ + "ua": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" + }, + "imp": [{ + "id": "test-imp-id-banner", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }, + { + "id": "test-imp-id-video", + "video": { + "mimes": ["video/mp4"], + "protocols": [2, 5], + "w": 1920, + "h": 1080 + }, + "ext": { + "bidder": { + "placementId": 13244 + } + } + } + ] + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [{ + "seat": "improvedigital", + "bid": [{ + "id": "randomid1", + "impid": "test-imp-id-banner", + "price": 0.500000, + "adid": "12345678", + "adm": "some-test-ad-html", + "cid": "987", + "crid": "12345678", + "h": 250, + "w": 300 + }, + { + "id": "randomid2", + "impid": "test-imp-id-video", + "price": 0.500000, + "adm": "some-test-ad-vast", + "crid": "1234567", + "w": 1920, + "h": 1080 + } + ] + }], + "cur": "USD" + } + } + }], + + "expectedBidResponses": [{ + "currency": "USD", + "bids": [{ + "bid": { + "id": "randomid1", + "impid": "test-imp-id-banner", + "price": 0.5, + "adm": "some-test-ad-html", + "adid": "12345678", + "cid": "987", + "crid": "12345678", + "w": 300, + "h": 250 + }, + "type": "banner" + }, + { + "bid": { + "id": "randomid2", + "impid": "test-imp-id-video", + "price": 0.5, + "adm": "some-test-ad-vast", + "crid": "1234567", + "w": 1920, + "h": 1080 + }, + "type": "video" + } + ] + }] +} diff --git a/adapters/improvedigital/improvedigitaltest/exemplary/simple-banner.json b/adapters/improvedigital/improvedigitaltest/exemplary/simple-banner.json index ea650ae0c44..19668c457b9 100644 --- a/adapters/improvedigital/improvedigitaltest/exemplary/simple-banner.json +++ b/adapters/improvedigital/improvedigitaltest/exemplary/simple-banner.json @@ -6,7 +6,6 @@ }, "imp": [{ "id": "test-imp-id", - "pid": 123, "banner": { "format": [{ "w": 300, diff --git a/adapters/improvedigital/improvedigitaltest/exemplary/site-multi.json b/adapters/improvedigital/improvedigitaltest/exemplary/site-multi.json new file mode 100644 index 00000000000..5fb4f061f50 --- /dev/null +++ b/adapters/improvedigital/improvedigitaltest/exemplary/site-multi.json @@ -0,0 +1,156 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "site": { + "page": "https://good.site/url", + "domain": "good.site", + "publisher": { + "id": "uniq_pub_id" + }, + "keywords": "omgword", + "ext": { + "amp": 0 + } + }, + "imp": [{ + "id": "test-imp-id-banner", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }, + { + "id": "test-imp-id-video", + "video": { + "mimes": ["video/mp4"], + "protocols": [2, 5], + "w": 1920, + "h": 1080 + }, + "ext": { + "bidder": { + "placementId": 13244 + } + } + } + ] + }, + + "httpCalls": [{ + "expectedRequest": { + "uri": "http://localhost/pbs", + "body": { + "id": "test-request-id", + "site": { + "page": "https://good.site/url", + "domain": "good.site", + "publisher": { + "id": "uniq_pub_id" + }, + "keywords": "omgword", + "ext": { + "amp": 0 + } + }, + "imp": [{ + "id": "test-imp-id-banner", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }, + { + "id": "test-imp-id-video", + "video": { + "mimes": ["video/mp4"], + "protocols": [2, 5], + "w": 1920, + "h": 1080 + }, + "ext": { + "bidder": { + "placementId": 13244 + } + } + } + ] + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [{ + "seat": "improvedigital", + "bid": [{ + "id": "randomid1", + "impid": "test-imp-id-banner", + "price": 0.500000, + "adid": "12345678", + "adm": "some-test-ad-html", + "cid": "987", + "crid": "12345678", + "h": 250, + "w": 300 + }, + { + "id": "randomid2", + "impid": "test-imp-id-video", + "price": 0.500000, + "adm": "some-test-ad-vast", + "crid": "1234567", + "w": 1920, + "h": 1080 + } + ] + }], + "cur": "USD" + } + } + }], + + "expectedBidResponses": [{ + "currency": "USD", + "bids": [{ + "bid": { + "id": "randomid1", + "impid": "test-imp-id-banner", + "price": 0.5, + "adm": "some-test-ad-html", + "adid": "12345678", + "cid": "987", + "crid": "12345678", + "w": 300, + "h": 250 + }, + "type": "banner" + }, + { + "bid": { + "id": "randomid2", + "impid": "test-imp-id-video", + "price": 0.5, + "adm": "some-test-ad-vast", + "crid": "1234567", + "w": 1920, + "h": 1080 + }, + "type": "video" + } + ] + }] +} diff --git a/adapters/improvedigital/improvedigitaltest/params/race/banner.json b/adapters/improvedigital/improvedigitaltest/params/race/banner.json index 0967ef424bc..0de1d580215 100644 --- a/adapters/improvedigital/improvedigitaltest/params/race/banner.json +++ b/adapters/improvedigital/improvedigitaltest/params/race/banner.json @@ -1 +1,12 @@ -{} +{ + "placementId": 123, + "publisherId": 321, + "placementKey": "uniq_name", + "size": { + "w": 100, + "h": 100 + }, + "keyValues": { + "testKey1": ["testValueA", "testValueB"] + } +} diff --git a/adapters/improvedigital/improvedigitaltest/params/race/video.json b/adapters/improvedigital/improvedigitaltest/params/race/video.json new file mode 100644 index 00000000000..0de1d580215 --- /dev/null +++ b/adapters/improvedigital/improvedigitaltest/params/race/video.json @@ -0,0 +1,12 @@ +{ + "placementId": 123, + "publisherId": 321, + "placementKey": "uniq_name", + "size": { + "w": 100, + "h": 100 + }, + "keyValues": { + "testKey1": ["testValueA", "testValueB"] + } +} diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/bad_response.json b/adapters/improvedigital/improvedigitaltest/supplemental/bad_response.json index cf6c498a5db..86fb40e20e8 100644 --- a/adapters/improvedigital/improvedigitaltest/supplemental/bad_response.json +++ b/adapters/improvedigital/improvedigitaltest/supplemental/bad_response.json @@ -14,6 +14,7 @@ }, "ext": { "bidder": { + "placementId": 13245 } } } @@ -39,6 +40,7 @@ }, "ext": { "bidder": { + "placementId": 13245 } } } diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/multi-seatbid.json b/adapters/improvedigital/improvedigitaltest/supplemental/multi-seatbid.json new file mode 100644 index 00000000000..575dfee0434 --- /dev/null +++ b/adapters/improvedigital/improvedigitaltest/supplemental/multi-seatbid.json @@ -0,0 +1,60 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + }, + + "httpCalls": [{ + "expectedRequest": { + "uri": "http://localhost/pbs", + "body": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [{ + "seat": "bad1", + "bid": [] + }, { + "seat": "bad2", + "bid": [] + }], + "cur": "USD" + } + } + }], + + "expectedMakeBidsErrors": [ + "Unexpected SeatBid! Must be only one but have: 2" + ] +} diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/native.json b/adapters/improvedigital/improvedigitaltest/supplemental/native.json new file mode 100644 index 00000000000..a6336651a3b --- /dev/null +++ b/adapters/improvedigital/improvedigitaltest/supplemental/native.json @@ -0,0 +1,61 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "native": { + "ver": "1.1" + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + }, + "httpCalls": [{ + "expectedRequest": { + "uri": "http://localhost/pbs", + "body": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "native": { + "request": "", + "ver": "1.1" + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [{ + "seat": "improvedigital", + "bid": [{ + "id": "randomid", + "impid": "test-imp-id", + "price": 0.500000, + "adid": "12345678", + "adm": "some-test-native", + "cid": "987", + "crid": "12345678", + "h": 250, + "w": 300 + }] + }], + "cur": "USD" + } + } + }], + + "expectedMakeBidsErrors": [ + "Unknown impression type for ID: \"test-imp-id\"" + ] +} diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/nobid-debug.json b/adapters/improvedigital/improvedigitaltest/supplemental/nobid-debug.json new file mode 100644 index 00000000000..8e70da371ac --- /dev/null +++ b/adapters/improvedigital/improvedigitaltest/supplemental/nobid-debug.json @@ -0,0 +1,48 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + }, + + "httpCalls": [{ + "expectedRequest": { + "uri": "http://localhost/pbs", + "body": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + } + }, + "mockResponse": { + "status": 204, + "body": "{\"ext\": {\"debug\": \"developer magic with special chars \\n \\t \n \t \"}}" + } + }], + + "expectedBidResponses": [] +} diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/nobid.json b/adapters/improvedigital/improvedigitaltest/supplemental/nobid.json new file mode 100644 index 00000000000..49ad0a72955 --- /dev/null +++ b/adapters/improvedigital/improvedigitaltest/supplemental/nobid.json @@ -0,0 +1,52 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + }, + + "httpCalls": [{ + "expectedRequest": { + "uri": "http://localhost/pbs", + "body": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [{"bid":[]}], + "cur": "USD" + } + } + }], + + "expectedBidResponses": [] +} diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/noseatbid.json b/adapters/improvedigital/improvedigitaltest/supplemental/noseatbid.json new file mode 100644 index 00000000000..4d75bedbf81 --- /dev/null +++ b/adapters/improvedigital/improvedigitaltest/supplemental/noseatbid.json @@ -0,0 +1,52 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + }, + + "httpCalls": [{ + "expectedRequest": { + "uri": "http://localhost/pbs", + "body": { + "id": "test-request-id", + "imp": [{ + "id": "test-imp-id", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + }] + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [], + "cur": "USD" + } + } + }], + + "expectedBidResponses": [] +} diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/status_204.json b/adapters/improvedigital/improvedigitaltest/supplemental/status_204.json index 614c06dc89d..25e9ae197e7 100644 --- a/adapters/improvedigital/improvedigitaltest/supplemental/status_204.json +++ b/adapters/improvedigital/improvedigitaltest/supplemental/status_204.json @@ -14,6 +14,7 @@ }, "ext": { "bidder": { + "placementId": 13245 } } } @@ -39,6 +40,7 @@ }, "ext": { "bidder": { + "placementId": 13245 } } } diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/status_400.json b/adapters/improvedigital/improvedigitaltest/supplemental/status_400.json index 4d3fd6fb1fb..837f08b318a 100644 --- a/adapters/improvedigital/improvedigitaltest/supplemental/status_400.json +++ b/adapters/improvedigital/improvedigitaltest/supplemental/status_400.json @@ -14,6 +14,7 @@ }, "ext": { "bidder": { + "placementId": 13245 } } } @@ -39,6 +40,7 @@ }, "ext": { "bidder": { + "placementId": 13245 } } } diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/status_418.json b/adapters/improvedigital/improvedigitaltest/supplemental/status_418.json index 0f97abb5ddc..e847cfbb717 100644 --- a/adapters/improvedigital/improvedigitaltest/supplemental/status_418.json +++ b/adapters/improvedigital/improvedigitaltest/supplemental/status_418.json @@ -14,6 +14,7 @@ }, "ext": { "bidder": { + "placementId": 13245 } } } @@ -39,6 +40,7 @@ }, "ext": { "bidder": { + "placementId": 13245 } } } diff --git a/adapters/improvedigital/improvedigitaltest/supplemental/wrong_impid.json b/adapters/improvedigital/improvedigitaltest/supplemental/wrong_impid.json new file mode 100644 index 00000000000..e874d1d3767 --- /dev/null +++ b/adapters/improvedigital/improvedigitaltest/supplemental/wrong_impid.json @@ -0,0 +1,77 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + } + ] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + } + ] + }, + + "httpCalls": [ + { + "expectedRequest": { + "uri": "http://localhost/pbs", + "body": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + } + ] + }, + "ext": { + "bidder": { + "placementId": 13245 + } + } + } + ] + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [{ + "seat": "improvedigital", + "bid": [{ + "id": "randomid", + "impid": "unknown_impid", + "price": 0.500000, + "adid": "12345678", + "adm": "some-test-ad", + "cid": "987", + "crid": "12345678", + "h": 250, + "w": 300 + }] + }], + "cur": "USD" + } + } + } + ], + + "expectedMakeBidsErrors": [ + "Failed to find impression for ID: \"unknown_impid\"" + ] +} diff --git a/adapters/improvedigital/params_test.go b/adapters/improvedigital/params_test.go index 5ecd5251fdd..13bdd807560 100644 --- a/adapters/improvedigital/params_test.go +++ b/adapters/improvedigital/params_test.go @@ -34,8 +34,9 @@ func TestInvalidParams(t *testing.T) { var validParams = []string{ `{"placementId":13245}`, - `{"size": {"w": 10, "h": 5}}`, - `{"other_optional": true}`, + `{"placementId":13245, "size": {"w":16, "h":9}}`, + `{"publisherId":13245, "placementKey": "slotA"}`, + `{"placementId":13245, "keyValues":{"target1":["foo"],"target2":["bar", "baz"]}}`, } var invalidParams = []string{ @@ -45,9 +46,14 @@ var invalidParams = []string{ `[]`, `true`, `2`, + `{"size": {"w": 10, "h": 5}}`, + `{"other_optional": true}`, `{"size":12345678}`, `{"size":""}`, + `{"placementId":-9}`, + `{"publisherId":9}`, `{"placementId": "1"}`, `{"size": true}`, `{"placementId": true, "size":"1234567"}`, + `{"placementId":13245, "publisherId":13245, "placementKey": "slotA"}`, } diff --git a/adapters/lifestreet/lifestreet.go b/adapters/lifestreet/lifestreet.go index 3df27b7b7e6..c9fff664922 100644 --- a/adapters/lifestreet/lifestreet.go +++ b/adapters/lifestreet/lifestreet.go @@ -5,6 +5,7 @@ import ( "context" "encoding/json" "fmt" + "github.com/prebid/prebid-server/openrtb_ext" "io/ioutil" "net/http" "strings" @@ -71,15 +72,27 @@ func (a *LifestreetAdapter) callOne(ctx context.Context, req *pbs.PBSRequest, re } bid := bidResp.SeatBid[0].Bid[0] + t := openrtb_ext.BidTypeBanner + + if bid.Ext != nil { + var e openrtb_ext.ExtBid + err = json.Unmarshal(bid.Ext, &e) + if err != nil { + return + } + t = e.Prebid.Type + } + result.Bid = &pbs.PBSBid{ - AdUnitCode: bid.ImpID, - Price: bid.Price, - Adm: bid.AdM, - Creative_id: bid.CrID, - Width: bid.W, - Height: bid.H, - DealId: bid.DealID, - NURL: bid.NURL, + AdUnitCode: bid.ImpID, + Price: bid.Price, + Adm: bid.AdM, + Creative_id: bid.CrID, + Width: bid.W, + Height: bid.H, + DealId: bid.DealID, + NURL: bid.NURL, + CreativeMediaType: string(t), } return } diff --git a/adapters/mgid/README.md b/adapters/mgid/README.md new file mode 100644 index 00000000000..32c06fd83be --- /dev/null +++ b/adapters/mgid/README.md @@ -0,0 +1 @@ +Please contact if you would like to build and deploy Prebid server and use it with Mgid. diff --git a/adapters/mgid/mgid.go b/adapters/mgid/mgid.go new file mode 100644 index 00000000000..ed63ebb9f27 --- /dev/null +++ b/adapters/mgid/mgid.go @@ -0,0 +1,169 @@ +package mgid + +import ( + "bytes" + "encoding/json" + "fmt" + "github.com/mxmCherry/openrtb" + "github.com/prebid/prebid-server/adapters" + "github.com/prebid/prebid-server/errortypes" + "github.com/prebid/prebid-server/openrtb_ext" + "net/http" +) + +type MgidAdapter struct { + endpoint string +} + +type ReqExt struct { + PlacementId string `json:"placementId"` + AccountId string `json:"accountId"` +} + +type RespBidExt struct { + CreativeType openrtb_ext.BidType `json:"crtype"` +} + +func (a *MgidAdapter) MakeRequests(request *openrtb.BidRequest) (adapterRequests []*adapters.RequestData, errs []error) { + + adapterReq, errs := a.makeRequest(request) + if adapterReq != nil && len(errs) == 0 { + adapterRequests = append(adapterRequests, adapterReq) + } + + return +} + +func (a *MgidAdapter) makeRequest(request *openrtb.BidRequest) (*adapters.RequestData, []error) { + var errs []error + + path, err := preprocess(request) + if err != nil { + errs = append(errs, err) + return nil, errs + } + + // Last Step + reqJSON, err := json.Marshal(request) + if err != nil { + errs = append(errs, err) + return nil, errs + } + + headers := http.Header{} + headers.Add("Content-Type", "application/json;charset=utf-8") + headers.Add("Accept", "application/json") + + return &adapters.RequestData{ + Method: "POST", + Uri: a.endpoint + path, + Body: reqJSON, + Headers: headers, + }, errs +} + +// Mutate the request to get it ready to send to yieldmo. +func preprocess(request *openrtb.BidRequest) (path string, err error) { + if request.TMax == 0 { + request.TMax = 200 + } + for i := 0; i < len(request.Imp); i++ { + var imp = request.Imp[i] + var bidderExt adapters.ExtImpBidder + + if err := json.Unmarshal(imp.Ext, &bidderExt); err != nil { + return "", &errortypes.BadInput{ + Message: err.Error(), + } + } + + var mgidExt openrtb_ext.ExtImpMgid + + if err := json.Unmarshal(bidderExt.Bidder, &mgidExt); err != nil { + return "", &errortypes.BadInput{ + Message: err.Error(), + } + } + + if path == "" { + path = mgidExt.AccountId + } + request.Imp[i].TagID = mgidExt.PlacementId + + cur := "" + if mgidExt.Currency != "" && mgidExt.Currency != "USD" { + cur = mgidExt.Currency + } + if cur == "" && mgidExt.Cur != "" && mgidExt.Cur != "USD" { + cur = mgidExt.Cur + } + bidfloor := mgidExt.BidFloor + if bidfloor <= 0 { + bidfloor = mgidExt.BidFloor2 + } + if bidfloor > 0 { + request.Imp[i].BidFloor = bidfloor + } + if cur != "" { + request.Imp[i].BidFloorCur = cur + } + } + if path == "" { + return "", &errortypes.BadInput{ + Message: "accountId is not set", + } + } + + return +} + +func (a *MgidAdapter) MakeBids(bidReq *openrtb.BidRequest, unused *adapters.RequestData, response *adapters.ResponseData) (*adapters.BidderResponse, []error) { + if response.StatusCode == http.StatusNoContent { + return nil, nil + } + + if response.StatusCode == http.StatusBadRequest { + return nil, []error{&errortypes.BadInput{ + Message: fmt.Sprintf("Unexpected status code: %d. Run with request.debug = 1 for more info", response.StatusCode), + }} + } + + if response.StatusCode != http.StatusOK { + return nil, []error{&errortypes.BadServerResponse{ + Message: fmt.Sprintf("Unexpected status code: %d. Run with request.debug = 1 for more info", response.StatusCode), + }} + } + + var bidResp openrtb.BidResponse + + if err := json.Unmarshal(response.Body, &bidResp); err != nil { + return nil, []error{err} + } + + bidResponse := adapters.NewBidderResponseWithBidsCapacity(1) + + bidResponse.Currency = bidResp.Cur + + for _, sb := range bidResp.SeatBid { + for i := range sb.Bid { + bidType := openrtb_ext.BidTypeBanner + if len(sb.Bid[i].Ext) > 0 && bytes.Contains(sb.Bid[i].Ext, []byte("crtype")) { + ext := RespBidExt{} + if err := json.Unmarshal(sb.Bid[i].Ext, &ext); err == nil && len(ext.CreativeType) > 0 { + bidType = ext.CreativeType + } + } + bidResponse.Bids = append(bidResponse.Bids, &adapters.TypedBid{ + Bid: &sb.Bid[i], + BidType: bidType, + }) + } + } + return bidResponse, nil +} + +func NewMgidBidder(endpoint string) *MgidAdapter { + return &MgidAdapter{ + endpoint: endpoint, + } +} diff --git a/adapters/mgid/mgid_test.go b/adapters/mgid/mgid_test.go new file mode 100644 index 00000000000..e7c5b9d904f --- /dev/null +++ b/adapters/mgid/mgid_test.go @@ -0,0 +1,10 @@ +package mgid + +import ( + "github.com/prebid/prebid-server/adapters/adapterstest" + "testing" +) + +func TestJsonSamples(t *testing.T) { + adapterstest.RunJSONBidderTest(t, "mgidtest", NewMgidBidder("https://prebid.mgid.com/prebid/")) +} diff --git a/adapters/mgid/mgidtest/exemplary/simple-banner.json b/adapters/mgid/mgidtest/exemplary/simple-banner.json new file mode 100644 index 00000000000..3e227ada92f --- /dev/null +++ b/adapters/mgid/mgidtest/exemplary/simple-banner.json @@ -0,0 +1,135 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + }, + { + "w": 300, + "h": 600 + } + ] + }, + "ext": { + "bidder": { + "accountId": "123", + "placementId": "456" + } + } + } + ], + "device": { + "ua": "test-user-agent", + "ip": "123.123.123.123", + "language": "en", + "dnt": 0 + }, + "site": { + "domain": "www.publisher.com", + "page": "http://www.publisher.com/awesome/site" + }, + "user": { + "buyeruid": "test_reader_id" + } + }, + + "httpCalls": [ + { + "expectedRequest": { + "headers": { + "Accept": ["application/json"], + "Content-Type": ["application/json;charset=utf-8"] + }, + "uri": "https://prebid.mgid.com/prebid/123", + "body": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + }, + { + "w": 300, + "h": 600 + } + ] + }, + "tagid": "456", + "ext": { + "bidder": { + "accountId": "123", + "placementId": "456" + } + } + } + ], + "site": { + "domain": "www.publisher.com", + "page": "http://www.publisher.com/awesome/site" + }, + "tmax": 200, + "user": { + "buyeruid": "test_reader_id" + }, + "device": { + "ua": "test-user-agent", + "ip": "123.123.123.123", + "dnt": 0, + "language": "en" + } + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [ + { + "bid": [ + { + "id": "test-bid-id", + "impid": "test-imp-id", + "price": 3.5, + "nurl": "nurl", + "adm": "some-test-ad", + "w": 300, + "h": 250 + } + ] + } + ] + } + } + } + ], + + "expectedBidResponses": [ + { + "currency": "USD", + "bids": [ + { + "bid": { + "id": "test-bid-id", + "impid": "test-imp-id", + "price": 3.5, + "adm": "some-test-ad", + "nurl": "nurl", + "w": 300, + "h": 250 + }, + "type": "banner" + } + ] + } + ] +} diff --git a/adapters/mgid/mgidtest/exemplary/simple-banner_no_device.json b/adapters/mgid/mgidtest/exemplary/simple-banner_no_device.json new file mode 100644 index 00000000000..58a58b77369 --- /dev/null +++ b/adapters/mgid/mgidtest/exemplary/simple-banner_no_device.json @@ -0,0 +1,129 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + }, + { + "w": 300, + "h": 600 + } + ] + }, + "ext": { + "bidder": { + "accountId": "123", + "placementId": "456", + "bidfloor": 1.1, + "cur": "GBP" + } + } + } + ], + "site": { + "domain": "www.publisher.com", + "page": "http://www.publisher.com/awesome/site" + }, + "user": { + "buyeruid": "test_reader_id" + } + }, + + "httpCalls": [ + { + "expectedRequest": { + "headers": { + "Accept": ["application/json"], + "Content-Type": ["application/json;charset=utf-8"] + }, + "uri": "https://prebid.mgid.com/prebid/123", + "body": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "bidfloor": 1.1, + "bidfloorcur": "GBP", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + }, + { + "w": 300, + "h": 600 + } + ] + }, + "tagid": "456", + "ext": { + "bidder": { + "accountId": "123", + "placementId": "456", + "bidfloor": 1.1, + "cur": "GBP" + } + } + } + ], + "site": { + "domain": "www.publisher.com", + "page": "http://www.publisher.com/awesome/site" + }, + "tmax": 200, + "user": { + "buyeruid": "test_reader_id" + } + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [ + { + "bid": [ + { + "id": "test-bid-id", + "impid": "test-imp-id", + "price": 3.5, + "nurl": "nurl", + "adm": "some-test-ad", + "w": 300, + "h": 250 + } + ] + } + ] + } + } + } + ], + + "expectedBidResponses": [ + { + "currency": "USD", + "bids": [ + { + "bid": { + "id": "test-bid-id", + "impid": "test-imp-id", + "price": 3.5, + "adm": "some-test-ad", + "nurl": "nurl", + "w": 300, + "h": 250 + }, + "type": "banner" + } + ] + } + ] +} diff --git a/adapters/mgid/mgidtest/exemplary/simple-banner_no_device_no_site.json b/adapters/mgid/mgidtest/exemplary/simple-banner_no_device_no_site.json new file mode 100644 index 00000000000..74cafd63691 --- /dev/null +++ b/adapters/mgid/mgidtest/exemplary/simple-banner_no_device_no_site.json @@ -0,0 +1,121 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + }, + { + "w": 300, + "h": 600 + } + ] + }, + "ext": { + "bidder": { + "accountId": "123", + "placementId": "456", + "bidFloor": 1.1, + "currency": "GBP" + } + } + } + ], + "user": { + "buyeruid": "test_reader_id" + } + }, + + "httpCalls": [ + { + "expectedRequest": { + "headers": { + "Accept": ["application/json"], + "Content-Type": ["application/json;charset=utf-8"] + }, + "uri": "https://prebid.mgid.com/prebid/123", + "body": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "bidfloor": 1.1, + "bidfloorcur": "GBP", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + }, + { + "w": 300, + "h": 600 + } + ] + }, + "tagid": "456", + "ext": { + "bidder": { + "accountId": "123", + "placementId": "456", + "bidFloor": 1.1, + "currency": "GBP" + } + } + } + ], + "tmax": 200, + "user": { + "buyeruid": "test_reader_id" + } + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [ + { + "bid": [ + { + "id": "test-bid-id", + "impid": "test-imp-id", + "price": 3.5, + "nurl": "nurl", + "adm": "some-test-ad", + "w": 300, + "h": 250 + } + ] + } + ] + } + } + } + ], + + "expectedBidResponses": [ + { + "currency": "USD", + "bids": [ + { + "bid": { + "id": "test-bid-id", + "impid": "test-imp-id", + "price": 3.5, + "adm": "some-test-ad", + "nurl": "nurl", + "w": 300, + "h": 250 + }, + "type": "banner" + } + ] + } + ] +} diff --git a/adapters/mgid/mgidtest/exemplary/simple-banner_with_crtype.json b/adapters/mgid/mgidtest/exemplary/simple-banner_with_crtype.json new file mode 100644 index 00000000000..87888149119 --- /dev/null +++ b/adapters/mgid/mgidtest/exemplary/simple-banner_with_crtype.json @@ -0,0 +1,141 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + }, + { + "w": 300, + "h": 600 + } + ] + }, + "ext": { + "bidder": { + "accountId": "123", + "placementId": "456" + } + } + } + ], + "device": { + "ua": "test-user-agent", + "ip": "123.123.123.123", + "language": "en", + "dnt": 0 + }, + "site": { + "domain": "www.publisher.com", + "page": "http://www.publisher.com/awesome/site" + }, + "user": { + "buyeruid": "test_reader_id" + } + }, + + "httpCalls": [ + { + "expectedRequest": { + "headers": { + "Accept": ["application/json"], + "Content-Type": ["application/json;charset=utf-8"] + }, + "uri": "https://prebid.mgid.com/prebid/123", + "body": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + }, + { + "w": 300, + "h": 600 + } + ] + }, + "tagid": "456", + "ext": { + "bidder": { + "accountId": "123", + "placementId": "456" + } + } + } + ], + "site": { + "domain": "www.publisher.com", + "page": "http://www.publisher.com/awesome/site" + }, + "tmax": 200, + "user": { + "buyeruid": "test_reader_id" + }, + "device": { + "ua": "test-user-agent", + "ip": "123.123.123.123", + "dnt": 0, + "language": "en" + } + } + }, + "mockResponse": { + "status": 200, + "body": { + "id": "test-request-id", + "seatbid": [ + { + "bid": [ + { + "id": "test-bid-id", + "impid": "test-imp-id", + "price": 3.5, + "nurl": "nurl", + "adm": "some-test-ad", + "w": 300, + "h": 250, + "ext" : { + "crtype": "banner" + } + } + ] + } + ] + } + } + } + ], + + "expectedBidResponses": [ + { + "currency": "USD", + "bids": [ + { + "bid": { + "id": "test-bid-id", + "impid": "test-imp-id", + "price": 3.5, + "adm": "some-test-ad", + "nurl": "nurl", + "w": 300, + "h": 250, + "ext" : { + "crtype": "banner" + } + }, + "type": "banner" + } + ] + } + ] +} diff --git a/adapters/mgid/mgidtest/supplemental/noaccountid.json b/adapters/mgid/mgidtest/supplemental/noaccountid.json new file mode 100644 index 00000000000..2c031c4d70c --- /dev/null +++ b/adapters/mgid/mgidtest/supplemental/noaccountid.json @@ -0,0 +1,29 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + } + ] + }, + "ext": { + "bidder": { + "placementId": "123" + } + } + } + ], + "site": { + "id": "fake-site-id" + } + }, + "expectedMakeRequestsErrors" : [ + "accountId is not set" + ] +} diff --git a/adapters/mgid/mgidtest/supplemental/status_204.json b/adapters/mgid/mgidtest/supplemental/status_204.json new file mode 100644 index 00000000000..6c133e5b924 --- /dev/null +++ b/adapters/mgid/mgidtest/supplemental/status_204.json @@ -0,0 +1,68 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + } + ] + }, + "ext": { + "bidder": { + "placementId": "321", + "accountId": "123" + } + } + } + ], + "site": { + "domain": "fake-site-domain" + } + }, + + "httpCalls": [ + { + "expectedRequest": { + "uri": "https://prebid.mgid.com/prebid/123", + "body": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "tagid": "321", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + } + ] + }, + "ext": { + "bidder": { + "placementId": "321", + "accountId": "123" + } + } + } + ], + "site": { + "domain": "fake-site-domain" + }, + "tmax": 200 + } + }, + "mockResponse": { + "status": 204, + "body": {} + } + } + ], + + "expectedBidResponses": [] +} diff --git a/adapters/mgid/mgidtest/supplemental/status_not200.json b/adapters/mgid/mgidtest/supplemental/status_not200.json new file mode 100644 index 00000000000..4572076ffab --- /dev/null +++ b/adapters/mgid/mgidtest/supplemental/status_not200.json @@ -0,0 +1,70 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + } + ] + }, + "ext": { + "bidder": { + "placementId": "321", + "accountId": "123" + } + } + } + ], + "site": { + "domain": "fake-site-domain" + } + }, + + "httpCalls": [ + { + "expectedRequest": { + "uri": "https://prebid.mgid.com/prebid/123", + "body": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "tagid": "321", + "banner": { + "format": [ + { + "w": 300, + "h": 250 + } + ] + }, + "ext": { + "bidder": { + "placementId": "321", + "accountId": "123" + } + } + } + ], + "site": { + "domain": "fake-site-domain" + }, + "tmax": 200 + } + }, + "mockResponse": { + "status": 404, + "body": {} + } + } + ], + "expectedMakeBidsErrors": [ + "Unexpected status code: 404. Run with request.debug = 1 for more info" + ], + "expectedBidResponses": [] +} diff --git a/adapters/mgid/mgidtest/supplemental/video.json b/adapters/mgid/mgidtest/supplemental/video.json new file mode 100644 index 00000000000..129dbe173b8 --- /dev/null +++ b/adapters/mgid/mgidtest/supplemental/video.json @@ -0,0 +1,57 @@ +{ + "mockBidRequest": { + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "video": { + "mimes": ["video/mp4"], + "protocols": [2, 5], + "w": 1024, + "h": 576 + }, + "ext":{ + "bidder":{ + "accountId": "123", + "placementId": "456" + } + } + } + ] + }, + + "httpCalls": [ + { + "expectedRequest": { + "uri": "https://prebid.mgid.com/prebid/123", + "body": { + "tmax" : 200, + "id": "test-request-id", + "imp": [ + { + "id": "test-imp-id", + "tagid": "456", + "video": { + "mimes": ["video/mp4"], + "protocols": [2, 5], + "w": 1024, + "h": 576 + }, + "ext": { + "bidder": { + "accountId": "123", + "placementId": "456" + } + } + } + ] + } + }, + "mockResponse": { + "status": 204, + "body": {} + } + } + ], + "expectedBidResponses": [] +} diff --git a/adapters/mgid/usersync.go b/adapters/mgid/usersync.go new file mode 100644 index 00000000000..fbdb95f01fc --- /dev/null +++ b/adapters/mgid/usersync.go @@ -0,0 +1,12 @@ +package mgid + +import ( + "text/template" + + "github.com/prebid/prebid-server/adapters" + "github.com/prebid/prebid-server/usersync" +) + +func NewMgidSyncer(temp *template.Template) usersync.Usersyncer { + return adapters.NewSyncer("mgid", 358, temp, adapters.SyncTypeRedirect) +} diff --git a/adapters/mgid/usersync_test.go b/adapters/mgid/usersync_test.go new file mode 100644 index 00000000000..7bb4d1e043b --- /dev/null +++ b/adapters/mgid/usersync_test.go @@ -0,0 +1,18 @@ +package mgid + +import ( + "github.com/stretchr/testify/assert" + "testing" + "text/template" +) + +func TestMgidSyncer(t *testing.T) { + temp := template.Must(template.New("sync-template").Parse("https://cm.mgid.com/m?cdsp=363893&adu=https%3A//external.com%2Fsetuid%3Fbidder%3Dmgid%26gdpr%3D{{.GDPR}}%26gdpr_consent%3D{{.GDPRConsent}}%26uid%3D%7Bmuidn%7D")) + syncer := NewMgidSyncer(temp) + syncInfo, err := syncer.GetUsersyncInfo("0", "") + assert.NoError(t, err) + assert.Equal(t, "https://cm.mgid.com/m?cdsp=363893&adu=https%3A//external.com%2Fsetuid%3Fbidder%3Dmgid%26gdpr%3D0%26gdpr_consent%3D%26uid%3D%7Bmuidn%7D", syncInfo.URL) + assert.Equal(t, "redirect", syncInfo.Type) + assert.EqualValues(t, 358, syncer.GDPRVendorID()) + assert.Equal(t, false, syncInfo.SupportCORS) +} diff --git a/adapters/pubmatic/params_test.go b/adapters/pubmatic/params_test.go index 598a6de6ce1..c8a300b9910 100644 --- a/adapters/pubmatic/params_test.go +++ b/adapters/pubmatic/params_test.go @@ -40,6 +40,9 @@ func TestInvalidParams(t *testing.T) { } var validParams = []string{ + `{"publisherId":"7890"}`, + `{"adSlot":"","publisherId":"7890"}`, + `{"adSlot":"AdTag_Div1","publisherId":"7890"}`, `{"adSlot":"AdTag_Div1@728x90","publisherId":"7890"}`, `{"adSlot":"AdTag_Div1@728x90","publisherId":"7890","keywords":[{"key": "pmZoneID", "value":["zone1"]},{"key": "dctr", "value":[ "v1","v2"]}]}`, `{"adSlot":"AdTag_Div1@728x90","publisherId":"7890","keywords":[{"key": "pmZoneID", "value":["zone1", "zone2"]}], "wrapper":{"profile":5123}}`, @@ -53,7 +56,6 @@ var invalidParams = []string{ `4.2`, `[]`, `{}`, - `{"publisherId":"7890"}`, `{"adSlot":"AdTag_Div1@728x90:0"}`, `{"adSlot":"AdTag_Div1@728x90:0","publisherId":1}`, `{"adSlot":123,"publisherId":"7890"}`, diff --git a/adapters/pubmatic/pubmatic.go b/adapters/pubmatic/pubmatic.go index 18808d8fee9..652af25383f 100644 --- a/adapters/pubmatic/pubmatic.go +++ b/adapters/pubmatic/pubmatic.go @@ -368,6 +368,73 @@ func (a *PubmaticAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapters }}, errs } +// validateAdslot validate the optional adslot string +// valid formats are 'adslot@WxH', 'adslot' and no adslot +func validateAdSlot(adslot string, imp *openrtb.Imp) error { + adSlotStr := strings.TrimSpace(adslot) + + if len(adSlotStr) == 0 { + return nil + } + + if !strings.Contains(adSlotStr, "@") { + imp.TagID = adSlotStr + return nil + } + + adSlot := strings.Split(adSlotStr, "@") + if len(adSlot) == 2 && adSlot[0] != "" && adSlot[1] != "" { + imp.TagID = strings.TrimSpace(adSlot[0]) + + adSize := strings.Split(strings.ToLower(adSlot[1]), "x") + if len(adSize) != 2 { + return errors.New(fmt.Sprintf("Invalid size provided in adSlot %v", adSlotStr)) + } + + width, err := strconv.Atoi(strings.TrimSpace(adSize[0])) + if err != nil { + return errors.New(fmt.Sprintf("Invalid width provided in adSlot %v", adSlotStr)) + } + + heightStr := strings.Split(adSize[1], ":") + height, err := strconv.Atoi(strings.TrimSpace(heightStr[0])) + if err != nil { + return errors.New(fmt.Sprintf("Invalid height provided in adSlot %v", adSlotStr)) + } + + //In case of video, size could be derived from the player size + if imp.Banner != nil { + imp.Banner.H = openrtb.Uint64Ptr(uint64(height)) + imp.Banner.W = openrtb.Uint64Ptr(uint64(width)) + } + } else { + return errors.New(fmt.Sprintf("Invalid adSlot %v", adSlotStr)) + } + + return nil +} + +func assignBannerSize(banner *openrtb.Banner) error { + if banner == nil { + return nil + } + + if banner.W != nil && banner.H != nil { + return nil + } + + if len(banner.Format) == 0 { + return errors.New(fmt.Sprintf("No sizes provided for Banner %v", banner.Format)) + } + + banner.W = new(uint64) + *banner.W = banner.Format[0].W + banner.H = new(uint64) + *banner.H = banner.Format[0].H + + return nil +} + // parseImpressionObject parse the imp to get it ready to send to pubmatic func parseImpressionObject(imp *openrtb.Imp, wrapExt *string, pubID *string) error { // PubMatic supports banner and video impressions. @@ -403,38 +470,14 @@ func parseImpressionObject(imp *openrtb.Imp, wrapExt *string, pubID *string) err *wrapExt = string(pubmaticExt.WrapExt) } - adSlotStr := strings.TrimSpace(pubmaticExt.AdSlot) - - adSlot := strings.Split(adSlotStr, "@") - if len(adSlot) == 2 && adSlot[0] != "" && adSlot[1] != "" { - imp.TagID = strings.TrimSpace(adSlot[0]) - - adSize := strings.Split(strings.ToLower(strings.TrimSpace(adSlot[1])), "x") - if len(adSize) == 2 { - width, err := strconv.Atoi(strings.TrimSpace(adSize[0])) - if err != nil { - return errors.New("Invalid width provided in adSlot") - } + if err := validateAdSlot(strings.TrimSpace(pubmaticExt.AdSlot), imp); err != nil { + return err + } - heightStr := strings.Split(strings.TrimSpace(adSize[1]), ":") - height, err := strconv.Atoi(strings.TrimSpace(heightStr[0])) - if err != nil { - return errors.New("Invalid height provided in adSlot") - } - if imp.Banner != nil { - imp.Banner.H = openrtb.Uint64Ptr(uint64(height)) - imp.Banner.W = openrtb.Uint64Ptr(uint64(width)) - } /* In case of video, params.adSlot would always be in the format adunit@0x0, - so we are not replacing video.W and video.H with size passed in params.adSlot - else { - imp.Video.H = uint64(height) - imp.Video.W = uint64(width) - }*/ - } else { - return errors.New("Invalid size provided in adSlot") + if imp.Banner != nil { + if err := assignBannerSize(imp.Banner); err != nil { + return err } - } else { - return errors.New("Invalid adSlot provided") } if pubmaticExt.Keywords != nil && len(pubmaticExt.Keywords) != 0 { diff --git a/adapters/pubmatic/pubmatictest/supplemental/invalidparam.json b/adapters/pubmatic/pubmatictest/supplemental/invalidparam.json index 93582f65ffd..65c1fd898e5 100644 --- a/adapters/pubmatic/pubmatictest/supplemental/invalidparam.json +++ b/adapters/pubmatic/pubmatictest/supplemental/invalidparam.json @@ -11,7 +11,7 @@ }, "ext": { "bidder": { - "adSlot": "AdTag_Div1", + "adSlot": "AdTag_Div1@", "publisherId": "999" } } @@ -58,6 +58,34 @@ "publisherId": "999" } } + },{ + "id": "test-imp-id-5", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "adSlot": "AdTag_Div1@300x", + "publisherId": "999" + } + } + },{ + "id": "test-imp-id-6", + "banner": { + "format": [{ + "w": 300, + "h": 250 + }] + }, + "ext": { + "bidder": { + "adSlot": "AdTag_Div1@x250", + "publisherId": "999" + } + } }], "site": { "id": "siteID", @@ -68,9 +96,11 @@ }, "expectedMakeRequestsErrors": [ - "Invalid adSlot provided", - "Invalid size provided in adSlot", - "Invalid width provided in adSlot", - "Invalid height provided in adSlot" + "Invalid adSlot AdTag_Div1@", + "Invalid size provided in adSlot AdTag_Div1@300", + "Invalid width provided in adSlot AdTag_Div1@valx250", + "Invalid height provided in adSlot AdTag_Div1@300xval", + "Invalid height provided in adSlot AdTag_Div1@300x", + "Invalid width provided in adSlot AdTag_Div1@x250" ] - } \ No newline at end of file + } diff --git a/adapters/rubicon/rubicon.go b/adapters/rubicon/rubicon.go index efd63d831c5..678912ca8a7 100644 --- a/adapters/rubicon/rubicon.go +++ b/adapters/rubicon/rubicon.go @@ -612,11 +612,13 @@ func (a *RubiconAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapters. request.Device = &deviceCopy } - if thisImp.Video != nil { + isVideo := isVideo(thisImp) + if isVideo { videoCopy := *thisImp.Video videoExt := rubiconVideoExt{Skip: rubiconExt.Video.Skip, SkipDelay: rubiconExt.Video.SkipDelay, RP: rubiconVideoExtRP{SizeID: rubiconExt.Video.VideoSizeID}} videoCopy.Ext, err = json.Marshal(&videoExt) thisImp.Video = &videoCopy + thisImp.Banner = nil } else { primarySizeID, altSizeIDs, err := parseRubiconSizes(thisImp.Banner.Format) if err != nil { @@ -631,6 +633,7 @@ func (a *RubiconAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapters. continue } thisImp.Banner = &bannerCopy + thisImp.Video = nil } siteExt := rubiconSiteExt{RP: rubiconSiteExtRP{SiteID: rubiconExt.SiteId}} @@ -673,6 +676,20 @@ func (a *RubiconAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapters. return requestData, errs } +func isVideo(imp openrtb.Imp) bool { + video := imp.Video + if video != nil { + // Do any other media types exist? Or check required video fields. + return imp.Banner == nil || isFullyPopulatedVideo(video) + } + return false +} + +func isFullyPopulatedVideo(video *openrtb.Video) bool { + // These are just recommended video fields for XAPI + return video.MIMEs != nil && video.Protocols != nil && video.MaxDuration != 0 && video.Linearity != 0 && video.API != nil +} + func (a *RubiconAdapter) MakeBids(internalRequest *openrtb.BidRequest, externalRequest *adapters.RequestData, response *adapters.ResponseData) (*adapters.BidderResponse, []error) { if response.StatusCode == http.StatusNoContent { return nil, nil @@ -706,7 +723,8 @@ func (a *RubiconAdapter) MakeBids(internalRequest *openrtb.BidRequest, externalR bidType := openrtb_ext.BidTypeBanner - if bidReq.Imp[0].Video != nil { + isVideo := isVideo(bidReq.Imp[0]) + if isVideo { bidType = openrtb_ext.BidTypeVideo } diff --git a/adapters/rubicon/rubicon_test.go b/adapters/rubicon/rubicon_test.go index 31ecea55eca..5b2752aa9e2 100644 --- a/adapters/rubicon/rubicon_test.go +++ b/adapters/rubicon/rubicon_test.go @@ -1134,6 +1134,122 @@ func TestOpenRTBRequest(t *testing.T) { } } +func TestOpenRTBRequestWithBannerImpEvenIfImpHasVideo(t *testing.T) { + SIZE_ID := getTestSizes() + bidder := new(RubiconAdapter) + + request := &openrtb.BidRequest{ + ID: "test-request-id", + Imp: []openrtb.Imp{{ + ID: "test-imp-id", + Banner: &openrtb.Banner{ + Format: []openrtb.Format{ + SIZE_ID[15], + SIZE_ID[10], + }, + }, + Video: &openrtb.Video{ + W: 640, + H: 360, + MIMEs: []string{"video/mp4"}, + }, + Ext: json.RawMessage(`{"bidder": { + "zoneId": 8394, + "siteId": 283282, + "accountId": 7891, + "inventory": {"key1" : "val1"}, + "visitor": {"key2" : "val2"} + }}`), + }}, + } + + reqs, errs := bidder.MakeRequests(request) + + if len(errs) > 0 { + t.Errorf("Got unexpected errors while building HTTP requests: %v", errs) + } + if len(reqs) != 1 { + t.Fatalf("Unexpected number of HTTP requests. Got %d. Expected %d", len(reqs), 1) + } + + rubiconReq := &openrtb.BidRequest{} + if err := json.Unmarshal(reqs[0].Body, rubiconReq); err != nil { + t.Errorf("Unexpected error while decoding request: %s", err) + } + + if len(rubiconReq.Imp) != 1 { + t.Fatalf("Unexpected number of request impressions. Got %d. Expected %d", len(rubiconReq.Imp), 1) + } + + if rubiconReq.Imp[0].Video != nil { + t.Error("Unexpected video object in request impression") + } + + if rubiconReq.Imp[0].Banner == nil { + t.Error("Banner object must be in request impression") + } +} + +func TestOpenRTBRequestWithVideoImpEvenIfImpHasBannerButAllRequiredVideoFields(t *testing.T) { + SIZE_ID := getTestSizes() + bidder := new(RubiconAdapter) + + request := &openrtb.BidRequest{ + ID: "test-request-id", + Imp: []openrtb.Imp{{ + ID: "test-imp-id", + Banner: &openrtb.Banner{ + Format: []openrtb.Format{ + SIZE_ID[15], + SIZE_ID[10], + }, + }, + Video: &openrtb.Video{ + W: 640, + H: 360, + MIMEs: []string{"video/mp4"}, + Protocols: []openrtb.Protocol{openrtb.ProtocolVAST10}, + MaxDuration: 30, + Linearity: 1, + API: []openrtb.APIFramework{}, + }, + Ext: json.RawMessage(`{"bidder": { + "zoneId": 8394, + "siteId": 283282, + "accountId": 7891, + "inventory": {"key1" : "val1"}, + "visitor": {"key2" : "val2"} + }}`), + }}, + } + + reqs, errs := bidder.MakeRequests(request) + + if len(errs) > 0 { + t.Errorf("Got unexpected errors while building HTTP requests: %v", errs) + } + if len(reqs) != 1 { + t.Fatalf("Unexpected number of HTTP requests. Got %d. Expected %d", len(reqs), 1) + } + + rubiconReq := &openrtb.BidRequest{} + if err := json.Unmarshal(reqs[0].Body, rubiconReq); err != nil { + t.Errorf("Unexpected error while decoding request: %s", err) + } + + if len(rubiconReq.Imp) != 1 { + t.Fatalf("Unexpected number of request impressions. Got %d. Expected %d", len(rubiconReq.Imp), 1) + } + + if rubiconReq.Imp[0].Banner != nil { + t.Error("Unexpected banner object in request impression") + } + + if rubiconReq.Imp[0].Video == nil { + t.Error("Video object must be in request impression") + } +} + func TestOpenRTBEmptyResponse(t *testing.T) { httpResp := &adapters.ResponseData{ StatusCode: http.StatusNoContent, diff --git a/adapters/sharethrough/butler.go b/adapters/sharethrough/butler.go new file mode 100644 index 00000000000..f16be7a9f17 --- /dev/null +++ b/adapters/sharethrough/butler.go @@ -0,0 +1,191 @@ +package sharethrough + +import ( + "encoding/json" + "fmt" + "github.com/mxmCherry/openrtb" + "github.com/prebid/prebid-server/adapters" + "github.com/prebid/prebid-server/errortypes" + "github.com/prebid/prebid-server/openrtb_ext" + "net/http" + "net/url" + "regexp" + "strconv" +) + +type StrAdSeverParams struct { + Pkey string + BidID string + ConsentRequired bool + ConsentString string + InstantPlayCapable bool + Iframe bool + Height uint64 + Width uint64 +} + +type StrOpenRTBInterface interface { + requestFromOpenRTB(openrtb.Imp, *openrtb.BidRequest) (*adapters.RequestData, error) + responseToOpenRTB(openrtb_ext.ExtImpSharethroughResponse, *adapters.RequestData) (*adapters.BidderResponse, []error) +} + +type StrAdServerUriInterface interface { + buildUri(StrAdSeverParams) string + parseUri(string) (*StrAdSeverParams, error) +} + +type UserAgentParsers struct { + ChromeVersion *regexp.Regexp + ChromeiOSVersion *regexp.Regexp + SafariVersion *regexp.Regexp +} + +type StrUriHelper struct { + BaseURI string +} + +type StrOpenRTBTranslator struct { + UriHelper StrAdServerUriInterface + Util UtilityInterface + UserAgentParsers UserAgentParsers +} + +func (s StrOpenRTBTranslator) requestFromOpenRTB(imp openrtb.Imp, request *openrtb.BidRequest) (*adapters.RequestData, error) { + headers := http.Header{} + headers.Add("Content-Type", "text/plain;charset=utf-8") + headers.Add("Accept", "application/json") + + var strImpExt adapters.ExtImpBidder + if err := json.Unmarshal(imp.Ext, &strImpExt); err != nil { + return nil, err + } + var strImpParams openrtb_ext.ExtImpSharethroughExt + if err := json.Unmarshal(strImpExt.Bidder, &strImpParams); err != nil { + return nil, err + } + + pKey := strImpParams.Pkey + + var height, width uint64 + if len(strImpParams.IframeSize) >= 2 { + height, width = uint64(strImpParams.IframeSize[0]), uint64(strImpParams.IframeSize[1]) + } else { + height, width = s.Util.getPlacementSize(imp.Banner.Format) + } + + return &adapters.RequestData{ + Method: "POST", + Uri: s.UriHelper.buildUri(StrAdSeverParams{ + Pkey: pKey, + BidID: imp.ID, + ConsentRequired: s.Util.gdprApplies(request), + ConsentString: s.Util.gdprConsentString(request), + Iframe: strImpParams.Iframe, + Height: height, + Width: width, + InstantPlayCapable: s.Util.canAutoPlayVideo(request.Device.UA, s.UserAgentParsers), + }), + Body: nil, + Headers: headers, + }, nil +} + +func (s StrOpenRTBTranslator) responseToOpenRTB(strResp openrtb_ext.ExtImpSharethroughResponse, btlrReq *adapters.RequestData) (*adapters.BidderResponse, []error) { + var errs []error + bidResponse := adapters.NewBidderResponse() + + bidResponse.Currency = "USD" + typedBid := &adapters.TypedBid{BidType: openrtb_ext.BidTypeNative} + + if len(strResp.Creatives) == 0 { + errs = append(errs, &errortypes.BadInput{Message: "No creative provided"}) + return nil, errs + } + creative := strResp.Creatives[0] + + btlrParams, parseHBUriErr := s.UriHelper.parseUri(btlrReq.Uri) + if parseHBUriErr != nil { + errs = append(errs, &errortypes.BadInput{Message: parseHBUriErr.Error()}) + return nil, errs + } + + adm, admErr := s.Util.getAdMarkup(strResp, btlrParams) + if admErr != nil { + errs = append(errs, &errortypes.BadServerResponse{Message: admErr.Error()}) + return nil, errs + } + + bid := &openrtb.Bid{ + AdID: strResp.AdServerRequestID, + ID: strResp.BidID, + ImpID: btlrParams.BidID, + Price: creative.CPM, + CID: creative.Metadata.CampaignKey, + CrID: creative.Metadata.CreativeKey, + DealID: creative.Metadata.DealID, + AdM: adm, + H: btlrParams.Height, + W: btlrParams.Width, + } + + typedBid.Bid = bid + bidResponse.Bids = append(bidResponse.Bids, typedBid) + + return bidResponse, errs +} + +func (h StrUriHelper) buildUri(params StrAdSeverParams) string { + v := url.Values{} + v.Set("placement_key", params.Pkey) + v.Set("bidId", params.BidID) + v.Set("consent_required", fmt.Sprintf("%t", params.ConsentRequired)) + v.Set("consent_string", params.ConsentString) + + v.Set("instant_play_capable", fmt.Sprintf("%t", params.InstantPlayCapable)) + v.Set("stayInIframe", fmt.Sprintf("%t", params.Iframe)) + v.Set("height", strconv.FormatUint(params.Height, 10)) + v.Set("width", strconv.FormatUint(params.Width, 10)) + + v.Set("supplyId", supplyId) + v.Set("strVersion", strVersion) + + return h.BaseURI + "?" + v.Encode() +} + +func (h StrUriHelper) parseUri(uri string) (*StrAdSeverParams, error) { + btlrUrl, err := url.Parse(uri) + if err != nil { + return nil, err + } + + params := btlrUrl.Query() + height, err := strconv.ParseUint(params.Get("height"), 10, 64) + if err != nil { + return nil, err + } + + width, err := strconv.ParseUint(params.Get("width"), 10, 64) + if err != nil { + return nil, err + } + + stayInIframe, err := strconv.ParseBool(params.Get("stayInIframe")) + if err != nil { + stayInIframe = false + } + + consentRequired, err := strconv.ParseBool(params.Get("consent_required")) + if err != nil { + consentRequired = false + } + + return &StrAdSeverParams{ + Pkey: params.Get("placement_key"), + BidID: params.Get("bidId"), + Iframe: stayInIframe, + Height: height, + Width: width, + ConsentRequired: consentRequired, + ConsentString: params.Get("consent_string"), + }, nil +} diff --git a/adapters/sharethrough/butler_test.go b/adapters/sharethrough/butler_test.go new file mode 100644 index 00000000000..2e258ecf775 --- /dev/null +++ b/adapters/sharethrough/butler_test.go @@ -0,0 +1,399 @@ +package sharethrough + +import ( + "fmt" + "github.com/mxmCherry/openrtb" + "github.com/prebid/prebid-server/adapters" + "github.com/prebid/prebid-server/errortypes" + "github.com/prebid/prebid-server/openrtb_ext" + "net/http" + "regexp" + "strings" + "testing" +) + +type MockUtil struct { + mockCanAutoPlayVideo func() bool + mockGdprApplies func() bool + mockGdprConsentString func() string + mockGenerateHBUri func() string + mockGetPlacementSize func() (uint64, uint64) + UtilityInterface +} + +func (m MockUtil) canAutoPlayVideo(userAgent string) bool { + return m.mockCanAutoPlayVideo() +} + +func (m MockUtil) gdprApplies(request *openrtb.BidRequest) bool { + return m.mockGdprApplies() +} + +func (m MockUtil) gdprConsentString(bidRequest *openrtb.BidRequest) string { + return m.mockGdprConsentString() +} + +func (m MockUtil) generateHBUri(baseUrl string, params StrAdSeverParams, app *openrtb.App) string { + return m.mockGenerateHBUri() +} + +func (m MockUtil) getPlacementSize(formats []openrtb.Format) (height uint64, width uint64) { + return m.mockGetPlacementSize() +} + +func assertRequestDataEquals(t *testing.T, testName string, expected *adapters.RequestData, actual *adapters.RequestData) { + t.Logf("Test case: %s\n", testName) + if expected.Method != actual.Method { + t.Errorf("Method mismatch: expected %s got %s\n", expected.Method, actual.Method) + } + if expected.Uri != actual.Uri { + t.Errorf("Uri mismatch: expected %s got %s\n", expected.Uri, actual.Uri) + } + if len(expected.Body) != len(actual.Body) { + t.Errorf("Body mismatch: expected %s got %s\n", expected.Body, actual.Body) + } + for headerIndex, expectedHeader := range expected.Headers { + if expectedHeader[0] != actual.Headers[headerIndex][0] { + t.Errorf("Header %s mismatch: expected %s got %s\n", headerIndex, expectedHeader[0], actual.Headers[headerIndex][0]) + } + } +} + +func TestSuccessRequestFromOpenRTB(t *testing.T) { + tests := map[string]struct { + inputImp openrtb.Imp + inputReq *openrtb.BidRequest + expected *adapters.RequestData + }{ + "Generates the correct AdServer request from Imp": { + inputImp: openrtb.Imp{ + ID: "abc", + Ext: []byte(`{ "bidder": {"pkey": "pkey", "iframe": true, "iframeSize": [10, 20]} }`), + Banner: &openrtb.Banner{ + Format: []openrtb.Format{{H: 30, W: 40}}, + }, + }, + inputReq: &openrtb.BidRequest{ + App: &openrtb.App{Ext: []byte(`{}`)}, + Device: &openrtb.Device{ + UA: "Android Chome/60", + }, + }, + expected: &adapters.RequestData{ + Method: "POST", + Uri: "http://abc.com", + Body: nil, + Headers: http.Header{ + "Content-Type": []string{"text/plain;charset=utf-8"}, + "Accept": []string{"application/json"}, + }, + }, + }, + } + + mockUriHelper := MockStrUriHelper{ + mockBuildUri: func() string { + return "http://abc.com" + }, + } + + adServer := StrOpenRTBTranslator{UriHelper: mockUriHelper, Util: Util{}, UserAgentParsers: UserAgentParsers{ + ChromeVersion: regexp.MustCompile(`Chrome\/(?P\d+)`), + ChromeiOSVersion: regexp.MustCompile(`CriOS\/(?P\d+)`), + SafariVersion: regexp.MustCompile(`Version\/(?P\d+)`), + }} + for testName, test := range tests { + outputSuccess, outputError := adServer.requestFromOpenRTB(test.inputImp, test.inputReq) + assertRequestDataEquals(t, testName, test.expected, outputSuccess) + if outputError != nil { + t.Errorf("Expected no errors, got %s\n", outputError) + } + } +} + +func assertBidderResponseEquals(t *testing.T, testName string, expected adapters.BidderResponse, actual adapters.BidderResponse) { + t.Logf("Test case: %s\n", testName) + if len(expected.Bids) != len(actual.Bids) { + t.Errorf("Expected %d bids in BidResponse, got %d\n", len(expected.Bids), len(actual.Bids)) + return + } + for index, expectedTypedBid := range expected.Bids { + if expectedTypedBid.BidType != actual.Bids[index].BidType { + t.Errorf("Bid[%d]: Type mismatch, expected %s got %s\n", index, expectedTypedBid.BidType, actual.Bids[index].BidType) + } + if expectedTypedBid.Bid.AdID != actual.Bids[index].Bid.AdID { + t.Errorf("Bid[%d]: AdID mismatch, expected %s got %s\n", index, expectedTypedBid.Bid.AdID, actual.Bids[index].Bid.AdID) + } + if expectedTypedBid.Bid.ID != actual.Bids[index].Bid.ID { + t.Errorf("Bid[%d]: ID mismatch, expected %s got %s\n", index, expectedTypedBid.Bid.ID, actual.Bids[index].Bid.ID) + } + if expectedTypedBid.Bid.ImpID != actual.Bids[index].Bid.ImpID { + t.Errorf("Bid[%d]: ImpID mismatch, expected %s got %s\n", index, expectedTypedBid.Bid.ImpID, actual.Bids[index].Bid.ImpID) + } + if expectedTypedBid.Bid.Price != actual.Bids[index].Bid.Price { + t.Errorf("Bid[%d]: Price mismatch, expected %f got %f\n", index, expectedTypedBid.Bid.Price, actual.Bids[index].Bid.Price) + } + if expectedTypedBid.Bid.CID != actual.Bids[index].Bid.CID { + t.Errorf("Bid[%d]: CID mismatch, expected %s got %s\n", index, expectedTypedBid.Bid.CID, actual.Bids[index].Bid.CID) + } + if expectedTypedBid.Bid.CrID != actual.Bids[index].Bid.CrID { + t.Errorf("Bid[%d]: CrID mismatch, expected %s got %s\n", index, expectedTypedBid.Bid.CrID, actual.Bids[index].Bid.CrID) + } + if expectedTypedBid.Bid.DealID != actual.Bids[index].Bid.DealID { + t.Errorf("Bid[%d]: DealID mismatch, expected %s got %s\n", index, expectedTypedBid.Bid.DealID, actual.Bids[index].Bid.DealID) + } + if expectedTypedBid.Bid.H != actual.Bids[index].Bid.H { + t.Errorf("Bid[%d]: H mismatch, expected %d got %d\n", index, expectedTypedBid.Bid.H, actual.Bids[index].Bid.H) + } + if expectedTypedBid.Bid.W != actual.Bids[index].Bid.W { + t.Errorf("Bid[%d]: W mismatch, expected %d got %d\n", index, expectedTypedBid.Bid.W, actual.Bids[index].Bid.W) + } + } +} + +func TestSuccessResponseToOpenRTB(t *testing.T) { + tests := map[string]struct { + inputButlerReq *adapters.RequestData + inputStrResp openrtb_ext.ExtImpSharethroughResponse + expectedSuccess *adapters.BidderResponse + expectedErrors []error + }{ + "Generates expected openRTB bid response": { + inputButlerReq: &adapters.RequestData{ + Uri: "http://uri.com?placement_key=pkey&bidId=bidid&height=20&width=30", + }, + inputStrResp: openrtb_ext.ExtImpSharethroughResponse{ + AdServerRequestID: "arid", + BidID: "bid", + Creatives: []openrtb_ext.ExtImpSharethroughCreative{{ + CPM: 10, + Metadata: openrtb_ext.ExtImpSharethroughCreativeMetadata{ + CampaignKey: "cmpKey", + CreativeKey: "creaKey", + DealID: "dealId", + }, + }}, + }, + expectedSuccess: &adapters.BidderResponse{ + Bids: []*adapters.TypedBid{{ + BidType: openrtb_ext.BidTypeNative, + Bid: &openrtb.Bid{ + AdID: "arid", + ID: "bid", + ImpID: "bidid", + Price: 10, + CID: "cmpKey", + CrID: "creaKey", + DealID: "dealId", + H: 20, + W: 30, + }, + }}, + }, + expectedErrors: []error{}, + }, + } + + adServer := StrOpenRTBTranslator{Util: Util{}, UriHelper: StrUriHelper{}} + for testName, test := range tests { + outputSuccess, outputErrors := adServer.responseToOpenRTB(test.inputStrResp, test.inputButlerReq) + assertBidderResponseEquals(t, testName, *test.expectedSuccess, *outputSuccess) + if len(outputErrors) != len(test.expectedErrors) { + t.Errorf("Expected %d errors, got %d\n", len(test.expectedErrors), len(outputErrors)) + } + } +} + +func TestFailResponseToOpenRTB(t *testing.T) { + tests := map[string]struct { + inputButlerReq *adapters.RequestData + inputStrResp openrtb_ext.ExtImpSharethroughResponse + expectedSuccess *adapters.BidderResponse + expectedErrors []error + }{ + "Returns nil if no creatives provided": { + inputButlerReq: &adapters.RequestData{}, + inputStrResp: openrtb_ext.ExtImpSharethroughResponse{ + Creatives: []openrtb_ext.ExtImpSharethroughCreative{}, + }, + expectedSuccess: nil, + expectedErrors: []error{ + &errortypes.BadInput{Message: "No creative provided"}, + }, + }, + "Returns nil if failed to parse Uri": { + inputButlerReq: &adapters.RequestData{ + Uri: "wrong format url", + }, + inputStrResp: openrtb_ext.ExtImpSharethroughResponse{ + Creatives: []openrtb_ext.ExtImpSharethroughCreative{{}}, + }, + expectedSuccess: nil, + expectedErrors: []error{ + &errortypes.BadInput{Message: `strconv.ParseUint: parsing "": invalid syntax`}, + }, + }, + } + + adServer := StrOpenRTBTranslator{UriHelper: StrUriHelper{}} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + outputSuccess, outputErrors := adServer.responseToOpenRTB(test.inputStrResp, test.inputButlerReq) + + if test.expectedSuccess != outputSuccess { + t.Errorf("Expected result %+v, got %+v\n", test.expectedSuccess, outputSuccess) + } + + if len(outputErrors) != len(test.expectedErrors) { + t.Errorf("Expected %d errors, got %d\n", len(test.expectedErrors), len(outputErrors)) + } + + for index, expectedError := range test.expectedErrors { + if fmt.Sprintf("%T", expectedError) != fmt.Sprintf("%T", outputErrors[index]) { + t.Errorf("Error type mismatch, expected %T, got %T\n", expectedError, outputErrors[index]) + } + if expectedError.Error() != outputErrors[index].Error() { + t.Errorf("Expected error %s, got %s\n", expectedError.Error(), outputErrors[index].Error()) + } + } + } +} + +func TestBuildUri(t *testing.T) { + tests := map[string]struct { + inputParams StrAdSeverParams + inputApp *openrtb.App + expected []string + }{ + "Generates expected URL, appending all params": { + inputParams: StrAdSeverParams{ + Pkey: "pkey", + BidID: "bid", + ConsentRequired: true, + ConsentString: "consent", + InstantPlayCapable: true, + Iframe: false, + Height: 20, + Width: 30, + }, + expected: []string{ + "http://abc.com?", + "placement_key=pkey", + "bidId=bid", + "consent_required=true", + "consent_string=consent", + "instant_play_capable=true", + "stayInIframe=false", + "height=20", + "width=30", + "supplyId=FGMrCMMc", + "strVersion=1.0.0", + }, + }, + } + + uriHelper := StrUriHelper{BaseURI: "http://abc.com"} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + output := uriHelper.buildUri(test.inputParams) + + for _, uriParam := range test.expected { + if !strings.Contains(output, uriParam) { + t.Errorf("Expected %s to be found in URL, got %s\n", uriParam, output) + } + } + } +} + +func assertStrAdServerParamsEquals(t *testing.T, testName string, expected *StrAdSeverParams, actual *StrAdSeverParams) { + t.Logf("Test case: %s\n", testName) + if expected.Pkey != actual.Pkey { + t.Errorf("Expected Pkey to be %s, got %s\n", expected.Pkey, actual.Pkey) + } + if expected.BidID != actual.BidID { + t.Errorf("Expected BidID to be %s, got %s\n", expected.BidID, actual.BidID) + } + if expected.Iframe != actual.Iframe { + t.Errorf("Expected Iframe to be %t, got %t\n", expected.Iframe, actual.Iframe) + } + if expected.Height != actual.Height { + t.Errorf("Expected Height to be %d, got %d\n", expected.Height, actual.Height) + } + if expected.Width != actual.Width { + t.Errorf("Expected Width to be %d, got %d\n", expected.Width, actual.Width) + } + if expected.ConsentRequired != actual.ConsentRequired { + t.Errorf("Expected ConsentRequired to be %t, got %t\n", expected.ConsentRequired, actual.ConsentRequired) + } + if expected.ConsentString != actual.ConsentString { + t.Errorf("Expected ConsentString to be %s, got %s\n", expected.ConsentString, actual.ConsentString) + } +} + +func TestSuccessParseUri(t *testing.T) { + tests := map[string]struct { + input string + expectedSuccess *StrAdSeverParams + }{ + "Decodes URI successfully": { + input: "http://abc.com?placement_key=pkey&bidId=bid&consent_required=true&consent_string=consent&instant_play_capable=true&stayInIframe=false&height=20&width=30&hbVersion=1&supplyId=FGMrCMMc&strVersion=1.0.0", + expectedSuccess: &StrAdSeverParams{ + Pkey: "pkey", + BidID: "bid", + Iframe: false, + Height: 20, + Width: 30, + ConsentRequired: true, + ConsentString: "consent", + }, + }, + } + + uriHelper := StrUriHelper{} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + output, actualError := uriHelper.parseUri(test.input) + + assertStrAdServerParamsEquals(t, testName, test.expectedSuccess, output) + if actualError != nil { + t.Errorf("Expected no errors, got %s\n", actualError) + } + } +} + +func TestFailParseUri(t *testing.T) { + tests := map[string]struct { + input string + expectedError string + }{ + "Fails decoding if unable to parse URI": { + input: "wrong URI", + expectedError: `strconv.ParseUint: parsing "": invalid syntax`, + }, + "Fails decoding if height not provided": { + input: "http://abc.com?width=10", + expectedError: `strconv.ParseUint: parsing "": invalid syntax`, + }, + "Fails decoding if width not provided": { + input: "http://abc.com?height=10", + expectedError: `strconv.ParseUint: parsing "": invalid syntax`, + }, + } + + uriHelper := StrUriHelper{} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + output, actualError := uriHelper.parseUri(test.input) + + if output != nil { + t.Errorf("Expected return value nil, got %+v\n", output) + } + if actualError == nil { + t.Errorf("Expected error not to be nil\n") + break + } + if actualError.Error() != test.expectedError { + t.Errorf("Expected error '%s', got '%s'\n", test.expectedError, actualError.Error()) + } + } +} diff --git a/adapters/sharethrough/params_test.go b/adapters/sharethrough/params_test.go new file mode 100644 index 00000000000..416f459341d --- /dev/null +++ b/adapters/sharethrough/params_test.go @@ -0,0 +1,58 @@ +package sharethrough + +import ( + "encoding/json" + "testing" + + "github.com/prebid/prebid-server/openrtb_ext" +) + +func TestValidParams(t *testing.T) { + validator, err := openrtb_ext.NewBidderParamsValidator("../../static/bidder-params") + if err != nil { + t.Fatalf("Failed to fetch the json-schemas. %v", err) + } + + for _, validParam := range validParams { + if err := validator.Validate(openrtb_ext.BidderSharethrough, json.RawMessage(validParam)); err != nil { + t.Errorf("Schema rejected Sharethrough params: %s", validParam) + } + } +} + +func TestInvalidParams(t *testing.T) { + validator, err := openrtb_ext.NewBidderParamsValidator("../../static/bidder-params") + if err != nil { + t.Fatalf("Failed to fetch the json-schemas. %v", err) + } + + for _, invalidParam := range invalidParams { + if err := validator.Validate(openrtb_ext.BidderSharethrough, json.RawMessage(invalidParam)); err == nil { + t.Errorf("Schema allowed unexpected params: %s", invalidParam) + } + } +} + +var validParams = []string{ + `{"pkey": "123"}`, + `{"pkey": "123", "iframe": true}`, + `{"pkey": "abc", "iframe": false}`, + `{"pkey": "abc123", "iframe": true, "iframeSize": [20, 20]}`, +} + +var invalidParams = []string{ + ``, + `null`, + `true`, + `5`, + `4.2`, + `[]`, + `{}`, + `{"pkey": 123}`, + `{"iframe": 123}`, + `{"iframeSize": [20, 20]}`, + `{"pkey": 123, "iframe": 123}`, + `{"pkey": 123, "iframe": true, "iframeSize": [20]}`, + `{"pkey": 123, "iframe": true, "iframeSize": []}`, + `{"pkey": 123, "iframe": true, "iframeSize": 123}`, +} diff --git a/adapters/sharethrough/sharethrough.go b/adapters/sharethrough/sharethrough.go new file mode 100644 index 00000000000..ea6a35cf619 --- /dev/null +++ b/adapters/sharethrough/sharethrough.go @@ -0,0 +1,72 @@ +package sharethrough + +import ( + "encoding/json" + "fmt" + "github.com/mxmCherry/openrtb" + "github.com/prebid/prebid-server/adapters" + "github.com/prebid/prebid-server/errortypes" + "github.com/prebid/prebid-server/openrtb_ext" + "net/http" + "regexp" +) + +const supplyId = "FGMrCMMc" +const strVersion = "1.0.0" + +func NewSharethroughBidder(endpoint string) *SharethroughAdapter { + return &SharethroughAdapter{ + AdServer: StrOpenRTBTranslator{ + UriHelper: StrUriHelper{BaseURI: endpoint}, + Util: Util{}, + UserAgentParsers: UserAgentParsers{ + ChromeVersion: regexp.MustCompile(`Chrome\/(?P\d+)`), + ChromeiOSVersion: regexp.MustCompile(`CriOS\/(?P\d+)`), + SafariVersion: regexp.MustCompile(`Version\/(?P\d+)`), + }, + }, + } +} + +type SharethroughAdapter struct { + AdServer StrOpenRTBInterface +} + +func (a SharethroughAdapter) MakeRequests(request *openrtb.BidRequest) ([]*adapters.RequestData, []error) { + var reqs []*adapters.RequestData + + for i := 0; i < len(request.Imp); i++ { + req, err := a.AdServer.requestFromOpenRTB(request.Imp[i], request) + + if err != nil { + return nil, []error{err} + } + reqs = append(reqs, req) + } + + // We never add to the errs slice (early return), so we just create an empty one to return + return reqs, []error{} +} + +func (a SharethroughAdapter) MakeBids(internalRequest *openrtb.BidRequest, externalRequest *adapters.RequestData, response *adapters.ResponseData) (*adapters.BidderResponse, []error) { + if response.StatusCode == http.StatusNoContent { + return nil, nil + } + + if response.StatusCode == http.StatusBadRequest { + return nil, []error{&errortypes.BadInput{ + Message: fmt.Sprintf("Unexpected status code: %d. Run with request.debug = 1 for more info", response.StatusCode), + }} + } + + if response.StatusCode != http.StatusOK { + return nil, []error{fmt.Errorf("unexpected status code: %d. Run with request.debug = 1 for more info", response.StatusCode)} + } + + var strBidResp openrtb_ext.ExtImpSharethroughResponse + if err := json.Unmarshal(response.Body, &strBidResp); err != nil { + return nil, []error{err} + } + + return a.AdServer.responseToOpenRTB(strBidResp, externalRequest) +} diff --git a/adapters/sharethrough/sharethrough_test.go b/adapters/sharethrough/sharethrough_test.go new file mode 100644 index 00000000000..b1d252d36d5 --- /dev/null +++ b/adapters/sharethrough/sharethrough_test.go @@ -0,0 +1,245 @@ +package sharethrough + +import ( + "fmt" + "github.com/mxmCherry/openrtb" + "github.com/prebid/prebid-server/adapters" + "github.com/prebid/prebid-server/errortypes" + "github.com/prebid/prebid-server/openrtb_ext" + "net/http" + "testing" +) + +type MockStrAdServer struct { + mockRequestFromOpenRTB func() (*adapters.RequestData, error) + mockResponseToOpenRTB func() (*adapters.BidderResponse, []error) + + StrOpenRTBInterface +} + +func (m MockStrAdServer) requestFromOpenRTB(imp openrtb.Imp, request *openrtb.BidRequest) (*adapters.RequestData, error) { + return m.mockRequestFromOpenRTB() +} + +func (m MockStrAdServer) responseToOpenRTB(strResp openrtb_ext.ExtImpSharethroughResponse, btlrReq *adapters.RequestData) (*adapters.BidderResponse, []error) { + return m.mockResponseToOpenRTB() +} + +type MockStrUriHelper struct { + mockBuildUri func() string + mockParseUri func() (*StrAdSeverParams, error) + + StrAdServerUriInterface +} + +func (m MockStrUriHelper) buildUri(params StrAdSeverParams) string { + return m.mockBuildUri() +} + +func (m MockStrUriHelper) parseUri(uri string) (*StrAdSeverParams, error) { + return m.mockParseUri() +} + +func TestSuccessMakeRequests(t *testing.T) { + stubReq := &adapters.RequestData{ + Method: "POST", + Uri: "http://test.com", + Body: nil, + Headers: http.Header{ + "Content-Type": []string{"text/plain;charset=utf-8"}, + "Accept": []string{"application/json"}, + }, + } + + tests := map[string]struct { + input *openrtb.BidRequest + expected []*adapters.RequestData + }{ + "Generates expected Request": { + input: &openrtb.BidRequest{ + App: &openrtb.App{Ext: []byte(`{}`)}, + Device: &openrtb.Device{ + UA: "Android Chome/60", + }, + Imp: []openrtb.Imp{{ + ID: "abc", + Ext: []byte(`{"pkey": "pkey", "iframe": true, "iframeSize": [10, 20]}`), + Banner: &openrtb.Banner{ + Format: []openrtb.Format{{H: 30, W: 40}}, + }, + }}, + }, + expected: []*adapters.RequestData{stubReq}, + }, + } + + mockAdServer := MockStrAdServer{ + mockRequestFromOpenRTB: func() (*adapters.RequestData, error) { + return stubReq, nil + }, + } + + adapter := SharethroughAdapter{AdServer: mockAdServer} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + output, actualErrors := adapter.MakeRequests(test.input) + + if len(output) != 1 { + t.Errorf("Expected one request in result, got %d\n", len(output)) + return + } + + assertRequestDataEquals(t, testName, test.expected[0], output[0]) + if len(actualErrors) != 0 { + t.Errorf("Expected no errors, got %d\n", len(actualErrors)) + } + } +} + +func TestFailureMakeRequests(t *testing.T) { + tests := map[string]struct { + input *openrtb.BidRequest + expected string + }{ + "Returns nil if failed to generate request": { + input: &openrtb.BidRequest{ + App: &openrtb.App{Ext: []byte(`{}`)}, + Device: &openrtb.Device{ + UA: "Android Chome/60", + }, + Imp: []openrtb.Imp{{ + ID: "abc", + Ext: []byte(`{"pkey": "pkey", "iframe": true, "iframeSize": [10, 20]}`), + Banner: &openrtb.Banner{ + Format: []openrtb.Format{{H: 30, W: 40}}, + }, + }}, + }, + expected: "error generating request", + }, + } + + mockAdServer := MockStrAdServer{ + mockRequestFromOpenRTB: func() (*adapters.RequestData, error) { + return nil, fmt.Errorf("error generating request") + }, + } + + adapter := SharethroughAdapter{AdServer: mockAdServer} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + output, actualErrors := adapter.MakeRequests(test.input) + + if output != nil { + t.Errorf("Expected result to be nil, got %d elements\n", len(output)) + } + if len(actualErrors) != 1 { + t.Errorf("Expected one error, got %d\n", len(actualErrors)) + } + if actualErrors[0].Error() != test.expected { + t.Errorf("Error mismatch: expected '%s' got '%s'\n", test.expected, actualErrors[0].Error()) + } + } +} + +func TestSuccessMakeBids(t *testing.T) { + stubBidderResponse := adapters.BidderResponse{} + + tests := map[string]struct { + inputResponse *adapters.ResponseData + expected *adapters.BidderResponse + }{ + "Returns nil,nil if ad server responded with no content": { + inputResponse: &adapters.ResponseData{ + StatusCode: http.StatusNoContent, + }, + expected: nil, + }, + "Generates response if ad server responded with 200": { + inputResponse: &adapters.ResponseData{ + StatusCode: http.StatusOK, + Body: []byte(`{}`), + }, + expected: &stubBidderResponse, + }, + } + + mockAdServer := MockStrAdServer{ + mockResponseToOpenRTB: func() (*adapters.BidderResponse, []error) { + return &stubBidderResponse, []error{} + }, + } + + adapter := SharethroughAdapter{AdServer: mockAdServer} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + response, errors := adapter.MakeBids(&openrtb.BidRequest{}, &adapters.RequestData{}, test.inputResponse) + if len(errors) > 0 { + t.Errorf("Expected no errors, got %d\n", len(errors)) + } + if response != test.expected { + t.Errorf("Response mismatch: expected '%+v' got '%+v'\n", test.expected, response) + } + } +} + +func TestFailureMakeBids(t *testing.T) { + tests := map[string]struct { + inputResponse *adapters.ResponseData + expected []error + }{ + "Returns BadInput error if ad server responds with BadRequest": { + inputResponse: &adapters.ResponseData{ + StatusCode: http.StatusBadRequest, + }, + expected: []error{&errortypes.BadInput{ + Message: fmt.Sprintf("Unexpected status code: %d. Run with request.debug = 1 for more info", http.StatusBadRequest), + }}, + }, + "Returns default error if ad server does not respond with Status OK": { + inputResponse: &adapters.ResponseData{ + StatusCode: http.StatusInternalServerError, + }, + expected: []error{fmt.Errorf("unexpected status code: %d. Run with request.debug = 1 for more info", http.StatusInternalServerError)}, + }, + "Returns error if failed parsing body": { + inputResponse: &adapters.ResponseData{ + StatusCode: http.StatusOK, + Body: []byte(`{ wrong json`), + }, + expected: []error{fmt.Errorf("invalid character 'w' looking for beginning of object key string")}, + }, + "Passes by errors from responseToOpenRTB": { + inputResponse: &adapters.ResponseData{ + StatusCode: http.StatusOK, + Body: []byte(`{}`), + }, + expected: []error{fmt.Errorf("failed in responseToOpenRTB")}, + }, + } + + mockAdServer := MockStrAdServer{ + mockResponseToOpenRTB: func() (*adapters.BidderResponse, []error) { + return nil, []error{fmt.Errorf("failed in responseToOpenRTB")} + }, + } + + adapter := SharethroughAdapter{AdServer: mockAdServer} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + response, errors := adapter.MakeBids(&openrtb.BidRequest{}, &adapters.RequestData{}, test.inputResponse) + if response != nil { + t.Errorf("Expected response to be nil, got %+v\n", response) + } + if len(errors) != 1 { + t.Errorf("Expected no errors, got %d\n", len(errors)) + } + if errors[0].Error() != test.expected[0].Error() { + t.Errorf("Error mismatch: expected '%s' got '%s'\n", test.expected[0].Error(), errors[0].Error()) + } + } +} diff --git a/adapters/sharethrough/sharethroughtest/params/race/banner.json b/adapters/sharethrough/sharethroughtest/params/race/banner.json new file mode 100644 index 00000000000..6702f4c2965 --- /dev/null +++ b/adapters/sharethrough/sharethroughtest/params/race/banner.json @@ -0,0 +1,5 @@ +{ + "pkey": "abc123", + "iframe": true, + "iframeSize": [50, 50] +} \ No newline at end of file diff --git a/adapters/sharethrough/sharethroughtest/params/race/native.json b/adapters/sharethrough/sharethroughtest/params/race/native.json new file mode 100644 index 00000000000..6702f4c2965 --- /dev/null +++ b/adapters/sharethrough/sharethroughtest/params/race/native.json @@ -0,0 +1,5 @@ +{ + "pkey": "abc123", + "iframe": true, + "iframeSize": [50, 50] +} \ No newline at end of file diff --git a/adapters/sharethrough/usersync.go b/adapters/sharethrough/usersync.go new file mode 100644 index 00000000000..a951fcd6a0a --- /dev/null +++ b/adapters/sharethrough/usersync.go @@ -0,0 +1,11 @@ +package sharethrough + +import ( + "github.com/prebid/prebid-server/adapters" + "github.com/prebid/prebid-server/usersync" + "text/template" +) + +func NewSharethroughSyncer(temp *template.Template) usersync.Usersyncer { + return adapters.NewSyncer("sharethrough", 80, temp, adapters.SyncTypeRedirect) +} diff --git a/adapters/sharethrough/usersync_test.go b/adapters/sharethrough/usersync_test.go new file mode 100644 index 00000000000..0cfc177f254 --- /dev/null +++ b/adapters/sharethrough/usersync_test.go @@ -0,0 +1,19 @@ +package sharethrough + +import ( + "github.com/stretchr/testify/assert" + "testing" + "text/template" +) + +func TestSharethroughSyncer(t *testing.T) { + temp := template.Must(template.New("sync-template").Parse("https://match.sharethrough.com?gdpr={{.GDPR}}&gdpr_consent={{.GDPRConsent}}")) + syncer := NewSharethroughSyncer(temp) + syncInfo, err := syncer.GetUsersyncInfo("0", "") + assert.NoError(t, err) + assert.Equal(t, "https://match.sharethrough.com?gdpr=0&gdpr_consent=", syncInfo.URL) + assert.Equal(t, "redirect", syncInfo.Type) + assert.EqualValues(t, 80, syncer.GDPRVendorID()) + assert.Equal(t, false, syncInfo.SupportCORS) + assert.Equal(t, "sharethrough", syncer.FamilyName()) +} diff --git a/adapters/sharethrough/utils.go b/adapters/sharethrough/utils.go new file mode 100644 index 00000000000..08f0ae3ae39 --- /dev/null +++ b/adapters/sharethrough/utils.go @@ -0,0 +1,193 @@ +package sharethrough + +import ( + "bytes" + "encoding/base64" + "encoding/json" + "fmt" + "github.com/buger/jsonparser" + "github.com/mxmCherry/openrtb" + "github.com/prebid/prebid-server/openrtb_ext" + "html/template" + "regexp" + "strconv" +) + +const minChromeVersion = 53 +const minSafariVersion = 10 + +type UtilityInterface interface { + gdprApplies(*openrtb.BidRequest) bool + gdprConsentString(*openrtb.BidRequest) string + + getAdMarkup(openrtb_ext.ExtImpSharethroughResponse, *StrAdSeverParams) (string, error) + getPlacementSize([]openrtb.Format) (uint64, uint64) + + canAutoPlayVideo(string, UserAgentParsers) bool + isAndroid(string) bool + isiOS(string) bool + isAtMinChromeVersion(string, *regexp.Regexp) bool + isAtMinSafariVersion(string, *regexp.Regexp) bool +} + +type Util struct{} + +func (u Util) getAdMarkup(strResp openrtb_ext.ExtImpSharethroughResponse, params *StrAdSeverParams) (string, error) { + strRespId := fmt.Sprintf("str_response_%s", strResp.BidID) + jsonPayload, err := json.Marshal(strResp) + if err != nil { + return "", err + } + + tmplBody := ` + + +
+ + ` + + if params.Iframe { + tmplBody = tmplBody + ` + + ` + } else { + tmplBody = tmplBody + ` + + + ` + } + + tmpl, err := template.New("sfpjs").Parse(tmplBody) + if err != nil { + return "", err + } + + var buf []byte + templatedBuf := bytes.NewBuffer(buf) + + b64EncodedJson := base64.StdEncoding.EncodeToString(jsonPayload) + err = tmpl.Execute(templatedBuf, struct { + Arid template.JS + Pkey string + StrRespId template.JS + B64EncodedJson string + }{ + template.JS(strResp.AdServerRequestID), + params.Pkey, + template.JS(strRespId), + b64EncodedJson, + }) + if err != nil { + return "", err + } + + return templatedBuf.String(), nil +} + +func (u Util) getPlacementSize(formats []openrtb.Format) (height uint64, width uint64) { + biggest := struct { + Height uint64 + Width uint64 + }{ + Height: 1, + Width: 1, + } + + for i := 0; i < len(formats); i++ { + format := formats[i] + if (format.H * format.W) > (biggest.Height * biggest.Width) { + biggest.Height = format.H + biggest.Width = format.W + } + } + + return biggest.Height, biggest.Width +} + +func (u Util) canAutoPlayVideo(userAgent string, parsers UserAgentParsers) bool { + if u.isAndroid(userAgent) { + return u.isAtMinChromeVersion(userAgent, parsers.ChromeVersion) + } else if u.isiOS(userAgent) { + return u.isAtMinSafariVersion(userAgent, parsers.SafariVersion) || u.isAtMinChromeVersion(userAgent, parsers.ChromeiOSVersion) + } + return true +} + +func (u Util) isAndroid(userAgent string) bool { + isAndroid, err := regexp.MatchString("(?i)Android", userAgent) + if err != nil { + return false + } + return isAndroid +} + +func (u Util) isiOS(userAgent string) bool { + isiOS, err := regexp.MatchString("(?i)iPhone|iPad|iPod", userAgent) + if err != nil { + return false + } + return isiOS +} + +func (u Util) isAtMinVersion(userAgent string, versionParser *regexp.Regexp, minVersion int64) bool { + var version int64 + var err error + + versionMatch := versionParser.FindStringSubmatch(userAgent) + if len(versionMatch) > 1 { + version, err = strconv.ParseInt(versionMatch[1], 10, 64) + } + if err != nil { + return false + } + + return version >= minVersion +} + +func (u Util) isAtMinChromeVersion(userAgent string, parser *regexp.Regexp) bool { + return u.isAtMinVersion(userAgent, parser, minChromeVersion) +} + +func (u Util) isAtMinSafariVersion(userAgent string, parser *regexp.Regexp) bool { + return u.isAtMinVersion(userAgent, parser, minSafariVersion) +} + +func (u Util) gdprApplies(request *openrtb.BidRequest) bool { + var gdprApplies int64 + + if request.Regs != nil { + if jsonExtRegs, err := request.Regs.Ext.MarshalJSON(); err == nil { + // 0 is the return value if error, so no need to handle + gdprApplies, _ = jsonparser.GetInt(jsonExtRegs, "gdpr") + } + } + + return gdprApplies != 0 +} + +func (u Util) gdprConsentString(request *openrtb.BidRequest) string { + var consentString string + + if request.User != nil { + if jsonExtUser, err := request.User.Ext.MarshalJSON(); err == nil { + // empty string is the return value if error, so no need to handle + consentString, _ = jsonparser.GetString(jsonExtUser, "consent") + } + } + + return consentString +} diff --git a/adapters/sharethrough/utils_test.go b/adapters/sharethrough/utils_test.go new file mode 100644 index 00000000000..a5ee882707e --- /dev/null +++ b/adapters/sharethrough/utils_test.go @@ -0,0 +1,410 @@ +package sharethrough + +import ( + "github.com/mxmCherry/openrtb" + "github.com/prebid/prebid-server/openrtb_ext" + "regexp" + "strings" + "testing" +) + +func TestGetAdMarkup(t *testing.T) { + tests := map[string]struct { + inputResponse openrtb_ext.ExtImpSharethroughResponse + inputParams *StrAdSeverParams + expectedSuccess []string + expectedError error + }{ + "Sets template variables": { + inputResponse: openrtb_ext.ExtImpSharethroughResponse{BidID: "bid", AdServerRequestID: "arid"}, + inputParams: &StrAdSeverParams{Pkey: "pkey"}, + expectedSuccess: []string{ + ``, + `
`, + ``, + }, + expectedError: nil, + }, + "Includes sfp.js without iFrame busting logic if iFrame param is true": { + inputResponse: openrtb_ext.ExtImpSharethroughResponse{BidID: "bid", AdServerRequestID: "arid"}, + inputParams: &StrAdSeverParams{Pkey: "pkey", Iframe: true}, + expectedSuccess: []string{ + ``, + }, + expectedError: nil, + }, + "Includes sfp.js with iFrame busting logic if iFrame param is false": { + inputResponse: openrtb_ext.ExtImpSharethroughResponse{BidID: "bid", AdServerRequestID: "arid"}, + inputParams: &StrAdSeverParams{Pkey: "pkey", Iframe: false}, + expectedSuccess: []string{ + ``, + }, + expectedError: nil, + }, + "Includes sfp.js with iFrame busting logic if iFrame param is not provided": { + inputResponse: openrtb_ext.ExtImpSharethroughResponse{BidID: "bid", AdServerRequestID: "arid"}, + inputParams: &StrAdSeverParams{Pkey: "pkey"}, + expectedSuccess: []string{ + ``, + }, + expectedError: nil, + }, + } + + util := Util{} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + outputSuccess, outputError := util.getAdMarkup(test.inputResponse, test.inputParams) + for _, markup := range test.expectedSuccess { + if !strings.Contains(outputSuccess, markup) { + t.Errorf("Expected Ad Markup to contain: %s, got %s\n", markup, outputSuccess) + } + } + if outputError != test.expectedError { + t.Errorf("Expected Error to be: %s, got %s\n", test.expectedError, outputError) + } + } +} + +func TestGetPlacementSize(t *testing.T) { + tests := map[string]struct { + input []openrtb.Format + expectedHeight uint64 + expectedWidth uint64 + }{ + "Returns default size if empty input": { + input: []openrtb.Format{}, + expectedHeight: 1, + expectedWidth: 1, + }, + "Returns size if only one is passed": { + input: []openrtb.Format{{H: 100, W: 100}}, + expectedHeight: 100, + expectedWidth: 100, + }, + "Returns biggest size if multiple are passed": { + input: []openrtb.Format{{H: 100, W: 100}, {H: 200, W: 200}, {H: 50, W: 50}}, + expectedHeight: 200, + expectedWidth: 200, + }, + } + + util := Util{} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + outputHeight, outputWidth := util.getPlacementSize(test.input) + if outputHeight != test.expectedHeight { + t.Errorf("Expected Height: %d, got %d\n", test.expectedHeight, outputHeight) + } + if outputWidth != test.expectedWidth { + t.Errorf("Expected Width: %d, got %d\n", test.expectedWidth, outputWidth) + } + } +} + +type userAgentTest struct { + input string + expected bool +} + +func runUserAgentTests(tests map[string]userAgentTest, fn func(string) bool, t *testing.T) { + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + output := fn(test.input) + if output != test.expected { + t.Errorf("Expected: %t, got %t\n", test.expected, output) + } + } +} + +func TestCanAutoPlayVideo(t *testing.T) { + uaParsers := UserAgentParsers{ + ChromeVersion: regexp.MustCompile(`Chrome\/(?P\d+)`), + ChromeiOSVersion: regexp.MustCompile(`CriOS\/(?P\d+)`), + SafariVersion: regexp.MustCompile(`Version\/(?P\d+)`), + } + + ableAgents := map[string]string{ + "Android at min Chrome version": "Android Chrome/60.0", + "iOS at min Chrome version": "iPhone CriOS/60.0", + "iOS at min Safari version": "iPad Version/14.0", + "Neither Android or iOS": "Some User Agent", + } + unableAgents := map[string]string{ + "Android not at min Chrome version": "Android Chrome/12", + "iOS not at min Chrome version": "iPod Chrome/12", + "iOS not at min Safari version": "iPod Version/8", + } + + tests := map[string]userAgentTest{} + for testName, agent := range ableAgents { + tests[testName] = userAgentTest{ + input: agent, + expected: true, + } + } + for testName, agent := range unableAgents { + tests[testName] = userAgentTest{ + input: agent, + expected: false, + } + } + + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + output := Util{}.canAutoPlayVideo(test.input, uaParsers) + if output != test.expected { + t.Errorf("Expected: %t, got %t\n", test.expected, output) + } + } +} + +func TestIsAndroid(t *testing.T) { + goodUserAgent := "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 6P Build/MMB29P)" + badUserAgent := "fake user agent" + + // This is an alternate way to do testing if you have many test cases that only change the input and output + tests := map[string]userAgentTest{ + "Match the Android user agent": { + input: goodUserAgent, + expected: true, + }, + "Does not match Android user agent": { + input: badUserAgent, + expected: false, + }, + } + + runUserAgentTests(tests, Util{}.isAndroid, t) +} + +func TestIsiOS(t *testing.T) { + iPhoneUserAgent := "Some string containing iPhone" + iPadUserAgent := "Some string containing iPad" + iPodUserAgent := "Some string containing iPOD" + badUserAgent := "Fake User Agent" + + tests := map[string]userAgentTest{ + "Match the iPhone user agent": { + input: iPhoneUserAgent, + expected: true, + }, + "Match the iPad user agent": { + input: iPadUserAgent, + expected: true, + }, + "Match the iPod user agent": { + input: iPodUserAgent, + expected: true, + }, + "Does not match Android user agent": { + input: badUserAgent, + expected: false, + }, + } + + runUserAgentTests(tests, Util{}.isiOS, t) +} + +func TestIsAtMinChromeVersion(t *testing.T) { + regex := regexp.MustCompile(`Chrome\/(?P\d+)`) + v60ChromeUA := "Mozilla/5.0 Chrome/60.0.3112.113" + v12ChromeUA := "Mozilla/5.0 Chrome/12.0.3112.113" + badUA := "Fake User Agent" + + tests := map[string]userAgentTest{ + "Return true if greater than min (53)": { + input: v60ChromeUA, + expected: true, + }, + "Return false if lower than min (53)": { + input: v12ChromeUA, + expected: false, + }, + "Return false if no version found": { + input: badUA, + expected: false, + }, + } + + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + output := Util{}.isAtMinChromeVersion(test.input, regex) + if output != test.expected { + t.Errorf("Expected: %t, got %t\n", test.expected, output) + } + } +} + +func TestIsAtMinChromeIosVersion(t *testing.T) { + regex := regexp.MustCompile(`CriOS\/(?P\d+)`) + v60ChrIosUA := "Mozilla/5.0 CriOS/60.0.3112.113" + v12ChrIosUA := "Mozilla/5.0 CriOS/12.0.3112.113" + badUA := "Fake User Agent" + + tests := map[string]userAgentTest{ + "Return true if greater than min (53)": { + input: v60ChrIosUA, + expected: true, + }, + "Return false if lower than min (53)": { + input: v12ChrIosUA, + expected: false, + }, + "Return false if no version found": { + input: badUA, + expected: false, + }, + } + + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + output := Util{}.isAtMinChromeVersion(test.input, regex) + if output != test.expected { + t.Errorf("Expected: %t, got %t\n", test.expected, output) + } + } +} + +func TestIsAtMinSafariVersion(t *testing.T) { + regex := regexp.MustCompile(`Version\/(?P\d+)`) + v12SafariUA := "Mozilla/5.0 Version/12.0.3112.113" + v07SafariUA := "Mozilla/5.0 Version/07.0.3112.113" + badUA := "Fake User Agent" + + tests := map[string]userAgentTest{ + "Return true if greater than min (10)": { + input: v12SafariUA, + expected: true, + }, + "Return false if lower than min (10)": { + input: v07SafariUA, + expected: false, + }, + "Return false if no version found": { + input: badUA, + expected: false, + }, + } + + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + output := Util{}.isAtMinSafariVersion(test.input, regex) + if output != test.expected { + t.Errorf("Expected: %t, got %t\n", test.expected, output) + } + } +} + +func TestGdprApplies(t *testing.T) { + bidRequestGdpr := openrtb.BidRequest{ + Regs: &openrtb.Regs{ + Ext: []byte(`{"gdpr": 1}`), + }, + } + bidRequestNonGdpr := openrtb.BidRequest{ + Regs: &openrtb.Regs{ + Ext: []byte(`{"gdpr": 0}`), + }, + } + bidRequestEmptyGdpr := openrtb.BidRequest{ + Regs: &openrtb.Regs{ + Ext: []byte(``), + }, + } + bidRequestEmptyRegs := openrtb.BidRequest{ + Regs: &openrtb.Regs{}, + } + + tests := map[string]struct { + input *openrtb.BidRequest + expected bool + }{ + "Return true if gdpr set to 1": { + input: &bidRequestGdpr, + expected: true, + }, + "Return false if gdpr set to 0": { + input: &bidRequestNonGdpr, + expected: false, + }, + "Return false if no gdpr set": { + input: &bidRequestEmptyGdpr, + expected: false, + }, + "Return false if no Regs set": { + input: &bidRequestEmptyRegs, + expected: false, + }, + } + + util := Util{} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + output := util.gdprApplies(test.input) + if output != test.expected { + t.Errorf("Expected: %t, got %t\n", test.expected, output) + } + } +} + +func TestGdprConsentString(t *testing.T) { + bidRequestWithConsent := openrtb.BidRequest{ + User: &openrtb.User{ + Ext: []byte(`{"consent": "abc"}`), + }, + } + bidRequestWithEmptyConsent := openrtb.BidRequest{ + User: &openrtb.User{ + Ext: []byte(`{"consent": ""}`), + }, + } + bidRequestWithoutConsent := openrtb.BidRequest{ + User: &openrtb.User{ + Ext: []byte(`{"other": "abc"}`), + }, + } + bidRequestWithUserExt := openrtb.BidRequest{ + User: &openrtb.User{}, + } + + tests := map[string]struct { + input *openrtb.BidRequest + expected string + }{ + "Return consent string if provided": { + input: &bidRequestWithConsent, + expected: "abc", + }, + "Return empty string if consent string empty": { + input: &bidRequestWithEmptyConsent, + expected: "", + }, + "Return empty string if no consent string provided": { + input: &bidRequestWithoutConsent, + expected: "", + }, + "Return empty string if User set": { + input: &bidRequestWithUserExt, + expected: "", + }, + } + + util := Util{} + for testName, test := range tests { + t.Logf("Test case: %s\n", testName) + + output := util.gdprConsentString(test.input) + if output != test.expected { + t.Errorf("Expected: %s, got %s\n", test.expected, output) + } + } +} diff --git a/config/config.go b/config/config.go index dc50c8a062c..f457f5501f5 100644 --- a/config/config.go +++ b/config/config.go @@ -119,9 +119,11 @@ func (cfg *AuctionTimeouts) LimitAuctionTimeout(requested time.Duration) time.Du } type GDPR struct { - HostVendorID int `mapstructure:"host_vendor_id"` - UsersyncIfAmbiguous bool `mapstructure:"usersync_if_ambiguous"` - Timeouts GDPRTimeouts `mapstructure:"timeouts_ms"` + HostVendorID int `mapstructure:"host_vendor_id"` + UsersyncIfAmbiguous bool `mapstructure:"usersync_if_ambiguous"` + Timeouts GDPRTimeouts `mapstructure:"timeouts_ms"` + NonStandardPublishers []string `mapstructure:"non_standard_publishers,flow"` + NonStandardPublisherMap map[string]int } func (cfg *GDPR) validate(errs configErrors) configErrors { @@ -384,6 +386,13 @@ func New(v *viper.Viper) (*Configuration, error) { if errs := c.validate(); len(errs) > 0 { return &c, errs } + + // To look for a request's publisher_id into the NonStandardPublishers in + // O(1) time, we fill this hash table located in the NonStandardPublisherMap field of GDPR + c.GDPR.NonStandardPublisherMap = make(map[string]int) + for i := 0; i < len(c.GDPR.NonStandardPublishers); i++ { + c.GDPR.NonStandardPublisherMap[c.GDPR.NonStandardPublishers[i]] = 1 + } return &c, nil } @@ -430,11 +439,13 @@ func (cfg *Configuration) setDerivedDefaults() { setDefaultUsersync(cfg.Adapters, openrtb_ext.BidderPulsepoint, "https://bh.contextweb.com/rtset?pid=561205&ev=1&rurl="+url.QueryEscape(externalURL)+"%2Fsetuid%3Fbidder%3Dpulsepoint%26gdpr%3D{{.GDPR}}%26gdpr_consent%3D{{.GDPRConsent}}%26uid%3D%25%25VGUID%25%25") setDefaultUsersync(cfg.Adapters, openrtb_ext.BidderRhythmone, "https://sync.1rx.io/usersync2/rmphb?gdpr={{.GDPR}}&gdpr_consent={{.GDPRConsent}}&redir="+url.QueryEscape(externalURL)+"%2Fsetuid%3Fbidder%3Drhythmone%26gdpr%3D{{.GDPR}}%26gdpr_consent%3D{{.GDPRConsent}}%26uid%3D%5BRX_UUID%5D") // openrtb_ext.BidderRubicon doesn't have a good default. + setDefaultUsersync(cfg.Adapters, openrtb_ext.BidderSharethrough, "https://sharethrough.adnxs.com/getuid?"+url.QueryEscape(externalURL)+"/setuid?bidder=sharethrough&gdpr={{.GDPR}}&gdpr_consent={{.GDPRConsent}}&uid=$UID") setDefaultUsersync(cfg.Adapters, openrtb_ext.BidderSomoaudience, "https://publisher-east.mobileadtrading.com/usersync?ru="+url.QueryEscape(externalURL)+"%2Fsetuid%3Fbidder%3Dsomoaudience%26gdpr%3D{{.GDPR}}%26gdpr_consent%3D{{.GDPRConsent}}%26uid%3D%24%7BUID%7D") setDefaultUsersync(cfg.Adapters, openrtb_ext.BidderSovrn, "https://ap.lijit.com/pixel?redir="+url.QueryEscape(externalURL)+"%2Fsetuid%3Fbidder%3Dsovrn%26gdpr%3D{{.GDPR}}%26gdpr_consent%3D{{.GDPRConsent}}%26uid%3D%24UID") setDefaultUsersync(cfg.Adapters, openrtb_ext.BidderSonobi, "https://sync.go.sonobi.com/us.gif?loc="+url.QueryEscape(externalURL)+"%2Fsetuid%3Fbidder%3Dsonobi%26consent_string%3D{{.GDPR}}%26gdpr%3D{{.GDPRConsent}}%26uid%3D%5BUID%5D") setDefaultUsersync(cfg.Adapters, openrtb_ext.BidderYieldmo, "https://ads.yieldmo.com/pbsync?gdpr={{.GDPR}}&gdpr_consent={{.GDPRConsent}}&redirectUri="+url.QueryEscape(externalURL)+"%2Fsetuid%3Fbidder%3Dyieldmo%26gdpr%3D{{.GDPR}}%26gdpr_consent%3D{{.GDPRConsent}}%26uid%3D%24UID") setDefaultUsersync(cfg.Adapters, openrtb_ext.BidderGamoshi, "https://rtb.gamoshi.io/pix/0000/scm?gdpr={{.GDPR}}&consent={{.GDPRConsent}}&rurl="+url.QueryEscape(externalURL)+"%2Fsetuid%3Fbidder%3Dgamoshi%26gdpr%3D{{.GDPR}}%26gdpr_consent%3D{{.GDPRConsent}}%26uid%3D%5Bgusr%5D") + setDefaultUsersync(cfg.Adapters, openrtb_ext.BidderMgid, "https://cm.mgid.com/m?cdsp=363893&adu="+url.QueryEscape(externalURL)+"%2Fsetuid%3Fbidder%3Dmgid%26gdpr%3D{{.GDPR}}%26gdpr_consent%3D{{.GDPRConsent}}%26uid%3D%7Bmuidn%7D") } @@ -500,6 +511,7 @@ func SetupViper(v *viper.Viper, filename string) { v.SetDefault("datacache.ttl_seconds", 0) v.SetDefault("category_mapping.filesystem.enabled", true) v.SetDefault("category_mapping.filesystem.directorypath", "./static/category-mapping") + v.SetDefault("category_mapping.http.endpoint", "") v.SetDefault("stored_requests.filesystem", false) v.SetDefault("stored_requests.directorypath", "./stored_requests/data/by_id") v.SetDefault("stored_requests.postgres.connection.dbname", "") @@ -587,9 +599,11 @@ func SetupViper(v *viper.Viper, filename string) { v.SetDefault("adapters.rhythmone.endpoint", "http://tag.1rx.io/rmp") v.SetDefault("adapters.gumgum.endpoint", "https://g2.gumgum.com/providers/prbds2s/bid") v.SetDefault("adapters.grid.endpoint", "http://grid.bidswitch.net/sp_bid?sp=prebid") + v.SetDefault("adapters.sharethrough.endpoint", "http://btlr.sharethrough.com/FGMrCMMc/v1") v.SetDefault("adapters.sonobi.endpoint", "https://apex.go.sonobi.com/prebid?partnerid=71d9d3d8af") v.SetDefault("adapters.yieldmo.endpoint", "http://ads.yieldmo.com/exchange/prebid-server") v.SetDefault("adapters.gamoshi.endpoint", "https://rtb.gamoshi.io") + v.SetDefault("adapters.mgid.endpoint", "https://prebid.mgid.com/prebid/") v.SetDefault("max_request_size", 1024*256) v.SetDefault("analytics.file.filename", "") @@ -598,8 +612,9 @@ func SetupViper(v *viper.Viper, filename string) { v.SetDefault("gdpr.usersync_if_ambiguous", false) v.SetDefault("gdpr.timeouts_ms.init_vendorlist_fetches", 0) v.SetDefault("gdpr.timeouts_ms.active_vendorlist_fetch", 0) + v.SetDefault("gdpr.non_standard_publishers", []string{""}) v.SetDefault("currency_converter.fetch_url", "https://cdn.jsdelivr.net/gh/prebid/currency-file@1/latest.json") - v.SetDefault("currency_converter.fetch_interval_seconds", 0) // #280 Not activated for the time being + v.SetDefault("currency_converter.fetch_interval_seconds", 1800) // fetch currency rates every 30 minutes v.SetDefault("default_request.type", "") v.SetDefault("default_request.file.name", "") v.SetDefault("default_request.alias_info", false) diff --git a/config/config_test.go b/config/config_test.go index 89cb7f8f203..ef7ebe78c1a 100644 --- a/config/config_test.go +++ b/config/config_test.go @@ -24,12 +24,15 @@ func TestDefaults(t *testing.T) { cmpInts(t, "host_cookie.ttl_days", int(cfg.HostCookie.TTL), 90) cmpStrings(t, "datacache.type", cfg.DataCache.Type, "dummy") cmpStrings(t, "adapters.pubmatic.endpoint", cfg.Adapters[string(openrtb_ext.BidderPubmatic)].Endpoint, "http://hbopenbid.pubmatic.com/translator?source=prebid-server") + cmpInts(t, "currency_converter.fetch_interval_seconds", cfg.CurrencyConverter.FetchIntervalSeconds, 1800) + cmpStrings(t, "currency_converter.fetch_url", cfg.CurrencyConverter.FetchURL, "https://cdn.jsdelivr.net/gh/prebid/currency-file@1/latest.json") } var fullConfig = []byte(` gdpr: host_vendor_id: 15 usersync_if_ambiguous: true + non_standard_publishers: ["siteID","fake-site-id","appID","agltb3B1Yi1pbmNyDAsSA0FwcBiJkfIUDA"] host_cookie: cookie_name: userid family: prebid @@ -157,6 +160,22 @@ func TestFullConfig(t *testing.T) { cmpInts(t, "http_client.idle_connection_timeout_seconds", cfg.Client.IdleConnTimeout, 30) cmpInts(t, "gdpr.host_vendor_id", cfg.GDPR.HostVendorID, 15) cmpBools(t, "gdpr.usersync_if_ambiguous", cfg.GDPR.UsersyncIfAmbiguous, true) + + //Assert the NonStandardPublishers was correctly unmarshalled + cmpStrings(t, "gdpr.non_standard_publishers", cfg.GDPR.NonStandardPublishers[0], "siteID") + cmpStrings(t, "gdpr.non_standard_publishers", cfg.GDPR.NonStandardPublishers[1], "fake-site-id") + cmpStrings(t, "gdpr.non_standard_publishers", cfg.GDPR.NonStandardPublishers[2], "appID") + cmpStrings(t, "gdpr.non_standard_publishers", cfg.GDPR.NonStandardPublishers[3], "agltb3B1Yi1pbmNyDAsSA0FwcBiJkfIUDA") + + //Assert the NonStandardPublisherMap hash table was built correctly + var found bool + for i := 0; i < len(cfg.GDPR.NonStandardPublishers); i++ { + _, found = cfg.GDPR.NonStandardPublisherMap[cfg.GDPR.NonStandardPublishers[i]] + cmpBools(t, "cfg.GDPR.NonStandardPublisherMap", found, true) + } + _, found = cfg.GDPR.NonStandardPublisherMap["appnexus"] + cmpBools(t, "cfg.GDPR.NonStandardPublisherMap", found, false) + cmpStrings(t, "currency_converter.fetch_url", cfg.CurrencyConverter.FetchURL, "https://currency.prebid.org") cmpInts(t, "currency_converter.fetch_interval_seconds", cfg.CurrencyConverter.FetchIntervalSeconds, 1800) cmpStrings(t, "recaptcha_secret", cfg.RecaptchaSecret, "asdfasdfasdfasdf") diff --git a/endpoints/auction.go b/endpoints/auction.go index 091c659e6f2..3bb2ba52cf4 100644 --- a/endpoints/auction.go +++ b/endpoints/auction.go @@ -33,13 +33,6 @@ type bidResult struct { const defaultPriceGranularity = "med" -// Constant keys for ad server targeting for responses to Prebid Mobile -const hbpbConstantKey = "hb_pb" -const hbBidderConstantKey = "hb_bidder" -const hbCacheIdConstantKey = "hb_cache_id" -const hbDealIdConstantKey = "hb_deal" -const hbSizeConstantKey = "hb_size" - func min(x, y int) int { if x < y { return x @@ -445,16 +438,16 @@ func sortBidsAddKeywordsMobile(bids pbs.PBSBidSlice, pbs_req *pbs.PBSRequest, pr hbSize = width + "x" + height } - hbPbBidderKey := hbpbConstantKey + "_" + bid.BidderCode - hbBidderBidderKey := hbBidderConstantKey + "_" + bid.BidderCode - hbCacheIdBidderKey := hbCacheIdConstantKey + "_" + bid.BidderCode - hbDealIdBidderKey := hbDealIdConstantKey + "_" + bid.BidderCode - hbSizeBidderKey := hbSizeConstantKey + "_" + bid.BidderCode + hbPbBidderKey := string(openrtb_ext.HbpbConstantKey) + "_" + bid.BidderCode + hbBidderBidderKey := string(openrtb_ext.HbBidderConstantKey) + "_" + bid.BidderCode + hbCacheIDBidderKey := string(openrtb_ext.HbCacheKey) + "_" + bid.BidderCode + hbDealIDBidderKey := string(openrtb_ext.HbDealIDConstantKey) + "_" + bid.BidderCode + hbSizeBidderKey := string(openrtb_ext.HbSizeConstantKey) + "_" + bid.BidderCode if pbs_req.MaxKeyLength != 0 { hbPbBidderKey = hbPbBidderKey[:min(len(hbPbBidderKey), int(pbs_req.MaxKeyLength))] hbBidderBidderKey = hbBidderBidderKey[:min(len(hbBidderBidderKey), int(pbs_req.MaxKeyLength))] - hbCacheIdBidderKey = hbCacheIdBidderKey[:min(len(hbCacheIdBidderKey), int(pbs_req.MaxKeyLength))] - hbDealIdBidderKey = hbDealIdBidderKey[:min(len(hbDealIdBidderKey), int(pbs_req.MaxKeyLength))] + hbCacheIDBidderKey = hbCacheIDBidderKey[:min(len(hbCacheIDBidderKey), int(pbs_req.MaxKeyLength))] + hbDealIDBidderKey = hbDealIDBidderKey[:min(len(hbDealIDBidderKey), int(pbs_req.MaxKeyLength))] hbSizeBidderKey = hbSizeBidderKey[:min(len(hbSizeBidderKey), int(pbs_req.MaxKeyLength))] } @@ -466,24 +459,24 @@ func sortBidsAddKeywordsMobile(bids pbs.PBSBidSlice, pbs_req *pbs.PBSRequest, pr kvs[hbPbBidderKey] = roundedCpm kvs[hbBidderBidderKey] = bid.BidderCode - kvs[hbCacheIdBidderKey] = bid.CacheID + kvs[hbCacheIDBidderKey] = bid.CacheID if hbSize != "" { kvs[hbSizeBidderKey] = hbSize } if bid.DealId != "" { - kvs[hbDealIdBidderKey] = bid.DealId + kvs[hbDealIDBidderKey] = bid.DealId } // For the top bid, we want to add the following additional keys if i == 0 { - kvs[hbpbConstantKey] = roundedCpm - kvs[hbBidderConstantKey] = bid.BidderCode - kvs[hbCacheIdConstantKey] = bid.CacheID + kvs[string(openrtb_ext.HbpbConstantKey)] = roundedCpm + kvs[string(openrtb_ext.HbBidderConstantKey)] = bid.BidderCode + kvs[string(openrtb_ext.HbCacheKey)] = bid.CacheID if bid.DealId != "" { - kvs[hbDealIdConstantKey] = bid.DealId + kvs[string(openrtb_ext.HbDealIDConstantKey)] = bid.DealId } if hbSize != "" { - kvs[hbSizeConstantKey] = hbSize + kvs[string(openrtb_ext.HbSizeConstantKey)] = hbSize } } } diff --git a/endpoints/auction_test.go b/endpoints/auction_test.go index 70963c91914..2abee138b88 100644 --- a/endpoints/auction_test.go +++ b/endpoints/auction_test.go @@ -148,56 +148,56 @@ func TestSortBidsAndAddKeywordsForMobile(t *testing.T) { t.Error("Ad server targeting should not be nil") } if bid.BidderCode == "audienceNetwork" { - if bid.AdServerTargeting["hb_size"] != "300x250" { - t.Error("hb_size key was not parsed correctly") + if bid.AdServerTargeting[string(openrtb_ext.HbSizeConstantKey)] != "300x250" { + t.Error(string(openrtb_ext.HbSizeConstantKey) + " key was not parsed correctly") } - if bid.AdServerTargeting["hb_pb"] != "2.00" { - t.Error("hb_pb key was not parsed correctly ", bid.AdServerTargeting["hb_pb"]) + if bid.AdServerTargeting[string(openrtb_ext.HbpbConstantKey)] != "2.00" { + t.Error(string(openrtb_ext.HbpbConstantKey)+" key was not parsed correctly ", bid.AdServerTargeting[string(openrtb_ext.HbpbConstantKey)]) } - if bid.AdServerTargeting["hb_cache_id"] != "test_cache_id1" { - t.Error("hb_cache_id key was not parsed correctly") + if bid.AdServerTargeting[string(openrtb_ext.HbCacheKey)] != "test_cache_id1" { + t.Error(string(openrtb_ext.HbCacheKey) + " key was not parsed correctly") } - if bid.AdServerTargeting["hb_bidder"] != "audienceNetwork" { - t.Error("hb_bidder key was not parsed correctly") + if bid.AdServerTargeting[string(openrtb_ext.HbBidderConstantKey)] != "audienceNetwork" { + t.Error(string(openrtb_ext.HbBidderConstantKey) + " key was not parsed correctly") } - if bid.AdServerTargeting["hb_deal"] != "2345" { - t.Error("hb_deal_id key was not parsed correctly ") + if bid.AdServerTargeting[string(openrtb_ext.HbDealIDConstantKey)] != "2345" { + t.Error(string(openrtb_ext.HbDealIDConstantKey) + " key was not parsed correctly ") } } if bid.BidderCode == "appnexus" { - if bid.AdServerTargeting["hb_size_appnexus"] != "320x50" { - t.Error("hb_size key for appnexus bidder was not parsed correctly") + if bid.AdServerTargeting[string(openrtb_ext.HbSizeConstantKey)+"_appnexus"] != "320x50" { + t.Error(string(openrtb_ext.HbSizeConstantKey) + " key for appnexus bidder was not parsed correctly") } - if bid.AdServerTargeting["hb_cache_id_appnexus"] != "test_cache_id2" { - t.Error("hb_cache_id key for appnexus bidder was not parsed correctly") + if bid.AdServerTargeting[string(openrtb_ext.HbCacheKey)+"_appnexus"] != "test_cache_id2" { + t.Error(string(openrtb_ext.HbCacheKey) + " key for appnexus bidder was not parsed correctly") } - if bid.AdServerTargeting["hb_bidder_appnexus"] != "appnexus" { - t.Error("hb_bidder key for appnexus bidder was not parsed correctly") + if bid.AdServerTargeting[string(openrtb_ext.HbBidderConstantKey)+"_appnexus"] != "appnexus" { + t.Error(string(openrtb_ext.HbBidderConstantKey) + " key for appnexus bidder was not parsed correctly") } - if bid.AdServerTargeting["hb_pb_appnexus"] != "1.00" { - t.Error("hb_pb key for appnexus bidder was not parsed correctly") + if bid.AdServerTargeting[string(openrtb_ext.HbpbConstantKey)+"_appnexus"] != "1.00" { + t.Error(string(openrtb_ext.HbpbConstantKey) + " key for appnexus bidder was not parsed correctly") } - if bid.AdServerTargeting["hb_pb"] != "" { - t.Error("hb_pb key was parsed for two bidders") + if bid.AdServerTargeting[string(openrtb_ext.HbpbConstantKey)] != "" { + t.Error(string(openrtb_ext.HbpbConstantKey) + " key was parsed for two bidders") } - if bid.AdServerTargeting["hb_deal_appnexus"] != "1234" { - t.Errorf("hb_deal_id_appnexus was not parsed correctly %v", bid.AdServerTargeting["hb_deal_id_appnexus"]) + if bid.AdServerTargeting[string(openrtb_ext.HbDealIDConstantKey)+"_appnexus"] != "1234" { + t.Errorf(string(openrtb_ext.HbDealIDConstantKey)+"_appnexus was not parsed correctly %v", bid.AdServerTargeting[string(openrtb_ext.HbDealIDConstantKey)+"_appnexus"]) } } - if bid.BidderCode == "rubicon" { + if bid.BidderCode == string(openrtb_ext.BidderRubicon) { if bid.AdServerTargeting["rpfl_1001"] != "15_tier0100" { t.Error("custom ad_server_targeting KVPs from adapter were not preserved") } } if bid.BidderCode == "nosizebidder" { - if _, exists := bid.AdServerTargeting["hb_size_nosizebidder"]; exists { - t.Error("hb_size key for nosize bidder was not parsed correctly", bid.AdServerTargeting) + if _, exists := bid.AdServerTargeting[string(openrtb_ext.HbSizeConstantKey)+"_nosizebidder"]; exists { + t.Error(string(openrtb_ext.HbSizeConstantKey)+" key for nosize bidder was not parsed correctly", bid.AdServerTargeting) } } if bid.BidderCode == "nodeal" { - if _, exists := bid.AdServerTargeting["hb_deal_nodeal"]; exists { - t.Error("hb_deal_id key for nodeal bidder was not parsed correctly") + if _, exists := bid.AdServerTargeting[string(openrtb_ext.HbDealIDConstantKey)+"_nodeal"]; exists { + t.Error(string(openrtb_ext.HbDealIDConstantKey) + " key for nodeal bidder was not parsed correctly") } } } @@ -409,7 +409,7 @@ func (m *auctionMockPermissions) BidderSyncAllowed(ctx context.Context, bidder o return m.allowBidderSync, nil } -func (m *auctionMockPermissions) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, consent string) (bool, error) { +func (m *auctionMockPermissions) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, PublisherID string, consent string) (bool, error) { return m.allowPI, nil } diff --git a/endpoints/cookie_sync_test.go b/endpoints/cookie_sync_test.go index 0d7d28d1146..8f126bb9444 100644 --- a/endpoints/cookie_sync_test.go +++ b/endpoints/cookie_sync_test.go @@ -198,6 +198,6 @@ func (g *gdprPerms) BidderSyncAllowed(ctx context.Context, bidder openrtb_ext.Bi return ok, nil } -func (g *gdprPerms) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, consent string) (bool, error) { +func (g *gdprPerms) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, PublisherID string, consent string) (bool, error) { return true, nil } diff --git a/endpoints/openrtb2/video_auction.go b/endpoints/openrtb2/video_auction.go index acb28307534..35cba3befbd 100644 --- a/endpoints/openrtb2/video_auction.go +++ b/endpoints/openrtb2/video_auction.go @@ -359,8 +359,10 @@ func minMax(array []int) (int, int) { func buildVideoResponse(bidresponse *openrtb.BidResponse, podErrors []PodError) (*openrtb_ext.BidResponseVideo, error) { adPods := make([]*openrtb_ext.AdPod, 0) + anyBidsReturned := false for _, seatBid := range bidresponse.SeatBid { for _, bid := range seatBid.Bid { + anyBidsReturned = true var tempRespBidExt openrtb_ext.ExtBid if err := json.Unmarshal(bid.Ext, &tempRespBidExt); err != nil { @@ -375,9 +377,9 @@ func buildVideoResponse(bidresponse *openrtb.BidResponse, podErrors []PodError) podId, _ := strconv.ParseInt(podNum, 0, 64) videoTargeting := openrtb_ext.VideoTargeting{ - Hb_pb: tempRespBidExt.Prebid.Targeting[string(openrtb_ext.HbpbConstantKey)], - Hb_pb_cat_dur: tempRespBidExt.Prebid.Targeting[string(openrtb_ext.HbCategoryDurationKey)], - Hb_cache_id: tempRespBidExt.Prebid.Targeting[string(openrtb_ext.HbVastCacheKey)], + HbPb: tempRespBidExt.Prebid.Targeting[string(openrtb_ext.HbpbConstantKey)], + HbPbCatDur: tempRespBidExt.Prebid.Targeting[string(openrtb_ext.HbCategoryDurationKey)], + HbCacheID: tempRespBidExt.Prebid.Targeting[string(openrtb_ext.HbVastCacheKey)], } adPod := findAdPod(podId, adPods) @@ -393,14 +395,14 @@ func buildVideoResponse(bidresponse *openrtb.BidResponse, podErrors []PodError) } } - if len(adPods) == 0 { + //check if there are any bids in response. + //if there are no bids - empty response should be returned, no cache errors + if len(adPods) == 0 && anyBidsReturned { //means there is a global cache error, we need to reject all bids err := errors.New("caching failed for all bids") return nil, err } - videoResponse := openrtb_ext.BidResponseVideo{} - // If there were incorrect pods, we put them back to response with error message if len(podErrors) > 0 { for _, podEr := range podErrors { @@ -412,9 +414,7 @@ func buildVideoResponse(bidresponse *openrtb.BidResponse, podErrors []PodError) } } - videoResponse.AdPods = adPods - - return &videoResponse, nil + return &openrtb_ext.BidResponseVideo{AdPods: adPods}, nil } func findAdPod(podInd int64, pods []*openrtb_ext.AdPod) *openrtb_ext.AdPod { diff --git a/endpoints/openrtb2/video_auction_test.go b/endpoints/openrtb2/video_auction_test.go index ab08299057a..198a06fbf90 100644 --- a/endpoints/openrtb2/video_auction_test.go +++ b/endpoints/openrtb2/video_auction_test.go @@ -54,7 +54,7 @@ func TestVideoEndpointImpressionsNumber(t *testing.T) { assert.Len(t, resp.AdPods[3].Targeting, 1, "Incorrect Targeting data in response") assert.Len(t, resp.AdPods[4].Targeting, 3, "Incorrect Targeting data in response") - assert.Equal(t, resp.AdPods[4].Targeting[0].Hb_pb_cat_dur, "20.00_395_30s", "Incorrect number of Ad Pods in response") + assert.Equal(t, resp.AdPods[4].Targeting[0].HbPbCatDur, "20.00_395_30s", "Incorrect number of Ad Pods in response") } @@ -398,8 +398,8 @@ func TestVideoBuildVideoResponseMissedCacheForOneBid(t *testing.T) { assert.NoError(t, err, "Should be no error") assert.Len(t, bidRespVideo.AdPods, 1, "AdPods length should be 1") assert.Len(t, bidRespVideo.AdPods[0].Targeting, 2, "AdPod Targeting length should be 2") - assert.Equal(t, "17.00_123_30s", bidRespVideo.AdPods[0].Targeting[0].Hb_pb_cat_dur, "AdPod Targeting first element hb_pb_cat_dur should be 17.00_123_30s") - assert.Equal(t, "17.00_456_30s", bidRespVideo.AdPods[0].Targeting[1].Hb_pb_cat_dur, "AdPod Targeting first element hb_pb_cat_dur should be 17.00_456_30s") + assert.Equal(t, "17.00_123_30s", bidRespVideo.AdPods[0].Targeting[0].HbPbCatDur, "AdPod Targeting first element hb_pb_cat_dur should be 17.00_123_30s") + assert.Equal(t, "17.00_456_30s", bidRespVideo.AdPods[0].Targeting[1].HbPbCatDur, "AdPod Targeting first element hb_pb_cat_dur should be 17.00_456_30s") } func TestVideoBuildVideoResponseMissedCacheForAllBids(t *testing.T) { @@ -478,6 +478,15 @@ func TestVideoBuildVideoResponsePodErrors(t *testing.T) { assert.Equal(t, int64(333), bidRespVideo.AdPods[2].PodId, "AdPods should contain error element at index 2") } +func TestVideoBuildVideoResponseNoBids(t *testing.T) { + openRtbBidResp := openrtb.BidResponse{} + podErrors := make([]PodError, 0, 0) + openRtbBidResp.SeatBid = make([]openrtb.SeatBid, 0) + bidRespVideo, err := buildVideoResponse(&openRtbBidResp, podErrors) + assert.NoError(t, err, "Error should be nil") + assert.Len(t, bidRespVideo.AdPods, 0, "AdPods length should be 0") +} + func mockDeps(t *testing.T, ex *mockExchangeVideo) *endpointDeps { theMetrics := pbsmetrics.NewMetrics(metrics.NewRegistry(), openrtb_ext.BidderList()) edep := &endpointDeps{ diff --git a/endpoints/setuid_test.go b/endpoints/setuid_test.go index df664c3833c..03b4b6c21ef 100644 --- a/endpoints/setuid_test.go +++ b/endpoints/setuid_test.go @@ -193,6 +193,6 @@ func (g *mockPermsSetUID) BidderSyncAllowed(ctx context.Context, bidder openrtb_ return false, nil } -func (g *mockPermsSetUID) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, consent string) (bool, error) { +func (g *mockPermsSetUID) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, PublisherID string, consent string) (bool, error) { return g.allowPI, nil } diff --git a/exchange/adapter_map.go b/exchange/adapter_map.go index 61d3cf16b42..3c835571673 100644 --- a/exchange/adapter_map.go +++ b/exchange/adapter_map.go @@ -23,11 +23,13 @@ import ( "github.com/prebid/prebid-server/adapters/improvedigital" "github.com/prebid/prebid-server/adapters/ix" "github.com/prebid/prebid-server/adapters/lifestreet" + "github.com/prebid/prebid-server/adapters/mgid" "github.com/prebid/prebid-server/adapters/openx" "github.com/prebid/prebid-server/adapters/pubmatic" "github.com/prebid/prebid-server/adapters/pulsepoint" "github.com/prebid/prebid-server/adapters/rhythmone" "github.com/prebid/prebid-server/adapters/rubicon" + "github.com/prebid/prebid-server/adapters/sharethrough" "github.com/prebid/prebid-server/adapters/somoaudience" "github.com/prebid/prebid-server/adapters/sonobi" "github.com/prebid/prebid-server/adapters/sovrn" @@ -60,6 +62,7 @@ func newAdapterMap(client *http.Client, cfg *config.Configuration, infos adapter cfg.Adapters[string(openrtb_ext.BidderRubicon)].XAPI.Username, cfg.Adapters[string(openrtb_ext.BidderRubicon)].XAPI.Password, cfg.Adapters[string(openrtb_ext.BidderRubicon)].XAPI.Tracker), + openrtb_ext.BidderSharethrough: sharethrough.NewSharethroughBidder(cfg.Adapters[string(openrtb_ext.BidderSharethrough)].Endpoint), openrtb_ext.BidderSomoaudience: somoaudience.NewSomoaudienceBidder(cfg.Adapters[string(openrtb_ext.BidderSomoaudience)].Endpoint), openrtb_ext.BidderSovrn: sovrn.NewSovrnBidder(client, cfg.Adapters[string(openrtb_ext.BidderSovrn)].Endpoint), openrtb_ext.Bidder33Across: ttx.New33AcrossBidder(cfg.Adapters[string(openrtb_ext.Bidder33Across)].Endpoint), @@ -67,6 +70,7 @@ func newAdapterMap(client *http.Client, cfg *config.Configuration, infos adapter openrtb_ext.BidderSonobi: sonobi.NewSonobiBidder(client, cfg.Adapters[string(openrtb_ext.BidderSonobi)].Endpoint), openrtb_ext.BidderYieldmo: yieldmo.NewYieldmoBidder(cfg.Adapters[string(openrtb_ext.BidderYieldmo)].Endpoint), openrtb_ext.BidderGamoshi: gamoshi.NewGamoshiBidder(cfg.Adapters[string(openrtb_ext.BidderGamoshi)].Endpoint), + openrtb_ext.BidderMgid: mgid.NewMgidBidder(cfg.Adapters[string(openrtb_ext.BidderMgid)].Endpoint), openrtb_ext.BidderImprovedigital: improvedigital.NewImprovedigitalBidder(cfg.Adapters[string(openrtb_ext.BidderImprovedigital)].Endpoint), } diff --git a/exchange/exchange.go b/exchange/exchange.go index da454563faf..7cd5c495e25 100644 --- a/exchange/exchange.go +++ b/exchange/exchange.go @@ -138,21 +138,26 @@ func (e *exchange) HoldAuction(ctx context.Context, bidRequest *openrtb.BidReque // Get currency rates conversions for the auction conversions := e.currencyConverter.Rates() - adapterBids, adapterExtra := e.getAllBids(auctionCtx, cleanRequests, aliases, bidAdjustmentFactors, blabels, conversions) - bidCategory, adapterBids, err := applyCategoryMapping(requestExt, adapterBids, *categoriesFetcher, targData) - auc := newAuction(adapterBids, len(bidRequest.Imp)) - if err != nil { - return nil, fmt.Errorf("Error in category mapping : %s", err.Error()) - } + adapterBids, adapterExtra, anyBidsReturned := e.getAllBids(auctionCtx, cleanRequests, aliases, bidAdjustmentFactors, blabels, conversions) + + if anyBidsReturned { + bidCategory, adapterBids, err := applyCategoryMapping(ctx, requestExt, adapterBids, *categoriesFetcher, targData) + if err != nil { + return nil, fmt.Errorf("Error in category mapping : %s", err.Error()) + } - if targData != nil && adapterBids != nil { - auc.setRoundedPrices(targData.priceGranularity) - cacheErrs := auc.doCache(ctx, e.cache, targData, bidRequest, 60, &e.defaultTTLs, bidCategory) - if len(cacheErrs) > 0 { - errs = append(errs, cacheErrs...) + auc := newAuction(adapterBids, len(bidRequest.Imp)) + + if targData != nil { + auc.setRoundedPrices(targData.priceGranularity) + cacheErrs := auc.doCache(ctx, e.cache, targData, bidRequest, 60, &e.defaultTTLs, bidCategory) + if len(cacheErrs) > 0 { + errs = append(errs, cacheErrs...) + } + targData.setTargeting(auc, bidRequest.App != nil, bidCategory) } - targData.setTargeting(auc, bidRequest.App != nil, bidCategory) } + // Build the response return e.buildBidResponse(ctx, liveAdapters, adapterBids, bidRequest, resolvedRequest, adapterExtra, errs) } @@ -169,11 +174,12 @@ func (e *exchange) makeAuctionContext(ctx context.Context, needsCache bool) (auc } // This piece sends all the requests to the bidder adapters and gathers the results. -func (e *exchange) getAllBids(ctx context.Context, cleanRequests map[openrtb_ext.BidderName]*openrtb.BidRequest, aliases map[string]string, bidAdjustments map[string]float64, blabels map[openrtb_ext.BidderName]*pbsmetrics.AdapterLabels, conversions currencies.Conversions) (map[openrtb_ext.BidderName]*pbsOrtbSeatBid, map[openrtb_ext.BidderName]*seatResponseExtra) { +func (e *exchange) getAllBids(ctx context.Context, cleanRequests map[openrtb_ext.BidderName]*openrtb.BidRequest, aliases map[string]string, bidAdjustments map[string]float64, blabels map[openrtb_ext.BidderName]*pbsmetrics.AdapterLabels, conversions currencies.Conversions) (map[openrtb_ext.BidderName]*pbsOrtbSeatBid, map[openrtb_ext.BidderName]*seatResponseExtra, bool) { // Set up pointers to the bid results adapterBids := make(map[openrtb_ext.BidderName]*pbsOrtbSeatBid, len(cleanRequests)) adapterExtra := make(map[openrtb_ext.BidderName]*seatResponseExtra, len(cleanRequests)) chBids := make(chan *bidResponseWrapper, len(cleanRequests)) + bidsFound := false for bidderName, req := range cleanRequests { // Here we actually call the adapters and collect the bids. @@ -228,9 +234,13 @@ func (e *exchange) getAllBids(ctx context.Context, cleanRequests map[openrtb_ext brw := <-chBids adapterBids[brw.bidder] = brw.adapterBids adapterExtra[brw.bidder] = brw.adapterExtra + + if !bidsFound && adapterBids[brw.bidder] != nil && len(adapterBids[brw.bidder].bids) > 0 { + bidsFound = true + } } - return adapterBids, adapterExtra + return adapterBids, adapterExtra, bidsFound } func (e *exchange) recoverSafely(inner func(openrtb_ext.BidderName, openrtb_ext.BidderName, *openrtb.BidRequest, *pbsmetrics.AdapterLabels, currencies.Conversions), chBids chan *bidResponseWrapper) func(openrtb_ext.BidderName, openrtb_ext.BidderName, *openrtb.BidRequest, *pbsmetrics.AdapterLabels, currencies.Conversions) { @@ -321,7 +331,7 @@ func (e *exchange) buildBidResponse(ctx context.Context, liveAdapters []openrtb_ return bidResponse, err } -func applyCategoryMapping(requestExt openrtb_ext.ExtRequest, seatBids map[openrtb_ext.BidderName]*pbsOrtbSeatBid, categoriesFetcher stored_requests.CategoryFetcher, targData *targetData) (map[string]string, map[openrtb_ext.BidderName]*pbsOrtbSeatBid, error) { +func applyCategoryMapping(ctx context.Context, requestExt openrtb_ext.ExtRequest, seatBids map[openrtb_ext.BidderName]*pbsOrtbSeatBid, categoriesFetcher stored_requests.CategoryFetcher, targData *targetData) (map[string]string, map[openrtb_ext.BidderName]*pbsOrtbSeatBid, error) { res := make(map[string]string) type bidDedupe struct { @@ -375,7 +385,7 @@ func applyCategoryMapping(requestExt openrtb_ext.ExtRequest, seatBids map[openrt continue } else { //if unique IAB category is present then translate it to the adserver category based on mapping file - category, err = categoriesFetcher.FetchCategories(primaryAdServer, publisher, bidIabCat[0]) + category, err = categoriesFetcher.FetchCategories(ctx, primaryAdServer, publisher, bidIabCat[0]) if err != nil || category == "" { //TODO: add metrics //if mapping required but no mapping file is found then discard the bid diff --git a/exchange/exchange_test.go b/exchange/exchange_test.go index 66823055207..868ca097d27 100644 --- a/exchange/exchange_test.go +++ b/exchange/exchange_test.go @@ -587,7 +587,7 @@ func TestCategoryMapping(t *testing.T) { adapterBids[bidderName1] = &seatBid - bidCategory, adapterBids, err := applyCategoryMapping(requestExt, adapterBids, categoriesFetcher, targData) + bidCategory, adapterBids, err := applyCategoryMapping(nil, requestExt, adapterBids, categoriesFetcher, targData) assert.Equal(t, nil, err, "Category mapping error should be empty") assert.Equal(t, "10.00_Electronics_30s", bidCategory["bid_id1"], "Category mapping doesn't match") @@ -652,7 +652,7 @@ func TestCategoryDedupe(t *testing.T) { adapterBids[bidderName1] = &seatBid - bidCategory, adapterBids, err := applyCategoryMapping(requestExt, adapterBids, categoriesFetcher, targData) + bidCategory, adapterBids, err := applyCategoryMapping(nil, requestExt, adapterBids, categoriesFetcher, targData) assert.Equal(t, nil, err, "Category mapping error should be empty") assert.Equal(t, 2, len(adapterBids[bidderName1].bids), "Bidders number doesn't match") diff --git a/exchange/targeting.go b/exchange/targeting.go index 66efa5f49c0..560f62dcfb3 100644 --- a/exchange/targeting.go +++ b/exchange/targeting.go @@ -51,7 +51,7 @@ func (targData *targetData) setTargeting(auc *auction, isApp bool, categoryMappi targData.addKeys(targets, openrtb_ext.HbVastCacheKey, vastID, bidderName, isOverallWinner) } if deal := topBidPerBidder.bid.DealID; len(deal) > 0 { - targData.addKeys(targets, openrtb_ext.HbDealIdConstantKey, deal, bidderName, isOverallWinner) + targData.addKeys(targets, openrtb_ext.HbDealIDConstantKey, deal, bidderName, isOverallWinner) } if isApp { diff --git a/exchange/utils.go b/exchange/utils.go index 8b7173390b1..d5c8924a287 100644 --- a/exchange/utils.go +++ b/exchange/utils.go @@ -45,7 +45,16 @@ func cleanOpenRTBRequests(ctx context.Context, for bidder, bidReq := range requestsByBidder { // Fixes #820 coreBidder := resolveBidder(bidder.String(), aliases) - if ok, err := gDPR.PersonalInfoAllowed(ctx, coreBidder, consent); !ok && err == nil { + + var publisherID string + if bidReq.Site != nil && bidReq.Site.Publisher != nil && bidReq.Site.Publisher.ID != "" { + publisherID = bidReq.Site.Publisher.ID + } else if bidReq.App != nil && bidReq.App.Publisher != nil { + publisherID = bidReq.App.Publisher.ID + } else { + publisherID = "" + } + if ok, err := gDPR.PersonalInfoAllowed(ctx, coreBidder, publisherID, consent); !ok && err == nil { cleanPI(bidReq, labels.RType == pbsmetrics.ReqTypeAMP) } } diff --git a/exchange/utils_test.go b/exchange/utils_test.go index 7bae4feacf2..91e4f0fb417 100644 --- a/exchange/utils_test.go +++ b/exchange/utils_test.go @@ -24,7 +24,7 @@ func (p *permissionsMock) BidderSyncAllowed(ctx context.Context, bidder openrtb_ return true, nil } -func (p *permissionsMock) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, consent string) (bool, error) { +func (p *permissionsMock) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, PublisherID string, consent string) (bool, error) { if bidder == "appnexus" { return true, nil } diff --git a/gdpr/gdpr.go b/gdpr/gdpr.go index 56b90e1786b..bdba008a77a 100644 --- a/gdpr/gdpr.go +++ b/gdpr/gdpr.go @@ -22,7 +22,7 @@ type Permissions interface { // Determines whether or not to send PI information to a bidder, or mask it out. // // If the consent string was nonsenical, the returned error will be an ErrorMalformedConsent. - PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, consent string) (bool, error) + PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, PublisherID string, consent string) (bool, error) } // NewPermissions gets an instance of the Permissions for use elsewhere in the project. diff --git a/gdpr/impl.go b/gdpr/impl.go index 424e1eff1eb..2fe6a67e99f 100644 --- a/gdpr/impl.go +++ b/gdpr/impl.go @@ -38,7 +38,12 @@ func (p *permissionsImpl) BidderSyncAllowed(ctx context.Context, bidder openrtb_ return false, nil } -func (p *permissionsImpl) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, consent string) (bool, error) { +func (p *permissionsImpl) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, PublisherID string, consent string) (bool, error) { + _, ok := p.cfg.NonStandardPublisherMap[PublisherID] + if ok { + return true, nil + } + id, ok := p.vendorIDs[bidder] if ok { return p.allowPI(ctx, id, consent) @@ -125,6 +130,6 @@ func (a AlwaysAllow) BidderSyncAllowed(ctx context.Context, bidder openrtb_ext.B return true, nil } -func (a AlwaysAllow) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, consent string) (bool, error) { +func (a AlwaysAllow) PersonalInfoAllowed(ctx context.Context, bidder openrtb_ext.BidderName, PublisherID string, consent string) (bool, error) { return true, nil } diff --git a/gdpr/impl_test.go b/gdpr/impl_test.go index 001fefc5e34..a1d4af3346d 100644 --- a/gdpr/impl_test.go +++ b/gdpr/impl_test.go @@ -175,11 +175,17 @@ func TestAllowPersonalInfo(t *testing.T) { } // PI needs both purposes to succeed - allowPI, err := perms.PersonalInfoAllowed(context.Background(), openrtb_ext.BidderAppnexus, "BOS2bx5OS2bx5ABABBAAABoAAAABBwAA") + allowPI, err := perms.PersonalInfoAllowed(context.Background(), openrtb_ext.BidderAppnexus, "", "BOS2bx5OS2bx5ABABBAAABoAAAABBwAA") assertNilErr(t, err) assertBoolsEqual(t, false, allowPI) - allowPI, err = perms.PersonalInfoAllowed(context.Background(), openrtb_ext.BidderPubmatic, "BOS2bx5OS2bx5ABABBAAABoAAAABBwAA") + allowPI, err = perms.PersonalInfoAllowed(context.Background(), openrtb_ext.BidderPubmatic, "", "BOS2bx5OS2bx5ABABBAAABoAAAABBwAA") + assertNilErr(t, err) + assertBoolsEqual(t, true, allowPI) + + // Assert that an item that otherwise would not be allowed PI access, gets approved because it is found in the GDPR.NonStandardPublishers array + perms.cfg.NonStandardPublisherMap = map[string]int{"appNexusAppID": 1} + allowPI, err = perms.PersonalInfoAllowed(context.Background(), openrtb_ext.BidderAppnexus, "appNexusAppID", "BOS2bx5OS2bx5ABABBAAABoAAAABBwAA") assertNilErr(t, err) assertBoolsEqual(t, true, allowPI) } diff --git a/macros/macros.go b/macros/macros.go index d84117b0cd6..d299a6f0d78 100644 --- a/macros/macros.go +++ b/macros/macros.go @@ -1,7 +1,7 @@ package macros import ( - "strings" + "bytes" "text/template" ) @@ -19,11 +19,12 @@ type UserSyncTemplateParams struct { // ResolveMacros resolves macros in the given template with the provided params func ResolveMacros(aTemplate template.Template, params interface{}) (string, error) { - strBuilder := strings.Builder{} - err := aTemplate.Execute(&strBuilder, params) + strBuf := bytes.Buffer{} + + err := aTemplate.Execute(&strBuf, params) if err != nil { return "", err } - res := strBuilder.String() + res := strBuf.String() return res, nil } diff --git a/openrtb_ext/bid.go b/openrtb_ext/bid.go index fc67a932fe0..f44d2320e2f 100644 --- a/openrtb_ext/bid.go +++ b/openrtb_ext/bid.go @@ -87,7 +87,7 @@ const ( // HbBidderConstantKey is the name of the Bidder. For example, "appnexus" or "rubicon". HbBidderConstantKey TargetingKey = "hb_bidder" HbSizeConstantKey TargetingKey = "hb_size" - HbDealIdConstantKey TargetingKey = "hb_deal" + HbDealIDConstantKey TargetingKey = "hb_deal" // HbCacheKey and HbVastCacheKey store UUIDs which can be used to fetch things from prebid cache. // Callers should *never* assume that either of these exist, since the call to the cache may always fail. diff --git a/openrtb_ext/bid_response_video.go b/openrtb_ext/bid_response_video.go index a91006f6e20..058d06c30f9 100644 --- a/openrtb_ext/bid_response_video.go +++ b/openrtb_ext/bid_response_video.go @@ -14,7 +14,7 @@ type AdPod struct { } type VideoTargeting struct { - Hb_pb string `json:"hb_pb"` - Hb_pb_cat_dur string `json:"hb_pb_cat_dur"` - Hb_cache_id string `json:"hb_cache_id"` + HbPb string `json:"hb_pb"` + HbPbCatDur string `json:"hb_pb_cat_dur"` + HbCacheID string `json:"hb_cache_id"` } diff --git a/openrtb_ext/bidders.go b/openrtb_ext/bidders.go index 6f8bcffa7f7..7a45c5b88d9 100644 --- a/openrtb_ext/bidders.go +++ b/openrtb_ext/bidders.go @@ -37,11 +37,13 @@ const ( BidderImprovedigital BidderName = "improvedigital" BidderIx BidderName = "ix" BidderLifestreet BidderName = "lifestreet" + BidderMgid BidderName = "mgid" BidderOpenx BidderName = "openx" BidderPubmatic BidderName = "pubmatic" BidderPulsepoint BidderName = "pulsepoint" BidderRhythmone BidderName = "rhythmone" BidderRubicon BidderName = "rubicon" + BidderSharethrough BidderName = "sharethrough" BidderSomoaudience BidderName = "somoaudience" BidderSovrn BidderName = "sovrn" BidderSonobi BidderName = "sonobi" @@ -68,10 +70,12 @@ var BidderMap = map[string]BidderName{ "ix": BidderIx, "lifestreet": BidderLifestreet, "openx": BidderOpenx, + "mgid": BidderMgid, "pubmatic": BidderPubmatic, "pulsepoint": BidderPulsepoint, "rhythmone": BidderRhythmone, "rubicon": BidderRubicon, + "sharethrough": BidderSharethrough, "somoaudience": BidderSomoaudience, "sovrn": BidderSovrn, "sonobi": BidderSonobi, diff --git a/openrtb_ext/imp_mgid.go b/openrtb_ext/imp_mgid.go new file mode 100644 index 00000000000..3cf4530e1de --- /dev/null +++ b/openrtb_ext/imp_mgid.go @@ -0,0 +1,11 @@ +package openrtb_ext + +// ExtImpMgid defines the contract for bidrequest.imp[i].ext.mgid +type ExtImpMgid struct { + AccountId string `json:"accountId"` + PlacementId string `json:"placementId"` + Cur string `json:"cur"` + Currency string `json:"currency"` + BidFloor float64 `json:"bidfloor"` + BidFloor2 float64 `json:"bidFloor"` +} diff --git a/openrtb_ext/imp_pubmatic.go b/openrtb_ext/imp_pubmatic.go index bafafe5f569..fd97836dd32 100644 --- a/openrtb_ext/imp_pubmatic.go +++ b/openrtb_ext/imp_pubmatic.go @@ -3,7 +3,8 @@ package openrtb_ext import "encoding/json" // ExtImpPubmatic defines the contract for bidrequest.imp[i].ext.pubmatic -// PublisherId and adSlot are mandatory parameters, others are optional parameters +// PublisherId is mandatory parameters, others are optional parameters +// AdSlot is identifier for specific ad placement or ad tag // Keywords is bid specific parameter, // WrapExt needs to be sent once per bid request @@ -14,7 +15,7 @@ type ExtImpPubmatic struct { Keywords []*ExtImpPubmaticKeyVal `json:"keywords,omitempty"` } -// ExtImpAppnexusKeyVal defines the contract for bidrequest.imp[i].ext.appnexus.keywords[i] +// ExtImpPubmaticKeyVal defines the contract for bidrequest.imp[i].ext.pubmatic.keywords[i] type ExtImpPubmaticKeyVal struct { Key string `json:"key,omitempty"` Values []string `json:"value,omitempty"` diff --git a/openrtb_ext/imp_sharethrough.go b/openrtb_ext/imp_sharethrough.go new file mode 100644 index 00000000000..21c6b4fd140 --- /dev/null +++ b/openrtb_ext/imp_sharethrough.go @@ -0,0 +1,102 @@ +package openrtb_ext + +import "encoding/json" + +type ExtImpSharethrough struct { + PlacementKey string `json:"pkey"` + Iframe bool `json:"iframe"` +} + +// ExtImpSharethrough defines the contract for bidrequest.imp[i].ext.sharethrough +type ExtImpSharethroughResponse struct { + AdServerRequestID string `json:"adserverRequestId"` + BidID string `json:"bidId"` + CookieSyncUrls []string `json:"cookieSyncUrls"` + Creatives []ExtImpSharethroughCreative `json:"creatives"` + Placement ExtImpSharethroughPlacement `json:"placement"` + StxUserID string `json:"stxUserId"` +} +type ExtImpSharethroughCreative struct { + AuctionWinID string `json:"auctionWinId"` + CPM float64 `json:"cpm"` + Metadata ExtImpSharethroughCreativeMetadata `json:"creative"` + Version int `json:"version"` +} + +type ExtImpSharethroughCreativeMetadata struct { + Action string `json:"action"` + Advertiser string `json:"advertiser"` + AdvertiserKey string `json:"advertiser_key"` + Beacons ExtImpSharethroughCreativeBeacons `json:"beacons"` + BrandLogoURL string `json:"brand_logo_url"` + CampaignKey string `json:"campaign_key"` + CreativeKey string `json:"creative_key"` + CustomEngagementAction string `json:"custom_engagement_action"` + CustomEngagementLabel string `json:"custom_engagement_label"` + CustomEngagementURL string `json:"custom_engagement_url"` + DealID string `json:"deal_id"` + Description string `json:"description"` + ForceClickToPlay bool `json:"force_click_to_play"` + IconURL string `json:"icon_url"` + ImpressionHTML string `json:"impression_html"` + InstantPlayMobileCount int `json:"instant_play_mobile_count"` + InstantPlayMobileURL string `json:"instant_play_mobile_url"` + MediaURL string `json:"media_url"` + ShareURL string `json:"share_url"` + SourceID string `json:"source_id"` + ThumbnailURL string `json:"thumbnail_url"` + Title string `json:"title"` + VariantKey string `json:"variant_key"` +} + +type ExtImpSharethroughCreativeBeacons struct { + Click []string `json:"click"` + Impression []string `json:"impression"` + Play []string `json:"play"` + Visible []string `json:"visible"` + WinNotification []string `json:"win-notification"` +} + +type ExtImpSharethroughPlacement struct { + AllowInstantPlay bool `json:"allow_instant_play"` + ArticlesBeforeFirstAd int `json:"articles_before_first_ad"` + ArticlesBetweenAds int `json:"articles_between_ads"` + Layout string `json:"layout"` + Metadata json.RawMessage `json:"metadata"` + PlacementAttributes ExtImpSharethroughPlacementAttributes `json:"placementAttributes"` + Status string `json:"status"` +} + +type ExtImpSharethroughPlacementThirdPartyPartner struct { + Key string `json:"key"` + Tag string `json:"tag"` +} + +type ExtImpSharethroughPlacementAttributes struct { + AdServerKey string `json:"ad_server_key"` + AdServerPath string `json:"ad_server_path"` + AllowDynamicCropping bool `json:"allow_dynamic_cropping"` + AppThirdPartyPartners []string `json:"app_third_party_partners"` + CustomCardCSS string `json:"custom_card_css"` + DFPPath string `json:"dfp_path"` + DirectSellPromotedByText string `json:"direct_sell_promoted_by_text"` + Domain string `json:"domain"` + EnableLinkRedirection bool `json:"enable_link_redirection"` + FeaturedContent json.RawMessage `json:"featured_content"` + MaxHeadlineLength int `json:"max_headline_length"` + MultiAdPlacement bool `json:"multi_ad_placement"` + PromotedByText string `json:"promoted_by_text"` + PublisherKey string `json:"publisher_key"` + RenderingPixelOffset int `json:"rendering_pixel_offset"` + SafeFrameSize []int `json:"safe_frame_size"` + SiteKey string `json:"site_key"` + StrOptOutURL string `json:"str_opt_out_url"` + Template string `json:"template"` + ThirdPartyPartners []ExtImpSharethroughPlacementThirdPartyPartner `json:"third_party_partners"` +} + +type ExtImpSharethroughExt struct { + Pkey string `json:"pkey"` + Iframe bool `json:"iframe"` + IframeSize []int `json:"iframeSize"` +} diff --git a/pbsmetrics/prometheus/prometheus.go b/pbsmetrics/prometheus/prometheus.go index 39864e7eace..3d09f10a4d8 100644 --- a/pbsmetrics/prometheus/prometheus.go +++ b/pbsmetrics/prometheus/prometheus.go @@ -90,12 +90,12 @@ func NewMetrics(cfg config.PrometheusMetrics) *Metrics { metrics.Registry.MustRegister(metrics.adaptBids) metrics.storedReqCacheResult = newCounter(cfg, "stored_request_cache_performance", "Number of stored request cache hits vs miss", - standardLabelNames, + []string{"cache_result"}, ) metrics.Registry.MustRegister(metrics.storedReqCacheResult) metrics.storedImpCacheResult = newCounter(cfg, "stored_imp_cache_performance", "Number of stored imp cache hits vs miss", - standardLabelNames, + []string{"cache_result"}, ) metrics.Registry.MustRegister(metrics.storedImpCacheResult) metrics.adaptPrices = newHistogram(cfg, "adapter_prices", @@ -365,6 +365,11 @@ func initializeTimeSeries(m *Metrics) { for _, l := range cookieLabels { _ = m.adaptCookieSync.With(l) } + cacheLabels := addDimension([]prometheus.Labels{}, "cache_result", cacheResultAsString()) + for _, l := range cacheLabels { + _ = m.storedImpCacheResult.With(l) + _ = m.storedReqCacheResult.With(l) + } } // addDimesion will expand a slice of labels to add the dimension of a new set of values for a new label name @@ -463,6 +468,15 @@ func adapterErrorsAsString() []string { return output } +func cacheResultAsString() []string { + list := pbsmetrics.CacheResults() + output := make([]string, len(list)) + for i, s := range list { + output[i] = string(s) + } + return output +} + func adaptersAsString() []string { list := openrtb_ext.BidderList() output := make([]string, len(list)) diff --git a/pbsmetrics/prometheus/prometheus_test.go b/pbsmetrics/prometheus/prometheus_test.go index a3dda4b4353..75119e7e1b0 100644 --- a/pbsmetrics/prometheus/prometheus_test.go +++ b/pbsmetrics/prometheus/prometheus_test.go @@ -279,6 +279,40 @@ func TestAdapterTimeMetrics(t *testing.T) { } +func TestRecordStoredReqCacheResult(t *testing.T) { + proMetrics := newTestMetricsEngine() + + metricCacheHit := dto.Metric{} + metricCacheMiss := dto.Metric{} + + proMetrics.RecordStoredReqCacheResult(pbsmetrics.CacheHit, 2) + proMetrics.RecordStoredReqCacheResult(pbsmetrics.CacheHit, 0) + proMetrics.RecordStoredReqCacheResult(pbsmetrics.CacheMiss, 1) + + proMetrics.storedReqCacheResult.WithLabelValues(string(pbsmetrics.CacheHit)).Write(&metricCacheHit) + proMetrics.storedReqCacheResult.WithLabelValues(string(pbsmetrics.CacheMiss)).Write(&metricCacheMiss) + + assertCounterValue(t, "stored_request_cache_performance[hit]", &metricCacheHit, 2) + assertCounterValue(t, "stored_request_cache_performance[miss]", &metricCacheMiss, 1) +} + +func TestRecordStoredImpCacheResult(t *testing.T) { + proMetrics := newTestMetricsEngine() + + metricCacheHit := dto.Metric{} + metricCacheMiss := dto.Metric{} + + proMetrics.RecordStoredImpCacheResult(pbsmetrics.CacheHit, 2) + proMetrics.RecordStoredImpCacheResult(pbsmetrics.CacheHit, 0) + proMetrics.RecordStoredImpCacheResult(pbsmetrics.CacheMiss, 1) + + proMetrics.storedImpCacheResult.WithLabelValues(string(pbsmetrics.CacheHit)).Write(&metricCacheHit) + proMetrics.storedImpCacheResult.WithLabelValues(string(pbsmetrics.CacheMiss)).Write(&metricCacheMiss) + + assertCounterValue(t, "stored_imp_cache_performance[hit]", &metricCacheHit, 2) + assertCounterValue(t, "stored_imp_cache_performance[miss]", &metricCacheMiss, 1) +} + func TestCookieMetrics(t *testing.T) { proMetrics := newTestMetricsEngine() @@ -332,6 +366,7 @@ func TestMetricsExist(t *testing.T) { if err := proMetrics.Registry.Register(prometheus.NewCounter(prometheus.CounterOpts{ Namespace: "prebid", + Subsystem: "server", Name: "active_connections", Help: "Current number of active (open) connections.", })); err == nil { @@ -343,7 +378,7 @@ func newTestMetricsEngine() *Metrics { return NewMetrics(config.PrometheusMetrics{ Port: 8080, Namespace: "prebid", - Subsystem: "", + Subsystem: "server", }) } diff --git a/static/bidder-info/improvedigital.yaml b/static/bidder-info/improvedigital.yaml index 9f221fddd89..41e49faab9c 100644 --- a/static/bidder-info/improvedigital.yaml +++ b/static/bidder-info/improvedigital.yaml @@ -1,6 +1,11 @@ maintainer: email: "j.bartek@improvedigital.com" capabilities: + app: + mediaTypes: + - banner + - video site: mediaTypes: - banner + - video diff --git a/static/bidder-info/mgid.yaml b/static/bidder-info/mgid.yaml new file mode 100644 index 00000000000..f8ba6db60b1 --- /dev/null +++ b/static/bidder-info/mgid.yaml @@ -0,0 +1,11 @@ +maintainer: + email: "prebid@mgid.com" +capabilities: + app: + mediaTypes: + - banner + - native + site: + mediaTypes: + - banner + - native diff --git a/static/bidder-info/sharethrough.yaml b/static/bidder-info/sharethrough.yaml new file mode 100644 index 00000000000..09530be508c --- /dev/null +++ b/static/bidder-info/sharethrough.yaml @@ -0,0 +1,11 @@ +maintainer: + email: "pubgrowth.engineering@sharethrough.com" +capabilities: + app: + mediaTypes: + - native + - banner + site: + mediaTypes: + - native + - banner diff --git a/static/bidder-params/improvedigital.json b/static/bidder-params/improvedigital.json index 03ed8817156..55412c2f513 100644 --- a/static/bidder-params/improvedigital.json +++ b/static/bidder-params/improvedigital.json @@ -6,8 +6,22 @@ "properties": { "placementId": { "type": "integer", + "minimum": 1, "description": "An ID which identifies this placement of the impression" }, + "publisherId": { + "type": "integer", + "minimum": 1, + "description": "An ID which identifies publisher. Required when using a placementKey" + }, + "placementKey": { + "type": "string", + "description": "An uniq name which identifies this placement of the impression. Must be used with publisherId" + }, + "keyValues": { + "type": "object", + "description": "Contains one or more key-value pairings for key-value targeting" + }, "size": { "type": "object", "properties": { @@ -22,5 +36,9 @@ "description": "Placement size" } }, - "required": [] + "oneOf": [{ + "required": ["placementId"] + }, { + "required": ["publisherId", "placementKey"] + }] } diff --git a/static/bidder-params/mgid.json b/static/bidder-params/mgid.json new file mode 100644 index 00000000000..45f6df83b5d --- /dev/null +++ b/static/bidder-params/mgid.json @@ -0,0 +1,34 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema#", + "title": "Mgid Adapter Params", + "description": "A schema which validates params accepted by the Mgid adapter", + + "type": "object", + "properties": { + "accountId": { + "type": "string", + "description": "Internal Mgid account ID" + }, + "placementId": { + "type": "string", + "description": "Internal Mgid Placement ID" + }, + "cur": { + "type": "string", + "description": "optional bidfloor currency" + }, + "currency": { + "type": "string", + "description": "optional bidfloor currency" + }, + "bidfloor": { + "type": "number", + "description": "optional minimum acceptable bid, in CPM, USD by default" + }, + "bidFloor": { + "type": "number", + "description": "optional minimum acceptable bid, in CPM, USD by default" + } + }, + "required": ["accountId","placementId"] +} diff --git a/static/bidder-params/pubmatic.json b/static/bidder-params/pubmatic.json index b44375ed10a..1b6a2f03512 100644 --- a/static/bidder-params/pubmatic.json +++ b/static/bidder-params/pubmatic.json @@ -49,5 +49,5 @@ } } }, - "required": ["publisherId", "adSlot"] + "required": ["publisherId"] } diff --git a/static/bidder-params/sharethrough.json b/static/bidder-params/sharethrough.json new file mode 100644 index 00000000000..03f4ec293ec --- /dev/null +++ b/static/bidder-params/sharethrough.json @@ -0,0 +1,28 @@ +{ + "$schema": "http://json-schema.org/draft-04/schema#", + "title": "Sharethrough Adapter Params", + "description": "A schema which validates params accepted by the Sharethrough adapter", + "type": "object", + "properties": { + "pkey": { + "type": "string", + "description": "placement key to use." + }, + "iframe": { + "type": "boolean", + "description": "whether or not to stay in iframe", + "default": false + }, + "iframeSize": { + "type": "array", + "minItems": 2, + "maxItems": 2, + "items": { + "type": "integer" + }, + "description": "iframe dimensions", + "default": [0, 0] + } + }, + "required": ["pkey"] +} diff --git a/stored_requests/backends/db_fetcher/fetcher.go b/stored_requests/backends/db_fetcher/fetcher.go index abb58731775..f12166eafec 100644 --- a/stored_requests/backends/db_fetcher/fetcher.go +++ b/stored_requests/backends/db_fetcher/fetcher.go @@ -93,7 +93,7 @@ func (fetcher *dbFetcher) FetchRequests(ctx context.Context, requestIDs []string return storedRequestData, storedImpData, errs } -func (fetcher *dbFetcher) FetchCategories(primaryAdServer, publisherId, iabCategory string) (string, error) { +func (fetcher *dbFetcher) FetchCategories(ctx context.Context, primaryAdServer, publisherId, iabCategory string) (string, error) { return "", nil } diff --git a/stored_requests/backends/empty_fetcher/fetcher.go b/stored_requests/backends/empty_fetcher/fetcher.go index 298449c59f0..25e8ead434b 100644 --- a/stored_requests/backends/empty_fetcher/fetcher.go +++ b/stored_requests/backends/empty_fetcher/fetcher.go @@ -27,6 +27,6 @@ func (fetcher EmptyFetcher) FetchRequests(ctx context.Context, requestIDs []stri return } -func (fetcher EmptyFetcher) FetchCategories(primaryAdServer, publisherId, iabCategory string) (string, error) { +func (fetcher EmptyFetcher) FetchCategories(ctx context.Context, primaryAdServer, publisherId, iabCategory string) (string, error) { return "", nil } diff --git a/stored_requests/backends/file_fetcher/fetcher.go b/stored_requests/backends/file_fetcher/fetcher.go index 61c5265ca3c..60853f65da7 100644 --- a/stored_requests/backends/file_fetcher/fetcher.go +++ b/stored_requests/backends/file_fetcher/fetcher.go @@ -22,12 +22,7 @@ func NewFileFetcher(directory string) (stored_requests.AllFetcher, error) { type eagerFetcher struct { FileSystem FileSystem - Categories map[string]map[string]Category -} - -type Category struct { - Id string - Name string + Categories map[string]map[string]stored_requests.Category } func (fetcher *eagerFetcher) FetchRequests(ctx context.Context, requestIDs []string, impIDs []string) (map[string]json.RawMessage, map[string]json.RawMessage, []error) { @@ -38,7 +33,7 @@ func (fetcher *eagerFetcher) FetchRequests(ctx context.Context, requestIDs []str return storedRequests, storedImpressions, errs } -func (fetcher *eagerFetcher) FetchCategories(primaryAdServer, publisherId, iabCategory string) (string, error) { +func (fetcher *eagerFetcher) FetchCategories(ctx context.Context, primaryAdServer, publisherId, iabCategory string) (string, error) { fileName := primaryAdServer if len(publisherId) != 0 { @@ -46,7 +41,7 @@ func (fetcher *eagerFetcher) FetchCategories(primaryAdServer, publisherId, iabCa } if fetcher.Categories == nil { - fetcher.Categories = make(map[string]map[string]Category) + fetcher.Categories = make(map[string]map[string]stored_requests.Category) } if data, ok := fetcher.Categories[fileName]; ok { return data[iabCategory].Id, nil @@ -56,7 +51,7 @@ func (fetcher *eagerFetcher) FetchCategories(primaryAdServer, publisherId, iabCa if file, ok := primaryAdServerDir.Files[fileName]; ok { - tmp := make(map[string]Category) + tmp := make(map[string]stored_requests.Category) if err := json.Unmarshal(file, &tmp); err != nil { return "", fmt.Errorf("Unable to unmarshal categories for adserver: '%s', publisherId: '%s'", primaryAdServer, publisherId) diff --git a/stored_requests/backends/file_fetcher/fetcher_test.go b/stored_requests/backends/file_fetcher/fetcher_test.go index 11628352f1f..2429a77cd25 100644 --- a/stored_requests/backends/file_fetcher/fetcher_test.go +++ b/stored_requests/backends/file_fetcher/fetcher_test.go @@ -114,7 +114,7 @@ func TestCategoriesFetcherWithPublisher(t *testing.T) { if err != nil { t.Errorf("Failed to create a category Fetcher: %v", err) } - category, err := fetcher.FetchCategories("test", "categories", "IAB1-1") + category, err := fetcher.FetchCategories(nil, "test", "categories", "IAB1-1") assert.Equal(t, nil, err, "Categories were loaded incorrectly") assert.Equal(t, "Beverages", category, "Categories were loaded incorrectly") } @@ -124,7 +124,7 @@ func TestCategoriesFetcherWithoutPublisher(t *testing.T) { if err != nil { t.Errorf("Failed to create a category Fetcher: %v", err) } - category, err := fetcher.FetchCategories("test", "", "IAB1-1") + category, err := fetcher.FetchCategories(nil, "test", "", "IAB1-1") assert.Equal(t, nil, err, "Categories were loaded incorrectly") assert.Equal(t, "VideoGames", category, "Categories were loaded incorrectly") } @@ -134,7 +134,7 @@ func TestCategoriesFetcherNoCategory(t *testing.T) { if err != nil { t.Errorf("Failed to create a category Fetcher: %v", err) } - _, fetchingErr := fetcher.FetchCategories("test", "", "IAB1-100") + _, fetchingErr := fetcher.FetchCategories(nil, "test", "", "IAB1-100") assert.Equal(t, fmt.Errorf("Unable to find category for adserver 'test', publisherId: '', iab category: 'IAB1-100'"), fetchingErr, "Categories were loaded incorrectly") } @@ -144,7 +144,7 @@ func TestCategoriesFetcherBrokenJson(t *testing.T) { if err != nil { t.Errorf("Failed to create a category Fetcher: %v", err) } - _, fetchingErr := fetcher.FetchCategories("test", "broken", "IAB1-100") + _, fetchingErr := fetcher.FetchCategories(nil, "test", "broken", "IAB1-100") assert.Equal(t, fmt.Errorf("Unable to unmarshal categories for adserver: 'test', publisherId: 'broken'"), fetchingErr, "Categories were loaded incorrectly") } @@ -154,7 +154,7 @@ func TestCategoriesFetcherNoCategoriesFile(t *testing.T) { if err != nil { t.Errorf("Failed to create a category Fetcher: %v", err) } - _, fetchingErr := fetcher.FetchCategories("test", "not_exists", "IAB1-100") + _, fetchingErr := fetcher.FetchCategories(nil, "test", "not_exists", "IAB1-100") assert.Equal(t, fmt.Errorf("Unable to find mapping file for adserver: 'test', publisherId: 'not_exists'"), fetchingErr, "Categories were loaded incorrectly") } diff --git a/stored_requests/backends/http_fetcher/fetcher.go b/stored_requests/backends/http_fetcher/fetcher.go index 34175c38ee6..b7e42c9e6cf 100644 --- a/stored_requests/backends/http_fetcher/fetcher.go +++ b/stored_requests/backends/http_fetcher/fetcher.go @@ -56,9 +56,10 @@ func NewFetcher(client *http.Client, endpoint string) *HttpFetcher { } type HttpFetcher struct { - client *http.Client - Endpoint string - hasQuery bool + client *http.Client + Endpoint string + hasQuery bool + Categories map[string]map[string]stored_requests.Category } func (fetcher *HttpFetcher) FetchRequests(ctx context.Context, requestIDs []string, impIDs []string) (requestData map[string]json.RawMessage, impData map[string]json.RawMessage, errs []error) { @@ -80,8 +81,54 @@ func (fetcher *HttpFetcher) FetchRequests(ctx context.Context, requestIDs []stri return } -func (fetcher *HttpFetcher) FetchCategories(primaryAdServer, publisherId, iabCategory string) (string, error) { - return "", nil +func (fetcher *HttpFetcher) FetchCategories(ctx context.Context, primaryAdServer, publisherId, iabCategory string) (string, error) { + if fetcher.Categories == nil { + fetcher.Categories = make(map[string]map[string]stored_requests.Category) + } + + //in NewFetcher function there is a code to add "?" at the end of url + //in case of categories we don't expect to have any parameters, that's why we need to remove "?" + var dataName, url string + if publisherId != "" { + dataName = fmt.Sprintf("%s_%s", primaryAdServer, publisherId) + url = fmt.Sprintf("%s/%s/%s.json", strings.TrimSuffix(fetcher.Endpoint, "?"), primaryAdServer, publisherId) + } else { + dataName = primaryAdServer + url = fmt.Sprintf("%s/%s.json", strings.TrimSuffix(fetcher.Endpoint, "?"), primaryAdServer) + } + + if data, ok := fetcher.Categories[dataName]; ok { + if val, ok := data[iabCategory]; ok { + return val.Id, nil + } else { + return "", fmt.Errorf("Unable to find category mapping for adserver: '%s', publisherId: '%s'", primaryAdServer, publisherId) + } + } + + httpReq, err := http.NewRequest("GET", url, nil) + if err != nil { + return "", err + } + + httpResp, err := ctxhttp.Do(ctx, fetcher.client, httpReq) + if err != nil { + return "", err + } + defer httpResp.Body.Close() + + respBytes, err := ioutil.ReadAll(httpResp.Body) + tmp := make(map[string]stored_requests.Category) + + if err := json.Unmarshal(respBytes, &tmp); err != nil { + return "", fmt.Errorf("Unable to unmarshal categories for adserver: '%s', publisherId: '%s'", primaryAdServer, publisherId) + } + fetcher.Categories[dataName] = tmp + + if val, ok := tmp[iabCategory]; ok { + return val.Id, nil + } else { + return "", fmt.Errorf("Unable to find category mapping for adserver: '%s', publisherId: '%s'", primaryAdServer, publisherId) + } } func buildRequest(endpoint string, requestIDs []string, impIDs []string) (*http.Request, error) { diff --git a/stored_requests/fetcher.go b/stored_requests/fetcher.go index f42fb985eb4..808495e4584 100644 --- a/stored_requests/fetcher.go +++ b/stored_requests/fetcher.go @@ -27,13 +27,13 @@ type Fetcher interface { type CategoryFetcher interface { // FetchCategories fetches the ad-server/publisher specific category for the given IAB category - FetchCategories(primaryAdServer, publisherId, iabCategory string) (string, error) + FetchCategories(ctx context.Context, primaryAdServer, publisherId, iabCategory string) (string, error) } // AllFetcher is an iterface that encapsulates both the original Fetcher and the CategoryFetcher type AllFetcher interface { FetchRequests(ctx context.Context, requestIDs []string, impIDs []string) (requestData map[string]json.RawMessage, impData map[string]json.RawMessage, errs []error) - FetchCategories(primaryAdServer, publisherId, iabCategory string) (string, error) + FetchCategories(ctx context.Context, primaryAdServer, publisherId, iabCategory string) (string, error) } // NotFoundError is an error type to flag that an ID was not found by the Fetcher. @@ -44,6 +44,11 @@ type NotFoundError struct { DataType string } +type Category struct { + Id string + Name string +} + func (e NotFoundError) Error() string { return fmt.Sprintf(`Stored %s with ID="%s" not found.`, e.DataType, e.ID) } @@ -176,7 +181,7 @@ func (f *fetcherWithCache) FetchRequests(ctx context.Context, requestIDs []strin return } -func (f *fetcherWithCache) FetchCategories(primaryAdServer, publisherId, iabCategory string) (string, error) { +func (f *fetcherWithCache) FetchCategories(ctx context.Context, primaryAdServer, publisherId, iabCategory string) (string, error) { return "", nil } diff --git a/stored_requests/fetcher_test.go b/stored_requests/fetcher_test.go index 13a003637c0..c1040acdb90 100644 --- a/stored_requests/fetcher_test.go +++ b/stored_requests/fetcher_test.go @@ -215,7 +215,7 @@ func (f *mockFetcher) FetchRequests(ctx context.Context, requestIDs []string, im return args.Get(0).(map[string]json.RawMessage), args.Get(1).(map[string]json.RawMessage), args.Get(2).([]error) } -func (f *mockFetcher) FetchCategories(primaryAdServer, publisherId, iabCategory string) (string, error) { +func (f *mockFetcher) FetchCategories(ctx context.Context, primaryAdServer, publisherId, iabCategory string) (string, error) { return "", nil } diff --git a/stored_requests/multifetcher.go b/stored_requests/multifetcher.go index 01cbb8098e6..24cf848448c 100644 --- a/stored_requests/multifetcher.go +++ b/stored_requests/multifetcher.go @@ -36,10 +36,10 @@ func (mf MultiFetcher) FetchRequests(ctx context.Context, requestIDs []string, i return } -func (mf MultiFetcher) FetchCategories(primaryAdServer, publisherId, iabCategory string) (string, error) { +func (mf MultiFetcher) FetchCategories(ctx context.Context, primaryAdServer, publisherId, iabCategory string) (string, error) { for _, f := range mf { if cf, ok := f.(CategoryFetcher); ok { - iabCategory, _ := cf.FetchCategories(primaryAdServer, publisherId, iabCategory) + iabCategory, _ := cf.FetchCategories(ctx, primaryAdServer, publisherId, iabCategory) if iabCategory != "" { return iabCategory, nil } diff --git a/usersync/usersyncers/syncer.go b/usersync/usersyncers/syncer.go index a0a69e7fb7e..f062bee36aa 100644 --- a/usersync/usersyncers/syncer.go +++ b/usersync/usersyncers/syncer.go @@ -2,6 +2,7 @@ package usersyncers import ( "github.com/prebid/prebid-server/adapters/gamoshi" + "github.com/prebid/prebid-server/adapters/sharethrough" "strings" "text/template" @@ -22,6 +23,7 @@ import ( "github.com/prebid/prebid-server/adapters/improvedigital" "github.com/prebid/prebid-server/adapters/ix" "github.com/prebid/prebid-server/adapters/lifestreet" + "github.com/prebid/prebid-server/adapters/mgid" "github.com/prebid/prebid-server/adapters/openx" "github.com/prebid/prebid-server/adapters/pubmatic" "github.com/prebid/prebid-server/adapters/pulsepoint" @@ -63,11 +65,13 @@ func NewSyncerMap(cfg *config.Configuration) map[openrtb_ext.BidderName]usersync insertIntoMap(cfg, syncers, openrtb_ext.BidderPulsepoint, pulsepoint.NewPulsepointSyncer) insertIntoMap(cfg, syncers, openrtb_ext.BidderRhythmone, rhythmone.NewRhythmoneSyncer) insertIntoMap(cfg, syncers, openrtb_ext.BidderRubicon, rubicon.NewRubiconSyncer) + insertIntoMap(cfg, syncers, openrtb_ext.BidderSharethrough, sharethrough.NewSharethroughSyncer) insertIntoMap(cfg, syncers, openrtb_ext.BidderSomoaudience, somoaudience.NewSomoaudienceSyncer) insertIntoMap(cfg, syncers, openrtb_ext.BidderSovrn, sovrn.NewSovrnSyncer) insertIntoMap(cfg, syncers, openrtb_ext.BidderSonobi, sonobi.NewSonobiSyncer) insertIntoMap(cfg, syncers, openrtb_ext.BidderYieldmo, yieldmo.NewYieldmoSyncer) insertIntoMap(cfg, syncers, openrtb_ext.BidderGamoshi, gamoshi.NewGamoshiSyncer) + insertIntoMap(cfg, syncers, openrtb_ext.BidderMgid, mgid.NewMgidSyncer) return syncers } diff --git a/usersync/usersyncers/syncer_test.go b/usersync/usersyncers/syncer_test.go index bb42b8622ac..11555827316 100644 --- a/usersync/usersyncers/syncer_test.go +++ b/usersync/usersyncers/syncer_test.go @@ -29,11 +29,13 @@ func TestNewSyncerMap(t *testing.T) { string(openrtb_ext.BidderImprovedigital): syncConfig, string(openrtb_ext.BidderIx): syncConfig, string(openrtb_ext.BidderLifestreet): syncConfig, + string(openrtb_ext.BidderMgid): syncConfig, string(openrtb_ext.BidderOpenx): syncConfig, string(openrtb_ext.BidderPubmatic): syncConfig, string(openrtb_ext.BidderPulsepoint): syncConfig, string(openrtb_ext.BidderRhythmone): syncConfig, string(openrtb_ext.BidderRubicon): syncConfig, + string(openrtb_ext.BidderSharethrough): syncConfig, string(openrtb_ext.BidderSomoaudience): syncConfig, string(openrtb_ext.BidderSovrn): syncConfig, string(openrtb_ext.Bidder33Across): syncConfig,