Changes to support using the newest API for fetching survey data, addressing #130

[jmobrien]

• fetch_survey now points to the modern API endpoint
• swapped/added some of the new parameters, not everything yet:
  https://api.qualtrics.com/reference#create-response-export-new
• updated some assertions & associated utilities to support argument checking, etc.
• url building redesigned to work better with new API, may be slightly more easily extensible going forward

Generally, fetch_survey seems to work, and a few internal changes were made so metadata wasn't broken by this. But remains to be seen if there's deeper stuff that needs addressing/cleaning up.

[jmobrien]

Updated 2019/02/13 to add a fix to all_surveys() that needed to work with the new URL generation scheme. (633ca33)

[juliasilge]

A question for you @jmobrien: did the option in the API for unanswer_recode really change to something integer-like? Not a string anymore? I am looking in the documentation trying to find this but not coming upon it easily.

[juliasilge]

Wait, I think I found it.

[juliasilge]

OK! I think this does it. 🎉 Thank you so much for your work on this.

I had to remove one test because the old testing framework does not play nicely with the new API calls. This makes #120 a higher priority.

[chrisumphlett]

Thanks to @jmobrien -- I'll give it some hardcore testing very soon, perhaps even tomorrow if I have time.

[jmobrien]

Yes, it's specified as int32 now. Can't recall if I still tried it as a string, but I did see the API fail with a non-integer number, so I assume it's not at all flexible. Joseph O'Brien, Ph.D. Adolescent Development Research Group Department of Psychology University of Texas at Austin [email protected]

…

On Thu, Feb 13, 2020 at 7:03 PM Julia Silge ***@***.***> wrote: Wait, I think I found it <https://api.qualtrics.com/reference#create-response-export-new>. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#135?email_source=notifications&email_token=ABWEMIVCNVJOGYXIMTSBZOTRCXUVFA5CNFSM4KUFTXF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELXF7GI#issuecomment-586047385>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWEMIXRYF2HDIX5LIY5L4DRCXUVFANCNFSM4KUFTXFQ> .

[jmobrien]

Glad to help. I've used it for ~400 data downloads so far, albeit with consistent parameters other than survey IDs. So far everything looks good, but I'll check in if anything looks off. Joseph O'Brien, Ph.D. Adolescent Development Research Group Department of Psychology University of Texas at Austin [email protected]

…

On Thu, Feb 13, 2020 at 8:01 PM Chris Umphlett ***@***.***> wrote: Thanks to @jmobrien <https://github.com/jmobrien> -- I'll give it some hardcore testing very soon, perhaps even tomorrow if I have time. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#135?email_source=notifications&email_token=ABWEMIQ2JH2OXJEVKMJZVN3RCX3PHA5CNFSM4KUFTXF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELXJMEY#issuecomment-586061331>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWEMIVYDTVEMHDBFO5EAATRCX3PHANCNFSM4KUFTXFQ> .

[chrisumphlett]

I think something changed... I"m not quite sure what. This is my process:

• fetch_survey, put in df called responses
• extract the column names and labels with: col_name_labels <- stack(sjlabelled::get_label(responses))
• this creates a df with columns values and ind.

Previously, I then had logic that filtered on values expecting to see the column name there. Now, I'm having to switch that filter to reference the ind column.

Am I making sense, and does that change in behavior make sense according to the change? Not saying it's a problem!

Context:
The goal of this process is to dynamically build a response_metadata table where I log all of the response metadata (start date, finished, progress now that it's there (yay!), browser) into a long table, for all of the surveys.

Another difference
Previously, there were 7 "Browser Meta Info" columns coming through. Now there are only 4. Flash version, java support, and user agent are dropped.
It's possible I've unknowingly done something here too.

[juliasilge]

Thanks for checking things out @chrisumphlett!

For the browser info, it looks like that is an actual difference between the old endpoint and the new endpoint, i.e. what Qualtrics is sending over when we request data.

For the labels, I think there may be some digging still to do, to figure out what the best end state for label handling is. If you would like, can you chime in on #134 about issues with the labels/attributes?

[chrisumphlett]

Re: Browsers. Makes sense to me-- I have no use for that information anyway, just need to update code.

I will comment on #134 momentarily.