Releases · vanderbilt-redcap/flight-tracker

10 Jan 16:17

scottjpearson

6.5.2

bebb5a1

Release 6.5.2

(Bug fix) I accidentally required that someone update the Data Dictionary for publication wrangling to work. This modification makes it optional. It has been tested successfully.
Sanitization of LDAP outputs.

Assets 2

10 Jan 00:57

scottjpearson

6.5.1

1c95a2a

Release 6.5.1

Bug fix: The cron's upgrade script was not functioning correctly when someone was upgrading while the crons were running. It is now fixed.
An unnecessary file was removed because it was causing a scanning tool to fail.

Assets 2

09 Jan 17:34

scottjpearson

6.5.0

802a53b

Release 6.5.0

(NEW FEATURE) An "ORCID Wrangler" has been added for streamlined handling of ORCID identifiers (www.orcid.org), used as a unique identifier for matching publications.
(NEW FEATURE) Program Roles (an optional checkbox) has been added per a Vanderbilt project's request. This field has been added to the Data Dictionary and Project Roles has been nuanced as a "dropdown."
(NEW FEATURE) Customizable Pilot Grants can be added to the Configuration. Appropriate fields have been added to the Data Dictionary, too.
(MAJOR) large sections of code have been moved into the MMAHelper class (MMA = Mentee-Mentor Agreements). This is not brand new code, but minor enhancements for clean code might have been made. Also, Psalm scanner improvements have been implemented here.
(MAJOR) The Mentee-Mentor Agreements have received a major upgrade. When first launched, I never cleaned up Chad's CSS and JS into more efficient organization. Here, I placed them into central structures and cleaned up their code.
(NEW FEATURE) The Scholar Portal, used by scholars not admins, now has a way for the scholars to validate their own publications.
(NEW FEATURE) Pilot Grants can now be associated with publications in the Publication Wrangler and in its equivalent in the Scholar Portal.
(NEW FEATURE) Tooltips have been added to headers in the NIH Training Tables.
(NEW FEATURE) Support for adding and removing ORCIDs has been added to the Scholar Portal.
As a part of the ORCID Wrangler changes, I centralized functionality to an ORCID class (in another submodule). To achieve this, I refactored the cron-script that runs weekly. In this script, I also disabled an email that I doubt is used much.
The Lexical Translator has been renamed to the "Grant Lexical Translator" for clarity.
Links to "hand picked" cohorts have been added to the list of existing cohorts.
Textual changes have been made to support Program Roles (checkboxes) on the application's configuration page. The code already supported checkbox options, but textual nuance was needed.
Because they are quick, NIH RePORTER runs will now be batched in batches up to 100 records instead of 30.
Some minor CSS modifications have been made.
When installing an optional field, only a Person Role textbox is shown when a prior item has been filled in. This avoids having 6 large, blank textboxes appear on the installation.
REDCap was having trouble displaying the special character of an accented e in resume. I have removed it in the Data Dictionary to display without the accent.
CSS on the Publication Wrangler has been cleaned up, and a few small, new classes have been added.
The grants for the initial survey and followup survey now use all PI grants (PI or Co-PI) instead of just career-defining awards.
Survey hooks have been optimized to download a smaller Data Dictionary.
(BUG FIX) Publications that were added were not displaying group author names; these are now parsed properly. This did not affect saving them in REDCap, only their display in the Publication Wrangler.
Some JS was moved into the Wrangler class. This is both from base.js and from a baseSelect.php file specific to data wranglers.
Some JS was modernized.
A new JS method checkForNextField was added to support configuring Associative Arrays.
(BUG FIX) When spoofing in the Scholar Portal, the scholar's name, instead of just their userid, is now displayed.
(BUG FIX) In the list of fields, word clouds do not display fields that aren't relevant.
Many menu items were removed from the left-hand bar in data wranglers.
The saving features for the Publication and Patent Wranglers have been updated. Archaic features like passing JSONs have been updated.
Code to copy the entire table 1 display into a word processor (at least, those that receive HTML, like MS Word) has been added.
ORCID functionality has been centralized from the driver into one class to support a new ORCID Wrangler. More on this in other commits. This code has been used for years, but is simply refactored here.
The NIH RePORTER's API has been upgraded from v1 to v2.
The optional checkbox variable Program Roles has been added, per request from a Vanderbilt project. This value feeds into making a Cohort. Of note, because they are checkboxes Program Roles do not have an "Other" option, unlike the dropdown Person Roles.
Cohorts now support checkboxes to support Program Roles.
An older function to lookup an email address for a REDCap user has been refactored to other classes.
A username and password for the proxy server is no longer required for cURL calls when using a proxy. This only impacts REDCap servers that use proxies. Thus, password-less proxies are now supported.
A $pid is now stored with each CitationCollection object. In turn, this is passed onto the Citation class. Other methods have been modified to represent this change.
A toggle for showing a link to REDCap for each citation is provided so that the Scholar Portal won't display inaccessible links to REDCap.
Refactoring to get the userid field either by $pid or with a $token. One project still uses the identifier_vunet notation.
Refactoring to get the full name of a user by $pid, not just by $token. This avoids a quick database call to convert $token's to $pid's.
The Scholar Portal's code for spoofing has more definition. Some methods have been refactored.
The title for the Scholar Portal page for Your Photo has been changed to Edit Your Photo.
Code for the Scholar Portal to authenticate users based on $pid and $recordId has been adapted.
The headers and buttons for the Publication Wrangler can now be customized and be changed for the Scholar Portal.
Pagination has also been added to the document, showing one section at a time.
I added a way for the user to turn on/off certain sections per their request. These changes are saved in the $_SESSION variable.
I also optimized code so that larger data structures (like the Data Dictionary) are only downloaded in part and are also stored in the $_SESSION variable. Generally, data downloads have been slightly streamlined.
I allowed users who have access to the REDCap project to "spoof" users to encourage wider adoption. This is what the MMA_DEBUG variable is now used for.
The code has been SIGNIFICANTLY restructured, with some modifications. The core is in tact. I have tested thoroughly.

Assets 2

26 Dec 17:01

scottjpearson

6.3.2

7582dc0

Release 6.3.2

Last week, I removed the Add Mentors for Existing Scholars page. I was asked to add it back, so I did.
On the Celebrations Email, I adjusted the "All New Publications" email to include middle-author publications. Perviously, it was only reporting first/last author publications. Then I added a separate dropdown to handle First/Last Author papers in a separate configuration.
On the Celebrations Email, do not require publications to be first or last author. Instead, include all unless the configuration states otherwise.
(Bug fix) On the Celebrations Email, change the screened date to the longer threshold date instead of a later warning date. Flight Tracker wasn't picking up the appropriate publications.
(Bug fix) The Celebrations Email was referencing an array incorrectly and not sending out publications. That has been fixed here.
On the Celebrations Email, I added support to test it out with the threshold on one date instead of the typical "weekly" or "monthly" designation. This won't be adjustable by users (yet).
On the Celebrations Email, only future grants were being considered, regardless of whatever threshold was specified by the user. This moves the grant threshold back 2 weeks or 2 months, depending on the user's request.
The Celebrations Email has a new setting to request first/last author only.
For DB optimization, the download class only downloads one name at a time instead of the whole cadre of names. I don't think this will affect performance much either way.
(Bug fix) Screen the cached metadata (in a SESSION variable) for a given set of fields if specific fields are requested.
Provide more debugging material when the API is used (which is rarely) instead of the REDCap class.
A function to download a single user's user-id.
Remove some unnecessary code in Download::fieldsForRecordsByPid. This function uses the REDCap class exclusively, and the unnecessary code was written for the API. It should have been cut at the outset.
(MSTP) Add in the mentee comments to the final signed document, per request.

Assets 2

12 Dec 00:32

scottjpearson

6.3.1

38b99f1

Release 6.3.1

(New feature) Configuration for multiple person roles has been enabled.
(Major) I switched the publication fetch to happen on Tuesdays-Wednesdays and Vanderbilt summaries to run on Saturdays-Sundays to reduce server load.
(Major bug fix) Setup repeating forms when copying a project. Also, redcap_repeat_instrument and redcap_repeat_instance are now relayed in the copy. The lack of these was a fatal bug for this feature.
(Enhancement) The page showing what trainees have become mentors in the same project has been beautified.
I adjusted the number of preprocessing projects to run at a time from 10 to 7 so that the portal user-ids can be readily accessible.
Shortened the mentoring menu with simplified configuration.
Adjusted a function name (downloadPortalPersonalData) to be simpler in a few places.
The grant table now has a JSON endpoint available with NOAUTH. This is by request so that outside applications can access the data. This might evolve into a separate API at some point, but that's not available yet. The JSON endpoint requires a token for security. The page has been allowed for NOAUTH, but then NOAUTH for its non-JSON features have been disabled.
(Bug fix) De-duplicate cleanup of resources was put into the wrong function in last week's release. That has been corrected here. I manually cleaned up Vanderbilt's last week, and most consortium users do not use this. Therefore, it's more of an insurance measure than a pressing need.
CSS was added for the mentoring agreement pages.
(Bug fix) Name matching in JS for adding resources was a bit too aggressive in shorter names. It was matching names like Ng and initials like A as a part of larger names. The adjusted algorithm requires that sub-parts of name be at least 5 characters long to work. It also handles initials better by removing any trailing periods.
JavaScript for looking up a REDCap user-id now can be configured to custom field ids.
Per user request, looking for new institutions in PubMed now displays an email address for a scholar if available.
The user-id field now explicitly ties the user-id to a REDCap user-id.
Person Roles are displayed on the Scholar Profile page.
Publications with a mentor are now downloadable in CSV format.
(Enhancement) Per project request, Person Roles now support up to 6 different person roles. This affects Cohorts/Filtering, too. To enact this, I broadened the Optional Roles to support a varying number.
(Bug fix) The queue of completed/run jobs was only populating the first entry because it failed to find a 0-th entry. However, the indexes began at 1. So it overwrote the 1st entry every time. This caused emails for completed jobs not to go out - not a huge deal, but a bug nonetheless.
Download::names alphabetizes names in a case-insensitive manner now.
(MSTP) For mentee-mentor agreements, only show users who have a userid in the list to begin a process.
(MSTP) Better error messaging.
(Enhancement) I simplified the configuration page with JavaScript, per a process suggested by Chris Williams of the Vanderbilt MSTP project. It handles collecting user-ids with much more hand-holding than the previous process.
(Enhancement) The agreement dashboard is also simpler, having introductory text moved to the configuration page.
I added a hidden page of the references we consulted about mentoring, which I occasionally email out.

Assets 2

30 Nov 23:02

scottjpearson

6.3.0

b5fbac0

Release 6.3.0

(New feature) Two new graphs were made to show how RCR values have changed over time in the score-distribution page. In addition, the timestamp field is used instead of date/month/year fields.
(New feature) I added a routine to report missing email addresses that are in other projects via email. This has been successfully running on the Vanderbilt server for some time, so I just made it available to our partners.
(Optimization) I rewrote a routine to find matched names into two separate jobs. This new routine will be significantly reduce the preprocessing on Fridays. The old algorithm took 28 hours for 7,000 scholars and increased as n^2, where n = the number of records. The new algorithm scales based on the number of **new records. This should reduce processing time, but I will monitor in future weeks.
(Bug fix) I plugged a data leak in repeating forms.
(Bug fix) I added a routine to deduplicate resources due to an improper data-sharing spill. This will be executed one time only. The routine has already been tested, vetted, and used on our production server.
(Improvement) I divvied up some processing to be handled in batches to avoid "all or nothing" batch jobs.
The External Module stores the CronManager centrally so that it can be accessed internally when launching future cron jobs. The matching routine launches future cron jobs after it completes to make sure that good values are in place.
Added a function to turn an ExtMod URL into base-64. This is because a partner's REDCap server was requiring a login for these files to be displayed in hooks. This function allows them to be displayed in base-64 formatting instead - thus avoiding the login requirements.
A first parameter of "0" is now displayed on the batch-queue monitor.
K99/R00s are adapted into "Bridge Award" designations. This affects code and JSONs for metadata.
(Bug fix) The publications widget in survey hooks were displaying incorrect counts in a few situations. This has now been corrected.
(MSTP) Revisions in text were made per request.
The global map can toggle its numerical labels.
Add support for batching a job that runs across multiple project-ids. This puts that job (a "multi-cron") into segmented chunks of project-ids so that it won't run for too long.
By default, the first parameter in a cron job is FALSE, unless specified otherwise. Previously, I had accepted all false-y values. This week, I had a case to use a first parameter of 0, so I made stricter comparisons.
Bug fix: I changed $batchQueue[0] to a $firstBatchQueue variable. I forgot to change this in one circumstance. I did a global grep and found no other occurrences.
The Psalm scanner found some unreachable code that was now removed.
I adapted a function getChoicesForField() into getChoicesForFields(). I then refactored getChoicesForField() to use the new function.
I added a method to scan to see if a list of metadata fields is filled. It checks whether there are 10 values and whether the first field is record_id. If one of these is FALSE, then an algorithm will skip over that project. I already have this methodology in place for metadata. This just adapts it to a field-list.
The Psalm scanner found that Download::getCachedMetadata() referred to a $fields variable that I did not pass to it. So I passed it now.
I adjusted some error messaging in the Download class, per the Psalm scanner.
To write a PDF, I required that DomPDF produced an actual value. This avoids writing a null file.
Per user request, the global map can now have its numerical labels turned off.
The Psalm scanner requested that min() and max() not be called with an empty array. Since the values are hard-coded, this is a non-issue logically. But I changed the code to make the scanner happy anyways.
I added support for a new pathway called "Bridge Award." These took over the K99/R00 pathway and included a new F99/K00 pathway from the NIH.
(MSTP) Per requested, I increased the amount of documentation for the user.
(MSTP) The Psalm scanner found some good code refinements to enhance quality, like referring to the correct variables.
In splitting a name, I had already removed parentheses. I also removed sets of single- and double-quotes.
(Bug fix) I looked up the $token and $server in Publications::xml2REDCap().
I refactored the makeHash() method to make the Psalm scanner happy.
I wrote a workaround to change HTML quotes back into real quotes. I also handled curly quotes while doing so. This is because ExtMods requires that ENT_QUOTES be displayed. This is almost always the right thing to do on our team because our data are displayed on the browser. However, Flight Tracker sometimes passes data to APIs, and quotes are needed here. The APIs don't accept them in escaped HTML.
The Vanderbilt MSTP program requested to calculate the end-of-training and start-of-training based on its prior data.
I adjusted how logging is handled with upload errors by supplying the $pid.
A user had an incomplete installation of the NIH Table 1 project. A project-id was generated, but a survey link was not. However, Flight Tracker was not allowing her to set up the project again because a project-id was available. I changed it so that a project-id and a public survey link are available.
A Psalm scan revealed that I had an extra parameter when getting a system setting. ExtMod system settings do not use project-ids.