Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only pull SODA salary data per badge number #48

Closed
mpuckett159 opened this issue May 20, 2021 · 3 comments · Fixed by #62
Closed

Only pull SODA salary data per badge number #48

mpuckett159 opened this issue May 20, 2021 · 3 comments · Fixed by #62

Comments

@mpuckett159
Copy link
Collaborator

With back end updates to support hosting full historical roster information there will be instances where lots of roster entries can be returned by queries, including scenarios where many entries will be fore the same badge number. We should implement a change to the SODA function to ensure that we only look up the data once per badge number rather than searching for each entry to reduce the time spent waiting for responses from the API.

Additionally, having historical information, we will encounter instances where we can't get historical pay information and should either denote as such that an officer isn't a current employee of the city and so we don't have data on their salary because of that, and note for any current officers that the displayed salary is only for their current salary and not applicable to any past positions.

@AetherUnbound
Copy link
Collaborator

AetherUnbound commented Jun 12, 2021

Updating the note on this: the augment with salary call should probably only be done if the record is current. Historical records shouldn't have this call. As part of #49 we added caching so the first portion of this isn't as necessary.

@sarayourfriend
Copy link
Contributor

@AetherUnbound I'd love to tackle this issue next but I'm not exactly sure how to start. Do you have any extra pointers/context around it for me?

@AetherUnbound
Copy link
Collaborator

AetherUnbound commented Jun 15, 2021

Sure thing! I think in extras.py, we might want to add an additional check here to ensure that the record is current:
https://github.com/SeattleDSA/1-312-hows-my-driving/blob/main/src/extras.py#L27

The way these extras are set up is dataset specific, so we can be confident that the "is current roster" field (I forget what it's actual name is) is in the roster record at this point. We only want to do the salary augment for officers that are on the current roster, because otherwise we can't be confident that the salary accurately reflects what the officer makes (or if they'll even be in the public city salary wage data).

I think the caching we're doing for this function mitigated the "one call per badge number" that this ticket was originally for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants