Trying to match AI size and age pop'n estimates prior to 2004

[pete-hulson]

Data product requested (stratum CPUE, etc.):
Filtered length frequency, CPUE, and specimen data for AI bottom trawl surveys from 2004 and prior

Species:
All

Region (GOA, AI, Bering Sea):
AI

Research team making the request (PI /requester name, and tag team members with GitHub accounts, or email address if requestor does not have a GitHub account. Division is nice too.):
Pete Hulson (@PeteHulson-NOAA, [email protected]) as part of a working group investigating the magnitude of sexed length samples. Others in the working group include Ben Williams (@ben-williams, [email protected]), Jason Conner ([email protected]), and Meaghan Bryant ([email protected])

[Ned-Laman-NOAA]

This seems like a request that @NancyRoberson could fill. If that's not the case, let me know.

[NancyRoberson]

@pete-hulson 2months later, do you still need these? I apologize for the delay and hopefully you already have them. Thanks, -Nancy

[NancyRoberson]

Hi All,
I am cleaning up past requests. Did this issue get resolved?
Thanks!

[pete-hulson]

Hey Nancy, No, it did not. What is requested here is to understand which data was used prior to 2006 - using the script developed from the original SQL code Wayne used we can match up the age and length population estimates from 2006 forward, but using this same script the estimates from 2004 back are different. This leads to the conclusion that it's not the functions used to estimate age and length population estimates but something in the filtering of data that changed in 2004. Pete

…

On Mon, Jul 11, 2022 at 8:19 AM Nancy Roberson ***@***.***> wrote: Hi All, I am cleaning up past requests. Did this issue get resolved? Thanks! — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKSBX7IWPM4ACOVZRQWDGTVTRCPDANCNFSM5OV525DA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Ned-Laman-NOAA]

Pete and Nancy,
It isn't clear to me what your comparing. Do you have a static copy of pre-2006 data that your comparing with? Or are you saying the pre-2006 data in tables we hold in Oracle are not reproducible with Wayne's SQL? Can you share a copy of Wayne's SQL for us to evaluate? I think answering these questions might provide a good starting point to sleuth out what's going on here.

[pete-hulson]

Hey Ned and Nancy, Apologies for the delayed response here, between summer field work and fall assessment season things get busy, I'm sure you can relate. I've got a couple of things attached for you. First, you had asked for Wayne's SQL, that's attached as 'SQL sizecomp.R'. A bit of development history here, over a number of years this code has been implemented into R based on this original SQL script - I originally transferred it over, then Matt Siskey worked on it, then Ben Williams helped to really polish it up. We now have a R project on GitHub that is working with this code (https://github.com/BenWilliams-NOAA/swo). I think you were involved somewhat in being aware of the work we did to investigate the number of sex length samples that are needed, or, more appropriately, how many we can reduce so that ergonomic injuries don't happen as frequently and we still get good info for assessments (had a meeting on that this spring). In that work we used the most recent 3 years worth of surveys in the EBS, AI, and GOA, and everything matches up well between what we're doing and what you guys produce in the SIZECOMP_TOTAL and AGECOMP_TOTAL tables (at least for the AI and GOA). That's where we landed with this data request. When I went back to try and match up what we're doing and what's in those tables for all years of the surveys I was able to match up everything almost perfectly with the GOA, and back to 2006 in the AI. To try and illustrate this I've attached a spreadsheet RACE comparison for you to look at ('RACE comparison.xlsx'). What these numbers are in the tables in the sum of the relative differences between what we're doing and the size/age tables you guys produce (SIZECOMP_TOTAL and AGECOMP_TOTAL tables). You'll notice in the GOA table that these numbers are really small, meaning that the length and age population estimates are matching (and I used pollock and POP as examples). But, when you flip over to the AI tab you'll see everything matches back thru 2006, then again in 2000, and again in 1980 for the size pop'n estimates, and slightly different for the age estimates. Trying to pull all that together, it doesn't seem like the differences are in the 'math' that we're doing since we can match all years in the GOA and at least half the years in the AI. What it looks like is that there are different filters in the data that may have been applied in the AI that are different than the current filters used in the GOA and more recent AI data. That was what the essence of this data request was about, to try and figure out what those different filters were. I've also attached the script I used in the SWO package to compare between our results and the SIZECOMP_TOTAL and AGECOMP_TOTAL tables ('match_RACE.R') in case you wanted to dig a bit more into what we've done. I'm also happy to meet whenever to chat about this, I know a 'face-to-face' conversation can go a long way to clarify things in comparison to an email. Thanks in advance for all the help, Pete

…

On Mon, Jul 11, 2022 at 3:02 PM Ned Laman ***@***.***> wrote: Pete and Nancy, It isn't clear to me what your comparing. Do you have a static copy of pre-2006 data that your comparing with? Or are you saying the pre-2006 data in tables we hold in Oracle are not reproducible with Wayne's SQL? Can you share a copy of Wayne's SQL for us to evaluate? I think answering these questions might provide a good starting point to sleuth out what's going on here. — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKSBXY4OF7INJJWUFCTEC3VTSRXPANCNFSM5OV525DA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[NancyRoberson]

Hi Ned and Pete,
Did this issue get resolved? If so, I will close it.

[Ned-Laman-NOAA]

Thanks for checking in, Nancy.
This remains on my list to get to, but I haven't made it there yet.
Let's leave it open and I'll try to get to it by the end of this calendar year.

[zoyafuso-NOAA]

Since this is concurrent with issues also seen in our index computation working group, I would like to be included in this thread. Pete, could you send me those attachments you referenced in your Sept 2 post?

[pete-hulson]

You bet, attachments attached (let me know you get them)

…

On Fri, Dec 30, 2022 at 10:01 PM zoyafuso-NOAA ***@***.***> wrote: Since this is concurrent with issues also seen in our index computation working group, I would like to be included in this thread. Pete, could you send me those attachments you referenced in your Sept 2 post? — Reply to this email directly, view it on GitHub <#8 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKSBX5AFJ3MEC4LB2X6JJ3WP7K6BANCNFSM5OV525DA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[zoyafuso-NOAA]

Hi Pete, I don't see the attachments, can you send it to me via email ([email protected])?

[Ned-Laman-NOAA]

Pete,
Can you cc me with those attachments when you send them to Zack?
I'd like to see them and haven't yet.
Ned