Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add xtalk error calculation #56

Merged
merged 4 commits into from
Nov 6, 2024
Merged

Add xtalk error calculation #56

merged 4 commits into from
Nov 6, 2024

Conversation

mwilensky768
Copy link

This adds the cross talk filter to the noise covariance calculation (and corresponding correction factors). It turned out to involve a sizable matrix multiply that made the calculation take 15 minutes without any adjustments.

I tried different speedup schemes to ameliorate this including generating one filtration operator per spectral window, only using one of the streams, and decimating in frequency. I found that decimating in frequency by a factor of 10 improves the speed by an order of magnitude (just under two minutes to complete) with an error compared to the full calculation of one part in 10^4 at worst on the couple baselines that I checked.

This option is adjusted with the CORR_MATRIX_FREQ_DECIMATION global, defaulted to 10.

Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@mwilensky768 mwilensky768 requested a review from jsdillon November 5, 2024 22:39
@@ -7,7 +7,7 @@
"source": [
Copy link
Member

@jsdillon jsdillon Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line #12.        if band_slice is None:

It is weird to me that you can have an inconsistent band_ind and band_slice. I'd suggest picking one or the other to specify.


Reply via ReviewNB

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just fixed this.

@@ -7,7 +7,7 @@
"source": [
Copy link
Member

@jsdillon jsdillon Nov 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line #24.                    xtalk_frop = get_frop_wrapper(pol=pol, stream_ind=stream_ind, band_ind=spw,

For baselines and bands with little to no overlap in the xtalk notch and the main beam, do we want to just skip all this?


Reply via ReviewNB

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, do we want to automate the check with some criterion or make a global switch?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I added a global setting called CORR_MATRIX_NOTCH_CUTOFF defaulted to 30m which means by default it will skip the notch filter in the error calculation in the EW projection is longer than 30m. This skips the notch filter part by a little less than 75% of the single-baseline runs I believe.

Copy link
Member

@jsdillon jsdillon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks reasonable. One requested change and one question/suggestion which might improve runtime.

@jsdillon jsdillon self-requested a review November 6, 2024 20:46
Copy link
Member

@jsdillon jsdillon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me. thanks @mwilensky768 !

@jsdillon jsdillon merged commit 3b60745 into master Nov 6, 2024
0 of 4 checks passed
@jsdillon jsdillon deleted the xtalk_error_cov branch November 6, 2024 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants