-
Notifications
You must be signed in to change notification settings - Fork 199
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Reduce particle shape when a particle approaches the EB #5209
Reduce particle shape when a particle approaches the EB #5209
Conversation
…lgorithms with PML BC
This reverts commit 0051a17.
….com:oshapoval/WarpX into add_analysis_embedded_boundary_removal_depth
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great, thank you for this! I added inline comments for small details.
I was wondering if it might help to split a the doEsirkepovDepositionShapeN
function up into inlined sub-functions, in case it gets too crowded.
} | ||
|
||
} | ||
|
||
ComputeDistanceToEB(); | ||
MarkReducedShapeCells( m_eb_reduce_particle_shape[lev], eb_fact, WarpX::nox ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you don't need to pass m_eb_reduce_particle_shape
(or even all) arguments here and could call it like ComputeDistanceToEB()
. (Because both are member functions of the same class.)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sounds good.
Part of the motivation here is to maybe do a follow-up PR where some of the Mark...Cells
functions become free functions.
|
||
// Every cell in box is regular: do not reduce particle shape in any cell | ||
amrex::ParallelFor(box, [=] AMREX_GPU_DEVICE (int i, int j, int k) { | ||
eb_reduce_particle_shape_arr(i, j, k) = 0; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Default from setVal
, but ok to be explicit.
@@ -656,7 +659,10 @@ void doEsirkepovDepositionShapeN (const GetParticlePosition<PIdx>& GetPosition, | |||
const amrex::XDim3 & xyzmin, | |||
amrex::Dim3 lo, | |||
amrex::Real q, | |||
[[maybe_unused]]int n_rz_azimuthal_modes) | |||
[[maybe_unused]]int n_rz_azimuthal_modes, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not important formatting of existing code:
[[maybe_unused]]int n_rz_azimuthal_modes, | |
[[maybe_unused]] int n_rz_azimuthal_modes, |
// If particle is close to the embedded boundary, recompute deposition with order 1 shape | ||
if constexpr (reduce_shape_control == has_reduced_shape) { | ||
if (reduce_shape_new) { | ||
for (int i=0; i<depos_order+3; i++) sx_new[i] = 0.; // Erase previous deposition |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I took a look on godbolt and the compiler seems to unroll this constant-size loop, so no need to write it more obscurely.
for (int i=0; i<depos_order+3; i++) sx_old[i] = 0.; // Erase previous deposition | ||
compute_shifted_shape_factor_order1( sx_old+depos_order/2, x_old, i_new+depos_order/2 ); // Redeposit with order 1 | ||
} | ||
// Note: depos_order/2 in the above code corresponds to the shift between the index of the lowest point |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the comment, very helpful to parse this.
If it helps to name this, you could also write this in an const int
before the function call.
The performance of CurrentDeposition was tested by running a uniform plasma test (input attached below) case three times, on Perlmutter. Results: feature branch does not change performance
EB was set via implicit function: |
amrex::Array4<int> const & eb_reduce_particle_shape_arr = eb_reduce_particle_shape->array(mfi); | ||
|
||
// Check if the box (including one layer of guard cells) contains a mix of covered and regular cells | ||
const amrex::Box& eb_info_box = mfi.tilebox(amrex::IntVect::TheCellVector()).grow(1); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Little bug: fix in #5635
Follow-up to #5209: My compiler says those locations would reference temporary objects that were destroyed at the end of the line. That seems to be the case indeed. Copy instead to make the temporary a named and thus persistent variable. ![Screenshot from 2025-02-03 16-59-22](https://github.com/user-attachments/assets/8259f6d7-099b-4d09-8382-f24baefb5793)
Description edited by @RemiLehe
Overview
This PR reduces the particle shape to order 1, when the particle gets closer to the embedded boundary:
![Screenshot 2025-01-23 at 8 46 34 AM](https://private-user-images.githubusercontent.com/6685781/406125614-2e206606-110e-4018-aedc-385567fe43e7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3Mzg5MDcyMDksIm5iZiI6MTczODkwNjkwOSwicGF0aCI6Ii82Njg1NzgxLzQwNjEyNTYxNC0yZTIwNjYwNi0xMTBlLTQwMTgtYWVkYy0zODU1NjdmZTQzZTcucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI1MDIwNyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNTAyMDdUMDU0MTQ5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZjNiZjkyZmE4ZDA2NTRlMTQyODMxNmE1YzJkMzg2OWY1YjZhNTJlMGNlZjdkNTIzZjZkNDcwMjlmMDc5OWUzNCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QifQ.fl_M2KlKLKIcdboEpLZNLLKDPMdLpDAixPLV1L-Y2TI)
This ensures that the particle does not deposit charge in valid cells, at the time when it is removed, which in turn ensures proper charge conservation with the electromagnetic solver.
Implementation
eb_reduce_particle_shape
(andiMultiFab
) that indicates in which cells to reduce the particle shape.Tests
This PR adds tests similar to the ones introduced in #5562 to check for charge conservation near the embedded boundary, but with higher-order shape factors:
development
for shape 2 and 3 but pass on this PR.development
for shape 3 ; they do pass for this PR. It is not clear why the tests do not fail ondevelopment
with shape 2.Note: For now, this PR only modifies the current deposition (and only the Esirkepov kernel). A follow-up PR will also modify the charge deposition.