InMemory: Consider implementing LeftJoin internally #21677
Assignees
Labels
area-in-memory
area-query
closed-fixed
The issue has been fixed and is/will be included in the release indicated by the issue milestone.
customer-reported
type-enhancement
Milestone
Comments
|
@dotnet/efteam - Pulling this in 6.0. This will vastly improve maintainability of in-memory (to make it production ready @maumar - Perhaps wait for this to be fixed before you look into in-memory bugs. |
)
smitpatel
added
the
closed-fixed
smitpatel
added a commit
that referenced
this issue
Mar 10, 2021
Implement left join as client method to reduce complexity To resolve the indexing issue stemming from #23934 Now all the projections are applied immediately to reshape the value buffer so that all our future bindings are always read value expressions which refers to a proper index. In order to do this, we apply select for entity type with hierarchy to avoid entity check conditional expression. For adding projection (through ReplaceProjectionMapping), - For client projection, we apply a select and re-generate client projection as read values - For projection mapping, we iterate the mappings, we apply a select and re-generate the mapping as read values - If projection mapping is empty then we add a dummy 1 so that it becomes non-range-variable When applying projection, we generate a selector lambda to form a value buffer and replaced all the expressions to read from new value buffer. Overall this solves the issue of having complex expressions to map or pull. This also removed PushDownIntoSubquery method. In order to avoid the issue of indexes changing when generating join due to iterating projection mappings, we now also have projectionMappingExpressions which remembers the all expressions inside projectionMapping (which are all read value as we generated before). So now we don't need to iterate the mapping and we use the existing expressions directly. This keeps existing indexes. Resolves #13561 Resolves #17539 Resolves #18194 Resolves #18435 Resolves #19344 Resolves #19469 Resolves #19667 Resolves #19742 Resolves #19967 Resolves #20359 Resolves #21677 Resolves #23360 Resolves #17537 Resolves #18394 Resolves #23934
smitpatel
added a commit
that referenced
this issue
Mar 10, 2021
Implement left join as client method to reduce complexity To resolve the indexing issue stemming from #23934 Now all the projections are applied immediately to reshape the value buffer so that all our future bindings are always read value expressions which refers to a proper index. In order to do this, we apply select for entity type with hierarchy to avoid entity check conditional expression. For adding projection (through ReplaceProjectionMapping), - For client projection, we apply a select and re-generate client projection as read values - For projection mapping, we iterate the mappings, we apply a select and re-generate the mapping as read values - If projection mapping is empty then we add a dummy 1 so that it becomes non-range-variable When applying projection, we generate a selector lambda to form a value buffer and replace all the expressions to read from new value buffer. Overall this solves the issue of having complex expressions to map or pull. This also removed PushDownIntoSubquery method. In order to avoid the issue of indexes changing when generating join due to iterating projection mappings, we now also have projectionMappingExpressions which remembers the all expressions inside projectionMapping (which are all read value as we generated before). So now we don't need to iterate the mapping and we use the existing expressions directly. This keeps existing indexes. Resolves #13561 Resolves #17539 Resolves #18194 Resolves #18435 Resolves #19344 Resolves #19469 Resolves #19667 Resolves #19742 Resolves #19967 Resolves #20359 Resolves #21677 Resolves #23360 Resolves #17537 Resolves #18394 Resolves #23934
smitpatel
added a commit
that referenced
this issue
Mar 10, 2021
Implement left join as client method to reduce complexity To resolve the indexing issue stemming from #23934 Now all the projections are applied immediately to reshape the value buffer so that all our future bindings are always read value expressions which refers to a proper index. In order to do this, we apply select for entity type with hierarchy to avoid entity check conditional expression. For adding projection (through ReplaceProjectionMapping), - For client projection, we apply a select and re-generate client projection as read values - For projection mapping, we iterate the mappings, we apply a select and re-generate the mapping as read values - If projection mapping is empty then we add a dummy 1 so that it becomes non-range-variable When applying projection, we generate a selector lambda to form a value buffer and replace all the expressions to read from new value buffer. Overall this solves the issue of having complex expressions to map or pull. This also removed PushDownIntoSubquery method. In order to avoid the issue of indexes changing when generating join due to iterating projection mappings, we now also have projectionMappingExpressions which remembers the all expressions inside projectionMapping (which are all read value as we generated before). So now we don't need to iterate the mapping and we use the existing expressions directly. This keeps existing indexes. Resolves #13561 Resolves #17539 Resolves #18194 Resolves #18435 Resolves #19344 Resolves #19469 Resolves #19667 Resolves #19742 Resolves #19967 Resolves #20359 Resolves #21677 Resolves #23360 Resolves #17537 Resolves #18394 Resolves #23934 Resolves #17620 Resolves #18912
smitpatel
added a commit
that referenced
this issue
Mar 10, 2021
Implement left join as client method to reduce complexity To resolve the indexing issue stemming from #23934 Now all the projections are applied immediately to reshape the value buffer so that all our future bindings are always read value expressions which refers to a proper index. In order to do this, we apply select for entity type with hierarchy to avoid entity check conditional expression. For adding projection (through ReplaceProjectionMapping), - For client projection, we apply a select and re-generate client projection as read values - For projection mapping, we iterate the mappings, we apply a select and re-generate the mapping as read values - If projection mapping is empty then we add a dummy 1 so that it becomes non-range-variable When applying projection, we generate a selector lambda to form a value buffer and replace all the expressions to read from new value buffer. Overall this solves the issue of having complex expressions to map or pull. This also removed PushDownIntoSubquery method. In order to avoid the issue of indexes changing when generating join due to iterating projection mappings, we now also have projectionMappingExpressions which remembers the all expressions inside projectionMapping (which are all read value as we generated before). So now we don't need to iterate the mapping and we use the existing expressions directly. This keeps existing indexes. Resolves #13561 Resolves #17539 Resolves #18194 Resolves #18435 Resolves #19344 Resolves #19469 Resolves #19667 Resolves #19742 Resolves #19967 Resolves #20359 Resolves #21677 Resolves #23360 Resolves #17537 Resolves #18394 Resolves #23934
smitpatel
added a commit
that referenced
this issue
Mar 10, 2021
Implement left join as client method to reduce complexity To resolve the indexing issue stemming from #23934 Now all the projections are applied immediately to reshape the value buffer so that all our future bindings are always read value expressions which refers to a proper index. In order to do this, we apply select for entity type with hierarchy to avoid entity check conditional expression. For adding projection (through ReplaceProjectionMapping), - For client projection, we apply a select and re-generate client projection as read values - For projection mapping, we iterate the mappings, we apply a select and re-generate the mapping as read values - If projection mapping is empty then we add a dummy 1 so that it becomes non-range-variable When applying projection, we generate a selector lambda to form a value buffer and replace all the expressions to read from new value buffer. Overall this solves the issue of having complex expressions to map or pull. This also removed PushDownIntoSubquery method. In order to avoid the issue of indexes changing when generating join due to iterating projection mappings, we now also have projectionMappingExpressions which remembers the all expressions inside projectionMapping (which are all read value as we generated before). So now we don't need to iterate the mapping and we use the existing expressions directly. This keeps existing indexes. Resolves #13561 Resolves #17539 Resolves #18194 Resolves #18435 Resolves #19344 Resolves #19469 Resolves #19667 Resolves #19742 Resolves #19967 Resolves #20359 Resolves #21677 Resolves #23360 Resolves #17537 Resolves #18394 Resolves #23934 # Conflicts: # test/EFCore.InMemory.FunctionalTests/Query/NorthwindGroupByQueryInMemoryTest.cs
ghost
closed this as
ghost
pushed a commit
that referenced
this issue
Mar 10, 2021
Implement left join as client method to reduce complexity To resolve the indexing issue stemming from #23934 Now all the projections are applied immediately to reshape the value buffer so that all our future bindings are always read value expressions which refers to a proper index. In order to do this, we apply select for entity type with hierarchy to avoid entity check conditional expression. For adding projection (through ReplaceProjectionMapping), - For client projection, we apply a select and re-generate client projection as read values - For projection mapping, we iterate the mappings, we apply a select and re-generate the mapping as read values - If projection mapping is empty then we add a dummy 1 so that it becomes non-range-variable When applying projection, we generate a selector lambda to form a value buffer and replace all the expressions to read from new value buffer. Overall this solves the issue of having complex expressions to map or pull. This also removed PushDownIntoSubquery method. In order to avoid the issue of indexes changing when generating join due to iterating projection mappings, we now also have projectionMappingExpressions which remembers the all expressions inside projectionMapping (which are all read value as we generated before). So now we don't need to iterate the mapping and we use the existing expressions directly. This keeps existing indexes. Resolves #13561 Resolves #17539 Resolves #18194 Resolves #18435 Resolves #19344 Resolves #19469 Resolves #19667 Resolves #19742 Resolves #19967 Resolves #20359 Resolves #21677 Resolves #23360 Resolves #17537 Resolves #18394 Resolves #23934 # Conflicts: # test/EFCore.InMemory.FunctionalTests/Query/NorthwindGroupByQueryInMemoryTest.cs
This issue was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Currently we translate it to GroupJoin/DefaultIfEmpty/SelectMany pattern. There are currently few tests skipped when they grow too much in weak entity tests. It may be beneficial from implementation, debugging, maintenance perspective to just implement LeftJoin internally rather than trying to generate a much larger tree.
The text was updated successfully, but these errors were encountered: