Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

InMemory: When adding subquery projection, take into account client projection #17620

Closed
smitpatel opened this issue Sep 4, 2019 · 10 comments · Fixed by #24805
Closed

InMemory: When adding subquery projection, take into account client projection #17620

smitpatel opened this issue Sep 4, 2019 · 10 comments · Fixed by #24805
Assignees
Labels
area-in-memory area-query closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. punted-for-3.1 type-bug
Milestone

Comments

@smitpatel
Copy link
Contributor

See #16963 (comment)

@zlepper
Copy link

zlepper commented Jan 23, 2020

So how exactly are we supposed to work around this? Using an anonymous object doesn't work either, and we can't create tuples in expressions?

We are just now attempting to update to dotnet 3 (from 2.2), and we have a bunch of failing tests because of this..

@tomasbruckner
Copy link

any update on this?

@brenwebber
Copy link

brenwebber commented Mar 9, 2020

I'm getting this exact error, but in a different, quite specific, situation.

Scenario: Fetching a Parent-ChildCollection and projecting to new Parent-ChildAggregate with a nested Parent.OwnedEntityProperty.

Json representation of source Parent entity structure:
{ "Config": { "SubConfig": { } } "Child": [] }

Whenever I try aggregate the child list AND retrieve the nested value object, I get the Unable to cast object of type 'System.Linq.Expressions.NewExpression' to type 'System.Linq.Expressions.MethodCallExpression' error.

I created a simple console app and proved that this is the only scenario it happens in.

  • No child aggregate, just nested config -> passes
  • Just child aggregate, no config -> passes
  • Child Aggregate, with non-nested config -> passes
  • Child list, with nested config -> passes
  • Child aggregate, with nested config -> FAILS

In my real project the last scenario succeeds against SQL Server, I have not tried this sample against SQL.

EFCoreTests.zip

To work around this I am checking if the Db is in memory, then split the query into 2. That way my tests pass, and I still have the performance of a single read to the real Db.

@PoyaManouchehri
Copy link

Really keen to hear if there is an ETA for this. It's a blocker for our migration to 3.1 at this point.

@jgbpercy
Copy link

For anyone coming across this because they're trying to use a .Count alongside a parameterized constructor (like my issue #21001 ), we've worked around this by wrapping the Count in a Convert.ToInt32 call. This fixes the tests that use the InMemory provider. In terms of the SQL generated for the SQL Server provider, this adds a CONVERT. Someone who is more of an expert on SQL perf can correct me if I'm wrong, but I don't think that should adversely affect performance, and for us it's superior to dropping to parameterless ctors.

@PoyaManouchehri don't know if this helps you?

@kvv842
Copy link

kvv842 commented Aug 3, 2020

I have this exception used Select and Count together.

var test = await db.Users.Select(u => new
                {
                    u.Id,
                    Acts = u.Activities.Select(a => new { a.Id }),
                    ActsCount = u.Activities.Count(),
                }).ToListAsync();

@PoyaManouchehri
Copy link

@jgbpercy I can confirm that our original issue (exception when using .Any() in a subquery) is fixed in EF Core InMemory 3.1.6.

@Noppey
Copy link

Noppey commented Aug 17, 2020

.Any() is still a problem for me in EF Core InMemory 3.1.7. When I replace .any() with .Count() > 0 my tests pass.

@tomasbruckner
Copy link

This still fails in 3.1.8 in InMemoryDatabase but works fine in SQL Server/PostgreSQL.

Exception

Unable to cast object of type 'System.Linq.Expressions.NewExpression' to type 'System.Linq.Expressions.MethodCallExpression'.

code

public static IQueryable<ContainerDto> ToDtoList(this IQueryable<Container> data)
{
    return data.Select(AsDto);
}

public static readonly Expression<Func<Container, ContainerDto>> AsDto =
  item => new ContainerDto
{
    Text = item.Text,
    Cars = item.Cars
        .AsQueryable()
        .Select(AsCarDto)
        .FirstOrDefault(),
};

public static readonly Expression<Func<Car, CarDto>> AsCarDto =
  item => new CarDto
  {
      Name = item.Name,
  };

@ryannasr
Copy link

ryannasr commented Oct 11, 2020

I am having the same problem as reply above. Works in SQL Server but not InMemoryDatabase. Is there a known workaround?

Thanks!

@ajcvickers ajcvickers modified the milestones: Backlog, 6.0.0 Nov 5, 2020
smitpatel added a commit that referenced this issue Mar 10, 2021
Implement left join as client method to reduce complexity
To resolve the indexing issue stemming from #23934
Now all the projections are applied immediately to reshape the value buffer so that all our future bindings are always read value expressions which refers to a proper index.
In order to do this, we apply select for entity type with hierarchy to avoid entity check conditional expression.
For adding projection (through ReplaceProjectionMapping),
- For client projection, we apply a select and re-generate client projection as read values
- For projection mapping, we iterate the mappings, we apply a select and re-generate the mapping as read values
- If projection mapping is empty then we add a dummy 1 so that it becomes non-range-variable
When applying projection, we generate a selector lambda to form a value buffer and replace all the expressions to read from new value buffer. Overall this solves the issue of having complex expressions to map or pull. This also removed PushDownIntoSubquery method.

In order to avoid the issue of indexes changing when generating join due to iterating projection mappings, we now also have projectionMappingExpressions which remembers the all expressions inside projectionMapping (which are all read value as we generated before). So now we don't need to iterate the mapping and we use the existing expressions directly. This keeps existing indexes.

Resolves #13561
Resolves #17539
Resolves #18194
Resolves #18435
Resolves #19344
Resolves #19469
Resolves #19667
Resolves #19742
Resolves #19967
Resolves #20359
Resolves #21677
Resolves #23360
Resolves #17537
Resolves #18394
Resolves #23934
Resolves #17620
Resolves #18912
smitpatel added a commit that referenced this issue Apr 29, 2021
@smitpatel smitpatel added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Apr 29, 2021
smitpatel added a commit that referenced this issue Apr 29, 2021
smitpatel added a commit that referenced this issue Apr 29, 2021
smitpatel added a commit that referenced this issue May 5, 2021
@ghost ghost closed this as completed in #24805 May 5, 2021
ghost pushed a commit that referenced this issue May 5, 2021
@ajcvickers ajcvickers modified the milestones: 6.0.0, 6.0.0-preview5 May 27, 2021
@ajcvickers ajcvickers modified the milestones: 6.0.0-preview5, 6.0.0 Nov 8, 2021
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-in-memory area-query closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. punted-for-3.1 type-bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants