Performance Issue with 3.3.0

[jessiewestlake]

Version 3.3.0 is significantly slower than 3.2.2. However, 3.2.2, seems to have a bug where the Smart Properties and special variables for the -Property parameter do not work. They always return $null, whether using {$Left.Name} or {$RightIndex}, etc. They always return $null results.

On doing a LeftJoin of two tables with about 2,000 records each, on a single ComputerName property, 3.2.2 takes less than 3 seconds to return the results. 3.3.0 takes about 10 seconds.

I was going to put in the below Issue before I realized I was on an old version (3.2.2). But the new (3.3.0) version is so slow, I don't think I can use it. Here was the issue I was going to put in for 3.2.2:

---

I haven't been able to get any of the Smart Properties or special variables to work on the -Property parameter.

---

Using Example 4 listed on the Readme:

PS C:\> $Employee = ConvertFrom-SourceTable '
Id Name    Country Department  Age ReportsTo
-- ----    ------- ----------  --- ---------
 1 Aerts   Belgium Sales        40         5
 2 Bauer   Germany Engineering  31         4
 3 Cook    England Sales        69         5
 4 Duval   France  Engineering  21         5
 5 Evans   England Marketing    35
 6 Fischer France  Engineering  29         4
 7 Geralds Belgium Sales        71         1
 '

PS C:\> LeftJoin $Employee -On ReportsTo -Equals Id -Property @{ Name = {$Left.Name} }, @{ Manager = {$Right.Name} }

Name Manager
---- -------
            
            
            
            
            
            

---

The weird thing is that if I highlight/select the output in the ISE, it shows what looks like whitespace wherever there is supposed to be a manager listed, but skips the line where it's not supposed to have one. Almost like it knows that Evans doesn't have a manager.

This is all fixed in 3.3.0, but the performance is 3-5x slower.

[iRon7]

Thanks for the notification.
I will dig into this once I am back from holidays (biegining October)

[iRon7]

Note that the readme has changed. Dynamic functions should still work (but I suspect there might a bug here),, smart properties are actually strings (e.g.: 'Left.Name') that refer to a predefined static function, as in the new 4th example:

LeftJoin $Employee -On ReportsTo -Equals Id -Property @{ Name = 'Left.Name' }, @{ Manager = 'Right.Name' }

[iRon7]

Missing properties

I am not able to reproduce the missing properties issue on either version 2.3.3 and version 3.3.0 on both Windows PowerShell (5.1) and Windows Core:

PS C:\> LeftJoin $Employee -On ReportsTo -Equals Id -Property @{ Name = {$Left.Name} }, @{ Manager = {$Right.Name} }

Name    Manager
----    -------
Aerts   Evans
Bauer   Duval
Cook    Aerts
Duval   Evans
Evans
Fischer Duval

Can you supply more information on environment you using ($PSVersionTable)?
Does it also appear if you start a new PowerShell session?

Performance

With regards to the performance issue, I am able to reproduce the performance issue using property expressions:

$Employee = foreach ($id in 1..9999) {
    [pscustomobject]@{
        Id = $Id
        Name = "Name$Id"
        Country = "Country$Id"
        Department = "Department$Id"
        Age = $Id
        ReportsTo = $Id % 8
    }
}

Version 2.3.3:

(Measure-Command {
    LeftJoin $Employee -On ReportsTo -Equals Id -Property @{ Name = {$Left.Name} }, @{ Manager = {$Right.Name} }
}).TotalSeconds
2.4010024

Version 3.3.0:

(Measure-Command {
    LeftJoin $Employee -On ReportsTo -Equals Id -Property @{ Name = {$Left.Name} }, @{ Manager = {$Right.Name} }
}).TotalSeconds
3.0639778

But if I actually use smart properties, there is hardly any performance difference between both versions:

(Measure-Command {
    LeftJoin $Employee -On ReportsTo -Equals Id -Property @{ Name = 'Left.Name' }, @{ Manager = 'Right.Name' }
}).TotalSeconds
2.5031613

There are two changes/features causing the performance degradation on property expressions:

1. The time to make the decision between smart properties and a property expression (for each integration, in the above example: 10,000 * 10,000 times)
2. The support for a safe hash table key syntaxes to prevent code injection:

$Name = 'Name"
LeftJoin $Employee -On ReportsTo -Equals Id -Property @{ Name = {$Left[$Name]} }, @{ Manager = {$Right[$Name]} }

I don't think I will be able to fix this.

[iRon7]

I have closed the issue as the performance degradation is caused by some new few features (smart properties) that could be use instead.

[iRon7]

The performance degradation was caused by the expensief function calls required for the ""smart property** feature (see also this discussion: Optimize static code reusage #19322).
I have fixed this in version 3.8.0, see: #39