Memory leak, or forgotten thread?

[RaptorCZ]

Hi,
there must be some problem inside LESS compiler in version 2.x. If I remember, it started around 2.0.8 build and persists.

I'll open solution and memory for VS process is around 600-700MB, CPU 0%. I'' start compilation of LESS file and do this a few times. Then VS process starts jumping to memory around 2GB and CPU 30%, then back to original values (800 MB and 0%) and back to 2GB and 30% and so on. Only way how to fix it is restart VS.

Any idea what is wrong inside? Tried previous version (pre 2.x) of WE and all works fine. So problem is maybe in any async method.

Is there any way how to trace problem?
Thanx

[SLaks]

This might be the sourcemap parser.

[am11]

No; SourceMap parser is there since 2.0 RC and according to OP the problem started around 2.0.8.

[RaptorCZ]

I can't remember exact version, but definitely this problem is relative new for me. Version 2.0 has been ok. It starts in version 8, maybe 9.

It is too bad there is no debug logging inside. Just something like "Thread start", "Thread end" etc. But I know it is complicated.

I'll try again after VS 2013 Update 2 and we will see. Maybe some magic will happen :-)

[am11]

@RaptorCZ, @ctolkien

Please try with this visx http://1drv.ms/1qyt0P3. You will probably get a graceful exception message when testing compiler to its limit.. but performance-wise, it should be better than before.

[am11]

I will sent the PR, once I get confirmation from you guys. TIA!

[ctolkien]

Hi @am11 - no noticeable change I'm afraid, LESS compilation time is still in the order of 10 seconds (and VS cpu/memory usage remains elevated for 30+ seconds after compilation is complete).

[am11]

@ctolkien, thanks for reverting.

Apparently, your issue is slightly different than one reported by @RaptorCZ (which I think should be fixed by the aforementioned build).

For your issue, please try with this build: http://1drv.ms/1gyDhBb

There is a new settings Process source maps under LESS and SCSS, set it t false and retry.

PS: we need to optimize Base64Vlq class code; I merely translated Mozilla's JavaScript code for Base64Vlq to C# and add few things. This is where its spending most time..

[SLaks]

I'll take a look at the performance now.

[SLaks]

This is caused by a horrible mix of dynamic and repetitive Substring() (creating a new string instance for each character in the source map).

I'm cleaning that up, which should make this orders of magnitude faster, and will hopefully finish tonight.
@am11, how good is the unit test coverage for this feature?
If the unit tests pass, what else should I check?

[SLaks]

On a related note, I believe that System.Web.Helpers.Json is rather slow.
We should replace all usages of it with JSON.Net (and avoid dynamic), especially for source maps.

[madskristensen]

@SLaks Do you think it's possible to make those changes within the next 6 hours? I'm going to announce the release of 2.1 at TechEd in my talk at 5pm CST so it would be cool it could be released before then.

[am11]

@SLaks, yes, we need to optimize Base64Vlq class code; I just translated Mozilla's JavaScript code to C#.

The unit tests were written the same way, not ditto but they were inspired by form Mozilla's repo too. I am pretty confident, the correctness is not the issue with the current implementation of the class (sadly the performance is). Nonetheless, I used this online tool http://murzwin.com/base64vlq.html as a reference (for manual verification).

Also, if you want I can help with testing with your interim commits.

[SLaks]

We should also remove all use of dynamic from the Node executor pipeline (https://github.com/madskristensen/WebEssentials2013/search?l=csharp&q=dynamic&ref=cmdform) and replace it with regular types & JObject.

[am11]

@SLaks, this is fixed by 6801720.

[am11]

@SLaks, when I try to test a Json.Net command in immediate window during debug, it says that Newtonsoft.json.dll is included twice. I can't find duplicate references. Could it be due to the fact that JSON editor in VS 2013 Update 2 is using JSON.NET and hence conflicting with the one we have included in project?

[SLaks]

It could also be coming from some other extension.
Check where each DLL comes from in the Modules window.

If it is always included in VS, we should disable Copy Local on our reference.

[am11]

It only happens with Newtonsoft.Json.Linq namespace:

The type 'Newtonsoft.Json.Linq.JObject' exists in both 'Newtonsoft.Json.dll' and 'Newtonsoft.Json.dll'

For Newtonsoft.Json it works fine.

[SLaks]

What version is in PrivateAssemblies?
@madskristensen, what editions/SPs of VS is that included in? (in particular, what about Express?)

[am11]

We are using latest: 6.00.3.17227

VS is using older: 4.05.11.15520

Here is the raw data:

Newtonsoft.Json.dll C:\USERS\ADEEL\APPDATA\LOCAL\MICROSOFT\VISUALSTUDIO\12.0EXP\EXTENSIONS\MADS KRISTENSEN\WEB ESSENTIALS 2013 FOR UPDATE 2\2.1\Newtonsoft.Json.dll No  No  Cannot find or open the PDB file.       460 6.00.3.17227    4/27/2014 6:12 AM   21980000-21A00000   [7872] devenv.exe   [1] DefaultDomain   
Newtonsoft.Json.dll C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PrivateAssemblies\Newtonsoft.Json.dll   No  No  Cannot find or open the PDB file.       275 4.05.11.15520   11/20/2012 12:45 PM 15D00000-15D66000   [7872] devenv.exe   [1] DefaultDomain   

[SLaks]

Then we should probably stick with the problem.
Adding an alias to our reference ought to allow you to use it in the debugger, but, IIRC, it doesn't.

[am11]

You are right.

I think VS team should use alias for it, so users don't get into trouble.

[SLaks]

Aliases are a compiler feature; VS using an alias wouldn't help at all.

If an alias helps (which I suspect it wouldn't), it would be because the debugger sees it in the current project configuration.

The VS debugger team ought to devise a way to allow usages with these conflicts.

[am11]

Don't know if it make sense to report it on Connect or just leave it with @madskristensen / @spadapet..

[spadapet]

For suggestions/bugs with the VS debugger, use Connect. It'll get to the right people.

[am11]

@spadapet, done. Here is the link to Connect feedback item: https://connect.microsoft.com/VisualStudio/feedback/details/873636/.