srcsrv doesnt support URLs with special chars

[gluck]

(better title would be that SRCSRV doesn't support that, but sourcelink could work around it)

Context:

Many (all ?) GIT viewers (e.g. cgit, gitweb) have raw URLs with HTTP params, e.g.

 http://git.zx2c4.com/cgit/plain/Makefile?id=84627609f5a7634732fe1d718e7d1c18254e64c8

SRCSRV doesn't like that

When using http protocol support, the URL is used to generate the local file path (where it'll be saved), which fails because of the special (e.g. ?) chars, with a cryptic error in VS 0x80004005 (you're not helping SRCSRV).

What can you do about it ?

Immediatly

Reject URLs with special chars when indexing

Powershell to the rescue

You could use an indexing strategy that wouldn't rely on SRCSRV http support, but use a powershell command for URL download instead. It's probably slower (400ms powershell startup time on my machine), requires the user to "trust" the command (get a warning in VS), but also opens up the range of possible features (e.g. could do the LF->CRLF conversion on the fly, or could use credentials).

Here's a PDBSTR indexing using powershell that works ™️

SRCSRV: ini ------------------------------------------------
VERSION=2
SRCSRV: variables ------------------------------------------
TRGFILE=%targ%\%fnbksl%(%var2%)
SRCSRVTRG=%TRGFILE%
RAWURL=%var3%
SRCSRVCMD=powershell -NoProfile -Command "(new-object net.webclient).DownloadFile('%RAWURL%','%TRGFILE%')"
SRCSRV: source files ---------------------------------------
C:\foobar\src\johndoe.cs*foobar/2d029c15787cd340eb666c39704732f86082dc53/src/johndoe.cs*http://git/cgit/foobar.git/plain/src/johndoe.cs?id=2d029c15787cd340eb666c39704732f86082dc53
...

Now you may wonder why am I not extracting stuff (URL root, SHA1, project) to variables ?
Glad you asked ! That is because I want a common variables section for every project, to allow/simplify merging assemblies (Hint: I'm the author of ILRepack), and their indexed pdbs.

[ctaggart]

Hi @gluck! I wonder if there is a way to get srcsrv.dll to actually support ? characters.

[gluck]

Simply with the http provider, I doubt it, that's also what MS support replied to this guy:
http://stackoverflow.com/a/18186709/615903

[ctaggart]

Well that sucks! I just reproduced it:

.\Exe\bin\Release\SourceLink.exe index -pr .\SourceLink\SourceLink.fsproj
-pp Configuration Release `
-u 'https://raw.githubusercontent.com/ctaggart/SourceLink/{0}/%var2%?a=abc'

[ctaggart]

Another simple work-a-round is to use a proxy or rewrite the URLs. Several web servers support rewriting URLs. Which one are you using. You can also repurpose the SourceLink-Proxy Node app I made to do what you want. http://blog.ctaggart.com/2015/07/sourcelink-10-with-private-github.html

[gluck]

I'm using cgit behind apache, and I already did that (mod_rewrite), mostly wanted to share the issue/findings, as well as give thoughts around the Powershelll approach.

Also the warning/error will probably help others spend less time troubleshooting this one.

Thx !

[ctaggart]

That is a good idea. Glad you got it working. Can you share the mod_rewrite config details?

[gluck]

Sure, that may be specific to our setup though. I remapped /raw/PROJECT/SHA1/PATH to the cgit raw URL (one of the benefits is that I can change from cgit to another viewer if the need arise).

    RewriteEngine  on
    RewriteRule    "^/raw/([^/]*)/([0-9a-fA-F]*)/(.*)$"  "/cgit/$1.git/plain/$3?id=$2" [PT]

[ctaggart]

My hope is that the Visual Studio implementation will eventually not use srcsrv.dll to download the code. May be that refactoring can be done with the portable pdb changes #93. It is definitely an orthogonal change.

cc @tmat