GET on ANY container with an index.html returns HTML, not Turtle

[jeff-zucker]

Please describe what you did in reproducible steps

Performed a GET on https://jeffz.solid.community/ (5.x) and on https://jeffz.solid.openlinksw.com:8444/ (4.x)

How did it work with 4.x series servers?

The GET returned Turtle

What happened when you tried the same with the 5.x series server?

The GET returns HTML

Any material that will help, logs, error messages, etc.

[dmitrizagidulin]

@jeff-zucker What was the Accept: header, on that GET, in each case?

[megoth]

I believe this might be connected to a hack we did earlier on NSS - https://github.com/solid/node-solid-server/blob/master/lib/handlers/allow.js#L56

IIRC the hack was there to mitigate the fact that the root ACL was preventing index.html to be served, which made all home pages for pods private by default. This is not a problem for newer pods, but was a problem for older pods (again, IIRC).

[jeff-zucker]

I tried without specifying an Accept header (which, should return Turtle anyway, I believe), and also with Accept "text/turtle". And got HTML in both cases.

[mikeadams1]

I agree with @megoth, I remember bringing this problem up but, the fix was not supposed to create this problem.

[jeff-zucker]

This looks to be the same issue as #1261

[mikeadams1]

I guess my question would be, what has changed in NSS that would cause this to occur? When did it change, And who changed it? Well I guess that’s a few questions that I have.

[jeff-zucker]

I have confirmed this is a problem on all containers, not just the root container. A GET on .../anyContainer/ returns HTML if there is an index.html file even if Accept "text/turtle" is specified. This effectively means there is no way to find out the contents of containers that have an index.html. Which breaks lots of things ...

[mikeadams1]

Hopefully it all gets figured out. Thanks for the update Jeff.

[rescribet]

I set up a local server (dev branch) from which I learned it's backed by a literal filesystem, so trying a DELETE to https://fletcher91.solid.community/index.html succeeded with a 200 (the home page now has the data browser).

The turtle representation works fine now, but trying to fetch that resource as turtle from my app results in a 403 however (I can't find the acl/trustedorigin settings My app has full r/w/a/c permissions).

[ludwigschub]

I think this PR fixes it, maybe you can give me some feedback

[mikeadams1]

I tried using solid IDE to edit route index.HTML but it is still not working.

[jeff-zucker]

Did you use a patched version of NSS that implements the PRs? If not, there's no reason to think it would work since the deployed version of NSS has not implemented any of the pending fixes. The problem will remain until that occurs.

[mikeadams1]

Just checking

[ludwigschub]

I tried using solid IDE to edit route index.HTML but it is still not working.

@mikeadams1 As far as I understand it this issue is about get requests, I don't exactly know what you want to do but editing sounds to me like you're doing a post/put request. So I guess that this is outside of the scope of this issue. Is that correct?

Did you use a patched version of NSS that implements the PRs? If not, there's no reason to think it would work since the deployed version of NSS has not implemented any of the pending fixes. The problem will remain until that occurs.

@jeff-zucker Yes it is my own server with the changes from my PR. I saw that there are other pull requests doing similar things so I closed it for now, but the changes there essentially allow the GET requests to work again

[jeff-zucker]

@ludwigschubi , apologies for the confusion, I was asking Mike if he was using a patched version, not you. There are several related bugs involved with root index.html, the GET being only one of them,

[mikeadams1]

No, not using patched version Jeff, maybe I will launch NSS, maybe not but, currently working on connecting some existing tools and experimenting with various use cases. Tim stated on gitter, "I feel we have a lot of tools with solid which we need to connect together to form solutions. Also we need to work on each one to get ot more polished." So I am trying to figure out ways that some of these tools can be used to build an enterprise.

[mikeadams1]

@ludwigschubi

@mikeadams1 As far as I understand it this issue is about get requests, I don't exactly know what you want to do but editing sounds to me like you're doing a post/put request. So I guess that this is outside of the scope of this issue. Is that correct?

No, not outside scope, I currently use curl –cookie “connect.sid=” https:.net/ -F data=”@/<your/file/index.html>” to edit root index.html but love using soild ide

[ludwigschub]

curl –cookie “connect.sid=” https:.net/ -F data=”@/<your/file/index.html>

@mikeadams1 When you use curl with the -F Option you’re sending a post request. But as @jeff-zucker pointed out there are multiple issues with the root index file so I guess you are right about posting your findings here 👍

[Otto-AA]

Has this already been fixed/updated on the solid.community server?

When I do a GET request with Accept: text/turtle on a folder with an index.html file in it I currently get a text/turtle response which is empty (ie it only contains @prefix : <#>.). So I experience a different issue than described here.

[michielbdejong]

@Otto-AA For solid.community specifically, that server will switch over to https://github.com/inrupt/pod-server soon, which does not have the same issues.

Until then, if you want to run a Solid pod server (on localhost or on your own server) and need help with that, ping me on gitter!

[jeff-zucker]

@michielbdejong - what then is the fate of NSS 5.x? Will it be patched? @Otto-AA, @bourgeoa,and I are trying to write solid-file-client and are, at this point, very confused about how to support NSS. Running a local server is not so much the question, rather, supporting the servers used by the apps which use our library.

[michielbdejong]

Yes, I understand it's a bit painful for you guys at the moment. In particular, @jaxoncreed and I wanted to fix this particular issue for you, but we couldn't confidently review @bourgeoa's PR with confidence because none of us truly understand how the ResourceMapper works exactly. This week, we'll put up the new server on dev.inrupt.net and then everybody can test against that. We'll try to keep the switch-over window as short as possible.

[mikeadams1]

To demonstrate what is happening and to study it out is simple. Go to https://jeff-zucker.github.io/solid-ide/?url=https://index.solid.community/ unfortunately we cant edit our home pages however, if you look at timbl's pod under the same at: https://jeff-zucker.github.io/solid-ide/?url=https://timbl.com/ you get a fully editable root index.html

[bourgeoa]

A problem everybody has is that the switch from dev.inrupt to solid.community is not notified to the community, so that developers can have some time to make their own checks and try to understand changes or unexpected problems.

Could there be a kind of panel with let say a few days delay ?

[jeff-zucker]

@michielbdejong, thanks for the information. I very strongly second @bourgeoa's suggestion. Could you please answer my question above - are there any plans to patch NSS or is it at end of life?

[michielbdejong]

Yes sorry, we should make sure that the switch is a positive experience for everyone involved.

So previously I thought it would be a good idea to update inrupt.net to the new inrupt-pod-server (IPS) TypeScript code, and leave solid.community running NSS. But that would split the development community in half. At first, that might be acceptable, but we expect a lot of spec changes to happen over the coming months, and it would require one or two full-time people to keep NSS up to date with the latest Solid spec.

So then we have to make a choice, do we invite / look for people who want to take on that project, or do we give up on the idea of maintaining two independent server implementations, and invite people to come work on IPS instead. Unless a group of enthusiastic people still show up who want to keep NSS alive separately from IPS, I think it's best to shut NSS code maintenance down cold-turkey.

As I said, I think the best course of action now is that we stop working on NSS and all start working on IPS, starting today. From inrupt's side, it's probably going to be mainly @jaxoncreed working on IPS, so he will no longer be available to keep maintaining NSS. But anybody can work on any code they want to, so it's also not a centralised decision. :)

But let's take that decision as a group! Tomorrow's call is in the Asia timezone so maybe not ideal, but maybe we can set up a call especially to discuss this topic and the transition process?

[mikeadams1]

So as far as the accounts on solid.community, will that server be shut down, and if so when?

[jeff-zucker]

Many thanks for the detailed explanation. I agree that having the typescript server on both inrupt.net and solid.community would be better than leaving the current NSS on solid.community. Are there links you'd recommend to get an idea of what changes we can expect with the new server? I'll make an effort to stay up tonight for the call.

[bourgeoa]

Even if I agree in principal I don't know if there could not be a last NSS version that could ease the time span between now and a functional IPS.

[jeff-zucker]

@michielbdejong - I tried and failed to stay up till 1:00 am, so missed the call. The minutes are uninformative, can you summarize where things now stand?

[jaxoncreed]

This should be fixed in #1282