[Bug] Connectivity in local instance

[davinci07]

##Community Note

• Please use a 👍 reaction to provide a +1/vote. This helps the community and maintainers prioritize this request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Description

I am trying to create a local instance of the graph-explorer on my machine (not docker because I have a forked branch with some custom features) to connect to my Neptune graph-database after updating a version of the G-E from 1.2.0 to 1.5.x (synced up to this PR. Previously I was able to access the graph database on a local instance of the graph-explorer on windows with npm run start and opening an SSH tunnel to the Neptune instance. However, with the upgrade I have not been able to access the database at all and ran into some network issue. with the change to POST when fetching from /gremlin but not from pg/statistics/summary?mode=detailed. This seems to be similar to this issue but there wasn't any issue with the size of the graph in the 1.2.0 instance.

Environment

• OS: Windows 11 Pro
• Browser: Firefox
• Graph Explorer Version: 1.5.x (synced up to PR#206
• Graph Database & Version: Amazon Neptune 1.2.1.0

Steps to Reproduce

1. Clone the repo until PR#206
2. Install the necessary packages by doing npm i in /packages/graph-explorer and /graph-explorer
3. Open a SSH tunnel as ssh -i <pem file> -N -L 8182:<neptune endpoint>:8182 <ec2 instance that port forwards neptune data>
4. Open on localhost:5173 and synchronize on https://localhost:8182
5. See error:

Expected Behavior

This should be providing the connection after port forwarding and not have issues to the gremlin request at start

---

[kmcginnes]

Hi, @davinci07. Thanks for the report.

As you mentioned, the switch to POST instead of GET for the database queries changed quite a bit. One of the side effects was requiring the use of the proxy server.

Are you using the proxy server in your setup?

[davinci07]

Ah, I was not. I tried to make a connection with proxy.py to my Neptune instance earlier but was unsuccessful. I'll adjust today and see if that fixes it. If you have any recommendations on how to setup one, that would be greatly appreciated.

[kmcginnes]

Typically, I would tell you to go read the development guide. However, it seems to be a bit out of date and lacking some detail. I'll make a note to get this updated.

For now, here are the basics. Feel free to ask for clarification on anything.

Make sure your code is updated

git checkout main
git pull

Ensure you are running the correct Node version

We are using v20 now. If you are using NVM, you can simply do:

nvm use

Otherwise, use whatever method you use to install Node v20.

Enable Corepack

We use Corepack to ensure the package manager used for the project is consistent.

corepack enable

Run the app

Install the dependencies and start the servers.

pnpm install
pnpm start

Open the app

Launch your web browser of choice and navigate to

http://localhost:5173/

Connect to database

At this point, Graph Explorer should be successfully running and it is asking you for connection details. This part is specific to your personal setup.

To ensure the proxy server is configured to be used, here are a few guidelines.

• Check the "Using Proxy-Server" checkbox
• Enter the proxy server in to the "Public or Proxy Endpoint" field:

  http://localhost
  

• Enter your database address in the "Graph Connection URL" field

[davinci07]

Hi @kmcginnes, thank you for the advice for how to best do the updates. After I got the code up to date I tried a few variations including your initial recommendation for the connection, a CORS extension for Firefox, and a proxy.py in the attached screenshots, however I still was not able to connect and received the following errors regardless of the variation. Do you know if this a CORS setting I need to change in Firefox? I tried in Chrome as well with the same results.

[kmcginnes]

From what I can tell you seem to be doing everything right.

Let's try a few things to see if we can narrow the scope of the issue.

• Try pulling and running the latest code changes in the aws/graph-explorer repo. I know you have customizations in your fork, but I want to rule out any code issues.
• Try disabling all extensions in your browsers, both Chrome and Firefox. I doubt this will have an impact, but you never know.
• Try connecting without proxy.py. I don't have one of these in my Neptune connection setup. I SSH, like you, and have the Neptune address in my /etc/hosts file pointed at 127.0.0.1. So I wonder if this proxy.py script is adding something to the mix that is making the browser unhappy.

[kmcginnes]

@davinci07 Can you copy the CORS error from the browser and paste the full text as a comment here? Feel free to scrub any private URLs. I'm just curious about the exact message.

[davinci07]

Yes here are the errors I received with the updated pull:

CORS:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://localhost/pg/statistics/summary?mode=detailed. (Reason: CORS request did not succeed). Status code: (null).

POST:
NS_ERROR_DOM_BAD_URI

GET:
NS_ERROR_DOM_BAD_URI

[Summary API] TypeError:
NetworkError when attempting to fetch resource.

Console:
ERROR (2964): request to https://**************.us-east-1.neptune.amazonaws.com:8182/pg/statistics/summary?mode=detailed failed, reason: connect ECONNREFUSED 127.0.0.1:8182

[davinci07]

Hi @kmcginnes I seemed to have solved the issue in the following manner for WSL:

Using:

• Node v20.13.1
• PNPM v8.15.6

Setup for G-E:

1. Change the port (if local only) in packages/graph-explorer-proxy-server/node-server.js Line 461 from 80 to {PORT > 1024} AND "Access-Control-Allow-Origin" : "*" on Line 312 if on WSL

2. Open SSH tunnel

3. run pnpm start

4. Set the following in the connections pop up:

   a. Public or Proxy Endpoint: `http://localhost:`{PORT > 1024}`
   b. Graph Connection URL: `https://{neptune-database}:8182`
   

This should return everything correctly. No need for a proxy-server since its built into G-E but it needs the changes in packages/graph-explorer-proxy-server/node-server.js

[kmcginnes]

That's great news! I'm glad you figured it out.

In hindsight, it makes sense that port 80 must've been used by another service. I believe we should move Graph Explorer off the assumption that port 80 will be open for any developer running Graph Explorer locally. I'll add an issue for this change.

I'd like to know more about the access control change you made.

AND "Access-Control-Allow-Origin" : "*" on Line 312

My line numbers don't align with yours since we seem to be using different versions of the code. Can you link within GitHub to the line of code you changed? Or better yet, make a small example branch with the changes you made.

[davinci07]

Yes, here is the link:

graph-explorer/packages/graph-explorer-proxy-server/node-server.js

Line 329 in 4d4ae7e

Accept: "application/json",

The headers within this line should be more like:

headers: {
  "Content-Type": "application/json",
  "Accept": "application/json",
  "Access-Control-Allow-Origin" : "*"
},

or 

    headers: {
  "Content-Type": "application/json",
  "Accept": "application/json",
  "Access-Control-Allow-Origin" : "ORIGIN"
},

This fixed any of the CORS issues within Firefox and Chrome so it should help

[kmcginnes]

That's an interesting result. I admit that my understanding of CORS is limited, but from what I'm reading, adding a Access-Control-Allow-Origin header to the request does nothing. It is the server that must determine if a request is allowed and the server places the Access-Control-Allow-Origin header in the response.

There seems to be a few allowed header options on the request side, but you are not using these and it works. Very confusing.

Additionally, we are enabling CORS pre-flight checks in Express middleware, which should handle these things for us.

graph-explorer/packages/graph-explorer-proxy-server/node-server.js

Line 167 in 4d4ae7e

app.use(cors());

@davinci07 would you mind trying one more thing and commenting out the new header you added to the request for Access-Control-Allow-Origin and see if it works?

[davinci07]

Huh... interesting. That seemed to have worked with and without the header change. CORS issues that were encountered earlier in Firefox/Chrome have disappeared even though using proxy-servers. Seems like the change in the ports/port forwarding was the necessary fix for local. Feel free to close this issue!

[kmcginnes]

@davinci07 That's great news! 🎉 Thanks for letting me know.