pg client fails to connect (silently) using Node 14 #2317

martinbliss · 2020-08-23T17:07:18Z

The pg client seems to fail quietly when connecting to postgress when using Node 14.x. Tested using Docker.

Dockerfile

FROM node:14.0.0-alpine3.10 as builder
RUN apk add postgresql-client
WORKDIR /src
COPY . .
ENTRYPOINT [ "/bin/sh" ]

package.json

{
  "devDependencies": {
    "pg": "^8.3.2",
    "ts-node": "^8.10.2",
    "typescript": "^3.7.5"
  }
}

test-pg-client.ts

const { Client } = require('pg');

start();

async function start() {
    const conString = "postgres://postgres:(password)@localhost:5432/postgres";

    const client = new Client(conString);
    await client.connect();

    const query = await client.query('select 1');

    query.rows.forEach(value => console.info(`row value`, value));

    await client.end();
}

Since we're using Docker, let's build and run the image

docker build -t my-image .
docker run -it --network=host my-image

(this runs the image with host network access, allowing you to connect to a postgres database that is running on the same machine. )

Expected Result (while inside the container):

❯ yarn install
❯ ts-node test-pg-client.ts
row value { '?column?': 1 }

Actual Result:

❯ ts-node test-pg-client.ts

(no errors, no messages, just nothing)

The behavior is remedied by changing the first line in Dockerfile to the following:

FROM node:12.18.3-alpine3.12 as builder

This behavior seems similar to #2180 but wasn't totally sure if the circumstances were the same, so chose to make a new issue just in case.

The text was updated successfully, but these errors were encountered:

krishchow · 2020-08-24T01:42:17Z

+1 I noticed this too, reverting to nodev12.18.3 fixed the issue for me as well

charmander · 2020-08-24T13:22:06Z

Your package.json has "pg": "^8.3.2", but can you make sure that it’s what’s being used? In the same script, try

console.log(require('pg/package.json').version);

martinbliss · 2020-08-24T13:58:32Z

Your package.json has "pg": "^8.3.2", but can you make sure that it’s what’s being used? In the same script, try
console.log(require('pg/package.json').version);

Adding this line to the above TS file:

console.log('PG VERSION', require('pg/package.json').version);

yields the following output:

PG VERSION 8.3.2

charmander · 2020-08-25T04:11:39Z

How about on the current version of Node 14?

FROM node:14.8.0-alpine3.12

martinbliss · 2020-08-25T05:10:11Z

Same result:

> node -v
v14.8.0
> ts-node test-pg-client.ts
PG VERSION 8.3.2

(still unable to connect to a live postgres instance that is accessible via a different script which uses ES6 fetch to verify the instance is accessible from inside the same container.)

rufreakde · 2020-08-25T06:37:13Z

I have the same issue this is blocking me as I need to upgrade to node v14.4.0.
So I cloned the repo and just run your tests but they fail on node 14. Could you please investigate? @brianc

npm run test

> [email protected] test /Users/X/git/node-postgres/packages/pg
> make test-all

***Testing optional native install***
Cannot find module 'pg-native'
Require stack:
- /Users/x/git/node-postgres/packages/pg/lib/native/client.js
- /Users/x/git/node-postgres/packages/pg/lib/native/index.js
- /Users/x/git/node-postgres/packages/pg/lib/index.js
- /Users/x/git/node-postgres/packages/pg/test/native/missing-native.js
configuration-tests.js.
startup-tests.js.......
inbound-parser-tests.js..............................................................
error-tests.js
  connection emits stream errors ✔
  connection emits ECONNRESET errors during normal operation ✔
  connection does not emit ECONNRESET errors during disconnect ✔
  connection does not emit ECONNRESET errors during disconnect also when using SSL ✔
  connection emits an error when SSL is not supported ✔
  connection emits an error when postmaster responds to SSL negotiation packet ✔

utils-tests.js.........................
sasl-scram-tests.js................
result-metadata-tests.js.......
simple-query-tests.js..............?.....
throw-in-type-parser-tests.js
  emits error ✔
  calls callback with error ✔
  rejects promise with error ✔

early-disconnect-tests.js
escape-tests.js................
md5-password-tests.js....
configuration-tests.js............
query-queue-tests.js.........
notification-tests.js.
cleartext-password-tests.js..
set-keepalives-tests.js
  setting keep alive ✔

stream-and-query-error-interaction-tests.js.
prepared-statement-tests.js...........
environment-variable-tests.js
  ConnectionParameters initialized from environment variables ✔
  ConnectionParameters initialized from mix ✔
  connection string parsing ✔
  connection string parsing - ssl ✔
  ssl is false by default ✔
  ssl is false when $PGSSLMODE= ✔
  ssl is false when $PGSSLMODE=disable ✔
  ssl is false when $PGSSLMODE=allow ✔
  ssl is true when $PGSSLMODE=prefer ✔
  ssl is true when $PGSSLMODE=require ✔
  ssl is true when $PGSSLMODE=verify-ca ✔
  ssl is true when $PGSSLMODE=verify-full ✔
  ssl is [object Object] when $PGSSLMODE=no-verify ✔

creation-tests.js.................?.....
***Testing connection***
/Users/x/git/node-postgres/packages/pg/script/create-test-tables.js:44
    throw err
    ^

Error: connect ECONNREFUSED 127.0.0.1:5432
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1141:16) {
  errno: -61,
  code: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 5432
}
make: *** [test-connection] Error 1
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] test: `make test-all`
npm ERR! Exit status 2
npm ERR!
npm ERR! Failed at the [email protected] test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/x/.npm/_logs/2020-08-25T06_30_56_178Z-debug.log // no useful info here

rufreakde · 2020-08-25T07:09:27Z

Update:
I deleted my node_modules folder and changed pg version to "pg": "8.3.0", and this seems to work fine with node 14.4

nicoburns · 2020-08-25T19:43:17Z

I had this same issue with pg version 7 and Node 14. It may be worth noting in the README that pg v8 or upwards is required for Node 14.

brianc · 2020-08-26T21:41:01Z

I had this same issue with pg version 7 and Node 14. It may be worth noting in the README that pg v8 or upwards is required for Node 14.

agreed - I can do this

martinbliss · 2020-08-26T22:28:28Z

I had this same issue with pg version 7 and Node 14. It may be worth noting in the README that pg v8 or upwards is required for Node 14.

That's really strange, considering I have verified that I'm running PG 8.3.2 on Node 14 in a clean docker build (that is to say, no node_modules).

yocontra · 2020-08-28T18:42:16Z

@martinbliss Working for me in node 14.9 w/ the latest pg version - prior to that I tested it on 14.0 and had the same problem. Make sure you wipe all of your node_modules folders and yarn/npm cache so that every native extension gets a full rebuild. Your solution will probably just be changing your docker base image to the latest 14 release instead of the first one - not sure why the first release (14.0.0) was bad, they changed a bunch of behavior with streams so it may be to blame.

martinbliss · 2020-08-28T19:00:14Z

@martinbliss Working for me in node 14.9 w/ the latest pg version - prior to that I tested it on 14.0 and had the same problem. Make sure you wipe all of your node_modules folders and yarn/npm cache so that every native extension gets a full rebuild. Your solution will probably just be changing your docker base image to the latest 14 release instead of the first one - not sure why the first release (14.0.0) was bad, they changed a bunch of behavior with streams so it may be to blame.

I failed to mention in my original post that I have a .dockerignore file which excludes node_modules from the COPY . . instruction, so there shouldn't be any concerns about conflicting node_modules / native binding issues with respect to node 14 vs. node 14.9 (inside a container.) If pg has an issue with certain versions of node 14, then I would expect that to be called out directly in the README. The current guidance that pg 8.3+ should work on Node 14 seems to be contradicted by this issue.

nicoburns · 2020-08-28T19:36:04Z

Node 14.0 made a bunch of backwards incompatible changes to streams that seemed to break quite a lot of libraries. I'd definitely avoid that version in general. They reverted the changes in 14.1, and the later releases are all fine. However, IIRC pg (once updated to v8) was not one of the libraries affected by this (at least for how we're using it).

yocontra · 2020-08-28T21:30:35Z

@brianc One thing that most people in the community are doing to avoid having node versions break streams is using https://github.com/nodejs/readable-stream instead of the core stream module directly. From the node repo: "If you want to guarantee a stable streams base, regardless of what version of Node you, or the users of your libraries are using, use readable-stream only and avoid the "stream" module in Node-core"

Not even sure if this is what is responsible for this particular issue - but I do see the stream module used directly in a lot of the packages in this monorepo. We had a lot of random bugs with streams across versions (particularly 14 and early 13 releases) that disappeared by switching everything to readable-stream.

charmander · 2020-08-28T21:48:31Z

@contra The issue with pg <8.0.3 was using a property that wasn’t part of the public API. I don’t think there’s anything like that left.

abenhamdine · 2020-09-02T09:46:30Z

Just adding a datapoint to this thread : we use in production pg 8.0.3 with nodejs 14.0.0 without any problem.

charmander · 2020-09-03T18:17:59Z

@martinbliss Is it also broken when connecting to a PostgreSQL server running in another container, not just the host?

ireznik · 2020-09-05T10:00:30Z

Thank you guys. I ran into this issue with an older version (<8) and thought I did something wrong

martinbliss · 2020-09-05T13:36:07Z

I don’t know, since I only run databases in containers.

…

On Thu, Sep 3, 2020 at 1:18 PM Charmander ***@***.***> wrote: @martinbliss <https://github.com/martinbliss> Is it also broken when connecting to a PostgreSQL server running in another container, not just the host? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2317 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC3FK5ZOESOMHRTBI5MACEDSD7MWRANCNFSM4QIYJ7KA> .

charmander · 2020-09-06T04:25:56Z

@martinbliss Oh, I thought this was accessing a database on the host because of the --network=host. Is there any difference on a different, non-host network, then?

martinbliss · 2020-09-06T04:27:45Z

No. I have a similar setup using docker-compose (where one container is accessing Postgres in a different container inside the same network) and the outcome is the same. I only chose to include the —network=host option to make it easier to repro the issue without revealing non open-source code. On September 5, 2020 at 11:26:09 PM, Charmander ([email protected]) wrote: @martinbliss <https://github.com/martinbliss> Oh, I thought this was accessing a database on the host because of the --network=host. Is there any difference on a different, non-host network, then? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2317 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC3FK55XVQC3SILBR6TZPFDSEMFOBANCNFSM4QIYJ7KA> .

samueldaviddelacruz · 2020-11-21T20:11:00Z

has this been solved for people using containers? still having the same issue using pg 8.5.X

codyhazelwood · 2020-11-21T20:22:45Z

@samueldaviddelacruz We are using node-Postgres now with node 14 in alpine and Debian containers. Upgrading to the latest version of node-Postgres solved the problem for us.

samueldaviddelacruz · 2020-11-21T20:33:22Z

@codyhazelwood exactly which version? im using this 14-alpine and it causing issues,pg version => "pg": "^8.5.1", this is my dockerfile:

FROM node:14-alpine
WORKDIR "/app"
COPY ./package.json ./
RUN npm install
COPY . .

CMD ["npm","run","dev"]```

samueldaviddelacruz · 2020-11-21T21:16:56Z

update: i fixed my issue by starting from scratch with my containers, deleted all the old images and containers and build them again, this fixed the issue on my side.

charmander · 2020-11-22T05:02:08Z

I set up a VM to reproduce the steps from the original issue exactly, and the output was:

/src # ts-node test-pg-client.js
/bin/sh: ts-node: not found

I’m not sure if that’s something that changed in a dependency, but either way:

/src # node_modules/.bin/ts-node test-pg-client.js
row value { '?column?': 1 }

@martinbliss Are you still having an issue connecting from containers with Node 14?

SteveFromAccounting · 2021-01-20T03:20:42Z

This was a real humdinger for me. I am using:

Node.js v14.15.4 (npm v6.14.10)
node-postgres v8.5.1

And the problem is now fixed. I was previously on 7.12.1 and same version of Node.

Fortegot · 2021-02-12T17:42:03Z

I've noticed this too... I cant tell the huge amount of hours i've wasted trying to make my app work with node 14...
Updating pg libraries to 8 version i could make it work again (with some further changes). Thank you all for giving the idea of upgrading pg...
But finally, i found out that a listener build on pg-pubsub has stopped working too and it seems to be no solution yet for that.
Can anyone give a piece of advice about pg-pubsub with node-14??

Thanks in advance!

charmander · 2021-02-13T07:54:00Z

@martinbliss Are you still having an issue connecting from containers with Node 14?

reopen if so!

tkssharma · 2021-06-03T08:05:02Z

This is why I don't like the Node JS Ecosystem
I spent 1 hour checking why my connection is not done with Postgres !!
As it was silent no errors!

BlackBo6 · 2021-06-04T20:24:28Z

Update:
I deleted my node_modules folder and changed pg version to "pg": "8.3.0", and this seems to work fine with node 14.4

yeah same , I uninstall pg and install pg version (^8.6.0) , and it works for me with node 15.14.0

brianc mentioned this issue Aug 26, 2020

Update welcome.mdx brianc/node-postgres-docs#96

Merged

cyjake mentioned this issue Nov 20, 2020

upgrade: node 14 requires pg >= 8 cyjake/leoric#48

Merged

charmander added the waiting for more info label Nov 22, 2020

Venryx mentioned this issue Feb 7, 2021

Random.js fails to update DB on Node v14.5.0 graphile/livesotope#12

Open

charmander closed this as completed Feb 13, 2021

monsterooo mentioned this issue Apr 11, 2021

doc.fetch did not execute callback share/sharedb#461

Closed

miccheng mentioned this issue Aug 1, 2021

[Production Deployment] Upgrade pg to v8 as 7 has some issues in Node 14 engineersftw/events-api#56

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pg client fails to connect (silently) using Node 14 #2317

pg client fails to connect (silently) using Node 14 #2317

martinbliss commented Aug 23, 2020

krishchow commented Aug 24, 2020

charmander commented Aug 24, 2020

martinbliss commented Aug 24, 2020

charmander commented Aug 25, 2020 •

edited

Loading

martinbliss commented Aug 25, 2020 •

edited

Loading

rufreakde commented Aug 25, 2020 •

edited

Loading

rufreakde commented Aug 25, 2020 •

edited

Loading

nicoburns commented Aug 25, 2020

brianc commented Aug 26, 2020

martinbliss commented Aug 26, 2020

yocontra commented Aug 28, 2020 •

edited

Loading

martinbliss commented Aug 28, 2020 •

edited

Loading

nicoburns commented Aug 28, 2020

yocontra commented Aug 28, 2020

charmander commented Aug 28, 2020

abenhamdine commented Sep 2, 2020

charmander commented Sep 3, 2020

ireznik commented Sep 5, 2020

martinbliss commented Sep 5, 2020 via email

charmander commented Sep 6, 2020

martinbliss commented Sep 6, 2020 via email

samueldaviddelacruz commented Nov 21, 2020

codyhazelwood commented Nov 21, 2020

samueldaviddelacruz commented Nov 21, 2020 •

edited

Loading

samueldaviddelacruz commented Nov 21, 2020

charmander commented Nov 22, 2020

SteveFromAccounting commented Jan 20, 2021

Fortegot commented Feb 12, 2021

charmander commented Feb 13, 2021

tkssharma commented Jun 3, 2021

BlackBo6 commented Jun 4, 2021 •

edited

Loading

pg client fails to connect (silently) using Node 14 #2317

pg client fails to connect (silently) using Node 14 #2317

Comments

martinbliss commented Aug 23, 2020

krishchow commented Aug 24, 2020

charmander commented Aug 24, 2020

martinbliss commented Aug 24, 2020

charmander commented Aug 25, 2020 • edited Loading

martinbliss commented Aug 25, 2020 • edited Loading

rufreakde commented Aug 25, 2020 • edited Loading

rufreakde commented Aug 25, 2020 • edited Loading

nicoburns commented Aug 25, 2020

brianc commented Aug 26, 2020

martinbliss commented Aug 26, 2020

yocontra commented Aug 28, 2020 • edited Loading

martinbliss commented Aug 28, 2020 • edited Loading

nicoburns commented Aug 28, 2020

yocontra commented Aug 28, 2020

charmander commented Aug 28, 2020

abenhamdine commented Sep 2, 2020

charmander commented Sep 3, 2020

ireznik commented Sep 5, 2020

martinbliss commented Sep 5, 2020 via email

charmander commented Sep 6, 2020

martinbliss commented Sep 6, 2020 via email

samueldaviddelacruz commented Nov 21, 2020

codyhazelwood commented Nov 21, 2020

samueldaviddelacruz commented Nov 21, 2020 • edited Loading

samueldaviddelacruz commented Nov 21, 2020

charmander commented Nov 22, 2020

SteveFromAccounting commented Jan 20, 2021

Fortegot commented Feb 12, 2021

charmander commented Feb 13, 2021

tkssharma commented Jun 3, 2021

BlackBo6 commented Jun 4, 2021 • edited Loading

charmander commented Aug 25, 2020 •

edited

Loading

martinbliss commented Aug 25, 2020 •

edited

Loading

rufreakde commented Aug 25, 2020 •

edited

Loading

rufreakde commented Aug 25, 2020 •

edited

Loading

yocontra commented Aug 28, 2020 •

edited

Loading

martinbliss commented Aug 28, 2020 •

edited

Loading

samueldaviddelacruz commented Nov 21, 2020 •

edited

Loading

BlackBo6 commented Jun 4, 2021 •

edited

Loading