Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docs: document how to proper make batch inserts/updates #1388

Open
abenhamdine opened this issue Jul 28, 2017 · 4 comments
Open

Docs: document how to proper make batch inserts/updates #1388

abenhamdine opened this issue Jul 28, 2017 · 4 comments
Assignees
Milestone

Comments

@abenhamdine
Copy link
Contributor

abenhamdine commented Jul 28, 2017

Hello,

Regularly there some questions about how to make batch inserts/updates and how to escape values safely.
Eg see :
#1400
#1387
#957
#880

I don't find any reference to this topic in the new doc site, but a user of this library can legitimately wonder how to handle this :

@charmander charmander added this to the docs milestone Jul 28, 2017
@qas
Copy link

qas commented Aug 2, 2017

👍

@brianc brianc self-assigned this Aug 4, 2017
joelmukuthu pushed a commit to knorm/knorm that referenced this issue Aug 18, 2017
@ariofrio
Copy link

ariofrio commented May 8, 2020

Any guidance on this? I'm thinking of this in comparison with the way JDBC's PostgreSQL driver works:

  1. JDBC batch API. That is PreparedStatement#addBatch() enables driver to send multiple "query executions" in a single network roundtrip. Current implementation, however would still split large batches into smaller ones to avoid TCP deadlock.

    The actions would be much better:

    execute query
    ...
    execute query
    execute query
    execute query
    sync <-- wait for the response from the DB
    

It seems that using Promise.all() might be the equivalent in node-pg. However:

  1. Does node-pg client state get corrupted if a client.query() call occurs before another client.query() call has resolved, on the same Client?
  2. Does the second client.query() wait until the first client.query() has received a response from the server, or only until the first client.query() has finished sending the query to the server?
  3. [Especially if we don't wrap time-overlapping client.query() calls in a transaction], does node-pg queue server responses appropriately to resolve the correct Promise with the correct response?
  4. Is there any performance advantage to wrapping the calls in a transaction?

I am planning to migrate a PostgreSQL JDBC microservice that uses 1000-batched inserts to achieve desired performance to node-pg, which is why I am trying to figure this out.

@charmander
Copy link
Collaborator

@ariofrio Batched queries – pipelining – is broader than batch insert. pg doesn’t have pipelining yet. I think @brianc is working on it.

I am planning to migrate a PostgreSQL JDBC microservice that uses 1000-batched inserts to achieve desired performance to node-pg, which is why I am trying to figure this out.

You should probably try to do it in fewer queries anyway, like #957 (comment).

@sebastianrothbucher
Copy link

case it still helps: temp tables can achieve it - see this comment: #956 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants