Some tools causes latex workshop to lose track of bibliography changes

[alexanderchr]

Preliminary questions

Disable all the other extensions except for LaTeX Workshop, restart VS Code, and check that you still see this issue. [Required]

You still see this issue?: Yes

Make sure to visit the wiki FAQ before filling an issue.

You visited the wiki?: Yes

If your issue is with compiling a document (not having to do with finding the root file of a project), check first that you can compile manually.

You can compile a TeX document manually?: Yes

Describe the bug

Some tools (for example https://github.com/retorquere/zotero-better-bibtex/) that work with .bib files export them using create-and-move. That is, they first write to a temporary file and then swap the old file for the temporary file. This seems to confuse the file watcher in LaTeX workshop so that changes to this file are only detected once. This means that Intellisense suggestions for citations are not updated until vs code is restarted.

To Reproduce

Create an tex document with a bibliography. Change the bibliography several times using a tool that uses create-and-move for example using the Better BibLaTeX extension for Zotero above).

Expected behavior

I expect changes to the bibliography beyond the first to be detected.

LaTeX Workshop Output [Required]

[13:20:09] Initializing LaTeX Workshop.
[13:20:09] Extension root: /Users/alexander/.vscode/extensions/james-yu.latex-workshop-8.20.2
[13:20:09] $PATH: /Users/alexander/.nvm/versions/node/v13.14.0/bin:/usr/local/anaconda3/condabin:/usr/local/anaconda3/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Frameworks/Mono.framework/Versions/Current/Commands
[13:20:09] $SHELL: /bin/zsh
[13:20:09] vscode.env.appName: Visual Studio Code
[13:20:09] vscode.env.remoteName: undefined
[13:20:09] vscode.env.uiKind: 1
[13:20:09] latex-workshop.bind.enter.key: true
[13:20:09] latex-workshop.docker.enabled: false
[13:20:09] latex-workshop.docker.image.latex: ""
[13:20:09] latex-workshop.intellisense.package.enabled: true
[13:20:09] latex-workshop.intellisense.update.aggressive.enabled: false
[13:20:09] latex-workshop.intellisense.update.delay: 1000
[13:20:09] latex-workshop.latex.autoBuild.run: "onFileChange"
[13:20:09] latex-workshop.latex.outDir: "%DIR%"
[13:20:09] latex-workshop.latex.recipes: [
 {
  "name": "latexmk 🔃",
  "tools": [
   "latexmk"
  ]
 },
 {
  "name": "latexmk (latexmkrc)",
  "tools": [
   "latexmk_rconly"
  ]
 },
 {
  "name": "latexmk (lualatex)",
  "tools": [
   "lualatexmk"
  ]
 },
 {
  "name": "pdflatex ➞ bibtex ➞ pdflatex × 2",
  "tools": [
   "pdflatex",
   "bibtex",
   "pdflatex",
   "pdflatex"
  ]
 },
 {
  "name": "Compile Rnw files",
  "tools": [
   "rnw2tex",
   "latexmk"
  ]
 },
 {
  "name": "Compile Jnw files",
  "tools": [
   "jnw2tex",
   "latexmk"
  ]
 },
 {
  "name": "tectonic",
  "tools": [
   "tectonic"
  ]
 }
]
[13:20:09] latex-workshop.latex.tools: [
 {
  "name": "latexmk",
  "command": "latexmk",
  "args": [
   "-synctex=1",
   "-interaction=nonstopmode",
   "-file-line-error",
   "-pdf",
   "-outdir=%OUTDIR%",
   "%DOC%"
  ],
  "env": {}
 },
 {
  "name": "lualatexmk",
  "command": "latexmk",
  "args": [
   "-synctex=1",
   "-interaction=nonstopmode",
   "-file-line-error",
   "-lualatex",
   "-outdir=%OUTDIR%",
   "%DOC%"
  ],
  "env": {}
 },
 {
  "name": "latexmk_rconly",
  "command": "latexmk",
  "args": [
   "%DOC%"
  ],
  "env": {}
 },
 {
  "name": "pdflatex",
  "command": "pdflatex",
  "args": [
   "-synctex=1",
   "-interaction=nonstopmode",
   "-file-line-error",
   "%DOC%"
  ],
  "env": {}
 },
 {
  "name": "bibtex",
  "command": "bibtex",
  "args": [
   "%DOCFILE%"
  ],
  "env": {}
 },
 {
  "name": "rnw2tex",
  "command": "Rscript",
  "args": [
   "-e",
   "knitr::opts_knit$set(concordance = TRUE); knitr::knit('%DOCFILE_EXT%')"
  ],
  "env": {}
 },
 {
  "name": "jnw2tex",
  "command": "julia",
  "args": [
   "-e",
   "using Weave; weave(\"%DOC_EXT%\", doctype=\"tex\")"
  ],
  "env": {}
 },
 {
  "name": "jnw2texmintex",
  "command": "julia",
  "args": [
   "-e",
   "using Weave; weave(\"%DOC_EXT%\", doctype=\"texminted\")"
  ],
  "env": {}
 },
 {
  "name": "tectonic",
  "command": "tectonic",
  "args": [
   "--synctex",
   "--keep-logs",
   "%DOC%.tex"
  ],
  "env": {}
 }
]
[13:20:09] latex-workshop.viewer.pdf.internal.keyboardEvent: "auto"
[13:20:09] Creating a new file watcher.
[13:20:09] watcherOptions: {"useFsEvents":false,"usePolling":false,"interval":300,"binaryInterval":1000,"awaitWriteFinish":{"stabilityThreshold":250}}
[13:20:09] Creating PDF file watcher.
[13:20:09] watcherOptions: {"useFsEvents":false,"usePolling":false,"interval":300,"binaryInterval":1000,"awaitWriteFinish":{}}
[13:20:09] Creating Bib file watcher.
[13:20:09] watcherOptions: {"useFsEvents":false,"usePolling":false,"interval":300,"binaryInterval":1000,"awaitWriteFinish":{"stabilityThreshold":250}}
[13:20:09] Set $LATEXWORKSHOP_DOCKER_LATEX: ""
[13:20:09] [Server] Creating LaTeX Workshop http and websocket server.
[13:20:09] LaTeX Workshop initialized.
[13:20:09] Trigger characters for intellisense of LaTeX documents: ["\\",",","{"]
[13:20:09] Bibtex format config: {"tab":"  ","case":"lowercase","left":"{","right":"}","trailingComma":false,"sort":["key"],"alignOnEqual":true,"sortFields":false,"fieldsOrder":[]}
[13:20:09] Current workspace folders: ["file:///Users/alexander/notes"]
[13:20:09] Current workspaceRootDir: file:///Users/alexander/notes
[13:20:09] Found root file from active editor: /Users/alexander/notes/notes.tex
[13:20:09] Root file changed: from undefined to /Users/alexander/notes/notes.tex
[13:20:09] Start to find all dependencies.
[13:20:09] Root file languageId: latex
[13:20:09] [Server] Server successfully started: {"address":"127.0.0.1","family":"IPv4","port":55084}
[13:20:09] Reset file watcher.
[13:20:09] Parsing a file and its subfiles: /Users/alexander/notes/notes.tex
[13:20:09] Found .bib file: /Users/alexander/notes/bibliography.bib
[13:20:09] Added to bib file watcher: /Users/alexander/notes/bibliography.bib
[13:20:09] Parsing .bib entries from /Users/alexander/notes/bibliography.bib
[13:20:09] Added to file watcher: /Users/alexander/notes/notes.tex
[13:20:09] Snippet data loaded.
[13:20:09] Parsed 9 bib entries from /Users/alexander/notes/bibliography.bib.
[13:20:09] Parse fls file.
[13:20:09] Fls file found: /Users/alexander/notes/notes.fls
[13:20:09] Parse aux file: /Users/alexander/notes/notes.aux
[13:20:09] Checking for duplicate labels: /Users/alexander/notes/notes.tex.
[13:20:11] Manager.fileWatcher.getWatched: {"/Users/alexander/notes":["notes.tex"]}
[13:20:11] Manager.filesWatched: ["/Users/alexander/notes/notes.tex"]
[13:20:11] BibWatcher.bibWatcher.getWatched: {"/Users/alexander/notes":["bibliography.bib"]}
[13:20:11] BibWatcher.bibsWatched: ["/Users/alexander/notes/bibliography.bib"]
[13:20:11] PdfWatcher.pdfWatcher.getWatched: {}
[13:20:11] PdfWatcher.pdfsWatched: []
[13:20:13] Bib file watcher - file changed: /Users/alexander/notes/bibliography.bib
[13:20:13] Parsing .bib entries from /Users/alexander/notes/bibliography.bib
[13:20:14] Parsed 9 bib entries from /Users/alexander/notes/bibliography.bib.
[13:20:14] Auto build started detecting the change of a file: /Users/alexander/notes/bibliography.bib
[13:20:14] BUILD command invoked.
[13:20:14] The document of the active editor: file:///Users/alexander/notes/notes.tex
[13:20:14] The languageId of the document: latex
[13:20:14] Building root file: /Users/alexander/notes/notes.tex
[13:20:14] Build root file /Users/alexander/notes/notes.tex
[13:20:14] outDir: /Users/alexander/notes
[13:20:14] Recipe step 1: latexmk, -synctex=1,-interaction=nonstopmode,-file-line-error,-pdf,-outdir=/Users/alexander/notes,/Users/alexander/notes/notes
[13:20:14] Recipe step env: {}
[13:20:14] cwd: /Users/alexander/notes
[13:20:14] LaTeX build process spawned. PID: 15453.
[13:20:14] Recipe of length 1 finished. PID: 15453.
[13:20:14] Successfully built /Users/alexander/notes/notes.tex.

Developer Tools Console [Required]

[felipe.nasc-touchbar]: Command `workbench.action.toggleSidebarVisibility` appears multiple times in the `commands` section.
abstractExtensionService.ts:899 [felipe.nasc-touchbar]: Command `workbench.action.togglePanel` appears multiple times in the `commands` section.
abstractExtensionService.ts:899 [felipe.nasc-touchbar]: Command `workbench.action.showCommands` appears multiple times in the `commands` section.
abstractExtensionService.ts:899 [felipe.nasc-touchbar]: Command `editor.action.toggleRenderWhitespace` appears multiple times in the `commands` section.
abstractExtensionService.ts:899 [felipe.nasc-touchbar]: Command `workbench.action.openGlobalSettings` appears multiple times in the `commands` section.
console.ts:137 [Extension Host] (node:15421) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.(Use `Code Helper (Renderer) --trace-deprecation ...` to show where the warning was created)

Desktop

Please write exact version numbers. Please don't write latest instead of exact numbers.

• OS: macOS Catalina (10.15.4)
• VS Code version: 1.59.0
• Extension version: 8.20.2
• TeX distribution version: TeX Live 2020

Additional questions

Are you using VSCodium?

No

Are you using the Snap or Flatpack versions of VS Code?

No

Are you using LaTeX Workshop with VS Code Remote?

No

Additional context

I first created an issue in the repository for the tool that writes the .bib-file: retorquere/zotero-better-bibtex#1886. In that issue there is also a minimal example reproducing this bug (with code extracted from https://github.com/James-Yu/LaTeX-Workshop/blob/master/src/components/managerlib/bibwatcher.ts). Setting useFsEvents: true when watching the file seems to solve this problem. Also, turning on polling solves the problem (for this there is even a setting latex-workshop.latex.watch.usePolling).

I think this is a problem in this extension because writing using create-and-move is not an unreasonable use case. Possible solutions I see is enabling useFsEvents, or watching for changes on directory level instead of on file level.

[tamuratak]

Also, turning on polling solves the problem (for this there is even a setting latex-workshop.latex.watch.usePolling).

It is the intended solution.

enabling useFsEvents

We have chosen to publish the extension without native libraries. See microsoft/vscode#658 (comment)

watching for changes on directory level instead of on file level.

It increases the maintenance burden.

We can also use a watcher through VS Code API. However, it watches only files within the current workspace folders. See

• https://code.visualstudio.com/api/references/vscode-api#workspace.createFileSystemWatcher
• Please up vote at API: Allow to use the file watcher for aribitrary folders microsoft/vscode#3025

[alexanderchr]

It is the intended solution.

Alright, thanks for taking the time to answer.

[tamuratak]

Please read FAQ next time before opening a new issue.

See https://github.com/James-Yu/LaTeX-Workshop/wiki/FAQ#suggestions-for-bibtex-files-are-not-updated