Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

texlive: how to get texdoc to work? #118757

Closed
benneti opened this issue Apr 7, 2021 · 10 comments
Closed

texlive: how to get texdoc to work? #118757

benneti opened this issue Apr 7, 2021 · 10 comments
Labels
0.kind: bug Something is broken 6.topic: TeX Issues regarding texlive and TeX in general

Comments

@benneti
Copy link
Contributor

benneti commented Apr 7, 2021

Describe the bug
I would like to use texdoc but it does not work.
Things I tried: using texlive.combined.scheme-full as well as scheme-medium with

    pkgFilter = (pkg:
      pkg.tlType == "run" || pkg.tlType == "bin" || pkg.pname == "core"
      || pkg.pname == "doc" || pkg.pname == "pgfplots");

results in

texdoc pgfplots
texdoc error: No texlive.tlpdb nor shipped tlpdb data found.

Trying

  tl = pkgs.texlive.combine {
    inherit (pkgs.texlive) scheme-medium;
    # see nixpkgs/pkgs/tools/typesetting/tex/texlive/combine.nix
    pkgFilter = (pkg:
      pkg.tlType == "run" || pkg.tlType == "bin" || pkg.pname == "core"
      || pkg.tlType == "doc");
  };

does not work at all due to a collision

warning: Git tree '/etc/nixos' is dirty
error: builder for '/nix/store/dgrfkd2kl5bzxjnkbkvzpx4wsz9nqmwk-texlive-combined-2020.drv' failed with exit code 25;
       last 1 log lines:
       > collision between `/nix/store/ghdsckyw0l0fhjwf7chpl3rns10psbpq-texlive-dvipdfmx.doc-56557/doc/man/man1/dvipdfmx.1' and `/nix/store/9w6mmym0qc9mwm2kx30p76p1x5zdm1yl-texlive-bin-2020-doc/doc/man/man1/dvipdfmx.1'
       For full logs, run 'nix log /nix/store/dgrfkd2kl5bzxjnkbkvzpx4wsz9nqmwk-texlive-combined-2020.drv'.
error: 1 dependencies of derivation '/nix/store/5wc2n4014h5zhxfdrlpnk6zmnvc16ys8-home-manager-path.drv' failed to build
error: 1 dependencies of derivation '/nix/store/i5yzv5jha91h8yvnw05aj9jpg0zphwp0-home-manager-generation.drv' failed to build
error: 1 dependencies of derivation '/nix/store/vamlcjm19wrgdrmv58xdds52pvinvc53-user-environment.drv' failed to build
error: 1 dependencies of derivation '/nix/store/6z1gipicql3q4lwbv9cgchi79bvdk5hz-etc.drv' failed to build
error: 1 dependencies of derivation '/nix/store/j4pgfb2wqqrzcvd6nzwfqi9isayydnkm-nixos-system-pulse-21.05.20210405.3d1a771.drv' failed to build

To Reproduce
Steps to reproduce the behavior:

  1. Install a texlive scheme that should have a working texdoc
  2. try texdoc command
  3. texdoc error: No texlive.tlpdb nor shipped tlpdb data found.

Expected behavior
Open the pdf with the documentation of the package

Additional context
Add any other context about the problem here.

Notify maintainers
@veprbl

Metadata

  • system: "x86_64-linux"
  • host os: Linux 5.11.11, NixOS, 21.05.20210405.3d1a771 (Okapi)
  • multi-user?: yes
  • sandbox: yes
  • version: nix-env (Nix) 2.4pre20210326_dd77f71
  • nixpkgs: /nix/store/sz015sc0y6adpacafc4zs35wf8364c6a-source

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:
@benneti benneti added the 0.kind: bug Something is broken label Apr 7, 2021
@veprbl veprbl added the 6.topic: TeX Issues regarding texlive and TeX in general label Apr 7, 2021
@bb010g
Copy link
Contributor

bb010g commented Apr 22, 2021

The following seems to work around the texlive-bin-${bin.texliveYear}-doc (bin.core.doc) collisions well enough:

let tl = pkgs.texlive; in (tl.combine ({
  pkgFilter = p: p.tlType == "run" || p.tlType == "bin" || p.tlType == "doc" || p.pname == "core";
} // { inherit (tl)
  scheme-medium
; })).override (super: {
  paths = let
    pathsSuper = super.paths or [ ];
    inherit (builtins) match storeDir;
    inherit (lib) isStorePath partition removePrefix substring;
    removeStoreAndHashPrefix = p: substring 34 (-1) (removePrefix storeDir p);
    isTlBinDoc = p: isStorePath p && match "texlive-bin-[0-9]{4}-doc(/.*)?" (removeStoreAndHashPrefix p) != null;
    rightThenWrong = { right, wrong, ... }: right ++ wrong;
  in rightThenWrong (partition isTlBinDoc pathsSuper);
  ignoreCollisions = true;
})

(Documentation for binaries may conflict with preferred documentation from Nixpkgs's TeX Live bin.)

However, even with that texlive-combined, texdoc still fails due to lack of texlive.tlpdb:

$ nix-shell --pure --run 'texdoc --debug texdoc'
texdoc debug-version: /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf/scripts/texdoc/texdoclib.tlu v3.3
texdoc debug-config: Setting "debug_list=all" from command line option "--debug".
texdoc debug-config: Setting "viewer_txt=less -RF" from environment variable "PAGER".
texdoc debug-config: Setting "suffix_list=doc, -doc, _doc, .doc, /doc, manual, /manual, -manual, userguide, /user_guide, -guide, -user, -man, notes, -info, ref" in file "/nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf/texdoc/texdoc.cnf" on line 94.
texdoc debug-config: Setting "lastfile_switch=true" in file "/nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf/texdoc/texdoc.cnf" on line 561.
texdoc debug-config: Setting "lang=en" from operating system locale.
texdoc debug-config: Setting "ext_list=pdf, htm, html, txt, dat, md, ps, dvi, " from built-in defaults.
texdoc debug-config: Setting "machine_switch=false" from built-in defaults.
texdoc debug-config: Setting "interact_switch=true" from built-in defaults.
texdoc debug-config: Setting "basename_list=readme, 00readme" from built-in defaults.
texdoc debug-config: Setting "badbasename_list=readme, 00readme" from built-in defaults.
texdoc debug-config: Setting "max_lines=20" from built-in defaults.
texdoc debug-config: Setting "verbosity_level=2" from built-in defaults.
texdoc debug-config: Setting "mode=view" from built-in defaults.
texdoc debug-config: Setting "fuzzy_level=3" from built-in defaults.
texdoc debug-config: Setting "badext_list=txt, dat, " from built-in defaults.
texdoc debug-config: Setting "zipext_list=" from built-in defaults.
texdoc debug-config: Setting "rm_file=rm -f" from built-in defaults.
texdoc debug-config: Setting "rm_dir=rmdir" from built-in defaults.
texdoc debug-files: Configuration file(s):
texdoc debug-files: not found   /home/bb010g/.texlive2021/texmf-config/texdoc/texdoc-bin.cnf
texdoc debug-files: not found   /home/bb010g/.texlive2021/texmf-config/texdoc/texdoc.cnf
texdoc debug-files: not found   /home/bb010g/.texlive2021/texmf-config/texdoc/texdoc-dist.cnf
texdoc debug-files: not found   /home/bb010g/.texlive2021/texmf-var/texdoc/texdoc-bin.cnf
texdoc debug-files: not found   /home/bb010g/.texlive2021/texmf-var/texdoc/texdoc.cnf
texdoc debug-files: not found   /home/bb010g/.texlive2021/texmf-var/texdoc/texdoc-dist.cnf
texdoc debug-files: not found   /home/bb010g/texmf/texdoc/texdoc-bin.cnf
texdoc debug-files: not found   /home/bb010g/texmf/texdoc/texdoc.cnf
texdoc debug-files: not found   /home/bb010g/texmf/texdoc/texdoc-dist.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-local/texdoc/texdoc-bin.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-local/texdoc/texdoc.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-local/texdoc/texdoc-dist.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-config/texdoc/texdoc-bin.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-config/texdoc/texdoc.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-config/texdoc/texdoc-dist.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-var/texdoc/texdoc-bin.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-var/texdoc/texdoc.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-var/texdoc/texdoc-dist.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf/texdoc/texdoc-bin.cnf
texdoc debug-files: active      /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf/texdoc/texdoc.cnf
texdoc debug-files: not found   /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf/texdoc/texdoc-dist.cnf
texdoc debug-texdocs: texdocs[7] = /home/bb010g/.texlive2021/texmf-config/doc (index_mandatory=false, recursion_allowed=true)
texdoc debug-texdocs: texdocs[6] = /home/bb010g/.texlive2021/texmf-var/doc (index_mandatory=false, recursion_allowed=true)
texdoc debug-texdocs: texdocs[5] = /home/bb010g/texmf/doc (index_mandatory=false, recursion_allowed=true)
texdoc debug-texdocs: texdocs[4] = /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-local/doc (index_mandatory=true, recursion_allowed=true)
texdoc debug-texdocs: texdocs[3] = /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-config/doc (index_mandatory=true, recursion_allowed=true)
texdoc debug-texdocs: texdocs[2] = /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf-var/doc (index_mandatory=true, recursion_allowed=true)
texdoc debug-texdocs: texdocs[1] = /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf/doc (index_mandatory=true, recursion_allowed=true)
texdoc debug-texdocs: texdocs[1] using index: /nix/store/5qcy1xdb40bx1fs1dlp25i2phmg0swwv-texlive-combined-2020/share/texmf (shift=doc/)
texdoc debug-tlpdb: Using shipped tlpdb data.
texdoc error: No texlive.tlpdb nor shipped tlpdb data found.

Texdoc stopped shipping its own fallback TLPDB information with v3.0: TeX-Live/texdoc#20 . Gentoo has a related bug ( https://bugs.gentoo.org/687592 ) and patch ( https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3c1eca3b76170d18bb229dd35ec34c256b3faa51 ) where it began packaging its own texlive-tlpdb-${year}.tar.xz, containing the sole file texlive.tlpdb. I don't know where that particular file is from or how it is created. It looks approximately equivalent to https://mirror.ctan.org/systems/texlive/tlnet/tlpkg/texlive.tlpdb.xz.

We could replicate that approach, as that file is used to generate pkgs.nix:

### Upgrade package information from texlive package database
```bash
curl -L http://mirror.ctan.org/tex-archive/systems/texlive/tlnet/tlpkg/texlive.tlpdb.xz \
| xzcat | uniq -u | sed -rn -f ./tl2nix.sed > ./pkgs.nix
```
This will download a current snapshot of the CTAN package database `texlive.tlpdb.xz`
and regenerate all of the sha512 hashes for the current upstream distribution in `pkgs.nix`.

Alternatively, Nixpkgs should be able to provide an appropriate TLPDB file when combining.

@veprbl
Copy link
Member

veprbl commented Apr 22, 2021

A proper solution for collisions is to make sure that there isn't a duplicate man page in texlive.bin.core{,-big}.

Shipping tlpdb should not be a problem, the question is how to make it so that texdoc finds it.

@benneti
Copy link
Contributor Author

benneti commented Apr 23, 2021

I think the simplest approach would be to put it in the correct place, see https://github.com/TeX-Live/texdoc/wiki/Packaging-Texdoc
And the correct place seems to be local texlive_tlpdb = get_tlroot() .. '/tlpkg/texlive.tlpdb' (https://github.com/TeX-Live/texdoc/blob/f5ec5891c69595b181f023d90c56c22afb600438/script/texdoclib-search.tlu#L661) .
And maybe additionally one needs to set local tlroot = kpse.expand_path('$TEXMFROOT') -- it should be exisitng one (https://github.com/TeX-Live/texdoc/blob/f5ec5891c69595b181f023d90c56c22afb600438/script/texdoclib-search.tlu#L33)
which is (at least for me) not set in the wrapper args but might not be necessary...

I hope this helps!

@nixos-discourse
Copy link

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/texdoc-cannot-find-texlive-tlpdb/9303/2

@xworld21
Copy link
Contributor

xworld21 commented Jun 19, 2021

I have a branch where the texdoc package works:
https://github.com/NixOS/nixpkgs/compare/master...xworld21:texlive-texdoc-data-tlpdb-lua?expand=1#diff-78dd1e8609118bb30c483b837f017110017218a9b7c388887a9c0ed0aa823f36R60
Rather than shipping texlive.tlpdb, I have used these instructions to generate Data.tlpdb.lua, which is a lot smaller (1.8M vs 17M). texdoc generates that cache on its first run anyway.

The only problem is that Data.tlpdb.lua is not reproducible. I have reported the reproducibility issue upstream, and for now I have added a patch so that we can keep the fixed hash.

I haven't done anything about the man pages, so you still need to use pkgFilter carefully.

Edit: the texdoc maintainers were incredibly quick and added the patch upstream, so I have made a PR now.

@stale
Copy link

stale bot commented May 2, 2022

I marked this as stale due to inactivity. → More info

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label May 2, 2022
@benneti
Copy link
Contributor Author

benneti commented May 2, 2022

For me, using this:

  tl = pkgs.texlive.combine {
    inherit (pkgs.texlive) scheme-full;
    # see nixpkgs/pkgs/tools/typesetting/tex/texlive/combine.nix
    pkgFilter = (pkg:
      pkg.tlType == "run" || pkg.tlType == "bin" || pkg.pname == "core"
      || pkg.tlType == "doc" );
  };

It now get's one step further (on nixos-unstable), i.e. no error, but it does not find any documentation.
@xworld21 with your PRs is there anything else that is necessary?

@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label May 2, 2022
@tobiasBora
Copy link
Contributor

tobiasBora commented May 2, 2022

In my case, texdoc works without much troubles (NixOs unstable 1ffba9f), including when I install the documentation for all packages (by default, it just comes with 30 docs, when I install the documentation for all packages I have 9393 docs):

    ((texlive.combine {
      inherit (texlive) scheme-full;
      pkgFilter = pkg: lib.elem pkg.tlType [ "run" "bin" "doc" ];
    })

Demo:

# List all documentation entries:
$ texdoc -l .
9393 results. Display them all? (y/N) N
$ texdoc pgf
# opens documentation for tikz/pgf ! :-D

However, I hit two compilation errors that I needed to solve to make this work:

To avoid these issues, I actually need to add to my configuration:

  security.pam.loginLimits = [{
    domain = "*";
    type = "soft";
    item = "nofile";
    value = "262144";
  }];

then switch to make sure this will be applied on the next compilation, then I add texlive allowing collisions :

  environment.systemPackages = with pkgs; [
    ((texlive.combine {
      inherit (texlive) scheme-full;
      pkgFilter = pkg: lib.elem pkg.tlType [ "run" "bin" "doc" ];
    }).override (args: { ignoreCollisions = true; }))
   # ...
  ];

@benneti
Copy link
Contributor Author

benneti commented May 2, 2022

Ah indeed, I think I was still in an old shell session while trying it out.
Also I have

    pam.loginLimits = [{
      domain = "*";
      type = "soft";
      item = "nofile";
      value = "4096"; # four times the default; hopefully sufficient
    }];

which is sufficient for me.

@benneti
Copy link
Contributor Author

benneti commented May 2, 2022

Closing as the main issue is solved, and the remaining problems seem to have separate issues open

@benneti benneti closed this as completed May 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: bug Something is broken 6.topic: TeX Issues regarding texlive and TeX in general
Projects
None yet
Development

No branches or pull requests

6 participants