Rendering to "pdf" doesn't work

[piiskop]

Once i want a file to be rendered to pdf, i get the following error message:

running xelatex - 1
command xelatex not found, attempting install
finding package for xelatex
1 package to install
installing xelatex (1 of 1)
ERROR: tlmgr returned a non zero status code
TLPDB: not a directory, not loading: /home/kalmer/texmf
tlmgr: running in usermode, did you call `tlmgr init-usertree'?

[dragonstyle]

I think perhaps there is an issue with your installation of TeXLive. Perhaps this would be of help:

https://tex.stackexchange.com/questions/137428/tlmgr-cannot-setup-tlpdb

I'm guessing that you need to run the tlmgr init-usertree command to enable usermode. See https://tug.org/texlive/doc/tlmgr.html#USER-MODE

[piiskop]

kalmer@test:~$ tlmgr init-usertree

tlmgr: user mode database already set up in
tlmgr: /home/kalmer/.TinyTeX/texmf-local/tlpkg/texlive.tlpdb
tlmgr: not overwriting it.

running xelatex - 1
command xelatex not found, attempting install
finding package for xelatex
1 package to install
installing xelatex (1 of 1)
ERROR: tlmgr returned a non zero status code
TLPDB: not a directory, not loading: /home/kalmer/texmf
tlmgr: running in usermode, did you call `tlmgr init-usertree'?

[dragonstyle]

I'm not enough of an expert on TeXlive to give too much additional help - it seems like the issue is that we're detecting that the xelatex command is missing and attempting to using the tlmgr to install it. Since something is amiss with your installation of TeXlive, however, that is failing. You could set something like:

pdf-engine: pdflatex

in your document front matter to instruct us not to use xelatex (note that pdflatex unicode character support is minimal). You also could manually install xelatex using tlmgr and we will use that installed version without problem.

[piiskop]

tlmgr install xelatex

tlmgr: package repository https://www.nic.funet.fi/pub/TeX/CTAN/systems/texlive/tlnet (verified)
tlmgr install: package xelatex not present in repository.
tlmgr: action install returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.

As for pdflatex:

pandoc
to: latex
output-file: authoring.tex
standalone: true
shift-heading-level-by: -1
pdf-engine: pdflatex
variables:
graphics: true
tables: true
default-image-extension: pdf
toc: true
number-sections: true
filters:
- crossref
- citeproc

metadata
documentclass: scrartcl
classoption:
- DIV=11
- numbers=noendperiod
papersize: letter
header-includes:
- '\KOMAoption{captions}{tableheading}'
block-headings: true
title: Housing Prices
author: Mine Çetinkaya-Rundel
bibliography:
- references.bib
geometry:
- top=30mm
- left=30mm

running pdflatex - 1
This is pdfTeX, Version 3.14159265-2.6-1.40.20 (TeX Live 2019/Debian) (preloaded format=pdflatex)
restricted \write18 enabled.
entering extended mode

updating tlmgr

updating existing packages
finding package for tcolorbox.sty

compilation failed- no matching packages
LaTeX Error: File `tcolorbox.sty' not found.

Type X to quit or to proceed,
or enter new name. (Default extension: sty)

Enter file name:
! Emergency stop.
<read *>

l.144 @ifpackageloaded
{fontawesome5}{}{\usepackage{fontawesome5}}^^M

see authoring.log for more information.

[dragonstyle]

Not looking good! I think until you figure out the underlying with the TeXLive installation, this may not work. Do you know how that TeXLive installation was installed / configured? We do support installation of TeXLive using quarto tools install tinytex, but that will require the exiting installation to be remove before it will install...

[piiskop]

i only know that building works just fine, rendering doesn't. Using bookdown, i can build a nice PDF-file.

i don't know exactly how TeXLive was installed / configured. How can i figure it out?

Your given command resulted with:

Installing tinytex
Install canceled - tinytex is already installed.

apt nor snap recognize tinytex.

[jmbuhr]

Currently running into a similar issue. There was some change in the tlmgr repositories:

ERROR: Your TexLive version is not updated enough to connect to the remote repository and download packages. Please update your installation of TexLive or TinyTex.

Underlying message: Local TeX Live (2021) is older than remote repository (2022).
Cross release updates are only supported with
  update-tlmgr-latest(.sh/.exe) --update
See https://tug.org/texlive/upgrade.html for details.


Error: Your TexLive version is not updated enough to connect to the remote repository and download packages. Please update your installation of TexLive or TinyTex.

Underlying message: Local TeX Live (2021) is older than remote repository (2022).
Cross release updates are only supported with
  update-tlmgr-latest(.sh/.exe) --update
See https://tug.org/texlive/upgrade.html for details.

Quarto detects an existing TexLive installation and thus doesn't want to instll tinytex:

$ quarto tools install tinytex  
Installing tinytex
An existing TexLive installation has been detected.

So now I am trying to find all the remnants of the old TexLive installation and remove them.

[jmbuhr]

Here is what worked for me:

• Follow the top answer here: https://tex.stackexchange.com/questions/95483/how-to-remove-everything-related-to-tex-live-for-fresh-install-on-ubuntu
• check if which tlmgr returns anything, remove it if it does
• rm -r ~/.TinyTeX/

Then quarto tools install tinytex (follow instructions if you get warnings)

[lyiqian]

Thanks. I only did quarto tools install tinytex and problem solved.

Relevant official doc: https://quarto.org/docs/output-formats/pdf-engine.html

[piiskop]

i also could solve the issue using a similar workaround. This however doesn't answer the question why PDF'ying works with bookdown and doesn't with Quarto without manual interfering into the installation of TinyTex.

[dragonstyle]

I think the issue is likely that Bookdown doesn't need to install any additional packages to render your PDF and Quarto does. Since TeXLive can't install any new packages, anything that needs a new package and attempts to use tlmgr to install will result in a failure.

I believe that if you add a package requirement to your rmarkdown document (for example tcolorbox, which Quarto uses for callouts), you will find that rendering fails as well.

---
title: "Using more LaTeX packages"
output: 
  pdf_document:
    extra_dependencies: ["tcolorbox"]
---

Until the TexLive installation is able to install new packages, rendering of documents will be dependent upon what packages are available locally / already installed.

[jmbuhr]

Random sidenote: I just love how the installation guide for TeX Live (https://www.tug.org/texlive/quickinstall.html), which was apparently updated in 2022, features this sentence:

"A separate page describes various ways to acquire the software. It boils down to either getting the DVD from a TeX user group (ideally by becoming a member), or downloading in various ways."

[jeremiahpslewis]

Posting in case helpful: if you are looking for a template repo with a Dockerfile and the appropriate configuration so that things (mostly) 'just work' with Docker + VSCode, you can check out this project: https://github.com/jeremiahpslewis/quarto-reproducible-gh-actions which might save you some of the 'works on my PC' pain of getting latex installed.

[melissa-barca]

@dragonstyle This has been causing the RSW doc builds to fail for almost a month now as well. Quarto initially recognizes this and skips over an attempt to reinstall it:

[2022-05-23T16:51:20.418Z] Installing tinytex
[2022-05-23T16:51:20.418Z] Install canceled - tinytex is already installed.

...but eventually quarto render fails with:

[2022-05-23T16:53:08.481Z] runnning xelatex - 1
[2022-05-23T16:53:08.481Z] 
[2022-05-23T16:53:08.481Z] No TeX installation was detected.
[2022-05-23T16:53:08.481Z] 
[2022-05-23T16:53:08.481Z] Please run 'quarto tools install tinytex' to install TinyTex.
[2022-05-23T16:53:08.481Z] If you prefer, you may install TexLive or another TeX distribution.
[2022-05-23T16:53:08.481Z] 
[2022-05-23T16:53:09.041Z] Error: Render failed with a non-zero error code
[2022-05-23T16:53:09.041Z] Execution halted
[2022-05-23T16:53:09.041Z] CMakeFiles/build_book.dir/build.make:57: recipe for target 'build_book' failed
[2022-05-23T16:53:09.041Z] make[2]: *** [build_book] Error 1
[2022-05-23T16:53:09.041Z] CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/build_book.dir/all' failed
[2022-05-23T16:53:09.041Z] make[1]: *** [CMakeFiles/build_book.dir/all] Error 2
[2022-05-23T16:53:09.041Z] Makefile:83: recipe for target 'all' failed
[2022-05-23T16:53:09.041Z] make: *** [all] Error 2
script returned exit code 2

It looks very similiar to the errors we debugged a few months ago. I'm trying to figure out the correct way to get it installed in our docker image but keep hitting similar errors.

[jjallaire]

If you are running an older version of Quarto you may need to update (there was a texlive update from 2021 to 2022 which required some changes to Quarto to work correctly). @dragonstyle is that your understanding as well?

[jmbuhr]

It is probably related to this bigger texlive update (https://tug.org/texlive/upgrade.html), which advises to just do a fresh installation. It also states that "You don't need to remove an existing installation.", but I am not sure how true that is. Maybe quarto tools install could have a --force option, to attempt an installation even if a previous version is detected?

[piiskop]

kalmer@test:~/rstudio-projects$ which tlmgr

/home/kalmer/bin/tlmgr

[dragonstyle]

Does that symlink point elsewhere?
ls -la /home/kalmer/bin/tlmgr (I think)

[piiskop]

lrwxrwxrwx 1 kalmer kalmer 44 mai 17 20:38 /home/kalmer/bin/tlmgr -> /home/kalmer/.TinyTeX/bin/x86_64-linux/tlmgr

[dragonstyle]

Ok this is pretty conclusive that we're just very confused trying to find a healthy happy TeX installation. Thx for going through all that!

We actually try to detect the TeX installation by see whether we can run the pdftex command and seeing whether that returns an error result. Is that runnable from the console? If not, that will likely be the root issue and we'll need to figure out if there is a better way for us to discover the TeX installation (for example, we could try the various pdf engines or the specific pdf engine that is use)...

[piiskop]

If that was a question then yes - pdftex is runnable in the terminal emulator but not in RStudio's console.

[dragonstyle]

Thanks, I appreciate the follow up- this is obviously very difficult to debug remotely.

---

@piiskop can you please try the following in the terminal:

• remove the tcolorbox package using: tlmgr remove tcolorbox
• Attempt to install the tcolorback package using: tlmgr install tcolorbox

If those both succeed, can you then try:

• remove the tcolorbox package using: tlmgr remove tcolorbox
• Render your bookdown project using the tcolorbox dependency and see whether it still successfully renders.

---

I am at a loss for what is going wrong here, we appear to be finding and attempting to use the existing TeXLive installation, and for some packages like xelatex and tcolorbox it appears to not be able to find those packages when we attempt to install them- I don't know why. If those commands fail when run in the terminal, they will also fail in Quarto since this is essentially what we do to orchestrate package installation.

[piiskop]

Both first commands succeeded.

Rendering after second removal went also fine even without any attempt to install tcolorbox.

[dragonstyle]

Rendering after second removal went also fine even without any attempt to install tcolorbox.

That seems very unexpected.

I have no currently viable explanation for what you're seeing other than there being more than one TeXLive environment on your machine. But this seems to have been conclusively ruled out, so I am not sure what to try to next.

Can you confirm that quarto rendering is still not working and the error that you receive is that the package tcolobox.sty can't be found?

[piiskop]

i can't render to PDF as there's no such option. There are only: Render, Render Website and Preview Website.

[LibrEars]

hi

maybe it helps: on a Debian 11 install (TexLive over repos and quarto via .deb) pdf output does fail as well:

running xelatex - 1                                                                              
command xelatex not found, attempting install                                                    
finding package for xelatex                                                                      
> 1 package to install                                                                           
> installing xelatex (1 of 1)                                                                    
ERROR: tlmgr returned a non zero status code                                                     
TLPDB: not a directory, not loading: /home/m/texmf                                               
tlmgr: user mode not initialized, please read the documentation!

I tried also to install the package 'r-cran-tinytex' from the repos or try the render command as root. No success with that.

Using pdf-engine: pdflatex or pdf-engine: lualatex in the YAML worked.

On the other hand pdf-engine: pdfcslatex or pdf-engine: xelatex or simply pdf-engine: latex did not work (list from https://manpages.debian.org/testing/texlive-xetex/xelatex.1.en.html)

[knuesel]

For @piiskop's problem, it seems to me there are several issues:

1. quarto doesn't find xelatex although it seems TinyTeX is installed.

   Maybe it's just a symlink missing? There is already a symlink /home/kalmer/bin/tlmgr -> /home/kalmer/.TinyTeX/bin/x86_64-linux/tlmgr, maybe there should be a similar one for xelatex?

2. quarto tries to install xelatex with tlmgr, but this can't work: there is no such package. There is a texlive xetex package, but even installing that would not provide the xelatex command anyway (see below).

I can reproduce @LibrEars's problem on Debian/Ubuntu, if I make a "small" texlive installation with the system package manager, for example if I install texlive-latex-base with apt, instead of texlive-full. In this case I run into a cascade of problems when quarto tries to install xelatex:

1. tlmgr complains "user mode not initialized", so I need to run tlmgr init-usertree
2. tlmgr complains that "Local TeX Live (2020) is older than remote repository (2022)" so I need to change the repository to an older version by running tlmgr option repository ftp://ftp.math.utah.edu/pub/tex/historic/systems/texlive/2020/tlnet-final
3. tlmgr complains that "package xelatex not present in repository".

The core problem seems to be: quarto uses tlmgr to install xelatex in the user tree but this is impossible: texlive binaries must be installed system-wide, see https://www.tug.org/texlive/doc/tlmgr.html:

All actions on packages in user mode can only be carried out on packages that are known as relocatable. This excludes all packages containing executables and a few other core packages. Of the 2500 or so packages currently in TeX Live the vast majority are relocatable and can be installed into a user tree.

and even so, I think the xelatex binary is not installable with tlmgr: it's part of the core installation of texlive. This core install happens to be split into several packages in Debian/Ubuntu, so the user needs to do e.g. sudo apt install texlive-xetex.

So for @LibrEars's issue the proper fix is probably to document that "when using the system texlive, the user must install texlive-xetex"

[knuesel]

More generally, maybe the right fix would be for quarto to stop using the system tlmgr (at least be default). The Debian/Ubuntu documentation in /usr/share/doc/texlive-base/README.tlmgr-on-Debian.md says:

Since package management (installation, update, ...) is the responsability
of APT (apt, apt-get, ...), tlmgr cannot interfere with it, but uses
the "TeX Live Manager User Mode" instead.
...
We strongly recommend not to user the TeX Live Manager user mode on Debian.
If you are using it despite the warnings, be prepared to fix your own TeX system.

This readme also gives a list of potential problems when using the system tlmgr.

By default, when using the system texlive quarto could ask the user to install the correct system package for the missing LaTeX package?

As I understand, the next version of quarto will manage its own TinyTeX in a way completely isolated from the system? That sounds great and will mostly fix this issue I guess... The documentation can then say something like:

quarto can use the texlive distribution installed in your system, but in this case you'll have to manually install TeX packages required by your document/project but missing from the system; Or you can run quarto install tool tinytex to let quarto manage its own TeX distribution.

[cderv]

As I understand, the next version of quarto will manage its own TinyTeX in a way completely isolated from the system? That sounds great and will mostly fix this issue I guess... The documentation can then say something like:

Yes. User who still have this issue should really try out the latest prerelease version (https://quarto.org/docs/download/) and reinstall TinyTeX with it.

If you are not using TinyTeX, but another distrution, then the constraint still remain: TeX Live should be in PATH for Quarto to found it. tlmgr path add is usually the way to do it.

[mcanouil]

Quarto documentation now states¹:

• we strongly recommend the use of TinyTeX

• We also recommend the use of Quarto’s built in PDF compilation engine, which among other things performs automatic installation of any missing TeX packages.

I would recommend to convert this to a discussion.

Footnotes

1. see https://quarto.org/docs/output-formats/pdf-engine.html. ↩

[AurelieQU]

Please help, I am really sorry if I am not in the right section:
running xelatex - 1
This is XeTeX, Version 3.141592653-2.6-0.999995 (TeX Live 2023) (preloaded format=xelatex)
restricted \write18 enabled.
entering extended mode

updating tlmgr

updating existing packages

compilation failed- error
LaTeX Error: Mismatched LaTeX support files detected.
(LaTeX) Loading 'expl3.sty' aborted!
(LaTeX)
(LaTeX) The L3 programming layer in the LaTeX format
(LaTeX) is dated 2023-11-01, but in your TeX tree the files require
(LaTeX) at least 2023-11-09.

For immediate help type H .
...

l.78 \ExplLoaderFileDate{expl3.sty}}
%