Bibtex stripping extra extensions

[kprussing]

The BibTeX and Biber actions when processing LaTeX files were stripping multiple extensions before passing the file to either bibtex or biber. This would result in the bibliography tool not finding the correct .aux file and running LaTeX multiple times for no gain.

Example

Given a main LaTeX file named main.extra.tex.

Old behavior:

1. latex main.extra produces the file main.extra.aux
2. bibtex main call issued by LaTeX Builder
3. BibTeX issues warning No file main.extra.aux
4. LaTeX reruns maximum number of times and does not typeset the bibliography

Expected behavior:

1. latex main.extra produces the file main.extra.aux
2. bibtex main.extra call issued by LaTeX Builder
3. BibTeX succeeds and processing finishes

Gist demonstrating the behavior

Contributor Checklist:

• [x] I have created a new test or updated the unit tests to cover the new/changed functionality.
• [x] I have updated CHANGES.txt and RELEASE.txt (and read the README.rst).
• [x] I have updated the appropriate documentation

[bdbaddog]

And this works if the file is named main_extra.tex right?

(I updated your test to use our new standard test header (you can see it in template/test.py)

[kprussing]

It does. The underscore isn't stripped by the .splitext in the original. It only happened when more that one . was in the file name (functionally, splitext is called multiple times instead of once).

(I saw the change. I was just lazy and copied one of the other TEX tests and made the logic fit. Now I know to look at the template for next time)

[bdbaddog]

This is not really a latex/bibtex issue at it's root, it's really about how SCons.Util.splitext treats a filename with two (or more) .'s in it. Which honestly has been a long term issue, and there have been many point fixes (like this one). Well combined with adding the builder suffix to a file which already apparently (though not in reality) has explicitly specified a suffix..

(this is the root of my delay in merging this PR BTW. I wanted to think a bit about this, and if perhaps we could resolve this by improving the SCons.Util.splitext logic instead. so please bear with me)

[kprussing]

No worries. It's your project and I'm just a visitor!

From what I can tell, Scons.Util.splitext behaves just like os.splitext and only splits off the last extension. The problem here was it (effectively) being called twice on the same file name (I don't know how TARGET.filebase is implemented under the hood).

The alternate explanation here was the wrong target and source were being passed to the Action. That situation would need to be point resolve no matter what.

[mwichmann]

Don't see how improving splitext helps the problem of being called twice, each time one chunk is hacked off?

[mwichmann]

From what I can tell, Scons.Util.splitext behaves just like os.splitext and only splits off the last extension.

It does a little less work is all to be just slightly faster... mainly, it doesn't account for bytes vs strings or for pathlib objects - the scons one just assumes always a plain string.

[bdbaddog]

I think if we just make this change, it will fix the issue

diff --git a/SCons/Tool/tex.py b/SCons/Tool/tex.py
index 6d366334a..cfabe4416 100644
--- a/SCons/Tool/tex.py
+++ b/SCons/Tool/tex.py
@@ -959,11 +959,11 @@ def generate_common(env) -> None:

     env['BIBTEX']      = 'bibtex'
     env['BIBTEXFLAGS'] = SCons.Util.CLVar('')
-    env['BIBTEXCOM']   = CDCOM + '${TARGET.dir} && $BIBTEX $BIBTEXFLAGS ${SOURCE.filebase}'
+    env['BIBTEXCOM']   = CDCOM + '${TARGET.dir} && $BIBTEX $BIBTEXFLAGS ${SOURCE.file}'

     env['BIBER']      = 'biber'
     env['BIBERFLAGS'] = SCons.Util.CLVar('')
-    env['BIBERCOM']   = CDCOM + '${TARGET.dir} && $BIBER $BIBERFLAGS ${SOURCE.filebase}'
+    env['BIBERCOM']   = CDCOM + '${TARGET.dir} && $BIBER $BIBERFLAGS ${SOURCE.file}'

     env['MAKEINDEX']      = 'makeindex'
     env['MAKEINDEXFLAGS'] = SCons.Util.CLVar('')

Thoughts? basically the code is stripping the suffix, and then the command was stripping the suffix, which for this file isn't the suffix it's the filename to be passed to bibtex?

[kprussing]

I'm not opposed. It's definitely the simplest change that fixes the problem. That is what I did to a local copy to get something rolling before I could look into the issue on more detail. After research, I thought the more "correct" solution would be to fix it at the source (the mismatched target/source objects).

[bdbaddog]

@kprussing I'm no latex/bibtex expert. From looking at some docs it looks like bibtex should be passed main.extra (and not main.extra.bib) ? And then it'll figure out the file extensions using that base? Is that right? (Does that apply to all the latex tools?)

[bdbaddog]

@kprussing - Does/would this same issue happen if makeindex get's run as well? (With a file named main.extra.tex)

[kprussing]

Bibtex looks for the stem plus the .aux file so the correct command line argument would be bibtex main.extra[.aux] (where .aux is optional). I am by no means fully proficient with all of the LaTeX tools, but looking for an assumed extension is common especially the .aux file.

I am less familiar with the command line for makeindex, but a look over the document says it wants an .idx file so it would be makeindex main.extra.idx (and generates a .ind on exit).