whipper
whipper copied to clipboard
whipper-team
Finding offset fails for one drive "file size 0 did not match expected size"
I have been setting up Whipper on all the machines I use. All Fedora 27, and 2 of the 3 machines everything works fine. On one machine with a drive:
HL-DT-ST Model: DVD+-RW GHA2N Rev: A103
Finding the offsets fails with:
% whipper offset find -o 667
/usr/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py:46: DeprecationWarning: OpenSSL.rand is deprecated - you should use os.urandom instead
import OpenSSL.SSL
Checking device /dev/sr0
Trying read offset 667 ...
WARNING:whipper.program.cdparanoia:file size 0 did not match expected size 52209740
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
WARNING: cannot rip with offset 667...
No matching offset found.
Consider trying again with a different disc.
I know the offset is 667 for this drive from the database and strangely Morituri can correctly pickup the offset:
% rip offset find -o 667
Checking device /dev/cdrom
Trying read offset 667 ...
** Message: pygobject_register_sinkfunc is deprecated (GstObject)
Offset of device is likely 667, confirming ...
Read offset of device is: 667. Adding read offset to configuration file.
If I set this as an offset in the config file ripping fails with this error too.
Drive PIONEER BDC-207D on Fedora 27 and whipper-0.6.0-6.fc27.x86_64
$ whipper drive analyze
/usr/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py:46: DeprecationWarning: OpenSSL.rand is deprecated - you should use os.urandom instead
import OpenSSL.SSL
cdparanoia can defeat the audio cache on this drive.
Adding drive cache behaviour to configuration file.
$ whipper offset find -o +667
/usr/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py:46: DeprecationWarning: OpenSSL.rand is deprecated - you should use os.urandom instead
import OpenSSL.SSL
Checking device /dev/sr0
Trying read offset 667 ...
WARNING:whipper.program.cdparanoia:file size 0 did not match expected size 53119964
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
WARNING: cannot rip with offset 667...
No matching offset found.
Consider trying again with a different disc.
I tried also to not to put + before 667
$ whipper offset find -o 667
/usr/lib/python2.7/site-packages/urllib3/contrib/pyopenssl.py:46: DeprecationWarning: OpenSSL.rand is deprecated - you should use os.urandom instead
import OpenSSL.SSL
Checking device /dev/sr0
Trying read offset 667 ...
WARNING:whipper.program.cdparanoia:file size 0 did not match expected size 53119964
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
WARNING: cannot rip with offset 667...
No matching offset found.
Consider trying again with a different disc.
I have a similar problem with the BH10LS38 drive. According to the DB it should be +667.
But whipper always ends in an error:
whipper offset find -o +667
Checking device /dev/sr0
Trying read offset 667 ...
WARNING:whipper.program.cdparanoia:file size 0 did not match expected size 27217388
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
WARNING: cannot rip with offset 667...
No matching offset found.
Consider trying again with a different disc.
I can confirm both my TSSTcorp DVDWBD SH-B123L and a PLDS DVD+/-RW DH-16AES encounter the same error.
A workaround I found was booting a kernel version earlier than 4.15. In Fedora 4.13.9-300 I am able to rip with no errors.
If you run the cd-paranoia command provided in the debug log* that supposedly returns a 0 length file manually, is the file still 0? Try replacing the 1[00:00:00.00]-1[...] with 1 if the command fails, and report back at least the lines containing the track offsets in HH:MM:SS.FF notation.
for example (from a cd that happens to be in my drive):
Ripping from sector 0 (track 1 [0:00.00])
to sector 22364 (track 1 [4:58.14])
does the "to sector ..." track 1 match the offset that whipper is trying?
(*) DEBUG:whipper.program.cdparanoia:Running cd-paranoia --stderr-progress --sample-offset=667 --force-cdrom-device /dev/sr0 1[00:00:00.00]-1[00:04:55.72] /tmp/tmpjLoljc.track01.offset667.whipper.wav in @simpz 's case.
Tested a disc that refuses to rip no matter what I try. Received the same error.
"WARNING:whipper.program.cdparanoia:non-integral amount of frames difference"
"WARNING:whipper.program.cdparanoia:file size 0 did not match expected size 42444236"
The debug file in /tmp/tmp3CF3Lo.whipper.wav is a 0 byte file. Tested another disc which refuses to rip and its debug file in /tmp/tmphrk67w.whipper.wav is also an empty 0 byte file.
Can you explain what "The debug file in /tmp/..." means-- I can't tell if you're referring to log messages that say "file size 0 did not match ...", the /tmp file as ripped by whipper, or files you ripped with cd-paranoia by hand that you've named the same as the files whipper tried.
If you rip the file with no offset passed to cd-paranoia, does it still return an empty file?
Sorry for being vague in my last response. I think I was describing the output files themselves being created as 0 byte files. This is the output I receive if I run whipper in Debug with the offset of my drive forced (6).
DEBUG:whipper.program.cdparanoia:Ripping from 0 to 18045 (inclusive)
DEBUG:whipper.program.cdparanoia:Starting at track 1, offset 0
DEBUG:whipper.program.cdparanoia:Stopping at track 1, offset 18045
DEBUG:whipper.program.cdparanoia:Running cd-paranoia --stderr-progress --sample-offset=6 --force-cdrom-device /dev/sr0 1[00:00:00.00]-1[00:04:00.45] /tmp/tmp9VJMBH.whipper.wav
WARNING:whipper.program.cdparanoia:file size 0 did not match expected size 42444236
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
DEBUG:whipper.program.cdparanoia:getTrackQuality: frames 18046, reads 0
DEBUG:whipper.program.cdparanoia:stop: exception FileSizeError(u'/tmp/tmp9VJMBH.whipper.wav', 'File size 0 did not match expected size 42444236')
DEBUG:whipper.command.cd:Got exception TaskException(FileSizeError(u'/tmp/tmp9VJMBH.whipper.wav', 'File size 0 did not match expected size 42444236'), 'exception %(exc)s at %(filename)s:%(line)s: ') on try 1
I next ran a wrong command to test your suggestion. The results may be useful.
I run cdparanoia manually it does rip the disc to .wav files with no problem. Here is output it gives for track 1.
cdparanoia III release 10.2 (September 11, 2008)
Ripping from sector 0 (track 1 [0:00.00])
to sector 251591 (track 12 [8:46.73])
outputting to track01.cdda.wav
(== PROGRESS == [ | 018045 00 ] == :^D * ==)
If I run cd-paranoia I get an error.
cdparanoia III release 10.2 libcdio 0.94
(C) 2001 Monty <[email protected]> and Xiphophorus
(C) 2004, 2005, 2008 Rocky Bernstein <[email protected]>
(C) 2014 Robert Kausch <[email protected]>
Report bugs to [email protected]
++ WARN: error in ioctl CDROMREADTOCHDR: No medium found
Selected span contains non audio track at track 13. Aborting.
I am hoping this information is helpful in solving this issue.
Thank you. If you could please use triple-backticks to quote large sections of console text it would make your reports easier to read.
Can you also please post the full command line you used with cdparanoia and cd-paranoia to rip the discs? I'm not sure if you ran them with a specific (correct? incorrect?) offset, or no offset at all.
It'd also be helpful to paste the musicbrainz URL for the CD in question.
Interesting that this is looking to be a divergence in behaviour between cdparanoia and libcdio's cd-paranoia.
If I run cd-paranoia with forced offset I get this:
$ cd-paranoia -B -d /dev/sr0 -o 6
cdparanoia III release 10.2 libcdio 0.94
(C) 2001 Monty <[email protected]> and Xiphophorus
(C) 2004, 2005, 2008 Rocky Bernstein <[email protected]>
(C) 2014 Robert Kausch <[email protected]>
Report bugs to [email protected]
Forcing search overlap to 6 sectors; ignoring autosense
Selected span contains non audio track at track 13. Aborting.
If I run cdparanoid with same forced offset I get this:
cdparanoia -B -d /dev/sr0 -o 6
cdparanoia III release 10.2 (September 11, 2008)
Forcing search overlap to 6 sectors; ignoring autosense
Ripping from sector 0 (track 1 [0:00.00])
to sector 251591 (track 12 [8:46.73])
outputting to track01.cdda.wav
(== PROGRESS == [ | 018045 00 ] == :^D * ==)
The musicbrainz URL is: https://musicbrainz.org/cdtoc/attach?toc=1+12+251742+150+18196+35827+49678+67595+94386+115802+132031+153418+174992+196843+212218&tracks=12&id=DdgxjFT7gVEdHxKuxi5SdDK9kJg-
I am also going to attach the whole debug log file to this posting.
Sorry not been repsonding quickly, the machine with this issue wasn't nearby.
But stealing the more cd-paranoia command from the log file:
DEBUG:whipper.program.cdparanoia:Running cd-paranoia --stderr-progress --sample-offset=667 --force-cdrom-device /dev/sr0 1[00:00:00.00]-1[00:04:55.72] /tmp/tmpMaZjhE.track01.offset667.whipper.wav
WARNING:whipper.program.cdparanoia:file size 0 did not match expected size 52209740
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
DEBUG:whipper.program.cdparanoia:getTrackQuality: frames 22198, reads 0
I get the following output:
% grep cd-paranoia whipper.log
DEBUG:whipper.program.cdparanoia:Running cd-paranoia --stderr-progress --sample-offset=667 --force-cdrom-device /dev/sr0 1[00:00:00.00]-1[00:04:55.72] /tmp/tmpMaZjhE.track01.offset667.whipper.wav
% cd-paranoia --stderr-progress --sample-offset=667 --force-cdrom-device /dev/sr0 '1[00:00:00.00]-1[00:04:55.72]' /tmp/tmpMaZjhE.track01.offset667.whipper.wav
Sending all callback output to stderr for wrapper script
cdparanoia III release 10.2 libcdio 0.94
(C) 2001 Monty <[email protected]> and Xiphophorus
(C) 2004, 2005, 2008 Rocky Bernstein <[email protected]>
(C) 2014 Robert Kausch <[email protected]>
Report bugs to [email protected]
Time/sector offset goes beyond end of specified track.
The generated wav file is zero length.
@simpz: Could you try adding --force-overread to the arguments of the cd-paranoia command you executed? I think this issue might be related to the fact that your offset is quite large. (You need to be on cd-paranoia version 10.2+0.94+2 for this argument to be added.)
I think @rry1983's problem is a different problem from the others in this issue, not sure what the problem is there (especially because changing kernel versions fixed it), but it's probably related to the fact that cd-paranoia uses libcdio to communicate with the drive, while cdparanoia implemented the communication itself, AFAIK.
@rry1983: This probably won't matter, but you didn't force offset on the commands you executed. You used -o which forces the overlap search window size. To force the offset you need to use -O. (This is true for both cdparanoia and cd-paranoia.)
Maybe my cd-paranoia isn't new enough (even though this is the one shipped in the Fedora 27 repo which is usually pretty bleeding edge):
% cd-paranoia -V
cdparanoia III release 10.2 libcdio 0.94
(C) 2001 Monty <[email protected]> and Xiphophorus
(C) 2004, 2005, 2008 Rocky Bernstein <[email protected]>
(C) 2014 Robert Kausch <[email protected]>
Report bugs to [email protected]
As I get
cd-paranoia: unrecognized option '--force-overread'
As far as I can tell, Fedora 27 branched off from rawhide on the 15th of august last year. cd-paranoia 10.2+0.94+2 was released on the 23rd of august last year, so I guess it makes sense that that version is not in Fedora 27.
The maintainer of libcdio didn’t realize cd-paranoia had been updated or he would have pushed it to f27. As it stands he built the update with cdio 2.0 so he doesn’t want to push anything to f27 to prevent breakage. 10.2+0.94+2 is coming with F28 in May. I would test it but right now the F28 branch is hosed completely. I am curious if the version of cd-paranoia in f27 is causing a lot of these issues. I'm seeing some offset issues myself that I can't seem to track down, just haven't submitted an issue yet.
@mruszczyk we could help him in making the needed updates in F27, since F27 is a supported version and bugs should be fixed
@Germano0 I spoke with him via email. He grouped the update in with his 2.0 update builds and didn't want to push it to f27. I didn't really know what to do past that point. At the time 0.5.1 was working with xiph cdparanoia so I figured the swap to cd-paranoia would be fine but it looks like it's causing issues. Let me see if I can get a spare pc up to test 10.2+0.94+2 using dnf system upgrade. Anaconda hasn't worked for rawhide for a month or so.
EDIT: I am seeing this same behavior when attempting to offset find large numbers with libcdio 2.0.0 and the latest cd-paranoia.
@a10footsquirrel Sorry for not mentioning it earlier. With an older kernel 2 of the 6 discs I have are able to be ripped. The other 4 still refuse to allow me to rip with the same file size 0 error no matter which of the 3 kernels I try from my grub menu.
I have the same issue as @simpz with a new drive with an offset of 667. I rebuilt latest libcdio and libcdio-paranoia from git and added --force-overread, but it didn't help:
$ cd-paranoia --force-overread --stderr-progress --sample-offset=667 --force-cdrom-device /dev/sr0 1[00:00:00.00]-1[00:04:50.12] /tmp/tmpYsAlCI.track01.offset667.whipper.wav
Sending all callback output to stderr for wrapper script
cdparanoia III release 10.2 libcdio 2.0.0 x86_64-pc-linux-gnu
(C) 2001 Monty <[email protected]> and Xiphophorus
(C) 2004, 2005, 2008 Rocky Bernstein <[email protected]>
(C) 2014 Robert Kausch <[email protected]>
Report bugs to [email protected]
Time/sector offset goes beyond end of specified track.
It works for me now. I patched cd-paranoia to just go ahead when this condition occurs.
diff --git a/src/cd-paranoia.c b/src/cd-paranoia.c
index 9443b62..68274db 100644
--- a/src/cd-paranoia.c
+++ b/src/cd-paranoia.c
@@ -227,7 +227,6 @@ parse_offset(cdrom_drive_t *d, char *offset, int begin)
if( i_track != CDIO_INVALID_TRACK ){
if (cdda_sector_gettrack(d,ret) != i_track) {
report("Time/sector offset goes beyond end of specified track.");
- exit(1);
}
}
@@ -1197,7 +1196,6 @@ main(int argc,char *argv[])
for( i=track1; i<=track2; i++ )
if(i != 0 && !cdda_track_audiop(d,i)){
report("Selected span contains non audio track at track %02d. Aborting.\n\n", i);
- exit(1);
}
report("Ripping from sector %7ld (track %2d [%d:%02d.%02d])\n"
The first hunk was needed to rip any track. The second hunk was needed to rip the last track. Furthermore, I needed to run whipper with option -x to enable --force-overread for cd-paranoia, in order to rip the last two tracks. The disc I used was: https://musicbrainz.org/release/ab5d678d-361f-3d5a-8f18-7736a89eb88d
“I can confirm this patch helps. Thanks!” Turns out that I spoke too soon. The last track still can't be ripped.
@ooglyhLL: Sorry, I forgot to update this patch after I had removed another line locally:
diff --git a/src/cd-paranoia.c b/src/cd-paranoia.c
index 9443b62..fbcf55d 100644
--- a/src/cd-paranoia.c
+++ b/src/cd-paranoia.c
@@ -227,7 +227,6 @@ parse_offset(cdrom_drive_t *d, char *offset, int begin)
if( i_track != CDIO_INVALID_TRACK ){
if (cdda_sector_gettrack(d,ret) != i_track) {
report("Time/sector offset goes beyond end of specified track.");
- exit(1);
}
}
@@ -235,7 +234,6 @@ parse_offset(cdrom_drive_t *d, char *offset, int begin)
if( ret>cdda_disc_lastsector(d) ) {
report("Time/sector offset goes beyond end of disc.");
- exit(1);
}
return(ret);
@@ -1197,7 +1195,6 @@ main(int argc,char *argv[])
for( i=track1; i<=track2; i++ )
if(i != 0 && !cdda_track_audiop(d,i)){
report("Selected span contains non audio track at track %02d. Aborting.\n\n", i);
- exit(1);
}
report("Ripping from sector %7ld (track %2d [%d:%02d.%02d])\n"
@RecursiveForest I ran into the same issue and followed what you said. The result is quite weird.
$ cd-paranoia --stderr-progress --sample-offset=667 --force-cdrom-device /dev/sr0 1[00:00:00.00]-1[00:03:13.44] /tmp/tmpoEuyVd.track01.offset667.whipper.wav
Sending all callback output to stderr for wrapper script
cdparanoia III release 10.2 libcdio 2.0.0 x86_64-pc-linux-gnu
(C) 2001 Monty <[email protected]> and Xiphophorus
(C) 2004, 2005, 2008 Rocky Bernstein <[email protected]>
(C) 2014 Robert Kausch <[email protected]>
Report bugs to [email protected]
Time/sector offset goes beyond end of specified track.
And here it is... working?!
$ cd-paranoia --stderr-progress --sample-offset=667 --force-cdrom-device /dev/sr0 1 /tmp/tmpoEuyVd.track01.offset667.whipper.wav
Sending all callback output to stderr for wrapper script
cdparanoia III release 10.2 libcdio 2.0.0 x86_64-pc-linux-gnu
(C) 2001 Monty <[email protected]> and Xiphophorus
(C) 2004, 2005, 2008 Rocky Bernstein <[email protected]>
(C) 2014 Robert Kausch <[email protected]>
Report bugs to [email protected]
Ripping from sector 1 (track 1 [0:00.00])
to sector 14520 (track 2 [0:00.-1])
outputting to /tmp/tmpoEuyVd.track01.offset667.whipper.wav
##: 0 [read] @ 21168
(== PROGRESS == [> | ...... 00 ] == :^D . ==) ##: 0 [read] @ 50568
(== PROGRESS == [> | ...... 00 ] == :^D o ==) ##: 0 [read] @ 79968
(== PROGRESS == [> | ...... 00 ] == :^D 0 ==) ##: 0 [read] @ 109368
##: 0 [read] @ 138768
(== PROGRESS == [> | ...... 00 ] == :^D O ==) ##: 0 [read] @ 168168
(== PROGRESS == [> | ...... 00 ] == :^D 0 ==) ##: 0 [read] @ 197568
(== PROGRESS == [> | ...... 00 ] == :^D o ==) ##: 0 [read] @ 226968
(== PROGRESS == [> | ...... 00 ] == :^D . ==) ##: 0 [read] @ 256368
...
The track ripped as a wav in the last command is perfect as far as I can see.
--force-overread makes no difference at all in the output, but I hope I brought some light to this issue.
Since the issue is found in GNU's implementation of cd-paranoia which uses libcdio, another "fix" is using Xiph's cdparanoia instead of cd-paranoia like the original morituri (https://github.com/thomasvs/morituri/blob/135b2f7bf27721177e3aeb1d26403f1b29116599/morituri/program/cdparanoia.py#L281-L282) (which is why it appears the work):
https://github.com/JoeLametta/whipper/blob/542e07144385638e5813148c25cf2d5e9e772412/whipper/program/cdparanoia.py#L286-L287
(by replacing the above and other lines with cdparanoia)
$ whipper offset find -o 667
Checking device /dev/sr0
Trying read offset 667 ...
Offset of device is likely 667, confirming ...
Read offset of device is: 667.
Adding read offset to configuration file.
Can anybody please summarise the actual situation of patches? Concerning third party libraries patch (if there are any): have you filled an upstream pull request?
No available patches for third party libraries, I have no idea of how to solve the problem (the problem is probably one level down at libcdio if I were to guess) and the devs probably don't have a way to reproduce it.
Regarding whipper patches, this would be "fixed" if we could choose cdparanoia's implementation, in this case to use Xiph's instead of GNU's.
@Germano0 Outside of someone submitting a pull request to select cdparanoia implementation I think we're at an impasse. No word on the bug in libcdio for a few weeks and the other patch here just ignores the failure. I think a config option for cdparanoia binary is a worthwhile pursuit but where that sits as a priority I do not know.
_
Regarding whipper patches, this would be "fixed" if we could choose cdparanoia's implementation, in this case to use Xiph's instead of GNU's.
Just to add my bits... I have amended the whipper code and although the 'offset find' now works and finds the offset: Checking device /dev/sr0 Trying read offset 667 ... Offset of device is likely 667, confirming ...
Read offset of device is: 667. Adding read offset to configuration file.
Actually ripping still does not work on the last track...
Ripping track 8 of 9: 08 Crying for Help VIII.flac
CRCs match for track 8
Peak level: 13871
Rip quality: 100.00%
Ripping track 9 of 9: 09 Sirens.flac
WARNING:whipper.program.cdparanoia:file size 144900140 did not match expected size 144967916
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
Ripping track 9 of 9 (try 2): 09 Sirens.flac
WARNING:whipper.program.cdparanoia:file size 144900140 did not match expected size 144967916
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
Ripping track 9 of 9 (try 3): 09 Sirens.flac
WARNING:whipper.program.cdparanoia:file size 144900140 did not match expected size 144967916
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
Ripping track 9 of 9 (try 4): 09 Sirens.flac
WARNING:whipper.program.cdparanoia:file size 144900140 did not match expected size 144967916
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
Ripping track 9 of 9 (try 5): 09 Sirens.flac
WARNING:whipper.program.cdparanoia:file size 144900140 did not match expected size 144967916
WARNING:whipper.program.cdparanoia:non-integral amount of frames difference
CRITICAL:whipper.command.cd:Giving up on track 9 after 5 times
track can't be ripped. Rip attempts number is equal to 'MAX_TRIES'