Vib-Ribbon (SCES-02873) - Checksum doesn't match Redump

[Rot-gut]

Firstly, I greatly appreciate the effort to improve this tool to recognize multi-track bins, as I thought it was abandoned.

While it's understood that the multi-track bins will have mismatched checksums due to re-conversion of audio, I'm still getting a main data track mismatch from Redump for Vib-Ribbon.

Redump Data track bin MD5: e116bccbc0828e45d7b3f19234cd3670
http://redump.org/disc/3474/

My DATA TRACK.BIN result MD5: 2875C53260C6DBD598A97C89423ED512

C:\RipAgain\SCES02873>"C:\psxtract-2\psxtract.exe" "C:\RipAgain\SCES02873\EBOOT.PBP" "C:\RipAgain\SCES02873\DOCUMENT.DAT" "C:\RipAgain\SCES02873\KEYS.BIN"
Decrypting DOCUMENT.DAT...
PGD: Invalid 0x80 MAC hash!
DOCUMENT.DAT successfully decrypted! Saving as DOCUMENT_DEC.DAT...

Using PGD key: redacted

Unpacking PBP C:\RipAgain\SCES02873\EBOOT.PBP...
[0] 944 bytes | PARAM.SFO
[1] 11541 bytes | ICON0.PNG
[2] 0 bytes | ICON1.PMF
[3] 38089 bytes | PIC0.PNG
[4] 2651 bytes | PIC1.PNG
[5] 0 bytes | SND0.AT3
[6] 12271 bytes | DATA.PSP
[7] 25073598 bytes | DATA.PSAR
Successfully unpacked C:\RipAgain\SCES02873\EBOOT.PBP!

Single disc game detected!

Found STARTDAT offset: 0x017e6350
Saving STARTDAT as STARTDAT.BIN...

Decrypting ISO header...
ISO header successfully decrypted! Saving as ISO_HEADER_0.BIN...

ISO disc: SCES_02873
ISO title: Vib-Ribbon

Found special data offset: 0x017e8bee
Decrypting special data...
Special data successfully decrypted! Saving as SPECIAL_DATA.BIN...

Found unknown data offset: 0x017e6240
Decrypting unknown data...
Unknown data successfully decrypted! Saving as UNKNOWN_DATA.BIN...

Offset 1m:40s:0f at 0428
Offset 4m:19s:0f at 0432
seeking to 100000 + 536dc0 (636dc0)
Extracting audio track 2 (11925 sectors, 2630784 bytes)
Offset 4m:19s:0f at 0432
Offset 6m:53s:0f at 043c
seeking to 100000 + 7b90c0 (8b90c0)
Extracting audio track 3 (11550 sectors, 2548224 bytes)
Offset 6m:53s:0f at 043c
Offset 9m:25s:0f at 0446
seeking to 100000 + a27140 (b27140)
Extracting audio track 4 (11400 sectors, 2515200 bytes)
Offset 9m:25s:0f at 0446
Offset 12m:0s:0f at 0450
seeking to 100000 + c8d0c0 (d8d0c0)
Extracting audio track 5 (11625 sectors, 2564736 bytes)
Offset 12m:0s:0f at 0450
Offset 14m:47s:0f at 045a
seeking to 100000 + eff1c0 (fff1c0)
Extracting audio track 6 (12525 sectors, 2763264 bytes)
Offset 14m:47s:0f at 045a
Offset 17m:20s:0f at 0464
seeking to 100000 + 11a1a40 (12a1a40)
Extracting audio track 7 (11475 sectors, 2531712 bytes)
Offset 17m:20s:0f at 0464
Offset 20m:22s:66f at 0414
seeking to 100000 + 140ba40 (150ba40)
Extracting audio track 8 (13716 sectors, 2992512 bytes)
7 audio tracks extracted to ATRAC3
Attempting to convert from ATRAC3 to WAV, this may take awhile...

C:\psxtract-2\at3tool.exe -d "D01 TRACK02.AT3" "D01 TRACK02.WAV"
Decoding 132 kbps (ATRAC3)
Decoded Bytes = 2630400 Bytes@6850frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK03.AT3" "D01 TRACK03.WAV"
Decoding 132 kbps (ATRAC3)
Decoded Bytes = 2547840 Bytes@6635frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK04.AT3" "D01 TRACK04.WAV"
Decoding 132 kbps (ATRAC3)
Decoded Bytes = 2514816 Bytes@6549frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK05.AT3" "D01 TRACK05.WAV"
Decoding 132 kbps (ATRAC3)
Decoded Bytes = 2564352 Bytes@6678frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK06.AT3" "D01 TRACK06.WAV"
Decoding 132 kbps (ATRAC3)
Decoded Bytes = 2762880 Bytes@7195frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK07.AT3" "D01 TRACK07.WAV"
Decoding 132 kbps (ATRAC3)
Decoded Bytes = 2531328 Bytes@6592frames(ave=384bytes)

C:\psxtract-2\at3tool.exe -d "D01 TRACK08.AT3" "D01 TRACK08.WAV"
Decoding 132 kbps (ATRAC3)
Decoded Bytes = 2992128 Bytes@7792frames(ave=384bytes)

7 audio tracks converted to WAV
Need to pad 348372 bytes
7 audio tracks converted to BIN

Building the data track...
ISO offset 100000
.....
Read 536dc0 bytes, wrote 10d4f00 bytes
Data track successfully reconstructed! Saving as ISO.BIN...

Offset 0m:0s:0f at 041e
Offset 1m:40s:0f at 0428
Converting the final image to BIN/CUE...
Patching ECC/EDC data...
Processing 7350 sectors
Encountered unknown mode! This is probably not a proper image.

Generating CUE file...
adding DATA TRACK.BIN.ISO
adding D01 TRACK02.BIN
adding D01 TRACK03.BIN
adding D01 TRACK04.BIN
adding D01 TRACK05.BIN
adding D01 TRACK06.BIN
adding D01 TRACK07.BIN
adding D01 TRACK08.BIN
Disc successfully converted to BIN/CUE format!

[has207]

Looking into it...

[has207]

Okay this is interesting, there's a few failures here. The entry in redump you want to be looking at is actually http://redump.org/disc/17774/, the one you were linking is the JP version. And the track file is actually TEMP/DATA_TRACK.BIN.ISO not TEMP/DATA_TRACK.BIN -- that's the initially dumped track before it's fixed, the .ISO file is the finalized track1 as it gets written to the BIN. So if we look at the md5 hashes for the correct files they do match up, BUT the resulting .BIN file is somehow borked, at least it seems to be the wrong size and Duckstation isn't able to properly md5 the tracks, let me investigate more...

[has207]

Okay I think I see the issue, this game uses some very unusual pregap values, generally every other game I've seen uses 2 seconds gaps between tracks but this one just kind of goes nuts with using random pregaps, and the psxtract code has some hardcoded expectations that are causing the final .BIN file to be pretty broken. Let me see if it's going to be possible to infer these values on the fly and do the right thing, but as kind of a workaround what you can do is grab the CUE file from redump, and then take individual BIN files from the TEMP directory (remembering to use DATA_TRACK.BIN.ISO for track 1) and combine them manually. I should have some kind of fix in psxtract for this hopefully soon though

[has207]

Well this keeps getting more and more interesting, seems that Sony also assumed 2 second gaps when making the EBOOT so their track listing is actually wrong and doesn't account for the actual pregaps in the audio. This makes me think this EBOOT might actually be a bit off in terms of when the audio starts playing in the game vs when it should start playing. On the other hand the 11 second pregap on Track2 on disc just seems weird as well, I don't know why there should be such a long silence on that track, though I'm not familiar with the game and don't know when it actually plays. In any case, it doesn't look like it's possible to reproduce the CUE file exactly, but it should be possible to fix the issue with data track length that causes the MD5 mismatch (and likely breaks the resulting image).

[has207]

I pushed some changes that introduce some initial support for this, it's not working correctly yet though. I've also noticed previous audio handling code might have had some issues too so need to do more tweaking and testing. Unfortunately it seems that Sony might have messed up with this game as the CUE entries they have in the EBOOT do not account for the variable pregaps for the audio tracks so my guess is they will play a bit wrong, and I've also discovered there might be some similar inconsistencies in other games as well. It's not fatal as the music starting a few frames sooner or later is probably not a big deal but it makes for a challenge reconstructing the audio layout and especially verifying it across a large number of games and tracks.

[has207]

Initial fix is committed, and I've compared PSOne releases with games that have unusual pregap values based on redump cue data. Here's what's likely to be the full list of currently affected titles:

• '99 Koushien (JP)
• A. IV Evolution Global (EU)
• Bowling (US)
• Centipede (EU)
• Centipede (US)
• Chou Aniki Kyuukyoku... (JP)
• Dai-4-Ji Super Robot Taisen S (JP)
• Grand Theft Auto (JP)
• Hanabi: Fantast (JP)
• Jet Copter X (JP)
• King of Fighters '96 (JP)
• King of Fighters Kyo (JP)
• Koushien V (JP)
• Motteke Tamago with Ganbare (JP)
• Nyan to Wonderful (JP)
• Perfect Weapon (US/EU)
• Slam Dragon (JP)
• Touge Max Saisoku Drift Master (JP)
• Tsuukai!! Slot Shooting (JP)
• Vib-Ribbon (EU/JP)
• Yamasa Digi Guide New Pulsar R (JP)
• Yamasa Digi Guide M771 (JP)
• Yamasa Digi Guide Hyper Rush (JP)

It will take a bit of time to test, as I think there may be a few titles in there that will reveal additional problems. Will make a new release after checking them.

[has207]

I have a fix close to working, I had to basically hard code pregap values for the affected games as there is no way to reconstruct them on the fly. Found some other issues while I was in there as well, so next release should hopefully be a big improvement across the board. Testing any changes is a real challenge with this so it's been taking some time to get it right.

[mr-blackmore]

I also have a list of games, which failed to verify checksum with Redump:
Dezaemon Plus (SLPS-00335)
Disney-Pixar Toy Story Racer (SLES-03396)
Disneys Atlantis - Det Forsvundne Rige (SCES-03534)
Galaxy Fight - Universal Warriors (SLPS-00138)
Kaisoku Tenshi - The Rapid Angel (SLPS-01553)
Mickey's Wild Adventure (SCES-00163)
Oddworld - Abe's Exoddus (Disc 2) (SLES-11480)
Resident Evil - Director's Cut (SLES-00969)
Street Fighter Alpha - Warriors' Dreams (SLES-00199)
Street Skater (SLES-01759)
Tekken (SCES-00005)
Tekken 2 (SLUS-00213)
Twisted Metal (SCES-00061)

There's also Driver (SLES-01816), but I suspect that has more to do with Ubisoft, as they bought the series in 2006, so they actually made small changes, like replacing any mentions of GT Interactive with themselves as publisher.

[has207]

@mr-blackmore thanks for the list, I'll check them out

[malvarenga123]

Hi @has207, are you sure you can't read the files to find out what the actual pre-gap is? If I'm not mistaken, it should be filled with zeroes and the start of audio data should have some distinct header.

[has207]

I also have a list of games, which failed to verify checksum with Redump: Dezaemon Plus (SLPS-00335) Disney-Pixar Toy Story Racer (SLES-03396) Disneys Atlantis - Det Forsvundne Rige (SCES-03534) Galaxy Fight - Universal Warriors (SLPS-00138) Kaisoku Tenshi - The Rapid Angel (SLPS-01553) Mickey's Wild Adventure (SCES-00163) Oddworld - Abe's Exoddus (Disc 2) (SLES-11480) Resident Evil - Director's Cut (SLES-00969) Street Fighter Alpha - Warriors' Dreams (SLES-00199) Street Skater (SLES-01759) Tekken (SCES-00005) Tekken 2 (SLUS-00213) Twisted Metal (SCES-00061)

There's also Driver (SLES-01816), but I suspect that has more to do with Ubisoft, as they bought the series in 2006, so they actually made small changes, like replacing any mentions of GT Interactive with themselves as publisher.

keeping track of this in #3

[has207]

Latest commit finally implements what should be as accurate an audio track generation as we're likely to get, using redump's cue files and disc sizes to ensure the audio tracks we generate match as closely as possible. I have a few more changes in mind before cutting a new release but at least this issue should finally be resolved (barring some more bugs of course, the code is pretty messy...)

[has207]

Hi @has207, are you sure you can't read the files to find out what the actual pre-gap is? If I'm not mistaken, it should be filled with zeroes and the start of audio data should have some distinct header.

yes that's true for the cdrom version, what we have in the eboot isn't raw pcm audio though but an encoded version and the pregap info is essentially lost. we reconstruct it through heuristics but it's quite tricky to get right so I've gone with the drastic approach of relying on redump cue files to tell us what the pregap should be. as it turns out psone classic versions of the games are not necessarily built off the same revisions as retail discs so even this can fail but it tends to be most reliable method if combined with the aforementioned heuristics. I'm still ironing it out for release but hopefully will have a shiny new version soon.