retool icon indicating copy to clipboard operation
retool copied to clipboard
verified publisher icon unexpectedpanda

GBA Parent/Clone mixed with compilations issues

Open katarak opened this issue 5 years ago • 5 comments

I feel like most of this will be a bug, but perhaps there are also some issues in the clone list. On GBA many of the compilations are in the dat, despite

  1. Games being available outside of the compilation
  2. Having a compilation in "English"

I generated a dat with (-bcdfrs) Nintendo - Game Boy Advance (1,696) (20201229-143743) [1G1R] (-bcdfrs) (Retool 2021-01-09 11-04-33).txt

For example. 2 Games in 1 - Monsters, Inc. + Finding Nemo (USA) is excluded from the dat because both of those games are available as single titles. However all other games listed as clones are in the dat, probably because the "english" was excluded.

    "2 Games in 1 - Monsters, Inc. + Finding Nemo": [
        ["2 Games in 1 - Die Monster AG + Findet Nemo", 1],
        ["2 Games in 1 - Monsters & Co. + Alla Ricerca di Nemo", 1],
        ["2 Games in 1 - Monsters en Co. + Finding Nemo", 1],
        ["2 Games in 1 - Monstres & Cie + Le Monde de Nemo", 1],
        ["2 Games in 1 - Monstruos, S.A. + Buscando a Nemo", 1]
    ],

2 Games in 1 - Finding Nemo - The Continuing Adventures + The Incredibles (USA) is not listed in the compilation sections. Also the single title is "Incredibles, The (USA)" if that could also cause issues.

2 Games in 1 - Golden Nugget Casino + Texas Hold 'em Poker (USA) is not listed in the compilation sections either, but is found in the renames section. Result is that the compilation and the single titles are in the dat.

In the end I think the real bug is that clones are not excluded if the compilation is excluded, and they should. The other examples are probably just clone list edits. Finding the issues, however, is difficult because of the noise created by that first bug.

I'm happy to do a pass on all the compilations that still show up after the bug is fixed.

katarak avatar Jan 09 '21 16:01 katarak

Ah compilations are really quite thorny. This won't make 0.87, but I'll revisit it when it's time for another update.

unexpectedpanda avatar Jan 20 '21 10:01 unexpectedpanda

Okay, finally put aside the time to dig into this. Yep, there are definite problems here that I'd like to solve for 0.88.

You've correctly pointed out that I've missed individual titles where the article has been swapped around in comparison to the compilation. For example, Monster AG, Die (Germany) vs 2 Games in 1 - Die Monster AG + Findet Nemo (Germany).

You're also right that with the -c option, it looks like the group is being broken and the other titles are no longer being associated with each other. That's definitely not desirable behavior. For future resilience when No-Intro and Redump change names, I need to ensure the remaining titles stay tied together if the key title disappears, whether it's a compilation or not. That might take a bit of tinkering, but I think it's important.

As far as compilations are concerned specifically... it's conceptually a mess. I could remove all the titles associated with those in the compilations section of the clone list (or just add them to the compilations list themselves), but I think this might have unintended consequences for different region orders/exclusions. This is because I'm currently following a standard for clone lists that likely doesn't work -- that is, only adding titles to the compilations list if their individual titles are available in the same region.

For example, these compilations are in the GBA dat:

  • 2 Games in 1 - Cartoon Network Block Party + Cartoon Network Speedway (USA)
  • Double Game! - Cartoon Network Block Party & Cartoon Network Speedway (Europe)

But the individual titles are only available in the USA:

  • Cartoon Network Speedway (USA)
  • Cartoon Network Block Party (USA)

Meaning the USA compilation qualifies for the compilations list, the European one doesn't, and the European title is set as a clone of the USA version. If you exclude compilations, the USA compilation gets removed, but the European compilation remains in the dat in addition to the USA individual titles. Not ideal.

Currently I'm thinking the best path forward is to remove the option to exclude compilations, then add the individual titles as clones of the compilations with a priority of 2. This would make the compilation the selected 1G1R title in many cases. I'm quite sure there are scenarios where there's a newer version of an individual title available than what was bundled in a compilation... but for the most part I'd imagine we're probably okay. It'll need some testing, and possibly some individualized tweaking.

Taking the above example, the proposed format would look like this as a clone list entry:

"2 Games in 1 - Cartoon Network Block Party + Cartoon Network Speedway": [
    ["Double Game! - Cartoon Network Block Party & Cartoon Network Speedway", 1],
    ["Cartoon Network Block Party", 2],
    ["Cartoon Network Speedway", 2]
]

The USA compilation would then become the 1G1R title if USA is higher, the European compilation the 1G1R title if Europe is higher. This makes sense to me.

But compilations are a pain, and GBA in particular is nasty. Take this particular set of titles:

  • 2 Games in 1 - Die Monster AG + Findet Nemo (Germany)
  • 2 Games in 1 - Monsters & Co. + Alla Ricerca di Nemo (Italy) (En,Fr,It+Es,It)
  • 2 Games in 1 - Monsters en Co. + Finding Nemo (Netherlands) (En,Es,Nl+Fr,Nl)
  • 2 Games in 1 - Monsters, Inc. + Finding Nemo (Europe)
  • 2 Games in 1 - Monsters, Inc. + Finding Nemo (USA)
  • 2 Games in 1 - Monstres & Cie + Le Monde de Nemo (France) (En,Fr,It+Fr,Nl)
  • 2 Games in 1 - Monstruos, S.A. + Buscando a Nemo (Spain) (En,Es,Nl+Es,It)
  • Findet Nemo (Germany) (Beta)
  • Findet Nemo (Germany)
  • Finding Nemo (USA, Europe)
  • Finding Nemo (Europe) (Fr,Nl)
  • Finding Nemo (Japan)
  • Finding Nemo (Europe) (Es,It)
  • Monster AG, Die (Germany)
  • Monsters, Inc. (USA, Europe)
  • Monsters, Inc. (Japan)
  • Monsters, Inc. (Europe) (En,Fr,It)
  • Monsters, Inc. (Europe) (En,Es,Nl)

Seems okay on the surface, until you discover Finding Nemo is actually in multiple compilations, so more titles have to be considered:

  • 2 Games in 1 - Alla Ricerca di Nemo + Gli Incredibili - Una 'Normale' Famiglia di Supereroi (Italy) (Es,It+It)
  • 2 Games in 1 - Buscando a Nemo + Los Increibles (Spain) (Es,It+Es)
  • 2 Games in 1 - Findet Nemo + Die Unglaublichen (Germany)
  • 2 Games in 1 - Findet Nemo + Findet Nemo - Das Abenteuer Geht Weiter (Germany) (De+Fr,De,Nl)
  • 2 Games in 1 - Finding Nemo + Finding Nemo - The Continuing Adventures (Europe) (En+En,Es,It,Sv,Da)
  • 2 Games in 1 - Finding Nemo + Finding Nemo - The Continuing Adventures (Europe) (Es,It+En,Es,It,Sv,Da)
  • 2 Games in 1 - Finding Nemo + Finding Nemo - The Continuing Adventures (Europe) (Fr,Nl+Fr,De,Nl)
  • 2 Games in 1 - Finding Nemo + The Incredibles (Europe)
  • 2 Games in 1 - Finding Nemo + The Incredibles (Europe) (Fr,Nl)
  • 2 Games in 1 - Finding Nemo - The Continuing Adventures + The Incredibles (USA)
  • Finding Nemo - The Continuing Adventures (USA, Europe)
  • Finding Nemo - The Continuing Adventures (Europe) (Fr,De,Nl)
  • Finding Nemo - The Continuing Adventures (Europe) (En,Es,It,Sv,Da)
  • Incredibles, The (USA, Europe)
  • Incredibles, The (Europe) (Fr,Nl)
  • Increibles, Los (Spain)
  • Incredibili, Gli - Una 'Normale' Famiglia di Supereroi (Italy)
  • Mr. Incredible (Japan)
  • Unglaublichen, Die (Germany)

Maybe with a heavily complicated set of conditional overrides it can be made to work. But gosh that's going to take a lot of thinking and testing to pull off cross-region.

For now I'll focus on the key title removal bug, then... we'll see if I can wrap my head around the more complex clone relationships.

unexpectedpanda avatar Feb 13 '21 04:02 unexpectedpanda

The bug side of this is fixed as of 0.88 -- now I just need to sort out a better way to handle compilations.

unexpectedpanda avatar Mar 04 '21 06:03 unexpectedpanda

A remove feature has been added to clone lists, which removes titles entirely from the dat instead of trying to assign them as clones. This happens whenever you run Retool, and effectively works as if the compilations filter was always on. As a result, excluding compilations is no longer a feature.

This should help whittle things down a little to get closer to the 1G1R ideal. I've managed to get around the mind explosion that was SMS compilations across multiple regions... GBA will have to wait for another day when I can put aside ample time.

unexpectedpanda avatar Apr 12 '21 07:04 unexpectedpanda

If it helps, here's my most up-to-date list for GBA. Except for SN Systems (Europe), these are all compilations.

  • 2 Disney Games - Disney Sports - Football + Disney Sports - Skateboarding (Europe) (En,Fr,De,Es,It)
  • 2 Disney Games - Lilo & Stitch 2 + Peter Pan - Return to Neverland (Europe) (En,Fr,De,Es+En,Fr,De,Es,It,Nl)
  • 2 Game Pack! - Hot Wheels - Stunt Track Challenge + Hot Wheels - World Race (USA, Europe)
  • 2 Games in 1 - Bionicle + Knights' Kingdom (Europe) (En,Fr,De,Da+En,De)
  • 2 Games in 1 - Brother Bear + The Lion King (Europe) (En,Fr,De,Es,It,Nl,Sv,Da)
  • 2 Games in 1 - Columns Crown + ChuChu Rocket! (Europe) (En+En,Ja,Fr,De,Es)
  • 2 Games in 1 - Disney Princesas + El Rey Leon (Spain) (Es+En,Fr,De,Es,It,Nl,Sv,Da)
  • 2 Games in 1 - Disney Princesas + Hermano Oso (Spain) (Es+En,Fr,De,Es,It,Nl,Sv,Da)
  • 2 Games in 1 - Dragon Ball Z - The Legacy of Goku I & II (USA)
  • 2 Games in 1 - Finding Nemo - The Continuing Adventures + The Incredibles (USA)
  • 2 Games in 1 - Golden Nugget Casino + Texas Hold 'em Poker (USA)
  • 2 Games in 1 - Monstruos, S.A. + Buscando a Nemo (Spain) (En,Es,Nl+Es,It)
  • 2 Games in 1 - Moto GP + GT Advance 3 - Pro Concept Racing (Europe) (En,Fr,De,Es,It+En)
  • 2 Games in 1 - Quad Desert Fury + Monster Trucks (USA)
  • 2 Games in 1 - Scooby-Doo + Scooby-Doo 2 - Desatado (Spain) (Es+En,Fr,De,Es,It)
  • 2 Games in 1 - Sonic Battle + ChuChu Rocket! (Europe) (En,Ja,Fr,De,Es,It+En,Ja,Fr,De,Es)
  • 2 Games in 1 - Sonic Pinball Party + Columns Crown (Europe) (En,Ja,Fr,De,Es,It+En)
  • 2 Games in 1 - SpongeBob SquarePants - Battle for Bikini Bottom + Jimmy Neutron Boy Genius (Europe) (En,Fr,De+En,Fr,De,Es)
  • 2 Games in 1 - SpongeBob SquarePants - Battle for Bikini Bottom + Nicktoons - Freeze Frame Frenzy (USA)
  • 2 Games in 1 - SpongeBob SquarePants - Battle for Bikini Bottom + The Fairly OddParents! - Breakin' da Rules (USA)
  • 2 Games in 1 - The SpongeBob SquarePants Movie + SpongeBob SquarePants and Friends in Freeze Frame Frenzy (Europe) (En,Fr,De,Es,It,Nl+En,Fr,De,Es,Nl)
  • 2 Games in 1! - Dragon Ball Z - Buu's Fury + Dragon Ball GT - Transformation (USA)
  • 2 in 1 - Asterix & Obelix - Bash Them All! + Asterix & Obelix XXL (Europe) (En,Fr,De,Es,It,Nl)
  • 2 in 1 - V-Rally 3 + Stuntman (Europe) (En,Fr,De,Es,It)
  • 2 in 1 Game Pack - Spider-Man - Mysterio's Menace + X2 - Wolverine's Revenge (USA, Europe)
  • 2 in 1 Game Pack - Tony Hawk's Underground + Kelly Slater's Pro Surfer (USA, Europe)
  • 3 Games in 1 - Rugrats - I Gotta Go Party + SpongeBob SquarePants - SuperSponge + Tak and the Power of Juju (Europe) (En+En+En,Fr,De)
  • 4 Games on One Game Pak (Nickelodeon Movies) (USA)
  • 4 Games on One Game Pak (Nicktoons) (USA)
  • 4 Games on One Game Pak (Racing) (USA) (En,Fr,De,Es,It)
  • Barbie Superpack - Secret Agent + Groovy Games (USA)
  • Castlevania Double Pack (USA)
  • Combo Pack - Sonic Advance + Sonic Pinball Party (USA) (En,Ja,Fr,De,Es+En,Ja,Fr,De,Es,It)
  • Crash & Spyro Super Pack Volume 1 (Europe) (En,Fr,De,Es,It,Nl)
  • Crash & Spyro Super Pack Volume 2 (Europe) (En,Fr,De,Es,It,Nl)
  • Crash & Spyro Superpack - Spyro - Season of Ice + Crash Bandicoot - The Huge Adventure (USA)
  • Crash Superpack - Crash Bandicoot 2 - N-Tranced + Crash Nitro Kart (USA)
  • Disney's Game + TV Episode - Lizzie McGuire 2 - Lizzie Diaries (USA) (En,Fr)
  • Dora the Explorer Double Pack (USA)
  • Double Game! - Cartoon Network Block Party & Cartoon Network Speedway (Europe)
  • Harry Potter Collection (Europe) (En,Fr,De,Es,It,Nl,Pt,Sv,No,Da)
  • Hugo 2 in 1 (Europe) (En,Fr,De,Es,It,Nl,Pt,Sv,No,Da,Fi,Pl)
  • Prince of Persia - The Sands of Time & Lara Croft Tomb Raider - The Prophecy (Europe) (En,Fr,De,Es,It,Nl+En,Fr,De,Es,It)
  • SN Systems (Europe)
  • Sportsmans Pack 2 in 1 - Cabela's Big Game Hunter + Rapala Pro Fishing (USA)
  • Spyro Superpack - Spyro - Season of Ice + Spyro 2 - Season of Flame (USA)
  • Teenage Mutant Ninja Turtles Double Pack (USA) (En,Fr,De,Es,It)
  • Winnie the Pooh's Rumbly Tumbly Adventure & Rayman 3 (Europe) (En,Fr,De,Es,It,Nl+En,Fr,De,Es,It,Nl,Sv,No,Da,Fi)

Retool version 1.01 Retool options aAbBcdDemMopPrv Clonelist last updated 22 October 2021 Datfile Nintendo - Game Boy Advance (Parent-Clone) (20211110-065833).dat

ghost avatar Nov 01 '21 22:11 ghost

Thanks both for your contributions and patience. After a huge engineering effort, this has finally been fixed as of v2 beta 8.

unexpectedpanda avatar Apr 07 '23 02:04 unexpectedpanda