#15 Implement searching for [title line 1] * [title line 2] * (country) * (version)

Open
opened 4 months ago by necklace · 5 comments

Example:

Metal Gear Solid 4 - Guns of the Patriots have several versions:

name size
Metal Gear Solid 4 - Guns of the Patriots (Japan) (v02.01) 32643547136
Metal Gear Solid 4 - Guns of the Patriots (USA) (En,Fr,De,Es,It) (v02.00) 32681033728
Metal Gear Solid 4 - Guns of the Patriots (Europe) (En,Fr,De,Es,It) (v02.00) 32681033728
Metal Gear Solid 4 - Guns of the Patriots (Japan) (v01.03) (FW2.20) 35575693312
Metal Gear Solid 4 - Guns of the Patriots (USA) (En,Fr,De,Es,It) (v01.01) 35613114368
Metal Gear Solid 4 - Guns of the Patriots (Europe) (En,Fr,De,Es,It) (v01.01) 35613114368

In the SFO they look like this:

'TITLE': 'METAL GEAR SOLID 4\nGUNS OF THE PATRIOTS', 
'TITLE_ID': 'BLES00246', 
'VERSION': '01.01'

A new wildcard search therefore needs to be added for these cases, something like: [title line 1] * [title line 2] * ([country]) * (v[version]) where * are wildcards, e.g. % in SQL.

Or actually, maybe the version isn't needed but just the separation of a newline with a wildcard between? Hm.

Example: Metal Gear Solid 4 - Guns of the Patriots have several versions: |name|size| |----|----| |Metal Gear Solid 4 - Guns of the Patriots (Japan) (v02.01)|32643547136| |Metal Gear Solid 4 - Guns of the Patriots (USA) (En,Fr,De,Es,It) (v02.00)|32681033728| |Metal Gear Solid 4 - Guns of the Patriots (Europe) (En,Fr,De,Es,It) (v02.00)|32681033728| |Metal Gear Solid 4 - Guns of the Patriots (Japan) (v01.03) (FW2.20)|35575693312| |Metal Gear Solid 4 - Guns of the Patriots (USA) (En,Fr,De,Es,It) (v01.01)|35613114368| |Metal Gear Solid 4 - Guns of the Patriots (Europe) (En,Fr,De,Es,It) (v01.01)| 35613114368| In the SFO they look like this: ``` 'TITLE': 'METAL GEAR SOLID 4\nGUNS OF THE PATRIOTS', 'TITLE_ID': 'BLES00246', 'VERSION': '01.01' ``` A new wildcard search therefore needs to be added for these cases, something like: `[title line 1] * [title line 2] * ([country]) * (v[version])` where `*` are wildcards, e.g. `%` in SQL. Or actually, maybe the version isn't needed but just the separation of a newline with a wildcard between? Hm.
clee commented 4 months ago

Any reason not to just convert the newline into a space, and then try the normal title matching? I bet that'd work.

Any reason not to just convert the newline into a space, and then try the normal title matching? I bet that'd work.

Well in this case if you just convert the \n to a space you end up with METAL GEAR SOLID 4 GUNS OF THE PATRIOTS which unfortunately will lead to no hits when searching in the SQLite database simply because there's a - between from the .dat file.

Verifiable with

SELECT * FROM games WHERE name LIKE 'METAL GEAR SOLID 4 GUNS OF THE PATRIOTS%';

Which returns nothing, however if we replace the newline with a % it works very well:

SELECT * FROM games WHERE name LIKE 'METAL GEAR SOLID 4%GUNS OF THE PATRIOTS%';

Returns the aforementioned games. Combined with size it should find the key without a need for a crc32. However, it might lead to unforeseen consequences.

I'll look around a bit more, maybe there are other cases that need the version as well.

Well in this case if you just convert the `\n` to a space you end up with `METAL GEAR SOLID 4 GUNS OF THE PATRIOTS` which unfortunately will lead to no hits when searching in the SQLite database simply because there's a `-` between from the .dat file. Verifiable with ``` SELECT * FROM games WHERE name LIKE 'METAL GEAR SOLID 4 GUNS OF THE PATRIOTS%'; ``` Which returns nothing, however if we replace the newline with a `%` it works very well: ``` SELECT * FROM games WHERE name LIKE 'METAL GEAR SOLID 4%GUNS OF THE PATRIOTS%'; ``` Returns the aforementioned games. Combined with size it should find the key without a need for a crc32. However, it might lead to unforeseen consequences. I'll look around a bit more, maybe there are other cases that need the version as well.
clee commented 4 months ago

Well in this case if you just convert the \n to a space you end up with METAL GEAR SOLID 4 GUNS OF THE PATRIOTS which unfortunately will lead to no hits when searching in the SQLite database simply because there's a - between from the .dat file.

Sure, if you run that query. But the way the code builds the title query, it effectively replaces all the spaces with %, and that actually seems to work for me. I tested this change locally and it appears to make the title-based fallback work for my copy of BLUS30109.

@@ -194,6 +194,7 @@ def multiman_title(title):
     '/': '-',
     '™': '',
     '®': '',
+    '\n': ' ',
   }
 
   for key, val in replace.items():
> Well in this case if you just convert the \n to a space you end up with METAL GEAR SOLID 4 GUNS OF THE PATRIOTS which unfortunately will lead to no hits when searching in the SQLite database simply because there's a - between from the .dat file. Sure, if you run _that_ query. But the way the code builds the title query, it effectively replaces all the spaces with `%`, and that actually seems to work for me. I tested this change locally and it appears to make the title-based fallback work for my copy of BLUS30109. ```patch @@ -194,6 +194,7 @@ def multiman_title(title): '/': '-', '™': '', '®': '', + '\n': ' ', } for key, val in replace.items(): ```
clee commented 4 months ago

Output from my locally-patched version:

libray -i ~/Games/PS3/MGS4.enc.iso

[*] Searching for PARAM.SFO
[*] PARAM.SFO found
[*] Checking for bundled redump keys
[*] Trying to find redump key based on size
[*] Trying to find redump key based on size, game title, and country
[*] Found potential redump key: "Metal Gear Solid 4 - Guns of the Patriots (USA) (En,Fr,De,Es,It) (v02.00)"
Output from my locally-patched version: ``` libray -i ~/Games/PS3/MGS4.enc.iso [*] Searching for PARAM.SFO [*] PARAM.SFO found [*] Checking for bundled redump keys [*] Trying to find redump key based on size [*] Trying to find redump key based on size, game title, and country [*] Found potential redump key: "Metal Gear Solid 4 - Guns of the Patriots (USA) (En,Fr,De,Es,It) (v02.00)" ```

Ah, forgot about the '%' + '%'.join(game_title.lower().split(' ') part, right.

Well in any case I think we have to thoroughly test with a bunch of games, there are 1796 games in the database with one or more - character in them. If nothing breaks, we might as well remove all the - characters in the database to save space, and then just replace the - character from the .iso with nothing.

Actually, trying to look for an example of something that would break I found this:

Transformers - The Game (USA) (v01.00)|3098673152
Transformers - The Game (USA) (v02.00)|3098673152

That one definitely needs version.

Edit: found more:

Tom Clancy's EndWar (Europe) (En,Fr,De,Es,It) (FW2.42) | 9147973632
Tom Clancy's EndWar (Europe) (En,Fr,De,Es,It) (FW2.50) | 9147973632

and

Assassin's Creed - Brotherhood (UK) (En,It) (v01.00) | 7492927488
Assassin's Creed - Brotherhood (UK) (En,It) (v02.00) | 7492927488
Ah, forgot about the `'%' + '%'.join(game_title.lower().split(' ')` part, right. Well in any case I think we have to thoroughly test with a bunch of games, there are 1796 games in the database with one or more `-` character in them. If nothing breaks, we might as well remove all the `-` characters in the database to save space, and then just replace the `-` character from the .iso with nothing. Actually, trying to look for an example of something that would break I found this: ``` Transformers - The Game (USA) (v01.00)|3098673152 Transformers - The Game (USA) (v02.00)|3098673152 ``` That one definitely needs version. Edit: found more: ``` Tom Clancy's EndWar (Europe) (En,Fr,De,Es,It) (FW2.42) | 9147973632 Tom Clancy's EndWar (Europe) (En,Fr,De,Es,It) (FW2.50) | 9147973632 ``` and ``` Assassin's Creed - Brotherhood (UK) (En,It) (v01.00) | 7492927488 Assassin's Creed - Brotherhood (UK) (En,It) (v02.00) | 7492927488 ```
Sign in to join this conversation.
No Milestone
No assignee
2 Participants
Loading...
Cancel
Save
There is no content yet.