#16 Logic for read_regions() method in iso class

Closed
opened 2 months ago by TheWhale23 · 5 comments

Hi,

I am not sure if this is the space for such questions, but I wondered about how you define the regions of the ISO file.

In the __init__ of the iso class you get the number of unencrypted regions form the first four info bytes that are always included with PS3 games. We also know that there is always a odd number of overall regions and that the first region is always decrypted. Thus if self.number_of_unencrypted_regions would equal 2 we have three regions.

  • Region 0 : decrypted
  • Region 1 : encrypted
  • Region 2 : decrypted

So far so good. I also understand that following the first four informations bytes (which contain the information on self.number_of_unencrypted_regions) there are four throwaway bytes which you skip with

# Skip unused bytes
input_iso.seek(input_iso.tell() + self.NUM_INFO_BYTES)

Further I also understand that following these skipped bytes we can extract information on the region bounds in 4 byte pairs (up until we'll read 4 bytes with a non-size) You also commented this in the read_regions() method

 # We'll read 4 bytes until we hit a non-size (<=0)

Now for my question. How do you allocate the byte borders to the regions?

For the first region you have:

# The first region is always unencrypted

encrypted = False

regions = [{
            'start': core.to_int(input_iso.read(self.NUM_INFO_BYTES)) * core.SECTOR,  # Should always be 0?
            'end': core.to_int(input_iso.read(self.NUM_INFO_BYTES)) * core.SECTOR + core.SECTOR,
            'enc': encrypted
          }]

So you include the very first byte and all of the bits of the end byte as you add + core.SECTOR to the end.

For all the consecutive regions you have:

regions.append({
                'start': regions[-1]['end'],
                'end': end + core.SECTOR - (core.SECTOR if encrypted else 0),
                'enc': encrypted
            })

I am confused by (core.SECTOR if encrypted else 0). Why do we have to subtract one sector from the encrypted regions but not form the unencrypted regions?

Lets say we have a disc like this:

  • Region 0 : decrypted - Sectors 0 - 2
  • Region 1 : encrypted - Sectors 2 - 5
  • Region 2 : decrypted - Sectors 5 - 8

If I get this right your code would do the following:

  • Region 0 : decrypted - Sectors 0 - 2 - All bits from byte 0,1 and 2
  • Region 1 : encrypted - Sectors 2 - 5 - All bits from byte 3,4
  • Region 2 : decrypted - Sectors 5 - 8 - All bits from byte 5,6,7,8

I assume you are correct since the code does the encryption perfectly.

But why do the first few bytes that provide information on the region borders specify borders for decrypted regions to be inclusive and those for encrypted borders to be exclusive?

Is there some documentation on this? I mean it could have been the other way around:

  • Region 0 : decrypted - Sectors 0 - 2 - All bits from byte 0 and 1
  • Region 1 : encrypted - Sectors 2 - 5 - All bits from byte 2,3,4, 5
  • Region 2 : decrypted - Sectors 5 - 8 - All bits from byte 6,7,8

This is more of an educational question. I hope this is fine.

Thanks!

Hi, I am not sure if this is the space for such questions, but I wondered about how you define the regions of the ISO file. In the `__init__` of the `iso` class you get the number of unencrypted regions form the first four info bytes that are always included with PS3 games. We also know that there is always a odd number of overall regions and that the first region is always decrypted. Thus if `self.number_of_unencrypted_regions` would equal 2 we have three regions. - Region 0 : decrypted - Region 1 : encrypted - Region 2 : decrypted So far so good. I also understand that following the first four informations bytes (which contain the information on `self.number_of_unencrypted_regions`) there are four throwaway bytes which you skip with ```python # Skip unused bytes input_iso.seek(input_iso.tell() + self.NUM_INFO_BYTES) ``` Further I also understand that following these skipped bytes we can extract information on the region bounds in 4 byte pairs (up until we'll read 4 bytes with a non-size) You also commented this in the `read_regions()` method ```python # We'll read 4 bytes until we hit a non-size (<=0) ``` Now for my question. How do you allocate the byte borders to the regions? For the first region you have: ```python # The first region is always unencrypted encrypted = False regions = [{ 'start': core.to_int(input_iso.read(self.NUM_INFO_BYTES)) * core.SECTOR, # Should always be 0? 'end': core.to_int(input_iso.read(self.NUM_INFO_BYTES)) * core.SECTOR + core.SECTOR, 'enc': encrypted }] ``` So you include the very first byte and all of the bits of the end byte as you add `+ core.SECTOR` to the end. For all the consecutive regions you have: ```python regions.append({ 'start': regions[-1]['end'], 'end': end + core.SECTOR - (core.SECTOR if encrypted else 0), 'enc': encrypted }) ``` I am confused by `(core.SECTOR if encrypted else 0)`. Why do we have to subtract one sector from the encrypted regions but not form the unencrypted regions? Lets say we have a disc like this: - Region 0 : decrypted - Sectors 0 - 2 - Region 1 : encrypted - Sectors 2 - 5 - Region 2 : decrypted - Sectors 5 - 8 If I get this right your code would do the following: - Region 0 : decrypted - Sectors 0 - 2 - All bits from byte 0,1 and 2 - Region 1 : encrypted - Sectors 2 - 5 - All bits from byte 3,4 - Region 2 : decrypted - Sectors 5 - 8 - All bits from byte 5,6,7,8 I assume you are correct since the code does the encryption perfectly. But why do the first few bytes that provide information on the region borders specify borders for decrypted regions to be inclusive and those for encrypted borders to be exclusive? Is there some documentation on this? I mean it could have been the other way around: - Region 0 : decrypted - Sectors 0 - 2 - All bits from byte 0 and 1 - Region 1 : encrypted - Sectors 2 - 5 - All bits from byte 2,3,4, 5 - Region 2 : decrypted - Sectors 5 - 8 - All bits from byte 6,7,8 This is more of an educational question. I hope this is fine. Thanks!

No worries, feel free to ask questions!

I'm gonna have to get home from work before I can 100% verify with an actual .iso, but, essentially we found in #11 that just reading and decrypting sequentially would result in the regions being wrongly decrypted. Comparing code and decrypted ISOs with PS3Dec, one could see that PS3Dec also reads the regions like this.

As to exactly where it's documented, that's a good question. The documentation I used is here, but I can't find anywhere saying that the regions should be read like that. In fact, kind of the opposite, they seem to suggest reading them sequentially.

Now why it is like this I couldn't tell you, I just made sure librays region hashes were matching the .IRDs and PS3Dec, that fixed #11 and made games run in rpcs3. It could be an error on the part of the OEM, it could be intended.

No worries, feel free to ask questions! I'm gonna have to get home from work before I can 100% verify with an actual .iso, but, essentially we found in #11 that just reading and decrypting sequentially would result in the regions being wrongly decrypted. Comparing code and decrypted ISOs with PS3Dec, one could see that PS3Dec also reads the regions like this. As to exactly where it's documented, that's a good question. The documentation I used is [here](https://www.psdevwiki.com/ps3/Bluray_disc#Encryption), but I can't find anywhere saying that the regions should be read like that. In fact, kind of the opposite, they seem to suggest reading them sequentially. Now why it is like this I couldn't tell you, I just made sure librays region hashes were matching the .IRDs and PS3Dec, that fixed #11 and made games run in rpcs3. It could be an error on the part of the OEM, it could be intended.
TheWhale23 commented 2 months ago
Poster

Hi,

thanks for answering. I used the same documentation and tried to decrypt using the information on the wiki which did not work as intended. I found your repo while trouble shooting my code.

EDIT: I should add that I implemented your method in my code and it works. So you are definitely correct. I was just curious how you figgured out to do it like this.

Hi, thanks for answering. I used the same documentation and tried to decrypt using the information on the wiki which did not work as intended. I found your repo while trouble shooting my code. EDIT: I should add that I implemented your method in my code and it works. So you are definitely correct. I was just curious how you figgured out to do it like this.

Here's the patch that added the fix in libray: fef4eee23a

I unfortunately did not write down exactly how I figured it out, but I remember comparing decrypted .isos from PS3Dec with binvis. At some point I must have noticed how it all worked. I'm sure I read parts of the PS3Dec source code as well.

How they figured it out, I don't know. Maybe they had decrypted dumps from a CFW PS3 they could compare to, or maybe they reversed code from the PS3. Or maybe clairvoyance, lmao.

Here's the patch that added the fix in libray: fef4eee23ae60f632706a0499d3746c1f16ebece I unfortunately did not write down exactly how I figured it out, but I remember comparing decrypted .isos from PS3Dec with [binvis](https://binvis.io/#/). At some point I must have noticed how it all worked. I'm sure I read parts of the PS3Dec source code as well. How they figured it out, I don't know. Maybe they had decrypted dumps from a CFW PS3 they could compare to, or maybe they reversed code from the PS3. Or maybe clairvoyance, lmao.
TheWhale23 commented 2 months ago
Poster

Yes I had the same issue as with your fix. Thanks for the insight. We can close this issue. Great work with libray.

Yes I had the same issue as with your fix. Thanks for the insight. We can close this issue. Great work with libray.

Thank you very much!

Just a friendly reminder that this project is licensed under GPLv3, which does apply certain rules regarding copyright for anyone who reads and copies the code.

I'm not a lawyer, but it might be worth doing some research around it, especially if one is planning to distribute code based on this project.

For example, see this answer on StackExchange

Thank you very much! Just a friendly reminder that this project is licensed under GPLv3, which does apply certain rules regarding copyright for anyone who reads and copies the code. I'm not a lawyer, but it might be worth doing some research around it, especially if one is planning to distribute code based on this project. For example, see [this answer on StackExchange](https://opensource.stackexchange.com/a/10186)
Sign in to join this conversation.
No Milestone
No assignee
2 Participants
Loading...
Cancel
Save
There is no content yet.