Join Us and become a Member for a Verified Badge to access private areas with the latest PS4 / PS5 PKGs.
Category PS4 CFW and Hacks       Thread starter Thread starter PSXHAX       Date / timeStart date Mar 26, 2022 at 1:26 PM       Replies 28      
Status
Not open for further replies.
This weekend PlayStation 4 Scene developer @iddaemon (aka id-daemon / daemon1 on XeNTaX) released a HFW Tool for PS4 / PS5 to aid in extracting skeletons, textures and static meshes from Horizon Forbidden West for data miners and game modders to examine. :geek:

Download: h2_extr.rar (603.97 KB - includes h2_extr.exe, Edgetosmd_i.exe, Edgetosmd.exe and decima.ini) / mesh_numbers.rar (3.79 KB - includes robots.txt, other_chars.txt, heads.txt and aloy.txt) / robots_codes.rar (950 Bytes - includes robots_codes.txt)

Spoiler: Depreciated

Those interested can also check out ID-Daemon's YouTube Channel, and this comes following several of his previous asset extraction tools including:
Below is a Horizon Forbidden West Tool update summary from daemon1, to quote:
  • So far I got all (or almost all) characters/robots with skeletons and textures, also static meshes.
  • Tests are ok so far, most of the models work fine. Currently working on placing destructible parts for robots, and then the tool will be complete and released.
  • Not all of them placed, but its working already. Horizon Forbidden West will NOT be released on PC in the nearest years. So this tool is for PS4 and PS5.
  • Finally. Fully working robots with all destructible parts. Tool released.
  • The process of extracting models is complicated, especially getting robots with all destructible parts. I will try to provide information on it as soon as possible. Unfortunately, its still impossible to post a file thats bigger than 256kb here, so its released on zenhax.
  • Material support later. So far you can find/apply textures manually. Animation support will be later. Aloy is easier to do, because its morpheme, and i have it all reversed before.
  • Robots are using internal decima animations, so i dont know if/when i can support it.
:arrow: Update: I'm going to start explaining the situation and what can we do about it.

First of all, data layout in this game is totally different from previous decima games. Assets are not organized in any logical structure, and have no names. There's no such a thing like assets for characters, any specific character or robot, they are not grouped in folders with textures, models, or shaders. Nothing like this. Instead, they are all grouped in so called "Streaming Graph".

As I understand, it was done for optimization purposes, so the game can load/use all of the assets (and there are millions) in the best way possible in terms of speed/memory usage. As a result, its hard to get anything from there. Especially considering there are almost no names anywhere.

There are 18 main big packages containing about 75000 asset groups. Then each group has individual assets inside. Groups are very different in size. One of them may have a 1000 of assets, and another one - only a few, or even one single asset. And all this is split weirdly. For example, we may have one big part of a robot in one group, and a small part of his left leg in another. Textures/skeletons may also be scattered around other groups, seemingly without any logic.

So what i did is a scan of all asset groups for skeletal meshes, and recorded their numbers. Then i searched for specific data/strings inside, and split them into 4 parts: character heads, aloy outfits, robots, and those that left. The ones left seems to have most other enemies, npcs and characters. I'm adding these lists here. Also note Aloy's head is ABAA, her long hairs is 6AA8, skeleton/eyes/teeth textures is 710.

Now current tool usage is very simple. Change path in INI file to your files location, make sure oo2core_8_win64.dll is in the same folder. If you dont have version 8, use 7 or 6, just rename it. Once all done, run the tool with 1 parameter: asset group number in hex:
Code:
h2_extr.exe <number>
For version 1.00 these numbers are from 0 to 126e4 (which is 75492 decimal).
Code:
example: h2_extr.exe 00AF5
or h2_extr.exe 00af5
or h2_extr.exe af5
If package loading will be successful (for 99% of all packages), tool will export textures as DDS and models as ASCII. If present, skeletons will be exported in separate files, so you have to copy-paste them to models (as it was done before for horizon 1).

During export, you can see which skeletons are used by each mesh in this form:
Code:
Mesh with skeleton 710 16
That means that skeleton is in group 710, asset 16. If thats the same group, it will be already exported. If its another group, you will have to extract that group to get the skeleton (if it was not done before).

1. More asset groups supported, now only 14 of 75000 groups crash.

2. Material output. Each model will now export "..._materials.txt" file with a list of all materials used. Also ascii file itself will have material numbers inside as a name of each submesh.

Materials will be exported to "materials" subfolder as individual files. Each file may contain several material sets. For each material there will be list of texture numbers + variables. Sometimes these variables contain important data, such as solid colors and other shader constants. Texture numbers may be either exact texture number (corresponding to texture file names exported) or texture set number. In this case you will have to guess texture numbers, because this info is not yet exported. Also some textures are not present in any sets. These are not exported yet, so you can't use them.

3. animations. Only morpheme animations supported, which are used only for Aloy, npcs, and only one robot - mole. These are group numbers containing them:
Code:
6b0a - aloy
99c - npcs
a935 - mole
Once exported, you will get all animations as in horizon zero dawn tool.
Unlike 1st game, forbidden west has no physics skeletons (which i used for animations), so you have to use mesh skeletons. Tool now exports these in both ASCII and SMD for this purpose.

Tool updated.
  • Edge animations support.
  • Robot animations & cutscene animations are also working now.
Time to explain the most complicated part, robots.

First, I will describe all assets needed. All of them were present in horizon 1, but now we also need EDGE assets. To get full animated robot, we will need ALL of these. And they could easily be in many different asset groups (groups described in a post above), or all in one. Which complicates things.

This is a list of all robot assets used in extraction:

1. Models:
  • main model
  • additional models (antennas etc)
  • destructible parts
2. Skeletons:
  • mesh skeleton
  • physics skeleton
  • edge skeleton
  • SkeletonHelpers
3. Other data:
  • destructibles description
  • animations
  • textures and materials
Now let me describe these assets a little.

1. Models are kind of usual, and they are same as horizon 1. There are many static small destructible parts, one big skeletal model, and maybe a few more skeletal/static parts. All destructible parts are usually in one group, rarely in two. Canisters are in separate small groups, one for each canister type.

2. Skeletons. Why do we need all these?

Mesh skeleton has all bones for visual model parts, weighted, and many of them are not present in any other skeleton.

SkeletonHelpers (called robotmodelhelpers in zero dawn) is the only place that has all helper bones - for destructible parts.

Physics skeleton. In addition to base skeleton, it also has bones needed to attach helpers, and again, some of them are not anywhere else. Nothing is weighted to them, they are just auxiliary bones to attach helpers, and then destructible parts to be attached to those helpers.

Edge skeleton is almost same, but its needed for sony tool to work.

3. other data:

Animations - original EDGE anim files. Most animations are in one group, just its usually different group, not the same as main model.

Destructibles description is a special table that connects SkeletonHelpers with static destructible parts. That part of the process was done manually in Horizon 1, because everything had names. Now we need those too.

So the 1st step in getting any robot is finding all these assets, in which group they are. We already have a list of all main parts for a start (attached). Now here's what the tool does when it encounters these assets:

1. Models - Models are exported as ASCII files. Note that destructible parts on the first run (when its unknown where to place them) are exported as static models. Later on, they will be exported as skeletal models, placed and weighted to corresponding bones.

2. Skeletons - All skeleton asset types (skeleton, physicsskeleton, skeletonhelpers, edgeskel) are exported as raw original assets with corresponding extension. Additionally, mesh skeletons are exported as ASCII + SMD, so they can be used to combine with ASCII model or SMD animation.

3. Other data - Destructibles description is not exported at all. This information is only used on the run, to update the data about how to connect destructible parts to helpers. The tool console output can also be used to create a list of all destructible parts and add them to the process. So far I'm only processing their "initial", undamaged states. Damaged states can also be added later.

Edge animations are exported to "anim_edge" subfolder with edge skeleton code in the end of its name. The group/asset number from this code will tell if its this robot animation, or another actor/character animation.

Textures and materials are exported to corresponding folders, just like it was before.

How to use the tool to get full robot.

1. Find mesh skeleton, physics skeleton and skeleton helpers.


If you don't care where each part is, just use the tool on all known groups for that robot. Run it as usual: h2_extr.exe <number>

After that, you will have a lot of stuff exported, and there will be several assets of each type. You need to choose only one of each type, the right one.
  • mesh skeleton is the .Skeleton file thats indicated when extracting main body (the same one that is used in usual models like before)
  • there are usually 2 .PhysicsSkeleton files, you need the big one
  • there are many helper files, open them and see inside which one has helper bones for destructible parts. Correct one will have bones called like "body_helper", "LegPlate_helper", "part12_helper" etc. Some others may be very small, or have "camera", "action" helpers inside, they are not needed.
2. Once you have those 3 files, run the tool with 3 parameters:
Code:
h2_extr.exe <skeleton> <physicsskeleton> <skeletonhelpers>
After that, you get FULL skeleton in ascii/smd called "..._robotskel" and a "robot_helpers" file. That robot_helpers file will gather information about all helper bones. ASCII skeleton will be needed to load the full model, and SMD version is for animations.

3. Process all destructible descriptions.

Again, if you don't know where they are, or don't care about it, just use the tool on all known groups and save the tool output. If in this output you see any missing groups for destructible parts, add them to your list. Messages looking like "Destructible mesh 235 - 8" mean that group 235 contains some needed parts.

4. Cleanup (optional)

Now everything is ready for final pass. At this point, it may be good idea to delete all ASCII files. It's not required, but that way you can get rid of leftover static parts that will now be replaced with proper skeletal parts.

5. Final pass.

Run the tool on all groups contaning models. Or simply run it again on all of them.

In the end, you get 3 kinds of ASCII files:
  • files with _destr suffix are destructibles
  • files with _static suffix are static parts
  • all other "usual" files with no suffix - these are skeletal parts
After everything is exported to ASCII, you can process all parts exactly how it was in Horizon 1. Static parts may be loaded directly. Destructibles must be all merged together, then count submeshes, add that number above and merge with skeleton. Skeletal parts must be merged with skeleton. Or you can also merge them together like destructibles, but in this case, before merging, remove submesh count from each file.

Extracting Edge animations.

1. Export .edgeanim and .edgeskel files from animation group (if its not done yet)

Each robot has several edge skeletons, you need to use correct one, that corresponds to animations. You can see skeleton code in the end of every animation name.

2. Use Edge anim decompiler tool (found in leaked Vita *** for example)

That tool is available in public, and has simple help option how to use it. Can be run in a batch. It outputs animation data as text. You need to save console (command window) output to text file for convertion.

3. Convert .txt to .smd with Edgetosmd.exe

Tool has only 1 parameter: input text file. As output, it creates .smd file with same name. Also can be run in a batch.

4. Merge full SMD skeleton with animations.

Resulting SMD files will only have animation frames, not the initial pose. The bind pose in these animations is usually all zeroes, and it can't be used to apply to the model, so I'm not exporting the bind pose. Instead, the full set of bones with their proper bind pose must be taken from full robot skeleton created before on the stage of model extraction.

You can merge each animation with skeleton individually, or merge several animation files putting skeleton in the beginning, so you can load them all as one big animation.

After that, you can apply animation to the model in your favourite 3d editor.

Added a version of Edgetosmd tool (Edgetosmd_i.exe) that adds all frames, with interpolated data. This is for noesis smd plugin which can't work properly without having all frames. Blender smd plugin can load only keyframes (this is what first tool did) making smaller smd files.

Cheers to @zecoxao for passing along the news on Twitter earlier! 🍻
Since someone already told what it was, I can confirm: it is not a final build, it is a beta build that didn't require 9.03 firmware.
Horizon Forbidden West Tool for Skeletons, Textures & Meshes by ID-Daemon.png

Horizon Forbidden West Tool for Skeletons, Textures & Meshes by ID-Daemon 2.png

Horizon Forbidden West Tool for Skeletons, Textures & Meshes by ID-Daemon 3.png

Horizon Forbidden West Tool for Skeletons, Textures & Meshes by ID-Daemon 4.png

Horizon Forbidden West Aloy Animation Test 1
Horizon Forbidden West Dredger Animation Test 1
 

Comments

The scene has held the game hostage for months now but no it doesnt mean a backport is happening unless they decide to send it to whoever is not afraid of posting it. I really dont understand how scene thinks sometimes, its not like releasing/sending the game files will reveal a new exploit...

I did try to come up with ways to exploit those files and i can think of 5 ways of obtaining higher fw games possible but 4 are scene private and 1 of them i had the idea but im not exactly sure its possible, which would require me to obtain access to the live memory of an updated console running those games
 
Status
Not open for further replies.
Back
Top