2014-09-03: v0.0.0.4: Support for drag and dropping files into the tool, passing a file to the executable (e.g. setting it as default program for .data files). It will also automatically load the file now after either browsing, dropping, or starting with it (though not if you just start the .exe and it recovers the last filename).
2014-09-03: v0.0.0.3: overwriting files now works (previously if the files existed, it wouldn't do anything since I forgot an additional check for that) Thanks to JettSchae for reporting this.
2014-08-31: v0.0.0.2: Editing values, cloning/deleting structs and list elements, saving of edited files
2014-08-25: updated link to Dropbox
I'm working on a tool that handles DATA (0x545AC67A) files.
I've now updated the tool to allow editing and saving (see "How to use" section on how and limitations). I haven't done much testing with edited files, so there are likely bugs. I did test editing the seed names for the CAS demo and that worked. The saving was also quite thoroughly tested. All files when loaded and left unchanged should be saved to a binary same file (except S4_545AC67A_006CA304_D89CB9186B79ACB7, which also doesn't read correctly). I won't get the game until Thursday, so I have only tested with the files available in the demo.
The data structure is complex enough that I didn't even know where to start implementing it in the s4pe wrapper (I'm capable of copy/pasting stuff to add a field, but this went beyond that and beyond my capabilities), so this is written in C++ as a quick and dirty prototype. Source code is attached and GPL3. It's a Qt Creator project, because I'm familiar with Qt from work. Unfortunately it makes the libraries the program depends on rather huge. The download is over 6MB, so I uploaded it elsewhere:
Sims 4 Data v0.0.0.5 (full version)
If you've downloaded the previous version, you can simply download the much smaller updated .exe (Sims4Data000xupdate.7z) attached to this post and overwrite the old one (the dependencies are the same and don't need to be replaced).
Caveat: I started looking at the data based on the info that s4pe already implements. The data field types are just my guesses. I'll make corrections as necessary. This will be changed to something useful once I add more columns.
How to install
- Use to unpack the folder/file. If you have the large package with the .dlls, it's enough to download the updated executable attached to this post and overwrite the old version.
How to start
- Just click the exe
How to use
There are several options to open a file. In all cases the DATA resource must have been previously exported with s4pe.
- Click "Browse" to select a DATA resource that was previously exported with s4pe.
- Pass the file path as a parameter to the executable (this is what happens when you set the .exe as default program for the .data file type)
- Drag a file onto the window
- You can also simply paste the path to the file in the line edit (this option doesn't remember the path)
Once you have set the file path in one of the above ways, the tool will mostly automatically load the file. In case of pasting the path, or if the tool restored the path from the last session, you'll have to manually click "Show File Info". (The loading of the data info might take a moment for the largest ones.)
- "Clear" clears everything
- "Save" saves the currently displayed file
- "Export All" is a convenience function that takes all .data files in the folder of the current file, then exports the meta info and data info as text files into a subfolder "as_text" (the folder needs to exist; if the files already exist nothing happens).
- Right-clicking on list elements and structs directly in the table will allow you to clone/delete them (you can't delete the last struct in the main tables, though)
- Double-clicking on values allows you to edit them (mode name is the exception)
Credit: Thanks to Rick, ChaosMageX, Kuree, Peter and Inge Jones and everyone else who helped with s4pe and in particular the DATA wrapper, which I used as a starting point.
Thanks: 382285 in 1066 Posts
I just committed a change to Data wrapper trying to fix the saving problem. The padding issue hasn't been solved entirely, but right now it works for small data files. As you can see in my commit, I'm trying to make it editable in grid view, which requires a lot of work. DATA looks very promising to tuning, but it's designed for reading rather than writing. We still have a lot of work to finish implementing DATA, but you're welcomed to join us on github.
@Kuree: I took a look at the changes you committed. As I said, I didn't know how to implement the data structure the way it's needed to work for grid view. I do have a few comments, though, from my observations of the data:
In the Data table unknown0C is the data type (same as the field data flags) and unknown10 is the field_size (for string tables this is 1).
0x00000001 is type string, only used as type in data tables, which are referenced by fields of type 0x0000000B
0x00000009 => CAS Modifier Instance, i.e. Instance ID of a resource of type 0xC5F6763E
VFX = 0x0000000B, => this is actually a string, it's the offset to the actual string in the string table (i.e. the table of type 0x01)
0x0000000C => Mode Name (only used in censor tuning), this is string offset + hash of that string
0x0000000D => offset to structure based data (and in tables means it's data according to the structure table it has a pos stored to)
Unknown3 = 0x0000000E, => this is a list with the offset to the first element and then the number of elements
0x0000000F => two floats
RGBColor = 0x00000010, => three floats, mostly color, but e.g. the padding in the censor bone data is probably dimensions
As far as padding goes: I believe the DT header, the individual Data blobs, and the ST header all begin at byte zero, so the 16 bytes line before them is filled with zeroes as needed.
And yes, the format is definitely meant for reading more than writing.
I've imported the category and flag data that you provided as a sample into Color Magic 0.2.3 (not yet released, expect it Saturday 8/30) and it is amazing how various items are flagged.
Thanks a bunch for tracking the information down.
codename: granthes -- Developer of Sims3/4 utilities
Now working on modding tools for the Sims 4 CAS Demo
@granthes: I'm glad the info helped and looking forward to seeing the flags and categories in action
ETA: I've added a new version with editing capabilities.
But I've added that now in v0.0.0.4. While I was at it, I also added drag/drop support, so you should be able to just drag .data files onto the tool. And it will then autoload the file, so the extra step of clicking "Show File Info" is not necessary anymore. (Except when pasting/typing a path and if you want to load the file from the last session.)
I was trying to test it as a helper for S4PE, yet it only showed the files that S4PE passed in. Any ideas? And do you mind integrating it into S4PE project? :D
@Kuree: I hadn't even thought of adding it as a helper. I just tried it and the tool didn't show the UI and was just running endlessly in the background. After investigating it seems that S4PE doesn't export the temporary file the same way it does in the actual export to file functionality. The temporary file is corrupt (the same issue actually exists for the hex preview). What do you mean by "it only showed the files that S4PE passed in"? Does the UI come up for you?
I wouldn't mind integration into S4PE. Did you mean just adding a .helper file in which people only have to change the path or distribution with the package? I'm fine with anything, but wouldn't the Qt dependency and the fact that it's C++ be an issue (with the filesize and the way the solution is built)?
Thanks: 121 in 6 Posts
And a bug report I guess - when you save file second time without loading any other, the application will crash.
Thanks: 1794 in 62 Posts
This way it would be much easier to find what we want to edit
I'm do not know English language very much, so I do not know if already asked, but a small note (I hope it's not hard to add).
Can you add a file-tree on the left of the main window? The reason - do not open each file individually. Just open the folder containing the data files in the file-tree and then view the files from the tree without opening / closing the program every time, or choosing to again file by clicking the "Browse".
I am not a programmer and do not know if it is difficult to add it, but if not - do, please.
P.S. I do not know how anyone, but I now import all the data files in a directory for a separate and yet look through almost straight (if not find an interesting group - miss). Without the file tree for me the process is slow. Of course, you can see the same thing in s4pe, but it also takes time with the transition between programs, etc.).
P.P.S. Sorry for my English. This text was translated with Google-translate.
P.P.P.S. I also had a crash when forgot to save the file or not, and hit the re-saving.
@Kodek: I hope I fixed the bug. Let me know if it still crashes. As for quicker saving: Do you want a shortcut instead of button or do you want to avoid the dialog coming up? I.e. would distinct Save/Save As buttons be okay? Shortcut shouldn't be a problem. I tend to be paranoid when overwriting things, but I see that it can get annoying. Out of curiosity: What changes are you making on so many files? Because this seems like something that might work better with a dedicated tool. Is it always the same change, just in different files?
@IceM: Yes, I'd like to be able to read from and write to package files. My first test failed to decompress the files and I haven't tried it again since then (though I assume this shouldn't be too hard to do). I can't say when I'll get to that.
@Kasany: Yes, I've also considered having a list of files for browsing a directory, which isn't quite a directory, though that should also be possible. I'm not sure yet how I'll handle it. If I get the package reading done quickly, that might make reading .data files directly almost obsolete. As for finding information in the data files: I find it much easier to do a text search on the info, which is why I implemented the "Export All" functionality to generate text files with all the data info. Those files are then easily searchable by keyword (and should also be easy to open in a text editor) with a normal search application (or even the windows search). The crash is hopefully fixed in the latest version 0.0.0.5. Let me know if it still crashes.
Thanks: 121 in 6 Posts
Originally Posted by velocitygrass
@Kodek: I hope I fixed the bug. Let me know if it still crashes. As for quicker saving: Do you want a shortcut instead of button or do you want to avoid the dialog coming up? I.e. would distinct Save/Save As buttons be okay? Shortcut shouldn't be a problem. I tend to be paranoid when overwriting things, but I see that it can get annoying. Out of curiosity: What changes are you making on so many files? Because this seems like something that might work better with a dedicated tool. Is it always the same change, just in different files?.
Great work, saving the same file second time or more doesn't crash application anymore. And I love the STBL integration, it's so much better now. Thank You!
Save and Save As buttons would make using the application easier, because anyway most of times you need to save the file with the same name. Shortcut for saving should be something for quick saving, so dialog probably should not come up then, but even if it would, the quicker way to save will be appreciated.
There are many files for tuning the same category of elements (like careers) and with scripting it will be much quicker and easier, but I wanted to change some things in that right now. It's just a small thing, nothing special.
Thanks: 9186 in 18 Posts
"You can do refraction by raymarching through the depth buffer" (c. Reddeyfish 2017)
I had aleady download dozen of dlls. And this programm was finally dead :c
"The procedure entry point not found in library"
I know, this troubles are all because of Qt