DEITM049.ITM Error Options

[BeachBum]

I am in the middle of installing Big World and specifcally the Lost Crossroads Spell Pack for Baldur's Gate 2 = SpellPackB5 and am encountering an error.

When trying to install the Core Files and Graphics it fails with the following error. "DEITM049.ITM: read out of bounds".

Can anyone help resolve this.

Thanks.

[Baronius]

If you want to suppress such installation errors in WeiDU, use "ELSE x" in the proper context, as TheBigg has said.

For example, here is a "patching" function call with two arguments that reads a 32-bit integer from file position 1234h to the variable "some_header_field":

READ_LONG 0x1234 "some_header_field"

If offset 0x1234 is out of range (e.g. the file size is smaller or equal to 0x1238 bytes), the operation will fail, causing an installation error. On the other hand, the following form works differently:

READ_LONG 0x1234 "some_header_field" ELSE 666
PATCH_IF ("%some_header_field%" != 666) ...

In this case, the value of some_header field will be 666, if the offset is beyond the end of stream. There will be no installation error. You can detect that there was an error by checking the returned value (obviously, to indicate the error you should use a value that can't occur in normal circumstances).

You said you learnt some Java earlier. You can think of the difference between the two solutions above as an Exception and a null check. That is, a run-time exception interrupts the normal flow of the program, while a null check can be used to detect a problem "silently" when you surely know that the returned value is a valid reference in normal circumstances (and you don't want to use exception handling, e.g. because the failure is expected to happen relatively often).

So you should add "ELSE 0" statements (with additional code if needed, I don't know your concrete code) to every function call where you expect failed reads and want to suppress (or specially process) the error. As I've emphasized, suppressing errors in mods is not always desired; if you specifically want to set a field/entry of a particular file and the file is corrupt, the installation won't fail, and the missing field (corrupt file) might break one or more features of the mod (which have the field or entry as their dependency). On the other hand, suppressing errors for features which absence can be tolerated is another question (it's a question of taste/style and/or of the mod's goals) .

Of course, there are other ways as well in WeiDU to detect that a file is corrupt (e.g. SOURCE_SIZE -- or something like that -- returns the buffer/file size, so you can check whether it has the required minimal size).

[..] how is the game able to run smoothly [..]

It ignores the problem if an invalid value is returned (exactly what you do with "ELSE 0" ). (I would've guessed that the game crashes when a referenced feature block doesn't exist, but if this inconsistent deitm049.itm works for people, then my guess was wrong and the game just ignores such problems). I didn't test how exactly the game handles item files that are invalid in this way, but it's a general fact that the file structure specifications are stricter than the implementation that deals with these files. This allows the game to be more fault-tolerant when it's possible (for example, a missing dialogue file doesn't cause any crash, but an area without a tileset cannot be tolerated ).

As far as I've noticed, DLTCEP tries to read most file formats in a robust way too. How can it be accomplished? I haven't checked the implementations in DLTCEP and NI, but I guess that e.g. DLTCEP often simply considers the file as a sequence of contiguous sections (e.g. header, extended headers, feature blocks, equipping feature blocks). Where possible, it reads all sections (when it finishes the reading of a Section A, it immediately starts to read Section B that follows Section A, regardless what the file header says about the position of Section B ). Of course, this is a bit simplified, but you get the idea. This allows DLTCEP to read files with inconsistent or sometimes even corrupt structure, and when you save it again, it provides valid values for the header fields that were incorrect in the originally loaded file -- which results in a consistent, valid output file. This is how "broken files" can be fixed.

However, this robust approach has its own drawbacks. First of all, since these editors (such as DLTCEP) aren't built for data recovery from corrupt IE mod/game files, the algorithms aren't too complicated, there aren't too advanced heuristics. Notice that data recovery and file editing have different goals; data recovery has its own uncertainty (the returned data might be corrupt) and it usually requires human interaction, while file editing should always guarantee a consistent structure, and when this isn't possible, it should indicate an error. It's typical example of the question whether we want to correct the error or to indicate it. The latter one is usually easier and -- more importantly -- safe, because correction may fail and return corrupt data without indicating that it's corrupt (but discussing this would exceed the scope of the present topic). This drawback is present in DLTCEP as well: when it reads dialog.tlk (probably with the aforementioned procedure of continous reading that ignores certain offsets) and there is an inconsistency, it offers you to make an attempt to correct the problem. If you click 'Yes', it applies its "corrections" and saves the file. Depending on the type of the faulty field in dialog.tlk, this can corrupt dialog.tlk! For example, the unofficial Russian dialog.tlk works perfectly in the game, but it contains a very tricky offset assignment (I wonder who had that idea and what he/she wanted to accomplish with it) which DLTCEP detects as a consistency error and offers to make an attempt to fix it. Its fix will corrupt the file (the game will not crash, but many dialog.tlk strings will become totally senseless, corrupt), because the assumptions made by its heuristics prove to be wrong. Needless to say that DLTCEP doesn't detect that it partially corrupted the file (this is a common drawback of correcting errors, as I've said), because structurally, the file will be consistent, but semantically, it will contain several corrupt, unreadable texts.

Considering BeachBum could open and save deitm049 in Near Infinity, this software probably uses the same or a similar algorithm to read items as DLTCEP (i.e. trying to be fault-tolerant when possible), but when you save the file, it does not correct the invalid fields. It probably writes back the same values as it read out (this has an advantage as well: it won't accidently corrupt an inconsistent file that possibly works in the game itself).

I hope I managed to answer your question Galc, and hopefully the readers of this topic also got an insight to the question of when "ELSE 0" should be used and when it shouldn't (and why).