The “Sound bug”

One of the most annoying bugs that plagued Battlefield 2: Project Reality for a whole decade was a bug people referred to as “Sound bug”. It was so incredibly common, that going a full round without experiencing at least a couple of sound bugs was impossible. The bug would cause vehicle sounds to continuously play from a point in the world where a vehicle used to be. You would constantly hear the sound of a helicopter from a position it was in minutes ago, or you could hear the sound of a tank driving just past the hill in front of you, only to climb the hill and see that there’s nothing there. We’ll discuss what caused this bug and the workaround I used to hide it.

Ghosts

Battlefield 2 supported 64 players simultaneously fighting on a single server, far back in 2004. For it to accomplish such a feat it had to prioritize what data to send to each player. Constantly sending updates about all objects to all players is a bad idea, no one had the bandwidth or the processing power for that. Therefore some sort of filter had to be implemented to decide what data gets sent to each client.
The system that decided this was called GhostManager, and every networked object in the game had a GhostObject instance per player. The system featured an extensive priority system which decided the frequency of the updates each object would get. It included in its calculations not only the distance from the player, but also if the object is in the player’s current field of view, giving bonuses to these objects. If you zoom in on a target, for example, your field of view decreases which causes less objects to receive the bonus. This results in more frequent updates to targets being aimed at, which means the client does not have to guess and extrapolate movements of targets he’s about to shoot. In addition to that, when an object is further away from the player than the view distance of the map, the GhostManager will tell the client that he will no longer be receiving any updates about the object and that he should disable the object. The client remembers the object, but understands that its currently not real. I assume this is what the developers meant by calling it Ghost. When an object is disabled, it should not be rendered, should not cause effects, should not collide with anything and, of course, should not play sounds…. Unfortunately, that last one tends to completely fail at times.
If you ask me, “Ghost” vehicle sounds being caused by a GhostManager is pretty ironic.

Bad quick fix

Not wanting to mess with writing assembly back then, I tried out a quick hack I knew was bad, mostly to see if it solves the problem and prove my theory regarding what the problem actually was. Every couple of minutes, we would make the server greatly increase what it thinks the view distance is, and then change it back to normal after a second. Clients would be told that all objects are no longer ghosts, sent an update of their actual position, and then told to turn them into ghosts again. This didn’t require any hacky methods, the variable was designed to be adjusted dynamically and could be changed in real time by the server. It took only few lines of Python. Also note that this did not affect client rendering and was purely server-side.

default = int(host.rcon_invoke("GameLogic.MaximumLevelViewDistance")) # Get default
host.rcon_invoke("GameLogic.MaximumLevelViewDistance 3000")
... # Later....
host.rcon_invoke("GameLogic.MaximumLevelViewDistance %s" % default)

It worked. Object positions were being reset every time it fired, fake sounds were being moved to the real object’s position. Because soundbugs started on the edge of a player’s view distance, firing the code every couple of minutes was enough that most people wouldn’t even noticed any soundbugs anymore. But it was a terrible solution, as every time the timer fired all objects would be freezing for about a quarter of a second. The server, having to tell the clients about all the objects that are no longer ghosts and send updates about them, had too much to send. Not a good trade off.

Slightly better quick fix that has been working great to this day

The real fix would have been to dive into the sound system assembly and try to figure out why disabled objects are playing sounds. Unfortunately this would have been very time consuming and I wasn’t even sure where to even start.
All the previous fix practically did was have the server tell the client to move these disabled objects away. Can’t we just do it ourselves without the server telling us to do it? First I tested with a friend that manually moving a disabled object away is enough to silence the bug
We both entered a helicopter, and using some previous knowledge from reversing other things in Battlefield 2 I could record the pointer of the object my client was currently in. I then exited the helicopter and changed team. The team change was important, as active vehicles that are on your team are never ghosted, so the client can keep showing them on the map even outside of view distance. I then had my friend fly all the way to the other side of the map, while viewing the helicopter’s transformation matrix in memory on another window. A short while later, the matrix I was watching stopped updating. After the helicopter has reached the furthest it could go, I could still hear the helicopter sounds, just as expected. I moved to the point where the fake sounds were being emitted from and confirmed that they are indeed caused by the bug. Then I changed the Y position in the transformation matrix to 10,000. The sound stopped instantly. I asked the friend to fly back to me and noted that there were no anomalies from this change, and the matrix was properly being reset when the object was enabled again.

The proof of concept worked, the sound engine relies on this matrix to decide where the sound should emit from. Now we just need to apply it every time the server tells our client to disable an object. Fortunately, the client parts of GhostManager have also been compiled into the server binaries, so we can use the Linux server binaries (which were compiled with ELF symbols) to look around a bit. Hoping to find a branch that triggers when the server tells the client to ghost an oobject, I found something much better. A branch within GhostManager::readData(dice::hfe::io::BitStream *) that calls disableObject(dice::hfe::world::IObject) function. That sounded like exactly like I imagined.

Next, I went to find that function in the Windows client. I load a game level with very little objects and find a spot on the level which causes a specific object to be disabled when I cross it. I can tell the object is being disabled because it disappears/appears on my map when crossing a specific distance from it. I attach IDA to the client and spread some breakpoints on functions belonging to GhostManager, seeing which branches get triggered when the object gets disabled. Once I’m sure I found the disableObject call on windows, I modify the call to instead call a code cave in the .exe where I can put some assembly in

; Call original
push    ecx ; ecx holds the object, back it up in stack
mov     eax, 0x5B3230 ; Offset of disableObject
call    eax ; call disableObject. (arg in ecx)
pop     ecx ; restore the object from stack

; The object's 4x4 transformation matrix is at 0x88. 
; offset 0xBC is row 4 column 2 of its matrix, which is Y translation.
mov     dword ptr [eax+0BCh], 464B2000h ; some high float
retn

The assembly first calls the original disableObject, and then applies the modification to the Y translation. When an object gets disabled, it gets teleported far up into the sky. Far enough for the object to not make any sound.
There’s a small very rare edge case that this doesn’t fix however. When clients receive packets out of order, they may receive a command to disable the object before receiving the final update for the object, which undos the Y translation change.

It’s finally possible to play Project Reality without the constant sound of fake helicopters everywhere.