This is an idea I’ve had for a while. Iteration times for native code are not improving much even though we have lots of CPU horsepower for compilation. The number one offender is link times which are as bad and non-parallelizable as ever.
So image that you didn’t have to wait for linking after changing one or more C or C++ files. Just recompile the individual object files and the game (or other application with an update loop) will pick up your changes, possibly without even restarting.
I think this could be achieved by replacing the link step with a system composed of two parts:
- A linker daemon (the host)
- A target process that loads code+data over the network from the host and then accepts updates from the daemon (the target)
The host process manages the address space of the target process. It keeps the target program’s object code and data in memory and can resolve relocations internally. When all symbol references are satisfied it will push the required changes to the address space of the target over a socket and tell it where to jump. The target process periodically checks for updates and “parks” its execution in a safe point where it can receive these updates.
Imagine a scenario with a program consisting of 100 object files. Here is a chain of events describing a programmer working on changes to a subsystem in a few of those object files:
- The programmer makes a build (but doesn’t link the program)
- The programmer starts the host, telling it to load all the objects. The host resolves symbols and prepares a memory image of the target program, much like a regular linker would, with the key difference that everything is kept in memory.
- The programmer starts the target stub, telling it to connect to the host.
- The target stub downloads a complete memory image and starts running a loop, calling the main entry point.
- The programmer changes a few source files and recompiles only those files.
- The host picks up on the object file changes (maybe through an explicit signal from the programmer)
- The host relinks the target image in memory, making sure symbols are resolved.
- The host synchronizes with the target to make sure it is safe to rearrange its memory image.
- The host garbage collects stale code and data that is no longer referenced, possibly by reaching out and scanning the target process memory to make sure there are no dynamically stored pointers into those blocks in a conservative fashion. If so, a warning is printed and the blocks are retained. They can be released on a full restart of the program.
- The host transmits the required changes in code and data to the target’s memory.
- The target resumes calling the main update function.
- The target runs the new code. Repeat to step #5
- Function pointers stored in the target memory would point to old versions of functions if they are updated. While we can scan for them and not remove the old code if it is referenced, it would be confusing to the programmer. The target layer can provide a callback to the user program that lets it adjust function pointers as an opt-in API.
- Relocation of code to avoid fragmenting the target memory too much. The host could compact most of the target’s memory space safely while it is suspended in the safe update point and update all relocations accordingly with a patch list. It would avoid moving functions and data that have pointers (pinning them) like described above.
- Certain C++ features like static constructors would not work well without additional support.