Much of our C code relies on various memory allocators: malloc, kalloc, zalloc, sal_malloc, and spin_malloc. Many of which are implemented in terms of the basic bsd/kern_malloc support. The problem is that they create untraced references to memory.
There were two memory leaks introduced by the networking code (mbufs datastructures):
Redirect all C untraced allocations to the traced heap, and rely on the garbage collector to catch memory leaks. For the allocations to be GC safe requires that the memory being referenced is immobilized before being passed to C. This approach requires that the M3 allocator functions even when called from an interrupt handler, and that it is available early in the boot phase.
Alternatively, a conservative garbage collector for untagged memory could be used that runs over the untraced heap.