Clean up the public link interfaces
This should only contain dependencies that are actually used in public headers.