entities may be different sizes
Sure, but you know what size your entities are and therefore what radius to compensate for.
or may not be travelling directly in front/behind of camera
Well if "in front" is a universal standard then get the z position.