Reiterating your copyright of the Blitz3D API and documentation seems the most logical course if any to take.
Ditto. The issue here is of copyright, which applies regardless. Someone has definitely stolen your documentation, copied the Blitz3d API exactly and copied your specific implementation of the Blitz3d API too. I think it is copyright law that applies here, not anything to do with an EULA or license agreement. (Which isn;t to say that such a thing would be a bad thing, provided it's an upfront EULA, not an "after you buy" thing).
Given that he has essentially hacked the exe into a DLL and exposed the commandname_ calls, could you release a b3d update that munges those internal calls such that they are unintelligible and different every time? Of course, this only slows down his "updates" based on your updates, and only really hurts if you were to release a highly functional update...