Should I contact my ISP or is there something I can do to prevent this or at least find out who/what is using my e-mail?
No there is nothing you can do to prevent this. If anything you should contact the company bouncing the mail (if it's an obviously legit company) and tell them that they're being used as an inverse SPAM relay.