I give up, its not that the spammers are actually getting the trackbacks into my blog, it’s that they are using more server resources than I want to babysit.

Simple solution, disable trackbacks and pingbacks. Caveat: this only disables trackbakcs for new posts, so make sure you turn them off for all old posts as well. You can do this in your database with:

UPDATE wp_posts
SET ping_status = "closed"