Since its out of the car already, for piece of mind, id take it to a machine shop. Have it looked at and/or welded.
I had a similar problem recently when the starter ejected from my 01 back in march ..
Btw: When i pulled my ground off, it took nearly all i had to get it to turn.