That's a fairly well known bug.
I found the Romans easier. I'm more of a melee player anyway and since my spell use is mostly limited to blessing, I didn't find the magical restrictions too hard. In addition to which, since I can still have the trait berserk and make up for it with Roman training, I'm able to hack through pretty much anything.