Not sure if this has been mentioned, but when the ogres attack him, they start pushing back. As long as you selectively place your characters and kill ogres so he keeps getting pushed back across one of the bridges- You can steal all of his treasure.