LadyWillow

Hi there! So I'm playing this game and loving it, but I've reached a problem. I took Gladwell's geas and now need to steal the Anama scrolls. Only problem is I can't get to them. I've run around for the last hour and a half trying to find the way up there and I can't figure it out. On a screen south of the city I can see up a cliff the place I need to be. But I can't find the way to actually get up there. I thought I read somewhere that I had to join them to get up there, but that makes no sense, because they won't LET me JOIN because of dark magic (as in gladwell's geas?). I'm obviously missing something... help please?