So, maybe I am confusing two different things. In the Beraza Woods, not the Beraza Pits, there is a switch on a wooden wall. My character hears the sound of stone scraping against stone, but I can never find any door that it opens. This may not be connected with the Hidden Ogres quest, but it must open something!