Yet we assume that placing the powerhouse item nearer the starting point means that the player would be able to access it immediately. Presuming that the same Rakshasi hold the Halberd, regardless of location, a level 10 party would no sooner be able to obtain it than a level 2. If it were located, for example, right outside the starting point, and the player were actually able to access the dungeon, that player would eventually either learn that the cave is beyond their current capability and come back later, or grow frustrated and stop playing.
We naturally assume that content further in the game's progress will be harder than currently accessible areas: such is good balance and design, as we understand it. Truly open worlds, which disrupt what we believe to be a natural flow, disrupt our expectations of progressive skill building. and many games built on open worlds and exploration (think metroidvania style) have to reinforce the concept of backtracking very early on in order to ensure that the player gets used to passing areas up and coming back when they're ready. Most games that work with this concept also give some very obvious hints that the player is not yet prepared for what lies beyond, by requiring certain equipment before allowing passage.
Granted, Jeff has safeguards built into his world (reputation, money requirements, sub-quests, and ultimately the maps and associated SDFs) that would easily prevent a fresh party from even knowing about high-end dungeons near the starting point, let alone access them, and so there are few real reasons for there not to be a high-end dungeon hidden near start, which is Trenton's point.
My guess would be real estate. The dungeons in Krizan Province (with a few Orb of Thralni-required exceptions) are designed with the goal of training a fresh party to get out and explore, and in explaining the reactions that the world will have to the actions that the player takes, when relevant.
With all of those starter and intermediate dungeons spread through the area, it seems like a bit too much to cram in that extra high-end dungeon when there's so much space elsewhere.
Further, placing the Black Halberd quite out of the way both encourages and rewards player exploration, which is what E3 is about.
As for only having the high-end spells at the very end... well, in-universe it seems logical for the best mage to be with the Queen. Otherwise, well, again, there's no reason why one couldn't have an arch-wizard hanging around teaching newbies for personal amusement, giving a small taste of the power that can be obtained with a ton of money and experience. That one's a matter of writer preference.
The Silent Assassin points out that the Mystical Item of Doom has to be in a dungeon near the end, or there's no point of having a game in between.