Almost every person will tell you to start with gf1 ... and I won't be an exception. Believe me, I started with gf3 and it was half as interesting as it'd have been otherwise, had I previously played gf1 and gf2.
And furthermore, once you get used to gf5's playing system (weight, moves etc.), believe me, it'll be very hard even to start playing gf3, let alone 1 or 2. But, again, if you start from the beginning, you'll notice only improvements in that area as well.
And also, I know it's hard to get yourself to playing geneforge, after only playing avernum. I've been there. But trust me, it's worth it.
And, besides, Avernum's coming to the end - what other option do you have