Phil B has some great info posted, but here are some additional things to think about.
Anything can go on clearance, and obviously adding 50% off will help the price per piece. However, sales and clearance are unpredictable in that you don't know when the sales will occur or what they will be comprised of. So I would skip doing calculations about hypothetical sales, unless you have a 70% off set in your hands.
Phil B made no mention of the Lego Pick a Brick wall. This is, for many types of pieces, the absolutely most cost effective way to get a lot of parts. If you live near a LEGO store, swing by once a month or so and load up on bulk parts.
He also talks about 'classic type' sets. This would be the Creator Line sets, which have the added bonus of having a lot of good basic pieces that aren't too specialized. Look at these first, they are going to be your best bet.
The really large sets tend to be very cost effective too, but it might not be in your budget to drop $150+ per box.
If you really want a lot of LEGO cheap, go buy 50 lb. used lots off of e-bay. Even after you pick out and throw away the inevitable mega-blocs, the price per piece is still way lower than anywhere else.