The strict minimum you need for PF motorization is:
Now, as oezi says, it's a bit senseless to have a train which you can't control. Well, you can control it, but it's limited to switching it on and off (if you can catch it). Yet, it's a start.
Now, as far as limited control goes, don't neglect the polarity switch! If you place it intelligently, it will be easier to stop your train. Or you could even mimic the old 4,5v system and have your train switch direction automatically at both ends of a track. Just a thought.
Of course, for more serious control, yes, you'll need:
Note that all these elements (tracks, motor, battery box, IR receiver and remote) are also present in these trains:
So it may be worth to do the math and see if you're not better off just buying one of these. Sets are usually cheaper than standalone elements, and considering the extra parts you'll get, it may be a wiser investment. Not to mention that they may be easier to find in regular shops, which means maybe Christmas discounts and the like. And the design of these trains is quite good too.
Of course, batteries are going to be a problem, so if you don't want to sink too much money in them you'd better use rechargeable batteries as fast as possible, or yes, switch to:
As you can see, it can be a costly investment in the long run, but nothing forces you to buy it all in one step, especially the rechargeable battery and charger which are far from cheap.