It's a good idea, but it often doesn't work well in practice. In general, rare parts from rare sets are more expensive, so you will frequently spend more money buying all the parts for a set. As a classic example, let's look at the Airport Shuttle:
This set is expensive on the secondary market, so it could be tempting to buy the parts for it instead of the complete set. However, many of the parts are very expensive individually. The motors and monorail track are specialized parts, relatively rare, and haven't been produced in 25 years, so they are priced accordingly.
Perhaps the most egregious example of this is the single molded piece of plastic that covers the motor:
That piece is currently $67 USD used on Bricklink.
These same economic factors apply to current sets as well, so it is often hard to save money buying the parts.