I'm working on a project that will require a list of all sets including all of the parts they contain. I'm aware of at least three sites which have this information (Peeron, BrickLink, and Rebrickable), but I haven't been able to find a way to download all of this data easily.
The closest thing I have been able to find so far is the Brickset API, but Brickset only has a limited amount of set metadata (year, theme, part count, etc) and does not seem to have a full listing of parts contained in each set.