The yellowing that is common in some colors is caused when a fire retardant that was added to the ABS plastic breaks down under UV light.
One of the elements used in the fire retardant is bromine. When the chemical degrades, bromine atoms can be freed and can make their way to the surface of the brick where they bond with an oxygen molecule (O2) and take on a characteristic brown tint.
The Retrobrite process uses UV light to break the weak bond between the bromine and oxygen and then replaces the oxygen molecule with hydrogen (from the hydrogen peroxide solution). This molecule is colorless, so the brown tint disappears.
Cellulose acetate bricks did not contain the brominated fire retardant chemical and so are not susceptible to the yellowing problem. You will usually find that the older cellulose acetate bricks are brighter and hold their color more consistently than newer bricks that contained the retardant. Modern LEGO bricks no longer contain brominated chemicals.
Hydrogen peroxide can have a bleaching effect on cellulose acetate plastic. As with any bleach, prolonged exposure can damage materials. The solutions of hydrogen peroxide that are sold as household strength are relatively weak and should not significantly damage the plastic for the few hours they are normally in the retrobrite solution.