The plastic is a bit heavier than water and should sink, however in practice I have experienced that 2x4 bricks (and the like) have a tendency to trap air inside them and actually float rather well.
It probably isn't failsafe, however for shorter periods of time (like a half to a full hour) would probably work fine.
(I haven't actually tested the time it could stay afloat, I'm trying it out now. When my small mini-boat has sunk, I will add the time here.)
Update:
24 hours have passed, it is still floating.
It sunk about 1mm compared to the beginning, but nothing of importance. However while I did push it around from time to time, it was still in still water and stationary most of the time. How it will perform during play is unknown.
Notice how the internal structure of the bricks causes the air to be trapped inside them.