Holger Matthes has a good page on SNOT building which includes a few techniques on how to get stud-down orientation.
Also, don't underestimate tiles, sometimes you can just lock the upside-down part into place without attaching it with studs at all (I did this when I was a kid to use a black 3943 — Cone 4 x 4 x 2 as a train chimney as the inverted part didn't exist at the time:
None of the inverted parts are actually attached, but they are locked in place by surrounding parts. You can see inverted parts in orange in this partial build; the blue parts are tiles in the usual direction:
Another good part to inverse stud direction is the 4081b — Plate 1 x 1 with Clip Light - Type 2 ( as the studs are exactly one plate apart. You'll need to hide it in the construction somehow, but it can be very handy.
For example, both that part and the tile-on-tile technique are used in this window roof to inverse the below arch:
It all rests on a tan Tile 4 x 4 with Studs on Edge, which is also used higher with a Train Roof 6 x 6 Double Slope 45 / Slope 33 to finish the roof. The following image illustrate a partial build, will all upside-down parts colored orange:
Edit: there are of course more techniques to be found on the Internet, but some are more simple or place-efficient than others. For plates for example, the use of levers is as compact as you'll ever find. For bricks, you can also stuff a Technic axle in the tubes, as illustrated here. Simple and efficient.
Another edit, this time answering the question with the intended stud direction.
On a 2x4 brick/plate, place a 3185 — Fence 1 x 4 x 2, then add two 87580 — Plate 2 x 2 with 1 Stud with the stud facing down in one of the empty holes of the fence. This is probably the smallest offset you'll get, but the thickness of the fence isn't really standard, so whether it's useful or not depends on the rest of your creation.
If you're willing, there's also the Clikits Bead, Ring Thick Small with Hole which accepts a stud in both directions.