Edit.
I came up with better idea how it can still look pretty in the front and a little ugly on the inside, but now have most of the window attached.
No changes at the front:

However this is how it looks now at the back:

Exploded view:

Original suggestion:
Another option, which I don't like too much since it relies on friction, with some elements not being attached to anything. However it is easier to integrate in a wall of a building.

Exploded view:

And as you can see the main part of the window is not attached to any studs. In such case it can be built to be pretty on both sides. And if not deliberately pushed, window will stay in place.