Same problem with both the native 9-pin serial port and with a USB to 9-pin Serial adapter. Using Lego's original cable.
Start BricxCC Version 3.3 (Build 188.8.131.52). It asks for the connection method.
Choose any option for the Port (I've tried them all with the same results), and RCX or RCX2 for the Brick Type. Standard Firmware.
(Actually, there isn't any firmware in the brick yet. Need to do that as soon as it connects, but I can't even get that far.)
Click OK. Communication fails immediately. "No connection to the programmable brick. Certain options will be unavailable."
Acknowledge the message. Click "Find Brick", and repeat from #2.
A google search turned up a note about using a terminal emulator to talk to the tower directly. 2400baud, no handshaking, should echo characters in the terminal, and the green activity light should make sense. That DOES work, for both the native port and the USB to serial adapter.
Using https://sourceforge.net/projects/realterm/ on Windows 10, fully up-to-date as of this writing.
When I close the native port, the green light comes on and stays on, and remains on while BricxCC tries to connect. It turns off when I reopen the port in the terminal emulator.
When I close the USB adapter, the green light turns off, and remains off while BricxCC tries to connect. It stays off when I reopen the port in the terminal emulator, except to indicate activity as expected.
I tried running BricxCC in Compatibility mode for Windows 98/ME, and then for Windows XP (Service Pack 2 & 3). Same result.