I don't have the RCX, and neither do I have any PF stuff, but as they both have wire connectors which are almost identical, I'm thinking that they can both work together and are compatible. Is this true? And can you connect more than one motor to one RCX port (RCX port has 4 studs but PF port has only 2)?
RCX is compatible with the older 9-V system that came before Power Functions. However, since PF motors are also 9V and still unregulated as the RCX motors, a simple conversion cable suffices. The Power Functions extension wires will do (8886 or 8871) as they have on one end an underside that is compatible with the 9V/RCX system (which hence goes onto the RCX port) With this setup, stacking additional motors is possible as well, you can stack them on both sides of the extension wire (RCX side or side of 1st motor)
The RCX uses "electric system 9V" style connectors. These are not compatible with power functions connectors. The "electric system 9V" connectors have the connections built into regular lego studs (with the electric part of each stud carefully arranged to allow connections to be made between parts placed at right angles) while the power functions connectors have a special connector section that is not compatible with regular lego studs in any way.
As the other answer mentions the power functions extension cables (8886 for regular length, 8871 for extra-long) can be used as an adapter between "power functions" and "electric system 9V". This cable has two different connectors, one end is a regular power functions connector with "power functions" connections on the top and bottom. The other end is a special connector with "power functions" connections on the top but "electric system 9V" connections on the bottom.
There are a few caveats though.
- The power functions only end of the extension cable can be a bit of a pain to mount, because the electrical positions won't stack on top of regular lego studs.
- The RCX has a 1-plate high step next to the motor ports. This was not a problem for electric system 9V wires which had the wire exit near the top of the brick but the power functions connectors have a much larger cable exit lump. This means that the power functions adapter cable can't be placed on a RCX in the normal orientation. For the side motor ports you can exit sideways, but for the center motor port you either have to exit over the screen, use an "electric system 9V" cable between the RCX and the adapter cable (so now you have three sections of cable to deal with....) or use a 1x2 electric plate (part 4755) to build-up the connector on the RCX.
- "Power functions" is a 4-wire system while "electric system 9" is only two wire. So while you can use basic power functions motors with the adapter cable things like the servo motor will not work and you can't supply power to things like the IR receiver from an "electric system 9V source.
- For smaller models actually building your model may be a pain because of the extra wire length from the adapter and because the RCX was designed with old-style technic in mind while power functions was designed with new-style technic in mind.
As for connecting multiple motors you can do so physically but if the motors are under heavy load you may hit the current limits in the motor drivers, heck even a single "RC race buggy motor" can overload the drivers in the RCX.