VPS is an independent "virtual server" running on a dedicated server along with other similar systems. So basically its not a real server, but rather a part of it that works independently.
VPS's can run on both Windows and Linux, and may be few more.
RDP is a protocol. You can connect to a remote computer from your own PC using this protocol called Remote Desktop Protocol.