1) Start Registry Editor (Start -> Run -> Type 'regedit' -> Enter)
2) Locate and then click the following registry subkey: HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TerminalServer\WinStations\RDP-Tcp\PortNumber
3) On the Edit menu, click Modify, and then click Decimal.
4) Type the new port number, and then click OK.
5) If you have any firewall installed, add a rule to allow traffic over new port number that you enter (Must!) - Please ensure that this step is followed correctly else you will lose connectivity with your VPS.
6) Go to Windows Firewall with Advanced Security >> Inbound Rules >> New Rule >> Enter port number >> Next >> Next >> Done
7) Quit Registry Editor.
8) Restart or reboot entire machine.