I can only think it is because of their wealth and status. Andrew Mountbatten Windsor, formerly Prince Andrew, has only lost his titles. Peter Mandelson lost his cushy job and resigned his place in the Lords, Sarah Ferguson's charity has shut down. No other consequences for any of them.