Curiosity is the latest, biggest and most sophisticated NASA rover to make it to the read planet, and like it's predecessors is controlled from the Jet Propulsion Laboratory (JPL) in California. NASA JPL sends instructions and receiving communications to the Curiosity Rover using either X band (radio waves), and also by relaying communication through one of the two spacecraft in orbit around Mars, the Mars Reconnaissance Orbiter and the Odyssey Orbiter, using UHF Electra-Lite. NASA JPL use the Deep Space Network (DSN), a series of large antenna dishes across the Earth, to send and receive these communications directly with Curiosity and with the two spacecraft in martian orbit.
The availability of the communication channel to the Curiosity Rover are critical to the £1.6billion mission, hence the resilience with the communication channels.
The data rates directly from Earth to Curiosity goes between 500bps 32kbps, remember the very early days of connecting to the Internet with a telephone modem? While the data rate relayed via the Mars Reconnaissance Orbiter is can be as high as 2Mbps, basic broadband speed. The data rate to the Odyssey orbiter is between 128kbps to 259kbps, think ISDN for this one. However there will be considerable and changing latency (lag) given the distance between Earth and Mars, which are both in motion.
Encrypted Comms? Not likely
Well I don't know whether the communications to the Curiosity rover are encrypted or not, and I'm trying to find out. But I very doubt if the communications would be encrypted given 'availability' to communicate with the rover is much more critical than any risks from potential threats to communication's confidentiality. By encrypting communications it presents increased risk to the rovers communications availability, should something go wrong with the rover comms, the encryption could be a mission killer. While threat wise against confidentiality; firstly nothing secret is being sent (so they tell us), and it's not as though hackers seeking to take control would have access to arrays of the largest communications antennas on earth, nor is it likely other nation states with the capability would have any motive to disrupt the mission. So NASA would have got their risk assessment right if encryption wasn't used in my view.
If any science boffins would like to shed more light with Curiosity communications and their security, please post a comment, I would be interested to learn more.