Having a mixed SSL settings in the same replica set is possible (although generally not recommended for a production deployment), however, it is important to understand the security implications of such an environment.
sslMode has four options:
disabled: The server does not use TLS/SSL.
allowSSL: Connections between servers do not use TLS/SSL. For incoming connections, the server accepts both TLS/SSL and non-TLS/non-SSL.
preferSSL: Connections between servers use TLS/SSL. For incoming connections, the server accepts both TLS/SSL and non-TLS/non-SSL.
requireSSL: The server uses and accepts only TLS/SSL encrypted connections.
To sync data between on-premise to AWS using SSL, all your severs (on-premise and AWS) should have SSL set to preferSSL or requireSSL. In both cases, communications between nodes in the replica set will be encrypted using SSL. However, preferSSL accepts connection in non-SSL where requireSSL doesn’t. So the choice between the two depends on what level of security you would like to have for the client connections.
You could also mix preferSSL and requireSSL, the main difference being whether clients connecting to them requires SSL or not:
Nodes which have preferSSL can accept non-SSL connections.
Nodes which have requireSSL cannot accept non-SSL connections.
Regarding the upgrade path, to avoid any downtime, you could start by changing all servers to allowSSL, where the node would accept incoming connections, but communications between nodes in the replica set will not be encrypted using SSL. Once all your replica set to allowSSL you could start changing them one by one to be requireSSL and later on (if you choose) to requireSSL. This would ensure that at every stage your nodes can still communicate with each other. You can find details about how to upgrade to SSL in the Upgrade a Cluster to Use TLS/SSL page
As with any major changes to a deployment, please ensure that all data are backed up and all procedures thoroughly tested.
