Mailman 3 January 2025 - Cryptography-dev

AES Key Type Issue
by Sriram R Jan. 20, 2025

Jan. 20, 2025

Hello, I'm testing a softHSM2 setup with the following code on rhel 9.4 system running in FIPS mode: import pkcs11 from cryptography.hazmat.primitives import serialization from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes lib = pkcs11.lib("/usr/local/lib/softhsm/libsofthsm2.so") token = lib.get_token(token_label='MyToken') with token.open(rw=True, user_pin="your_pin") as session: key = session.generate_key(pkcs11.KeyType.AES, 256, label="MySymmetricKey", template={Attribute.SENSITIVE: False, Attribute.EXTRACTABLE: True,}, store=True) data = b"Hello, world!" iv = os.urandom(16) cipher = Cipher(algorithms.AES(key), modes.CBC(iv)) encryptor = cipher.encryptor() ciphertext = encryptor.update(data) + encryptor.finalize() decryptor = cipher.decryptor() plaintext = decryptor.update(ciphertext) + decryptor.finalize() print("Plaintext:", plaintext) When I run the above code, I get the following key type error: TypeError: memoryview: a bytes-like object is required, not 'SecretKey' While I understand that the AES Cipher() method doesn't like the key format, I don't know how to convert the key from session.generate_key() method to a format that's acceptable. I also tried using the key[Attribute.VALUE] but no luck! Thanks in advance! Best, PE

3 2

KDF Question
by Sriram R Jan. 12, 2025

Jan. 12, 2025

Hello, I'm generating a key using Scrypt from a password supplied by the user. I then use this key as follows. This works but my question to the experts: is this an acceptable way to use the AESGCMSIV cipher? Am I doing something that's fundamentally against best practices? In the examples on the Cryptography site, the sample code snippet uses AESGCMSIV.generate_key() method to generate the key instead. The requirement I have is to generate the key that's based on a password. def gen_salt(size=32): return secrets.token_bytes(size) def der_key(salt, password): kdf = Scrypt(salt=salt, length=32, n=2**20, r=8, p=1) return kdf.derive(password.encode()) def gen_symmkey(salt, password): symmkey = der_key(salt, password) return symmkey key = gen_symmkey(salt, password) aesgcmsiv = AESGCMSIV(key) ct = aesgcmsiv.encrypt(nonce, file_data, aad) with open(fname, "wb") as outfile: outfile.write(ct) Best, PE

3 12

Re: [Cryptography-dev] AESGCMSIV
by Alex Gaynor Jan. 4, 2025

Jan. 4, 2025

It's correct that nonces do not need to be secret. Alex On Sat, Jan 4, 2025 at 5:56 PM Sriram R <sramacha2(a)gmail.com> wrote: > > Alex, > > Found the flaw in the code logic, it had to do with the location of the nonce file. I was storing the nonce file in the same directory that was getting encrypted/decrypted. > Once I changed the nonce file location to be outside the target directory, the encryption and decryption works flawlessly now. > > One last question, I read that the nonce file contents are not sensitive and it doesn't need to be encrypted. Is that your understanding also? > > Best, > PE > > On Sat, Jan 4, 2025 at 2:37 PM Sriram R <sramacha2(a)gmail.com> wrote: >> >> Fair point, here's the entire code: >> >> import cryptography >> from cryptography.fernet import Fernet >> from cryptography.hazmat.primitives.kdf.scrypt import Scrypt >> import secrets, base64, getpass, os, os.path, argparse >> >> from cryptography.hazmat.primitives.ciphers.aead import AESGCMSIV >> >> from pathlib import Path >> >> from collections import defaultdict, OrderedDict >> >> aad = b"super secret authenticated but unencrypted data" >> >> rootpath = "/tmp/cstore/" >> #rootpath = "/tmp/certstore/" >> #dirs = ['certs/', 'csrs/', 'keys/'] >> dirs = ['csrs/'] >> files = [] >> >> saltfile = Path.home().joinpath("salt.salt") >> keyfile = Path.home().joinpath(".key") >> saltsize = int(32) >> >> def lsdir(path): >> for nodes in Path(path).iterdir(): >> if nodes.is_dir(): >> #print("Dir is: ", nodes) >> lsdir(nodes) >> else: >> #print("File is: ", nodes) >> files.append(nodes) >> >> def gen_salt(size=32): >> return secrets.token_bytes(size) >> >> def der_key(salt, password): >> kdf = Scrypt(salt=salt, length=32, n=2**20, r=8, p=1) >> return kdf.derive(password.encode()) >> >> def gen_symmkey(salt, password): >> symmkey = der_key(salt, password) >> return symmkey >> #return base64.urlsafe_b64encode(symmkey) >> >> def encrypt(nonce, fname, aesgcmsiv): >> with open(fname, "rb") as infile: >> file_data = infile.read() >> >> # encrypt data >> edata = aesgcmsiv.encrypt(nonce, file_data, aad) >> with open(fname, "wb") as outfile: >> outfile.write(edata) >> >> >> def decrypt(nonce, fname, aesgcmsiv): >> with open(fname, "rb") as infile: >> edata = infile.read() >> >> try: >> data = aesgcmsiv.decrypt(nonce, edata, aad) >> except cryptography.exceptions.InvalidTag: >> print("something went wrong") >> return False >> >> with open(fname, "wb") as outfile: >> outfile.write(data) >> >> def call_crypto(saltsize, password): >> if os.path.isfile(saltfile): >> salt = open(saltfile, 'rb').read() >> else: >> salt = gen_salt(saltsize) >> with open(saltfile, 'wb') as sfile: >> sfile.write(salt) >> >> key = gen_symmkey(salt, password) >> aesgcmsiv = AESGCMSIV(key) >> >> # dir walk and store files to process in a list >> lsdir(rootpath) >> >> # encrypt/decrypt all elements in the 'files' list >> if ENC: >> for file in files: >> nonce = os.urandom(12) >> encrypt(nonce, file, aesgcmsiv) >> #nfile = Path.joinpath(file, ".nonce") >> nfile = "/tmp/cstore/csrs/eud1.csr.nonce" >> with open(nfile, 'wb') as f: >> f.write(nonce) >> else: >> for file in files: >> #nfile = Path.joinpath(file, ".nonce") >> nfile = "/tmp/cstore/csrs/eud1.csr.nonce" >> with open(nfile, 'rb') as f: >> nonce = f.read() >> decrypt(nonce, file, aesgcmsiv) >> >> if __name__ == "__main__": >> >> password = some_password >> >> ENC = False >> call_crypto(saltsize, password) >> >> Best, >> PE >> >> On Sat, Jan 4, 2025 at 2:32 PM Sriram R <sramacha2(a)gmail.com> wrote: >>> >>> Fair point, here's the entire code: >>> >>> [certadmin@r9kc01-silver ~]$ !source >>> source ./testvenv/bin/activate >>> (testvenv) [certadmin@r9kc01-silver ~]$ vi ./gcm.py >>> (testvenv) [certadmin@r9kc01-silver ~]$ cat !$ >>> cat ./gcm.py >>> import cryptography >>> from cryptography.fernet import Fernet >>> from cryptography.hazmat.primitives.kdf.scrypt import Scrypt >>> import secrets, base64, getpass, os, os.path, argparse >>> >>> from cryptography.hazmat.primitives.ciphers.aead import AESGCMSIV >>> >>> from pathlib import Path >>> >>> from collections import defaultdict, OrderedDict >>> >>> aad = b"super secret authenticated but unencrypted data" >>> >>> rootpath = "/tmp/cstore/" >>> #rootpath = "/tmp/certstore/" >>> #dirs = ['certs/', 'csrs/', 'keys/'] >>> dirs = ['csrs/'] >>> files = [] >>> >>> saltfile = Path.home().joinpath("salt.salt") >>> keyfile = Path.home().joinpath(".key") >>> saltsize = int(32) >>> >>> def lsdir(path): >>> for nodes in Path(path).iterdir(): >>> if nodes.is_dir(): >>> #print("Dir is: ", nodes) >>> lsdir(nodes) >>> else: >>> #print("File is: ", nodes) >>> files.append(nodes) >>> >>> def gen_salt(size=32): >>> return secrets.token_bytes(size) >>> >>> def der_key(salt, password): >>> kdf = Scrypt(salt=salt, length=32, n=2**20, r=8, p=1) >>> return kdf.derive(password.encode()) >>> >>> def gen_symmkey(salt, password): >>> symmkey = der_key(salt, password) >>> return symmkey >>> #return base64.urlsafe_b64encode(symmkey) >>> >>> def encrypt(nonce, fname, aesgcmsiv): >>> with open(fname, "rb") as infile: >>> file_data = infile.read() >>> >>> # encrypt data >>> edata = aesgcmsiv.encrypt(nonce, file_data, aad) >>> with open(fname, "wb") as outfile: >>> outfile.write(edata) >>> >>> >>> def decrypt(nonce, fname, aesgcmsiv): >>> with open(fname, "rb") as infile: >>> edata = infile.read() >>> >>> try: >>> data = aesgcmsiv.decrypt(nonce, edata, aad) >>> except cryptography.exceptions.InvalidTag: >>> print("something went wrong") >>> return False >>> >>> with open(fname, "wb") as outfile: >>> outfile.write(data) >>> >>> def call_crypto(saltsize, password): >>> if os.path.isfile(saltfile): >>> salt = open(saltfile, 'rb').read() >>> else: >>> salt = gen_salt(saltsize) >>> with open(saltfile, 'wb') as sfile: >>> sfile.write(salt) >>> >>> key = gen_symmkey(salt, password) >>> aesgcmsiv = AESGCMSIV(key) >>> >>> #print('key is: ', key) >>> >>> # dir walk and store files to process in a list >>> lsdir(rootpath) >>> >>> # encrypt/decrypt all elements in the 'files' list >>> if ENC: >>> for file in files: >>> nonce = os.urandom(12) >>> encrypt(nonce, file, aesgcmsiv) >>> nfile = "/tmp/cstore/csrs/eud1.red.ftf.net.csr.nonce" >>> with open(nfile, 'wb') as f: >>> f.write(nonce) >>> else: >>> for file in files: >>> #nfile = Path.joinpath(file, ".nonce") >>> nfile = "/tmp/cstore/csrs/eud1.red.ftf.net.csr.nonce" >>> with open(nfile, 'rb') as f: >>> nonce = f.read() >>> decrypt(nonce, file, aesgcmsiv) >>> >>> if __name__ == "__main__": >>> >>> password = some_password >>> >>> ENC = False >>> call_crypto(saltsize, password) >>> >>> >>> Best, >>> PE >>> >>> On Sat, Jan 4, 2025 at 1:25 PM Alex Gaynor <alex.gaynor(a)gmail.com> wrote: >>>> >>>> It's impossible to say without seeing your code. If >>>> encrypt()/decrypt() isn't working, then some of your parameters don't >>>> match between them. >>>> >>>> Alex >>>> >>>> On Sat, Jan 4, 2025 at 4:11 PM Sriram R <sramacha2(a)gmail.com> wrote: >>>> > >>>> > Also, I'm saving and reading the ct and the nonce values in binary mode during encryption and decryption. Not sure if that needs to be done using a codec if the read/write operation is damaging the values. >>>> > >>>> > Appreciate taking the time to respond to this issue. >>>> > >>>> > >>>> > On Sat, Jan 4, 2025, 12:52 PM Sriram R <sramacha2(a)gmail.com> wrote: >>>> >> >>>> >> Yes, all the other kdf parameters are the same. Like the salt size, n, p, r values etc. >>>> >> Any ideas as to why the decryption fails? >>>> >> I'm passing the same nonce to the .decrypt() method. >>>> >> >>>> >> >>>> >> On Sat, Jan 4, 2025, 12:46 PM Alex Gaynor <alex.gaynor(a)gmail.com> wrote: >>>> >>> >>>> >>> Assuming the salt, and all other parameters passed to the KDF are the >>>> >>> same, then yes. >>>> >>> >>>> >>> On Sat, Jan 4, 2025 at 3:34 PM Sriram R <sramacha2(a)gmail.com> wrote: >>>> >>> > >>>> >>> > Appreciate the symmetric key explanation. If the key is derived using a password based kdf during encryption, isn't it reasonable to expect the same key would be generated by the kdf during decryption phase if the password >>>> >>> > is also the same? >>>> >>> > >>>> >>> > PE >>>> >>> > >>>> >>> > >>>> >>> > On Sat, Jan 4, 2025, 9:30 AM Alex Gaynor <alex.gaynor(a)gmail.com> wrote: >>>> >>> >> >>>> >>> >> Based on "it creates a new key from the password and is used for >>>> >>> >> decryption" it sounds like you're using different keys for encryption >>>> >>> >> and decryption. >>>> >>> >> >>>> >>> >> You need to use the same key to decrypt a value as you used to encrypt >>>> >>> >> it -- this is the point of symmetric encryption! It wouldn't make much >>>> >>> >> sense if you could decrypt a value without possessing the key. >>>> >>> >> >>>> >>> >> How exactly you manage your keys depends a lot on your application, >>>> >>> >> deployment environment, and threat model, so I can't really provide >>>> >>> >> any general purpose advice. You generally should not store the key >>>> >>> >> with the ciphretext, as there's basically no threat model that makes >>>> >>> >> sense under. >>>> >>> >> >>>> >>> >> Alex >>>> >>> >> >>>> >>> >> On Sat, Jan 4, 2025 at 12:28 PM Sriram R via Cryptography-dev >>>> >>> >> <cryptography-dev(a)python.org> wrote: >>>> >>> >> > >>>> >>> >> > Hello, >>>> >>> >> > >>>> >>> >> > I'm reaching out to the cryptography experts for this issue. >>>> >>> >> > >>>> >>> >> > Using Python 3.9 on a RHEL 9.4 platform with the cryptography module installed on it. >>>> >>> >> > The requirements are to encrypt some data using classcryptography.hazmat.primitives.ciphers.aead.AESGCM(key) >>>> >>> >> > >>>> >>> >> > I've written the encryption code to use the above cipher and it works without any issues. >>>> >>> >> > >>>> >>> >> > The issue I'm running into is in the decryption step. The same Python code is called with the flag set to decryption but it raises the InvalidToken exception. >>>> >>> >> > Reading the documentation it says this could occur if the ciphertext is changed, which it is not. Or if the nonce is different, I adjusted the code to save each nonce during the encryption process and I supply this same nonce during decryption. >>>> >>> >> > The documentation also says this exception can occur if the key is different. >>>> >>> >> > >>>> >>> >> > This is where my confusion is, the .decrypt() only takes in the ct, nonce and aad parameters. >>>> >>> >> > Obviously when the Python module is run for decryption, it creates a new key from the password and is used for decryption. Which will be different than what was used during encryption. >>>> >>> >> > I don't think the solution is to also store the key along with the nonce, since it's a very bad security practice. >>>> >>> >> > >>>> >>> >> > What am I missing? >>>> >>> >> > Do I need to parse the ciphertext and extract the tag or something? But then .decrypt() method doesn't use a tag parameter. >>>> >>> >> > >>>> >>> >> > FYI, I'm saving the ciphertext and the nonce with the 'wb' flag in the open() statement and the decryption step uses the 'rb' flag to read in the ciphertext and the nonce. >>>> >>> >> > >>>> >>> >> > The example code snippet in the cryptography website shows the encrypt and decrypt operation in sequence. >>>> >>> >> > In my situation, encryption happens at one time and the decryption has to happen some time later. >>>> >>> >> > >>>> >>> >> > Several websites/users are using the Pycryptodome module for this. >>>> >>> >> > I'd rather not because the cryptography module works great for my other requirements. >>>> >>> >> > Also I'd prefer not to use cryptography module and the Pycryptodome module as well. I think that would be overkill. >>>> >>> >> > >>>> >>> >> > Best >>>> >>> >> > PE >>>> >>> >> > _______________________________________________ >>>> >>> >> > Cryptography-dev mailing list >>>> >>> >> > Cryptography-dev(a)python.org >>>> >>> >> > https://mail.python.org/mailman/listinfo/cryptography-dev >>>> >>> >> >>>> >>> >> >>>> >>> >> >>>> >>> >> -- >>>> >>> >> All that is necessary for evil to succeed is for good people to do nothing. >>>> >>> >>>> >>> >>>> >>> >>>> >>> -- >>>> >>> All that is necessary for evil to succeed is for good people to do nothing. >>>> >>>> >>>> >>>> -- >>>> All that is necessary for evil to succeed is for good people to do nothing. -- All that is necessary for evil to succeed is for good people to do nothing.

2 1

Re: [Cryptography-dev] AESGCMSIV
by Alex Gaynor Jan. 4, 2025

Jan. 4, 2025

It's impossible to say without seeing your code. If encrypt()/decrypt() isn't working, then some of your parameters don't match between them. Alex On Sat, Jan 4, 2025 at 4:11 PM Sriram R <sramacha2(a)gmail.com> wrote: > > Also, I'm saving and reading the ct and the nonce values in binary mode during encryption and decryption. Not sure if that needs to be done using a codec if the read/write operation is damaging the values. > > Appreciate taking the time to respond to this issue. > > > On Sat, Jan 4, 2025, 12:52 PM Sriram R <sramacha2(a)gmail.com> wrote: >> >> Yes, all the other kdf parameters are the same. Like the salt size, n, p, r values etc. >> Any ideas as to why the decryption fails? >> I'm passing the same nonce to the .decrypt() method. >> >> >> On Sat, Jan 4, 2025, 12:46 PM Alex Gaynor <alex.gaynor(a)gmail.com> wrote: >>> >>> Assuming the salt, and all other parameters passed to the KDF are the >>> same, then yes. >>> >>> On Sat, Jan 4, 2025 at 3:34 PM Sriram R <sramacha2(a)gmail.com> wrote: >>> > >>> > Appreciate the symmetric key explanation. If the key is derived using a password based kdf during encryption, isn't it reasonable to expect the same key would be generated by the kdf during decryption phase if the password >>> > is also the same? >>> > >>> > PE >>> > >>> > >>> > On Sat, Jan 4, 2025, 9:30 AM Alex Gaynor <alex.gaynor(a)gmail.com> wrote: >>> >> >>> >> Based on "it creates a new key from the password and is used for >>> >> decryption" it sounds like you're using different keys for encryption >>> >> and decryption. >>> >> >>> >> You need to use the same key to decrypt a value as you used to encrypt >>> >> it -- this is the point of symmetric encryption! It wouldn't make much >>> >> sense if you could decrypt a value without possessing the key. >>> >> >>> >> How exactly you manage your keys depends a lot on your application, >>> >> deployment environment, and threat model, so I can't really provide >>> >> any general purpose advice. You generally should not store the key >>> >> with the ciphretext, as there's basically no threat model that makes >>> >> sense under. >>> >> >>> >> Alex >>> >> >>> >> On Sat, Jan 4, 2025 at 12:28 PM Sriram R via Cryptography-dev >>> >> <cryptography-dev(a)python.org> wrote: >>> >> > >>> >> > Hello, >>> >> > >>> >> > I'm reaching out to the cryptography experts for this issue. >>> >> > >>> >> > Using Python 3.9 on a RHEL 9.4 platform with the cryptography module installed on it. >>> >> > The requirements are to encrypt some data using classcryptography.hazmat.primitives.ciphers.aead.AESGCM(key) >>> >> > >>> >> > I've written the encryption code to use the above cipher and it works without any issues. >>> >> > >>> >> > The issue I'm running into is in the decryption step. The same Python code is called with the flag set to decryption but it raises the InvalidToken exception. >>> >> > Reading the documentation it says this could occur if the ciphertext is changed, which it is not. Or if the nonce is different, I adjusted the code to save each nonce during the encryption process and I supply this same nonce during decryption. >>> >> > The documentation also says this exception can occur if the key is different. >>> >> > >>> >> > This is where my confusion is, the .decrypt() only takes in the ct, nonce and aad parameters. >>> >> > Obviously when the Python module is run for decryption, it creates a new key from the password and is used for decryption. Which will be different than what was used during encryption. >>> >> > I don't think the solution is to also store the key along with the nonce, since it's a very bad security practice. >>> >> > >>> >> > What am I missing? >>> >> > Do I need to parse the ciphertext and extract the tag or something? But then .decrypt() method doesn't use a tag parameter. >>> >> > >>> >> > FYI, I'm saving the ciphertext and the nonce with the 'wb' flag in the open() statement and the decryption step uses the 'rb' flag to read in the ciphertext and the nonce. >>> >> > >>> >> > The example code snippet in the cryptography website shows the encrypt and decrypt operation in sequence. >>> >> > In my situation, encryption happens at one time and the decryption has to happen some time later. >>> >> > >>> >> > Several websites/users are using the Pycryptodome module for this. >>> >> > I'd rather not because the cryptography module works great for my other requirements. >>> >> > Also I'd prefer not to use cryptography module and the Pycryptodome module as well. I think that would be overkill. >>> >> > >>> >> > Best >>> >> > PE >>> >> > _______________________________________________ >>> >> > Cryptography-dev mailing list >>> >> > Cryptography-dev(a)python.org >>> >> > https://mail.python.org/mailman/listinfo/cryptography-dev >>> >> >>> >> >>> >> >>> >> -- >>> >> All that is necessary for evil to succeed is for good people to do nothing. >>> >>> >>> >>> -- >>> All that is necessary for evil to succeed is for good people to do nothing. -- All that is necessary for evil to succeed is for good people to do nothing.

1 0

AESGCMSIV
by Sriram R Jan. 4, 2025

Jan. 4, 2025

Hello, I'm reaching out to the cryptography experts for this issue. Using Python 3.9 on a RHEL 9.4 platform with the cryptography module installed on it. The requirements are to encrypt some data using *class* cryptography.hazmat.primitives.ciphers.aead.AESGCM(*key*) I've written the encryption code to use the above cipher and it works without any issues. The issue I'm running into is in the decryption step. The same Python code is called with the flag set to decryption but it raises the InvalidToken exception. Reading the documentation it says this could occur if the ciphertext is changed, which it is not. Or if the nonce is different, I adjusted the code to save each nonce during the encryption process and I supply this same nonce during decryption. The documentation also says this exception can occur if the key is different. This is where my confusion is, the .decrypt() only takes in the ct, nonce and aad parameters. Obviously when the Python module is run for decryption, it creates a new key from the password and is used for decryption. Which will be different than what was used during encryption. I don't think the solution is to also store the key along with the nonce, since it's a very bad security practice. What am I missing? Do I need to parse the ciphertext and extract the tag or something? But then .decrypt() method doesn't use a tag parameter. FYI, I'm saving the ciphertext and the nonce with the 'wb' flag in the open() statement and the decryption step uses the 'rb' flag to read in the ciphertext and the nonce. The example code snippet in the cryptography website shows the encrypt and decrypt operation in sequence. In my situation, encryption happens at one time and the decryption has to happen some time later. Several websites/users are using the Pycryptodome module for this. I'd rather not because the cryptography module works great for my other requirements. Also I'd prefer not to use cryptography module and the Pycryptodome module as well. I think that would be overkill. Best PE

2 2