Import encrypted csv into Python 3
So I am planning to do some data analysis using a Jupyter notbook (Python 3), for collaborative reasons I want to store the data on a github repo, however the data set is sensitive.
As such I would like to store the data (currently .csv) as an encrypted file on the repo and then decrypt it at runtime (with a password prompt I guess).
What is the best method to do this?
One Solution collect form web for “Import encrypted csv into Python 3”
In the end, I used python 3.6 and SimpleCrypt to encrypt the file and then uploaded it.
I think this is the code I used to encrypt the file:
f = open('file.csv','r').read() ciphertext = encrypt('USERPASSWORD',f.encode('utf8'))#this .encode('utf8') is the bit im unsure about e = open('file.enc','wb') # file.enc doesn't need to exist, python will create it e.write(ciphertext) e.close
This is the code I use to decrypt at runtime, I run
getpass("password: ") as an argument so I don’t have to store a
password variable in memory
from io import StringIO import pandas as pd from simplecrypt import encrypt, decrypt from getpass import getpass # opens the file f = open('file.enc','rb').read() print('Please enter the password and press the enter key \n Decryption may take some time') # Decrypts the data, requires a user-input password CSVplaintext = decrypt(getpass("password: "), f).decode('utf8') print('Data have been Decrypted') #create a temp csv-like file to pass to pandas.read_csv() DATA=StringIO(CSVplaintext) # Makes a panda dataframe with the data df = pd.read_csv(DATA)
Note, the UTF-8 encoding behaviour is different in python 2.7 so the code will be slightly different.