Hash a file in Python
Hash a file with hashlib.file_digest in Python 3.11 and later, which reads in chunks. A read() call loads the whole file into memory first.
You need the SHA-256 of a file that may be larger than memory, and the checksum has to match sha256sum.
import hashlib
with open("data.bin", "wb") as f:
f.write(b"hello world\n" * 1000)
with open("data.bin", "rb") as f:
print(hashlib.file_digest(f, "sha256").hexdigest())
h = hashlib.sha256()
with open("data.bin", "rb") as f:
for chunk in iter(lambda: f.read(1 << 20), b""):
h.update(chunk)
print(h.hexdigest())555812d3df91c5390f2211d729e58eb2005369934ac2ba69eb42913f897f7a7e
555812d3df91c5390f2211d729e58eb2005369934ac2ba69eb42913f897f7a7ehashlib.file_digest, added in Python 3.11, reads the file object in chunks and returns the hash object, so memory use stays flat, as the hashlib documentation explains. The loop below it does the same by hand and is needed on earlier versions. Both print the same digest, which is the one sha256sum data.bin gives.
The file must be opened in binary mode, because the digest is of bytes. Do not use MD5 or SHA-1 to check that a download was not tampered with. For a hash that is compared with a published value, read the value from the same channel the file came from, over HTTPS.