SHA-256 in Bash and Python – A speed comparison

When experimenting with generating Bitcoin private key and public address pairs, also widely known as brainwallet cracking (which is a subject that we will revisit many times), what you do is computing SHA-256 hashes of words, passwords or passphrases. You then take the hashes, which are conveniently 64 character hexadecimal strings (32 bytes or 256 bits, if you will), and convert them to Bitcoin private keys and their corresponding public addresses. Then you look up the public addresses on the blockchain, and if you are lucky you will get a few hits.

An infamous example is (not discovered by us, we will report our private findings in due time):

As you can see on the blockchain, this public address has received more than 10 BTC. Should it be funded again, you now have its private key!

No but seriously… In order to engage in these endeavors, you will need to test hashes of large wordlists (and for any realistic chance, lists that no one else has access to). Let’s begin with downloading a list of nearly 470 thousand English words (about 4.6 MB). In your root folder, type:

wget https://github.com/dwyl/english-words/raw/master/words.txt

Good, you now have a wordlist called “words.txt” to work with. It should be exactly 466,551 lines (you can count the lines of a text file with the “wc -l filename” command), and contain:

Let’s move on and create two separate scripts, one in Bash and one in Python, that produce the exact same outputs.

Here is the Bash script – copy and save it as “sha256-bash”

#!/usr/bin/env bash
#Reads a file line by line and outputs the corresponding SHA-256 hashes

if [ -z "$1" ]; then
  echo "Error: No input file specified"
else
  while IFS= read -r line; do
    line=$(echo -n "$line" | tr -d '\r\n' | sha256sum | cut -c1-64)
    echo "$line"
  done < "$1"
fi

And here is the Python script – copy and save it as “sha256-python”

#!/usr/bin/env python3
#Reads a file line by line and outputs the corresponding SHA-256 hashes

import sys, hashlib

try:
  filename=sys.argv[1]
except IndexError:
  print('Error: No input file specified')
  sys.exit()

f = open(filename, 'r')

for line in f:
  line = line.replace('\n', '').replace('\r', '')
  sha256 = hashlib.sha256(line.encode('utf-8')).hexdigest()
  print(sha256)

They should be straightforward and quite self-explicatory. As usual, make these executable by running

chmod +x sha256-bash
chmod +x sha256-python

Great, now we have all in place to get to work. Let’s begin with the Python script. Also, bring in the “time” command, so we know how long the operation takes. We will pipe the output to “hashes.txt”. In other words, run:

time ./sha256-python words.txt > hashes-python.txt

How long did it take you to calculate are print out almost 470,000 lines of hashes? On my several-year-old notebook (4 cores, 8 GB RAM) it took 1.4 seconds. Not too shabby! How do you think Bash will do? Try:

time ./sha256-bash words.txt > hashes-bash.txt

I suggest you go make yourself a cup of coffee or something and then take out the trash while waiting for it to finish. My computer needed 21 minutes and 22 seconds to finish.

The content of the output files should be

For perfection, let’s verify that the two output files are bit-by-bit identical, run (hint: save this command somewhere, it is fast and useful)

cmp -s hashes-python.txt hashes-bash.txt && echo 'SUCCESS: Files are identical' || echo 'WARNING: Files not identical'

which will result in “SUCCESS: Files are identical”. Good.

Let’s do some numbers. 21 minutes is 21*60 = 1260 + 22 = 1288 seconds. And 1288 / 1.4 = 920

What have we learned here? Well, in conclusion, we have shown that file processing and hash calculations are almost 1,000 times faster in Python than in Bash. I’d say we have a very clear winner. The moral of the story is that for batch processing, you have to master Python. Or you will die of a coffee overdose while waiting for your Bash scripts to complete.

Python it is!

Comments or questions?

• Category: Scripts
• Tag: bash, hacking, python, security, sha-256