TL;DR
Skip to the end if you just want the tool to do this yourself, otherwise, enjoy the story about how we debugged a nagging network issue.
Corruption!
So, we’ve been experiencing some annoying network issues lately and we were almost positive it was our ISP (Comcast). The issue would occur on large files (think major OS updates, video on demand, music downloads, etc). Basically we would perform our 20 minute download and get stuck with a corrupted file. Worse, we would pay for a VOD movie and start to watch it only to find it is a glitchy mess with intermittent sound.
Yikes! Something had to be done.
How hard can it be?
But first — a little about our network configuration. We have a Comcast cable modem that goes to our main firewall router that then gets distributed throughout our house via switches and wireless access points. The initial hint was that we saw this corruption on both the wired (Tivo) and wireless downloads.
The worst part of all this is that the problem was intermittent. Sometimes large files would work and other times they would not. I needed data if I was going to convince Comcast that this was their problem (note the negative bias).
In the beginning
I started by manually downloaded the large file. In this case, I leaned heavily on my buddies in Cupertino and one of their 1.1GB major software updates. Thankfully, their update site provides a SHA1 checksum. The process was this:
- Note the start time
- Download said file
- Note the end time
- Open a terminal window and type “shasum /path/to/filename.dmg”
- Make a note in some file whether or not the download was corrupt.
Did I mention
Oh yeah, each one of these downloads took me 6-7 minutes and I have kids now. This manual data gathering was becoming a pain. I needed relief and automation so that I could run these tests unattended all night long and gather the precious data in all the configurations and locations of our network.
Thank you Guido
Enter python and 30 minutes of time. My son, Leo, thankfully cooperated by not injuring himself during those precious 30 minutes. My daughter, Dagny, also helped out by taking a nice long nap. The stars were aligned and all was go.
The script
So, I knocked out a quick and dirty little python script that does the above steps while also outputs the notes and results into a csv file. Don’t worry, the downloaded GB file is deleted every time. A link to the script is below (NOTE: You will need to tweak the download URL and checksum to suit your needs and desires)
The results
So, after all the work, what does the CSV file look like when imported into Excel?
Output from the python downloader script with some conditional formatting applied.
And the Winner is………
Surpise! It was not Comcast! In the end, I was able to trace the corruption to our Netgear FVS318G router which was running some old firmware (3.0.5 or something). After installing the latest firmware (3.1.1+) the problem vanished.
The lesson here — it’s not always Comcast’s fault and it definitely helps to throw a little python around now and then.
Leave a Reply