I'll have to remember to try tracerouting in my troubleshooting. I made the mistake of assuming that because it happened right at the backup that it was an issue with the server.
Thank goodness if it's an ISP issue. At least we know then that the server is healthy.