Wednesday, August 13, 2014

Wizards of the Coast Broke the Internet Wayback Machine

Attempts to use The Internet Wayback Machine to search are now broken.

Here, try it:*/

Where it used to give a huge listing of files, there is now a "cannot be displayed because, robots" message.

I think this changed happened when Wizards of the Coast updated the D&D website to the god-awful navigational mess that is the new D&D site. They introduced a robots.txt file which excludes websites from crawling certain directories.

This represents a major break with the ability to search & retrieve old material from the domain in an easy-to-navigate interface.

Site-specific Google searches still work, but only bring up current content, not older, "disappeared" material.

Luckily, I have ALL the free 3.0 & 3.5 free adventure PDFs downloaded. I even managed to snag the d20 Modern and Star Wars d20 & SAGA freebies (well, most of the Star Wars stuff). This weighs in at roughly 1GB of data. That's a lot of free material Wizards released over the d20 years.

I'm not in the mind to check all of the links to free 4e material that I had previously compiled. I think most of it is still accessible, but it most likely won't be for long.

Also, the new CMS that is using to manage the updated D&D site hosts most of its downloads on They do not have a consistent file directory system or naming scheme for files, either. I'll be posting a 5e resource list soon.