Wednesday, August 13, 2014

Wizards of the Coast Broke the Internet Wayback Machine

Attempts to use The Internet Wayback Machine to search are now broken.

Here, try it:*/

Where it used to give a huge listing of files, there is now a "cannot be displayed because, robots" message.

I think this changed happened when Wizards of the Coast updated the D&D website to the god-awful navigational mess that is the new D&D site. They introduced a robots.txt file which excludes websites from crawling certain directories.

This represents a major break with the ability to search & retrieve old material from the domain in an easy-to-navigate interface.

Site-specific Google searches still work, but only bring up current content, not older, "disappeared" material.

Luckily, I have ALL the free 3.0 & 3.5 free adventure PDFs downloaded. I even managed to snag the d20 Modern and Star Wars d20 & SAGA freebies (well, most of the Star Wars stuff). This weighs in at roughly 1GB of data. That's a lot of free material Wizards released over the d20 years.

I'm not in the mind to check all of the links to free 4e material that I had previously compiled. I think most of it is still accessible, but it most likely won't be for long.

Also, the new CMS that is using to manage the updated D&D site hosts most of its downloads on They do not have a consistent file directory system or naming scheme for files, either. I'll be posting a 5e resource list soon.

1 comment:

reynomatt said...

Been searching the internet for the last two days for the d20 Modern adventures. It sounds like I've been following the same digital investigation you did. Are you willing to share the files by any chance?