webcrawler

Ancient Web: Matthew Gray Counted the Web Before There Were a Thousand Sites

Matthew Gray's MIT Internet Statistics page preserves his 1993 World Wide Web Wanderer research, an early attempt to count websites with a wandering robot.