.Google.com's Gary Illyes uncovers Googlebot picks up links as opposed to "following" all of them, negating main documents.Googlebot gathers links instead of following all of them in real-time.Gary Illyes discussed this details in Google's 'Search Off The File' podcast.Google.com's information contradicts the real crawler actions.