Get the latest digital marketing news, views and
more direct to your inbox
Sep 16th, 2015
In a Google+ Hangout on September 11, Webmaster Trends Analyst John Mueller confirmed that Googlebots do submit forms — but only in certain situations.
“Submitting forms” means that spiders enter a query into a search box (either in a website or in a search engine) then crawl through the links that are generated. It became a big issue in 2011 when Google accused Bing of doing this on Google-generated links, effectively “cheating” and copying Google’s search results. (Bing denied the accusation and said they were taking it as a “back-handed compliment”.)
Mueller clarified that Google only submits forms when they come across pages that look like search forms and suspect that they’re not “getting all of the content that we suspect is available on this website”. When this happens, Google will take keywords from the page’s existing content and do a search to see if they can find more relevant information on the site. Mueller stressed that this is “the really rare situation”.
He added: “In practice, those forms don’t lead us to new or interesting content. Search forms sometimes do lead us to content that we can’t otherwise find, especially if a site doesn’t have any navigation, doesn’t have a sitemap file. If we see that maybe the homepage just has a search form, and we suspect there is a lot of really good content there, but we can’t really reach it in our crawl, so that is one situation where our algorithms might say well, we know this site is about this topic, let’s plug in a keyword and see what we find.”
Share this article!