Googlebot soon able to crawl all JavaScript
If we look back to when Google first started way back in the late 90’s, the web was a very different place.
JavaScript and CSS weren’t a real worry for crawlers and usage of these languages was minimal. JavaScript for example, was written off as a ‘simple’ language. At this point JavaScript wasn’t used within page elements like it is today so they didn’t have to worry about crawling it for valuable content.
As the web grew we started seeing content being rendered by JavaScript and due to limitations with crawling, Google struggled with displaying content such as this within their search results, meaning users weren’t able to find the right information. This presented a problem for Google and ever since they’ve been attempting to correctly render JavaScript for many years with varied success. Originally they tried mashing together bits of JavaScript they found, which wasn’t too successful. However, as they’ve grown, so has their ability to render more and more JavaScript.
We’ve recently seen a number of updates to Webmaster Tools, providing us with the ability to render our pages as Googlebot (or Fetch as Google), and see potential breakpoints where spiders could be having serious problems crawling our sites.
Above: (A render of the Parallax site in Fetch as Google)
Google have been working hard to get Googlebot fully rendering JavaScript and at a recent AngularJS Meetup in New York, Brad Green answered a question regarding crawl-ability of JavaScript (particularly AngularJS). He said that by the end of the year, Googlebot will be able to render all JavaScript and the tool within Webmaster Tools will show exactly what they look like. We are told that will work across all JavaScript platforms and frameworks, from backbone.js to AngularJS.
Check out the video below. Flick to 41:00 where there’s an interesting question from the audience.
What are your thoughts on this? Will it have a real effect on SEO? Feel free to share with us in the comments below or get in touch on Twitter