I have been active in the Internet space since 2006, and have been tracking Google’s search engine evolution very closely since about 2008. I am sure most people would agree that the years 2011 and 2012 have witnessed tremendous advancements in Google’s search engine ranking algorithms, the now famous Panda and Penguin updates being the highlights. I would like to explain my understanding of these changes, and what that means for sites and what this means for traffic from Google going forward.

Google’s crawling infrastructure has seen great advancements in the last 2-3 years, with Caffeine they are able to crawl an amazing amount of the web every day. This gives them a unique ability to stay on top of the content on the web, and to improve the quality of search results they give to users. This along with a couple of smart algorithm updates, and access to more user data has led to these improvements.

Uptil 2011, it was relatively easy to game Google’s algorithm.  If you had a trusted site, with good quality of links coming in, you could add millions of low quality pages to your site to target traffic from the long tail of search queries where it was easy to rank since there wasn’t enough content targeting these searches. Google, I’m sure, was for a long time looking for reliable signals to measure the overall quality of a website. From everything I’ve heard from Google on the Panda algorithm update – my understanding is that they have managed to find relatively ‘noiseless’ and reliable signals based on which they are able to determine overall how much a user or a group of users likes a particular site (domain or subdomain). Google has said that they use indicators such as how often a user goes back to the site, whether they bookmark it etc, to use as a proxy for the quality of pages on the site. I would entertain a guess that data from chrome is involved, also they are probably measuring mean time before the user returns to the search results as an indicator of the extent to which the user is happy with the result. i.e. if the user intent matches with the content on the page he lands up on, it’s a win for all, the user, the search engine as well as the site. This is the main aim for a search engine, and Panda has gone a long way in making this happen more. The beauty is that even there are some good pages on the site, a large number of low quality pages can bring down traffic to the entire site. How Panda works is – it looks at a certain cutoff after taking in all the data – sites on one side of the cutoff are not impacted – sites on the other side have their rankings reduced across the domain – and sites somewhat in the middle – and impacted, but not to a great extent.

Next up was Penguin – while Panda targeted at low quality content and low quality sites, Penguin was targeted at link spam and other spammy SEO techniques. Although detecting paid links is a huge task, Google is getting better at it – they need to get to a point where people would be scared to buy links. But with Penguin, Google targeted sites that were clearly buying a lot of links, or their anchor text profile was skewed towards non-brand, money keywords anchor texts. The intent here is to move the SEO community from ‘link building’ to ‘link earning’.

Indeed, it is much easier to attract links naturally if you have great compelling content, and some amount of link building in the form of – ‘hey, I have some great content here relevant to your site, would you like to link to me?’ is very important to do. Google has a great deal of influence in the direction the web takes, and getting people to create great content versus spending time on gaming the system would be a big plus.

I wish Google had the ability to clean up low quality a while ago. Then everyone would have not had it so easy. The writing is on the wall – focus on creating high quality, unique, relevant content – and a fantastic user experience on the site – and sooner or later, everything else will be demoted in search results. This is infact brilliant, as the only thing people have to worry about is building a great product with great content. However, Google search is algorithmic, and no algorithm can be perfect. When you fix one hole, another one opens up – and also you still need to do SEO. You still need to understand the basics and how to best optimize your site for Google. SEO is far from dead, but today SEO is more aligned to your overall product, content, and marketing strategy than ever before. Some have said – the best SEO is no SEO at all. I would disagree, everyone should do the certain number of basic things required to rank. What we need not do is force results either by creating thin pages, or by creating unnatural links.

Even today, Google has a long way to go. Google is not perfect, and probably never will be. People who understand the ins and outs of Google’s ranking system still win big with overall lowish quality of user experience. But when I see such examples, I wonder how long it will be before another Google update hits their site. Google is serious about cleaning up the web, and I for one would rather focus on creating high quality unique content and a great product experience for our users, and if not today, over time hope to win the traffic race.

Vivek Pahwa is the founder of Accentium Web, which runs a network of content sites, including a comparison shopping site called ishopper.com. He is also the co-founder of a digital marketing agency called AdLift.