Nofollow and Noindex on Robots.txt and Meta Tags

Nofollow and Noindex on Robots.txt and Meta Tags
Author: Eileen Maravillas

I know a lot of you are wondering what's the difference between the “nofollow” and “noindex” in the robots.txt file and the ones declared on the meta tags. Some search engine optimization specialists claims that there's no difference while others believe that one method is less valuable than the other.

You can ask it out on several webmaster/SEO forums, but you'll just get mixed responses. In here, I will give you the REAL answer! According to Eric Enge's interview with Google's software engineer Matt Cutts way back in 2007, there are differences in using NoIndex, NoFollow, and Robots.txt.

In The Robots.txt File

Even if you restrict Google's spider from indexing certain pages of your website, they can still accumulate page rank. For example, if you disallow the crawling of your “About Us” page yet your homepage links to it, PR juice is still passed on.

In addition, websites that disallow crawling can still accumulate page rank and be visible in search results. Why? If another website links to it, PR juice will be passed on. Google can sometimes use the information of a website submitted to ODP (also known as DMOZ) in order to display it on their results page, or when another website links to it.

In The Meta Tag

Let's understand what Nofollow and Noindex really means.

The Nofollow is usually used on outgoing links, when declared in the meta tag, it means “do not follow all links on this page.” With this, you are also telling the spider not to pass PR juice. However, the nofollow on the meta tag also applies to your links that points to the other pages of your website. This also means that you are depriving the flow of PR juice to some of your pages.

On the other hand, the Noindex means, “do not index this page.” Pages with noindex can still accumulate page rank if a there's a dofollow link pointing to that page.

Dofollow is the opposite of Nofollow - a link with no “nofollow” attribute assigned means it's a dofollow link. A term used by webmasters and SEO experts.

To sum it all up:

• NoFollow means you're telling the search engine spider not to follow a link and also not to pass PR juice to that link.
• NoIndex means you're telling the search engine spider not to index your website and not to show it on SERP.
• The website or webpage in a Nofollow link can still gain PR if another site links to it without the nofollow attribute.
• Pages with the Noindex tag can still gain PR if another site links to it without a nofollow attribute.
• Pages with the Noindex tag can still be visible on SEPR using the information collected from ODP (DMOZ) or when another website links to it using Dofollow.
  
• Nofollow on the meta tag of a webpage, applies to all the links on that page.
• Nofollow attribute on any link applies only to that link.

So whether you use the Robots.txt or the Meta Tag, the results are still the same. The reason why Robots.txt is mostly used is because “it is the fundamental method of putting up an electronic no trespassing sign that people have used since 1996,” and it's much easier to declare which pages of your website not to crawl instead of placing the NoIndex code manually on individual pages' meta tags.

Commonly, SEO services do provide complete analysis of a website as well as recommendations. So whenever you are looking for a good search engine optimization service, never forget to ask for a website analysis first, don't just jump in and hire somebody you do not know.

The “Nofollow” Attribute

The “Nofollow” Attribute
Author: Eileen Maravillas

The nofollow attribute was created by Google's Matt Cutts and Blogger.com's Jason Shellen mainly to stop spam comments on blogs. However, some people still believes that this is a TAG used to restrict search engine spiders to index a webpage.

In search engine optimization, the “nofollow” attribute is placed on an outgoing link (a link that points to another website). By placing this attribute, you are telling search engine spiders “not to score” or “don't pass my page rank juice” to this website. You cannot block access to a content or prevent that content to be indexed.

Example:


In another sense, this attribute is used to control the flow of page rank (PR) from one website to another. You don't want to pass on your hard earned PR to some untrusted (and oftentimes illegal) sites. That's why bloggers are using this to prevent such sites from hurting them.

Other search engines like Yahoo, Bing (former MSN) and Ask respects the rel=nofollow but only Google honours this attribute completely.

Controlling the Flow of Page Rank

Since it works on outgoing links, many search engine optimizers (SEOs) believed that this can also be used on internal links. The ability to pass on PR juice to the most important pages of your website using the nofollow attribute gave SEOs another method called PageRank Sculpting.

Several SEOs have suggested that pages such as "About Us", "Terms of Service", "Contact Us", and "Privacy Policy" pages are not important enough to earn PageRank, and so should have nofollow on internal links pointing to them. Google employee Matt Cutts has provided indirect responses on the subject, but has never publicly endorsed this point of view. From Wikipedia

There are many BPO companies that offers SEO Services, no matter the cost, that would suggest to use the nofollow attribute on not-so-important pages and focus mainly on the high valued pages like the homepage. Even so, there's this contradiction: instead of having just one page appear (or rank) in SERP, you can have two or even more pages appear by making them rank high as well. So instead of just focusing on boosting one page of your site up to the first page in SERP, you can have two or more thus gaining you more traffic and sales.

If you're wondering which of this two strategies you're going to use, my advice is focus on your target goal. If your client wants to rank his homepage only then you may use the nofollow attribute on low-valued pages. Just remember this, always INFORM your client of the additions and/or changes you're about to make and make the decision TOGETHER to avoid any conflict in the long run. Your strategy may get his website on top but once your ranking fluctuates and drops the website way down, you'll be having a hard time explaining it to him! (Especially if the client knows nothing about SEO).

Other Purpose

Aside from human and automated spam comments on blogs, the nofollow attribute is also advised to be used on paid links so that these links do not get credit in search engine results page (SERP).

Abuse of Using the Nofollow Attribute

While this attribute greatly helps blog owners to sweep away spam comments, there are also bloggers who uses the nofollow when referring or quoting to a quality content from another website. There may be a link to follow but as long as there's a rel=“nofollow” included, these links are deprived of their well deserved PR juice.

Another abuse I see is mostly at the sidebar of any blog (or in a dedicated link partners' page of any website). Yes, I'm talking about their link exchange partners! Though link trading is not highly recommended in search engine optimization, they too have a value. There are many webmasters and SEO specialist who practices this scheme to fool their link partners. It is not nice to receive quality PR juice from your link partners while you use rel=“nofollow” on their links. Seriously, it's like stealing on daylight!

The NOFOLLOW attribute is created to help combat spammers and not to provide another way for evil geniuses (bad webmasters and SEO specialists) to cause more problems on the web.