Author Topic: Looking for Robots.txt for phpBB forum  (Read 139 times)

0 Members and 1 Guest are viewing this topic.

Offline h2obubbler

  • Full Member
  • *
  • Posts: 228
  • Karma: +0/-0
  • Life is like a box of chocolate.
    • View Profile
Looking for Robots.txt for phpBB forum
« on: August 13, 2008, 12:14:07 PM »
Hi everyone,

 I\'m running the latest phpBB forum on my website, which parts should I have included in robots.txt so they don\'t get spidered? any ideas? You have sample code to share? Many Thanks.

Techronnati | where technology never sleeps

Looking for Robots.txt for phpBB forum
« on: August 13, 2008, 12:14:07 PM »

Mountain View

Offline rpmolecule

  • Full Member
  • *
  • Posts: 228
  • Karma: +1/-0
  • Seize the Day!
    • View Profile
Re: Looking for Robots.txt for phpBB forum
« Reply #1 on: August 13, 2008, 12:15:11 PM »
Try this one:

User-agent: *
Disallow: /forums/admin/
Disallow: /forums/images/
Disallow: /forums/includes/
Disallow: /forums/language/
Disallow: /forums/templates/
Disallow: /forums/common.php
Disallow: /forums/config.php
Disallow: /forums/groupcp.php
Disallow: /forums/memberlist.php
Disallow: /forums/modcp.php
Disallow: /forums/posting.php
Disallow: /forums/profile.php
Disallow: /forums/privmsg.php
Disallow: /forums/viewonline.php
Disallow: /forums/search.php
Disallow: /forums/faq.php

Offline JusticeLeague

  • Sr. Member
  • *
  • Posts: 354
  • Karma: +1/-0
    • View Profile
Re: Looking for Robots.txt for phpBB forum
« Reply #2 on: August 13, 2008, 12:17:12 PM »
Just bit of caution though,

The file robots.txt is great for controlling the indexing of bots, for your site, if you have duplicate content elsewhere on the web. Duplicate content is a no no for google.

 

Related Topics

  Subject / Started by Replies Last post
1 Replies
200 Views
Last post August 22, 2008, 05:51:54 AM
by CyborgRepublic
1 Replies
224 Views
Last post October 04, 2008, 05:52:33 AM
by MoneyRepublic
0 Replies
106 Views
Last post December 16, 2008, 07:56:19 AM
by rpmolecule
1 Replies
214 Views
Last post December 19, 2008, 05:16:00 AM
by MrSpecialist
0 Replies
40 Views
Last post January 16, 2013, 07:27:33 AM
by Owen741

Posting Disclaimer: Any individual may post a message in this forum and may do so anonymously. Therefore, the sole author is exclusively and entirely responsible for all opinions in that message. They do not represent the official opinions of Techronnati, its administrators or moderators or the Techronnati Management. Techronnati is merely acting as an impartial conduit for constitutionally protected free speech and is not responsible and will not be held liable for the content of such messages. All images and service logos are trademarks of their respective owners.