Opened 7 years ago

Closed 7 years ago

#1765 closed unknown (fixed) needs to Disallow everything

Reported by: guaka Owned by: planetcruiser
Priority: major Milestone: 1.1
Component: ServerSetup Keywords: seo, robots.txt

Change History (3)

comment:1 Changed 7 years ago by jsfan

This file is not specific to alpha. The robots.txt is in git and thus shared between alpha and www.

comment:2 Changed 7 years ago by guaka

Then should be a mechanism to have a different robots.txt.

Otherwise there's tons of duplicate content, and people might actually arrive at alpha when searching for stuff.

Version 0, edited 7 years ago by guaka (next)

comment:3 Changed 7 years ago by jsfan

  • Milestone changed from unassigned to 1.1
  • Resolution set to fixed
  • Status changed from new to closed

I have now removed robots.txt from the repo and created one for alpha on puma just as the one you suggested. With the next release www will get a static robots.txt as well.

Note: See TracTickets for help on using tickets.