#-----------------------------------------------------------------------------------------------------------------
#
# INTRODUCTION
# This robot file is created for an XLNTcms application and will contain generic information for robots.
# This file can be changed freely as long as the right syntax is followed.
# For more information goto : http://www.robotstxt.org/
#
# 
# THE METHOD
# The method used to exclude robots from a server is to create a file on the server which specifies an 
# access policy for robots. This file must be accessible via HTTP on the local URL "/robots.txt".
# 
#
# FORMAT
# The format and semantics of the "/robots.txt" file are as follows: 
# The file consists of one or more records separated by one or more blank lines (terminated by CR,CR/NL, or NL).
# Each record contains lines of the form "<field>:<optionalspace><value><optionalspace>". 
# The field name is case insensitive.
#
#-----------------------------------------------------------------------------------------------------------------

#
# Disallow peeking in some of the parts of the system folder for all robots. 
# The pages folder should be crawled because the robots will gather all the information through he PPage script
# Change this if the system folder is in a different location
#
User-agent: *
Disallow: /xlntcms/system/
Disallow: /system/

User-agent: *
Allow: /xlntcms/system/PPage.php
Allow: /system/PPage.php

#
# There is no need to index the configuration file
#
User-agent: *
Disallow: /xlntcms/user/system_config.php
Disallow: /system/system_config.php

# 
# Users can add their own rules for the user folder
#
#User-agent: *
#DisAllow: /