cugeCrawler 1.2 crawler information

What is cugeCrawler 1.2 and what is it doing?

cugeCrawler 1.2 is a web crawler that is surfing the web automatically. It has been developed by cuge.
It finds websites that are publicly reachable and saves the following information to the international
websites directory on cuge.org in order to make the appropriate website available for being evaluated
with regard to their service quality and reliability by the website's customers themselves:

- domain name (e.g. cuge.org)
- website title (if available)
- description meta contents (if available)
- keywords meta contents (if available)

Neither contents from any page within the website nor the pages themselves are saved.

Identification

cugeCrawler 1.2 identifies himself using the following user agent: "cugeCrawler 1.2 - cuge.org"

Exclusion standards respected

Having been developed for finding nothing but entire websites and making them available to the
public for leaving an evaluation of the appropriate website's reliability, cugeCrawler 1.2 respects the
NOINDEX statement in the robots meta tag content. (ex.: <META NAME="robots" CONTENT="NOINDEX">).
This statement must be made in the domain's start page head area.

How to contact us

If you have more questions or requests, feel free to contact us at cugeCrawler@cuge.org please. Thank you.