Who or what is myArtSpider?
myArtSpider is a new search engine, which is specialized in art or culture related results. It is based upon the experiences of the former art platform 'mercant arts international'. This art search engine is not meant to replace such search engines like Google, Yahoo! or others, which are focused on mass results. myArtspider is not able not meant to serve this huge amount of result. In the foreground stands the relevance of the search results. This is achieved by the combination of url indexing by an automatic indexing software (crawler, spider, robot) and the examining of the relevance by the software and a human.
KrauliBot
The core of myArtSpider is the automatic searching and indexing software called 'KrauliBot'. You can check the HTTP user-agents and the HTTP referrers in the web server logfiles to see if KrauliBot has already visited Your web server. Depending on the page content KrauliBot dertermines the art or culture relevance of the page and decides autonomous whether to add the page to the search index or not. The KrauliBot tries to be polite and to spare the ressources of Your web server and to attend indexing prohibitions by HTTP or HTML standard (HTTP header, meta-robots-tag and robots.txt). If it's not the case You can send us an email and we will check this.