State of the cigar web
As many of you may know CigarCities constantly crawls loads of different cigar related pages. Here is a little summery of the pages that are offline the most.
So our first error is on the 7. November 2012 at 16:00:20 and the last ist on 19. November 2013 11:04:40. We have analysed about 1.7 Gigabyte of data. While most crwals ran successfully there were still 664 errors of various kinds. The most obvious was that the server was down. But we also had some errors that the site would return junk or we would trigger an error in the server. The sites (top 10) with the most errors were:
18 Miguel Private Cigars
22 My Cigar Blog
25 Aficionados Blog
29 LCDH Tabac Benden
38 Premium Cigars
65 Zigarren News Blog
Of course this is not really a good statistic as some sites were added later and so had less time to generate errors. But I still find it interresting, that is is again my good old friend long tail:
It is also interresting, that some sites got a high rank in this anti-rank because they were offline for a few days and others because they are on and off all the time. If you are interrested in more please let me know and I will do some more analysis.