trAvis - MANAGER

Edit File: awstats_benchmark.html

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<meta name="description" content="AWStats Documentation - Benchmark page">
<meta name="keywords" content="awstats, awstat, benchmark, speed, dns, lookup, time">
<meta name="robots" content="index,follow">
<meta name="title" content="AWStats Documentation - Benchmark comparison page">
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<title>AWStats Documentation - Benchmark page</title>
<link rel="stylesheet" href="styles.css" type="text/css">
<link href="https://plus.google.com/+LaurentDestailleur-Open-Source-Expert" rel="publisher" />
</head>

<tr style="font: 10pt arial,helvetica,verdana">
<td bgcolor=#9999cc align=center><a href="/"><img src="images/awstats_logo6.png" border=0></a></td>
<td bgcolor=#9999cc align=center>
 
AWStats logfile analyzer 7.8 Documentation 
 
</td>
<td bgcolor=#9999cc align=center>
&nbsp;
</td>
</tr>

</table>

<H1 style="font: 26px arial,helvetica,sans-serif">Benchmarks</H1>

AWStats update process must be ran frequently, so it's important to know what is AWStats
speed to choose an optimum delay between each update process according to AWStats speed and
the refresh rate you need to have. 
AWStats speed depends on AWStats version and options/setup you use in configuration file.

This is benchmark results with AWStats version 6.0 and a common configuration: 
 
HARDWARE: Athlon 1 GHz / 256MB 
SOFTWARE: Windows 2000 / Perl 5.8 (Cygwin Perl) 
CONFIG OPTIONS: Default values were used: <a href="awstats_config.html#LogFormat">LogFormat</a>=1, <a href="awstats_config.html#DNSLookup">DNSLookup</a>=0, <a href="awstats_config.html#URLWithQuery">URLWithQuery</a>=0, <a href="awstats_config.html#URLReferrerWithQuery">URLReferrerWithQuery</a>=0, <a href="awstats_config.html#URLWithAnchor">URLWithAnchor</a>=0, No plugins 
AVERAGE SPEED: 5200 lines by seconds 
Other times for different kind of web sites sizes are shown later in this page...

This is other important information to know: 
 
- A log file size is about 150 (NCSA common/CLF log files) to 320 times (NCSA extended/XLF/ELF log files) its number of lines, 
- 1,000 visits = 8,000 pages (with 8 pages/visits) = 64,000 lines (with 8 hits/page) = 20 MB file => 15 seconds (Athlon 1GHz, Standard Perl 5.8) 
- History files (AWStats database, resuming the log analysis) has the following size (one file a month) :
15000+90*x+100*y bytes (where x is number of unique visitors a month and y is number of different pages on web sites).
If you use option <a href="awstats_config.html#BuildHistoryFormat">BuildHistoryFormat</a>=xml, you must 
multiplie this value by 3. 
 
WARNING ! All those data are average values for a common public site with default configuration.
Calculation rule can be seriously changed according to web server or AWStats configuration and web site content.

Don't forget that benchmarks of log analyzers are made without reverse DNS lookup because DNS
lookup is so slow (depending on Internet network and your system), that if enabled in AWStats
configuration file, it would take more than 99% of the time of a log analysis !
Take a look at the following chart to: 
- Get more real ideas on benchmarks results 
- Get more information and advice on a good setup for your site.

This is examples of frequency/parameters you should use to have a good use of AWStats: 
 
<table border=1 cellspacing=0 cellpadding=2 style="font: 12px; arial,verdana">
<tr bgcolor=#EEEEEE><td rowspan=2>Your Web site traffic</td><td rowspan=2>Perl distrib</td><td colspan=3>Values for parameters</td><td rowspan=2>Recommended update frequency (Rotate log delay)</td><td rowspan=2>Memory required**</td><td rowspan=2>Update process duration***</td></tr>
<tr bgcolor=#EEEEEE><td>DNSLookup*</td><td>URLWithQuery</td><td>URLReferrerWithQuery</td></tr>
<tr><td rowspan=2> 0 - 1,000 visits/month</td><td rowspan=2>Your choice</td><td>0 (or 2)</td><td>0 or 1</td><td>0 or 1</td><td>Once a day Log files are 0-1 MB 2000 lines to process</td><td>4 MB free</td><td>1s</td></tr>
<tr> <td>1</td><td>0 or 1</td><td>0 or 1</td><td>Once a day Log files are 0-1 MB 2000 lines to process</td><td>4 MB free</td><td>2mn</td></tr>
<tr><td rowspan=2> 1,000 - 10,000 visits/month</td><td rowspan=2>Your choice</td><td>0 (or 2)</td><td>0 or 1</td><td>0 or 1</td><td>Once a day Log files are 1-10 MB 2000-20000 lines</td><td>4-8 MB free</td><td>1s-10s</td></tr>
<tr> <td>1</td><td>0 or 1</td><td>0 or 1</td><td>Once a day Log files are 1-10 MB 2000-20000 lines</td><td>4-8 MB free</td><td>2mn-10mn</td></tr>
<tr><td rowspan=2>10,000 - 100,000 visits/month</td><td rowspan=2>Your choice</td><td>0 (or 2)</td><td>0 or 1</td><td>0 or 1</td><td>One a day Log Files are 10-100 MB 20000-200000 lines</td><td>8-32 MB free</td><td>10s-120s</td></tr>
<tr> <td>1</td><td>0 or 1</td><td>0 or 1</td><td>Once a day Log files are 10-100 MB 20000-200000 lines</td><td>8-32 MB free</td><td>10mn-50mn</td></tr>
<tr><td rowspan=2>100,000 - 500,000 visits/month</td><td rowspan=2>Your choice</td><td>0 (or 2)</td><td>0</td><td>0 or 1</td><td>Every 6 hours Log Files are 24-96 MB 50000-300000 lines</td><td>16-92 MB free</td><td>30s-3mn</td></tr>
<tr> <td>1</td><td>0</td><td>0 or 1</td><td>Every 6 hours Log Files are 24-96 MB 50000-300000 lines</td><td>16-92 MB free</td><td>15mn-60mn</td></tr>
<tr><td> 500,000 - 2,000,000 visits/month</td><td>See next section on pb with ActiveState</td><td>0 (or 2)</td><td>0</td><td>0</td><td>Every 6 hours Log Files are 96-384 MB 300000-1200000 lines</td><td>64-256 MB free</td><td>3mn-12mn</td></tr>
<tr><td> 2,000,000 - 4,000,000 visits/month</td><td>See next section on pb with ActiveState</td><td>0 (or 2)</td><td>0</td><td>0</td><td>Every 6 hours Log Files are 384-768 MB 1200000-2400000 lines</td><td>256-512 MB free</td><td>12mn-24mn</td></tr>
<tr><td> +4,000,000 visits/month</td><td>See next section on troubles with ActiveState</td><td colspan=6>AWStats is a good choice for such web sites only if you use a dedicated server with a large amount of memory. Try a tool with less features but faster like "row counter log analyzers" like Webalizer if not.</td></tr>
</table>
* You should set <a href="awstats_config.html#DNSLookup">DNSLookup</a> parameter to 0 (or 2) if 
&nbsp;&nbsp; &nbsp;&nbsp; - reverse DNS lookup is already done in your log file, 
&nbsp;&nbsp; &nbsp;&nbsp; - or if your web site has more than 250,000 visits a month. 
Note: Country report can works without reverse DNS lookup if plugin 'geoip' is enabled (faster and more accurate than reverse DNS lookup). 
** This is free memory required for update process (in MB), this is not hardware memory installed ! 
Warning: If you use the <a href="awstats_config.html#URLWithQuery">URLWithQuery</a> or <a href="awstats_config.html#URLReferrerWithQuery">URLReferrerWithQuery</a> option, or forget
to complete correctly <a href="awstats_config.html#URLQuerySeparators">URLQuerySeparators</a> for some sites, this value can be dramatically increased. 
*** Duration with DNSLookup set to 1 is very long because of DNS lookup whatever is speed of your computer.
Duration with DNSLookup set to 0 (or 2) is with Athlon 1GHz/256MB, Cygwin Perl 5.8 and LogFormat=1.

<!--
Configuration used for tests:

# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 4
model name      : Intel(R) Celeron(R) CPU 2.66GHz
stepping        : 1
cpu MHz         : 2672.884
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 5
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe pni monitor ds_cpl cid
bogomips        : 5334.63

# free
memoire : 524Mo
Mem: 505680k total, 492548k used, 13132k free, 18244k buffers
Swap: 522104k total, 12752k used, 509352k free, 188820k cached
-->
 
 
<a name="ADVICES"></a>SOME IMPORTANT ADVICES FOR A GOOD USE OF AWSTATS: 
 
<img src="images/star.png"><img src="images/star.png"><img src="images/star.png"><img src="images/star.png">
- Check that <a href="awstats_config.html#DNSLookup">DNSLookup</a> is disabled in AWStats (DNSLookup should not
be set to 1). If you need a 'Country' report, you should prefer using the 'geoip' plugin
using <a href="http://www.maxmind.com/en/city?rId=awstats">Maxmind database</a> instead of a DNS lookup. Those plugins allow you to have more accurate results, faster with no network queries (With
DNSLookup enabled, log analyze speed is decreased by 40 to 100 times, so use it only if required).
Note that without DNS lookup and without the 'geoip' plugins, 'Country' report might work but results will be
less accurate than the 'geoip' usage and it works only if hosts addresses in your log file are already resolved
(need to setup your web server to do so, your web server will be slowed). 
<img src="images/star.png"><img src="images/star.png"><img src="images/star.png"><img src="images/star.png">
- Use carefully parameters <a href="awstats_config.html#URLWithQuery">URLWithQuery</a>, <a href="awstats_config.html#URLReferrerWithQuery">URLReferrerWithQuery</a> and <a href="awstats_config.html#URLWithAnchor">URLWithAnchor</a>
(Let them set to 0 if you don't know what they means) and check your web site URLs' syntax to know if you don't need to complete the parameter <a href="awstats_config.html#URLQuerySeparators">URLQuerySeparators</a>.
If you really need to use <a href="awstats_config.html#URLWithQuery">URLWithQuery</a>=1, check that
<a href="awstats_config.html#URLWithQueryWithOnlyFollowingParameters">URLWithQueryWithOnlyFollowingParameters</a> or
<a href="awstats_config.html#URLWithQueryWithoutFollowingParameters">URLWithQueryWithoutFollowingParameters</a>
are set properly. 
<img src="images/star.png"><img src="images/star.png"><img src="images/star.png"><img src="images/star.png">
- Use last Perl version (For example Perl 5.8 is 5% faster than 5.6) and, more important, for large log files,
use standard Perl distribution instead of ActiveState. This is because ActiveState 5.006 (and may be also
other versions), has very important memory hole problem making speed of analysis slower and slower
reaching 0 lines/seconds and using all your memory. You can see the decrease by adding the -showsteps option
on command line. The speed should be constant to value given in top of this page, even for
several Gigabytes log files ! 
<img src="images/star.png"><img src="images/star.png"><img src="images/star.png">
- Rotate your log (See <a href="awstats_faq.html#ROTATE">FAQ-SET500</a>) and launch AWStats more often (from crontab or a scheduler, See <a href="awstats_faq.html#CRONTAB">FAQ-SET550</a>).
The more often you launch AWStats, the less AWStats has new lines in log to process.
This can also solve the ActiveState memory problem (see next advice). 
<img src="images/star.png"><img src="images/star.png">
- Be sure that your <a href="awstats_config.html#HostAliases">HostAliases</a> parameter list is complete. 
<img src="images/star.png"><img src="images/star.png">
- Use last AWStats version (For example AWStats 6.0 is 15% faster than 5.9). 
<img src="images/star.png"><img src="images/star.png">
- For geeks users, you can also recompile your Perl with differents options.
For example, it seems that the "use64bitint=define usemymalloc=y" can increase speed by 10%.

<hr>

<div class="htmldoc-ignore">

Article written by <a href="https://plus.google.com/+LaurentDestailleur-Open-Source-Expert?rel=author" rel="author" style="color: #ccc; font-weight: normal;">Laurent Destailleur</a>. 
 
<script type="text/javascript" src="https://apis.google.com/js/plusone.js"></script>
<g:plusone></g:plusone>

<div id="fb-root"></div>
<script>(function(d, s, id) {
 var js, fjs = d.getElementsByTagName(s)[0];
 if (d.getElementById(id)) return;
 js = d.createElement(s); js.id = id;
 js.src = "//connect.facebook.net/fr_FR/all.js#xfbml=1";
 fjs.parentNode.insertBefore(js, fjs);
}(document, 'script', 'facebook-jssdk'));</script>
<div class="fb-like" data-href="http://www.awstats.org/docs/awstats_benchmark.html" data-layout="button_count" data-action="recommend" data-show-faces="false" data-share="true"></div>
 

<a href="https://twitter.com/awstats_project" class="twitter-follow-button" data-show-count="false">Follow @awstats_project</a>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
</div>

</body>
</html>