Introduction to Webalizer

Webalizer is a free log file analysis application that generates highly detailed usage summaries for Web and FTP servers in easily comprehensible graphical and tabular formats. The logs generated are a statistical encapsulation of user traffic on your server.

Webalizer reports are generated at a scheduled time and in accordance with the configuration options you configured for the site at the time of creating the site.

Note: The webalizer.conf file for the Web server is located at /home/virtual/<site_name>/etc/webalizer/web, where <site_name> is the name of the site for which you configured Webalizer.

The webalizer.conf file for the FTP server is located at /home/virtual/<site_name>/etc/webalizer/ftp, where <site_name> is the name of the site for which you configured Webalizer.

Features of Webalizer

• Fast processing of log files

  Log files, over a period of time, grow to very large sizes. A relatively small Web site that has several thousand visitors a month will have approximately 25 megabytes of log files. Most log file analysis tools are slow, processing the log files at speeds of only 2-3 megabytes per minute. Webalizer can be very fast; on a 200Mhz Pentium machine, over 10,000 records can be processed in one second, with a 40 Megabyte file taking roughly 15 seconds (over 150,000 records).

• Portable

  Webalizer is compatible with a range of hardware configurations ( Intel x86, Power PC, Sparc, Mips4 ) and operating systems (Linux, Solaris, OS/2) and can be easily ported from one system to another.

• Configurable reports

  Webalizer can be configured with custom options at any time by modifying the Webalizer configuration file.

• Incremental processing of logs

  Incremental processing of logs enables large log files to be split up into multiple files for processing, without any loss of data.

  Incremental processing of logs enables large log files to be split up into multiple files for processing, without any loss of data. By default, Webalizer does not enable incremental processing.

  Why do you need incremental processing

  Log files grow to very large sizes, over a period of time. A relatively small Web site that has several thousand visitors a month will have approximately 25 megabytes of log files. Processing large log files can thus be intensive on time and resources.

  Incremental processing aids faster processing of log data by fragmenting large files into smaller data entities, without compromising on data. This is achieved by saving and restoring necessary information to a disk file (by default, webalizer.current, located in the output directory of the application), so that data between runs is  preserved and log processing resumes without any loss of detail.

  For example, when you run Webalizer, it stores all the log data in a special disk file; this information also includes the time stamp of the log record, last processed. When you run Webalizer the next time, it scans the data in the existing disk file, reads the timestamp, and generates reports only for those that were logged after the timestamp.

  Caveats of incremental processing

  Incremental processing requires you to adhere to the following guidelines.

  Ensure that Webalizer configuration options are not changed between runs, as this could cause corruption of the data that encapsulates the state of the previous run.

  If you need to change configuration options, it is advisable to do it at the end of the month after normal processing of the previous month and before processing the current month.

Supported log formats

Server logs follow defined standards in the way information is stored in a log file.  

Webalizer supports the following log formats.

• Common Log File (CLF) format
• The CLF format specifies data such as the machine name or IP address from which the user made the request, as well as the information on the resource requested. This is the most commonly used format.
• Combined log format

  An extension of the Common Log File format, with additional information on referrals, user agents and cookies.

• Xferlog format
• The xferlog format is generated by the wu-ftpd and contains information on inbound and outbound requests.

  Squid proxy logs

• Squid logs record access information, system configuration errors and resource consumption (for example, memory, disk space).
  

  Squid is a high-performance proxy caching server for Web clients, supporting FTP, gopher, and HTTP data objects.

Report generation

Webalizer provides Web and FTP log reports in HTML format that can be viewed with any browser. It provides a highly detailed report of the activity on your server, manifest in a variety of reports.

When a log report is requested, Webalizer performs the following actions to produce the report.

1. Reads for the Webalizer configuration information (contained in the webalizer.conf file) for the site. Webalizer can read both GZ and ZIP file formats. If it gets a GZ or ZIP format, it decompresses the file on the fly.
2. Creates an output directory to store the log reports. If no directory is specified, the current working directory is used.
3. Reads the history file (contains totals for previous months) for the site, if it exists.
4. If incremental processing of logs is enabled, then it looks for a data file, named webalizer.current, that contains information about the previous run.

   Incremental processing of logs enables large log files to be split up into multiple files for processing, without any loss of data. By default, Webalizer does not enable incremental processing.

   Why do you need incremental processing

   Log files grow to very large sizes, over a period of time. A relatively small Web site that has several thousand visitors a month will have approximately 25 megabytes of log files. Processing large log files can thus be intensive on time and resources.

   Incremental processing aids faster processing of log data by fragmenting large files into smaller data entities, without compromising on data. This is achieved by saving and restoring necessary information to a disk file (by default, webalizer.current, located in the output directory of the application), so that data between runs is  preserved and log processing resumes without any loss of detail.

   For example, when you run Webalizer, it stores all the log data in a special disk file; this information also includes the time stamp of the log record, last processed. When you run Webalizer the next time, it scans the data in the existing disk file, reads the timestamp, and generates reports only for those that were logged after the timestamp.

   Caveats of incremental processing

   Incremental processing requires you to adhere to the following guidelines.

   Ensure that Webalizer configuration options are not changed between runs, as this could cause corruption of the data that encapsulates the state of the previous run.

   If you need to change configuration options, it is advisable to do it at the end of the month after normal processing of the previous month and before processing the current month.

5. Processes the Web server log file. If the log file contains information for multiple  months, a separate HTML file is created for each month.
6. Creates the usage summary (encapsulated in the index.htm file). The usage summary shows totals by month, and links to the HTML page of each month.
7. A new history file is saved to disk, which contains the totals generated by Webalizer during the current run.
8. If incremental processing is enabled, a data file (webalizer.current) is written, which captures necessary information about the current run.