This is a collection of utilities and scripts built on top of the
excellent logtools utilities which we use to automate the collection
and diplay of logs for the Project Pilgrim Archives.  (All these files
are in the top level directory for us, replacing the distributed files
where there is a conflict).

We run servers on multiple architectures all mounting the same archive
directory thru NFS, which kind of makes for a convoloted
configuration; for example, we have 3 different configuration files
for the logtools package corresponding to the 3 architectures we
support, and the proper file is selected depending on uname output. An
example of the output may be seen at 
      http://www.pilgrim.umass.edu/pub/misc/usage/usage.html



The scripts nightly, daily, weekly, monthly, and plotusage.sh are run
periodically  thru cron. 
    o) Nightly scans, sorts, builds summaries, and
       squeezes the logs (run near midnight).
    o) daily, run soon after, prints summaries of the previous day and
       week. 
    o) weekly, run once every saturday, generates summaries for this
       and the previous month.
    
    o) plotusage.sh, run every morning, generates transparent gifs of
       the data.
All these scripts get thier defaults from the script conf_guess.sh,
which provides a single spot of configuration for your site.

We also provide the customizations we have made to the standard perl
files for our environment.  The details are:

     clean_by_type.pl:
             Do Pilgrim specific cleanup
     count-to-gnuplot: 
             Set out put to pbm 
     sum:
             Do pilgrim specific stuff         
     sum-format:
             Add stuff about the pilgrim webmaster.


Probably the most usefull script is graph-log, a perl 5.001 script
that is a different take on count-to-gnuplot, which is used by the
scripts above to generate data and gnuplot scripts. One of the main
advantages of this script is that it enables us to generate graphs for
both accesses and bytes transferred, and also break down the report by
protocol types (in addition to the totals reported by
count-to-gnuplot). The usage string is given below(unfortunately, even
though the scripts pretends to be a man page, the man page has not yet
been written. This is all the documentation there is): 
__> graph-log --help
 graph-log 1.2
usage: graph-log [options] [filename]
  [--help]                    This message
  [--type=per_hour|per_day|per_month|date|month|year] Type of graph needed
  [--otype=count|bytes|both]  Type of output needed
  [--prefix=<file-prefix>]    Output file  prefix
  [--title=<Title of plot>]   A general title for all plots
  [--[no]total]               [do not] output totals
  [--[no]ftp]                 [do not] output ftp data
  [--[no]gopher]              [do not] output gopher data
  [--[no]http]                [do not] output http  data


We tend to use gnuplot 3.6, perl 5.001m, ppmtogif from the netpbm
distribution, and giftrans for generating transparent gifs.  Bash is
needed for some of the scripts (the !&^$!@# /bin/sh is badly broken
for ultrix), and, the GNU time program generates some nice timing
information. 
