Uninitialized value in my Apache script

[Uday64]

As always in my script of Apache log and iis, I currently have a problem with my regexp. For the moment only with Apache, the regexp works well, although it is perhaps not quite optimal for errors. However when I add a regexp to see if the row is a iis line, I have a small error.

Here is the code of two regexp:

my $APACHE  = q{(\S+)\s+\S*\s+\S*\s+\[(\d+)\/(\D+)\/(\d+):\d+:\d+:\d+\s+\+\d+\] \"(.*) (.*)\" (\S+) (\S+) \"(.*)\" \"(.*)\"};
my $IIS     = q{(\S+)\s+\S*\s+\S*\s+\[(\D+)\/(\d+)\/(d\+):\d+:\d+:\d+\s+\+\d+\] \"(.*) (.*)\" (\S+) (\S+) \"(.*)\" \"(.*)\"};
 
open (File, "< $file") or die "Can not open file $ARGV[0]\n";
while (my $line = <File>){
 
($ip,$day,$month,$year,$method,$page,$ret_code,$byte,$referer,$user_a) = $line =~ $APACHE
($ip,$month,$day,$year,$method,$page,$ret_code,$byte,$referer,$user_a) = $line =~ $IIS;
 
...

I just do a reversal for the month to test. So my lines from my file looks like this:

190.90.90.90 - - [14/Feb/2009:19:45:42 +0100] "GET /index.php" 100 200 "http://www.techarena.in" "Mozilla/4.0 (compatible; MSIE 7.0; Windows)"
190.90.90.90 - - [Feb/14/2009:19:45:42 +0100] "GET /index.php" 100 200 "http://www.techarena.in" "Mozilla/4.0 (compatible; MSIE 7.0; Windows)"

My Apache lines only run, I try with the iis = error

Use of uninitialized value $month in string eq at C:\log.pl line 66, <File> line 2.

This error is just after $month that has been initialized since if I just put Apache, it works well. I think there I need the condition for the regexp, twice "=" is not enough? Can I paste the complete code required, I don't understand why I have this error.

[fellah]

Try to make simple, just one regexp.

#!/usr/bin/perl
use strict;
use warnings;
open my $file ,'<', 'test.txt' or die "The file does not exist !";
while (my $line = <$file>){
	if ( $line =~ m{([\d.]+)[-\s]+\[(\w+)/(\w+)/(\d+)[\d:]+\s\+\d+\] "(\w+) /(.*?)" (\d+) (\d+) "(.*?)" "(.*?)"}){
		my ($day, $month);
		my ($ip,$val2, $val3, $year,$method,$page,$ret_code,$byte,$referer,$user_a) = ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10);
		# Looking between $2 and $3 which is the month and which is the day
		if ($2 =~ m{\d+}){
			$day = $val2;
			$month = $val3;
		}
		else{
			$day = $val3;
			$month = $val2;			
		}
		print "ip = $ip\nday = $day\nmonth = $month\nyear = $year\nmeth = $method\npage = $page\ncode = $ret_code\nby = $byte\nref = $referer\nuser = $user_a\n\n\n";
	}
 
}
 
close $file;

Result

ip = 190.90.90.90
day = 15
month = Feb
year = 2009
meth = GET
page = index.php
code = 100
by = 200
ref = http://www.techarena.in
user = Mozilla/4.0 (compatible; MSIE 7.0; Windows)


ip = 190.90.90.90
day = 15
month = Feb
year = 2009
meth = GET
page = index.php
code = 100
by = 200
ref = http://www.techarena.in
user = Mozilla/4.0 (compatible; MSIE 7.0; Windows)

[Uday64]

I includes the way you do is true that immediately simplest. But in reality between the Apache and IIS log, there is not the date that differs. For example, the date will be first on iis, tracking ip etc. The regular expression will be entirely different if the hit can not be done (or code that will take quite a line in duplicate). This is the complete code (regular expression IIS has not been changed):

#!/usr/bin/perl -w
use strict;
use warnings;
use Date::Calc qw( Date_to_Days );
 
########## VARIABLES ##########
 
my $choice;
my $EXPREG  = q{(\S+)\s+\S*\s+\S*\s+\[(\d+)\/(\D+)\/(\d+):\d+:\d+:\d+\s+\+\d+\] \"(.*) (.*)\" (\S+) (\S+) \"(.*)\" \"(.*)\"};
my $EXPREG1 = q{(\S+)\s+\S*\s+\S*\s+\[(\d+)\/(\D+)\/(\d+):\d+:\d+:\d+\s+\+\d+] \"(.*) (.*)\" (\S+) (\S+) \"(.*)\" \"(.*)\"};
my ($ip,$day,$month,$year,$method,$page,$ret_code,$byte,$referer,$user_a);
my ($lower,$upper,$date);
my ($dayin,$dayout,$monthin,$monthout,$yearin,$yearout);
my (%haship,%hashpage,%hashbyte,%hashref,%hashua);
 
###############################
 
#### TEST FILE AND DATE #####
 
if ($#ARGV != 2){
	print ("\nUtilisation : script.pl <file_log> <date_debut> <date_fin>\n");
	print ("Example : script.pl log.txt 20/01/2009 20/02/2009 \n");
	exit();
}
 
my $file = $ARGV[0];
my $datein = $ARGV[1];
my $dateout = $ARGV[2];
 
if (!(-e $file)){
	print $file," does not exist.\n";  
	exit();
}
 
if ($datein =~/([0-9]*)\/([0-9]*)\/([0-9]*)/) {
	$dayin = $1;
	$monthin = $2;
	$yearin = $3;
}
 
else {
	print ("Error of size on date_debut");
	exit();
}
 
if ($dateout =~/([0-9]*)\/([0-9]*)\/([0-9]*)/) {
	$dayout = $1;
	$monthout = $2;
	$yearout = $3;
}
 
else {
	print ("Error of size on date_fin");
	exit();
}
 
###############################
 
############ MAIN #############
 
open (File, "< $file") or die "Can not open file $ARGV[0]\n";
 
while (my $line = <File>){
		($ip,$day,$month,$year,$method,$page,$ret_code,$byte,$referer,$user_a) = $line =~ $EXPREG;
		($ip,$day,$month,$year,$method,$page,$ret_code,$byte,$referer,$user_a) = $line =~ $EXPREG1;
		$month = '01' if ($month eq "Jan");
		$month = '02' if ($month eq "Feb");
		$month = '03' if ($month eq "Mar"); 
		$month = '04' if ($month eq "May");
		$month = '05' if ($month eq "Apr");
		$month = '06' if ($month eq "Jun");
		$month = '07' if ($month eq "Jul");
		$month = '08' if ($month eq "Aug");
		$month = '09' if ($month eq "Sep");
		$month = '10' if ($month eq "Oct");
		$month = '11' if ($month eq "Nov");
		$month = '12' if ($month eq "Dec");
		$lower = Date_to_Days($yearin,$monthin,$dayin);
		$upper = Date_to_Days($yearout,$monthout,$dayout);
		$date = Date_to_Days($annee,$month ,$jour);
		if (($date >= $lower) && ($date <= $upper)) {
			$haship{$ip} += 1;
			$hashpage{$page} += 1;
			$hashbyte{$byte} += 1;
			$hashref{$referer} += 1;
			$hashua{$user_a} += 1;	
		}
}
 
&subdisplay();
$choice = (<STDIN>);
 
if ($choice == "1") {
	&subip();
}
 
elsif ($choice == "2") {
	&subpage();
}
 
elsif ($choice == "3") {
	&subbyte();
}
 
elsif ($choice == "4") {
	&subref();
}
 
elsif ($choice == "5") {
	&subua();
}
 
###############################
 
############# SUB #############
 
sub subdisplay {
	system("cls"); # A change to reflect clear
 
	print " |      Welcome to the program log analyzer        |\n";
	print " |                Programmer par Uday                    |\n";
	print " |-----------------------------------------------------------|\n\n";
 
	print " Please enter the number corresponding to desired action :\n\n";
	print "    1.Hits IP addresses\n";
	print "    2.Hits of the most visited pages\n";
	print "    3.Hits kb of the most important\n";
	print "    4.Hits of referer of the most presents\n";
	print "    5.Hits of the user agent the most presents\n";	
}
 
sub subip {
	system("cls"); # A change to reflect clear
	open(SORT, ">log-ip.txt");
	foreach $ip ( sort { $haship{$b} <=> $haship{$a} } keys %haship) {
		print "IP : $ip has been found $haship{$ip} times\n";
		print SORT "IP : $ip has been found $haship{$ip} times\n";
		}
	print "\nWriting in the file log-ip.txt under way...\n";
	sleep(1);
	print "\nScripture successful\n";
	print "\n ** A Soon **\n";
	exit;
}
 
sub subpage {
	system("cls"); # A change to reflect clear
	open(SORT, ">log-page.txt"); 
	foreach $page ( sort { $hashpage{$b} <=> $hashpage{$a} } keys %hashpage) {
		print "Page : $page has been found $hashpage{$page} times\n";
		print SORT "Page : $page has been found $hashpage{$page} times\n";
		}
	print "\nWriting in the file log-page.txt under way...\n";
	sleep(1);
	print "\nScripture successful\n";
	print "\n ** A Soon **\n";
	exit;
}
 
sub subbyte {
	system("cls"); # A change to reflect clear
	open(SORT, ">log-byte.txt");
	foreach $byte ( sort { $hashbyte{$b} <=> $hashbyte{$a} } keys %hashbyte) {
		print "Kilo-bytes : $byte kb  - $hashbyte{$byte} times\n";
		print SORT "Kilo-bytes : $byte kb  - $hashbyte{$byte} times\n";
		}
	print "\nWriting in the file log-byte.txt in progress...\n";
	sleep(1);
	print "\nScripture successful\n";
	print "\n ** A Soon **\n";
	exit;
}
 
sub subref {
	system("cls"); 
	open(SORT, ">log-referer.txt");
	foreach $referer ( sort { $hashref{$b} <=> $hashref{$a} } keys %hashref) {
		print "Referer : $referer has been found $hashref{$referer} times\n";
		print SORTIE "Referer : $referer has been found $hashref{$referer} times\n";
		}
	print "\nWriting in the file log-referer.txt in progress...\n";
	sleep(1);
	print "\nScripture successful\n";
	print "\n ** A Soon **\n";
	exit;
}
 
sub subua {
	system("cls"); 
	open(SORT, ">log-user_agent.txt");
	foreach $user_a ( sort { $hashua{$b} <=> $hashua{$a} } keys %hashua) {
		print "User agent : $user_a has been found  $hashua{$user_a} times\n";
		print SORT "User agent : $user_a has been found $hashua{$user_a} times\n";
		}
	print "\nWriting in the file log-user-_agent.txt in progress...\n";
	sleep(1);
	print "\nScripture successful\n";
	print "\n ** A Soon **\n";
	exit;
}
 
#############################
close(File);

My aim was therefore to directly identify the line type: If line type is $ regexp then Apache if $ EXPREG1 style so it's IIS. Only I have the error mentioned above. Have you an idea about debugging?

[fellah]

I do not see what your code requires absolutely regexp 2, but if you do not want my solution only uses one, it's the same. Here's the problem

($ip,$day,$month,$year,$method,$page,$ret_code,$byte,$referer,$user_a) = $line =~ $EXPREG;($ip,$day,$month,$year,$method,$page,$ret_code,$byte,$referer,$user_a) = $line =~ $EXPREG1;

make use of "if"

if ($line =~ $EXPREG){
	($ip,$day,$month,$year,$method,$page,$ret_code,$byte,$referer,$user_a) = ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10);
}
elsif ($line =~ $EXPREG1){
	($ip,$day,$month,$year,$method,$page,$ret_code,$byte,$referer,$user_a) = ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10);
}

For

my $EXPREG  = q{(\S+)\s+\S*\s+\S*\s+\[(\d+)\/(\D+)\/(\d+):\d+:\d+:\d+\s+\+\d+\] \"(.*) (.*)\" (\S+) (\S+) \"(.*)\" \"(.*)\"};

use

my $EXPREG  = q{(\S+)\s+\S*\s+\S*\s+\[(\d+)/(\D+)/(\d+):\d+:\d+:\d+\s+\+\d+\] "(.*) (.*)" (\S+) (\S+) "(.*)" "(.*)"};

The double quotes should not be escaped by a backslash. To escape the slash by a backslash is necessary if you use m// (it is also advisable to use m () instead)