Extract unique IP addresses from an apache log


Transfered from Linux Config Disqus comments:


Hi how do I extract all IP addresses from my httpd log. I need to extract only unique IP addresses from my apache log file.

Here is a my sample apache log entry:

XXX.64.70.XXX - - [26/Mar/2011:00:28:23 -0700] "GET / HTTP/1.1" 403 4609 
"-" "Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.16 (KHTML, like 
Gecko) Chrome/10.0.648.204 Safari/534.16"


Upon the apache log entry format you have supplied, the easiest way to extract in IP addresses from this kind of apache log entries is to use a combination of awk, sort and uniq commands. First we need to get a long list of IP addresses. This can be done with awk command:

$ awk '{ print $1 } ' apache_log

Next step is to sort IP addresses with sort command:

$ awk '{ print $1 } ' apache_log | sort 

And finally we get only unique IP address by streaming the output of the above command to uniq:

$ awk '{ print $1 } ' apache_log | sort | uniq

This will create a long list of short IP addresses, each on separate line. If you intend to count unique visitors on your page simply redirect the output to wc -l.

$ awk '{ print $1 } ' apache_log | sort | uniq | wc -l 

Linux questions and answers

opened #2