There are handful feasible flash based charting solutions, and they just work really great - with interaction. But unfortunately most of those are not open source, nor free. Few months ago(may be it has been there for longer?), google released a chart api which can generate a PNG image based on your data.

The charts API very easy to use, and everything is encoded in the URL. So, you can either download the image and host it on your servers, or rather use that in the web page itself without any troubles at all.

Yesterday, I gave it a try. I got some of my apache logs, parsed them, and used google APIs to draw the charts. The results look good.

The log files are too big to parse quickly and contains lot of junk data. So, I wrote the code so that it only takes the “200 OK”, and ignores “js,css,png,jpg,gif” requests. Here is the php code.

	$ignored = array('css','js','png','jpg','gif');

	$vals = array();

	$f = fopen("/var/log/apache2/access.log","r");
	while (!feof($f)){
		$line = fgets($f);
		preg_match("/(.*) - - \[(.*)\] \”GET (.*) HTTP\/1.1\” 200/”,$line,$matches);
		// If it’s not 200 OK - just ignore it
		if (empty($matches))
			continue;
		$path = pathinfo($matches[3]);
		if (in_array($path["extension"],$ignored))
			continue;
		$time = strtotime($matches[2]);
		$month = date(”M/y”,$time);
		$vals[$month]++;
	}

	function text_encode($vals,$max) {
		$tvals = array();
		foreach($vals as $val) {
			$tvals[] = round($val/$max*100,1);
		}
		return implode($tvals,”,”);
	}

	function get_y_axis($vals) {
		$max = max($vals)+3000;
		$step = $max/5.0;
		$temp = array();
		for($i=0;$i<5;$i++)
			$temp[] = round($i*$step);
		$temp[] = $max;
		return implode($temp,"|");
	}

	echo "http://chart.apis.google.com/chart?chs=500x300&chd=t:"
			.urlencode(text_encode($vals,max($vals)+3000)).
			"&cht=bvg&chbh=50&chxt=x,y&chxl=0:|".
			urlencode(implode(array_keys($vals),'|')).
			"|1:|".urlencode(get_y_axis($vals))."\n";

Even though this contains code for ommiting unrelated results, it takes too much time. So, you can use the piping to create a simple log file, which is reasonably speed than running this through the whole log file. For example I use this command to create a smaller version of the log file.

cat sandaru1_log | grep 'HTTP/1.1" 200' | grep -v '.css HTTP'
	| grep -v '.png HTTP' | grep -v '.js HTTP'
	| grep -v '.jpg HTTP' | grep -v '.gif HTTP' > log

Google Charts also provides ability to generate pie charts. It’s easier to use a pie chart for browser percentages.

Here is the code :

	$vals = array();

	function parseUserAgent($ua)
  	{

    	$userAgent = array();
 		$agent = $ua;
    	$products = array();

		$pattern  = "([^/[:space:]]*)” . “(/([^[:space:]]*))?”
		.”([[:space:]]*\[[a-zA-Z][a-zA-Z]\])?” . “[[:space:]]*”
		.”(\\((([^()]|(\\([^()]*\\)))*)\\))?” . “[[:space:]]*”;

		while( strlen($agent) > 0 )
		{
			if ($l = ereg($pattern, $agent, $a))
			{
				// product, version, comment
				array_push($products, array($a[1],    // Product
                                        $a[3],    // Version
                                        $a[6]));  // Comment
				$agent = substr($agent, $l);
			}
			else
			{
				$agent = “”;
			}
		}

		// Directly catch these
		foreach($products as $product)
		{
			switch($product[0])
			{
				case ‘Firefox’:
				case ‘Netscape’:
				case ‘Safari’:
				case ‘Camino’:
				case ‘Mosaic’:
				case ‘Galeon’:
				case ‘Opera’:
					$userAgent[0] = $product[0];
					$userAgent[1] = $product[1];
					break;
			}
		}

		if (count($userAgent) == 0)
		{
			// Mozilla compatible (MSIE, konqueror, etc)
			if ($products[0][0] == ‘Mozilla’ &&
            	!strncmp($products[0][2], ‘compatible;’, 11))
			{
				$userAgent = array();
				if ($cl = ereg(”compatible; ([^ ]*)[ /]([^;]*).*”,
                           $products[0][2], $ca))
				{
					$userAgent[0] = $ca[1];
					$userAgent[1] = $ca[2];
				}
				else
				{
					$userAgent[0] = $products[0][0];
					$userAgent[1] = $products[0][1];
				}
			}
			else
			{
				$userAgent = array();
				$userAgent[0] = $products[0][0];
				$userAgent[1] = $products[0][1];
			}
		}

		if (strstr($userAgent[1],”http:/”))
			$userAgent[1] = “”;

		return $userAgent[0]
			.($userAgent[0]==”"||$userAgent[1]==”"?”":” “)
			.$userAgent[1];
	}

	$f = fopen(”log”,”r”);
	while (!feof($f)){
		$line = fgets($f);
		preg_match(”/([\d.]+).* [^ ] [^ ] \[(.*?)\] (.*?) (.*) (.*)”
				.” (\d+) ([^ ]+) (.*?) \”(.*?)\”/”,
					$line,$matches);
		$bot = parseUserAgent($matches[9]);
		$bot = preg_replace(”/Firefox 2.*/”,”Firefox 2″,$bot);
		$bot = preg_replace(”/Firefox 1.*/”,”Firefox 1″,$bot);
		$vals[$bot]++;
	}

	$others = 0;

	foreach($vals as $key => $val)
		if ($val<500) {
			$others += $val;
			unset($vals[$key]);
		}

	$vals['Others'] = $others;
	$vals['Unknown'] = $vals['-'];
	unset($vals['-']);

	$lables = array();
	function text_encode($vals,$sum) {
		global $lables;
		$tvals = array();
		foreach($vals as $key => $val) {
			$tvals[] = round($val/$sum*100,1);
			$lables[] = $key.” (”.round($val/$sum*100,2).” %)”;
		}
		return implode($tvals,’,');
	}

	echo “http://chart.apis.google.com/chart?cht=p&chd=t:”.
			urlencode(text_encode($vals,array_sum($vals))).
			“&chs=700×400&chl=”.
			urlencode(implode($lables,”|”)).”\n”;

This code will get the user agent, parse it and generate a pie chart. The user agent parse function is by dotvoid.com.

This entry was posted on Thursday, January 31st, 2008 at 1:12 pm and is filed under Google, PHP, Programming. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.