Mysql and Top pruner

The server had a setup in which a cronjob was used to dump output of top command to a directory accessible via http once every 5 minutes. To get rid of the logs and also to prune mysql logs the following script was made.


#! /bin/sh

LOGGER_PATH='/usr/local/apache/htdocs/top/'
N_DAY=3
MYSQL_DATA_DIR='/var/lib/mysql'
LOGSTOKEEP=3

################################################
#find and delete all files
#in logger path older than n days
################################################

find $LOGGER_PATH -mtime +$N_DAY -exec rm -f {} ;

################################################
#mysql pruner section.
#flush logs first,
#find the last filename
#and purge binary logs upto that
################################################

#mysql flush logs here

FILENAMES=`ls $MYSQL_DATA_DIR | grep -e "^mysql-bin.0+[[:digit:]]+$"|sort|tail -n $LOGSTOKEEP`
LINECOUNT=`echo $FILENAMES|wc -w`

#linecount will be lesser than or equal to the number of logs we want to keep

if [ $LINECOUNT -lt $LOGSTOKEEP ]; then
echo "$LOGSTOKEEP files were not found. So no purge is being done."
else
LASTFILENAME=`echo $FILENAMES|cut -d' ' -f 1`
#do purge upto lastfilename
fi
exit

Check if a server is down and email admin

A simple shell cronjob to mail the admin when a particular ip is down.

ping -c 1 192.168.0.99 || echo “ipdown” | mail -s “down” jebasingh.emmanuel@gmail.com

Or a bit more larger version

PINGCOUNT=4
IPADDR="192.168.0.59"

COUNT=`ping -c $PINGCOUNT $IPADDR|grep 'received'|awk -F ',' '{print $2}'|awk '{print $1}'`
if [ $COUNT -eq 0 ]; then
  echo "Ping reported no responses from $IPADDR over $PINGCOUNT pings."|mail -s "Ping Failure" "jebasingh.emmanuel@gmail.com"
fi

Function to create traffic graph in PHP with gd

Just finished a couple of projects in which this code came in useful.

You just pass in an array of traffic date of the format
arrayname[0]['date']=’15/06′;
arrayname[0]['data']=200;
arrayname[1]['date']=’16/06′;
arrayname[1]['data']=100;

I am Indian so the data code is in format d/m, but m/d should not matter either if you create the array in proper order. Also if the data is greater than 1024 for any of the array values, the graph switches to GB display.

Here is a sample image generated by this code.

Sample graph

Sample graph

Below is the code, enjoy! Let me know if it helped you.

<?php
function generategraph($traffic){
$days=count($traffic);
$imagewidth=700; $imageheight=300;
$yaxistextarea=35; 	$lefttextarea=30; $xaxistextarea=30; $bottomtextarea=30;
$topmargin=20; 	$rightmargin=20; $graphtop=10;
$yaxissplits=10;
$datascale=1;
$dataunit='MBs';

$graphwidth=$imagewidth-$yaxistextarea-$lefttextarea-$rightmargin;
$graphheight=$imageheight-$xaxistextarea-$bottomtextarea-$topmargin-$graphtop;

//xaxis is one greater because one line the start and end of the graph will not have any bars on them.
$xaxissplits=$days+1;
$barwidth=$graphwidth/($xaxissplits+1)/2;

if ($barwidth>30) $barwidth=30;
$i=0;
$maxvalue=0;$minvalue=0;$totaldata=0;
foreach($traffic as $value){
if ($maxvalue<$value&#91;'data'&#93;) $maxvalue=$value&#91;'data'&#93;;
$totaldata+=$value&#91;'data'&#93;;
}

if ($maxvalue>=1024) { $datascale=1024; $dataunit='GBs';}

//The y axis is for the amount of traffic. so we find the minimum of the incoming and the outcoming traffic and
//it will be the lower and upper bounds.

//Since we will have a graph of only 20 lines, we split the difference into 20 and find the linerange
$linerange=($maxvalue-$minvalue)/$yaxissplits;
$graphscale=$graphheight/($maxvalue-$minvalue);

//we now have a neat map of i => in and out
//remember that each linerange is equal to
$im = ImageCreate ($imagewidth,$imageheight);
if (!$im) echo "Error in gd library";

//some of these variables are not used in the current code, but nice to let it remain in case future coding requires additions like borders around bars, lines etc.
imagecolorallocatealpha($im,255,255,255,100);
$imagebackground=ImageColorAllocate($im, 255,255,255);
$graphbackground=ImageColorAllocate($im, 255,255,255);
$textcolor=ImageColorAllocate($im, 100,100,100);
$datalinecolor=ImageColorAllocate($im, 100, 255,100);
$databarcolor=ImageColorAllocate($im, 233,131,18);
$databarbordercolor=ImageColorAllocate($im, 100, 100,100);
$axiscolor=ImageColorAllocate($im, 50,50,50);
$axislinecolormajor=ImageColorAllocate($im, 186,180,24);
$axislinecolorminor=ImageColorAllocate($im, 255,255,255);

//create some frequently used variables
//these are the min and max values on the graph.
$xmin=$yaxistextarea + $lefttextarea;
$ymin=$imageheight-$bottomtextarea-$xaxistextarea;
$xmax=$imagewidth-$rightmargin;
$ymax=$topmargin;

//create the graph area
imagefilledrectangle($im, $xmin, $ymin, $xmax, $ymax, $graphbackground);
imagestringup($im, 3, 0+5, $ymin-30 , "Data Transmitted in ".$dataunit, $textcolor);

for ($i=1; $i<=$yaxissplits; $i++){
//each major y axis line locations
$ymajor = $ymin - ($i * ($graphheight/$yaxissplits));
imagestring($im, 2,  $lefttextarea,$ymajor -5, round(($minvalue + $i * $linerange)/$datascale,2), $textcolor);
$yminor = $ymajor + ($graphheight/$yaxissplits)/2;
imageline($im, $xmin, $ymajor, $xmax, $ymajor, $axislinecolormajor);
}

for ($i=1; $i<$xaxissplits; $i++){
//each major x axis line locations
$xmajor = $xmin + ($i * ($graphwidth/$xaxissplits));
$xminor = $xmajor - ($graphwidth/$xaxissplits)/2;
imageline($im, $xmajor, $ymin, $xmajor, $ymax, $axislinecolormajor);
imagestringup($im, 1,  $xmajor-3, $ymin+$xaxistextarea, $traffic&#91;$i-1&#93;&#91;'date'&#93;, $textcolor);
}

//plot the actual graph
$plotxold=$xmin; $plotyold=$ymin;
for($key=1; $key<=$xaxissplits-1; $key++){
$value=$traffic&#91;$key-1&#93;;
$plotx=$xmin + ($key) *($graphwidth/$xaxissplits);
$ploty= $ymin - ($value&#91;'data'&#93; - $minvalue) * $graphscale;
if ($plotx>$xmax) $plotx=$xmax; //to avoid bleed due to rounding off of graphscale
if ($ploty<$ymax) $ploty=$ymax; //to avoid bleed due to rounding off of graphscale
$plotxold=$plotx; $plotyold=$ploty;
if ($value&#91;'data'&#93;>0){
imagefilledrectangle($im, $plotx-$barwidth/2, $ymin, $plotx+$barwidth/2, $ploty, $databarcolor);
}
}
$plotxold=$xmin; $plotyold=$ymin;
imagestring($im, 2,  $yaxistextarea+$lefttextarea,$imageheight-$bottomtextarea*3/4, "Total Data: " . $totaldata . " ".$dataunit, $textcolor);
imagestring($im, 1,  $lefttextarea,5, "Image generated at " . date('d/m/y  H:i'), $textcolor);
imagerectangle($im, $xmin, $ymin, $xmax, $ymax, $axiscolor);
imagejpeg($im, "graph.jpg");
return "<div><img src='graph.jpg' /></div>";
?>

Python script to get links from yahoo search

This was a quick script I made to pull links from yahoo search using the boss search api, and then list the unique domains.

If you want the entire links, just modify so that the whole links are appended to the list. Yahoo does not allow to get all the results, but only a certain predefined number so this code only extracts about 800 domains. But it is still good enough for a start and for most uses.

I am also working on getting citation values for google scholar for a friend. I will post that soon here. Heres the code for now.

#! /usr/bin/python
import urllib,json
from urlparse import urlparse

yahoo_application_id="Ht18VqTV34EMRWTJKOOh4rNBWTqkrjTSSQj9JwWlsqTMK41_3oFWFnhivJipX0wnvU4qzXc9VAw-"
nextresult=0;
links=list()
linksdump=list()
#print yahoo_application_id

#print "http://boss.yahooapis.com/ysearch/web/v1/Jeba+Singh+Emmanuel?appid="+yahoo_application_id+"&format=xml"
while(True):
	print "trying result from " + str(nextresult)
	f = urllib.urlopen("http://boss.yahooapis.com/ysearch/web/v1/search+engine+optimization+software?appid="+yahoo_application_id+"&format=json&count=100&start="+str(nextresult))
	ss=json.JSONDecoder()
	ssjson= ss.decode(f.read())
	#count=ssjson["ysearchresponse"]["count"]
	#start=ssjson["ysearchresponse"]["start"]
	totalhits=int(ssjson["ysearchresponse"]["totalhits"])
	print totalhits
	for x in ssjson["ysearchresponse"]["resultset_web"]:
		url= x["url"]
		o = urlparse(url)
		linksdump.append(url)
		link = o[0]+"://"+o[1]
		if link not in links:
			links.append(link)
		nextresult=nextresult+1
	if (nextresult>10000):
		break
print "Obtained results: " + str(nextresult) + " of which " + str(len(links)) + " were unique."
for x in links:
	print x

Cool huh? If you want any help modifying this, drop me a line.

Positioning text vertically and horizontally using gd

I was working on a PHP script yesterday for creating a couple of graphs which required lots of text in various places aligned to text on the same line. Some on the x axix and some on y. Finding no inbuilt function in gd on php, I created this simple function to align text vertically and horizontally as required based on the parameters passed to it.

[sourcecode language="php"]

function positiontextinimage($img, $font, $size, $color, $string, $x,$y, $horiz, $vert)
{
$bounds=imageftbbox  ( $size  , 0  , $font  , $string);
switch($horiz){
case “left”:
break;
case “right”:
$x=$x-$bounds[4];
break;
case “center”:
case “centre”:
$x=$x-($bounds[4] -$bounds[0])/2;
break;
}
switch($vert){
case “top”:
$y=$y-$bounds[5];
break;
case “bottom”:
break;
case “middle”:
$y=$y+($bounds[1]-$bounds[5])/2;

break;

}
imagefttext  ( $img  , $size  , 0  , $x, $y, $color, $font  , $string);
}

[/sourcecode]

PHP Proxy script to solve javascript and jquery cross domain issues

I just finished a project in which a jquery dialog used a jquery.ajax call using get to obtain details of a transaction from a url on another domain, effectively a cross domain ajax request. Since firefox 3.0 does not allow itself to be set to allow cross domain requests, I had to use some other way. I could have just used an iframe, but that is another story in itself.

Here is the bit of javascript code.

[sourcecode language="js"]

function refaxorder(){
//idfield=$(“#idfield”).val();
query=”urlid=1&inOID=” + idfield + “&inKey=” + key + “&inAction=reFax”;
$(“#actiondiv”).html(“

Loading…

“);
$(“#actiondivin”).dialog({autoOpen:false, modal:true, title:”ReFax”, buttons:{OK: function(){$(this).dialog(‘destroy’).remove(); }}});;
$(“#actiondivin”).dialog(“open”);

$.ajax({type:”GET”, url: “proxy.php”, data: query,  dataType:”html”, success:function(msg){
$(“#actiondivin”).html(msg);
}
});
}

[/sourcecode]

To get around this, I implemented a couple of lines as proxy.php to act as a proxy. Please note that I am only allowing it to be passed the urlid instead of a url to prevent it from becoming an open proxy. Also IP based filtering could be implemented to make it more safer. Have a look, maybe it will come in useful for you too.

[sourcecode language='php']

$value){
if ($key!=’url’) $query[]=$key.”=”.$value;
}
if (count($query)) $querystring=implode(“&”,$query);
$fullurl=$url[$urlid].”?”.$querystring;

$fp=fopen($fullurl,’rb’);
$page = ”;
if ($fp===false) die(“Unable to open”);
while (!feof($fp)) {
$page .= fread($fp, 8192);
}
fclose($fp);
print $page;
?>

[/sourcecode]

Unable to uninstall Audio Driver – Realtek, Sound not working

A system at the office was having the Realtek audio control panel icon showing up in the system tray, with the 3d test working, but windows wasn’t showing the audio device in the sound and audio control panel applet. Instead windows said that not audio playback device was found. As a result no applications were able to play audio.

This post at tomshardware.com provided an easy solution.

Just open regedit and delete the key folder //HKEY_LOCAL_MACHINE/SYSTEM/CONTROLSET001/ENUM/HDAUDIO and restart your computer. It should prompt you to install the hardware again.

That fixed the hardware issue and the user was able to view www.abcnews.go.com/whatwouldyoudo happily. :)

Python script to get addresses from google maps

A simple python script to get addresses of businesses in a city. Just a quick demo for a client I wrote in an hour.

import urllib;
def getdata(idstr, matchstr):
	matchstrlen=len(matchstr)
line=idstr[idstr.find(matchstr,0)+matchstrlen:idstr.find(""",idstr.find(matchstr,0)+matchstrlen)]
	return line
url="http://maps.google.ca/maps"
data="f=q&source=s_q&output=js&hl=en&geocode="
location="&q=" + urllib.quote("airport loc: New Delhi, India") + "&btnG=" + urllib.quote("Search Maps")

fp=urllib.urlopen(url + "?" + data + location)
filecontents=''
for line in fp.readlines():
	filecontents=filecontents + line
morecontent=True
startloc=0
while morecontent==True:
	startpos=filecontents.find("id:", startloc)
	if startpos>-1:
		endpos=filecontents.find("}}}", startpos)
		if endpos>-1:
			startloc=endpos+1
			section=filecontents[startpos:endpos]
			sxti=getdata(section, "sxti:"")
			sxsn=getdata(section, "sxsn:"")
			sxst=getdata(section, "sxst:"")
			sxpr=getdata(section, "sxpr:"")
			sxpo=getdata(section, "sxpo:"")
			sxph=getdata(section, "sxph:"")
			actual_url=getdata(section, "actual_url:"")
			print sxti + ", " + sxsn + ", " + sxst + ", " + sxpr + ", " + sxpo + ", " + sxph + ", " + actual_url
		else:
			morecontent=False
	else:
		morecontent=False

PHP and XML RPC – Searching for values

Many servers provide xmlrpc interfaces which allow other web applications to call and execute functions. It is actually quite simple once you get the hang of it. The servers also return the response and any variables as xmlrpc response messages which are xml responses basically

There are probably a lot of ways to get this done, but this actually turned out pretty well using a combination of the phpxmlrpc library and the Domdoc class in php.

Let us say that the server url is https://rpc.server.com/admin/admin at port 4567.
In order to make the call you first have to download the phpxmlrpc library from http://phpxmlrpc.sourceforge.net/ and extract it to a folder in your web directory root.

In the beginning you may want to turn the debugging feature of the rpc client on. To do uncomment the setDebug line in the code.

<?php

//error_reporting(E_ALL);

//ini_set("display_errors", 1);

require_once('xmlrpc-2.2.2/lib/xmlrpc.inc');

//You do not need to set the transport as https here.

$xmlrpc_client=new xmlrpc_client('admin/admin', 'rpc.server.com',4567);

//$xmlrpc_client->setDebug(1);

//An xmlrpc call without any parameters is below.

$xmlrpc_msg=new xmlrpcmsg('rpcFunctionName');

//the next one is an xmlrpc call with parameters.

//$xmlrpc_msg=new xmlrpcmsg('rpcFunctionName', array(new xmlrpcval(7, "int"),new xmlrpcval("", "string"),new xmlrpcval(2, "int")));

//Here is where you set the transport as https. Look at the manual for more options

$xmlrpc_resp=$xmlrpc_client->send($xmlrpc_msg, 200, 'https');

if ($xmlrpc_resp==False)

{

print "No response";

die ('Error');

}

if (!$xmlrpc_resp->faultCode())

{

//If you just want the xml, print out the return value from serialize()

//print $xmlrpc_resp->serialize() ;

$dom=new DomDocument();

$dom->loadXML($xmlrpc_resp->serialize());

//examine the xml to find out the path to the actual things you need.

$xpathString = "//methodResponse/params/param/value/struct/member[name='a_value']/value/string";

$xp=new DOMXPath($dom);

$domNodeList = $xp->query($xpathString);

foreach($domNodeList as $domNode){

//You may need to do more xpath queries as $xp->query($anotherXpathstr, $domNode). search will be done under $domNode

$server_name=$domNode->nodeValue ;

}

}

else

{

print "Error: " . $xmlrpc_resp->faultString();

}

?>

Also do look up the documentation for xmlrpc_client property return_type.