gem install mysql failing with checking for mysql_query() in -lmysqlclient… no

Aside

If you run into an error similar to the follow while attempting to install mysql gem on OSX Lion

sudo gem install -V mysql -v '2.8.1'
GET http://rubygems.org/specs.4.8.gz
302 Moved Temporarily
-- snipped --
Installing gem mysql-2.8.1
/Library/Ruby/Gems/1.8/gems/mysql-2.8.1/COPYING
-- snipped --
Building native extensions.  This could take a while...
ERROR:  Error installing mysql:
	ERROR: Failed to build gem native extension.

/System/Library/Frameworks/Ruby.framework/Versions/
1.8/usr/bin/ruby extconf.rb
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lm... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lz... yes
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lsocket... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lnsl... no
checking for mysql_query() in -lmysqlclient... no
checking for main() in -lmygcc... no
checking for mysql_query() in -lmysqlclient... no
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/System/Library/Frameworks/Ruby.framework/
              Versions/1.8/usr/bin/ruby
	--with-mysql-config
	--without-mysql-config
	--with-mysql-dir
	--without-mysql-dir
	--with-mysql-include
	--without-mysql-include=${mysql-dir}/include
	--with-mysql-lib
	--without-mysql-lib=${mysql-dir}/lib
	--with-mysqlclientlib
	--without-mysqlclientlib
	--with-mlib
	--without-mlib
	--with-mysqlclientlib
	--without-mysqlclientlib
	--with-zlib
	--without-zlib
	--with-mysqlclientlib
	--without-mysqlclientlib
	--with-socketlib
	--without-socketlib
	--with-mysqlclientlib
	--without-mysqlclientlib
	--with-nsllib
	--without-nsllib
	--with-mysqlclientlib
	--without-mysqlclientlib
	--with-mygcclib
	--without-mygcclib
	--with-mysqlclientlib
	--without-mysqlclientlib

Gem files will remain installed in /Library/Ruby/Gems/1.8/gems/mysql-2.8.1 for inspection.
Results logged to /Library/Ruby/Gems/1.8/gems/mysql-2.8.1/ext/mysql_api/gem_make.out

You can fix it passing along the mysql_config5 param to the command. On Lion, if you installed mysql by using the command sudo /opt/local/bin/port install mysql5-server or something similar the command to install the gem is

sudo gem install -V -v '2.8.1' mysql -- --with-mysql-config=/opt/local/bin/mysql_config5 

Assembla Subversion Repository moving code to trunk

A colleague set up a new subversion repository on assembla and invited me in. Unfortunately he missed importing the code into the trunk directory and as a result all the code was out in the root of the repo.

A bit of googling gave the solution at http://svn.haxx.se/users/archive-2007-10/0600.shtml

Simply put

cd ~/localdir
svn checkout https://path/to/assembla/repo localdir
cd localdir
svn mkdir trunk
for f in *; do svn mv $f trunk; done
svn mkdir branches tags
svn commit -m 'Finally got organized'

Gallery Upgrade Issues

I had an issue today with a gallery upgrade in that I upgraded the gallery to 2.3.1 version as per the documentation. However, the galleries (there were about 14k images in about 9GB) displayed ok, while a couple of them had broken thumbnail placeholders. Gallery’s logs were no help and neither were apache’s logs.

I tried to use the maintenance section of the site admin to regenerate all thumbnails and derivatives, but got various errors including ERROR_STORAGE_FAILURE and ERROR_LOCK_FAILURE among a few others. I was also getting quite a lot of mysql server has gone away errors which probabaly were due to mysql query restrictions on the server to prevent abuse.

After a lot of googling and running through the faqs and troubleshooting sections of menalto’s gallery site, finally figured out a set of actions that seemed to temporarily fix the issue.
1. Go to http://site/lib/support, enter the setup password, and click on cache maintenance. Leave the logs and the thumbnails unchecked and run it.
2. In http://site/main.php after login, go to Site Admin>Maintenance>Delete Template Cache. If this gives some errors, it seemed ok to ignore it.
3. Open the database via phpmyadmin and run find the max value of g_id in g2_Entity. Increment the value by 1 and insert that value to the row in g2_Sequenceid (There should be only one row and if the value of the query ‘select max(g_id) from g2_Entity’ is 34523, the row in g2_Sequenceid should be 34524)

Now rerun the generate all thumbnails and derivatives from Site Admin>Maintenance

I still have issues running it as the memory limit seems to setup at 32mb. Working on fixing that.

ExtJS line Charts and improper string data from PHP

While working on the main dashboard for the current extjs project I am working I had to create a line chart which showed the performance over the last 1 week. This was across 3 different variables.

The code to show that with tips and all as an item in an ExtJS Panel is

new Ext.chart.LineChart({
                    store: new Ext.data.JsonStore({
                        url:PHP_AJAX_URL + '/hometab.php',
                        baseParams:{
                            task:'graphdata'
                        },
                        idProperty: 'gdate',
                        root:'data',
                        fields: ['gdate', 'f1', 'f2','f3'],
                        autoLoad: true
                    }),
                    xField:'gdate',
                    series:[
                    {
                        type:'line',
                        displayName: 'FONE',
                        yField: 'f1',
                        style: {
                            color: 0x889fbb
                        }
                    }, {
                        type: 'line',
                        displayName: 'FTWO',
                        yField: 'f2',
                        style: {
                            color: 0x889fcc
                        }
                    }, {
                        type:'line',
                        displayName: 'FTHREE',
                        yField: 'f3',
                        style: {
                            color: 0x6666bb
                        }
                    }
                    ],
                    tipRenderer: function (chart, record, index, series){
                        if (series.yField=='f1')
                            return Ext.util.Format.number(record.data.f1, '0,0') +
                            ' f1 on ' + record.data.gdate;
                        if (series.yField=='f2')
                            return Ext.util.Format.number(record.data.f2, '0,0')+
                            ' f2 on ' + record.data.gdate;
                        if (series.yField=='f3')
                            return Ext.util.Format.number(record.data.f3, '0,0') + 
                            ' f3 on ' + record.data.gdate;

                    },
                    extraStyle: {
                        padding: 10,
                        animationEnabled: true,
                        legend:{
                            display:'bottom'
                        },
                        xAxis: {
                            color: 0x3366cc,
                            majorGridLines: {
                                size: 0,
                                color: 0xdddddd
                            }
                        },
                        yAxis: {
                            color: 0x3366cc,
                            majorTicks: {
                                color: 0x3366cc,
                                length: 0
                            },
                            minorTicks: {
                                color: 0x3366cc,
                                length: 0
                            },
                            majorGridLines: {
                                size: 0,
                                color: 0xdddddd
                            }
                        }
                    }
                })

Like all other parts of the backend the code was being pulled via a quick mysql_query from the database and provided to the ExtJS frontend via JSON.

        $graph_query="select 
				date(f_date) as gdate,
                sum(f1) as f1,
                sum(f2) as f2,
                sum(f3) as f3
                from stats
				where id=user
                and date(f_date)>date_sub(current_date, interval 7 day)
                group by date(f_date)";
        $graph_recordset=mysql_query($graph_query);
		while($graph_row=mysql_fetch_assoc($graph_recordset)){
			$response['data']=$graph_row;
		}
		$response['success']=true;
		print json_encode($response);

However, the LineChart wasn’t rendering properly and it was showing wierd values for some and not rendering at all for the others. The axes were fine and firebug showed proper values for the JSON string.

After a bit of looking around, it seems that ExtJS does not convert from string numeric values to numeric values. Remember that mysql_fetch_assoc provides an associative array with string values. All that was needed was to manually cast to integer/float values that the LineChart needs.

The bit of modified code is below. Note the casts inside the while loop

        $graph_query="select 
				date(f_date) as gdate,
                sum(f1) as f1,
                sum(f2) as f2,
                sum(f3) as f3
                from stats
				where id=user
                and date(f_date)>date_sub(current_date, interval 7 day)
                group by date(f_date)";
        $graph_recordset=mysql_query($graph_query);
		while($graph_row=mysql_fetch_assoc($graph_recordset)){
			$graph_row['f1']=intval($graph_row['f1']); //or floatval as needed;
			$graph_row['f2']=intval($graph_row['f2]); //or floatval as needed;
			$graph_row['f3']=intval($graph_row['f3]); //or floatval as needed;
			$response['data']=$graph_row;
		}
		$response['success']=true;
		print json_encode($response);

To Cake or not to cake

To Cake or not
I am starting work on a project for a client that involves complicated workflows when the customer orders a product and also a simple user interface. Even thought I have worked with a few php frameworks, this is the first time I am starting solo on a project from scratch. While looking around for the best options that fit the bill, I shortlisted 3 options
1. Core PHP
2. Drupal
3. CakePHP or Zend Framework
Option 1 might be the simplest to start with but would be not be a wise decision since it would be a maintenance and extensibility nightmare. I started working on a prototype for Drupal, but a few weeks down the line, the limits of a CMS for such a project became obvious. The best option now seems to be to use a framework, which should have been obvious from the start in hindsight.
I considered going with Zend but Cake seems to be much lighter and also places more emphasis on convention which supposedly makes me a ‘better’ programmer. Additionally, the fact that Zend seems to be more feature rich that cake actually pushed me more towards cake since the project is quite simple technically. And Zend seems to be too powerful for that.
So I am off to the bakery.
Cheers and Peace!
Jeba Singh Emmanuel
————————————-
Those who would joyously march in rank and file have already earned my contempt, for they were given a large brain by accident when a spinal chord would have sufficed.
–Albert Einstein

I am starting work on a project for a client that involves complicated workflows when the customer orders a product and also a simple user interface. Even thought I have worked with a few php frameworks, this is the first time I am starting solo on a project from scratch. While looking around for the best options that fit the bill, I shortlisted 3 options

1. Core PHP

2. Drupal

3. CakePHP or Zend Framework

Option 1 might be the simplest to start with but would be not be a wise decision since it would be a maintenance and extensibility nightmare. I started working on a prototype for Drupal, but a few weeks down the line, the limits of a CMS for such a project became obvious. The best option now seems to be to use a framework, which should have been obvious from the start in hindsight.

I considered going with Zend but Cake seems to be much lighter and also places more emphasis on convention which supposedly makes me a ‘better’ programmer. Additionally, the fact that Zend seems to be more feature rich that cake actually pushed me more towards cake since the project is quite simple technically. And Zend seems to be too powerful for that.

So I am off to the bakery.

Exim and Multiple Domains – PHP Mail function and junk mail issue

I am now working at a place where I get a chance to home my php, javascript and mysql skills, work with cake and other similar projects The job has System Admin work also and it is fun since I get to learn lots of new things. At the moment, I am just being given debugging and troubleshooting to do on the servers, but it is a matter of time before I impress and blow my team leader away. :D

Meanwhile, one issue that came up was a client with a server that had multiple domains on it. While sending mails with php’s mail function, the Received and other headers were not being set correctly, because of which mails to hotmail and live.com were ending up in the junk folder.

For eg.

Return-Path: <admin@correct.com>
Received: from noc.wrong.com ([76.76.11.105])
by mx.google.com with ESMTP id 42si3780510ywh.71.2009.;
Wed, 11 Nov 2009 22:59:31 -0800 (PST)
Received-SPF: neutral (google.com: .......;
Authentication-Results: mx.google.com; .......
Received: from wronguser by noc.wrong.com with local(Exim 4.69)
(envelope-from <admin@wrong1.com>)ntid 1N8TeT-0007On-F6ntfor jebasingh.emmanuel@gmail.com ; Thu, 12 Nov 2009 01:59:29 -0500

See the envelope-from? That was not being set properly event by using the 5th parameter in php mail as ‘-fadmin@correct.com’

The fix for that was easy, set

trusted_users=domainusername

in /etc/exim.conf.

The trusted_users variable is a colon delimited list of user accounts whose domains should be allowed to send mail as themselves.

Now for the actual code fixes, which can be found here.

General Exim method which doesn’t exactly work as advised. Might need modifications.

WHM specific method.

The problem with the first method was that $interface_address was not set at all for outgoing mail. Since we do not receive incoming mail for our domains, I didn’t get to check how the incoming mails were working out. If you are using WHM, however, you just need to create the files /etc/mailhelo and /etc/mailips and you are good to go after restarting exim. (test with exim -bV first)

I get these headers after the edit.


Return-Path: <admin@correct.com>
Received: from correct.com ([76.76.11.105])
by mx.google.com with ESMTP id 42si3780510ywh.71.2009.;
Wed, 11 Nov 2009 22:59:31 -0800 (PST)
Received-SPF: neutral (google.com: …….;
Authentication-Results: mx.google.com; …….
Received: from correctuser by noc.wrong.com with local(Exim 4.69)
(envelope-from &amp;lt;admin@correct.com&amp;gt;)ntid 1N8TeT-0007On-F6ntfor jebasingh.emmanuel@gmail.com ; Thu, 12 Nov 2009 01:59:29 -0500</pre>

Couldn’t figure out how to set the smtp_active_hostname in exim.conf since $interface_address wasn’t being set. Still, the mails no longer end up in the junk mail folder on gmail, yahoo, aol and hotmail, so I think I am golden.

Took ages to get the above links. Just hope it helps someone else too.

Check if a server is down and email admin

A simple shell cronjob to mail the admin when a particular ip is down.

ping -c 1 192.168.0.99 || echo “ipdown” | mail -s “down” jebasingh.emmanuel@gmail.com

Or a bit more larger version

PINGCOUNT=4
IPADDR="192.168.0.59"

COUNT=`ping -c $PINGCOUNT $IPADDR|grep 'received'|awk -F ',' '{print $2}'|awk '{print $1}'`
if [ $COUNT -eq 0 ]; then
  echo "Ping reported no responses from $IPADDR over $PINGCOUNT pings."|mail -s "Ping Failure" "jebasingh.emmanuel@gmail.com"
fi

Function to create traffic graph in PHP with gd

Just finished a couple of projects in which this code came in useful.

You just pass in an array of traffic date of the format
arrayname[0]['date']=’15/06′;
arrayname[0]['data']=200;
arrayname[1]['date']=’16/06′;
arrayname[1]['data']=100;

I am Indian so the data code is in format d/m, but m/d should not matter either if you create the array in proper order. Also if the data is greater than 1024 for any of the array values, the graph switches to GB display.

Here is a sample image generated by this code.

Sample graph

Sample graph

Below is the code, enjoy! Let me know if it helped you.

<?php
function generategraph($traffic){
$days=count($traffic);
$imagewidth=700; $imageheight=300;
$yaxistextarea=35; 	$lefttextarea=30; $xaxistextarea=30; $bottomtextarea=30;
$topmargin=20; 	$rightmargin=20; $graphtop=10;
$yaxissplits=10;
$datascale=1;
$dataunit='MBs';

$graphwidth=$imagewidth-$yaxistextarea-$lefttextarea-$rightmargin;
$graphheight=$imageheight-$xaxistextarea-$bottomtextarea-$topmargin-$graphtop;

//xaxis is one greater because one line the start and end of the graph will not have any bars on them.
$xaxissplits=$days+1;
$barwidth=$graphwidth/($xaxissplits+1)/2;

if ($barwidth>30) $barwidth=30;
$i=0;
$maxvalue=0;$minvalue=0;$totaldata=0;
foreach($traffic as $value){
if ($maxvalue<$value&#91;'data'&#93;) $maxvalue=$value&#91;'data'&#93;;
$totaldata+=$value&#91;'data'&#93;;
}

if ($maxvalue>=1024) { $datascale=1024; $dataunit='GBs';}

//The y axis is for the amount of traffic. so we find the minimum of the incoming and the outcoming traffic and
//it will be the lower and upper bounds.

//Since we will have a graph of only 20 lines, we split the difference into 20 and find the linerange
$linerange=($maxvalue-$minvalue)/$yaxissplits;
$graphscale=$graphheight/($maxvalue-$minvalue);

//we now have a neat map of i => in and out
//remember that each linerange is equal to
$im = ImageCreate ($imagewidth,$imageheight);
if (!$im) echo "Error in gd library";

//some of these variables are not used in the current code, but nice to let it remain in case future coding requires additions like borders around bars, lines etc.
imagecolorallocatealpha($im,255,255,255,100);
$imagebackground=ImageColorAllocate($im, 255,255,255);
$graphbackground=ImageColorAllocate($im, 255,255,255);
$textcolor=ImageColorAllocate($im, 100,100,100);
$datalinecolor=ImageColorAllocate($im, 100, 255,100);
$databarcolor=ImageColorAllocate($im, 233,131,18);
$databarbordercolor=ImageColorAllocate($im, 100, 100,100);
$axiscolor=ImageColorAllocate($im, 50,50,50);
$axislinecolormajor=ImageColorAllocate($im, 186,180,24);
$axislinecolorminor=ImageColorAllocate($im, 255,255,255);

//create some frequently used variables
//these are the min and max values on the graph.
$xmin=$yaxistextarea + $lefttextarea;
$ymin=$imageheight-$bottomtextarea-$xaxistextarea;
$xmax=$imagewidth-$rightmargin;
$ymax=$topmargin;

//create the graph area
imagefilledrectangle($im, $xmin, $ymin, $xmax, $ymax, $graphbackground);
imagestringup($im, 3, 0+5, $ymin-30 , "Data Transmitted in ".$dataunit, $textcolor);

for ($i=1; $i<=$yaxissplits; $i++){
//each major y axis line locations
$ymajor = $ymin - ($i * ($graphheight/$yaxissplits));
imagestring($im, 2,  $lefttextarea,$ymajor -5, round(($minvalue + $i * $linerange)/$datascale,2), $textcolor);
$yminor = $ymajor + ($graphheight/$yaxissplits)/2;
imageline($im, $xmin, $ymajor, $xmax, $ymajor, $axislinecolormajor);
}

for ($i=1; $i<$xaxissplits; $i++){
//each major x axis line locations
$xmajor = $xmin + ($i * ($graphwidth/$xaxissplits));
$xminor = $xmajor - ($graphwidth/$xaxissplits)/2;
imageline($im, $xmajor, $ymin, $xmajor, $ymax, $axislinecolormajor);
imagestringup($im, 1,  $xmajor-3, $ymin+$xaxistextarea, $traffic&#91;$i-1&#93;&#91;'date'&#93;, $textcolor);
}

//plot the actual graph
$plotxold=$xmin; $plotyold=$ymin;
for($key=1; $key<=$xaxissplits-1; $key++){
$value=$traffic&#91;$key-1&#93;;
$plotx=$xmin + ($key) *($graphwidth/$xaxissplits);
$ploty= $ymin - ($value&#91;'data'&#93; - $minvalue) * $graphscale;
if ($plotx>$xmax) $plotx=$xmax; //to avoid bleed due to rounding off of graphscale
if ($ploty<$ymax) $ploty=$ymax; //to avoid bleed due to rounding off of graphscale
$plotxold=$plotx; $plotyold=$ploty;
if ($value&#91;'data'&#93;>0){
imagefilledrectangle($im, $plotx-$barwidth/2, $ymin, $plotx+$barwidth/2, $ploty, $databarcolor);
}
}
$plotxold=$xmin; $plotyold=$ymin;
imagestring($im, 2,  $yaxistextarea+$lefttextarea,$imageheight-$bottomtextarea*3/4, "Total Data: " . $totaldata . " ".$dataunit, $textcolor);
imagestring($im, 1,  $lefttextarea,5, "Image generated at " . date('d/m/y  H:i'), $textcolor);
imagerectangle($im, $xmin, $ymin, $xmax, $ymax, $axiscolor);
imagejpeg($im, "graph.jpg");
return "<div><img src='graph.jpg' /></div>";
?>

Positioning text vertically and horizontally using gd

I was working on a PHP script yesterday for creating a couple of graphs which required lots of text in various places aligned to text on the same line. Some on the x axix and some on y. Finding no inbuilt function in gd on php, I created this simple function to align text vertically and horizontally as required based on the parameters passed to it.

[sourcecode language="php"]

function positiontextinimage($img, $font, $size, $color, $string, $x,$y, $horiz, $vert)
{
$bounds=imageftbbox  ( $size  , 0  , $font  , $string);
switch($horiz){
case “left”:
break;
case “right”:
$x=$x-$bounds[4];
break;
case “center”:
case “centre”:
$x=$x-($bounds[4] -$bounds[0])/2;
break;
}
switch($vert){
case “top”:
$y=$y-$bounds[5];
break;
case “bottom”:
break;
case “middle”:
$y=$y+($bounds[1]-$bounds[5])/2;

break;

}
imagefttext  ( $img  , $size  , 0  , $x, $y, $color, $font  , $string);
}

[/sourcecode]

PHP Proxy script to solve javascript and jquery cross domain issues

I just finished a project in which a jquery dialog used a jquery.ajax call using get to obtain details of a transaction from a url on another domain, effectively a cross domain ajax request. Since firefox 3.0 does not allow itself to be set to allow cross domain requests, I had to use some other way. I could have just used an iframe, but that is another story in itself.

Here is the bit of javascript code.

[sourcecode language="js"]

function refaxorder(){
//idfield=$(“#idfield”).val();
query=”urlid=1&inOID=” + idfield + “&inKey=” + key + “&inAction=reFax”;
$(“#actiondiv”).html(“

Loading…

“);
$(“#actiondivin”).dialog({autoOpen:false, modal:true, title:”ReFax”, buttons:{OK: function(){$(this).dialog(‘destroy’).remove(); }}});;
$(“#actiondivin”).dialog(“open”);

$.ajax({type:”GET”, url: “proxy.php”, data: query,  dataType:”html”, success:function(msg){
$(“#actiondivin”).html(msg);
}
});
}

[/sourcecode]

To get around this, I implemented a couple of lines as proxy.php to act as a proxy. Please note that I am only allowing it to be passed the urlid instead of a url to prevent it from becoming an open proxy. Also IP based filtering could be implemented to make it more safer. Have a look, maybe it will come in useful for you too.

[sourcecode language='php']

$value){
if ($key!=’url’) $query[]=$key.”=”.$value;
}
if (count($query)) $querystring=implode(“&”,$query);
$fullurl=$url[$urlid].”?”.$querystring;

$fp=fopen($fullurl,’rb’);
$page = ”;
if ($fp===false) die(“Unable to open”);
while (!feof($fp)) {
$page .= fread($fp, 8192);
}
fclose($fp);
print $page;
?>

[/sourcecode]