GraphWeather PHP for 3600?

[James007]

Your excellent PHP software says Graph Weather est un ensemble de script php permettant de tracer des courbes météo à partir d'un fichier history.dat généré par les stations météo WS2300. ... but I've a 3600, and therefore it won't work with these files.

The file format for both files are at http://www.niftythings.org/HeavyWeather ... Format.txt which contains the differing file formats; I've tried to change the PHP, but have failed to make it make any sense, unfortunately.

Is there anyone who might be able to help?

Je suis desole: je ne parle pas francais; mais est possible que me lire francais. J'espere que vous m'aider.

[TiToine]

Hi,

You must rewrite a major part of mysqlweather.php in this way :
- modify the method to retrieve numbers of records, last and first timestamp
- modify the unpack string
- possibly add sql field (requieres other changes in mysqlweather_mean.php)

So, you should try this script...I've made all changes, but not tested...
GraphWeather PHP is beta beta version, in rewriting this, i've found
minors bugs, and odd algorithm (i was very inspirated the day I wrote it )

Code : Tout sélectionner

<? 
// ----------------------------------------------------------------------
// - UPDATE data DATABASE
// ----------------------------------------------------------------------

echo '## UPDATE data DATABASE ##<br>';

// $INFORMATION_LENGTH=28;	no infos at the end of WS3600 file

$DATA_LENGTH=56;	// Size is now 56 bytes
$DELPHI1970=25569	// need a conversion from delphi to unix timestamp

// Get file size
$HISTORY_DAT_SIZE=filesize($HISTORY_DAT);

// Get the numbers of new records
$handle=@fopen($HISTORY_DAT,"r");
if ($handle)
{
	$RECORDS=$HISTORY_DAT_SIZE/$DATA_LENGTH;

	$buffer=fread($handle, 8);	// timestamp is double so read 8 bytes
	$date=unpack("d",$buffer);	// unpack timestamp as double
	$FIRST_HISTORYDAT_RECORD=($date-$DELPHI1970)*86400; // delphi/unix conversion

	fseek($handle,-$DATA_LENGTH,SEEK_END);
	$buffer=fread($handle, 8);
	$date=unpack("d",$buffer);
	$LAST_HISTORYDAT_RECORD= ($date-$DELPHI1970)*86400;

	echo 'Records in history.dat : ',$RECORDS,'<br>';
	echo 'First record in history.dat : ',date("d/m/y H:i",$FIRST_HISTORYDAT_RECORD),'<br>';
	echo 'Last record in history.dat : ',date("d/m/y H:i",$LAST_HISTORYDAT_RECORD),'<br>';

	$sql="SELECT MAX(`timestamp`) FROM data";
	$query=mysql_query($sql);
	$list=mysql_fetch_array($query);
	
	if (isset($list[0]))
	{
		$LAST_UPDATE=$list[0];
		echo 'Last update : ',date("d/m/y H:i", $LAST_UPDATE),'<br>';

		if($HISTORY_DAT_SIZE>0)
		{
			// iter from the end of file
			$RECORDS_TO_UPDATE=-1;
			fseek($handle,-$DATA_LENGTH,SEEK_END);
			do
			{
				$buffer=fread($handle, 8);
				$date=unpack("d",$buffer);
				$date=($date-$DELPHI1970)*86400;
				$RECORDS_TO_UPDATE++;

				if ((ftell($handle)-$DATA_LENGTH-8)<0) break;
				fseek($handle,-$DATA_LENGTH-8,SEEK_CUR);
			}
			while ($date>$LAST_UPDATE);
			fseek($handle,$DATA_LENGTH-8,SEEK_CUR);
		}
		else $RECORDS_TO_UPDATE=0;
	}
	else 
	{
		$RECORDS_TO_UPDATE=$RECORDS;
		fseek($handle,0);
	}

	echo 'Records to update : ',$RECORDS_TO_UPDATE,'<br>';

	if ($RECORDS_TO_UPDATE!=0)
	{
		$sql = "INSERT INTO data (Timestamp,abs_pressure,wind_speed,wind_direction,wind_chill,total_rainfall,indoor_temp,outdoor_temp,outdoor_dew_point,indoor_humidity,outdoor_humidity)";
		$sql.= "VALUES ";
	
	for($i=0;$i<$RECORDS_TO_UPDATE;$i++)
	{

		$unpackstr="dTimestamp/fAbsPressure/fPressure/fWindSpeed/S2WindDirection/fWindGust/fTotalRainfall";
		$unpackstr.="/fNewRainfall/fIndoorTemp/fOutdoorTemp";
		$unpackstr.="/fIndoorHumidity/fOutdoorHumidity/VUnknown";

		$buffer=fread($handle,$DATA_LENGTH);
		$data=unpack($unpackstr,$buffer);

		$timestamp=($Timestamp-$DELPHI1970)*86400;
	
		//Dewpoint calculation
		$es=6.11*pow(10,7.5*$data["OutdoorTemp"]/($data["OutdoorTemp"]+237.7));
		$data["OutdoorDewPoint"]=( 237.7*log($es*$data["OutdoorHumidity"]/611,10) )/( 7.5-log($es*$data["OutdoorHumidity"]/611,10) );

		//Windchild calculation


		if (($data["OutdoorTemp"]<5)&&($data["WindSpeed"]>5))
		{	
			$data["WindChill"]=13.12+0.6215*$data["OutdoorTemp"]-11.37*pow($data["WindSpeed"],0.16)+0.3965*$data["OutdoorTemp"]*pow($data["WindSpeed"],0.16);
		}
		else $data["WindChill"]=$data["OutdoorTemp"];

		$data["Pressure"]=round($data["Pressure"],1);
		$data["WindSpeed"]=round($data["WindSpeed"]*3.6,1);
		$data["WindChill"]=round($data["WindChill"],1);
		$data["TotalRainfall"]=round($data["TotalRainfall"],1);
		$data["IndoorTemp"]=round($data["IndoorTemp"],1);
		$data["OutdoorTemp"]=round($data["OutdoorTemp"],1);
		$data["OutdoorDewPoint"]=round($data["OutdoorDewPoint"],1); 
		$data["IndoorHumidity"]=round($data["IndoorHumidity"],1);
		$data["OutdoorHumidity"]=round($data["OutdoorHumidity"],1);
	
		/*
		$count=$i+1;
		echo "Record $count at ".date("d/m/y H:i",$timestamp)." ";
		echo $data["Unknown"]." - ";
		echo $data["Timestamp1"]." - ";
		echo $data["Timestamp2"]." - ";
		echo $data["Pressure"]." - ";
		echo $data["WindSpeed"]." - ";
		echo $data["WindDirection1"]." - ";
		echo $data["WindChill"]." - ";
		echo $data["TotalRainfall"]." - ";
		echo $data["IndoorTemp"]." - ";
		echo $data["OutdoorTemp"]." - ";
		echo $data["OutdoorDewPoint"]." - "; 
		echo $data["IndoorHumidity"]." - ";
		echo $data["OutdoorHumidity"]."<br>";
		*/	

		$sql.="('".$timestamp;
		$sql.= "','".$data["Pressure"];
		$sql.= "','".$data["WindSpeed"];
		$sql.= "','".$data["WindDirection1"];
		$sql.= "','".$data["WindChill"];
		$sql.= "','".$data["TotalRainfall"];
		$sql.= "','".$data["IndoorTemp"];
		$sql.= "','".$data["OutdoorTemp"];
		$sql.= "','".$data["OutdoorDewPoint"];
		$sql.= "','".$data["IndoorHumidity"];
		$sql.= "','".$data["OutdoorHumidity"];
		if ($i==$RECORDS_TO_UPDATE-1) $sql.= "')";
		else $sql.= "'),";
		}

		$query = mysql_query($sql) or die( "Erreur SQL : ".$sql." ".mysql_error() );
		echo 'Database successfully updated.<br>';

	}
	fclose($handle);
}
else echo "Unable to open history.dat<br>";

?>

[TiToine]

The connect.php missing in .zip. It contains :

Code : Tout sélectionner

<? 
	$server="localhost";
	$user="root";
	$pass="";
	$db="dbname";
	mysql_connect($server, $user, $pass) or die('SQL connexion error.');
	mysql_select_db($db) or die('Error SQL');
?>

Antoine.

[James007]

Merci beaucoup! I will give this a go.

If you're interested, you can see my current details at http://james.cridland.net/weather/ - I'll publish the quick and dirty parser for current.lst shortly.

[James007]

There are a few errors in the above; I've debugged this and enclose it here.

This is actually all I really wanted; now it's in a database I can play with this in a different way: so many thanks for your help.

Code : Tout sélectionner

<? 
// ----------------------------------------------------------------------
// - UPDATE data DATABASE
// ----------------------------------------------------------------------

// Error reporting
// Comment the next lines once you're happy it all works
ini_set('display_errors', 1);
ini_set('error_reporting', E_ALL);

echo '## UPDATE data DATABASE ##<br>';

$DATA_LENGTH=56;    // Size is 56 bytes
$DELPHI1970=25569;    // need a conversion from delphi to unix timestamp

// Get file size
$HISTORY_DAT_SIZE=filesize($HISTORY_DAT);


// Get the numbers of new records
$handle = fopen($HISTORY_DAT,'r') or exit("Unable to open the history file.");

    $RECORDS=$HISTORY_DAT_SIZE/$DATA_LENGTH;

    $buffer=fread($handle, 8); // timestamp is double so read 8 bytes
    $date=unpack("d",$buffer); // unpack timestamp as double
    $FIRST_HISTORYDAT_RECORD=(($date[1]-$DELPHI1970)*86400); // delphi/unix conversion

    fseek($handle,-$DATA_LENGTH,SEEK_END);
    $buffer=fread($handle, 8);
    $date=unpack("d",$buffer);
    $LAST_HISTORYDAT_RECORD= ($date[1]-$DELPHI1970)*86400;

    echo 'Records in history.dat : ',$RECORDS,'<br>';
    echo 'First record in history.dat : ',date("d/m/y H:i",$FIRST_HISTORYDAT_RECORD),'<br>';
    echo 'Last record in history.dat : ',date("d/m/y H:i",$LAST_HISTORYDAT_RECORD),'<br>';

    $sql="SELECT MAX(`timestamp`) FROM data";
    $query=mysql_query($sql);
    $list=mysql_fetch_array($query);
    
    if (isset($list[0]))
    {
        $LAST_UPDATE=$list[0];
        echo 'Last update : ',date("d/m/y H:i", $LAST_UPDATE),'<br>';

        if($HISTORY_DAT_SIZE>0)
        {
            // iter from the end of file
            $RECORDS_TO_UPDATE=-1;
            fseek($handle,-$DATA_LENGTH,SEEK_END);
            do
            {
                $buffer=fread($handle, 8);
                $date=unpack("d",$buffer);
                $date=($date[1]-$DELPHI1970)*86400;
                $RECORDS_TO_UPDATE++;

                if ((ftell($handle)-$DATA_LENGTH-8)<0) break;
                fseek($handle,-$DATA_LENGTH-8,SEEK_CUR);
            }
            while ($date>$LAST_UPDATE);
            fseek($handle,$DATA_LENGTH-8,SEEK_CUR);
        }
        else $RECORDS_TO_UPDATE=0;
    }
    else 
    {
        $RECORDS_TO_UPDATE=$RECORDS;
        fseek($handle,0);
    }

    echo 'Records to update : ',$RECORDS_TO_UPDATE,'<br>';

    if ($RECORDS_TO_UPDATE!=0)
    {
        $sql = "INSERT INTO data (Timestamp,abs_pressure,wind_speed,wind_direction,wind_chill,total_rainfall,indoor_temp,outdoor_temp,outdoor_dew_point,indoor_humidity,outdoor_humidity)";
        $sql.= "VALUES ";
    
    for($i=0;$i<$RECORDS_TO_UPDATE;$i++)
    {

        $unpackstr="dTimestamp/fAbsPressure/fPressure/fWindSpeed/S2WindDirection/fWindGust/fTotalRainfall";
        $unpackstr.="/fNewRainfall/fIndoorTemp/fOutdoorTemp";
        $unpackstr.="/fIndoorHumidity/fOutdoorHumidity/VUnknown";

        $buffer=fread($handle,$DATA_LENGTH);
        $data=unpack($unpackstr,$buffer);
        $timestamp=($data["Timestamp"]-$DELPHI1970)*86400;
    
        //Dewpoint calculation
        $es=6.11*pow(10,7.5*$data["OutdoorTemp"]/($data["OutdoorTemp"]+237.7));
        $data["OutdoorDewPoint"]=( 237.7*log($es*$data["OutdoorHumidity"]/611,10) )/( 7.5-log($es*$data["OutdoorHumidity"]/611,10) );

        //Windchild calculation


        if (($data["OutdoorTemp"]<5)&&($data["WindSpeed"]>5))
        {    
            $data["WindChill"]=13.12+0.6215*$data["OutdoorTemp"]-11.37*pow($data["WindSpeed"],0.16)+0.3965*$data["OutdoorTemp"]*pow($data["WindSpeed"],0.16);
        }
        else $data["WindChill"]=$data["OutdoorTemp"];

        $data["Pressure"]=round($data["Pressure"],1);
        $data["WindSpeed"]=round($data["WindSpeed"]*3.6,1);
        $data["WindChill"]=round($data["WindChill"],1);
        $data["TotalRainfall"]=round($data["TotalRainfall"],1);
        $data["IndoorTemp"]=round($data["IndoorTemp"],1);
        $data["OutdoorTemp"]=round($data["OutdoorTemp"],1);
        $data["OutdoorDewPoint"]=round($data["OutdoorDewPoint"],1); 
        $data["IndoorHumidity"]=round($data["IndoorHumidity"],1);
        $data["OutdoorHumidity"]=round($data["OutdoorHumidity"],1);
    
        /*
        $count=$i+1;
        echo "Record $count at ".date("d/m/y H:i",$timestamp)." ";
        echo $data["Unknown"]." - ";
        echo $data["Timestamp1"]." - ";
        echo $data["Timestamp2"]." - ";
        echo $data["Pressure"]." - ";
        echo $data["WindSpeed"]." - ";
        echo $data["WindDirection1"]." - ";
        echo $data["WindChill"]." - ";
        echo $data["TotalRainfall"]." - ";
        echo $data["IndoorTemp"]." - ";
        echo $data["OutdoorTemp"]." - ";
        echo $data["OutdoorDewPoint"]." - "; 
        echo $data["IndoorHumidity"]." - ";
        echo $data["OutdoorHumidity"]."<br>";
        */    

        $sql.="('".$timestamp;
        $sql.= "','".$data["Pressure"];
        $sql.= "','".$data["WindSpeed"];
        $sql.= "','".$data["WindDirection1"];
        $sql.= "','".$data["WindChill"];
        $sql.= "','".$data["TotalRainfall"];
        $sql.= "','".$data["IndoorTemp"];
        $sql.= "','".$data["OutdoorTemp"];
        $sql.= "','".$data["OutdoorDewPoint"];
        $sql.= "','".$data["IndoorHumidity"];
        $sql.= "','".$data["OutdoorHumidity"];
        if ($i==$RECORDS_TO_UPDATE-1) $sql.= "')";
        else $sql.= "'),";
        }

        $query = mysql_query($sql) or die( "Erreur SQL : ".$sql." ".mysql_error() );
        echo 'Database successfully updated.<br>';

    }
    fclose($handle);
    

?>

[James007]

PS:

I've <a href='http://james.cridland.net/blog/2006/05/and-now-for-weather.html'>blogged about your help</a>, and finished, I think, my <a href='http://james.cridland.net/weather/'>weather page</a>. It's only supposed to be a bit of fun, but it appears to work well.

[James007]

PS:

I've blogged about your help, and finished, I think, my weather page. It's only supposed to be a bit of fun, but it appears to work well. Many thanks for your help.

[TiToine]

It works great !
Sparklines seems very interesting.
London temperature are higher than in Normandy, France...bouuhhh

[fikkaud]

Hi,

I've almost got this graphweather-php for 3600 working.

If the database is BLANK it updates without any problem (only thing I spot is that timestamps reported is one hour ahead) Graphing also works (Very Nice:)
This is what is reported with a blank database:

## UPDATE data DATABASE ##
Records in history.dat : 12103
First record in history.dat : 25/12/06 22:00
Last record in history.dat : 05/02/07 23:05
Records to update : 12103
Database successfully updated.

## UPDATE data_1h DATABASE ##
Empty database, first timestamp will be at 25/12/06 22:30
Database updated.

## UPDATE data_12h DATABASE ##
Empty database, first timestamp will be at 26/12/06 04:00
Database updated.

## UPDATE data_24h DATABASE ##
Empty database, first timestamp will be at 26/12/06 10:00
Database updated.


Script executed in 25.1478 seconds.


However when I try to do an UPDATE to this dabase, It crashes.
Can somebody assist me. I can't crack this one myself.

## UPDATE data DATABASE ##
Records in history.dat : 12105
First record in history.dat : 25/12/06 22:00
Last record in history.dat : 05/02/07 23:15
Last update : 05/02/07 23:05
Records to update : 3
Erreur SQL : INSERT INTO data (Timestamp,abs_pressure,wind_speed,wind_direction,wind_chill,total_rainfall,indoor_temp,outdoor_temp,outdoor_dew_point,indoor_humidity,outdoor_humidity)VALUES ('-2209161600','7.1','3589.9','3277','-21.2','0','264.7','0','-40.8','23.8','2.8'),('-2209161600','7.1','3590.3','4915','-21.2','0','264.7','0','-40.8','23.8','2.8'),('-2209161600','7.1','3589.9','3277','-21.2','0','264.7','0','-40.8','23.9','2.8') Out of range value adjusted for column 'timestamp' at row 1

[TiToine]

Hi,

The timestamp seems to be wrong (negative), and other values too.
Are you using the James007 script ?

[fikkaud]

Hi,

The timestamp seems to be wrong (negative), and other values too.
Are you using the James007 script ?

Yes i'm using the mysqlweather.php script from James007 without any modifications.

Something happens where it's referencing last timestamp in mysql database, and is about to append to it.
Any suggestions??

regards
Audun
http://vindkast.com
http://vindkast.com/astro

[TiToine]

Do you know PHP langage ?
You can try to see what happens in these lines in debugging the script :

$buffer=fread($handle,$DATA_LENGTH);
$data=unpack($unpackstr,$buffer);
$timestamp=($data["Timestamp"]-$DELPHI1970)*86400;

[fikkaud]

Hi Antoine

I've done a workaround and ported my history database to MySql.
Your graphweather php works fine ) (except for the update of 'data' which fails on the timestamp issue previously mentioned.
However I just update the 'data' database with a regular perlscript to keep it always up to date.
Every 24 hours I'll be running a script to update the '_1h _12 _24' databases.

I do have one question for you Regarding your Graphweather 1.6.1 application :
I can't connect to MySql. It gives me an error: http://vindkast.com/grafikk/gw_error.gif

My setup on the plugins page look like this: http://vindkast.com/grafikk/sql_plugin.gif

INFO: Database name= 'meteo', Table = 'data'
How/Where do I specify what Table to read?
I've tried to type 'data' in source fields, and also browsed to the data.frm file. But none of them works.
Do you have any information on how to connect to MySql? Since the phpversion connects fine and displays nicely. There shouldn't be a problem with the database itself.


regards
Audun
http://vindkast.com

[TiToine]

Hi,

Your plugin setup seems to be correct.
You should set the "optional source file 1" with the name of the table "data" (optional source file 1 because you are using the optional plugin 1)

Did you install the driver "MySQL ODBC 3.51 Driver" or later ?

Don't forget to modify the stylesheets you use with correct probe name. For example : outdoor_temperature become outdoor_temperature_1.

Regards,
Antoine.

[fikkaud]

Ahhhhh It was the ODBC driver.... Nice catch

It works fantastic... Now I can make the final swap to MySQL only! Lovely

Thanks again.
Audun
http://vindkast.com