Sprite xml by php script

ไม่ได้ update เป็นเดือนเพราะว่าไปบวชมาน่ะครับและก็ไม่รู้จะเขียนอะไรดี วันนี้เลยไปเอา php script มาแจก
พอดีวันหนึ่งต้องเขียน script อ่าน xml ขนาดประมาณ 100 MB นั้งกลุ้มอยู่นาน เลยไป search จาก google
ไปเจอ PHP script สำหรับ sprite file XML จาก http://www.techtalkpoint.com/articles/how-to-handle-large-xml-files-in-php/
แต่พอลองเอามาใช้มัน error เลยแก้ดูพอไม่มี error แล้วลองรันดู result ยังใช้ไม่ได้อีกก็เลย modify ซะเลย
ลองเอาไปใช้กันดู

01
<?php
02
//initialize vars
03
$begin=time(); // script start time
04
$filenum = 1; // start chunk file number at 1
05
$recordnum = 0; // start at record 1
06
//file settings
07
$basefilename = "chunks"; // the base file name for the chunks
08
$xmlfile = "TURL0001.XML"; // the xml file name to be processed
09
$xmldatadelimiter = "ONIXmessage"; // core data delimiter
10
$xmlitemdelimiter = "product"; // record delimiter
11
$chunksize = "2000"; // number of records in each chunk file
12
$xmlheader ="<?xml version=\"1.0\" encoding=\"UTF-8\">\n";
13
$xmlheader.="<$xmldatadelimiter>\n"; // xmlchunk file header
14
//dirs and files
15
$dir = ""; // path to where splits will be stored
16
$exportfile = "$dir"."$basefilename-".str_pad($filenum, 4, "0",STR_PAD_LEFT).".xml";
17
echo "Processing (".$dir."/$xmlfile)\n";
18
 
19
$handle = @fopen($dir."$xmlfile","r");
20
if ($handle) {
21
	while ($buffer = fread($handle,4096)) {
22
		$recordnum += substr_count($buffer,"</$xmlitemdelimiter>"); //count amount of item in buffer, $recordnum: hold number of record in current chunk file
23
 
24
		if ($recordnum>$chunksize) {//when recordnum more then chunksize do end file 
25
			preg_match("(<\/$xmlitemdelimiter>)", $buffer, $hit, PREG_OFFSET_CAPTURE); // find end item tag in buffer
26
			if ($hit){//set file point back to end item tag and remove data after end tag form buffer
27
				$str = $hit[0][0];
28
				$pos = (int)$hit[0][1];
29
				$buff_length  = strlen($buffer);
30
				$curr_pointer = ftell($handle);
31
				$new_pointer  = $curr_pointer-$buff_length+$pos+strlen($str);
32
				fseek($handle, (int)$new_pointer);
33
				$buffer =  substr($buffer,0,$pos+strlen($str));
34
			}
35
			error_log("$buffer",3,$exportfile);//write the last piece of chunk file
36
			error_log("</$xmldatadelimiter>",3,$exportfile);//write close data tag
37
			$recordnum = 0; //reset counter
38
			$filenum++;
39
			$exportfile = "$dir"."$basefilename-".str_pad($filenum, 4, "0",STR_PAD_LEFT).".xml"; //create next chunk file
40
			echo"Segment $filenum. Record ".($chunksize*$filenum).".\n";
41
			error_log($xmlheader,3,$exportfile);
42
		}else{
43
			error_log("$buffer",3,$exportfile); //write buffer to chunk file
44
		}
45
	}
46
	fclose($handle);
47
}else{
48
	echo"Unable to open file! (".$dir.$xmlfile.")\n";
49
}
50
$procend = time();
51
echo "\n####\n";
52
echo "Split Complete (".floor((($procend-$begin)/60))." Minutes)\n";
53
$end = microtime(true);
54
$time = $end - $start;
55
print "parse time: $time seconds<br/>";
56
?>

but this code isn’t 100% perfect because it’s to slow if I modify more.

source code and example xml: source

Tags: ,

This entry was posted on Sunday, June 7th, 2009 at 4:10 am and is filed under PHP. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

Leave a Reply