對PHP導(dǎo)出海量數(shù)據(jù)進(jìn)行優(yōu)化的方法

對PHP導(dǎo)出海量數(shù)據(jù)進(jìn)行優(yōu)化的方法？這個問題可能是我們?nèi)粘W(xué)習(xí)或工作經(jīng)常見到的。希望通過這個問題能讓你收獲頗深。下面是小編給大家?guī)淼膮⒖純?nèi)容，讓我們一起來看看吧！

導(dǎo)出數(shù)據(jù)量很大的情況下，生成excel的內(nèi)存需求非常龐大，服務(wù)器吃不消，這個時候考慮生成csv來解決問題，cvs讀寫性能比excel高。
測試表student 數(shù)據(jù)(大家可以腳本插入300多萬測數(shù)據(jù)。這里只給個簡單的示例了)

SET NAMES utf8mb4;
SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
-- Table structure for student
-- ----------------------------
DROP TABLE IF EXISTS `student`;
CREATE TABLE `student`  (
  `ID` int(11) NOT NULL AUTO_INCREMENT,
  `StuNo` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
  `StuName` varchar(10) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
  `StuAge` int(11) NULL DEFAULT NULL,
  PRIMARY KEY (`ID`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 12 CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Compact;

-- ----------------------------
-- Records of student
-- ----------------------------
INSERT INTO `student` VALUES (1, 'A001', '小明', 22);
INSERT INTO `student` VALUES (2, 'A005', '小李', 23);
INSERT INTO `student` VALUES (3, 'A007', '小紅', 24);
INSERT INTO `student` VALUES (4, 'A003', '小明', 22);
INSERT INTO `student` VALUES (5, 'A002', '小李', 23);
INSERT INTO `student` VALUES (6, 'A004', '小紅', 24);
INSERT INTO `student` VALUES (7, 'A006', '小王', 25);
INSERT INTO `student` VALUES (8, 'A008', '喬峰', 27);
INSERT INTO `student` VALUES (9, 'A009', '歐陽克', 22);
INSERT INTO `student` VALUES (10, 'A010', '老頑童', 34);
INSERT INTO `student` VALUES (11, 'A011', '黃老邪', 33);

SET FOREIGN_KEY_CHECKS = 1;

導(dǎo)出腳本export.php

<?php
set_time_limit(0);
ini_set('memory_limit', '128M');

$fileName = date('YmdHis', time());
header('Content-Encoding: UTF-8');
header("Content-type:application/vnd.ms-excel;charset=UTF-8");
header('Content-Disposition: attachment;filename="' . $fileName . '.csv"');
//注意，數(shù)據(jù)量在大的情況下。比如導(dǎo)出幾十萬到幾百萬，會出現(xiàn)504 Gateway Time-out,請修改php.ini的max_execution_time參數(shù)
//打開php標(biāo)準(zhǔn)輸出流以寫入追加的方式打開
$fp = fopen('php://output', 'a');
//連接數(shù)據(jù)庫
$dbhost = '127.0.0.1';
$dbuser = 'root';
$dbpwd = 'root';
$con = mysqli_connect($dbhost, $dbuser, $dbpwd);
if (mysqli_connect_errno())
    die('connect error');

$database = 'test';//選擇數(shù)據(jù)庫
mysqli_select_db($con, $database);
mysqli_query($con, "set names UTF8");//如果需要請設(shè)置編碼

//用fputcsv從數(shù)據(jù)庫中導(dǎo)出1百萬的數(shù)據(jù),比如我們每次取1萬條數(shù)據(jù)，分100步來執(zhí)行
//一次性讀取1萬條數(shù)據(jù)，也可以把$nums調(diào)小，$step相應(yīng)增大。
$step = 100;
$nums = 10000;
$where = "where 1=1"; //篩選條件,可自行添加
//設(shè)置標(biāo)題
$title = array('id', '編號', '姓名', '年齡'); //注意這里是小寫id,否則ID命名打開會提示Excel 已經(jīng)檢測到"xxx.xsl"是SYLK文件,但是不能將其加載: CSV 文或者XLS文件的前兩個字符是大寫字母"I"，"D"時，會發(fā)生此問題。
foreach ($title as $key => $item)
    $title[$key] = iconv("UTF-8", "GB2312//IGNORE", $item);
fputcsv($fp, $title);
for ($s = 1; $s <= $step; $s++) {
    $start = ($s - 1) * $nums;
    $result = mysqli_query($con, "SELECT ID,StuNo,StuName,StuAge FROM `student` " . $where . " ORDER BY `ID` LIMIT {$start},{$nums}");
    if ($result) {
        while ($row = mysqli_fetch_assoc($result)) {
            foreach ($row as $key => $item)
                $row[$key] = iconv("UTF-8", "GBK", $item); //這里必須轉(zhuǎn)碼，不然會亂碼
            fputcsv($fp, $row);
        }
        mysqli_free_result($result); //釋放結(jié)果集資源
        ob_flush();  //每1萬條數(shù)據(jù)就刷新緩沖區(qū)
        flush();
    }
}
mysqli_close($con);//斷開連接

導(dǎo)出效果：

感謝各位的閱讀！看完上述內(nèi)容，你們對對PHP導(dǎo)出海量數(shù)據(jù)進(jìn)行優(yōu)化的方法大概了解了嗎？希望文章內(nèi)容對大家有所幫助。如果想了解更多相關(guān)文章內(nèi)容，歡迎關(guān)注億速云行業(yè)資訊頻道。