本文共 823 字,大约阅读时间需要 2 分钟。
我已经用
fgetcsv读取一个120MB的csv流式(这是正确的英语吗?)。它逐行读取,然后我将每行插入数据库。这样一来,在每次迭代中只有一行被保留在内存中。脚本仍然需要20分钟。跑步。也许我下次尝试Python …不要试图加载一个巨大的csv文件到一个数组,这真的会消耗大量的内存。
// WDI_GDF_Data.csv (120.4MB) are the World Bank collection of development indicators:
// http://data.worldbank.org/data-catalog/world-development-indicators
if(($handle = fopen('WDI_GDF_Data.csv', 'r')) !== false)
{
// get the first row, which contains the column-titles (if necessary)
$header = fgetcsv($handle);
// loop through the file line-by-line
while(($data = fgetcsv($handle)) !== false)
{
// resort/rewrite data and insert into DB here
// try to use conditions sparingly here, as those will cause slow-performance
// I don't know if this is really necessary, but it couldn't harm;
// see also: http://php.net/manual/en/features.gc.php
unset($data);
}
fclose($handle);
}
转载地址:https://blog.csdn.net/weixin_33132553/article/details/116107795 如侵犯您的版权,请留言回复原文章的地址,我们会给您删除此文章,给您带来不便请您谅解!