玩蛇网提供最新Python编程技术信息以及Python资源下载!

pandas内存错误MemoryError如何解决

  File "D:\WinPython-32bit-2.7.9.3\python-2.7.9\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 680, in runfile
    execfile(filename, namespace)

  File "D:\WinPython-32bit-2.7.9.3\python-2.7.9\lib\site-packages\spyderlib\widgets\externalshell\sitecustomize.py", line 71, in execfile
    exec(compile(scripttext, filename, 'exec'), glob, loc)

  File "D:/pywork/Marking/DataProcessing.py", line 113, in <module>
    df = pd.DataFrame(data, columns=colNames)

  File "D:\WinPython-32bit-2.7.9.3\python-2.7.9\lib\site-packages\pandas\core\frame.py", line 258, in __init__
    dtype=dtype)

  File "D:\WinPython-32bit-2.7.9.3\python-2.7.9\lib\site-packages\pandas\core\frame.py", line 4627, in _arrays_to_mgr
    return create_block_manager_from_arrays(arrays, arr_names, axes)

  File "D:\WinPython-32bit-2.7.9.3\python-2.7.9\lib\site-packages\pandas\core\internals.py", line 3539, in create_block_manager_from_arrays
    blocks = form_blocks(arrays, names, axes)

  File "D:\WinPython-32bit-2.7.9.3\python-2.7.9\lib\site-packages\pandas\core\internals.py", line 3614, in form_blocks
    int_blocks = _multi_blockify(int_items)

  File "D:\WinPython-32bit-2.7.9.3\python-2.7.9\lib\site-packages\pandas\core\internals.py", line 3681, in _multi_blockify
    list(tup_block), dtype)

  File "D:\WinPython-32bit-2.7.9.3\python-2.7.9\lib\site-packages\pandas\core\internals.py", line 3725, in _stack_arrays
    stacked = np.empty(shape, dtype=dtype)

MemoryError

使用pandas.DataFrame读175MB数据(111W行 X 38列),程序执行过程中报如上错误。

是读入错误还是计算中错误?
如果是后者,可以试试读入后 del 一些无用的 col,再计算
百万行我试过,但这么多列没试过

玩蛇网文章,转载请注明出处和文章网址:https://www.iplaypy.com/wenda/wd18719.html

相关文章 Recommend

玩蛇网Python互助QQ群,欢迎加入-->: 106381465 玩蛇网Python新手群
修订日期:2017年05月18日 - 10时32分33秒 发布自玩蛇网

您现在的位置: 玩蛇网首页 > Python问题解答 > 正文内容
我要分享到:

必知PYTHON教程 Must Know PYTHON Tutorials

必知PYTHON模块 Must Know PYTHON Modules