玩蛇网提供最新Python编程技术信息以及Python资源下载!

大神帮忙看下20行的python代码,文件io和数据库插入太慢

新学python,在做一个把爬虫爬下来的股票数据从文件插入到mysql数据库,速度很慢,不知道瓶颈是文件处理还是sql插入问题,望大神不吝赐教python问题的解答,感激不尽!

# encoding: utf-8
#!/usr/bin/python

import MySQLdb
import os
db = MySQLdb.connect(host="localhost",user="root",passwd="4242",\
        db="STOCKS", charset="utf8", use_unicode=True)
cursor = db.cursor()
files = os.listdir("shanghaiHistory")
for fileName in files:
    print "start insert " + fileName
    isTheFirstLine = True
    for line in open("shanghaiHistory/" + fileName):
        if isTheFirstLine:
            isTheFirstLine = False
            continue
        lineList = line.split(',')
        lineList.insert(0,fileName[:6])
        sql = "insert into shanghaiHistory(Code,Date,Open,High,Low,Close,Volume,AdjClose)\
                values ('%s','%s','%s','%s','%s','%s','%s','%s')" % tuple(lineList)
        try:
            cursor.execute(sql)
            db.commit()
        except:
            db.rollback()
db.close()

文件格式是形如:

Date,Open,High,Low,Close,Volume,Adj Close
2012-12-21,0.9151,0.9151,0.9151,0.9151,000,0.9151
2012-12-20,0.896,0.898,0.896,0.898,22924300,0.898
2012-12-19,0.895,0.898,0.895,0.897,37858100,0.897
2012-12-18,0.896,0.896,0.895,0.896,43542500,0.896
2012-12-17,0.895,0.897,0.895,0.896,41877700,0.896
2012-12-14,0.895,0.897,0.894,0.897,35734700,0.897
2012-12-13,0.894,0.896,0.894,0.895,11613800,0.895
2012-12-12,0.89,0.895,0.89,0.894,26586500,0.894

两个多小时才插入45W条数据,怎么才能提速一点呢.

玩蛇网文章,转载请注明出处和文章网址:https://www.iplaypy.com/wenda/wd18323.html

相关文章 Recommend

玩蛇网Python互助QQ群,欢迎加入-->: 106381465 玩蛇网Python新手群
修订日期:2020年08月03日 - 14时27分26秒 发布自玩蛇网

您现在的位置: 玩蛇网首页 > Python问题解答 > 正文内容
我要分享到:

必知PYTHON教程 Must Know PYTHON Tutorials

必知PYTHON模块 Must Know PYTHON Modules