VeighNa量化社区
你的开源社区量化交易平台
Member
avatar
加入于:
帖子: 8
声望: 0

载入文件: rb20190301.csv
Traceback (most recent call last):
File "D:\Program Files\load_csv.py", line 74, in <module>
run_load_csv()
File "D:\Program Files\load_csv.py", line 19, in run_load_csv
csv_load(file)
File "D:\Program Files\load_csv.py", line 68, in csv_load
database_manager.save_tick_data(ticks)
File "D:\Program Files\vnstudio\lib\site-packages\vnpy\database\mongodb\mongodb_database.py", line 219, in save_tick_data
DbTickData.objects(
File "D:\Program Files\vnstudio\lib\site-packages\mongoengine\queryset\manager.py", line 37, in get
queryset = queryset_class(owner, owner._get_collection())
File "D:\Program Files\vnstudio\lib\site-packages\mongoengine\document.py", line 214, in _get_collection
if cls._meta.get("auto_create_index", True) and db.client.is_primary:
File "D:\Program Files\vnstudio\lib\site-packages\pymongo\mongo_client.py", line 1031, in is_primary
return self._server_property('is_writable')
File "D:\Program Files\vnstudio\lib\site-packages\pymongo\mongo_client.py", line 856, in _server_property
writable_server_selector)
File "D:\Program Files\vnstudio\lib\site-packages\pymongo\topology.py", line 243, in select_server
address))
File "D:\Program Files\vnstudio\lib\site-packages\pymongo\topology.py", line 200, in select_servers
selector, server_timeout, address)
File "D:\Program Files\vnstudio\lib\site-packages\pymongo\topology.py", line 217, in _select_servers_loop
(self._error_message(selector), timeout, self.description))
pymongo.errors.ServerSelectionTimeoutError: 172.0.0.1:27017: timed out, Timeout: 30s, Topology Description: <TopologyDescription id: 607a9f7acaa2dde19fe3862f, topology_type: Single, servers: [<ServerDescription ('172.0.0.1', 27017) server_type: Unknown, rtt: None, error=NetworkTimeout('172.0.0.1:27017: timed out')>]>

Member
avatar
加入于:
帖子: 8
声望: 0

mongodb服务打开了,MongoDBCompass打开能连接看到数据。

Member
avatar
加入于:
帖子: 8
声望: 0
import os 
import csv
from datetime import datetime, time

from vnpy.trader.constant import Exchange
from vnpy.trader.database import database_manager
from vnpy.trader.object import TickData


def run_load_csv():
    """
    遍历同一文件夹内所有csv文件,并且载入到数据库中
    """
    for file in os.listdir("."): 
        if not file.endswith(".csv"): 
            continue

        print("载入文件:", file)
        csv_load(file)


def csv_load(file):
    """
    读取csv文件内容,并写入到数据库中    
    """
    with open(file, "r") as f:
        reader = csv.DictReader(f)

        ticks = []
        start = None
        count = 0

        for item in reader:

            # generate datetime
            date = item["交易日"]
            second = item["最后修改时间"]
            millisecond = item["最后修改毫秒"]

            standard_time = date + " " + second + "." + millisecond
            dt = datetime.strptime(standard_time, "%Y%m%d %H:%M:%S.%f")

            # filter
            if dt.time() > time(15, 1) and dt.time() < time(20, 59):
                continue

            tick = TickData(
                symbol="rb1905",
                datetime=dt,
                exchange=Exchange.SHFE,
                last_price=float(item["最新价"]),
                volume=float(item["数量"]),
                bid_price_1=float(item["申买价一"]),
                bid_volume_1=float(item["申买量一"]),
                ask_price_1=float(item["申卖价一"]),
                ask_volume_1=float(item["申卖量一"]), 
                gateway_name="DB",       
            )
            ticks.append(tick)
            #print(ticks)

            # do some statistics
            count += 1
            if not start:
                start = tick.datetime

        end = tick.datetime
        database_manager.save_tick_data(ticks)

        print("插入数据", start, "-", end, "总数量:", count)      


if __name__ == "__main__":
    run_load_csv()
Member
avatar
加入于:
帖子: 8
声望: 0

以下是rb20190301.csv文件内容,哪位大侠能试试吗?版本是最新的2.2.0
交易日,合约代码,交易所代码,合约在交易所的代码,最新价,上次结算价,昨收盘,昨持仓量,今开盘,最高价,最低价,数量,成交金额,持仓量,今收盘,本次结算价,涨停板价,跌停板价,昨虚实度,今虚实度,最后修改时间,最后修改毫秒,申买价一,申买量一,申卖价一,申卖量一,申买价二,申买量二,申卖价二,申卖量二,申买价三,申买量三,申卖价三,申卖量三,申买价四,申买量四,申卖价四,申卖量四,申买价五,申买量五,申卖价五,申卖量五,当日均价,业务日期
20190301,rb1905,SHFE, ,3756,3733,3750,2631922,3756,3756,3756,2634,98933040,2632938,0,0,3994,3471,0,0,20:59:00,500,3756,269,3757,630,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,37560,20190228
20190301,rb1905,SHFE, ,3760,3733,3750,2631922,3756,3760,3755,6726,252688500,2633532,0,0,3994,3471,0,0,21:00:00,500,3758,492,3760,321,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,37568.9,20190228
20190301,rb1905,SHFE, ,3760,3733,3750,2631922,3756,3762,3755,15370,577602680,2635138,0,0,3994,3471,0,0,21:00:01,0,3760,849,3762,27,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,37579.9,20190228
20190301,rb1905,SHFE, ,3761,3733,3750,2631922,3756,3765,3755,19102,718002700,2635632,0,0,3994,3471,0,0,21:00:01,500,3761,23,3762,9,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,37587.8,20190228
20190301,rb1905,SHFE, ,3764,3733,3750,2631922,3756,3766,3755,25084,943140360,2633440,0,0,3994,3471,0,0,21:00:02,0,3762,385,3764,32,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,37599.3,20190228
20190301,rb1905,SHFE, ,3763,3733,3750,2631922,3756,3768,3755,31246,1175098580,2633258,0,0,3994,3471,0,0,21:00:02,500,3762,245,3763,194,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,37608,20190228
20190301,rb1905,SHFE, ,3762,3733,3750,2631922,3756,3768,3755,32994,1240862660,2633246,0,0,3994,3471,0,0,21:00:03,0,3761,40,3762,459,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,37608.7,20190228

Member
avatar
加入于:
帖子: 8
声望: 0

我错了!
原来vt_setting.json里只需要这样写 "database.host": "localhost",而不是“127.0.0.1”!!!
我在这里卡了1个月了,虽然只是周末弄弄,但是这挫败感真的很大。
接下来,load历史数据,写策略,跑回测吧。

Administrator
avatar
加入于:
帖子: 4502
声望: 321

所以新手的话不要折腾数据库,直接用默认的SQLite就行

Member
avatar
加入于:
帖子: 8
声望: 0

谢谢,因为之前用的就是mongodb,这次就直接用了。其实也不难,只是有些细节没太在意。这里也接受了个教训,下次一定好好看message,不会再出这种低级错误。

© 2015-2022 上海韦纳软件科技有限公司
备案服务号:沪ICP备18006526号

沪公网安备 31011502017034号

【用户协议】
【隐私政策】
【免责条款】