今天发现一个很奇怪的问题。

从 vntrader 中的CTA回测模块中，进行多进程优化没有问题。
但是自己从 examples 中的 jupyter notebook中进行 backtesting, 源代码是遗传算法，简单更改为 多进行优化的时候，就出现错误。 是发生再多进程 load_data 的时候，访问 sqlite3 的 fetch_one() 出错。

1。同样的代码从 UI 中运行就没有问题, 也是同时读取 sqlite3
2。自己写成单独的脚本运行，多进程读取的时候很快就出问题。

请教一下，大家有没有遇到过?

我最终的办法是给 load_data 加了个全局锁。
这样问题解决了，但是还是不知道原因。

我的电脑是 6 cores, 对于前6个任务，是没有多进程加速的。

我的电脑是 6 cores, 对于前6个任务，是没有多进程加速的 (这个是全局锁导致的)

请贴下完整的报错截图信息？

好的。 下面是我不加锁时的错误信息。
sqlite3 的数据库是没有问题的。加锁之后，就一直好用。

multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 3005, in execute_sql
cursor.execute(sql, params or ())
sqlite3.DatabaseError: database disk image is malformed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/multiprocessing/pool.py", line 121, in worker
result = (True, func(args, **kwds))
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/vnpy/app/cta_strategy/backtesting.py", line 1204, in optimize
engine.load_data()
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/vnpy/app/cta_strategy/backtesting.py", line 243, in load_data
end
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/vnpy/app/cta_strategy/backtesting.py", line 1252, in load_bar_data
symbol, exchange, interval, start, end
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/vnpy/trader/database/database_sql.py", line 352, in load_bar_data
data = [db_bar.to_bar() for db_bar in s]
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 6655, in iter
self.execute()
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 1812, in inner
return method(self, database, args, *kwargs)
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 1883, in execute
return self._execute(database)
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 2054, in _execute
cursor = database.execute(self)
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 3018, in execute
return self.execute_sql(sql, params, commit=commit)
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 3012, in execute_sql
self.commit()
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 2783, in exit
reraise(new_type, new_type(exc_args), traceback)
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 183, in reraise
raise value.with_traceback(tb)
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/peewee.py", line 3005, in execute_sql
cursor.execute(sql, params or ())
peewee.DatabaseError: database disk image is malformed
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/zlu/Work/vnpy/test/vnpy/tui_apps/cta_backtesting.py", line 59, in <module>
df = engine.run_optimization(setting)
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/vnpy/app/cta_strategy/backtesting.py", line 572, in run_optimization
result_values = [result.get() for result in results]
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/site-packages/vnpy/app/cta_strategy/backtesting.py", line 572, in <listcomp>
result_values = [result.get() for result in results]
File "/home/zlu/anaconda3/envs/py37/lib/python3.7/multiprocessing/pool.py", line 657, in get
raise self._value
peewee.DatabaseError: database disk image is malformed

https://github.com/passiomatic/coldsweat/issues/104

查了下网上的资料，这应该是个Linux系统上SQlite老版本的问题，建议升级下试试

我的问题是多进程优化似乎没有充分调动cpu， 而且非常慢

请问你的版本是2.4.0吗？
可以pip install --upgrade vnpy_ctabacktester试试看

xiaohe wrote:

请问你的版本是2.4.0吗？
可以pip install --upgrade vnpy_ctabacktester试试看
是2.4.0，现在成了是回测中途跑了一半会崩溃

请问这次优化cpu跑满了吗？
请问是回测就崩溃了，还是优化崩溃了。如果回测就崩溃了可以用run.py或者在cmd用python -m vnstation打开VN Trader然后复现你的操作看看底层是否有报错信息