1. 本帖的目的
本帖旨在通过对CTP接口推送的tick行情数据进行过滤和分析，揭示利用未经有效性处理的tick驱动各种上层应用可能带来的错误，以及这些错误所带来的影响。
依据事实说话，讲道理，不是空谈！仔细阅读，耐心分析，你一定会用巨大收获！
2. 脏数据的获得方法
按照如何更有效地利用合约交易状态信息——拒绝CTP接口中的脏数据一文中的4.4节的方法对原始数据进行过滤，其中OmsEngine新增了函数process_origin_tick_event()，代码是这样的：
def process_origin_tick_event(self,event: Event):#-> None: # hxxjava debug
""" 原始tick数据处理 """
tick = event.data
status:StatusData = self.trade_status_manager.get_tick_status(tick)
# hxxjava debug
if not status:
print(f"{datetime.now()} {tick.vt_symbol} 还没有收到交易状态")
return
# 有效交易状态
if status.instrument_status in VALID_TRADE_STATUSES:
# 这里是所有有效数据的发源地
self.event_engine.put(Event(EVENT_TICK, tick))
self.event_engine.put(Event(EVENT_TICK + tick.vt_symbol, tick))
else:
print(f"{datetime.now()} 特别交易状态={status} {tick}")3. 脏数据展示
下面是实际运行python -m vnstation之后，对CTP接口推送的数据的过滤而得到的部分结果，暂时称之为脏数据。
之所以称之为脏数据，是因为它们和当前合约的交易状态不符。
其中包含了当时的本地实际、当前合约交易状态和tick数据库三部分。
因为打印出来的内容每行比较长，建议您把下面的内容复制粘贴到其他的文本编辑器（如vscode）中，方便对左右移动，也可以上下对齐。
2021-07-15 07:07:05.767769 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.NO_TRADING: '非交易'>, trading_segment_sn=15, enter_time='23:00:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 23, 0, 0, 500000), name='铁矿石2109', volume=121262, open_interest=435280.0, last_price=1206.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1215.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1206.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1206.5, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=51, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=33, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 10:14:59.837070 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.NO_TRADING: '非交易'>, trading_segment_sn=25, enter_time='10:15:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 10, 15), name='铁矿石2109', volume=225979, open_interest=448149.0, last_price=1230.5, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1233.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1230.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1231.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=33, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=68, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 10:14:59.994983 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.NO_TRADING: '非交易'>, trading_segment_sn=25, enter_time='10:15:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 10, 15, 0, 500000), name='铁矿石2109', volume=225984, open_interest=448151.0, last_price=1231.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1233.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1230.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1231.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=39, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=63, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 10:15:00.253865 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.NO_TRADING: '非交易'>, trading_segment_sn=25, enter_time='10:15:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 10, 15, 1), name='铁矿石2109', volume=225984, open_interest=448151.0, last_price=1231.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1233.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1230.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1231.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=7, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=64, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 11:29:59.852447 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.NO_TRADING: '非交易'>, trading_segment_sn=29, enter_time='11:30:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 11, 30), name='铁矿石2109', volume=283444, open_interest=465948.0, last_price=1230.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1229.5, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1230.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=23, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=29, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 11:29:59.923405 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.NO_TRADING: '非交易'>, trading_segment_sn=29, enter_time='11:30:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 11, 30, 0, 500000), name='铁矿石2109', volume=283453, open_interest=465947.0, last_price=1230.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1229.5, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1230.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=23, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=22, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 11:30:00.185258 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.NO_TRADING: '非交易'>, trading_segment_sn=29, enter_time='11:30:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 11, 30, 1), name='铁矿石2109', volume=283456, open_interest=465945.0, last_price=1230.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1229.5, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1230.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=21, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=30, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 11:30:02.015268 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.NO_TRADING: '非交易'>, trading_segment_sn=29, enter_time='11:30:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 11, 30, 1, 500000), name='铁矿石2109', volume=283457, open_interest=465944.0, last_price=1230.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1229.5, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1230.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=21, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=29, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 15:00:01.727185 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.CLOSE: '收盘'>, trading_segment_sn=99, enter_time='15:00:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 15, 0 ), name='铁矿石2109', volume=359871, open_interest=484883.0, last_price=1234.5, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1233.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1234.5, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=169, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=40, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 15:00:01.834127 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.CLOSE: '收盘'>, trading_segment_sn=99, enter_time='15:00:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 15, 0, 0, 500000), name='铁矿石2109', volume=359884, open_interest=484882.0, last_price=1234.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1233.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1234.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=167, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=13, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 15:00:01.927076 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.CLOSE: '收盘'>, trading_segment_sn=99, enter_time='15:00:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 15, 0, 1 ), name='铁矿石2109', volume=359885, open_interest=484883.0, last_price=1234.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1233.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1233.5, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=167, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=1, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 15:00:01.931069 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.CLOSE: '收盘'>, trading_segment_sn=99, enter_time='15:00:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 15, 0, 1, 500000), name='铁矿石2109', volume=359886, open_interest=484883.0, last_price=1234.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1233.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1234.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=172, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=12, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 15:01:04.571289 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.CLOSE: '收盘'>, trading_segment_sn=99, enter_time='15:00:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 15, 1, 5, 500000), name='铁矿石2109', volume=359886, open_interest=484883.0, last_price=1234.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1233.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1234.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=172, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=12, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)
2021-07-15 15:10:52.207334 特别交易状态=StatusData(gateway_name='CTP', symbol='i', exchange=<Exchange.DCE: 'DCE'>, settlement_group_id='00000001', instrument_status=<InstrumentStatus.CLOSE: '收盘'>, trading_segment_sn=99, enter_time='15:00:00', enter_reason=<StatusEnterReason.AUTOMATIC: '自动切换'>, exchange_inst_id='i') TickData(gateway_name='CTP', symbol='i2109', exchange=<Exchange.DCE: 'DCE'>, datetime=datetime.datetime(2021, 7, 15, 15, 10,53 ), name='铁矿石2109', volume=359886, open_interest=484883.0, last_price=1234.0, last_volume=0, limit_up=1335.5, limit_down=1093.5, open_price=1208.0, high_price=1240.0, low_price=1196.0, pre_close=1219.5, bid_price_1=1233.0, bid_price_2=0, bid_price_3=0, bid_price_4=0, bid_price_5=0, ask_price_1=1234.0, ask_price_2=0, ask_price_3=0, ask_price_4=0, ask_price_5=0, bid_volume_1=172, bid_volume_2=0, bid_volume_3=0, bid_volume_4=0, bid_volume_5=0, ask_volume_1=12, ask_volume_2=0, ask_volume_3=0, ask_volume_4=0, ask_volume_5=0)4. 脏数据分类
为了方便描述，我们把上面的内容进行编号，编号的顺序为打印信息的本地时间，
1. 2021-07-15 07:07:05.767769
2. 2021-07-15 10:14:59.837070
3. 2021-07-15 10:14:59.994983
4. 2021-07-15 10:15:00.253865
5. 2021-07-15 11:29:59.852447
6. 2021-07-15 11:29:59.923405
7. 2021-07-15 11:30:00.185258
8. 2021-07-15 11:30:02.015268
9. 2021-07-15 15:00:01.727185
10. 2021-07-15 15:00:01.834127
11. 2021-07-15 15:00:01.927076
12. 2021-07-15 15:00:01.931069
13. 2021-07-15 15:01:04.571289
14. 2021-07-15 15:10:52.207334下面是对这几条的分析：
再次连接CTP接口后，接口自动推送i2109合约昨天夜里23:00收盘的价格和盘口信息，
已经进入上午10:15休市时段，volume=225979, last_price=1230.5
- 已经进入上午10:15休市时段，volume=225984, last_price=1231.0 bid_volume_1=39 ask_volume_1=63
已经进入上午10:15休市时段，volume=225984, last_price=1231.0 bid_volume_1=7 ask_volume_1=64
已经进入上午11:30休市时段，volume=283444, open_interest=465948.0, last_price=1230.0,
- 已经进入上午11:30休市时段，volume=283453, open_interest=465947.0, last_price=1230.0,
- 已经进入上午11:30休市时段，volume=283456, open_interest=465945.0, last_price=1230.0,
已经进入上午11:30休市时段，volume=283457, open_interest=465944.0, last_price=1230.0,
已经进入下午15:00休市时段，volume=359871, open_interest=484883.0, last_price=1234.5,ask_price_1=1234.5,ask_volume_1=40,
10.已经进入下午15:00休市时段，volume=359884, open_interest=484882.0, last_price=1234.0,ask_price_1=1234.0,ask_volume_1=13,
11.已经进入下午15:00休市时段，volume=359885, open_interest=484883.0, last_price=1234.0,ask_price_1=1234.5,ask_volume_1=1,
12.已经进入下午15:00休市时段，open_interest=484883.0, last_price=1234.0
10.已经进入下午15:00休市时段，open_interest=484883.0, last_price=1234.0 —— 除了和时间戳与12不同之外，tick的其他字段完全相同
11.已经进入下午15:00休市时段，open_interest=484883.0, last_price=1234.0 —— 除了和时间戳与12不同之外，tick的其他字段完全相同
5. 如果不过滤这些tick数据，对各种应用的有何影响？
对于1中的tick，不过滤可以在刚启动的行情列表中显示，对启动应用后再连接接口的上层应用会产生多余的K线，过滤了更好
对于2，3，4中的tick，是进入休市后仍然有撮合， 不过滤可能造成上层应用会产生多余的K线（如5分钟K线），过滤了更好
对于5，6，7中的tick，是进入休市后仍然有撮合， 不过滤可能造成上层应用会产生多余的K线（如5分钟K线），过滤了更好
对于9，10，11中的tick，是进入收盘后仍然有撮合， 不过滤可能造成上层应用会产生多余的K线（如5分钟K线），过滤了影响跨收盘时刻K线的持仓兴趣、成交量等信息的统计
对于12，13，14中的tick，是进入收盘后已经没有撮合， 完全可以过滤掉
为什么说2-7中的tick过滤了更好？
因为这些tick是在非交易时间段延续的撮合，其价格、持仓兴趣、成交量可以忽略，它们采用下一个交易状态下tick中的价格、持仓兴趣、成交量更好！
