influxdb3-python icon indicating copy to clipboard operation
influxdb3-python copied to clipboard

请问如何使用多线程入库?

Open allran opened this issue 3 months ago • 7 comments
trafficstars

    def bulk_save_item_data(obj: SyncInfluxDbObject):
        """批量同步数据"""
        try:
            conn = _create_connection()
            # 插入pd.Dataframe数据
            conn.write(record=obj.bulk_data_pd,
                       data_frame_measurement_name=obj.coll,
                       data_frame_tag_columns=[obj.code_field],
                       data_frame_timestamp_column='time')
            conn.close()
        except Exception as err:
            logging.error(f"Failed to insert to influxdb, ErrMessage:{err}")
            raise err

  with ThreadPoolExecutor(max_workers=10) as executor:
      futures = [executor.submit(bulk_save_item_data, ops) for ops in batch_group]
      for future in as_completed(futures):
          pbar_save_db.update(1)
          result = future.result()

如果是单线程不会出错,后面开的10线程,开始执行插入时没报错,执行一会儿后就会报如下错误:

ErrMessage:HTTPConnectionPool(host='localhost', port=8181): Read timed out. (read timeout=0)

请问如何使用多线程插入?跪求,谢谢

allran avatar Aug 08 '25 09:08 allran