MySQL get_arrow2 type convert error

[Arionbat]

What language are you using?

Rust

What version are you using?

1.67.1

What database are you using?

mysql

What dataframe are you using?

arrow

Can you describe your bug?

Use the get_arrow2 method to read the database, and encounter a type conversion error when performing type conversion. Mysql types {text, longtext} are all converted to LargeList, resulting in the inability to write data frames when using CsvWriter to write

What are the steps to reproduce the behavior?

If possible, please include a minimal simple example including:

Database setup if the error only happens on specific data or data type

Table schema and example data

Example query / code

    let mut source_conn = SourceConn::try_from("mysql://root:[email protected]:3306/xx")
        .expect("parse conn str failed");
    let queries = &[CXQuery::from("SELECT description from xxx")];
    let destination = get_arrow2(&source_conn, None, queries).expect("run failed");
    let mut data_frame = destination.polars().unwrap();
    println!("{:#?}", data_frame);
    let mut file =
        std::fs::File::create("/Users/xxxx/RustProjects/dbpipe/help_info.csv").unwrap();
    CsvWriter::new(&mut file)
        .with_delimiter(FILE_DELIMITER)
        .finish(&mut data_frame)
        .unwrap();

What is the error?

thread 'main' panicked at 'data type: LargeList(Field { name: "item", data_type: UInt8, is_nullable: true, metadata: {} }) not supported by csv writer', /Users/xxx/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow2-0.10.1/src/io/csv/write/serialize.rs:458:15
stack backtrace:
   0: rust_begin_unwind
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/panicking.rs:575:5
   1: core::panicking::panic_fmt
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/panicking.rs:64:14
   2: arrow2::io::csv::write::serialize::new_serializer
             at /Users/xxx/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow2-0.10.1/src/io/csv/write/serialize.rs:458:15
   3: arrow2::io::csv::write::new_serializers::{{closure}}
             at /Users/xxx/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow2-0.10.1/src/io/csv/write/mod.rs:22:23
   4: core::iter::adapters::map::map_try_fold::{{closure}}
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/iter/adapters/map.rs:91:28
   5: core::iter::traits::iterator::Iterator::try_fold
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/iter/traits/iterator.rs:2238:21
   6: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/iter/adapters/map.rs:117:9
   7: <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::try_fold
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/iter/adapters/mod.rs:195:9
   8: core::iter::traits::iterator::Iterator::try_for_each
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/iter/traits/iterator.rs:2299:9
   9: <core::iter::adapters::GenericShunt<I,R> as core::iter::traits::iterator::Iterator>::next
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/iter/adapters/mod.rs:178:9
  10: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/alloc/src/vec/spec_from_iter_nested.rs:26:32
  11: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/alloc/src/vec/spec_from_iter.rs:33:9
  12: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/alloc/src/vec/mod.rs:2748:9
  13: core::iter::traits::iterator::Iterator::collect
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/iter/traits/iterator.rs:1836:9
  14: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter::{{closure}}
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/result.rs:2075:49
  15: core::iter::adapters::try_process
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/iter/adapters/mod.rs:164:17
  16: <core::result::Result<V,E> as core::iter::traits::collect::FromIterator<core::result::Result<A,E>>>::from_iter
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/result.rs:2075:9
  17: core::iter::traits::iterator::Iterator::collect
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/iter/traits/iterator.rs:1836:9
  18: arrow2::io::csv::write::new_serializers
             at /Users/xxx/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow2-0.10.1/src/io/csv/write/mod.rs:20:5
  19: arrow2::io::csv::write::write_chunk
             at /Users/xxx/.cargo/registry/src/github.com-1ecc6299db9ec823/arrow2-0.10.1/src/io/csv/write/mod.rs:67:27
  20: <polars_io::csv::CsvWriter<W> as polars_io::SerWriter<W>>::finish
             at /Users/xxx/.cargo/registry/src/github.com-1ecc6299db9ec823/polars-io-0.20.0/src/csv.rs:97:13
  21: dbpipe::main
             at ./src/main.rs:42:5
  22: core::ops::function::FnOnce::call_once
             at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/ops/function.rs:507:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.