torchchat
torchchat copied to clipboard
Published
20 hours ago •
pytorch
pytorch
Update packaging in AOTI path
Added a aoti_package path, dependent on https://github.com/pytorch/pytorch/pull/129895. Follow up will be to delete the --output-dso-path.
To export, use the --output-aoti-package-path to specify a file with a .pt2 extension. This will generate an artifact containing all the AOTI generated files
python3 torchchat.py export stories15M --output-aoti-package-path exportedModels/stories15M_artifacts_cpu.pt2 --device cpu
If we look into the contents of the package, we can see the following:
unzip -l exportedModels/stories15M_artifacts_cpu.pt2
Archive: exportedModels/stories15M_artifacts_cpu.pt2
Length Date Time Name
--------- ---------- ----- ----
1 09-13-2024 08:36 version
3 09-13-2024 08:36 archive_format
299452 09-13-2024 08:36 data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys.cpp
26 09-13-2024 08:35 data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys_metadata.json
644608 09-13-2024 08:36 data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys.o
1727 09-13-2024 08:36 data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys_compile_flags.json
1738 09-13-2024 08:36 data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys_linker_flags.json
173405576 09-13-2024 08:36 data/aotinductor/model/ca4wxt6cqzwws5meb44ouvurtpu5elwsqmsxayqx4cgmhvogmhys.so
172941232 09-13-2024 08:36 data/aotinductor/model/cghfy3rmc5pj6omsgyzll5b7btki66najqkyd3544pmitg26ihvj.o
--------- -------
347294363 9 files
To run with Python:
python3 torchchat.py generate stories15M --aoti-package-path exportedModels/stories15M_artifacts_cpu.pt2 --prompt "Hello my name is"
To run with the C++ runner, first build the runner binary:
torchchat/utils/scripts/build_native.sh aoti
To run:
cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z `python3 torchchat.py where stories15M `/tokenizer.model -l 2 -i "Once upon a time"
AOTI changes in PyTorch also now allow users to not need to specify --device when generating, as we will save this information during export time as metadata and use it during runtime to determine what device to use. A followup can be to extend the metadata to save model information so that users do not need to specify tokenizer version to use during runtime.
:link: Helpful Links
:test_tube: See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/torchchat/896
- :page_facing_up: Preview Python docs built from this PR
Note: Links to docs will display an error until the docs builds have been completed.
:white_check_mark: No Failures
As of commit 32c5807edaadd90e50b7b637cd689534f35ba31d with merge base f20f5e792d1114c9ee501f92b6fed3f2e840c99e (
This comment was automatically generated by Dr. CI and updates every 15 minutes.
![pytorch-bot[bot] avatar](https://avatars.githubusercontent.com/in/40112?v=4 "Published by a pub.dev verified publisher"){kind=small}
Weird that the pin bump in https://github.com/pytorch/torchchat/pull/1144 doesn't have the same errors as we see here....
Side question: Is it possible for us to push -l 2 into the dso as metadata as well or does that require looping through pytorch/pytorch? It's an arg when running the binary, but we know AOT what the arg should be
This command fails out of the box. I had to create the directory exportedModels for this to run
the command should create the directories in the specified path if they don't exist already.
python3 torchchat.py export stories15M --output-aoti-package-path exportedModels/stories15M_artifacts_cpu.pt2 --device cpu
command to build the runner fails
torchchat/utils/scripts/build_native.sh aoti
Building aoti native runner...
Defaulting TORCHCHAT_ROOT to /home/warden/source/torchchat/torchchat/utils/scripts/../../.. since it is unset.
~/source/torchchat ~/source/torchchat
Synchronizing submodule url for 'tokenizer/third-party/abseil-cpp'
Synchronizing submodule url for 'tokenizer/third-party/re2'
Synchronizing submodule url for 'tokenizer/third-party/sentencepiece'
~/source/torchchat
-- VERSION: 0.2.1
-- Not Found TCMalloc: TCMALLOC_LIB-NOTFOUND
-- Using ET BUILD DIR: --[et-build]--
-- TORCHCHAT_ROOT="/home/warden/source/torchchat"
-- Looking for excutorch in /home/warden/source/torchchat/et-build/install
-- Could NOT find executorch (missing: executorch_DIR)
-- Caffe2: CUDA detected: 12.0
-- Caffe2: CUDA nvcc is: /usr/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr
-- Caffe2: Header version is: 12.0
-- Could NOT find nvtx3 (missing: nvtx3_dir)
-- USE_CUDNN is set to 0. Compiling without cuDNN support
-- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
-- USE_CUDSS is set to 0. Compiling without cuDSS support
-- USE_CUFILE is set to 0. Compiling without cuFile support
-- Autodetected CUDA architecture(s): 8.9 8.6
-- Added CUDA NVCC flags for: -gencode;arch=compute_89,code=sm_89;-gencode;arch=compute_86,code=sm_86
-- Configuring done (0.3s)
-- Generating done (0.1s)
-- Build files have been written to: /home/warden/source/torchchat/cmake-out
[1/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_strings_internal.a
[2/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_cordz_functions.a
[3/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/win32_waiter.cc.o
[4/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_strings.a
[5/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/debugging/libabsl_symbolize.a
[6/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_str_format_internal.a
[7/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/crc/libabsl_crc32c.a
[8/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/crc/libabsl_crc_cord_state.a
[9/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/strings/libabsl_cord_internal.a
[10/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/types/CMakeFiles/bad_optional_access.dir/bad_optional_access.cc.o
[11/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_posix.cc.o
[12/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/types/CMakeFiles/bad_variant_access.dir/bad_variant_access.cc.o
[13/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/types/libabsl_bad_optional_access.a
[14/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/flags/libabsl_flags_marshalling.a
[15/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/types/libabsl_bad_variant_access.a
[16/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/flags/libabsl_flags_commandlineflag.a
[17/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/hash/libabsl_hash.a
[18/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/flags/libabsl_flags_private_handle_accessor.a
[19/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/bitmap256.cc.o
[20/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/civil_time.dir/internal/cctz/src/civil_time_detail.cc.o
[21/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_fixed.cc.o
[22/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/zone_info_source.cc.o
[23/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/time/libabsl_civil_time.a
[24/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/perl_groups.cc.o
[25/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_libc.cc.o
[26/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_lookup.cc.o
[27/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/graphcycles_internal.dir/internal/graphcycles.cc.o
[28/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_if.cc.o
[29/127] Linking CXX static library tokenizer/third-party/abseil-cpp/absl/synchronization/libabsl_graphcycles_internal.a
[30/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_format.cc.o
[31/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/sem_waiter.cc.o
[32/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/pthread_waiter.cc.o
[33/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/kernel_timeout_internal.dir/internal/kernel_timeout.cc.o
[34/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/blocking_counter.cc.o
[35/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/clock.cc.o
[36/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/format.cc.o
[37/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/barrier.cc.o
[38/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/per_thread_sem.cc.o
[39/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/unicode_casefold.cc.o
[40/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/futex_waiter.cc.o
[41/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/notification.cc.o
[42/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/waiter_base.cc.o
[43/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/util/rune.cc.o
[44/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/create_thread_identity.cc.o
[45/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/unicode_groups.cc.o
[46/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/internal/stdcpp_waiter.cc.o
[47/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_impl.cc.o
[48/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/bitstate.cc.o
[49/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/io_win32.cc.o
[50/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/time.cc.o
[51/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/civil_time.cc.o
[52/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/util/strutil.cc.o
[53/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/mimics_pcre.cc.o
[54/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time.dir/duration.cc.o
[55/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/synchronization/CMakeFiles/synchronization.dir/mutex.cc.o
[56/127] Building CXX object tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o
FAILED: tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o
/usr/bin/c++ -I/home/warden/source/torchchat/tokenizer -I/home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src -I/home/warden/source/torchchat/tokenizer/third-party/re2 -I/home/warden/source/torchchat/tokenizer/third-party/abseil-cpp -D_GLIBCXX_USE_CXX11_ABI=0 -MD -MT tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o -MF tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o.d -o tokenizer/CMakeFiles/tokenizer.dir/tiktoken.cpp.o -c /home/warden/source/torchchat/tokenizer/tiktoken.cpp
In file included from /home/warden/source/torchchat/tokenizer/tiktoken.cpp:18:
/home/warden/source/torchchat/tokenizer/base64.h:37:11: error: ‘uint32_t’ does not name a type
37 | constexpr uint32_t DECODE_TABLE[] = {
| ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:29:1: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
28 | #include <string>
+++ |+#include <cstdint>
29 | #include <string_view>
/home/warden/source/torchchat/tokenizer/base64.h:57:13: error: variable or field ‘validate’ declared void
57 | inline void validate(uint32_t v) {
| ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:57:22: error: ‘uint32_t’ was not declared in this scope
57 | inline void validate(uint32_t v) {
| ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:57:22: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h: In function ‘void base64::detail::decode(const std::string_view&, std::string&)’:
/home/warden/source/torchchat/tokenizer/base64.h:70:3: error: ‘uint32_t’ was not declared in this scope
70 | uint32_t val = 0;
| ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:70:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:72:3: error: ‘uint8_t’ was not declared in this scope
72 | uint8_t c = input[0];
| ^~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:72:3: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:73:12: error: ‘DECODE_TABLE’ was not declared in this scope
73 | auto v = DECODE_TABLE[c];
| ^~~~~~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:73:25: error: ‘c’ was not declared in this scope
73 | auto v = DECODE_TABLE[c];
| ^
/home/warden/source/torchchat/tokenizer/base64.h:74:3: error: ‘validate’ was not declared in this scope
74 | validate(v);
| ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:75:3: error: ‘val’ was not declared in this scope
75 | val = v;
| ^~~
/home/warden/source/torchchat/tokenizer/base64.h: In function ‘void base64::detail::decode_1_padding(const std::string_view&, std::string&)’:
/home/warden/source/torchchat/tokenizer/base64.h:105:3: error: ‘uint32_t’ was not declared in this scope
105 | uint32_t val = 0;
| ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:105:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:107:3: error: ‘uint8_t’ was not declared in this scope
107 | uint8_t c = input[0];
| ^~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:107:3: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:108:12: error: ‘DECODE_TABLE’ was not declared in this scope
108 | auto v = DECODE_TABLE[c];
| ^~~~~~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:108:25: error: ‘c’ was not declared in this scope
108 | auto v = DECODE_TABLE[c];
| ^
/home/warden/source/torchchat/tokenizer/base64.h:109:3: error: ‘validate’ was not declared in this scope
109 | validate(v);
| ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:110:3: error: ‘val’ was not declared in this scope
110 | val = v;
| ^~~
/home/warden/source/torchchat/tokenizer/base64.h: In function ‘void base64::detail::decode_2_padding(const std::string_view&, std::string&)’:
/home/warden/source/torchchat/tokenizer/base64.h:131:3: error: ‘uint32_t’ was not declared in this scope
131 | uint32_t val = 0;
| ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:131:3: note: ‘uint32_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:133:3: error: ‘uint8_t’ was not declared in this scope
133 | uint8_t c = input[0];
| ^~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:133:3: note: ‘uint8_t’ is defined in header ‘<cstdint>’; did you forget to ‘#include <cstdint>’?
/home/warden/source/torchchat/tokenizer/base64.h:134:12: error: ‘DECODE_TABLE’ was not declared in this scope
134 | auto v = DECODE_TABLE[c];
| ^~~~~~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:134:25: error: ‘c’ was not declared in this scope
134 | auto v = DECODE_TABLE[c];
| ^
/home/warden/source/torchchat/tokenizer/base64.h:135:3: error: ‘validate’ was not declared in this scope
135 | validate(v);
| ^~~~~~~~
/home/warden/source/torchchat/tokenizer/base64.h:136:3: error: ‘val’ was not declared in this scope
136 | val = v;
| ^~~
[57/127] Building CXX object tokenizer/third-party/abseil-cpp/absl/time/CMakeFiles/time_zone.dir/internal/cctz/src/time_zone_info.cc.o
[58/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/parse.cc.o
[59/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/bytestream.cc.o
[60/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/simplify.cc.o
[61/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/statusor.cc.o
[62/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_enum_util.cc.o
[63/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/common.cc.o
[64/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/prefilter.cc.o
[65/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/onepass.cc.o
[66/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/status.cc.o
[67/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/tostring.cc.o
[68/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/set.cc.o
[69/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/nfa.cc.o
[70/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/arena.cc.o
[71/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/int128.cc.o
[72/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/stringpiece.cc.o
[73/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/implicit_weak_message.cc.o
[74/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/filtered_re2.cc.o
[75/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/stringprintf.cc.o
[76/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/arenastring.cc.o
[77/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/coded_stream.cc.o
[78/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/re2.cc.o
[79/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/prog.cc.o
[80/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/compile.cc.o
[81/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/parse_context.cc.o
[82/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/message_lite.cc.o
In file included from /usr/include/string.h:548,
from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/stubs/port.h:39,
from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/stubs/macros.h:34,
from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/stubs/common.h:46,
from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/message_lite.h:45,
from /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/third_party/protobuf-lite/message_lite.cc:36:
In function ‘void* memcpy(void*, const void*, size_t)’,
inlined from ‘google::protobuf::uint8* google::protobuf::io::EpsCopyOutputStream::WriteRaw(const void*, int, google::protobuf::uint8*)’ at /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/io/coded_stream.h:699:16,
inlined from ‘virtual google::protobuf::uint8* google::protobuf::internal::ImplicitWeakMessage::_InternalSerialize(google::protobuf::uint8*, google::protobuf::io::EpsCopyOutputStream*) const’ at /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/src/../third_party/protobuf-lite/google/protobuf/implicit_weak_message.h:85:28,
inlined from ‘bool google::protobuf::MessageLite::SerializePartialToZeroCopyStream(google::protobuf::io::ZeroCopyOutputStream*) const’ at /home/warden/source/torchchat/tokenizer/third-party/sentencepiece/third_party/protobuf-lite/message_lite.cc:419:30:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:33: warning: ‘void* __builtin___memcpy_chk(void*, const void*, long unsigned int, long unsigned int)’ specified size between 18446744071562067968 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Wstringop-overflow=]
29 | return __builtin___memcpy_chk (__dest, __src, __len,
| ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~
30 | __glibc_objsize0 (__dest));
| ~~~~~~~~~~~~~~~~~~~~~~~~~~
[83/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/prefilter_tree.cc.o
[84/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/dfa.cc.o
[85/127] Building CXX object tokenizer/third-party/re2/CMakeFiles/re2.dir/re2/regexp.cc.o
[86/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_message_util.cc.o
[87/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/generated_message_table_driven_lite.cc.o
[88/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/repeated_field.cc.o
[89/127] Building CXX object tokenizer/third-party/sentencepiece/src/CMakeFiles/sentencepiece-static.dir/__/third_party/protobuf-lite/extension_set.cc.o
ninja: build stopped: subcommand failed.
the command should create the directories in the specified path if they don't exist already.
That should be fixed with https://github.com/pytorch/pytorch/pull/138919
the command should create the directories in the specified path if they don't exist already.
That should be fixed with pytorch/pytorch#138919
Seems like we need to bump the pin to pytorch as well?
https://github.com/pytorch/torchchat/pull/1319/files
There's a bump here that's not actively being worked on, seems like it's not a free bump (+ the infra issue we're seeing here)
currently running into this issue when trying to run using the runner.
(.venv) warden@Vikander:~/source/torchchat$ cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z `python3 torchchat.py where stories15M `/tokenizer.model -l 2 -i "Once upon a time"
2024-11-01:14:45:53,463 INFO [sdpa_with_kv_cache.py:29] Loading custom ops library: /home/warden/source/torchchat/.venv/lib/python3.11/site-packages/executorch/extension/llm/custom_ops/libcustom_ops_aot_lib.so
terminate called after throwing an instance of 'std::runtime_error'
what(): Failed to initialize zip archive: file open failed
Aborted (core dumped)
currently running into this issue when trying to run using the runner.
(.venv) warden@Vikander:~/source/torchchat$ cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z `python3 torchchat.py where stories15M `/tokenizer.model -l 2 -i "Once upon a time" 2024-11-01:14:45:53,463 INFO [sdpa_with_kv_cache.py:29] Loading custom ops library: /home/warden/source/torchchat/.venv/lib/python3.11/site-packages/executorch/extension/llm/custom_ops/libcustom_ops_aot_lib.so terminate called after throwing an instance of 'std::runtime_error' what(): Failed to initialize zip archive: file open failed Aborted (core dumped)
path issue. Just need to update the instructions in the first comment so it's
cmake-out/aoti_run exportedModels/stories15M_artifacts_cpu.pt2 -z python3 torchchat.py where stories15M /tokenizer.model -l 2 -i "Once upon a time"
and not
cmake-out/aoti_run exportedModels/exportedModels/stories15M_artifacts_cpu.pt2 -z python3 torchchat.py where stories15M /tokenizer.model -l 2 -i "Once upon a time"