twitter-to-sqlite
twitter-to-sqlite copied to clipboard
Import archive error 'withheld_in_countries'
Importing the twitter archive I received this error:
$ twitter-to-sqlite import archive.db twitter-2021-12-31-<hash>.zip
birdwatch-note-rating: not yet implemented
birdwatch-note: not yet implemented
branch-links: not yet implemented
community-tweet: not yet implemented
contact: not yet implemented
device-token: not yet implemented
direct-message-mute: not yet implemented
mute: not yet implemented
periscope-account-information: not yet implemented
periscope-ban-information: not yet implemented
periscope-broadcast-metadata: not yet implemented
periscope-comments-made-by-user: not yet implemented
periscope-expired-broadcasts: not yet implemented
periscope-followers: not yet implemented
periscope-profile-description: not yet implemented
professional-data: not yet implemented
protected-history: not yet implemented
reply-prompt: not yet implemented
screen-name-change: not yet implemented
smartblock: not yet implemented
spaces-metadata: not yet implemented
sso: not yet implemented
Traceback (most recent call last):
File "/home/paulox/.virtualenvs/dogsheep/bin/twitter-to-sqlite", line 8, in <module>
sys.exit(cli())
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
return self.main(*args, **kwargs)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/twitter_to_sqlite/cli.py", line 759, in import_
archive.import_from_file(db, filename, content)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/twitter_to_sqlite/archive.py", line 246, in import_from_file
db[table_name].insert_all(rows, pk=pk, replace=True)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 2625, in insert_all
self.insert_chunk(
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 2406, in insert_chunk
result = self.db.execute(query, params)
File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 422, in execute
return self.conn.execute(sql, parameters)
sqlite3.OperationalError: table archive_tweet has no column named withheld_in_countries
I found only a single tweet with the key withheld_in_countries
in tweet.js
that seems the problems:
[
{
"tweet" : {
"retweeted" : false,
"source" : "<a href=\"http://twitter.com/download/android\" rel=\"nofollow\">Twitter for Android</a>",
"entities" : {
"hashtags" : [
{
"text" : "NowOnAndroid",
"indices" : [
"64",
"77"
]
}
],
"symbols" : [ ],
"user_mentions" : [
{
"name" : "Periscope",
"screen_name" : "PeriscopeCo",
"indices" : [
"3",
"15"
],
"id_str" : "1111111111",
"id" : "222222222"
}
],
"urls" : [
{
"url" : "https://t.co/xxxxxxxxx",
"expanded_url" : "https://vine.co/v/xxxxxxxxx",
"display_url" : "vine.co/v/xxxxxxxxxx",
"indices" : [
"78",
"101"
]
}
]
},
"display_text_range" : [
"0",
"101"
],
"favorite_count" : "0",
"id_str" : "1111111111111111111111",
"truncated" : false,
"retweet_count" : "0",
"withheld_in_countries" : [
"TR"
],
"id" : "000000000000000000",
"possibly_sensitive" : false,
"created_at" : "Fri Aug 14 06:04:03 +0000 2015",
"favorited" : false,
"full_text" : "RT @periscopeco: Travel the world. LIVE. The Global Map is here #NowOnAndroid https://t.co/NZXdsPWROk",
"lang" : "en"
}
}
]
I solved the error removing the key from the tweet.js
but I'm reporting this error to improve the project.