UNIQUE constraint failed: files.name

[EmptySoft]

during make create_dbs and on each make i got a lot of following messages [SQL: INSERT INTO files (id, name, type, serial_number) VALUES (?, ?, ?, ?)] [parameters: ('6078279536', '6078279536_ACTIVITY.fit', 'fit_activity', 3348242342)] (Background on this error at: http://sqlalche.me/e/13/gkpj) (Background on this error at: http://sqlalche.me/e/13/7s2a) Failed to write message <MessageType.device_info: 23> type DataMessage(<MessageType.device_info: 23>: [timestamp(datetime.datetime(2021, 1, 9, 12, 10, 24, tzinfo=datetime.timezone.utc) (979128624)), serial_number([invalid] (0)), cum_operating_time([invalid] (4294967295)), unknown_8([invalid] (4294967295)), ant_related([invalid] (4294967295)), unknown_16([invalid] (4294967295)), unknown_17([invalid] ([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])), sensor_id([invalid] (0)), unknown_31([invalid] (4294967295)), manufacturer(<Manufacturer.Garmin: 1> (1)), product(<GarminProduct.GPS_3107: 3107> (3107)), software_version('4.80' (480)), battery_voltage([invalid] v (65535)), unknown_13([invalid] (65535)), ant_device_number([invalid] (0)), device_index(2), device_type(<LocalDeviceType.gps: 0> (0)), hardware_version([invalid] (255)), unknown_9([invalid] (255)), battery_status([invalid] (255)), sensor_position([invalid] (255)), ant_transmission_type([invalid] (0)), ant_network([invalid] (255)), unknown_23([invalid] (255)), source_type(<SourceType.local: 5> (5)), unknown_29([invalid] ([255, 255, 255, 255, 255, 255])), unknown_30([invalid] (255))]): This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (sqlite3.IntegrityError) UNIQUE constraint failed: files.name [bugreport_1612938165.zip](https://github.com/tcgoetz/GarminDB/files/5956244/bugreport_1612938165.zip)

[tcgoetz]

UNIQUE constraint failed: files.name => '6078279536_ACTIVITY.fit'

stack is: File "/home/max/GarminDB/fit_file_processor.py", line 50, in __write_generic function(fit_file, message.fields) File "/home/max/GarminDB/fit_file_processor.py", line 232, in _write_user_profile_entry self._write_attributes(timestamp, message_fields, attribute_names) File "/home/max/GarminDB/fit_file_processor.py", line 208, in _write_attributes self._write_attribute(timestamp, message_fields, attribute_name) File "/home/max/GarminDB/fit_file_processor.py", line 204, in _write_attribute GarminDB.Attributes.s_set_newer(self.garmin_db_session, db_attribute_name, attribute, timestamp) File "/home/max/GarminDB/utilities/key_value.py", line 32, in s_set_newer item = cls.s_get(session, key) File "/home/max/GarminDB/utilities/db_object.py", line 247, in s_get instance = session.query(cls).get(instance_id)

but that's where it did the DB flush and recognized the failed unique constraint, not where the bad write is.

[EmptySoft]

Sorry, there are really a lot of messages, is this part better?

Failed to write message <MessageType.device_info: 23> type DataMessage(<MessageType.device_info: 23>: [timestamp(datetime.datetime(2021, 1, 9, 11, 23, 50, tzinfo=datetime.timezone.utc) (979125830)), serial_number(3348242342), cum_operating_time([invalid] (4294967295)), unknown_8([invalid] (4294967295)), ant_related([invalid] (4294967295)), unknown_16([invalid] (4294967295)), unknown_17([invalid] ([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])), sensor_id([invalid] (0)), unknown_31([invalid] (4294967295)), manufacturer(<Manufacturer.Garmin: 1> (1)), product(<GarminProduct.Fenix_6S_Pro: 3288> (3288)), software_version('13.10' (1310)), battery_voltage([invalid] v (65535)), unknown_13([invalid] (65535)), ant_device_number([invalid] (0)), device_index(0), device_type(<MainDeviceType.fitness_tracker: 0> (255)), hardware_version([invalid] (255)), unknown_9([invalid] (255)), battery_status([invalid] (255)), sensor_position([invalid] (255)), ant_transmission_type([invalid] (0)), ant_network([invalid] (255)), unknown_23([invalid] (255)), source_type(<SourceType.local: 5> (5)), unknown_29([invalid] ([255, 255, 255, 255, 255, 255])), unknown_30([invalid] (255))]): (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (sqlite3.IntegrityError) UNIQUE constraint failed: files.name [SQL: INSERT INTO files (id, name, type, serial_number) VALUES (?, ?, ?, ?)] [parameters: ('6078279536', '6078279536_ACTIVITY.fit', 'fit_activity', 3348242342)] (Background on this error at: http://sqlalche.me/e/13/gkpj) Failed to write message <MessageType.device_info: 23> type DataMessage(<MessageType.device_info: 23>: [timestamp(datetime.datetime(2021, 1, 9, 11, 23, 50, tzinfo=datetime.timezone.utc) (979125830)), serial_number([invalid] (0)), cum_operating_time([invalid] (4294967295)), unknown_8([invalid] (4294967295)), ant_related([invalid] (4294967295)), unknown_16([invalid] (4294967295)), unknown_17([invalid] ([0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])), sensor_id([invalid] (0)), unknown_31([invalid] (4294967295)), manufacturer(<Manufacturer.Garmin: 1> (1)), product(<GarminProduct.Fenix_6S_Pro: 3288> (3288)), software_version('13.10' (1310)), battery_voltage([invalid] v (65535)), unknown_13([invalid] (65535)), ant_device_number([invalid] (0)), device_index(1), device_type(<LocalDeviceType.barometer: 4> (4)), hardware_version([invalid] (255)), unknown_9([invalid] (255)), battery_status([invalid] (255)), sensor_position([invalid] (255)), ant_transmission_type([invalid] (0)), ant_network([invalid] (255)), unknown_23([invalid] (255)), source_type(<SourceType.local: 5> (5)), unknown_29([invalid] ([255, 255, 255, 255, 255, 255])), unknown_30([invalid] (255))]): This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (raised as a result of Query-invoked autoflush; consider using a session.no_autoflush block if this flush is occurring prematurely) (sqlite3.IntegrityError) UNIQUE constraint failed: files.name

[tcgoetz]

I saw those logs in the garmin.log in the bug report. It's not a matter of more logging, it's a matter of logging at the right time. The issue is that for efficiency, the DB code is flushing a whole file worth of data at once and only once that whole file's worth is flushed is the constraint violation seen. We want to see it right when it happens.

Please apply this patch: `diff --git a/fit_file_processor.py b/fit_file_processor.py index 21046dc..ce55342 100644 --- a/fit_file_processor.py +++ b/fit_file_processor.py @@ -81,9 +81,11 @@ class FitFileProcessor(object): priority_message_types = [Fit.MessageType.file_id, Fit.MessageType.device_info] for message_type in priority_message_types: self.__write_message_type(fit_file, message_type)

•        self.garmin_db_session.commit()
     for message_type in message_types:
         if message_type not in priority_message_types:
             self.__write_message_type(fit_file, message_type)
  

•            self.garmin_db_session.commit()
  

  def write_file(self, fit_file): with self.garmin_db.managed_session() as self.garmin_db_session:`

Then run "make deps" to make sure your python packages are up-to-date then rerun your create db and we will hopefully get a more usable error.

[EmptySoft]

Hi Tom.!

I am running on Linux and havent done before applieng a patch. Can you describe, how to do?

max@kalymnos:~/GarminDB> diff --git a/fit_file_processor.py b/fit_file_processor.py diff: Unbekannte Option »--git«

[tcgoetz]

Use the program "patch" For instructions "man patch".

[EmptySoft]

Thanks for the tip, but unfortunately I do not understand the documentation. I am not a programmer and have never done anything like this. I understand roughly that the command changes a file, but where does the file come from? From GIT, if so, the parameter --git does not work for me (because I apply it wrong). The documentation gives to git only that symbolic links are no longer supported.

[tcgoetz]

closing as old