Provide some background on how datasets are updated and how it affects internal fields

[chrismetcalf]

Might be useful:

First, just so its super clear, the :created_at and :updated_at fields (and all the :* internal fields) are internal, Socrata maintained metadata fields that track the metadata associated with each record stored in our platform. They’re read-only values, so data publishers have no control over what shows up there. :created_at and :id are set when a particular record is first created, and the :updated_at field is updated if that particular record is updated. Records exist at the Socrata level, and are not necessarily tied to what you might think of as a ‘record’ in the dataset itself. So, a particular customer record could be deleted and re-added to update it, and it’d have a different internal :id.

There are basically three different ways a dataset can be updated in our platform:

A dataset can be updated by completely replacing all of its records from an updated CSV source, similar to doing a TRUNCATE followed by a bulk INSERT in SQL. In this case, every single customer record would have a new Socrata :id and new :created_at values The publisher can perform an “Upsert” (Insert + Update) operation using our API. In this case, they must have a primary key within their dataset, which we use to determine what rows to update. In this case, :created_at will show when the record was first created, and :updated_at will show when it was updated. A dataset can be updated using “Smart Update” and our DataSync tool. In this case, we examine a new CSV source and essentially run “diff” at the record level to determine what records are new in the data file and what needs to be removed from the dataset. Since this can be performed without a primary key, records that are “updated” will actually be deleted and added as a new record, so you’ll see new internal :ids and new :created_at and :updated_at values.

[Romerolweb]

Good afternoon @chrismetcalf, I hope you are doing welll, How can I know which records has been updated?, Are there some metadata related with?.