[Bug]: InternalError on iOS when Tasks is installed

[juanbretti]

Describe the bug

I get this error:

And the backup is not completed. To solve the issue, I have to not install Tasks, and clone again the repository.

Relevant errors (if available)

InternalError: An internal error caused this command to fail. Please file a bug report at 
https://github.com/isomorphic-git/isomorphic-git/issues 
with this error message: SHA check failed! Expected 27eb43e79ec4b1236d2ba1b27e738f279420bd93, 
computed b77dOb17f624d487b84cc67d4510794aeb349819

Steps to reproduce

The steps I am doing:

• Created a new vault.
• Clone a repository from Github.
• The repository has .obsidian.
• Set depth to 1.

No issues so far. Then...

• Edit a note.
• Run Obsidian Git: Create backup
• Backup is complete

No issues so far. Then...

• Install the Tasks plugin (https://publish.obsidian.md/tasks/)
• Run Obsidian Git: Create backup
• I get the mentioned error
• Backup is not done

Expected Behavior

No response

Addition context

Obsidian: 1.4.4 (98) << From TestFligth Also tested with the stable release from the AppStore.

iPhone 12 iOS: 16.4.1 Calendar: 1.5.10 Dataview: 0.5.55 Obsidian Git: 2.19.1

No other plugin installed on my phone.

Operating system

iOS

Installation Method

Other

Plugin version

2.19.1

[juandastic]

I am having the exact same issue

[GJRobert]

I'm having this issue on iOS, but I don't have the Tasks plugin installed, FYI.

[juanbretti]

I'm having this issue on iOS, but I don't have the Tasks plugin installed, FYI.

@GJRobert, could you list the plugins you have installed on your iOS?

[GJRobert]

I'm having this issue on iOS, but I don't have the Tasks plugin installed, FYI.

@GJRobert, could you list the plugins you have installed on your iOS?

Hi thanks. Here they are:

1. Front Matter Title
2. Obsidian Git

Very few because this is just a vault in its early stage for testing Git syncing between 3 systems.

[GJRobert]

I don't know the cause. All my commits on iPad cannot be pushed any more, this error message starts today (5/12 UTC+8) on my iPad. It was doing okay yesterday.

[anpin]

I'm having exact same issue since yesterday on my iOS device. While trying to pinpoint the cause I was switching branches and were able to push and pull most of the commits, but then issue came back. The issue also crashes obsidian UI and becomes totally black and unresponsive until app restart, I can only prevent it by shutting down network with airplane mode.

[juandastic]

I couldn't solve it. I just copy all the files from my iPad to google drive and commit the changes from my PC, then I delete the vault and create a new one and clone the repo again. I tried using Working Copy as an alternative but the feature to auto pull on startup and auto push every x minutes are time saving

[GJRobert]

After copying the vault to iCloud and got it to my MacBook, on MacBook the error message would read:

corrupt loose object 'dee91ecf0955b6a59ad486510a8fd4400dbbda9c'
fatal: unable to read dee91ecf0955b6a59ad486510a8fd4400dbbda9c

and I searched for "corrupt loose object" and found this bug #423

I did upgrade my iPad to iOS 16 recently, but this issue is occurring on my iPad 1 week after the upgrade. Not sure if iOS 16 is the cause.

And now I'm trying to look for how to fix "loose object" of a Git repo to see if I can push them normally.

[GJRobert]

Although I have researched and tried several ways to try to fix the corrupted vault repo (even with the one Linus Torvalds documented), I still can't correct it. So finally I re-cloned the repo and replaced the .git/ in my vault, and sync the vault back to iPad, re-set the auth information, and pushed with this rebuilt vault repo. Old unpushed commits are lost and combined as a new commit, but at least the vault on iOS is back working again.

I'll keep testing to see if iOS cause any problem. And perhaps everybody using Obsidian-git on iOS should watch not to modify files during pushing, or take care not having any disk issue to try to prevent this issue from happening again.

[dbischof90]

I see that @GJRobert is reporting here as well :)

@tamiroh there is an issue open at https://github.com/isomorphic-git/isomorphic-git/issues/1760 that tracks the git side of things. Is it possible that files are hashed (not pushed, but hashed) while they are being modified? Could that lead to a broken state? If yes, then you could try to lock the file for the time (e.g., copy the currently active file to a temporary file/buffer, finish the git task and then replace the committed file with the buffer (which should be commited file plus the delta of typed stuff). Something there seems to be corrupting the commit.

I had the same problems but only another LaTeX-helper installed, not the tasks add-on.

[tamiroh]

Hmm...

@Vinzent03 How about the above issue?

[anpin]

After copying the vault to iCloud

@GJRobert have you been able to copy the .git folder? It is not showing up for me in the files on iOS, so I'm only able to copy latest version of notes files, without the commit history

[GJRobert]

@anpin iOS will surely copy the whole vault including .git for you, and you can check it on your Mac(Book). At least on my side.

[dbischof90]

After copying the vault to iCloud

@GJRobert have you been able to copy the .git folder? It is not showing up for me in the files on iOS, so I'm only able to copy latest version of notes files, without the commit history

I can access it via shell: https://forum.obsidian.md/t/mobile-ios-app-to-work-with-hidden-folder/25741/2

[anpin]

thanks @dbischof90, I'll try that. Got this issue again after recovering it and using for like 5 days :(

[dead-end]

The error message

SHA check failed! Expected
27eb43e79ec4b1236d2ba1b27e738f279420bd93, computed 
b77dOb17f624d487b84cc67d4510794aeb349819

is a cryptic way to say that the .git/index file is broken. The file consists of 3 parts:

1. fixed size header
2. list of staging informations for each file
3. hash which is computed over 1. and 2.

The error message says that the hash over the first two parts is not correct.

Isomorphic-git has the class GitIndexManager.js which maintances the .git/index file. The class ensures that the file has a proper format, the hash is always computed when the list in 2. changed and has a lock to ensure that concurrent writes that could corrupt the file can never occur. In isomorphic-git the .git/index is never directy accessed, only through the GitIndexManager class.

In the obsidian-git repository there is a file: myAdapter.ts which contains the following code that overwrites the .git/index:

    async saveAndClear(): Promise<void> {
        if (this.index !== undefined) {
            await this.adapter.writeBinary(
                this.plugin.gitManager.getVaultPath(this.gitDir + "/index"),
                this.index,
                {
                    ctime: this.indexctime,
                    mtime: this.indexmtime,
                }
            );
        }
        this.clearIndex();
    }

Why do you do this?

The issue in isomorohic-git: https://github.com/isomorphic-git/isomorphic-git/issues/1760

[devzero]

I have this issue as well, but can still replicate it without the Tasks plugins. I sync from multiple desktops and an iOS phone and starting in may, I can no longer commit changes from the iOS phone. This is a huge issue for me.

[juanbretti]

I have this issue as well, but can still replicate it without the Tasks plugins. I sync from multiple desktops and an iOS phone and starting in may, I can no longer commit changes from the iOS phone. This is a huge issue for me.

My personal solution, limiting the number of plugins on iOS. Alternatively, maybe you could .gitignore all the plugins on iOS.

[devzero]

Even with no plugins other than git installed, with my repo, this happens

[juanbretti]

Even with no plugins other than git installed, with my repo, this happens

I am now at the same stage. I am not able to sync anymore. Any ideas?

[Vinzent03]

@dead-end The reason for me to cache the .git/index file is a HUGE performance gain. Without that caching layer, it would reread the index file for every minor action, which is very slow. Maybe there is some issue resulting in a broken index file. But I can't reproduce your issue, so I don't know what exactly to fix. I'm very sure it has nothing to do with any specific plugin installed, but rather the files count. Though I think the tasks plugin is huge and should therefore not be committed.

[juanbretti]

[...] Though I think the tasks plugin is huge and should therefore not be committed.

For me, it works with version Obsidian Git 2.19.1 and Dataview and Calendar plugin installed. But if I try to update to Obsidian Git 2.20.4, it breaks. So, to make it work again, I had to downgraded to 2.19.1. Could it be something related the version?

I have 122 .md and 1 .jpg files. A total of 1.70 MiB.

Do you have any ideas to try? Thanks!

[Vinzent03]

From a quick look at the CHANGELOG, I can't find any change that could affect this, so I doubt the plugin version is responsible. I guess it's a more race condition or similar and thus hard to reproduce. Though once the index is broken, it's not that easy to recover.

[juanbretti]

Is there any trick to re-create the index on the mobile (iOS)? Or I must restart the git repository from GitHub?

[dead-end]

@Vinzent03 As I said in the note above, the GitIndexManager.js has a lock, that ensures that concurrent writing to the index results in a valid index file. You delegate the writing to the saveAndClear function, which pushes the write outside the lock context and with this concurrent writes are possible. As far as I understand the implementation you pull before you commit. Both can update the index. If you look at the first image in this thread you see

1. commit
2. pull
3. error

which has the wrong order. I think this is a race problem, but I have no evidence jet.

With your implementation you only save the reading of the index file, but the file has to be parsed each time. A better way to improve performance is to use the isomorphic git cache:

https://isomorphic-git.org/docs/en/cache

Create a global cache object, pass this to the function calls and flush the cache from time to time. You get better performance because you cache pared objects and you relay on the official api.

[Vinzent03]

I'm aware of the cache object and pretty sure I've tested that as well, but will look into that again.

[Vinzent03]

I've just tried it and it still reads the index file for every file, so using a cache does not fix it. It's only useful if I call a isogit function many times, but methods like statusMatrix use their own cache if none is provided.

[dead-end]

@Vinzent03 I think the intended way to use the cache is to create one (global) cache object and to pass it to all function calls. If this does not work, create an issue at isomorphic git with some example code and we will fix it. This is better than implementing a cache for yourself.

[maakbaas]

I have the same with:

• a clean vault
• a succesfully cloned repo with a single md file
• no other plugins

Anything I can do, or is the plugin basically broken?

[juanbretti]

Anything I can do, or is the plugin basically broken?

For me, now, it is working on iOS with version 2.19.1. But I'm not sure if it is a stable condition.

If I try to update, it breaks.