Support git integration in the community edition

[synergiator]

Following the feature matrix, I have discovered that currently, there is no Git integration support in the community edition.

If I have read the matrix properly, it's not possible even in the Pro edition - only in the cloud version.

I've also found an old request which has been suggested for reopening as a new issue.

Done. :-)

[synergiator]

Clarification: does the "raw git access" work anyway with community edition or not? https://www.scivision.dev/overleaf-with-github/

[jdleesmiller]

does the "raw git access" work anyway with community edition or not?

Not at this time, unfortunately. It only works with the cloud edition.

[synergiator]

@jdleesmiller thx

[MengXiangxi]

does the "raw git access" work anyway with community edition or not?

Not at this time, unfortunately. It only works with the cloud edition.

Thanks for answering, and I really wonder if it is something like a commercial strategy to keep it an advanced feature, or there is a technical limitation from using git in self-hosted versions?

[jdleesmiller]

there is a technical limitation

There are indeed technical limitations. The short version is that https://github.com/overleaf/writelatex-git-bridge, while open source, relies on a different project history service than the one used in Overleaf v2 (namely the one from Overleaf v1, pre-integration with ShareLaTeX). We're running both history services in the SaaS edition with a connector between them, but doing that in CE would add a lot of complexity. We are working toward a better solution.

[Alex-Muirhead]

@jdleesmiller Would this better git-bridge solution help solve issue #365? And is there any way we can contribute to finding / implementing this solution? It seems a lot of my personal issues with Overleaf would be solved with more git capabilities.

[eeriksp]

Yes, Git integration would indeed be very helpful. To make the issue less complicated, would there be a way of setting up Git integration on the server side without the user interface? For instance so that Overleaf just places the files in some directory and then one can configure Git inside this directory.

[gunwald]

To make the issue less complicated, would there be a way of setting up Git integration on the server side without the user interface? Would be a big step because it would enable us to also work offline. I think a UI is would not be that important here.

[axkibe]

I hacked together a bridge to be used as workaround on CE/Pro Servers https://gitlab.com/axkibe/olgitbridge

It uses the web API*, thus it's way slower and also cruder than the v1 java bridge also it doesn't translate the git changes to real-time changes; it uploads any changed file. But it gets the job done as far as I can tell.

'* After studying the code a bit, that seemed to be the layer of least complexity to insert to, albeit handling the cookies and the one needed socket.io call to get the folder IDs proofed a bit more complicated than I expected.

[youkaichao]

2022 now, any new progress?

[goerz]

Any updates on this?

[ivanperez-keera]

Any updates on this? We have a local installation of overleaf. We can pay for the plugin as a premium feature, but we really would like to have this feature.

[TerenceLiu98]

I hacked together a bridge to be used as workaround on CE/Pro Servers https://gitlab.com/axkibe/olgitbridge

It uses the web API*, thus it's way slower and also cruder than the v1 java bridge also it doesn't translate the git changes to real-time changes; it uploads any changed file. But it gets the job done as far as I can tell.

'* After studying the code a bit, that seemed to be the layer of least complexity to insert to, albeit handling the cookies and the one needed socket.io call to get the folder IDs proofed a bit more complicated than I expected.

Hi, I hope everything goes well. I checked the bridge you wrote and I am wondering if is there any instruction on how to use the bridge? Seem I am a newbie on JS and I am not sure I am doing it the right way:

1. modify the olServer in config.js
2. build the docker image
3. run the image

After these three steps, I am confused about what is the next step. Could you provide Any hint :)

---

update:

I run the image however when I run the git clone http://localhost:5000/<project-id> project

it came out errors:

olgitbridge-gitbridge-1  | Node.js v18.14.2
olgitbridge-gitbridge-1  | [*] listening on port 5000
olgitbridge-gitbridge-1  | [1] getting request for <project-id>
olgitbridge-gitbridge-1  | [1] requesting authorization
olgitbridge-gitbridge-1  | [2] getting request for <project-id>
olgitbridge-gitbridge-1  | [2] requesting user semaphore
olgitbridge-gitbridge-1  | [2] auth email [email protected]
olgitbridge-gitbridge-1  | [2] overleaf login
olgitbridge-gitbridge-1  | /var/olgitbridge/src/server.js:126
olgitbridge-gitbridge-1  | 		if( e.response.status === 401 )
olgitbridge-gitbridge-1  | 		               ^
olgitbridge-gitbridge-1  |
olgitbridge-gitbridge-1  | TypeError: Cannot read properties of undefined (reading 'status')
olgitbridge-gitbridge-1  |     at handleAuth (/var/olgitbridge/src/server.js:126:18)
olgitbridge-gitbridge-1  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
olgitbridge-gitbridge-1  |     at async Server.serve (/var/olgitbridge/src/server.js:218:15)

[axkibe]

Since my university at large licensed the Overleaf cloud version a few months ago, I'm no longer running a CE server for my department, so I'm no longer mainting this, sorry.

As you can see in the code https://gitlab.com/axkibe/olgitbridge/-/blob/main/src/server.js#L126 You are in the error handling here (which seems to have a bug, assuming a response code while something else happened, I suggest logging the error with console.log(e);

Also I didn't use the docker image but ran it directly, the docker image was a PR I accepted.

[TerenceLiu98]

Thanks! Let me check whether I have missed anything necessary. I use postman to generate a post, however, the login is forbidden. I do not know whether it is the reason for the 401 error.

[Sciroccogti]

Thanks! Let me check whether I have missed anything necessary. I use postman to generate a post, however, the login is forbidden. I do not know whether it is the reason for the 401 error.

Hi @TerenceLiu98 , have you finally succeeded in using the git-bridge? I just don't understand what is the role of git-bridge. Is it hosting a git server by itself? Or just forwarding git commands to another git server?

[TerenceLiu98]

Thanks! Let me check whether I have missed anything necessary. I use postman to generate a post, however, the login is forbidden. I do not know whether it is the reason for the 401 error.

Hi @TerenceLiu98 , have you finally succeeded in using the git-bridge? I just don't understand what is the role of git-bridge. Is it hosting a git server by itself? Or just forwarding git commands to another git server?

I gave up :(

[zenone87]

This seems to work for the self-hosted CE Overleaf version: https://gitlab.inria.fr/sed-rennes/sharelatex/python-sharelatex

[Sciroccogti]

This seems to work for the self-hosted CE Overleaf version: https://gitlab.inria.fr/sed-rennes/sharelatex/python-sharelatex

Thanks a lot!

BTW, is this a cmdline tool that can sync code from sharelatex? So we can maintain a local repo, and use git slatex to sync with overleaf, then use original git to sync with a remote git, is that true?

[zenone87]

Unfortunately I do not know how it works with respect to the local and remote repos. However, it works with Overleaf CE 3.0.1. It seems a good start to build an external tool for using git with Overleaf CE.

[jidhub]

@axkibe :

It uses the web API*, thus it's way slower and also cruder than the v1 java bridge also it doesn't translate the git changes to real-time changes; it uploads any changed file. But it gets the job done as far as I can tell.

I think the git and github access of overleaf.com accounts have the same limitations; I don't know if this runs on overleaf.com's servers or on my PC.

There is a better way, because latexmkrc allows arbitrary perl programs, which can be used to setup a mongo secondary server, or at least apply these mongoDB backup scripts: https://github.com/overleaf/overleaf/issues/512#issuecomment-1099152438

[diviyank]

This seems to work for the self-hosted CE Overleaf version: https://gitlab.inria.fr/sed-rennes/sharelatex/python-sharelatex

It seems that the latest version of CE Overleaf had its upload file API changed: I'm getting an 422: Unprocessable entity for URL.

The logs indicate :

{"name":"web","hostname":"...","pid":140,"level":50,"projectId":"...","msg":"bad name when trying to upload file","time":"2024-04-01T22:10:42.608Z","v":0}

Has anyone encountered this issue?

[Musicminion]

Let me talk about my thoughts after several months of developing the Overleaf Community Edition, which is also my personal point of view.

It is almost impossible for the overleaf community version to support git-bridge. If possible, it should be 2 years or more later, and it is unlikely that the github integration will be open source. There was a feature called a visual editor a long time ago. It was probably a feature 4-5 years ago. It was only recently open sourced in Overleaf 4.0.

Some community patches, such as olgitbridge, are actually based on earlier versions. It is impossible for anyone to always be willing to maintain this, because Overleaf's API is constantly changing, and if you use the previous version, you will not be able to use the next version. And then, debug and fix......

Setting up an Overleaf community edition server requires 2 cores and 4GB of memory. The price of this server configuration is enough to purchase a one-year student edition subscription on the Overleaf official website.

To give another inappropriate example, suppose Overleaf makes all codes public. For an individual, the cost of building an Overleaf with exactly the same functions as the official website is more expensive than subscribing to membership.

So I chose to give up and Overleaf Common was the best choice.

[Musicminion]

Thanks! Let me check whether I have missed anything necessary. I use postman to generate a post, however, the login is forbidden. I do not know whether it is the reason for the 401 error.

Glad to see someone have already tried to use postman to debug overleaf! But they may got confused about Forbidden result, since overleaf have _csrf token check before web server can hanle request.

So, this tutorial is about How to use POSTMAN to debug overleaf

Use Postman to debug overleaf By Musicminion

Step one:

Login in your overleaf instance in your browser, and copy cookies[Using edge dev tool to get cookie]

s%3AONjupRamJeQtO-xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Then, set cookie in postman, this is my local dev.(I think there is no need to tell how to set cookie)

Step two

Add this code to your Postman Pre request Script., don't forget to modify 192.XXX to your overleaf instance.

pm.sendRequest('http://192.168.48.128/dev/csrf', function (err, res) {
    if (err) {
        console.log(err);
    } else {
        console.log(res.text());
        postman.setGlobalVariable("csrftoken", res.text());
    }
});

And then, time to start you dev, like this:

Step Three

Add X-Csrf-Token to header

Reminder

dev/csrf is a private API, so, You need to login before you can get this token.