Need an example of creating a git server using go-git

[zh-jn]

trafficstars

Hi all, is there have an example of creating a git server using this repository? thanks!

[MatisseB]

Hello, also have a hard time to use this lib to create a git server. Trying to find the way with this sample using old repo.

If someone have any example, even of a simple http upload/receive-pack server, it would be much appreciated.

Thanks!

[saul-data]

I would also like to see an example of http server implementation.

[seankhliao]

For future reference, an demo server for just git-upload-pack with go-git/v5: https://github.com/seankhliao/gitreposerver

[royletron]

@seankhliao this has been super useful. Any tips on the order of operations for receive. I tried but am clearly missing some steps.

Happy to PR what I have to your repo

[nathanblair]

I have been able to get receive-pack and upload-pack to work over a very basic https://localhost server by using the following:

func receivePack(
	context context.Context,
	session transport.Session,
	body io.ReadCloser,
	writer http.ResponseWriter,
) (err error) {
	receivePackRequest := packp.NewReferenceUpdateRequest()
	receivePackRequest.Decode(body)

	receivePackSession, ok := session.(transport.ReceivePackSession)
	if !ok {
		err = fmt.Errorf("Could not create receive-pack session")
		return
	}

	reportStatus, err := receivePackSession.ReceivePack(context, receivePackRequest)
	if err != nil || reportStatus == nil {
		return
	}

	return reportStatus.Encode(writer)
}

and then calling it like:

err = receivePack(request.Context(), session, request.Body, writer)

where the request is coming in through an *http.Request and the writer is simply an http.ResponseWriter.

I use essentially the same flow for the upload-pack route but replacing the respective types and functions for upload operations.

[prologic]

I'm trying to do this over ssh and run into this error: create receive-pack response: reference delta not found

code sample:

func handleReceivePack(dir string, ch ssh.Channel) error {
	ctx := context.Background()

	ep, err := transport.NewEndpoint("/")
	if err != nil {
		return fmt.Errorf("create transport endpoint: %w", err)
	}
	bfs := osfs.New(dir)
	ld := server.NewFilesystemLoader(bfs)
	svr := server.NewServer(ld)
	sess, err := svr.NewReceivePackSession(ep, nil)
	if err != nil {
		return fmt.Errorf("create receive-pack session: %w", err)
	}

	ar, err := sess.AdvertisedReferencesContext(ctx)
	if err != nil {
		return fmt.Errorf("get advertised references: %w", err)
	}
	err = ar.Encode(ch)
	if err != nil {
		return fmt.Errorf("encode advertised references: %w", err)
	}

	rur := packp.NewReferenceUpdateRequest()
	err = rur.Decode(ch)
	if err != nil {
		return fmt.Errorf("decode reference-update request: %w", err)
	}

	res, err := sess.ReceivePack(ctx, rur)
	if err != nil {
		return fmt.Errorf("create receive-pack response: %w", err)
	}
	err = res.Encode(ch)
	if err != nil {
		return fmt.Errorf("encode receive-pack response: %w", err)
	}

	return nil
}

This is based on gitreposerver by @seankhliao -- Pretty sure I'm doing something wrong here 🤔

[andskur]

@prologic this issue mentioned here - https://github.com/go-git/go-git/issues/190

[andskur]

@prologic I have added no-thin capability to advertise reference and it helps:

		if err := ar.Capabilities.Add("no-thin"); err != nil {
			http.Error(rw, err.Error(), 500)
			log.Println(err)
			return
		}

full code of updated InfoRefs handler:

func httpInfoRefs(dir string) http.HandlerFunc {
	return func(rw http.ResponseWriter, r *http.Request) {
		log.Printf("httpInfoRefs %s %s", r.Method, r.URL)

		service := r.URL.Query().Get("service")
		if service != "git-upload-pack" && service != "git-receive-pack" {
			http.Error(rw, "only smart git", 403)
			return
		}

		rw.Header().Set("content-type", fmt.Sprintf("application/x-%s-advertisement", service))

		ep, err := transport.NewEndpoint("/")
		if err != nil {
			http.Error(rw, err.Error(), 500)
			log.Println(err)
			return
		}
		bfs := osfs.New(dir)
		ld := server.NewFilesystemLoader(bfs)
		svr := server.NewServer(ld)

		var sess transport.Session

		if service == "git-upload-pack" {
			sess, err = svr.NewUploadPackSession(ep, nil)
			if err != nil {
				http.Error(rw, err.Error(), 500)
				log.Println(err)
				return
			}
		} else {
			sess, err = svr.NewReceivePackSession(ep, nil)
			if err != nil {
				http.Error(rw, err.Error(), 500)
				log.Println(err)
				return
			}
		}

		ar, err := sess.AdvertisedReferencesContext(r.Context())
		if err != nil {
			http.Error(rw, err.Error(), 500)
			log.Println(err)
			return
		}
		ar.Prefix = [][]byte{
			[]byte(fmt.Sprintf("# service=%s", service)),
			pktline.Flush,
		}

		if err := ar.Capabilities.Add("no-thin"); err != nil {
			http.Error(rw, err.Error(), 500)
			log.Println(err)
			return
		}

		err = ar.Encode(rw)
		if err != nil {
			http.Error(rw, err.Error(), 500)
			log.Println(err)
			return
		}
	}
}

[prologic]

I assume I can do the same/similar with the SSH handler too? 🤔

[prologic]

So this helps with pushing over http, but over ssh I end up with an error I don't understand:

On the server side:

prologic@JamessMacStudio
Thu Dec 29 01:09:25
~/Projects/legit
 (main) 0
$ ./legit
2022/12/29 01:09:37 starting HTTP server on 127.0.0.1:5555
2022/12/29 01:09:37 starting SSH server on 127.0.0.1:2222
2022/12/29 01:09:59 args: #[git-upload-pack /foo]
2022/12/29 01:09:59 dir: /Users/prologic/Projects/foo/.git
2022/12/29 01:11:14 args: #[git-receive-pack /foo]
2022/12/29 01:11:14 dir: /Users/prologic/Projects/foo/.git
2022/12/29 01:11:14 encode receive-pack response: EOF

On the client side:

prologic@JamessMacStudio
Thu Dec 29 01:11:11
~/tmp/foo
 (main) 0
$ git push
Enumerating objects: 5, done.
Counting objects: 100% (5/5), done.
Delta compression using up to 10 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 1.13 KiB | 1.13 MiB/s, done.
Total 4 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: the remote end hung up unexpectedly

[prologic]

Full code here: https://git.mills.io/prologic/legit

[scabala]

@prologic

I've been hit by the same issue but I've been able to figure it out.

sess.ReceivePack calls Close on io.Reader it gets and any further attempts to write something fails. I have a workaround - implement real io.Reader (wrapping ssh.Channel) but fake io.Closer (i.e. returning only nil) and pass that to ReceivePackRequest.Decode.

And it works.

Hopefully someone finds it helpful.

[prologic]

@scabala Do you have code sample of your workaround? It would be good if this bug was fixed 👌

[scabala]

Hi, my code is in such a mess that I am not able to share it but here's a snippet that solved it for me.

type ReaderFakeCloser struct {
        r io.Reader
}

func (rfc ReaderFakeCloser) Read(data []byte) (int, error) {
        return rfc.r.Read(data)
}

func (rfc ReaderFakeCloser) Close() error {
        fmt.Println("Gotcha!")
        return nil
}

// .. somewhere later  in the code ...
// `s` is ssh.Channel instance

receivePackRequest := packp.NewReferenceUpdateRequest()
err = receivePackRequest.Decode(ReaderFakeCloser{r: s})

// ... and later as in other examples

In essence, there's assumption that io.Reader passed to ReceivePackRequest.Decode can be closed, if closable. And that's not always the case.

[prologic]

@scabala Thanks! I'll try this out 👌

[prologic]

I'm not sure that this solved one of my original problems though :/ But I'd need to spend a bit more time to go back and understand what'g going on here hmmm

prologic@JamessMacStudio
Wed Dec 13 01:54:30
~/Projects/gitxt
 (main) 130
$ make dev
Running in debug mode...
INFO[0000]/Users/prologic/Projects/gitxt/cmd/gitxt/main.go:28 main.main() starting HTTP server on 0.0.0.0:5555
ERRO[0009]/Users/prologic/Projects/gitxt/internal/git/handler.go:257 gitxt.net/gitxt/internal/routes.Handlers.UploadPackHandler.func3() error encoding upload-pack response           error="write tcp 127.0.0.1:5555->127.0.0.1:50116: write: broken pipe"
2023/12/13 01:55:11 http: superfluous response.WriteHeader call from gitxt.net/gitxt/internal/routes.Handlers.UploadPackHandler.func3 (handler.go:258)
^Cmake: *** [dev] Interrupt: 2

[siaikin]

thks @prologic @scabala 🎉

I was able to pull and push on ssh after combining your code and debugging all afternoon.

@prologic I was able to get it to work after making the following modifications to yours.

	ar, err := sess.AdvertisedReferencesContext(ctx)
	if err != nil {
		return fmt.Errorf("get advertised references: %w", err)
	}
+	if err := ar.Capabilities.Add("no-thin"); err != nil {
+		return fmt.Errorf("get advertised references: %w", err)
+	}
	err = ar.Encode(ch)
	if err != nil {
		return fmt.Errorf("encode advertised references: %w", err)
	}

	rur := packp.NewReferenceUpdateRequest()
+	err = rur.Decode(ReaderFakeCloser{r: ch}) // from @scabala's ReaderFakeCloser 
	if err != nil {
		return fmt.Errorf("decode reference-update request: %w", err)
	}

[scabala]

Glad it helped you @siaikin.

Did you manage to have success for two, consecutive git pushes? In my example, it is failing on parsing response from client.

[siaikin]

Glad it helped you @siaikin.

Did you manage to have success for two, consecutive git pushes? In my example, it is failing on parsing response from client.

Yes I can push multiple times on the normal call flow. clone repository > modify file > push > modify again > push ...

Here is my code for this, maybe it will help you.

[scabala]

@siaikin Ah, so you did not try git push && git push? I did, for sake of completeness and found out that there's a bug in go-git - reported it here.

[github-actions[bot]]

To help us keep things tidy and focus on the active tasks, we've introduced a stale bot to spot issues/PRs that haven't had any activity in a while.

This particular issue hasn't had any updates or activity in the past 90 days, so it's been labeled as 'stale'. If it remains inactive for the next 30 days, it'll be automatically closed.

We understand everyone's busy, but if this issue is still important to you, please feel free to add a comment or make an update to keep it active.

Thanks for your understanding and cooperation!