Example demonstrating how to resume an upload

[markmcd]

File uploads are resumable, but we don't have any examples of how that works.

When we know the precise chunks already, it's pretty easy, but the example should demonstrate how to recover from an error - e.g. what info to save and how to resume from whatever is there, specifying the right offset.

[markmcd]

(idea suggested by @jochenkirstaetter)

[TYMichaelChen]

Is there a specific language you'd like the example in? I think we should just support this via the python SDK directly

[markmcd]

Just the standard stuff we use here - Python and maybe REST. We need the SDK support too but depending on the complexity we should either update the existing guides here to show resumption or add new ones.

[jochenkirstaetter]

Hi there, thanks @markmcd to bring this up, appreciated.

Any language would be great. I'm interested in any code so that I am able to adapt it into my Gemini SDK for .NET ;-)

And as we are talking about resume. How about parallel upload channel/streams? Is this something possible on the API side?

[TYMichaelChen]

Hi @jochenkirstaetter, I don't think we'll have examples or SDK support for .NET.

But to unblock you, I think you can rely on the googleapiclient library for .NET and create a DiscoveryService using our API documentation. Then upload files using ResumableUpload class (similar to the Drive example).

This is how we're doing it in JS right now until we have SDK Support:

   const GENAI_DISCOVERY_URL = `https://generativelanguage.googleapis.com/$discovery/rest?version=v1beta&key=${API_KEY}`;
   // Initialize API Client
    const genaiService = await google.discoverAPI({url: GENAI_DISCOVERY_URL});
    const auth = new google.auth.GoogleAuth().fromAPIKey(API_KEY);

    // Prepare file to upload to GenAI File API
    const media = {
        mimeType: mime.lookup(filePath),
        body: fs.createReadStream(filePath),
    };
    var body = {"file": {"displayName": fileDisplayName}};
    try {
        // Upload the file
        const createFileResponse = await genaiService.media.upload({
            media: media, auth: auth, requestBody:body});
        const file = createFileResponse.data.file;
        const fileUri = file.uri;
        

[jochenkirstaetter]

Hi @TYMichaelChen Thanks for the example code which I already assimilated into my SDK. The issue is about resuming an upload which I don't see in the Javascript code. And I couldn't find it in the source code of the Python SDK.

Maybe it could be of interest to you that I implemented a client for the Gemini API in NET supporting Google AI and Vertex AI in one package.

https://github.com/mscraftsman/generative-ai

Actually, the repository contains three projects

• a core package for Gemini and PaLM
• a package for ASP.NET Core, and
• a package using Google Cloud Client Library for NET and lots of tests to verify functionality.

This is based on the REST API and a little bit of network capture to correct my initial file upload from the (for me) insufficient sample.sh using multiple requests to the correct use of the multipart request as done in the Python (create_file method) and Javascript SDK. Something I exchanged with @markmcd recently.

I also had a look at the official dotnet-docs-samples repository provided by Google. Which is totally different to anything else you provide in the other SDKs. Which is actually the reason I implemented the client in NET. End of February there was nothing available for Gemini, only PaLM 2.

Anyway, I'll have a look at the Drive sources, maybe I'll be able to find something. 🤔

FYI, I have a signed CLA and I'm a GDE for Cloud. I'd be glad to assist with an official Gemini SDK for NET. 🙏

[jochenkirstaetter]

Hi @TYMichaelChen and @markmcd

I think that I found the right information here: https://cloud.google.com/storage/docs/performing-resumable-uploads

Those cURL examples are easy to understand. Using the suggested ResumableUpload class should allow me to give it a shot and see what happens. 🙏

[MarkDaoust]

https://github.com/google-gemini/generative-ai-python/pull/275

[jochenkirstaetter]

Hi @MarkDaoust

Thanks for following up. I noticed a while ago that the handling in .NET is based on using Streams instead of using chunks like in Python.

// other code before
var multipartContent = new MultipartContent("related");
multipartContent.Add(new StringContent(json, Encoding.UTF8, Constants.MediaType));
multipartContent.Add(new StreamContent(new FileStream(uri, FileMode.Open), (int)Constants.ChunkSize)
// other code after

Which already works nice and stable. I'm going to review the Python code you linked to see how it works and how I could possibly adapt it in .NET. Thank you very much. Cheers, JoKi