Issues with ServingUrl

[Veejay]

Hi we're using the google.golang.org/appengine/image package to construct a serving URL for an object stored in Google Cloud Storage and it's shown to be unreliable under certain circumstances.

The code the ServingUrl is calling to is located in the internal package (see https://github.com/golang/appengine/blob/master/internal/api.go#L435 for reference)

Our workflow

• We get a signed URL from the golang application to upload a file to Google Cloud Storage
• We send a PUT request to that URL to upload the file to a Google Cloud Storage bucket
• Upon receiving the response for that PUT request, we retrieve a serving URL using the following code:

ctx := appengine.NewContext(r)
objectName := fmt.Sprintf("%s", "/gs/"+bucketID+"/"+objectID)
key, err := blobstore.BlobKeyForFile(ctx, objectName)
client, _ := storage.NewClient(ctx)
_ , error := client.Bucket(bucketID).Object(objectID).Attrs(ctx)
res, err := image.ServingURL(ctx,appengine.BlobKey(key),&image.ServingURLOptions{Secure: true})

Summary of the issue

What we're noticing is that this scheme works (as the saying goes) "perfectly 99% of the time", returning a serving URL that allows us to dynamically resize, crop our images with ease.

There are cases in the remaining 1% where the call to ServingUrl (with correct parameters) ends up returning an API error 8 - OBJECT_NOT_FOUND.

Since we're only asking for the serving URL when the PUT request's response is coming back to us, this doesn't really make sense.

Additional note

When that error happens, the object is actually in the bucket (as observed in the Storage browser of the Google Cloud Platform console), which is consistent with the successful response from the PUT request.

We've implemented retries with exponential backoff as per the documentation, but there's only so much retrying we can do and we'd like to fully understand why the internal call to the API is indicating that the object is not present in storage, when it, in fact, is.

Thanks a lot for any help you might provide on this issue.

[calvernaz]

Any update on this issue? I have a similar problem using the go111 runtime with cloud API datastore to store blobs and use the blobstore API with the image package in a local environment. The returning URL comes in the form of gs_encoded_file:...

[jschoedt]

I ran into the same error. For me it helped closing reader/writer/client before getting the serving url. I was using defer at first but that would defer the closing and sometimes error. Closing first makes sure all data is flushed to storage.