go-force icon indicating copy to clipboard operation
go-force copied to clipboard

Enhancement: Support gzip compression

Open rickb777 opened this issue 5 years ago • 3 comments

The Salesforce API allows gzip compression on either request entities or response entities (or both). It's not to hard to support this by using the necessary Content-Encoding and Accept-Encoding headers and then passing the json marshalling through compress/gzip functions.

The only method affected is ForceApi.request, I think.

rickb777 avatar Oct 22 '20 11:10 rickb777

BTW in my experience, low compression levels are better than the defaults for this kind of use case; level 4 is used in the suggested code here. This amends ForceApi.request:

	// Build body
	var body io.Reader
	if payload != nil {
		buf := &bytes.Buffer{}

		gbuf, err := gzip.NewWriterLevel(buf, 4)
		if err != nil {
			return fmt.Errorf("Error compressing encoded payload: %v", err)
		}

		err = json.NewEncoder(gbuf).Encode(payload)
		if err != nil {
			return fmt.Errorf("Error marshaling encoded payload: %v", err)
		}

		body = buf
	}

	// Build Request
	req, err := http.NewRequest(method, uri.String(), body)
	if err != nil {
		return fmt.Errorf("Error creating %v request: %v", method, err)
	}

	// Add Headers
	req.Header.Set("User-Agent", userAgent)
	req.Header.Set("Content-Type", contentType)
	req.Header.Set("Content-Encoding", "gzip")
	req.Header.Set("Accept-Encoding", "gzip")
	req.Header.Set("Accept", responseType)
	req.Header.Set("Authorization", fmt.Sprintf("%v %v", "Bearer", forceApi.oauth.AccessToken))

	// Send
	forceApi.traceRequest(req)
	resp, err := http.DefaultClient.Do(req)
	if err != nil {
		return fmt.Errorf("Error sending %v request: %v", method, err)
	}
	defer resp.Body.Close()
	forceApi.traceResponse(resp)

	// Sometimes the force API returns no body, we should catch this early
	if resp.StatusCode == http.StatusNoContent {
		return nil
	}

	body = resp.Body
	if resp.Header.Get("Content-Encoding") == "gzip" {
		body, err = gzip.NewReader(resp.Body)
		if err != nil {
			return fmt.Errorf("Error decompressing %v response: %v", method, err)
		}
	}
	respBytes, err := ioutil.ReadAll(body)
	...  etc

The new bits of code would probably be best in new helper methods instead of inline.

rickb777 avatar Oct 22 '20 11:10 rickb777

I would have made a PR but when I checked out the code I got many test failures.

rickb777 avatar Oct 22 '20 11:10 rickb777

BTW the suggested code above uses the standard encoding/json package instead of the forcejson package. I'm not clear why forcejson exists, but if it's really needed it should have NewEncoder and NewDecoder functions like the standard library does.

rickb777 avatar Oct 22 '20 11:10 rickb777