Frame.GetImage returns just black or just white image instead of rendered frame

[kaxap]

This is due to lack of output normalization:

// GetImage returns an image.Image representation the frame, using default
// processing. This default processing is basic at the moment, and does not
// autoscale pixel values or use window width or level info.
func (n *NativeFrame) GetImage() (image.Image, error) {
	i := image.NewGray16(image.Rect(0, 0, n.Cols, n.Rows))
	for j := 0; j < len(n.Data); j++ {
		i.SetGray16(j%n.Cols, j/n.Rows, color.Gray16{Y: uint16(n.Data[j][0])}) // for now, assume we're not overflowing uint16, assume gray image
	}
	return i, nil
}

This link outlines some info on lookup tables, but it's currently impossible to get access to tags from within NativeFrame.

Temp fix is to obtain min-max values with additional loop and then normalize pixel values:

func (n *NativeFrame) GetImage() (image.Image, error) {
	i := image.NewGray16(image.Rect(0, 0, n.Cols, n.Rows))
	step := len(n.Data) / n.Rows / n.Cols

	max := n.Data[0]
	min := n.Data[0]
	// get min and max for normalization
	for j := step; j < len(n.Data); j += step {
		if n.Data[j] > max {
			max = n.Data[j]
		}
		if n.Data[j] < min {
			min = n.Data[j]
		}
	}

	for j := 0; j < len(n.Data); j += step {
		i.SetGray16(j%n.Cols, j/n.Rows,
			color.Gray16{Y: uint16(0xFFFF * (1 - float64(n.Data[j]-min)/float64(max-min)))}) // for now, assume we're not overflowing uint16, assume gray image
	}
	return i, nil
}

[suyashkumar]

Yes, this is a known behavior (see #2 for autoscale of pixel values issue). The reason it is done as it is now is to preserve the clinical data in the PixelData (which often times map to actual measurements like Hounsfield units). This currently allows to caller to rescale as appropriate for their context.

For example, imagine a caller that is a image viewer. The user will input/change some value of window width and window center, that should be used in the rescaling formula that applies to the original values in the pixel data, so could be applied directly to the image.Image as is today.

100% agree we should offer a default option for rescaling the pixel values based on the current values of various relevant tags (e.g. rescale intercept and slope, etc), but I think that should potentially be a set of utilities in a utility package (maybe pkg/imgutil or pkg/frameutil?) that helps with the rescaling. If the user applies a flag to the dicomutil utility, we can invoke some of this autoscaling.

Also of course this currently applies to NativePixelData, though rescaling based on formulas and selections of window width and window center may also apply.

Thoughts?

[suyashkumar]

There may also be value in adding a simple normalization for the dicomutil utility to start with

[kaxap]

There may also be value in adding a simple normalization for the dicomutil utility to start with

something like GetNormalizaedImage(frame *NativeFrame) (image.Image, error) {...} ?

[BastienVigneron]

any update regarding this issue ?

[BastienVigneron]

Hi @kaxap

In the meantime, I've tried to apply you're quick fix but it seems I've miss something, how can you n.Data[j] > max or n.Data[j]-min where both are []int ?

[bernardopericacho]

I am experiencing this issue too. I am basically trying to generate a png from the pixel data. This worked for me for frames that are not encapsulated.

nativeFrame := pixelDataInfo.Frames[0].NativeData
img := image.NewGray16(image.Rect(0, 0, nativeFrame.Cols, nativeFrame.Rows))

max := nativeFrame.Data[0][0]
min := nativeFrame.Data[0][0]
log.Printf("Max. %d. Min %d\n", max, min)
for j := 0; j < len(nativeFrame.Data); j++ {
	if nativeFrame.Data[j][0] > max {
		max = nativeFrame.Data[j][0]
	}
	if nativeFrame.Data[j][0] < min {
		min = nativeFrame.Data[j][0]
	}
}

for j := 0; j < len(nativeFrame.Data); j++ {
	img.SetGray16(j%nativeFrame.Cols, j/nativeFrame.Rows, color.Gray16{Y: uint16(0xFFFF * (float64(nativeFrame.Data[j][0]-min) / float64(max-min)))}) // for now, assume we're not overflowing uint16, assume gray image
}

for encapsulated frames, I tried to get the bytes and decode the image

if pixelDataInfo.Frames[0].IsEncapsulated() {
    img, _, err = image.Decode(bytes.NewReader(pixelDataInfo.Frames[0].EncapsulatedData.Data))
    if err != nil {
	    log.Printf("Error encoding image. %s\n", err.Error())
	    return err
    }
}

and basically it said that it doesn't have the right format Error encoding image. unsupported JPEG feature: unknown marker although I can see the image properly in Osirix viewer locally. (It is an MRI). Any thoughts on this? jpeg.Decode also didnt work.

I think scaling pixels based on the standard would be a good feature. This is preventing me to migrate from features from python to Go. Pixel utilities to:

• Apply a modality lookup table (based on LUT Descriptor (0028,3002) and (0028,3000) *Modality LUT Sequence*)
• Apply rescale operation (based on (0028,1052) *Rescale Intercept* and (0028,1053) *Rescale Slope*)
• Apply a VOI lookup table (based on LUT Descriptor (0028,3002) and (0028,3010) *VOI LUT Sequence*)
• Apply windowing operation (based on (0028,1050) *Window Center* and (0028,1051) *Window Width*)

[bbeckrest]

@bernardopericacho see https://github.com/suyashkumar/dicom/issues/85 for open issue regarding parsing encapsulated images

[suyashkumar]

Yep, as @bbeckrest said, the second issue you're facing @bernardopericacho is #85 -- namely that lossless JPEG decoding doesn't yet have native support in Go. At some point I may take this up myself and see if I can write a decoder in Go, but I'll have to see when I have time for that (and also if there's other interest outside the DICOM community to help drive that). Note that other encapsulated images (non LJPEG) should be decoded just fine by this library!

As for supporting the various schemes of DICOM based rescaling of pixel values--I agree that this should be done (thus this issue and #2). For now, this library will at least help users get the raw data themselves (the value of which may be clinically meaningful) and rescale (or not) as appropriate in the case of Native PixelData.

In terms of my time bandwidth, I can plan on taking a closer look later this weekend and then in a couple weeks when I should have some more time!