Avalonia
Avalonia copied to clipboard
AvaloniaUI
The app crashes when we resize it while it renders an image
Describe the bug I am acquiering images with a camera very fast and I'm displaying them on the UI using an Observable. During the aquisition i'm resizing the application and it crashes. I guess its because I am resizing the application while it renders the image.
Exception
System.NullReferenceException: Object reference not set to an instance of an object.
at Avalonia.Media.Imaging.Bitmap.get_Size() in /_/src/Avalonia.Visuals/Media/Imaging/Bitmap.cs:line 134
at Avalonia.Controls.Image.MeasureOverride(Size availableSize) in /_/src/Avalonia.Controls/Image.cs:line 105
at Avalonia.Layout.Layoutable.MeasureCore(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 559
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Layout.LayoutHelper.MeasureChild(ILayoutable control, Size availableSize, Thickness padding) in /_/src/Avalonia.Layout/LayoutHelper.cs:line 46
at Avalonia.Layout.LayoutHelper.MeasureChild(ILayoutable control, Size availableSize, Thickness padding, Thickness borderThickness) in /_/src/Avalonia.Layout/LayoutHelper.cs:line 39
at Avalonia.Controls.Border.MeasureOverride(Size availableSize) in /_/src/Avalonia.Controls/Border.cs:line 187
at Avalonia.Layout.Layoutable.MeasureCore(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 559
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Controls.Grid.MeasureOverride(Size constraint) in /_/src/Avalonia.Controls/Grid.cs:line 230
at Avalonia.Layout.Layoutable.MeasureCore(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 559
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Controls.Grid.MeasureCell(Int32 cell, Boolean forceInfinityV) in /_/src/Avalonia.Controls/Grid.cs:line 1150
at Avalonia.Controls.Grid.MeasureCellsGroup(Int32 cellsHead, Size referenceSize, Boolean ignoreDesiredSizeU, Boolean forceInfinityV, Boolean& hasDesiredSizeUChanged) in /_/src/Avalonia.Controls/Grid.cs:line 1005
at Avalonia.Controls.Grid.MeasureCellsGroup(Int32 cellsHead, Size referenceSize, Boolean ignoreDesiredSizeU, Boolean forceInfinityV) in /_/src/Avalonia.Controls/Grid.cs:line 968
at Avalonia.Controls.Grid.MeasureOverride(Size constraint) in /_/src/Avalonia.Controls/Grid.cs:line 489
at Avalonia.Layout.Layoutable.MeasureCore(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 559
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Controls.Grid.MeasureOverride(Size constraint) in /_/src/Avalonia.Controls/Grid.cs:line 230
at Avalonia.Layout.Layoutable.MeasureCore(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 559
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Controls.Grid.MeasureCell(Int32 cell, Boolean forceInfinityV) in /_/src/Avalonia.Controls/Grid.cs:line 1150
at Avalonia.Controls.Grid.MeasureCellsGroup(Int32 cellsHead, Size referenceSize, Boolean ignoreDesiredSizeU, Boolean forceInfinityV, Boolean& hasDesiredSizeUChanged) in /_/src/Avalonia.Controls/Grid.cs:line 1005
at Avalonia.Controls.Grid.MeasureCellsGroup(Int32 cellsHead, Size referenceSize, Boolean ignoreDesiredSizeU, Boolean forceInfinityV) in /_/src/Avalonia.Controls/Grid.cs:line 968
at Avalonia.Controls.Grid.MeasureOverride(Size constraint) in /_/src/Avalonia.Controls/Grid.cs:line 489
at Avalonia.Layout.Layoutable.MeasureCore(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 559
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Layout.LayoutHelper.MeasureChild(ILayoutable control, Size availableSize, Thickness padding) in /_/src/Avalonia.Layout/LayoutHelper.cs:line 46
at Avalonia.Layout.LayoutHelper.MeasureChild(ILayoutable control, Size availableSize, Thickness padding, Thickness borderThickness) in /_/src/Avalonia.Layout/LayoutHelper.cs:line 39
at Avalonia.Controls.Presenters.ContentPresenter.MeasureOverride(Size availableSize) in /_/src/Avalonia.Controls/Presenters/ContentPresenter.cs:line 366
at Avalonia.Layout.Layoutable.MeasureCore(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 559
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Layout.LayoutHelper.MeasureChild(ILayoutable control, Size availableSize, Thickness padding) in /_/src/Avalonia.Layout/LayoutHelper.cs:line 46
at Avalonia.Controls.Decorator.MeasureOverride(Size availableSize) in /_/src/Avalonia.Controls/Decorator.cs:line 54
at Avalonia.Controls.Primitives.VisualLayerManager.MeasureOverride(Size availableSize) in /_/src/Avalonia.Controls/Primitives/VisualLayerManager.cs:line 133
at Avalonia.Layout.Layoutable.MeasureCore(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 559
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Layout.Layoutable.MeasureOverride(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 625
at Avalonia.Layout.Layoutable.MeasureCore(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 559
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Layout.Layoutable.MeasureOverride(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 625
at Avalonia.Controls.Window.MeasureOverride(Size availableSize) in /_/src/Avalonia.Controls/Window.cs:line 937
at Avalonia.Controls.WindowBase.MeasureCore(Size availableSize) in /_/src/Avalonia.Controls/WindowBase.cs:line 247
at Avalonia.Layout.Layoutable.Measure(Size availableSize) in /_/src/Avalonia.Layout/Layoutable.cs:line 364
at Avalonia.Layout.LayoutManager.Measure(ILayoutable control) in /_/src/Avalonia.Layout/LayoutManager.cs:line 297
at Avalonia.Layout.LayoutManager.ExecuteMeasurePass() in /_/src/Avalonia.Layout/LayoutManager.cs:line 261
at Avalonia.Layout.LayoutManager.InnerLayoutPass() in /_/src/Avalonia.Layout/LayoutManager.cs:line 243
at Avalonia.Layout.LayoutManager.ExecuteLayoutPass() in /_/src/Avalonia.Layout/LayoutManager.cs:line 145
at Avalonia.Controls.WindowBase.HandleResized(Size clientSize, PlatformResizeReason reason) in /_/src/Avalonia.Controls/WindowBase.cs:line 225
at Avalonia.Controls.Window.HandleResized(Size clientSize, PlatformResizeReason reason) in /_/src/Avalonia.Controls/Window.cs:line 1018
at Avalonia.Win32.WindowImpl.AppWndProc(IntPtr hWnd, UInt32 msg, IntPtr wParam, IntPtr lParam) in /_/src/Windows/Avalonia.Win32/WindowImpl.AppWndProc.cs:line 399
at Avalonia.Win32.WindowImpl.WndProc(IntPtr hWnd, UInt32 msg, IntPtr wParam, IntPtr lParam) in /_/src/Windows/Avalonia.Win32/WindowImpl.WndProc.cs:line 30
at Avalonia.Win32.Interop.UnmanagedMethods.DefWindowProc(IntPtr hWnd, UInt32 msg, IntPtr wParam, IntPtr lParam)
at Avalonia.Win32.WindowImpl.AppWndProc(IntPtr hWnd, UInt32 msg, IntPtr wParam, IntPtr lParam) in /_/src/Windows/Avalonia.Win32/WindowImpl.AppWndProc.cs:line 539
at Avalonia.Win32.WindowImpl.WndProc(IntPtr hWnd, UInt32 msg, IntPtr wParam, IntPtr lParam) in /_/src/Windows/Avalonia.Win32/WindowImpl.WndProc.cs:line 30
To Reproduce Steps to reproduce the behavior: 0 - Load an image and make sure you copy it at each call 1 - Create an observable of Avalonia.Media.Imaging.Bitmap, to simulate the camera, i'm using a Interval Observble ticking very fast (10 ms) and project the time to a copy of _mySkBitmap. The Observavle converts the SKBitmap to the WriteableBitmap
private SKBitmap _mySkBitmap = SKBitmap.Decode(System.IO.File.ReadAllBytes("image_00028.bmp"));
private SKBitmap MySkBitmap => _mySkBitmap.Copy();
public IObservable<Bitmap?> DisplayInputImageObservable => Observable.Interval(TimeSpan.FromMilliseconds(10)).Select(_ => MySkBitmap)
.ObserveOn(Scheduler.Default)
.Select(img =>
{
// bitmap is a SkiaSharp.SKBitmap
using var skImage = SKImage.FromBitmap(img);
using var data = skImage.Encode(SKEncodedImageFormat.Jpeg, 100);
using var stream = data.AsStream();
var writeableBitmap = WriteableBitmap.Decode(stream);
img.Dispose();
if (_prevRef != writeableBitmap && _prevRef is not null)
{
_prevRef.Dispose();
}
_prevRef = writeableBitmap;
return writeableBitmap;
})
.Catch<Bitmap?, Exception>(e => { return Observable.Empty<Bitmap>(); });
2 - Connect it to the View
<Image Source="{Binding DisplayInputImageObservable^}"
Name="MyImage"
Stretch="Fill" />
Expected behavior The app should resize correcly without crashing when it renders an image
Screenshots
Desktop (please complete the following information):
- OS: windows 11
- Avalonia version 0.10.16
.ObserveOn(Scheduler.Default)
That's dangerous. UI changes should be done only on UI thread. .ObserveOn(RxApp.MainThreadScheduler) should be used at least before Catch call.
Probably that's a reason of your problem as well. As Image.Source is updated from another thread.
Thank you @maxkatz6
Unfortunately, I have the same issue with .ObserveOn(RxApp.MainThreadScheduler) as well as without any ObserveOn
Why are you recreating the Bitmap every frame? Try to just allocate one Bitmap and override its content. Your camera feed should have fixed dimensions.
Hello @Gillibald and thank you for your help.
I have not found any way to allocate the pixels to an existing Avalonia.Media.Imaging.Bitmap or Avalonia.Media.Imaging.WriteableBitmap. The only way I found is to creating a new object from a stream. Am I missing something ?
best regards
var bitmap = new WriteableBitmap(new PixelSize(100, 100), new Vector(96, 96), Avalonia.Platform.PixelFormat.Rgba8888, Avalonia.Platform.AlphaFormat.Premul);
using(var buffer = bitmap.Lock())
{
buffer.Address //Write here
}
Thank you @Gillibald, the Address pointer property is get only if i'm not wrong.
A memory address is just the start of a memory region. You have to write to that region. You know how many pixels you have and how much space a pixel takes.
A memory address is just the start of a memory region. You have to write to that region. You know how many pixels you have and how much space a pixel takes.
Yes that's right! How would you then trigger the framework to redraw as it is the same referenced object? With this approach, you use an observable or a property with RaiseAndSetIfChanged ?
Hello, following up on this topic, just raising the change event does not work. Nothing is happening, the image does not show on the app.
Here is what I have :
fields:
private WriteableBitmap _outputImage = new WriteableBitmap(new PixelSize(1920, 1200), new Vector(96, 96), Avalonia.Platform.PixelFormat.Rgba8888, Avalonia.Platform.AlphaFormat.Premul);
private WriteableBitmap _inputImage = new WriteableBitmap(new PixelSize(1920, 1200), new Vector(96, 96), Avalonia.Platform.PixelFormat.Rgba8888, Avalonia.Platform.AlphaFormat.Premul);
When I recive a new image I'm doing this.
using SKBitmap outputToCopy = img.Output;
using var bufferDestination = _outputImage.Lock();
Marshal.Copy(outputToCopy.Bytes, 0, bufferDestination.Address, outputToCopy.Width * outputToCopy.Height);
this.RaisePropertyChanged();
The property linked from the view to the view model:
public WriteableBitmap OutputImage => _outputImage;
public WriteableBitmap InputImage => _inputImage;
Your best bet is to provide a minimal sample. Then we can have a deeper look
Thank you,
There it is. Basically there is an interval observable of 10 ms that simulate a camera stream. The image is displayed on the main view. Try to resize the application and it crashes
best regards
https://github.com/broadside74/avalonia.issue.rezize_while_render_image
I wonder why you dispose your image over and over again and create a new one. I thought WriteableBitmap was there to just replace the Pixels?
You can also do new Bitmap(stream) if you don't need WriteableBitmap.
@avaloniaui-team : I found that in https://github.com/AvaloniaUI/Avalonia/blob/8f788883315c66ad1b9af27a9103e6b2049cfa3c/src/Avalonia.Base/Media/Imaging/Bitmap.cs#L110 Item can be null and that is where it crashes on resize. I don't know if we can / should mark Item as nullable and make a null check where needed?
I wonder why you dispose your image over and over again and create a new one. I thought WriteableBitmap was there to just replace the Pixels? It is related to this discussion https://github.com/AvaloniaUI/Avalonia/discussions/7669#discussioncomment-2221375 You can also do
new Bitmap(stream)if you don't need WriteableBitmap.@avaloniaui-team : I found that in
https://github.com/AvaloniaUI/Avalonia/blob/8f788883315c66ad1b9af27a9103e6b2049cfa3c/src/Avalonia.Base/Media/Imaging/Bitmap.cs#L110
Item can be null and that is where it crashes on resize. I don't know if we can / should mark Item as nullable and make a null check where needed?
I'd suggest to not recreate images but to use WritableBitmap instead.
That means: don't create the image over and over again. Just replace the pixels. You need to lock your Bitmap to do so I think. ATM I don't have a sample for you, but maybe you can find one online.
I think the issue is, that some times the Binding has not the new Bitmap picked up while it actually gets disposed. To work around this I've sent you a PR.
https://github.com/broadside74/avalonia.issue.rezize_while_render_image/pull/1
It may only be a workaround, but at least for me I had no more crash
Hello @timunie
First of all, big thank you for your help and your PR! it works in this case but I have some questions and remarks.
- It works with avalonia 0.10.18 and not for 0.10.14. I did not really checked why
- It is important to me to display every single images in my application even though in reality, the application won't display at that kind of speed (every 10ms). This sample is just to demonstrate the concept and show it can crashes (even if it is very rare)
With this workaround, what if the camera update the MySkBitmap object quicker than the tick of the timer ?
https://github.com/AvaloniaUI/Avalonia/blob/8f788883315c66ad1b9af27a9103e6b2049cfa3c/src/Avalonia.Base/Media/Imaging/Bitmap.cs#L110
@avaloniaui-team By design, the property PlatformImpl.Item is not marked as nullable but it can actually be.
I tested the following code and it works fine. I think we definitely need to check the image is not null before actually reaching the Size property. This must be the same with the Dpi property.
public Size Size
{
get
{
if(PlatformImpl.Item == null)
return Size.Empty;
return PlatformImpl.Item.PixelSize.ToSizeWithDpi(Dpi);
}
}
Best regards
Alex
I agree that my PR is just a workaround. Things to add:
If you need to render every frame, your Observable may fail as well. Instead you need to listen to your camera change event if it exists.
Moreover you may want to pre cache some frames.
Also try update to 11.0 preview 1 as this is faster in rendering.
Item can only be null if the PlatformImpl is disposed before it is being used by the renderer. This should not happen. So we need to investigate when it is being disposed too early.
Item can only be null if the PlatformImpl is disposed before it is being used by the renderer. This should not happen. So we need to investigate when it is being disposed too early.
Exactly ! It was because I had previously memory issues (see this thread https://github.com/AvaloniaUI/Avalonia/discussions/7669#discussioncomment-2221375). I decided to manualy dispose previous avalonia.imaging.Bitmap object which did the trick but when we resize the app, we need apparently need the reference.
if (_prevRef != writeableBitmap && _prevRef is not null)
{
_prevRef.Dispose();
}
I suggest closing this issue and find a better way to correctly dispose these Bitmap
Thank you all