symbol layer - images missing on initial zoom

[mfazekas]

Environment

• Android OS version: c7a4d0a06e7ee01808bf08bbe6046a0d15031f04 aka 10.14.1 and 5d9e37d2e7b8da090536b89feea965bd4afb51cf aka 10.5.0-rc.1
• Devices affected: Android Emulator Pixel 3a API 34 extension elevel 7 arm64-v8a
• Maps SDK Version:

Observed behavior and steps to reproduce

To reproduce use the following component as SimpleMapActivity.kt, then run the Display a map view example

package com.mapbox.maps.testapp.examples

import android.graphics.Bitmap
import android.graphics.drawable.BitmapDrawable
import android.os.Bundle
import android.os.Handler
import android.util.Log
import androidx.appcompat.app.AppCompatActivity
import androidx.appcompat.content.res.AppCompatResources
import com.mapbox.geojson.Point
import com.mapbox.maps.CameraOptions
import com.mapbox.maps.Image
import com.mapbox.maps.MapView
import com.mapbox.maps.extension.style.layers.addLayer
import com.mapbox.maps.extension.style.layers.generated.symbolLayer
import com.mapbox.maps.extension.style.sources.addSource
import com.mapbox.maps.extension.style.sources.generated.geoJsonSource
import com.mapbox.maps.testapp.R
import java.nio.ByteBuffer

/**
 * Example of displaying a map.
 */

fun Bitmap.toImage() : Image {
  if (getConfig() != Bitmap.Config.ARGB_8888) {
    throw RuntimeException("Only ARGB_8888 bitmap config is supported!")
  }
  val byteBuffer = ByteBuffer.allocate(getByteCount())
  copyPixelsToBuffer(byteBuffer)
  return Image(getWidth(), getHeight(), byteBuffer.array())
}
class SimpleMapActivity : AppCompatActivity() {
  override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    val mapView = MapView(this)
    setContentView(mapView)
    mapView.getMapboxMap()
      .apply {
        setCamera(
          CameraOptions.Builder()
            .center(Point.fromLngLat(LONGITUDE, LATITUDE))
            .zoom(9.0)
            .build()
        )
      }
    mapView.getMapboxMap().addOnStyleImageMissingListener {
      Log.e("SimpleMapActivity", "on image missing: $it")
    }
    mapView.getMapboxMap().addOnStyleLoadedListener {
      val mapboxMap = mapView.getMapboxMap()
      val style = mapboxMap.getStyle()!!


      val bitmap = (AppCompatResources.getDrawable(this, R.drawable.mapbox_logo_icon) as BitmapDrawable).bitmap

      style.addSource(
        geoJsonSource("points-src") {
          geometry(Point.fromLngLat(LONGITUDE, LATITUDE))
        }
      )
      style.addLayer(
        symbolLayer("points-layer", "points-src") {
          textField("Hello world")
          iconImage("an-image")
        }
      )

      Handler().postDelayed({
        Log.e("SimpleMapActivity", "Add style image")
        style.addStyleImage(
          "an-image",
          1.0f,
          bitmap.toImage(),
          false,
          listOf(),
          listOf(),
          null
        )
      },
      1000)
    }
  }

  companion object {
    private const val LATITUDE = 40.0
    private const val LONGITUDE = -74.5
  }
}

On default zoom it looks like this:

Zooming out a bit will show this image correctly.

See also video image showing/hiding when changing zoom:

https://github.com/mapbox/mapbox-maps-android/assets/52435/ff604eb5-1ae4-4b9b-b041-02e042730fe1

Expected behavior

Show on all resolutions. The issue also doesn't happens when I load the image immediately or changing delay in Handler().postDelayed({ to 0.

Notes / preliminary analysis

Sounds like some kind of race condition, also I do see on image missing called before I add the image:

2023-07-18 16:03:36.604  6525-6525  SimpleMapActivity       com.mapbox.maps.testapp              E  on image missing: StyleImageMissingEventData(begin=330237873989, end=null, id=an-image)
...
2023-07-18 16:03:37.202  6525-6525  SimpleMapActivity       com.mapbox.maps.testapp              E  Add style image

Additional links and references

[veb-ioki]

Thanks for the issue!

This is indeed affecting our app in production. Is there any hope this will be fixed soon?

[mfazekas]

I've checked the v10.16 branch and the issue was there, and also on the #main branch (5e9a5140afef64fefb16ddeed7f87f7245e08a4b) and the issue was there.

[mfazekas]

this is the version of the code for the v11 beta:

package com.mapbox.maps.testapp.examples

import android.graphics.Bitmap
import android.graphics.drawable.BitmapDrawable
import android.os.Bundle
import android.os.Handler
import android.util.Log
import androidx.appcompat.app.AppCompatActivity
import androidx.appcompat.content.res.AppCompatResources
import com.mapbox.bindgen.DataRef
import com.mapbox.geojson.Point
import com.mapbox.maps.CameraOptions
import com.mapbox.maps.Image
import com.mapbox.maps.MapView
import com.mapbox.maps.extension.observable.eventdata.StyleImageMissingEventData
import com.mapbox.maps.extension.style.layers.addLayer
import com.mapbox.maps.extension.style.layers.generated.symbolLayer
import com.mapbox.maps.extension.style.sources.addSource
import com.mapbox.maps.extension.style.sources.generated.geoJsonSource
import com.mapbox.maps.testapp.R
import java.nio.ByteBuffer

/**
 * Example of displaying a map.
 */

fun Bitmap.toImage() : Image {
  if (getConfig() != Bitmap.Config.ARGB_8888) {
    throw RuntimeException("Only ARGB_8888 bitmap config is supported!")
  }
  val byteBuffer = ByteBuffer.allocateDirect(getByteCount())
  copyPixelsToBuffer(byteBuffer)
  return Image(getWidth(), getHeight(), DataRef(byteBuffer))
}

class SimpleMapActivity : AppCompatActivity() {
  override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    val mapView = MapView(this)
    setContentView(mapView)
    mapView.getMapboxMap()
      .apply {
        setCamera(
          CameraOptions.Builder()
            .center(Point.fromLngLat(LONGITUDE, LATITUDE))
            .zoom(9.0)
            .build()
        )
      }
    mapView.getMapboxMap().addOnStyleImageMissingListener {
      Log.e("SimpleMapActivity", "on image missing: $it")
    }
    mapView.getMapboxMap().addOnStyleLoadedListener {
      val mapboxMap = mapView.getMapboxMap()
      val style = mapboxMap.getStyle()!!

      val bitmap = (AppCompatResources.getDrawable(this, R.drawable.mapbox_logo_icon) as BitmapDrawable).bitmap

      fun addImage() {
        Log.e("SimpleMapActivity", "Add style image")
        val ret = style.addStyleImage(
          "an-image",
          1.0f,
          bitmap.toImage(),
          false,
          listOf(),
          listOf(),
          null
        )
        Log.e("SimpleMapActivity", "[x] Result of addStyleImage: ret=${ret.isValue} error=${ret.error?: "n/a"}")
        val image_check = style.getStyleImage("an-image")
        if (image_check != null) {
          Log.e("SimpleMapActivity", "[x] Check after add image: ${image_check.width} x ${image_check.height}")
        } else {
          Log.e("SimpleMapActivity", "[x] Check after add image: missing")
        }
      }

      style.addSource(
        geoJsonSource("points-src") {
          geometry(Point.fromLngLat(LONGITUDE, LATITUDE))
        }
      )
      style.addLayer(
        symbolLayer("points-layer", "points-src") {
          textField("Hello world")
          iconImage("an-image")
        }
      )

      if (false) {
        addImage()
      } else {
        Handler().postDelayed(
          {
            addImage()
          },
          1000
        )
      }

    }
  }

  companion object {
    private const val LATITUDE = 40.0
    private const val LONGITUDE = -74.5
  }
}

[kiryldz]

@mfazekas thanks for the code. The image should be loaded beforehand if you refer to it in some layer. In your case you're referring to it here:

style.addLayer(
        symbolLayer("points-layer", "points-src") {
          textField("Hello world")
          iconImage("an-image")
  }
)

That explains why you see some weird behaviour when trying to actually firstly add the layer with a reference to the image which is not yet loaded.

If - for some reason - you do not want / have no ability to addStyleImage before you add a layer that refers to it, you could actually add it in OnStyleImageMissingListener. I've updated your code for Maps v11:

package com.mapbox.maps.testapp.examples

import android.graphics.Bitmap
import android.graphics.drawable.BitmapDrawable
import android.os.Bundle
import android.os.Handler
import android.util.Log
import androidx.appcompat.app.AppCompatActivity
import androidx.appcompat.content.res.AppCompatResources
import com.mapbox.bindgen.DataRef
import com.mapbox.geojson.Point
import com.mapbox.maps.CameraOptions
import com.mapbox.maps.Image
import com.mapbox.maps.MapView
import com.mapbox.maps.Style
import com.mapbox.maps.extension.observable.eventdata.StyleImageMissingEventData
import com.mapbox.maps.extension.style.layers.addLayer
import com.mapbox.maps.extension.style.layers.generated.symbolLayer
import com.mapbox.maps.extension.style.sources.addSource
import com.mapbox.maps.extension.style.sources.generated.geoJsonSource
import com.mapbox.maps.testapp.R
import java.nio.ByteBuffer

/**
 * Example of displaying a map.
 */

fun Bitmap.toImage() : Image {
  if (getConfig() != Bitmap.Config.ARGB_8888) {
    throw RuntimeException("Only ARGB_8888 bitmap config is supported!")
  }
  val byteBuffer = ByteBuffer.allocateDirect(getByteCount())
  copyPixelsToBuffer(byteBuffer)
  return Image(getWidth(), getHeight(), DataRef(byteBuffer))
}

class SimpleMapActivity : AppCompatActivity() {

  fun addImage(style: Style, bitmap: Bitmap) {
    Log.e("SimpleMapActivity", "Add style image")
    val ret = style.addStyleImage(
      "an-image",
      1.0f,
      bitmap.toImage(),
      false,
      listOf(),
      listOf(),
      null
    )
    Log.e("SimpleMapActivity", "[x] Result of addStyleImage: ret=${ret.isValue} error=${ret.error?: "n/a"}")
    val image_check = style.getStyleImage("an-image")
    if (image_check != null) {
      Log.e("SimpleMapActivity", "[x] Check after add image: ${image_check.width} x ${image_check.height}")
    } else {
      Log.e("SimpleMapActivity", "[x] Check after add image: missing")
    }
  }

  override fun onCreate(savedInstanceState: Bundle?) {
    super.onCreate(savedInstanceState)
    val mapView = MapView(this)
    setContentView(mapView)
    mapView.getMapboxMap()
      .apply {
        setCamera(
          CameraOptions.Builder()
            .center(Point.fromLngLat(LONGITUDE, LATITUDE))
            .zoom(9.0)
            .build()
        )
      }
    mapView.getMapboxMap().addOnStyleImageMissingListener {
      Log.e("SimpleMapActivity", "on image missing: $it")
      val bitmap = (AppCompatResources.getDrawable(this, R.drawable.mapbox_logo_icon) as BitmapDrawable).bitmap
      addImage(mapView.getMapboxMap().getStyle()!!, bitmap)
    }
    mapView.getMapboxMap().addOnStyleLoadedListener {
      val mapboxMap = mapView.getMapboxMap()
      val style = mapboxMap.getStyle()!!

      style.addSource(
        geoJsonSource("points-src") {
          geometry(Point.fromLngLat(LONGITUDE, LATITUDE))
        }
      )
      style.addLayer(
        symbolLayer("points-layer", "points-src") {
          textField("Hello world")
          iconImage("an-image")
        }
      )
    }
  }

  companion object {
    private const val LATITUDE = 40.0
    private const val LONGITUDE = -74.5
  }
}

[mfazekas]

@kiryldz thanks much for looking into it. Note that we're talking about RNMapbox - and open-source React Native library for building maps with the Mapbox SDK.

So the image is have to be provided: a.) Before the symbol layer referring to the image was added. b.) From the StyleImageMissingListener callback. If you provide the image at a later point, the image might not render correctly.

The issue is that in our environment the app code is in javascript so we might not be able to provide the image from the StyleImageMissingListener callback. From the StyleImageMissingListener we send event (about the missing image) to the user code which is javascript and running on another thread. And they provide an image at a later point, and we have no way to block the StyleImageMissingListener, until they have provided the image.

So the following is not supported as the image is added outside of the image missing listener?

mapView.getMapboxMap().addOnStyleImageMissingListener {
  Handler().postDelayed(
            {
              addImage()
            },
            1000
          )
}

[kiryldz]

@mfazekas no, it is not supported. Can you clarify why is it not possible to build the code in a way that firstly you add an image to the style (with addStyleImage) and then setup the layer that uses this image?

[mfazekas]

@kiryldz One example is that one might generate icon-image using and expression and then generate the image dynamically for onStyleImageMissing.

For example to implement the equivalent of this demo: https://docs.mapbox.com/mapbox-gl-js/example/cluster-html/ (to clarify this demo uses marker views, but the similar effect could be achieved using expression as iconImage based on clusterProperties and the using onStyleImageMissing to generate the image when needed.)

We're providing and library, and onStyleImageMissing provides a lot of flexibility. But because of react-native architecture our code is not sync, so we cannot perform the addImage from onStyleImageMissing

[mfazekas]

@kiryldz I've also tried adding a dummy image until the final one is available and using addStyleImage to update the image, but it was not updated in the current zoom.

But the docs mention image updates, so i'd assume that should work:

https://docs.mapbox.com/android/maps/api/10.6.1/mapbox-maps-android/com.mapbox.maps/-style/add-style-image.html

| This API can also be used for updating an image. If the image id was already added, it gets replaced by the new image.

[alexshalamov]

But because of react-native architecture our code is not sync, so we cannot perform the addImage from onStyleImageMissing

@mfazekas, the best option would be to add a transparent placeholder image immediately from onStyleImageMissing and then request (generate) an image asynchronously. Once the image is available, you can use addStyleImage to update the placeholder.

I've also tried adding a dummy image until the final one is available and using addStyleImage to update the image

What was the size of a dummy image? Do you know the size of the image in advance? Would be great to add a placeholder image that is of the same size as the final image.

[mfazekas]

But because of react-native architecture our code is not sync, so we cannot perform the addImage from onStyleImageMissing

@mfazekas, the best option would be to add a transparent placeholder image immediately from onStyleImageMissing and then request (generate) an image asynchronously. Once the image is available, you can use addStyleImage to update the placeholder.

I've also tried adding a dummy image until the final one is available and using addStyleImage to update the image

What was the size of a dummy image? Do you know the size of the image in advance? Would be great to add a placeholder image that is of the same size as the final image.

When the image sizes were the same the map refreshed - so blue_marker_view => red_marker worked fine in the example app. But different sizes it still showed the old image in the current zoom. (ic_taxi_top => ic_car_top in the example app)

https://github.com/mapbox/mapbox-maps-android/assets/52435/2068b1af-6948-44b8-ba0c-99947faf95f8

[alexshalamov]

But different sizes it still showed the old image in the current zoom.

@mfazekas Thanks, that could be a bug on our side. We will take a look. However, modifying an image size will trigger tile re-layout, thus, affecting rendering performance. It is advisable to update images without changing their dimensions.

[mfazekas]

@alexshalamov thanks should I open a different issue for image size change issue?

Being able to change image sizes even at a cost of re-layout sounds fine. I'm also modifying the RNMapbox code to generate the the placeholders with the same size as the image that's resolved only later. But sometimes the size is not available, like when the user specifies an image from the web.

Also in case of react-native we can't really do async callback from Kotlin to javascript, so we can't update in onImageMissing callback just somewhat later. This might get fixed with new architecture of react-native but that's still work in progress.

[baleboy]

@alexshalamov thanks should I open a different issue for image size change issue?

If you could, that would be appreciated. Thanks!

[jakub-oone]

@alexshalamov thanks should I open a different issue for image size change issue?

Being able to change image sizes even at a cost of re-layout sounds fine. I'm also modifying the RNMapbox code to generate the the placeholders with the same size as the image that's resolved only later. But sometimes the size is not available, like when the user specifies an image from the web.

Also in case of react-native we can't really do async callback from Kotlin to javascript, so we can't update in onImageMissing callback just somewhat later. This might get fixed with new architecture of react-native but that's still work in progress.

Hi @mfazekas , We are facing the same issue about missing SymbolLayer images on initial zoom. After some time of zooming in/out (the time depends on amount of markers on map) the images finally appear. Do you have some idea how to fix this issue? Here is the video recording https://discord.com/channels/1004826913229000704/1194228577978290206/1194228577978290206

Thank you! We would be grateful for any help

[mfazekas]

@jakub-oone pls use rnmapbox repo for rnmapbox maps specific issue. Please post a bug report or discussion with a simple component to reproduce the issue.