question: bloc with many datasets

[IntCleastwood]

Bloc is everything about representing a state, that will be manipulated by events and finally gets "drawn" by the UI. Thats clear so far.

But i struggle when it comes to large amount of datasets. For example there is a product list view with 10000 products that should be represented in a editable data grid to the user. Especially I use PlutoGrid for this to make it easy to edit all the products data in a single table view format.

The products are represented then using a member variable in the corresponding bloc state. Something like List<Products> products

But lets imagine i wanna mark a changed line as orange to signal the user, that this line was modified, but not saved. The flow would be as following

1. On leaving the line, send a "LineChangedEvent" with the corresponding item id to the bloc.
2. On the bloc: Iterate through 10000 items and find the one that has been changed and set the "isModified" flag to true
3. emit the new "LoadCompleteState" with the current 10000 items
4. Throw the complete PlutoGrid widget away and build a new one (via BlocBuilder). Consider the one that was changed and draw it orange instead of the usual color.

Isn't this way too much overhead, drawing the whole grid every time new? Or is this more or less some kind of "incompatibility" that the PlutoGrid in this case is not optimised to be used in a bloc-way? I mean, there is a a manager on the PlutoGrid with "addRow" and "deleteRow" function, but anyway, at some point, the difference between the current bloc state and the grid state has to be figured out. And that is exepnsive when it comes to iterating over each dataset over and over again.

What key concept do i miss for bloc? Any chance to just send a small part (a single dataset) of the whole state to the frontend? Any solution to this?

Best regards

[codesculpture]

Ok i understand this question and also got this question (pain). Lets divide this problem into two parts:

First Problem: Iterating each time the list when some change happened. You can go for map for to resolve this or u need to divide ur list into batches, each batch have 100 item and for each 100 items there shld be seperate bloc.

Second Problem (Rendering): Actually this is a important problem, because this problem itself can be classified into two. The ui tree first need to visit each element in list and then draw for each element so two operations being happening here (visiting and drawing or rendering). But after researching quite a bit i came to understand that flutter is intelligent enough to not render same ui again and over again by caching them. So rendering problem could be omitted here. But anyway the ui needs to visit each element. To eliminate this, u may plan to use pagination, which means only render elements as user scrolls. Also known as Lazy rendering

[IntCleastwood]

Hello and thanks for your answer .. i tried to evaluate it, but it seems at least for the second problem, "re-redering only the parts that where changed" its not working as suggested ... or i misunderstood ... Lets look at the bloc builder code part from my app

...
return BlocBuilder<MyScreenBloc, MyScreenState>(
      builder: (context, state) {
       
        List<PlutoRow> rows = state.items
            .map((e) {
          return PlutoRow(cells: {
            fieldName: PlutoCell(value: e.title)
          });
        }).toList();

        return PlutoGrid(
          columns: columns,
          rows: rows,
          onChanged: (PlutoGridOnChangedEvent event) {
            MyItem item = MyItem(title: event.row.cells['title']?.value);
            context.read<MyScreenBloc>().add(RowItemChangedEvent(item: item));
          },
        );
      },
    );

Beside the RowItemChangedEvent there is also a 'LoadItemsEvent' which is called once when starting the app. This will load all items. However, when a row is changed, a RowItemChangedEvent is fired and the whole Plutogrid-Widget with all its content is re-rendered again, which is ridicolous for a grid with thousands of entries ... honestly, i don't know how flutter should be intelligent enough to just decide which lines to re-render and which not, since on every line change it gets a new list on the build method, even if it has only slight changes ...

All in all, at least for this pluto grid example it feels like bloc is not really suitable, since the PlutoGrid has its own callbacks and a StateManager for adding/deleting rows. A naive idea would be to setup the PlutoGrid Statemanage rin the bloc which would be a clear violation of the bloc approach ...

For the first problem:

You mean a map in the format: id -> item ? Since the map should be more efficient (O(1)) as the list (O(n)), this seems a good idea

[codesculpture]

Whenever a re render occurs, flutter try to diff (by comparing the new props and old props) the tree and only create new widget or re render. And also at some point it would just 'update" the widget which is cheaper process than "re-rendering". You can look for "keys" in widgets which would ( i believe) helps to solve ur problem. Feel free to correct me or ask question

[IntCleastwood]

I come more and more to the conclusion that especially with third party modules like PlutoGrid, its simply not or not meaningful possible to control it via the bloc approach. The attempt to apply bloc here feels like making everything overcomplicated ...

Example: I tried to change the row color when a dataset changed.

In bloc it would be:

1. Fire a RowChangedEvent
2. emit the state with a list of changed entries
3. on UI either return the widget with the new colorized rows using BlocBuilder or use a BlocListener to call a method on the widget which will colorize the rows as desired

Both is not possible, since on the one hand building the whole widget with new data again using BlocBuilder, its incredibly inefficient (re-render several tousand linse even if they did not changed, see comments before) On the other hand, the implementation of PlutoGrid does not provide a callable function that would set the color on a BlocListener ... Instead, PlutoGrid itself provides a callback function to change a color on each row change ...

Re-render a single row seems not possible also ... even if yes, probably you would loos the focus in frontend

Hopefully this was understandable, but it feels like fighting against with bloc against the PlutoGrid implementation ... either i missed a very fundamental key concept here, or its simply not possible ...

I currently think about to not manage anything using bloc here at all, just passing the state to the bloc when something changed but don't intervent.

I could live with this approach but i am afraid that the frontend might not reflect the actual bloc state

[codesculpture]

I guess, its possible and better to control the list (PlutoGrid) via bloc. Flutter wont re construct every widget during re render phase, its intelligent enough to note whether this widget had a change or not, if yes only it would perform re render.

[IntCleastwood]

Yes, but like in this case where the Widget is the complete Grid where i do not have any chance to intervent into the rerendering of a sub part (like a single row), the widget is rendered over and over again completely, since from the widget view its changed completely everytime i make a Text input ...

I tried to setup a grid with 1000 lines of random data and just edit one line ... the change event on the grid produces the bloc event and the bloc emits a state which will lead to the re-render of the grid.

Two effects:

1. After i leave the line after editing, the rerendering takes 2 seconds (i assume the whole grid is rebuild/replaced)
2. The whole grid is loosing the cursor focus, which is also not what i want (grid gets replaced by the builder)

Omitting the bloc event let the grid runs smoothly

So if there is a proper way to implement it, i really don't see it, since i am experimenting several days ... but i do also not have years of experience using the bloc approach. I just wonder if there might be cases where bloc is simply not meaningful possible like in this case

[codesculpture]

Alright, (am also dont have years of experience, but lets play). Can we do something like this, lets say u have 100 rows, can we provide a bloc for each rows. Which rows wont intervent with each other rendering. It seems weird but i did something similar to post features for social media app. Where there is a list of posts and each have a like feature there, so if i modify a single post whole list would get re render, but what i did was had a like cubit for each posts. And ended up this problem and i dont see any problem there.

[IntCleastwood]

This idea of a bloc for a single row/entry was also on my mind ... but i am not sure how to apply a bloc to a dynamic set of entries ... i thought, every item has its own bloc implementation ... so if i have a email form, i have a email form bloc ... if i have another email form, i would have an email form bloc 2 ... or am i wrong?

[codesculpture]

No, u might have a single row bloc but create multiple instances of them under the widget tree

For ex: PlutoGrid: RowBloc: Row 1 RowBloc: Row 2 ... Row n

Row i, only have access to its own bloc through the context, since by default bloc builder looks for its nearest bloc on the tree. All u need to have unique identifier in the Row Bloc implementation to know what row it belongs to something like a row id

[IntCleastwood]

How to provide such instances for each row? I thought this is some kind of singleton via the bloc provider?

[codesculpture]

I dont know how the Plutogrid build widgets, but its pretty straight forward to implement. Lets say we have list of todos which each has its own states completed and not Completed

class Todo {
      final String status; // Could be Completed or Not Completed
}


// this is our bloc
class GlobalTodosBloc extends Bloc<List<Todo>, SomeEvent<..> {

}

And then we had a list of todos and we supposed to render the all todos with status, where user can change any todos state at any time, but if we store all todos as a list in a bloc's state, if any single todo needs to update then the whole list of todos get re rendered.

But what we can do is having a seperate bloc for each todo, and which means each has its own state and if we need to modify a todo, we need to call particular bloc instance which belongs to that particular todo and modify the state and re render only occurs for that only todo.

class TodoBloc extends Bloc<Todo,SomeEvent> 
 }

So, in UI after we got the list, we can create blocs for each todo like this

Widget build(context) {
      final allTodos = someRepo.fetchTodos();
       return allTodos.map((todo) => BlocProvider(create: () => TodoBloc()), child: () => TodoRenderer())
}

Imagine TodoRenderer is the widget where u going to have fit the BlocBuilder

// Todo Renderer

Widget build(context) {
        return BlocBuilder<TodoBloc>(build: (context, state) => {
    // here u have the state and context where u can emit events to the bloc which is nearest to this widget.
     return Text(state)
})
}

Sorry for the poor syntax.

[IntCleastwood]

Let me check this, i will response later :) But first look sounds promising! Thanks a lot for your effort so far

[IntCleastwood]

It seems, in your example the TodoRenderer() must return a Widget ... PlutoRow isn't a widget ... ... but the Plutogrid want's to have a List<PlutoRow> for the rows: property ... or did i misunderstand?

[codesculpture]

Ok whats the row parameter expecting a widget or ?

[IntCleastwood]

Its expecting a list of PlutoRow objects ...

[codesculpture]

Objects ? U mean raw data, can u provide me a example

[IntCleastwood]

Copied from the official README https://pub.dev/packages/pluto_grid/example ... I just skipped the uninteresting parts

 final List<PlutoRow> rows = [
    PlutoRow(
      cells: {
        'id': PlutoCell(value: 'user1'),
        'name': PlutoCell(value: 'Mike'),
        'age': PlutoCell(value: 20),
        'role': PlutoCell(value: 'Programmer'),
        'joined': PlutoCell(value: '2021-01-01'),
        'working_time': PlutoCell(value: '09:00'),
        'salary': PlutoCell(value: 300),
      },
    ),
    ...
  ];

@override
  Widget build(BuildContext context) {
    return Scaffold(
      body: Container(
        child: PlutoGrid(
          rows: rows,
          ...
        ),
      ),
    );
  }

[codesculpture]

Im away from my laptop, will look at later. I took a glance at pluto grid where seems it provide a stateManager itself PlutoStateManager doesnt that work for you, since that could be more efficient because it directly provided by the PlutoGrid. You may consider that.

[IntCleastwood]

Yes thats what i say ... PlutoGrid has a state manager by itself ... i think it would be very bad practise to inject this into the bloc ... so there are not many chances ... i think bloc cant be applied ... i dont know how to proper manage app state then

[codesculpture]

Considering PlutoGrid is a complex widget, its better to use its own state manager. You can pass this state manager to bloc, but here bloc is only used to segregate logic from Ui. And u can control the PlutoStateManager through bloc

[IntCleastwood]

Is this considered from BLOC to be a good practise? What if i switch from Plutogrid to something else? Or is this more are workaround approach?

[codesculpture]

Well, this is my suggestion for ur case. Am not a official bloc member, also bloc is maintained by @felangel . He may help here. But Let me clearly my suggestion, If ur entire app is handled through blocs, then any new cases (like this) shld also use bloc. But the way of using bloc for them is differently. Because Pluto Widget is a complex widget which is not open configurable widgets, so its hard to inject bloc there. But eventually, its providing a state manager itself. And thats a good thing, considering it is a complex widget we not entirely know how its being handling state so injecting bloc requires so much of time, so its better to use their own state manager.

So, picking the state manager instance and saving it in the bloc instance and controlling through bloc would be good option in my opinion. Since here, you are not breaking the law that ur entire app is driven by bloc. But for this bloc, u dont need any blocbuilders since those handled by pluto widget. But its okay, since u have the pluto state manager in the bloc that means still ur having the control in bloc which is i see a big advantage in this approach that isHaving control through ur widget.

[IntCleastwood]

Okay, so maybe considering a complete isolation for this pluto grid bloc might also be an advantage ... so at least the "pollution" of other blocs are as small as possible ...

[IntCleastwood]

I think i faced the next problem with this approach ... the state manager of the pluto grid is bound late ... so when the PlutoGrid is created, we can assign the manager from the creation of the grid to the late variable and use it... but thats the moment were its already too late to pass it to the bloc, since the bloc provider has to be created way before ...

Maybe the code can explain better

runApp(MultiBlocProvider(
    providers: [
      BlocProvider(
          create: (context) =>
              PlutoGridBloc(PlutoGridStateManager())),
    ],
    child: MyApp(
        ...
        late PlutoGridStateManager stateManager;
        PlutoGrid grid = PlutoGrid(
            onLoaded: (event) {
            stateManager = event.stateManager;
      });
      ...
    ),
  ));

As you see, how do i pass the state manager to the bloc, that has already been created at this moment? Feels very dirty imo^^

[codesculpture]

U can have a event like StoreStateManager to the bloc and pass the statemanager and store it in the bloc as a member

[codesculpture]

class MyPlutoBloc extends Bloc<...> { 
 late final PlutoStateManager stateManager; //Uninitialized
...
on<ReceivePlutoStateManager>(event, emit)  {
   stateManager = event.stateManager
}

}

// In pluto grid, widget
runApp(MultiBlocProvider( providers: [ BlocProvider( create: (context) => PlutoGridBloc(PlutoGridStateManager())), ], child: MyApp( ... late PlutoGridStateManager stateManager; PlutoGrid grid = PlutoGrid( onLoaded: (event) { 
context.read<MyPlutoBloc>.add(ReceivePlutoStateManager(stateManager: event.stateManager))
 }); ... ), ));

*You may think a good event name than me

[felangel]

Closing this issue for now since it's been a while and there doesn't appear to be any additional action needed. If you feel this is still an issue please let me know and I'm happy to take a closer look/revisit the conversation 👍