Integration in mineflayer

[rom1504]

Related #65 Things to do :

• [x] #115 use Minecraft data
• [x] pchunk bedrock support
• [x] pblock and pitem support
• [ ] Actual implementation in mineflayer, 2 choices. I prefer the first choice, but it depends on how difficult it would be in practice
  • 1. [ ] Build an abstraction layer for java and bedrock packets, build new mineflayer plugins relying on these stable clean and nice packets instead of raw packets
  • 2. [ ] Build mineflayer plugins for bedrock raw packets

[extremeheat]

Progress Report, (trying to finally get this bedrock stuff out of the way)

• https://github.com/PrismarineJS/minecraft-data/pull/445 has been opened to move the protocol data to minecraft-data, #115 will implement it here in bedrock-protocol, with https://github.com/PrismarineJS/node-minecraft-data/pull/110
  • #98 will be done in https://github.com/PrismarineJS/minecraft-data/pull/454
• prismarine-block support in https://github.com/PrismarineJS/prismarine-block/pull/40
• prismarine-item is blocked by https://github.com/PrismarineJS/prismarine-item/pull/38 to avoid a conflict
• the easy approach to pchunk for now I think is to depend on bedrock-provider (it doesn't depend on leveldb anymore), I'll have something to update it to update it to 1.17.10 + new prismarine-block later this week
• prismarine-recipes support for bedrock recipe schema in https://github.com/PrismarineJS/prismarine-recipe/pull/9
  • main difference in the schema is "ingredients" is always set as a flat array, with indexes for the shapes in the "inputs" field.
  • This also uses strings when addressing items as there are no hardcoded item IDs on bedrock.
  • We could make it more conformant to the PC schema (it was just easier to code with this schema), but after implementing it in prismarine-recipie imo I think it's alot easier keeping it this way

For consideration:

• New lib to do inventories and/or expand scope of p-windows ?
  • Inventory logic needs to be shared between the client and the server, and nickelpro is also working on revamping java edition inventories for pc 1.17.
  • https://extremeheat.github.io/bedrock-protocol/protocol/1.16.220.html#ItemStackRequest - on bedrock, the inventory actions are high level instead of being click based, so actions like take, place, swap, etc. are sent instead of individual clicks.
  • Clients can also batch inventory requests (e.g. click spreading in one packet), so it needs to be able to simulate these requests from the server. If the server rejects, client needs to discard and undo the batch.

[u9g]

In terms of inventory, nickelpro was talking about a similar way to do high level actions with p-window where you would be able to say window.swap(itemOne, itemTwo) and everything with slots would be handled by p-window

[extremeheat]

Yeah, that could be joined along with a transaction group API to batch many actions at once. Something like this is also needed for GUI inventories in Minecraft, and I've implemented it into minecraft-inventory-gui for creative mode (just not connected to network APIs) : https://github.com/extremeheat/minecraft-inventory-gui/blob/bedrock/web/CW2.js

For example, doing spread operations in the inventory, then cancelling it requires the client to simulate the spread, then if not cancelled, send the group to the server for confirmation (or rejection).

For the mineflayer side an API like this :

interface Inventory {
  // take from slot and put count into hand
  take({ from?: string, fromSlot?: number, count?: number })
  // take item from slot and put count into hand
  take({ from?: string, item?: ItemName, count?: number })
  // place from hand to slot specific if slot is specified, else distribute
  place({ toSlot?: number, count?: number })
  // move from slot to another slot
  move({ from?: string, fromSlot?: number, to?: string, toSlot?: number })
  // from item from one place to another place (specific slot if specified)
  move({ from?: string, item?: number, to?: string, toSlot?: number })
}

const tx = bot.inventory.startTransaction()
tx.take({ fromSlot: 0, count }) // put into hand
tx.place({ toSlot: 0, count }) // put from hand into slot
tx.swap({ fromSlot: 0, toSlot: 1 })
tx.move({ from: 'chest', fromSlot: 2, to: 'hand' })
tx.send()

There some duplicates APIs which do the same thing, I just added them from the bedrock protocol. For automation in agents, the action space can be simplified to just moving x item to y.

[u9g]

prismarine-item pr has been merged

[RUGMJ]

Any updates?

[extremeheat]

Remaining work backlog

• prismarine-registry - implementation needed for protocol runtime ID mapping
• prismarine-item - needs to use the runtime ID mapping
• prismarine-chunk - move bedrock-provider chunk implementation to prismarine-chunk https://github.com/PrismarineJS/prismarine-chunk/issues/152

What's done

• https://github.com/PrismarineJS/bedrock-provider/pull/9 - support for 1.18 level format
• mc-data updated with most essential data, biomes, blocks, items, collisions, etc.
• prismarine-viewer kind of works, but needs some updates to block model mapping

Non-essential backlog

• https://github.com/PrismarineJS/prismarine-recipe/pull/9 - bedrock recipes

[neohunter]

https://github.com/PrismarineJS/mineflayer/discussions/2411

How to make mineflayer use bedrock-protocol instead of minecraft-protocol?

[rom1504]

by finishing this task, if you want to help @neohunter feel free to join #bedrock-protocol in discord and ask what you can do at this time to make things easier

[extremeheat]

June 2022

Remaining work backlog

• prismarine-item - awaiting bedrock support impl
• prismarine-entity - awaiting bedrock support impl
• prismarine-physics - awaiting bedrock support impl
• prismarine-registry - support remapping from server item palette (in StartGamePacket), similar to how pc biome palette data is handled

What's done

• prismarine-chunk supports 1.18.0 & below
• minecraft-data, all relevant data for bedrock 1.18 & below 👏
• prismarine-block
• prismarine-biome, works out of box
• prismarine-registry / nmd supports bedrock

Non-essential backlog

• prismarine-chunk support for 1.18.30, 1.19.1 chunks
• https://github.com/PrismarineJS/prismarine-recipe/pull/9 - bedrock recipes, depends on bedrock prismarine-item to use & test
• prismarine-world
• prismarine-chat

Want to help?

• Get a good understanding of how the prismarine projects work currently, specifically mineflayer.
• Then take a look at the bedrock protocol documentation (on https://minecraft-data.prismarine.js.org)
• use a bedrock-protocol proxy (called Relay) over a vanilla server and client to understand how the packets are used in correlation to ingame activity
• Update packages in Remaining work backlog to implement in game features (the packages provide abstraction on topp of the game protocol)

[ATXLtheAxolotl]

What problems are there currently with prismarine-physics that prevents it from supporting bedrock?

[extremeheat]

What problems are there currently with prismarine-physics that prevents it from supporting bedrock?

Problems if any for physics will come up when making tests for it on bedrock to see what works / what needs changes

[extremeheat]

Some notes from discord discussion:

• prismarine-entity will likely need some changes to make it consistent across all versions (like effects, attributes, properties)
  • After looking at prismarine-physics/entity, one of the things we can do I think is make the effect/attribute/entity names consistent across all versions, that would simplify the code alot and remove alot of code like this: https://github.com/PrismarineJS/prismarine-physics/blob/708ff2341bbe8e542f9d29a82c9c102c33a02c02/index.js#L649
  • since effectNamesAreRegistryNames is only valid on pc1.17, mc-data can probably be updated to rename the effects to 1.19/1.16 naming. A protocolName field could be added to keep the actual protocol naming instead so name can be consistent across all pc versions / bedrock versions

Prismarine item will require loading item palette data from StartGame packet similar to pc dimension codec. Some work in https://github.com/PrismarineJS/prismarine-registry/pull/26#issuecomment-1452506834. Enchant data was added in PrismarineJS/Minecraft-data#689, so all the data for pitem should now be in place.

Additionally, prismarine window handling will also likely require some changes to pitem

[rom1504]

pitem bedrock is done

[minidogg]

is this done yet?

[rom1504]

I think it would be great to get this one moving.

I wonder if we could do a minimal implementation in mineflayer with some basic stuff working (chat) and have the rest not work, and go from there

I think that may be a better approach than blocking on "be on par with pc"