bcachefs device add seems to scale default bucket size by total volume size

[kode54]

I have created two qcow2 volumes roughly sized according to my two large hard drives:

qemu-img create -f qcow2 bcache-1.qcow2 18000G
qemu-img create -f qcow2 bcache-2.qcow2 18000G

and mounted them with qemu-nbd:

sudo modprobe nbd
sudo qemu-nbd -c /dev/nbd0 bcache-1.qcow2
sudo qemu-nbd -c /dev/nbd1 bcache-2.qcow2

If I format them together:

sudo bcachefs format /dev/nbd0 /dev/nbd1

Then they both end up with 512 KiB bucket size, like my first formatted drive:

/dev/nbd0 contains a bcachefs filesystem
Proceed anyway? (y,n) y
/dev/nbd1 contains a bcachefs filesystem
Proceed anyway? (y,n) y
External UUID:                              da5b2aa7-0c5c-4aa7-a4c2-240f43d87932
Internal UUID:                              64cda1f8-319d-4682-a608-3a9a7928c729
Magic number:                               c68573f6-66ce-90a9-d96a-60cf803df7ef
Device index:                               1
Label:
Version:                                    1.4: member_seq
Version upgrade complete:                   0.0: (unknown version)
Oldest version on disk:                     1.4: member_seq
Created:                                    Mon Jan 15 12:57:22 2024
Sequence number:                            0
Time of last write:                         Wed Dec 31 16:00:00 1969
Superblock size:                            1144
Clean:                                      0
Devices:                                    2
Sections:                                   members_v1,members_v2
Features:                                   new_siphash,new_extent_overwrite,btree_ptr_v2,extents_above_btree_updates,btree_updates_journalled,new_varint,journal_no_flush,alloc_v2,extents_across_btree_nodes
Compat features:

Options:
  block_size:                               512 B
  btree_node_size:                          256 KiB
  errors:                                   continue [ro] panic
  metadata_replicas:                        1
  data_replicas:                            1
  metadata_replicas_required:               1
  data_replicas_required:                   1
  encoded_extent_max:                       64.0 KiB
  metadata_checksum:                        none [crc32c] crc64 xxhash
  data_checksum:                            none [crc32c] crc64 xxhash
  compression:                              none
  background_compression:                   none
  str_hash:                                 crc32c crc64 [siphash]
  metadata_target:                          none
  foreground_target:                        none
  background_target:                        none
  promote_target:                           none
  erasure_code:                             0
  inodes_32bit:                             1
  shard_inode_numbers:                      1
  inodes_use_key_cache:                     1
  gc_reserve_percent:                       8
  gc_reserve_bytes:                         0 B
  root_reserve_percent:                     0
  wide_macs:                                0
  acl:                                      1
  usrquota:                                 0
  grpquota:                                 0
  prjquota:                                 0
  journal_flush_delay:                      1000
  journal_flush_disabled:                   0
  journal_reclaim_delay:                    100
  journal_transaction_names:                1
  version_upgrade:                          [compatible] incompatible none
  nocow:                                    0

members_v2 (size 272):
Device:                                     0
  Label:                                    (none)
  UUID:                                     9cc991d7-2462-4457-b0f8-bcca9d841e64
  Size:                                     17.6 TiB
  read errors:                              0
  write errors:                             0
  checksum errors:                          0
  seqread iops:                             0
  seqwrite iops:                            0
  randread iops:                            0
  randwrite iops:                           0
  Bucket size:                              512 KiB
  First bucket:                             0
  Buckets:                                  36864000
  Last mount:                               (never)
  Last superblock write:                    0
  State:                                    rw
  Data allowed:                             journal,btree,user
  Has data:                                 (none)
  Durability:                               1
  Discard:                                  0
  Freespace initialized:                    0
Device:                                     1
  Label:                                    (none)
  UUID:                                     ccac6898-821d-4597-ab7d-7895fcebd5c0
  Size:                                     17.6 TiB
  read errors:                              0
  write errors:                             0
  checksum errors:                          0
  seqread iops:                             0
  seqwrite iops:                            0
  randread iops:                            0
  randwrite iops:                           0
  Bucket size:                              512 KiB
  First bucket:                             0
  Buckets:                                  36864000
  Last mount:                               (never)
  Last superblock write:                    0
  State:                                    rw
  Data allowed:                             journal,btree,user
  Has data:                                 (none)
  Durability:                               1
  Discard:                                  0
  Freespace initialized:                    0
mounting version 1.4: member_seq
initializing new filesystem
going read-write
initializing freespace

If I format one, then add the second device (later) using device add, then the second drive ends up with 1024 KiB bucket size:

sudo bcachefs format /dev/nbd0
sudo mkdir /tmp/bcache
sudo mount /dev/nbd0 /tmp/bcache
sudo bcachefs device add /tmp/bcache /dev/nbd1

> sudo bcachefs fs usage /tmp/bcache
Filesystem: efdc2be6-9027-4a7c-8847-a71701f0f69c
Size:                 35562329210880
Used:                    12897222656
Online reserved:                   0

Data type       Required/total  Durability    Devices
btree:          1/1             1             [nbd0]               4456448

(no label) (device 0):          nbd0              rw
                                data         buckets    fragmented
  free:               19323047903232        36855789
  sb:                        3149824               7        520192
  journal:                4294967296            8192
  btree:                     4456448              12       1835008
  user:                            0               0
  cached:                          0               0
  parity:                          0               0
  stripe:                          0               0
  need_gc_gens:                    0               0
  need_discard:                    0               0
  capacity:           19327352832000        36864000

(no label) (device 1):          nbd1              rw
                                data         buckets    fragmented
  free:               19318758703104        18423804
  sb:                        3149824               4       1044480
  journal:                8589934592            8192
  btree:                           0               0
  user:                            0               0
  cached:                          0               0
  parity:                          0               0
  stripe:                          0               0
  need_gc_gens:                    0               0
  need_discard:                    0               0
  capacity:           19327352832000        18432000