airmail icon indicating copy to clipboard operation
airmail copied to clipboard

double free or corruption (fasttop) when running the indexer

Open raphael10-collab opened this issue 9 months ago • 7 comments

root@WorldMap:~/airmail# cargo run --bin indexer -- \
> --wof-db data/whosonfirst-data-admin-latest.db \
> --index data/index/ \
> load-osm-pbf data/planet-daily.osm.pbf
    Finished dev [unoptimized + debuginfo] target(s) in 3.77s
     Running `target/debug/indexer --wof-db data/whosonfirst-data-admin-latest.db --index data/index/ load-osm-pbf data/planet-daily.osm.pbf`
[2025-03-17T16:08:06Z INFO  indexer::osm_pbf] Generating OSM node map from: data/planet-daily.osm.pbf
[2025-03-31T12:05:33Z INFO  indexer::osm_pbf] 9737670035 node locations are cached
[2025-03-31T12:05:33Z INFO  indexer::osm_pbf] Parsing POIs
[2025-03-31T12:05:33Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
[2025-03-31T12:05:33Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
[2025-03-31T12:05:33Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
[2025-03-31T12:05:33Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
[2025-03-31T12:05:33Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
[2025-03-31T12:05:33Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
[2025-03-31T12:05:33Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
[2025-03-31T12:05:33Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
[2025-03-31T12:05:33Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
[2025-03-31T12:05:34Z WARN  airmail_indexer::importer] Failed to populate admin areas, error returned from database: (code: 1) no such table: main.point_in_polygon
double free or corruption (fasttop)
double free or corruption (fasttop)
Aborted (core dumped)

Other info:

OS: Ubuntu 24.04

root@WorldMap:~/airmail# df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           2.6G 1004K  2.6G   1% /run
/dev/sda1       3.1T  691G  2.4T  23% /
tmpfs            13G     0   13G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            13G     0   13G   0% /run/qemu
/dev/sda16      881M  112M  707M  14% /boot
/dev/sda15      105M  6.1M   99M   6% /boot/efi
tmpfs           2.6G   12K  2.6G   1% /run/user/0


root@WorldMap:~/airmail# free -m
               total        used        free      shared  buff/cache   available
Mem:           26047         652       25146           0         636       25395
Swap:              0           0           0

@ellenhp what can I do?

Looking forward to any hints

raphael10-collab avatar Apr 04 '25 09:04 raphael10-collab

I'm not sure what's causing the double-free but it looks like you downloaded the wrong Who's On First database. The one you need is the SpatiaLite database at the bottom of the page. https://geocode.earth/data/whosonfirst/combined/

That explains the "no such table" errors.

ellenhp avatar Apr 04 '25 15:04 ellenhp

I downloaed the first SpatiaLite database at the bottom of the page, the one with administrative boundaries https://data.geocode.earth/wof/dist/spatial/whosonfirst-data-admin-latest.spatial.db.bz2 ,

root@WorldMap:~/airmail/data# ls -lah
total 94G
drwxr-xr-x  3 root root 4.0K Apr  6 11:40 .
drwxr-xr-x 10 root root 4.0K Feb 25 07:21 ..
drwxr-xr-x  2 root root 4.0K Mar  5 22:14 index
-rw-r--r--  1 root root  83G Feb 24 21:10 planet-daily.osm.pbf
-rw-r--r--  1 root root  12G Mar 27 16:01 whosonfirst-data-admin-latest.spatial.db.bz2

I repeat the indexer step. I will let you know about the end result. Thank you

raphael10-collab avatar Apr 06 '25 09:04 raphael10-collab

Great, let me know. You got the right database this time though so I'm hoping it'll work. I'm a little bit upset about the double-free, that's not an acceptable failure mode for software like this, and I don't know what could be causing it. 🤔

ellenhp avatar Apr 06 '25 18:04 ellenhp

Hi @ellenhp

after 3 weeks this is the output :

root@WorldMap:~/airmail# cargo run --bin indexer -- --wof-db data/whosonfirst-data-admin-latest.spatial.db --index data/index/ load-osm-pbf data/planet-daily.osm.pbf

[2025-04-26T18:51:25Z INFO  airmail_indexer::importer] 90130000 POIs parsed in 1705765 seconds, 52.838450920454754 per second.
[2025-04-26T18:52:13Z INFO  airmail_indexer::importer] 90140000 POIs parsed in 1705812 seconds, 52.84284845368677 per second.
[2025-04-26T18:52:58Z INFO  airmail_indexer::importer] 90150000 POIs parsed in 1705857 seconds, 52.84731256553397 per second.
[2025-04-26T18:53:35Z INFO  airmail_indexer::importer] 90160000 POIs parsed in 1705895 seconds, 52.852015269781774 per second.
[2025-04-26T18:54:18Z INFO  airmail_indexer::importer] 90170000 POIs parsed in 1705938 seconds, 52.8565367218204 per second.
[2025-04-26T18:54:59Z INFO  airmail_indexer::importer] 90180000 POIs parsed in 1705979 seconds, 52.86113726487826 per second.
[2025-04-26T18:55:41Z INFO  airmail_indexer::importer] 90190000 POIs parsed in 1706020 seconds, 52.865708805850794 per second.
[2025-04-26T18:56:21Z INFO  airmail_indexer::importer] 90200000 POIs parsed in 1706060 seconds, 52.870333479421134 per second.
[2025-04-26T18:57:00Z INFO  airmail_indexer::importer] 90210000 POIs parsed in 1706100 seconds, 52.87497324323473 per second.
[2025-04-26T18:57:49Z INFO  airmail_indexer::importer] 90220000 POIs parsed in 1706149 seconds, 52.87930126845185 per second.
[2025-04-26T18:58:25Z INFO  airmail_indexer::importer] 90230000 POIs parsed in 1706184 seconds, 52.88407413352758 per second.
[2025-04-26T18:59:01Z INFO  airmail_indexer::importer] 90240000 POIs parsed in 1706221 seconds, 52.888797628244696 per second.
[2025-04-26T18:59:36Z INFO  airmail_indexer::importer] 90250000 POIs parsed in 1706256 seconds, 52.89357562585835 per second.
[2025-04-26T19:00:11Z INFO  airmail_indexer::importer] 90260000 POIs parsed in 1706291 seconds, 52.898361859633695 per second.
[2025-04-26T19:00:49Z INFO  airmail_indexer::importer] 90270000 POIs parsed in 1706328 seconds, 52.9030506205524 per second.
[2025-04-26T19:01:22Z INFO  airmail_indexer::importer] 90280000 POIs parsed in 1706362 seconds, 52.90787714657196 per second.
[2025-04-26T19:01:55Z INFO  airmail_indexer::importer] 90290000 POIs parsed in 1706395 seconds, 52.912704876165904 per second.
[2025-04-26T19:02:28Z INFO  airmail_indexer::importer] 90300000 POIs parsed in 1706427 seconds, 52.91756553698958 per second.
[2025-04-26T19:03:05Z INFO  airmail_indexer::importer] 90310000 POIs parsed in 1706464 seconds, 52.92227519971143 per second.
[2025-04-26T19:03:41Z INFO  airmail_indexer::importer] 90320000 POIs parsed in 1706501 seconds, 52.92699477958372 per second.
[2025-04-26T19:04:27Z INFO  airmail_indexer::importer] 90330000 POIs parsed in 1706546 seconds, 52.93144949244868 per second.
[2025-04-26T19:04:58Z INFO  airmail_indexer::importer] 90340000 POIs parsed in 1706578 seconds, 52.93633301254568 per second.
[2025-04-26T19:05:40Z INFO  airmail_indexer::importer] 90350000 POIs parsed in 1706619 seconds, 52.94091094032554 per second.
[2025-04-26T19:06:15Z INFO  airmail_indexer::importer] 90360000 POIs parsed in 1706655 seconds, 52.94566424564821 per second.
[2025-04-26T19:06:52Z INFO  airmail_indexer::importer] 90370000 POIs parsed in 1706691 seconds, 52.950387558667515 per second.
[2025-04-26T19:07:32Z INFO  airmail_indexer::importer] 90380000 POIs parsed in 1706732 seconds, 52.95499635409539 per second.
[2025-04-26T19:08:05Z INFO  airmail_indexer::importer] 90390000 POIs parsed in 1706765 seconds, 52.959822288103936 per second.
[2025-04-26T19:08:37Z INFO  airmail_indexer::importer] 90400000 POIs parsed in 1706796 seconds, 52.96471101007854 per second.
[2025-04-26T19:09:04Z INFO  tantivy::indexer::index_writer] Buffer limit reached, flushing segment with maxdoc=152897.
[2025-04-26T19:09:16Z INFO  airmail_indexer::importer] 90410000 POIs parsed in 1706836 seconds, 52.96934895144078 per second.
[2025-04-26T19:09:54Z INFO  airmail_indexer::importer] 90420000 POIs parsed in 1706873 seconds, 52.97403747783768 per second.
[2025-04-26T19:10:31Z INFO  airmail_indexer::importer] 90430000 POIs parsed in 1706910 seconds, 52.97875357744874 per second.
[2025-04-26T19:10:58Z INFO  airmail_indexer::importer] 90440000 POIs parsed in 1706938 seconds, 52.983753786459964 per second.
[2025-04-26T19:11:39Z INFO  airmail_indexer::importer] 90450000 POIs parsed in 1706979 seconds, 52.98834515264004 per second.
[2025-04-26T19:12:12Z INFO  airmail_indexer::importer] 90460000 POIs parsed in 1707012 seconds, 52.993166439893685 per second.
[2025-04-26T19:12:44Z INFO  airmail_indexer::importer] 90470000 POIs parsed in 1707043 seconds, 52.99805142883914 per second.
[2025-04-26T19:13:15Z INFO  airmail_indexer::importer] 90480000 POIs parsed in 1707075 seconds, 53.00293253998744 per second.
[2025-04-26T19:13:50Z INFO  airmail_indexer::importer] 90490000 POIs parsed in 1707109 seconds, 53.00771497689868 per second.
[2025-04-26T19:14:23Z INFO  airmail_indexer::importer] 90500000 POIs parsed in 1707143 seconds, 53.01253409669436 per second.
[2025-04-26T19:14:58Z INFO  airmail_indexer::importer] 90510000 POIs parsed in 1707178 seconds, 53.0173028856781 per second.
[2025-04-26T19:15:08Z WARN  sqlx::query] slow statement: execution time exceeded alert threshold summary="SELECT place.source, place.id, place.class, …" db.statement="\n\nSELECT\n  place.source,\n  place.id,\n  place.class,\n  place.type\nFROM\n  main.point_in_polygon\n  LEFT JOIN place USING (source, id)\nWHERE\n  search_frame = MakePoint(?1, ?2, 4326)\n  AND INTERSECTS(\n    point_in_polygon.geom,\n    MakePoint(?1, ?2, 4326)\n  )\n  AND place.source IS NOT NULL\n  AND (\n    place.type != 'planet'\n    AND place.type != 'marketarea'\n    AND place.type != 'county'\n    AND place.type != 'timezone'\n  )\nLIMIT\n  1000\n" rows_affected=0 rows_returned=13 elapsed=1.030085715s elapsed_secs=1.030085715 slow_threshold=1s
[2025-04-26T19:15:34Z INFO  airmail_indexer::importer] 90520000 POIs parsed in 1707213 seconds, 53.02206515066858 per second.
[2025-04-26T19:16:15Z INFO  airmail_indexer::importer] 90530000 POIs parsed in 1707254 seconds, 53.02665316520257 per second.
[2025-04-26T19:16:54Z INFO  airmail_indexer::importer] 90540000 POIs parsed in 1707293 seconds, 53.031290463274125 per second.
[2025-04-26T19:17:26Z INFO  airmail_indexer::importer] 90550000 POIs parsed in 1707326 seconds, 53.036150158354836 per second.
[2025-04-26T19:18:01Z INFO  airmail_indexer::importer] 90560000 POIs parsed in 1707360 seconds, 53.04092609948414 per second.
[2025-04-26T19:18:39Z INFO  airmail_indexer::importer] 90570000 POIs parsed in 1707399 seconds, 53.04558603661568 per second.
[2025-04-26T19:19:16Z INFO  airmail_indexer::importer] 90580000 POIs parsed in 1707436 seconds, 53.05029053315556 per second.
[2025-04-26T19:19:53Z INFO  airmail_indexer::importer] 90590000 POIs parsed in 1707472 seconds, 53.055019645108175 per second.
[2025-04-26T19:20:33Z INFO  airmail_indexer::importer] 90600000 POIs parsed in 1707512 seconds, 53.05963601039173 per second.
[2025-04-26T19:21:13Z INFO  airmail_indexer::importer] 90610000 POIs parsed in 1707553 seconds, 53.06423529240592 per second.
[2025-04-26T19:21:53Z WARN  sqlx::query] slow statement: execution time exceeded alert threshold summary="SELECT place.source, place.id, place.class, …" db.statement="\n\nSELECT\n  place.source,\n  place.id,\n  place.class,\n  place.type\nFROM\n  main.point_in_polygon\n  LEFT JOIN place USING (source, id)\nWHERE\n  search_frame = MakePoint(?1, ?2, 4326)\n  AND INTERSECTS(\n    point_in_polygon.geom,\n    MakePoint(?1, ?2, 4326)\n  )\n  AND place.source IS NOT NULL\n  AND (\n    place.type != 'planet'\n    AND place.type != 'marketarea'\n    AND place.type != 'county'\n    AND place.type != 'timezone'\n  )\nLIMIT\n  1000\n" rows_affected=0 rows_returned=5 elapsed=1.365321782s elapsed_secs=1.365321782 slow_threshold=1s
[2025-04-26T19:21:53Z WARN  sqlx::query] slow statement: execution time exceeded alert threshold summary="SELECT place.source, place.id, place.class, …" db.statement="\n\nSELECT\n  place.source,\n  place.id,\n  place.class,\n  place.type\nFROM\n  main.point_in_polygon\n  LEFT JOIN place USING (source, id)\nWHERE\n  search_frame = MakePoint(?1, ?2, 4326)\n  AND INTERSECTS(\n    point_in_polygon.geom,\n    MakePoint(?1, ?2, 4326)\n  )\n  AND place.source IS NOT NULL\n  AND (\n    place.type != 'planet'\n    AND place.type != 'marketarea'\n    AND place.type != 'county'\n    AND place.type != 'timezone'\n  )\nLIMIT\n  1000\n" rows_affected=0 rows_returned=13 elapsed=1.468240744s elapsed_secs=1.468240744 slow_threshold=1s
[2025-04-26T19:21:53Z WARN  sqlx::query] slow statement: execution time exceeded alert threshold summary="SELECT place.source, place.id, place.class, …" db.statement="\n\nSELECT\n  place.source,\n  place.id,\n  place.class,\n  place.type\nFROM\n  main.point_in_polygon\n  LEFT JOIN place USING (source, id)\nWHERE\n  search_frame = MakePoint(?1, ?2, 4326)\n  AND INTERSECTS(\n    point_in_polygon.geom,\n    MakePoint(?1, ?2, 4326)\n  )\n  AND place.source IS NOT NULL\n  AND (\n    place.type != 'planet'\n    AND place.type != 'marketarea'\n    AND place.type != 'county'\n    AND place.type != 'timezone'\n  )\nLIMIT\n  1000\n" rows_affected=0 rows_returned=7 elapsed=1.468630777s elapsed_secs=1.468630777 slow_threshold=1s
Killed

Why "Killed" ? I didn't do anything to kill the process

raphael10-collab avatar Apr 28 '25 06:04 raphael10-collab

Usually this indicates an out of memory condition. 50 POIs per second is also extremely slow. What hardware is this running on?

ellenhp avatar Apr 29 '25 00:04 ellenhp

@ellenhp which hardware problems do you see in my Contabo 10 Core Storage VPS?

https://contabo.com/en/storage-vps/storage-vps-10c/?image=ubuntu.332&qty=1&contract=12

Image

root@WorldMap:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           2.6G 1004K  2.6G   1% /run
/dev/sda1       3.1T  717G  2.4T  24% /
tmpfs            13G     0   13G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs            13G     0   13G   0% /run/qemu
/dev/sda16      881M  112M  707M  14% /boot
/dev/sda15      105M  6.1M   99M   6% /boot/efi
tmpfs           2.6G   12K  2.6G   1% /run/user/0




root@WorldMap:~# free -m
               total        used        free      shared  buff/cache   available
Mem:           26047         606       25292           0         490       25440
Swap:              0           0           0







root@WorldMap:~# lspci -vv
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
	Subsystem: Red Hat, Inc. Qemu virtual machine
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
	Subsystem: Red Hat, Inc. Qemu virtual machine
	Physical Slot: 1
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-

00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II] (prog-if 80 [ISA Compatibility mode-only controller, supports bus mastering])
	Subsystem: Red Hat, Inc. Qemu virtual machine
	Physical Slot: 1
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Region 0: I/O ports at 01f0 [size=8]
	Region 1: I/O ports at 03f4
	Region 2: I/O ports at 0170 [size=8]
	Region 3: I/O ports at 0374
	Region 4: I/O ports at e0c0 [size=16]
	Kernel driver in use: ata_piix

00:01.2 USB controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (rev 01) (prog-if 00 [UHCI])
	Subsystem: Red Hat, Inc. QEMU Virtual Machine
	Physical Slot: 1
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin D routed to IRQ 11
	Region 4: I/O ports at e080 [size=32]
	Kernel driver in use: uhci_hcd

00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
	Subsystem: Red Hat, Inc. Qemu virtual machine
	Physical Slot: 1
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 9

00:02.0 VGA compatible controller: Device 1234:1111 (rev 02) (prog-if 00 [VGA controller])
	Subsystem: Red Hat, Inc. Device 1100
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Region 0: Memory at fd000000 (32-bit, prefetchable) [size=16M]
	Region 2: Memory at fea50000 (32-bit, non-prefetchable) [size=4K]
	Expansion ROM at 000c0000 [disabled] [size=128K]

00:03.0 Unclassified device [00ff]: Red Hat, Inc. Virtio memory balloon
	Subsystem: Red Hat, Inc. Virtio memory balloon
	Physical Slot: 3
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 10
	Region 0: I/O ports at e000 [size=64]
	Region 4: Memory at 381800000000 (64-bit, prefetchable) [size=16K]
	Capabilities: [84] Vendor Specific Information: VirtIO: <unknown>
		BAR=0 offset=00000000 size=00000000
	Capabilities: [70] Vendor Specific Information: VirtIO: Notify
		BAR=4 offset=00003000 size=00001000 multiplier=00000004
	Capabilities: [60] Vendor Specific Information: VirtIO: DeviceCfg
		BAR=4 offset=00002000 size=00001000
	Capabilities: [50] Vendor Specific Information: VirtIO: ISR
		BAR=4 offset=00001000 size=00001000
	Capabilities: [40] Vendor Specific Information: VirtIO: CommonCfg
		BAR=4 offset=00000000 size=00001000
	Kernel driver in use: virtio-pci

00:05.0 SCSI storage controller: Red Hat, Inc. Virtio SCSI
	Subsystem: Red Hat, Inc. Virtio SCSI
	Physical Slot: 5
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 10
	Region 0: I/O ports at e040 [size=64]
	Region 1: Memory at fea51000 (32-bit, non-prefetchable) [size=4K]
	Region 4: Memory at 381800004000 (64-bit, prefetchable) [size=16K]
	Capabilities: [98] MSI-X: Enable+ Count=13 Masked-
		Vector table: BAR=1 offset=00000000
		PBA: BAR=1 offset=00000800
	Capabilities: [84] Vendor Specific Information: VirtIO: <unknown>
		BAR=4 offset=00001000 size=00000001
	Capabilities: [70] Vendor Specific Information: VirtIO: Notify
		BAR=4 offset=00003000 size=00001000 multiplier=00000004
	Capabilities: [60] Vendor Specific Information: VirtIO: DeviceCfg
		BAR=4 offset=00002000 size=00001000
	Capabilities: [50] Vendor Specific Information: VirtIO: ISR
		BAR=4 offset=00001000 size=00001000
	Capabilities: [40] Vendor Specific Information: VirtIO: CommonCfg
		BAR=4 offset=00000000 size=00001000
	Kernel driver in use: virtio-pci

00:12.0 Ethernet controller: Red Hat, Inc. Virtio network device
	Subsystem: Red Hat, Inc. Virtio network device
	Physical Slot: 18
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 11
	Region 0: I/O ports at e0a0 [size=32]
	Region 1: Memory at fea52000 (32-bit, non-prefetchable) [size=4K]
	Region 4: Memory at 381800008000 (64-bit, prefetchable) [size=16K]
	Expansion ROM at fea00000 [disabled] [size=256K]
	Capabilities: [98] MSI-X: Enable+ Count=4 Masked-
		Vector table: BAR=1 offset=00000000
		PBA: BAR=1 offset=00000800
	Capabilities: [84] Vendor Specific Information: VirtIO: <unknown>
		BAR=4 offset=00000014 size=00000001
	Capabilities: [70] Vendor Specific Information: VirtIO: Notify
		BAR=4 offset=00003000 size=00001000 multiplier=00000004
	Capabilities: [60] Vendor Specific Information: VirtIO: DeviceCfg
		BAR=4 offset=00002000 size=00001000
	Capabilities: [50] Vendor Specific Information: VirtIO: ISR
		BAR=4 offset=00001000 size=00001000
	Capabilities: [40] Vendor Specific Information: VirtIO: CommonCfg
		BAR=4 offset=00000000 size=00001000
	Kernel driver in use: virtio-pci

00:1e.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 10
	Region 0: Memory at 380000000000 (64-bit, non-prefetchable) [size=256]
	Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
	I/O behind bridge: d000-dfff [size=4K] [16-bit]
	Memory behind bridge: fe800000-fe9fffff [size=2M] [32-bit]
	Prefetchable memory behind bridge: 381000000000-3817ffffffff [size=32G] [32-bit]
	Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [4c] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [48] Slot ID: 0 slots, First+, chassis 01
	Capabilities: [40] Hot-plug capable

00:1f.0 PCI bridge: Red Hat, Inc. QEMU PCI-PCI bridge (prog-if 00 [Normal decode])
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Interrupt: pin A routed to IRQ 11
	Region 0: Memory at 380000001000 (64-bit, non-prefetchable) [size=256]
	Bus: primary=00, secondary=02, subordinate=02, sec-latency=0
	I/O behind bridge: c000-cfff [size=4K] [16-bit]
	Memory behind bridge: fe600000-fe7fffff [size=2M] [32-bit]
	Prefetchable memory behind bridge: 380800000000-380fffffffff [size=32G] [32-bit]
	Secondary status: 66MHz+ FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR-
	BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B-
		PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn-
	Capabilities: [4c] MSI: Enable- Count=1/1 Maskable+ 64bit+
		Address: 0000000000000000  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [48] Slot ID: 0 slots, First+, chassis 02
	Capabilities: [40] Hot-plug capable








root@WorldMap:~# lshw
worldmap                    
    description: Computer
    product: Standard PC (i440FX + PIIX, 1996)
    vendor: QEMU
    version: pc-i440fx-9.0
    width: 64 bits
    capabilities: smbios-2.8 dmi-2.8 smp vsyscall32
    configuration: boot=normal uuid=2920a889-5d66-4fd8-954f-3546ec25dde0
  *-core
       description: Motherboard
       physical id: 0
     *-firmware
          description: BIOS
          vendor: SeaBIOS
          physical id: 0
          version: rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org
          date: 04/01/2014
          size: 96KiB
     *-cpu
          description: CPU
          product: Intel Core Processor (Broadwell, no TSX, IBRS)
          vendor: Intel Corp.
          physical id: 400
          bus info: cpu@0
          version: 6.61.2
          slot: CPU 0
          size: 2GHz
          capacity: 2GHz
          width: 64 bits
          capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp x86-64 constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault pti ssbd ibrs ibpb stibp fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap xsaveopt arat md_clear
          configuration: cores=10 enabledcores=10 microcode=1 threads=10
     *-memory
          description: System Memory
          physical id: 1000
          size: 26GiB
          capabilities: ecc
          configuration: errordetection=multi-bit-ecc
        *-bank:0
             description: DIMM RAM
             vendor: QEMU
             physical id: 0
             slot: DIMM 0
             size: 16GiB
        *-bank:1
             description: DIMM RAM
             vendor: QEMU
             physical id: 1
             slot: DIMM 1
             size: 10GiB
     *-pci
          description: Host bridge
          product: 440FX - 82441FX PMC [Natoma]
          vendor: Intel Corporation
          physical id: 100
          bus info: pci@0000:00:00.0
          version: 02
          width: 32 bits
          clock: 33MHz
        *-isa
             description: ISA bridge
             product: 82371SB PIIX3 ISA [Natoma/Triton II]
             vendor: Intel Corporation
             physical id: 1
             bus info: pci@0000:00:01.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: isa
             configuration: latency=0
           *-pnp00:00
                product: PnP device PNP0501
                physical id: 0
                capabilities: pnp
                configuration: driver=serial
           *-pnp00:01
                product: PnP device PNP0303
                physical id: 1
                capabilities: pnp
                configuration: driver=i8042 kbd
           *-pnp00:02
                product: PnP device PNP0f13
                physical id: 2
                capabilities: pnp
                configuration: driver=i8042 aux
           *-pnp00:03
                product: PnP device PNP0700
                physical id: 3
                capabilities: pnp
           *-pnp00:04
                product: PnP device PNP0b00
                physical id: 4
                capabilities: pnp
                configuration: driver=rtc_cmos
        *-ide
             description: IDE interface
             product: 82371SB PIIX3 IDE [Natoma/Triton II]
             vendor: Intel Corporation
             physical id: 1.1
             bus info: pci@0000:00:01.1
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: ide isa_compat_mode bus_master
             configuration: driver=ata_piix latency=0
             resources: irq:0 ioport:1f0(size=8) ioport:3f6 ioport:170(size=8) ioport:376 ioport:e0c0(size=16)
        *-usb
             description: USB controller
             product: 82371SB PIIX3 USB [Natoma/Triton II]
             vendor: Intel Corporation
             physical id: 1.2
             bus info: pci@0000:00:01.2
             version: 01
             width: 32 bits
             clock: 33MHz
             capabilities: uhci bus_master
             configuration: driver=uhci_hcd latency=0
             resources: irq:11 ioport:e080(size=32)
           *-usbhost
                product: UHCI Host Controller
                vendor: Linux 6.8.0-55-generic uhci_hcd
                physical id: 1
                bus info: usb@1
                logical name: usb1
                version: 6.08
                capabilities: usb-1.10
                configuration: driver=hub slots=2 speed=12Mbit/s
        *-bridge UNCLAIMED
             description: Bridge
             product: 82371AB/EB/MB PIIX4 ACPI
             vendor: Intel Corporation
             physical id: 1.3
             bus info: pci@0000:00:01.3
             version: 03
             width: 32 bits
             clock: 33MHz
             capabilities: bridge
             configuration: latency=0
        *-display UNCLAIMED
             description: VGA compatible controller
             physical id: 2
             bus info: pci@0000:00:02.0
             version: 02
             width: 32 bits
             clock: 33MHz
             capabilities: vga_controller
             configuration: latency=0
             resources: memory:fd000000-fdffffff memory:fea50000-fea50fff memory:c0000-dffff
        *-generic
             description: Unclassified device
             product: Virtio memory balloon
             vendor: Red Hat, Inc.
             physical id: 3
             bus info: pci@0000:00:03.0
             version: 00
             width: 64 bits
             clock: 33MHz
             capabilities: bus_master cap_list
             configuration: driver=virtio-pci latency=0
             resources: iomemory:38180-3817f irq:10 ioport:e000(size=64) memory:381800000000-381800003fff
           *-virtio0 UNCLAIMED
                description: Virtual I/O device
                physical id: 0
                bus info: virtio@0
                configuration: driver=virtio_balloon
        *-scsi
             description: SCSI storage controller
             product: Virtio SCSI
             vendor: Red Hat, Inc.
             physical id: 5
             bus info: pci@0000:00:05.0
             version: 00
             width: 64 bits
             clock: 33MHz
             capabilities: scsi msix bus_master cap_list
             configuration: driver=virtio-pci latency=0
             resources: iomemory:38180-3817f irq:10 ioport:e040(size=64) memory:fea51000-fea51fff memory:381800004000-381800007fff
           *-virtio1
                description: Virtual I/O device
                physical id: 0
                bus info: virtio@1
                logical name: scsi0
                configuration: driver=virtio_scsi
              *-disk
                   description: SCSI Disk
                   product: QEMU HARDDISK
                   vendor: QEMU
                   physical id: 0.0.0
                   bus info: scsi@0:0.0.0
                   logical name: /dev/sda
                   version: 2.5+
                   size: 3200GiB (3435GB)
                   capabilities: 5400rpm gpt-1.00 partitioned partitioned:gpt
                   configuration: ansiversion=5 guid=2cc9efa8-f404-47d6-804f-06b18d2960cc logicalsectorsize=512 sectorsize=512
                 *-volume:0
                      description: EXT4 volume
                      vendor: Linux
                      physical id: 1
                      bus info: scsi@0:0.0.0,1
                      logical name: /dev/sda1
                      logical name: /
                      version: 1.0
                      serial: 314dc64b-478e-4831-8f35-5640abff7e09
                      size: 3198GiB
                      capacity: 3198GiB
                      capabilities: journaled extended_attributes large_files huge_files dir_nlink recover 64bit extents ext4 ext2 initialized
                      configuration: created=2025-01-15 09:43:16 filesystem=ext4 label=cloudimg-rootfs lastmountpoint=/ modified=2025-04-29 14:23:59 mount.fstype=ext4 mount.options=rw,relatime,discard,errors=remount-ro,commit=30 mounted=2025-03-17 16:43:04 state=mounted
                 *-volume:1
                      description: BIOS Boot partition
                      vendor: EFI
                      physical id: e
                      bus info: scsi@0:0.0.0,14
                      logical name: /dev/sda14
                      serial: 9f38750f-3d5a-4c48-9a0b-dd486ba769ce
                      capacity: 4095KiB
                      capabilities: nofs
                 *-volume:2
                      description: Windows FAT volume
                      vendor: mkfs.fat
                      physical id: f
                      bus info: scsi@0:0.0.0,15
                      logical name: /dev/sda15
                      logical name: /boot/efi
                      version: FAT32
                      serial: 53cc-adda
                      size: 100MiB
                      capacity: 105MiB
                      capabilities: boot fat initialized
                      configuration: FATs=2 filesystem=fat label=UEFI mount.fstype=vfat mount.options=rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro state=mounted
                 *-volume:3
                      description: EXT4 volume
                      vendor: Linux
                      physical id: 10
                      bus info: scsi@0:0.0.0,16
                      logical name: /dev/sda16
                      logical name: /boot
                      version: 1.0
                      serial: eb7ca90e-1f45-4605-a740-d47fe1f8e5b3
                      size: 913MiB
                      capacity: 913MiB
                      capabilities: journaled extended_attributes large_files huge_files dir_nlink recover 64bit extents ext4 ext2 initialized
                      configuration: created=2025-01-15 09:43:40 filesystem=ext4 label=BOOT lastmountpoint=/boot modified=2025-04-24 06:26:08 mount.fstype=ext4 mount.options=rw,relatime mounted=2025-03-17 16:43:05 state=mounted
        *-network
             description: Ethernet controller
             product: Virtio network device
             vendor: Red Hat, Inc.
             physical id: 12
             bus info: pci@0000:00:12.0
             version: 00
             width: 64 bits
             clock: 33MHz
             capabilities: msix bus_master cap_list rom
             configuration: driver=virtio-pci latency=0
             resources: iomemory:38180-3817f irq:11 ioport:e0a0(size=32) memory:fea52000-fea52fff memory:381800008000-38180000bfff memory:fea00000-fea3ffff
           *-virtio2
                description: Ethernet interface
                physical id: 0
                bus info: virtio@2
                logical name: eth0
                serial: 00:50:56:59:05:e0
                capabilities: ethernet physical logical
                configuration: autonegotiation=off broadcast=yes driver=virtio_net driverversion=1.0.0 ip=173.249.4.143 link=yes multicast=yes
        *-pci:0
             description: PCI bridge
             product: QEMU PCI-PCI bridge
             vendor: Red Hat, Inc.
             physical id: 1e
             bus info: pci@0000:00:1e.0
             version: 00
             slot: 0, chassis 1
             width: 64 bits
             clock: 66MHz
             capabilities: pci msi slotid normal_decode cap_list
             resources: iomemory:38000-37fff memory:380000000000-3800000000ff ioport:d000(size=4096) memory:fe800000-fe9fffff ioport:381000000000(size=34359738368)
        *-pci:1
             description: PCI bridge
             product: QEMU PCI-PCI bridge
             vendor: Red Hat, Inc.
             physical id: 1f
             bus info: pci@0000:00:1f.0
             version: 00
             slot: 0, chassis 2
             width: 64 bits
             clock: 66MHz
             capabilities: pci msi slotid normal_decode cap_list
             resources: iomemory:38000-37fff memory:380000001000-3800000010ff ioport:c000(size=4096) memory:fe600000-fe7fffff ioport:380800000000(size=34359738368)
  *-input:0
       product: Power Button
       physical id: 1
       logical name: input0
       logical name: /dev/input/event0
       capabilities: platform
  *-input:1
       product: AT Translated Set 2 keyboard
       physical id: 2
       logical name: input1
       logical name: /dev/input/event1
       logical name: input1::capslock
       logical name: input1::numlock
       logical name: input1::scrolllock
       capabilities: i8042
  *-input:2
       product: VirtualPS/2 VMware VMMouse
       physical id: 3
       logical name: input3
       logical name: /dev/input/event3
       logical name: /dev/input/mouse1
       capabilities: i8042
  *-input:3
       product: VirtualPS/2 VMware VMMouse
       physical id: 4
       logical name: input4
       logical name: /dev/input/event2
       logical name: /dev/input/mouse0
       capabilities: i8042
root@WorldMap:~# 

raphael10-collab avatar Apr 29 '25 13:04 raphael10-collab

@ellenhp

based on these hardware settings : https://github.com/ellenhp/airmail/issues/34#issuecomment-2838773710

How to make it work?

raphael10-collab avatar May 03 '25 09:05 raphael10-collab