trezor-firmware
trezor-firmware copied to clipboard
trezor
Firmware.elf analysis tool
Connected with https://github.com/trezor/trezor-firmware/issues/1947:
- creating a tool for analyzing
firmware.elfand extracting useful information about the sizes of all symbols
Is using bloaty under the hood, which I found better than nm. However, the tool is independent of the mechanism from which it gets the raw data.
Transforms the symbols into a human-readable format and also adds a location where the symbol is defined:
fun_data_apps_monero_xmr_bulletproof__lt_module_gt__BulletProofPlusBuilder__prove_batch_main
secp256k1_ecmult_gen_prec_table
trezor_lib::protobuf::encode::Encoder::encode_field::he48bd51ac180cbd5
becomes
src/apps/monero/xmr/bulletproof.py:2444 BulletProofPlusBuilder._prove_batch_main()
vendor/secp256k1-zkp/src/precomputed_ecmult_gen.c:14 secp256k1_ecmult_gen_prec_table
embed/rust/src/protobuf/encode.rs:95 Encoder::encode_field()
Thanks to this we can see how much space is taken by individual functions, files, modules, directories, etc.
It also allows for easy groupings of symbols, for example by each application we have, showing their size:
102_577: ethereum (2_486 symbols)
72_831: monero (1_650 symbols)
60_534: bitcoin (1_557 symbols)
35_966: cardano ( 924 symbols)
29_693: common ( 723 symbols)
27_881: webauthn ( 787 symbols)
20_187: management ( 512 symbols)
14_849: nem ( 408 symbols)
11_876: stellar ( 308 symbols)
9_179: eos ( 220 symbols)
8_442: tezos ( 201 symbols)
4_382: misc ( 101 symbols)
4_160: ripple ( 125 symbols)
3_742: zcash ( 136 symbols)
3_310: binance ( 87 symbols)
3_095: homescreen ( 82 symbols)
1_029: debug ( 26 symbols)
As for the structure of the new code, it is a bin_size package on its own, so it is isolated from all other tools and can be made as readable as possible. It also contains a bunch of tests, that are necessary here, as there is a lot of low-level logic (mostly string parsing and manipulation).
All the exposed objects can be found in core/tools/bin_size/__init__.py.
The main object is BinarySize, which puts together all other components. The example usage of it is:
BinarySize().load_file(
"firmware.elf", sections=(".flash", ".flash2")
).add_basic_info().aggregate().filter(
lambda row: row.size > 50 or row.language == "Rust"
).sort(
lambda row: row.size, reverse=True
).add_definitions(
lambda row: row.language != "C"
).show(
"report.txt"
)
- loading data from file, enriching the data, filtering them, sorting them, adding definitions and generating a report to a file
- the clients are allowed to control all its behavior with simple lambda functions operating on the row data (
DataRowclass) - it is also highly customizable, custom objects for data loading, row processing or final formatting can be supplied
There is a CLI tool - core/tools/size_firmware_elf.py - that allows to control these things through the command line. Example usage of that is python size_firmware_elf.py get --app ethereum --add-definitions
Another useful exposed object is StatisticsPlugin, which allows for example for the above-mentioned statistics about application size. Its example usage is (this generates the table above):
def _apps_categories(row: DataRow) -> str | None:
pattern = r"^src/apps/(\w+)/" # dir name after apps/
match = re.search(pattern, row.module_name)
if not match:
return None
else:
return match.group(1)
BS = (
BinarySize()
.load_file(BIN_TO_ANALYZE, sections=(".flash", ".flash2"))
.add_basic_info()
)
StatisticsPlugin(BS, _apps_categories).show()
- it is just about supplying a "grouping" function operating again on the
DataRowand returning a string describing the category (or None for no category)
This can be used for any grouping of the data and seeing how much space is each group taking. Another example is:
def _categories_func(row: DataRow) -> str | None:
if "ui" in row.definition_place.lower():
return "UI"
elif (
"crypto" in row.definition_place.lower()
or "vendor/secp256k1-zkp" in row.definition_place.lower()
):
return "Crypto"
elif "trezor" in row.definition_place.lower():
return "Trezor"
elif "micropython" in row.definition_place.lower():
return "Micropython env"
elif "bitcoin" in row.definition_place.lower():
return "Bitcoin"
elif "ethereum" in row.definition_place.lower():
return "Ethereum"
elif "apps/management" in row.definition_place.lower():
return "Management"
elif "src/apps/" in row.definition_place.lower():
return "Other apps"
elif row.language == "Rust":
return "Rust"
BS = (
BinarySize()
.load_file(BIN_TO_ANALYZE, sections=(".flash", ".flash2"))
.add_basic_info()
.aggregate()
.add_definitions()
)
StatisticsPlugin(BS, _categories_func).show(include_none=True)
which generates this data:
470_357: None ( 291 symbols)
389_909: Crypto (1_161 symbols)
232_737: Other apps (1_895 symbols)
207_791: Micropython env (1_405 symbols)
103_682: UI (1_036 symbols)
102_577: Ethereum ( 113 symbols)
60_534: Bitcoin ( 492 symbols)
58_498: Trezor ( 658 symbols)
20_187: Management ( 137 symbols)
8_080: Rust ( 45 symbols)
This PR is also adding a couple of checks into CI, that are monitoring the current state of the flash space:
- check how much space we have in
.flashand.flash2sections - if less than X, fail the job - check the comparison between the current binary and the
masterbinary - if we are bigger for more than X, fail the job (NOTE: this is not really done ATM, as the latestmasterfirmware.elfis not exposed as an artifact, so we are faking it - after merging this the artifact will start to be generated and the job results will be "valid") - printing all the differences between current and
master(again, using a mock master now) - generating a report with all the symbols in the current binary
These checks can be also run locally as core/tools/size_checker_firmware_flash.py and core/tools/size_compare_flash_master.py
NOTE for local usage:
- saving this data as
core/tools/bin_size/src/bin_size/DEFINITIONS_CACHE.jsonwill save you around 20 minutes on the first run withadd_definitions()
In 65d5559 I am taking advantage of a newly-discovered nm capability - it can show the exact source code definitions of most of the C symbols inside. In case of micropython, it shows the build location from frozen_mpy.c, which can be also useful. This info is available by nm --line-numbers --radix=dec --size-sort firmware.elf. Example output:
00036857 r fun_data_apps_ethereum_tokens__lt_module_gt__token_by_chain_address /home/jmusil/trezor-firmware/core/build/firmware/frozen_mpy.c:183006
00037084 T nist256p1 /home/jmusil/trezor-firmware/core/vendor/trezor-crypto/nist256p1.c:26
00037084 T secp256k1 /home/jmusil/trezor-firmware/core/vendor/trezor-crypto/secp256k1.c:26
00065536 R secp256k1_ecmult_gen_prec_table /home/jmusil/trezor-firmware/core/vendor/secp256k1-zkp/src/precomputed_ecmult_gen.c:14
It identified even some symbols that I could not find "by hand" - for example t_fl in vendor/trezor-crypto/aes/aestab.h:129.
Thanks to this we do not need to grep for symbol definitions in the source code, which was taking a lot of time at the first run (it is still needed for some symbols, but not for the majority).
It also means that we do not need to store all the C definitions in the cache file, as they will be always fetched freshly from the binary. (Thinking about it, the micropython line numbers can also be found out very quickly even without cache - so the only time-consuming-to-find definitions are Rust functions - but I hadn't checked for some ast-like tool for Rust)
It had a small effect on the group's statistics, as it corrected my inaccuracies in finding the symbol definitions "by hand" - it "removed" some symbols from Micropython group and "added" them in Crypto group.
In deed458 (indeed) I am adding core/tools/size_group_finder.py tool that can be quickly used to get the group statistics data.
The [section .flash] and [section .flash2] still remain a mystery to me.
When I increased the flash sizes core/embed/firmware/memory_T.ld and built firmware with UI2=1 - allowing .flash overflow - it also increased the [section .flash] and decreased the [section .flash2] - in tandem with the overall .flash and .flash2 sizes. That could mean these mysterious data are some "overhead" of storing the "real" data. It does not seem like it would pad with empty data as we discussed.
master
.flash: 758K / 768K
.flash2: 857K / 896K
189_281 [section .flash]
230_050 [section .flash2]
UI2
.flash: 798K / 768K
.flash2: 811K / 896K
192_424 [section .flash]
224_769 [section .flash2]
One hint that I could find at https://github.com/google/bloaty/blob/master/doc/how-bloaty-works.md is
In practice we usually can't achieve perfect 100% coverage. To compensate for this, we have various kinds of "fallback" labels we attach to mystery regions of the file. This is how we guarantee an important invariant of Bloaty: the totals given in Bloaty's output will always match the total size of the file. This ensures that we always account for the entire file, even if we don't have detailed information for every byte. - stating that it is just "some mysterious data"
And as a fun fact, nm does not show these [section .flash] at all, and the difference between overall sizes reported by bloaty and nm is almost exactly the size [section .flash] + [section .flash2]
Trying to modify micropython configurations in embed/firmware/mpconfigport.h had only a very little effect. Most of the things are already turned off (0) there, so there is very little decreasing potential.
I was only lucky in // optimisations section, changing MICROPY_OPT_COMPUTED_GOTO and MICROPY_OPT_MPZ_BITWISE to (0) saved around 5kb in binary, while the unittests were still OK.
For example turning off stuff in // compiler configuration or // Python internal features did not help at all (size-wise).
I tried to bring objdump to the rescue, but it did not help me much. Interesting look at the data is objdump -s firmware.elf, showing the raw data in all the locations, like:
8137ee0 0869735f 66697273 7400aa81 0769735f .is_first....is_
8137ef0 6c617374 00b48004 70726576 00c98105 last....prev....
8137f00 5f706167 6500f634 23747265 7a6f722f _page..4#trezor/
I could find exact byte-sizes of .flash and .flash2 there, but no hint about the [section .flash]. I tried to count the number of dots, zero bytes, or the 03 byte, which seems to delimit the symbols, but none of those fit the expected ~200 kB.
What could maybe bring insight is bloaty -d sections,symbols -v firmware.elf | grep "\[section \.flash2", outputting
[f0000, 126aae] LOAD #4 [R], .flash2, [section .flash2]
[126c8e, 126c90] LOAD #4 [R], .flash2, [section .flash2]
[196cb7, 196cb8] LOAD #4 [R], .flash2, [section .flash2]
[1a8744, 1a8a0d] LOAD #4 [R], .flash2, [section .flash2]
[1a8a3a, 1a8a3c] LOAD #4 [R], .flash2, [section .flash2]
[1a8bad, 1a8bb0] LOAD #4 [R], .flash2, [section .flash2]
[1a8d7e, 1a8d80] LOAD #4 [R], .flash2, [section .flash2]
[1bad80, 1bae00] LOAD #4 [R], .flash2, [section .flash2]
[8120000, 8156aae] LOAD #4 [R], .flash2, [section .flash2]
[8156c8e, 8156c90] LOAD #4 [R], .flash2, [section .flash2]
[81c6cb7, 81c6cb8] LOAD #4 [R], .flash2, [section .flash2]
[81d8744, 81d8a0d] LOAD #4 [R], .flash2, [section .flash2]
[81d8a3a, 81d8a3c] LOAD #4 [R], .flash2, [section .flash2]
[81d8bad, 81d8bb0] LOAD #4 [R], .flash2, [section .flash2]
[81d8d7e, 81d8d80] LOAD #4 [R], .flash2, [section .flash2]
[81ead80, 81eae00] LOAD #4 [R], .flash2, [section .flash2]
27.0% 219Ki 27.0% 219Ki [section .flash2]
That shows that the majority (126aae - f0000 = 223_918 bytes) is allocated at the one spot (the beginning?)
when examining
bootloader.elf, I still see unresolved symbols looking like this:_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$::fmt::hbce8bf475f6abdc1 _$LT$$RF$mut$u20$W$u20$as$u20$core..fmt..Write$GT$::write_str::h8c7bbb38f915a22d _$LT$trezor_lib..ui..component..text..layout..TextRenderer$u20$as$u20$trezor_lib..ui..component..text..layout..LayoutSink$GT$::text::h3ffbe9ab3d2082e1
Right, these strange things are currently not being translated. What do we want to show in these cases?
We could do
_$LT$$RF$T$u20$as$u20$core..fmt..Debug$GT$::fmt::hbce8bf475f6abdc1
Rust.core.fmt Debug:fmt()
_$LT$$RF$mut$u20$W$u20$as$u20$core..fmt..Write$GT$::write_str::h8c7bbb38f915a22d
Rust.core.fmt Write::write_str()
here's a translation table:
REPLACEMENTS = {
"$LT$": "<",
"$GT$": ">",
"$u20$": " ",
"$RF$": "&",
"..": "::",
}
the last entry messes with the :: separator detection though
I would like to point your attention to bootloader.map and/or firmware.map files. These contain the linker verbose output, and AFAICT list all linked symbols and their positions and sizes.
This should allow us to identify every byte of the elf file, including the "section" blocks (which I now suspect are .rodata)
here's a translation table:
REPLACEMENTS = { "$LT$": "<", "$GT$": ">", "$u20$": " ", "$RF$": "&", "..": "::", }the last entry messes with the
::separator detection though
Thanks for the table. In the end there were more symbols encoded like this.
Better resolving of Rust functions done in 22caf59
here is a rough script that extracts data from bootloader.map
some things are hardcoded, it would need modifications for firmware (or for any sort of general usage)
https://gist.github.com/matejcik/6499988646b11eb1e33d6c4cebe67549
2529960 tries to differentiate between the logic size and the data size.
For micropython I am deciding this according to symbol name - data symbols start with const_obj.
For C, I am looking at the definition of the symbol and checking for const keyword in the definition.
Rust does not seem to have (much) data, so everything is logic.
It also shows how many symbols got aggregated under one function - usually the more data there, the more aggregated symbols (applicable only for micropython)
It creates this output ("L"ogic column and "D"ata column)
SUMMARY: 6_331 rows, 1_655_376 bytes in total (L1_300_742 D354_634).
.flash2 230_976 1 L230_976 D0 [section .flash2]
.flash 188_970 1 L188_970 D0 [section .flash]
.flash2 77_037 2011 L44_909 D32_128 src/apps/ethereum/tokens.py:15 token_by_chain_address()
.flash2 65_536 1 L0 D65_536 vendor/secp256k1-zkp/src/precomputed_ecmult_gen.c:14 secp256k1_ecmult_gen_prec_table
.flash 37_084 1 L0 D37_084 vendor/trezor-crypto/nist256p1.c:26 nist256p1
.flash 37_084 1 L0 D37_084 vendor/trezor-crypto/secp256k1.c:26 secp256k1
.flash2 27_544 1 L0 D27_544 vendor/micropython/ports/powerpc/mpconfigport.h mp_qstr_frozen_const_pool
.flash 24_576 1 L0 D24_576 vendor/trezor-crypto/ed25519-donna/ed25519-donna-basepoint-table.c:4 ge25519_niels_base_multiples
.flash 14_224 1 L14_224 D0 vendor/trezor-crypto/blake2b.c:213 blake2b_compress
...
b2f82ae contains the map-file analysis (thanks @matejcik !) and translates these "mysterious" data into known symbols.
All these data can be divided (roughly) into three parts:
-
.bootloader- the whole bootloader (cannot be divided anymore) -
.rodata- a whole section, and a lot of symbols like.rodata::L__unnamed_70or.rodata::L__unnamed_25- aggregated under.rodata::L__unnamed -
str1.1- a whole section, and a lot of symbols likenorcow_write.str1.1, which are then aggregated together withnorcow_writeas data - so there are both logic and data entries fornorcow_writefunction
The current results look like:
SUMMARY: 5_937 rows, 1_564_752 bytes in total (L921_223 D643_529).
.flash2 179_468 1 L0 D179_468 str1
.flash 94_208 1 L94_208 D0 .bootloader
.flash2 77_037 2011 L44_909 D32_128 src/apps/ethereum/tokens.py:15 token_by_chain_address()
.flash2 65_536 1 L0 D65_536 vendor/secp256k1-zkp/src/precomputed_ecmult_gen.c:14 secp256k1_ecmult_gen_prec_table
.flash2 43_326 1 L0 D43_326 .rodata
.flash 37_186 1 L0 D37_186 str1
.flash 37_084 1 L0 D37_084 vendor/trezor-crypto/nist256p1.c:26 nist256p1
.flash 37_084 1 L0 D37_084 vendor/trezor-crypto/secp256k1.c:26 secp256k1
.flash2 26_384 1 L0 D26_384 vendor/micropython/ports/powerpc/mpconfigport.h mp_qstr_frozen_const_pool
.flash 25_917 1 L0 D25_917 .rodata
.flash 24_576 1 L0 D24_576 vendor/trezor-crypto/ed25519-donna/ed25519-donna-basepoint-table.c:4 ge25519_niels_base_multiples
.flash 14_224 1 L14_224 D0 vendor/trezor-crypto/blake2b.c:213 blake2b_compress
.flash 11_856 71 L0 D11_856 .rodata::L__unnamed
.flash2 10_636 116 L8_828 D1_808 src/apps/ethereum/networks.py:45 _networks_iterator()
.flash2 10_557 112 L8_813 D1_744 src/apps/common/coininfo.py:93 by_name()
At the bottom, there are the leftovers of the "mysterious" sections.
.flash2 -30 1 L-30 D0 [section .flash2]
.flash -6_342 1 L-6_342 D0 [section .flash]
These negative numbers tell that we have added more data from the map file than the size of the mysterious section - however, the difference is really small (but could be made more precise).
There is a slight disadvantage of adding the map-file data - the new symbols do not have a definition taken from the nm file, so it takes some time to find their definitions (at the first time of running).
Force-pushed after cleaning and rebasing on master.
Current groupings look like:
305_408: .rodata + str1.1 ( 5 symbols)
305_084: Crypto ( 979 symbols)
196_103: Micropython (1_281 symbols)
108_792: Secp256 ( 159 symbols)
105_860: UI ( 845 symbols)
102_577: Ethereum app ( 85 symbols)
94_208: .bootloader ( 1 symbols)
72_738: Monero app ( 448 symbols)
60_534: Bitcoin app ( 405 symbols)
55_558: Altcoin apps ( 354 symbols)
35_966: Cardano app ( 242 symbols)
32_175: Rest of src/ ( 253 symbols)
29_693: Common apps ( 155 symbols)
28_055: Webauthn app ( 200 symbols)
27_112: Storage ( 189 symbols)
21_241: Trezorio ( 181 symbols)
21_020: Trezorhal ( 189 symbols)
20_534: Management app ( 102 symbols)
13_613: Other apps ( 74 symbols)
11_886: Rust ( 77 symbols)
2_984: None ( 51 symbols)
2_495: Trezorconfig ( 38 symbols)
1_753: Embed firmware ( 22 symbols)
1_523: Trezorutils ( 20 symbols)
SUMMARY: 24 categories, 6_355 symbols, 1_656_912 bytes in total.
What is interesting, firmware built with TREZOR_MODEL=R is almost 100kb smaller than the "regular" one. The biggest difference is in the UI category.
I do not have any further ideas on how to make the tool better or what functionality to add - the current state seems OK for me.
About the possibility of moving this tool into its own repository and creating its own python package - as suggested - it would be really nice, but it seems to me a lot of the logic is specific to our repository (especially all the path resolving and special symbols we encounter). Looking forward to discussing this.
very nice!
so now str1 is reported both as its own section, AND counts towards data in the functions from which it comes? or are there leftover str1s that don't belong?
maybe separate out bootloader from rest of .rodata so that we see what more is left there.
Force-pushed after rebase on master (needed for the last commit).
9dabafc newly adds a script to compare history sizes of firmware master - as suggested by @matejcik.
Results of going 500 commits into past with step of 15 commits are:
a5824ed1 2022-01-31 {'.flash': 768552, '.flash2': 849960}
c755c417 2022-02-07 {'.flash': 768552, '.flash2': 850984}
bb71f9f3 2022-02-10 {'.flash': 768552, '.flash2': 850984}
ab0eef5d 2022-02-10 {'.flash': 768552, '.flash2': 850472}
3c0cb4d7 2022-02-15 {'.flash': 768552, '.flash2': 850984}
69eb0958 2022-02-17 {'.flash': 768552, '.flash2': 850984}
b22c0a0c 2022-02-24 {'.flash': 768552, '.flash2': 850984}
c3b28d8f 2022-03-10 {'.flash': 768552, '.flash2': 850984}
94fa6d25 2022-03-17 {'.flash': 768552, '.flash2': 852008}
9f9535ab 2022-03-18 {'.flash': 769064, '.flash2': 857640}
45276963 2022-03-23 {'.flash': 769064, '.flash2': 858664}
4f95ec25 2022-03-30 {'.flash': 769064, '.flash2': 857640}
38f4ab09 2022-04-04 {'.flash': 769576, '.flash2': 857640}
e4a6608d 2022-04-12 {'.flash': 769576, '.flash2': 858152}
26d1fad2 2022-05-02 {'.flash': 769576, '.flash2': 863784}
c9b521a8 2022-05-03 {'.flash': 771112, '.flash2': 865320}
09d14307 2022-05-03 {'.flash': 771112, '.flash2': 865320}
768d577d 2022-05-03 {'.flash': 771112, '.flash2': 867368}
bd476769 2022-05-10 {'.flash': 771112, '.flash2': 865832}
5e6582a3 2022-05-16 {'.flash': 775208, '.flash2': 875048}
c5f1bec4 2022-05-18 {'.flash': 775720, '.flash2': 876584}
a69e43d1 2022-05-30 {'.flash': 775720, '.flash2': 878120}
dfa4b1d9 2022-06-01 {'.flash': 776232, '.flash2': 879656}
7d37109e 2022-06-14 {'.flash': 775720, '.flash2': 880680}
ccf364f1 2022-06-28 {'.flash': 776232, '.flash2': 881192}
45588493 2022-06-30 {'.flash': 776232, '.flash2': 881192}
687a981d 2022-07-04 {'.flash': 776232, '.flash2': 884776}
f4d0dd98 2022-07-08 {'.flash': 776232, '.flash2': 890920}
38f7d32d 2022-07-20 {'.flash': 776232, '.flash2': 890920}
c353135b 2022-08-02 {'.flash': 777256, '.flash2': 898600}
745be6ea 2022-08-08 {'.flash': 776744, '.flash2': 897576}
0b52ffb9 2022-08-18 {'.flash': 776744, '.flash2': 897576}
d5b0650c 2022-08-29 {'.flash': 776744, '.flash2': 897576}
It means there was almost 50 kb added to .flash2 in the last 7 months while .flash increased "only" by 8 kb.
If we would have some server available for use, we can run this script for a longer period and with a smaller granularity (analyze every commit for the last 2000 of them e.g.) to really get insight into which commits caused the biggest increases.
Here is the detailed diff of that 7 months period - between a5824ed1 and d5b0650c
(generated by python tools/size_firmware_elf.py compare build/firmware/firmware.elf_a5824ed1ff79c8bf3b03b09039299d2a9fcef6e9 build/firmware/firmware.elf_d5b0650cc2485af5d0e887f95ff2067be9e98cd8 -d -o diff.txt)
A lot of symbols are there both in plus and minus category, mostly for renaming purposes.
It seems at the first glance that the biggest space-eaters are monero and cardano.
66fa214 builds on top of the previous commit and introduces a tool to get the flash-size difference of a specific commit.
Usage: python size_of_commit.py 26d1fad2
- it builds the binary for that commit and for the commit before and checks the size difference
The previous size_history.py modified a little bit for this use-case and made to accept arguments for the commit history:
-
python size_history.py 1000 50- going 1000 commits into past and with 50 commits steps
Force-pushing after rebasing on master and some changes:
- removing the
tools/bin_sizein favour of installing it from PyPI - firmware size-related scripts moved to
tools/sizeand they are interacting with abinsizelibrary/command, supplying it firmware-specific settings
what's the status here? didn't we move out some things to separate pypi packages?
what's the status here? didn't we move out some things to separate pypi packages?
The status is that this PR has been forgotten :) We have moved the core functionality to its own binsize package, and that code was removed form this PR.
What is staying here are some firmware-specific functionalities for analyzing our flash-space and including some checks into CI
pipfile.lock diff is too big. please use something like pipfile lock --no-update (not sure about the exact command) to only add the new dependency and not update everything
pipfile.lock diff is too big. please use something like
pipfile lock --no-update(not sure about the exact command) to only add the new dependency and not update everything
Right, did poetry lock --no-update now and the diff is much smaller. It updates just platformdirs and termcolor, as binsize specifies some higher version of them.
It also added category: main|dev - hopefully that is not a problem. The poetry version change reverts the change from dependabot, so therefore the decrease