gno icon indicating copy to clipboard operation
gno copied to clipboard

feat(p/ufmt): Optimize `ufmt` package

Open notJoon opened this issue 8 months ago • 4 comments

Description

This is an additional works for PR #4136. However, I couldn't push it directly, so I made this as a separate PR.

This PR includes various optimizations to improve the performance and memory usage of the ufmt package. In particular, it introduces lookahead[^1] approches to the format string parsing mechanism to enhance processing speed.

Key Improvements

1. Format Metadata Parsing Optimization

The existing parsing approach used multiple function calls and nested loops to parse format metadata. This has been replaced with a single-pass loop and lookahead technique, significantly improving processing speed.

Example parsing process (format string: %#+-0 15.8llx):

Existing method: Parsing each element individually
1. Parse '#' flag -> separate check
2. Parse '+', '-', '0', ' ' flags -> separate check for each
3. Parse '15' width -> call parseNumber function
4. Parse '.8' precision -> check for dot then call parseNumber function
5. Parse 'l', 'l' length modifiers -> iterate through all possible modifiers

Lookahead method: Parsing in a single pass
1. Detect '#' character -> set flagHashtag
2. Detect each flag character '+', '-', '0', ' ' -> set corresponding flag
3. Detect '15' number -> parse as width
4. Detect '.' dot -> check next character (lookahead) to parse precision
5. Detect 'l' -> lookahead next character to check if another 'l' exists
   -> If more 'l' exists, process as 'll' modifier at once

This approach is much more efficient, especially when parsing complex format specifiers.

2. Improved Buffer Management

Buffer management has been improved to optimize memory allocation:

  • Enhanced Printf function to output directly without intermediate buffers

3. Function Inlining and Optimization

Several small functions have been partially inlined to reduce function call overhead

  • Inlined the parseDigit function within parseNumber

Performance Measurement Method

Since we cannot directly verify performance line by line yet, we measured performance by outputting gas consumption from txtar tests that failed in the original PR. Although this isn't completely precise due to various influencing factors, it was considered a relatively reliable measurement method.

Tests were conducted using the TestRunMultiple_Integration test from gno.land/pkg/gnoclient/integration_test.go[^2].

go test -v ./gno.land/pkg/gnoclient -run TestRunMultiple_Integration

The gas consumption output from the original PR was:

=== RUN   TestRunMultiple_Integration
    integration_test.go:457: Gas used for first run: 23630623
(... failure afterwards ...)

We can see that only the first value is printed, and subsequent values are not output because the gas usage exceeds the requested value. When calculating the improvement ratio later, we will use 23630623 obtained here as an original usage.

The gas consumption after the modification was:

=== RUN   TestRunMultiple_Integration
    integration_test.go:457: Gas used for first run: 22194438
    integration_test.go:463: Gas used for second run: 22194595
--- PASS: TestRunMultiple_Integration (3.36s)
PASS
ok      github.com/gnolang/gno/gno.land/pkg/gnoclient   3.992s

Comparing only the numbers, we confirmed that the gas consumption improved as follows:

Reduction = 23,630,623 - 22,194,438 = 1,436,185
Improvement ratio = (Reduction / Original usage) × 100%
                  = (1,436,185 / 23,630,623) × 100%
                  ≈ 6.08%

[^1]: A method of making parsing decisions by examining tokens that appear after the current input token in advance. [^2]: https://github.com/gnolang/gno/blob/bb6a8842a921609ebd5e2fa4204de4d68ff5fe4d/gno.land/pkg/gnoclient/integration_test.go#L364

notJoon avatar Apr 17 '25 02:04 notJoon

🛠 PR Checks Summary

All Automated Checks passed. ✅

Manual Checks (for Reviewers):
  • [ ] IGNORE the bot requirements for this PR (force green CI check)
Read More

🤖 This bot helps streamline PR reviews by verifying automated checks and providing guidance for contributors and reviewers.

✅ Automated Checks (for Contributors):

🟢 Maintainers must be able to edit this pull request (more info) 🟢 Pending initial approval by a review team member, or review from tech-staff

☑️ Contributor Actions:
  1. Fix any issues flagged by automated checks.
  2. Follow the Contributor Checklist to ensure your PR is ready for review.
    • Add new tests, or document why they are unnecessary.
    • Provide clear examples/screenshots, if necessary.
    • Update documentation, if required.
    • Ensure no breaking changes, or include BREAKING CHANGE notes.
    • Link related issues/PRs, where applicable.
☑️ Reviewer Actions:
  1. Complete manual checks for the PR, including the guidelines and additional checks if applicable.
📚 Resources:
Debug
Automated Checks
Maintainers must be able to edit this pull request (more info)

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 The pull request was created from a fork (head branch repo: notJoon/gno-core)

Then

🟢 Requirement satisfied
└── 🟢 Maintainer can modify this pull request

Pending initial approval by a review team member, or review from tech-staff

If

🟢 Condition met
└── 🟢 And
    ├── 🟢 The base branch matches this pattern: ^master$
    └── 🟢 Not (🔴 Pull request author is a member of the team: tech-staff)

Then

🟢 Requirement satisfied
└── 🟢 If
    ├── 🟢 Condition
    │   └── 🟢 Or
    │       ├── 🔴 At least one of these user(s) reviewed the pull request: [jefft0 leohhhn n0izn0iz notJoon omarsy x1unix] (with state "APPROVED")
    │       ├── 🟢 At least 1 user(s) of the team tech-staff reviewed pull request
    │       └── 🔴 This pull request is a draft
    └── 🟢 Then
        └── 🟢 And
            ├── 🟢 Not (🔴 This label is applied to pull request: review/triage-pending)
            └── 🟢 At least 1 user(s) of the team tech-staff reviewed pull request

Manual Checks
**IGNORE** the bot requirements for this PR (force green CI check)

If

🟢 Condition met
└── 🟢 On every pull request

Can be checked by

  • Any user with comment edit permission

Gno2D2 avatar Apr 17 '25 02:04 Gno2D2

Codecov Report

:white_check_mark: All modified and coverable lines are covered by tests.

:loudspeaker: Thoughts on this report? Let us know!

codecov[bot] avatar Apr 17 '25 02:04 codecov[bot]

Does this depend on #4136?

thehowl avatar May 14 '25 14:05 thehowl

Does this depend on #4136?

Yes, but not a prerequisite. it's an sort of alternative takes

notJoon avatar May 14 '25 14:05 notJoon