App
App copied to clipboard
[$500] HIGH: Add `Avatar > About > Troubleshooting` section
Strategy: A billion users means a billion problems. To diagnose and solve them at scale, we capture client-side logs and upload centrally.
Problem: Some problems can't be diagnosed cleanly after the fact from the logs, for a few reasons:
- External contributors don't have access to production logs.
- Internal engineers do have production access, but they're often cumbersome to access.
- The logs are often ambiguous, such as if you have multiple devices connected and tabs open at the same time -- reverse engineering what happened for a particular client is confusing and time consuming.
- Observing logs in realtime is cumbersome or sometimes impossible, which slows down and complicates testing.
- Even if you have all the access you need, when talking with a user sometimes it's difficult to find the logs specific to them.
Solution: Let's expand our ability to remotely diagnose production devices with better dev tools built into the app itself. Specifically:
- Add a
Troubleshooting
section to theAvatar > About
page, just aboveReport a bug
- This will always be available, both on staging and production builds
- Move the following
Test preferences
settings into this new menu:- Use staging server
- Force offline
- Simulate failing network requests
- Authentication status: Invalidate
- Device credentials: Destroy
- Add
Reset and refresh
, which wipes Onyx clean except for the minimum needed to call OpenApp (so you don't need to reauthenticate) - Add
View console
which opens a simple console:- This would first open up and initialize with all locally stored logs, so you can scroll back and see what happened
- It would also append new logs in realtime
- There would also be a
Save
button that just copies all available logs into a text file that is saved on the device - Add a
Share
button that brings up aSearch
selector and just posts the logs into the selected room (and then leaves the user navigated there, so they can talk about them). I'm imagining this would be useful for chatting with a real world user and asking them to do something and send you the logs. - At the bottom is a command line that works similar to the Chrome console, where you can execute arbitrary JavaScript, with the results output to the console
This is blocking https://github.com/Expensify/App/issues/33040.
Upwork Automation - Do Not Edit
- Upwork Job URL: https://www.upwork.com/jobs/~01c2a08b8b8ae90e21
- Upwork Job ID: 1744202378047897600
- Last Price Increase: 2024-01-15
- Automatic offers:
- fedirjh | Reviewer | 28115102
Job added to Upwork: https://www.upwork.com/jobs/~01c2a08b8b8ae90e21
Triggered auto assignment to @greg-schroeder (NewFeature
), see https://stackoverflowteams.com/c/expensify/questions/14418#:~:text=BugZero%20process%20steps%20for%20feature%20requests for more details.
Triggered auto assignment to Contributor-plus team member for initial proposal review - @fedirjh (External
)
Proposal
Please re-state the problem that we are trying to solve in this issue.
External contributors don't have access to production logs. Internal engineers do have production access, but they're often cumbersome to access. The logs are often ambiguous, such as if you have multiple devices connected and tabs open at the same time -- reverse engineering what happened for a particular client is confusing and time consuming. Observing logs in realtime is cumbersome or sometimes impossible, which slows down and complicates testing. Even if you have all the access you need, when talking with a user sometimes it's difficult to find the logs specific to them.
What is the root cause of that problem?
What changes do you think we should make in order to solve the problem?
Expand app ability to remotely diagnose production devices with better dev tools built into the app itself. Specifically:
Add a Troubleshooting section to the Avatar > About page, just above Report a bug This will always be available, both on staging and production builds Move the following Test preferences settings into this new menu: Use staging server Force offline Simulate failing network requests Authentication status: Invalidate Device credentials: Destroy Add Reset and refresh, which wipes Onyx clean except for the minimum needed to call OpenApp (so you don't need to reauthenticate) Add View console which opens a simple console: This would first open up and initialize with all locally stored logs, so you can scroll back and see what happened It would also append new logs in realtime There would also be a Save button that just copies all available logs into a text file that is saved on the device Add a Share button that brings up a Search selector and just posts the logs into the selected room (and then leaves the user navigated there, so they can talk about them). I'm imagining this would be useful for chatting with a real world user and asking them to do something and send you the logs. At the bottom is a command line that works similar to the Chrome console, where you can execute arbitrary JavaScript, with the results output to the console
Can I directly work on this without a proposal?
Proposal
Please re-state the problem that we are trying to solve in this issue.
Some problems can't be diagnosed cleanly after the fact from the logs.
What is the root cause of that problem?
The settings page is not optimal for diagnosing.
What changes do you think we should make in order to solve the problem?
Troubleshoot
- In the
AboutPage
, create a menu item titledTroubleshoot
above theReportBug
menu item on this line Sample Component
{
translationKey: 'initialSettingsPage.aboutPage.troubleshoot',
icon: Expensicons.Troubleshoot,
action: waitForNavigate(Report.navigateToTroubleshoot),
},
Test preferences
- Add a new menu item called
Test preferences
to the settings page. - Create a menu item on the settings menu for the new page.
- Move the necessary menu items (use staging server, force offline, etc..) to the new page.
Reset and refresh
- Add another item in the
Test preferences
page calledReset and refresh
. - Set its action to a created function to refresh the page after setting every onyx collection, except for data required to call the
OpenApp
endpoint, to null.
View console
- Add a new menu item called
View console
to theTest preferences
page. - Bind on
console.log, console.info, console.info, etc...
depending on the type of messages we want to collect, then render before/after callingconsole.stdlog.apply
. - Create a component to get the logs from the console and render them.
- Create an input component for the js script and execute the input script using the inbuilt
eval
function. I recommend a feature to access the history of input scripts and replay if desired. - Add a share button on the header component to take the required logs and send them to the appropriate channels.
Please submit proposals, especially for how you intend to do the console feature. The more detail you provide, the greater likelihood we select your proposal.
On Sun, Jan 7, 2024, 7:57 PM Antony Kithinzi @.***> wrote:
Proposal Please re-state the problem that we are trying to solve in this issue.
Some problems can't be diagnosed cleanly after the fact from the logs, for a few reasons What is the root cause of that problem?
The settings page is not optimal for diagnosing. What changes do you think we should make in order to solve the problem?
Add a new page in settings called Test preferences. Then move the necessary components to the new page. Create a menu item on the settings menu for the page.
Then add another item in Test preferences page called Reset and refresh. This option will clear the onyx collection except for the auth data. Finally, it will refresh the page.
— Reply to this email directly, view it on GitHub https://github.com/Expensify/App/issues/34082#issuecomment-1880353158, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAEMNUQ4EYCWKZ7HZ4VAK63YNNVEDAVCNFSM6AAAAABBQ2U4NSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBQGM2TGMJVHA . You are receiving this because you authored the thread.Message ID: @.***>
This is a thoughtful and comprehensive proposal that tackles the challenge that's encountered in diagnosing and solving issues at scale. I appreciate the clarity in the suggested changes, especially the addition of the Troubleshooting section and the integration of better dev tools within the app.
The proposed enhancements, such as the Console View, Reset and Refresh option, and the ability to Share Logs, would significantly improve the ability to remotely diagnose production devices. I particularly like the idea of having a command line for executing JavaScript, resembling the Chrome console.
I'm eager to contribute to the implementation of these features. Before diving in, is there any specific technical considerations or preferences regarding the implementation? Additionally, if there are any specific testing scenarios or edge cases to keep in mind, please provide guidance.
Looking forward to collaborating on this improvement
📣 @samsonmobisa! 📣 Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork. Please follow these steps:
- Make sure you've read and understood the contributing guidelines.
- Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
- Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
- Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
Format:
Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>
I have some questions about the View console
expected result.
-
Should the
View console
page be full screen because long lines of code will be displayed and the console input component be wide enough to type in scripts? -
Should we have a separate output log component to differentiate the output from the user-performed scripts? Another option is to color or tag it differently from the rest of the application's console log.
-
Are we going to preserve logs after the window refreshes?
Contributor details Your Expensify account email: [email protected] Upwork Profile Link: https://www.upwork.com/freelancers/~018e78ebb343f52495
✅ Contributor details stored successfully. Thank you for contributing to Expensify!
Have already submitted my details
Problem Overview: The current challenge involves diagnosing issues at scale, where existing logs may not provide a clear understanding of problems. External contributors lack access to production logs, and even for internal engineers with access, retrieving logs can be cumbersome and time-consuming. The logs' ambiguity, coupled with difficulties in observing real-time logs, further complicates issue resolution.
Proposed Changes:
-
Troubleshooting Section:
- Add a Troubleshooting section to the Avatar > About page, above Report a bug.
- Always available on staging and production builds.
-
Test Preferences Settings:
- Move Test preferences settings to the Troubleshooting menu.
- Settings include Use staging server, Force offline, Simulate failing network requests, Invalidate Authentication status, and Destroy Device credentials.
-
Reset and Refresh:
- Introduce a Reset and refresh option in Troubleshooting.
- Wipes Onyx clean, retaining only essentials for OpenApp calls, avoiding ReAuthentication.
-
View Console:
- Add a View console option in Troubleshooting.
- Opens a console with locally stored logs, allowing real-time updates.
- Includes a Save button for exporting logs to a text file on the device.
-
Share Logs:
- Include a Share button in Troubleshooting.
- Facilitates posting logs into a selected room for collaboration.
- Enhances communication with users for issue resolution.
-
Command Line:
- Integrate a command line at the bottom of the console.
- Allows execution of arbitrary JavaScript, akin to the Chrome console.
Implementation Details:
- Troubleshooting sections should have a clear UI design, ensuring easy navigation.
- Test preferences settings must seamlessly transition to the Troubleshooting menu.
- Reset and refresh should be implemented efficiently to maintain essential data.
- View consoles should offer a user-friendly interface, supporting both existing and real-time logs.
- Share logs feature should provide options for selecting rooms and streamline the collaboration process.
- Command line functionality should resemble the Chrome console, with proper error handling.
Expected Results:
- Improved accessibility for external contributors and internal engineers.
- Enhanced clarity in log interpretation and diagnosis.
- Streamlined troubleshooting process with efficient tools.
- Real-time log observation for quicker issue resolution.
- Seamless collaboration through log sharing.
I would appreciate your feedback and insights on this proposal. Looking forward to discussing this further.
📣 @Piyush-Kumar-Ghosh! 📣 Hey, it seems we don’t have your contributor details yet! You'll only have to do this once, and this is how we'll hire you on Upwork. Please follow these steps:
- Make sure you've read and understood the contributing guidelines.
- Get the email address used to login to your Expensify account. If you don't already have an Expensify account, create one here. If you have multiple accounts (e.g. one for testing), please use your main account email.
- Get the link to your Upwork profile. It's necessary because we only pay via Upwork. You can access it by logging in, and then clicking on your name. It'll look like this. If you don't already have an account, sign up for one here.
- Copy the format below and paste it in a comment on this issue. Replace the placeholder text with your actual details.
Format:
Contributor details
Your Expensify account email: <REPLACE EMAIL HERE>
Upwork Profile Link: <REPLACE LINK HERE>
Contributor details Your Expensify account email: [email protected] Upwork Profile Link: https://www.upwork.com/freelancers/~017087736e94a4f9b8
✅ Contributor details stored successfully. Thank you for contributing to Expensify!
Proposal
Please re-state the problem that we are trying to solve in this issue.
HIGH: Add Avatar > About > Troubleshooting section
What is the root cause of that problem?
NA - Feature request.
What changes do you think we should make in order to solve the problem?
- Creating a new Menu i.e TroubleShooting.
We already have About
page in which we show multiple menu's under it, we should use MenuItem
component to render the new TroubleShoot
item.
Once user clicks on it, use Navigation
to navigate to the troubleshoot page.
-
Moving
Test Preferences
toTroubleShoot
page, since thisTroubleshoot
will be a new page, we need to moveMenuItem
's from thePreferences
page. -
Reset and Refresh
, we only need to wipe the onyx keys which doesn't require the user to be login again, this means we need to remove the data inside onyx that are not needed.
The majority of data we store are related to reports
, transactions
, reportActions
, policy
, reportDraftComment
, network
.
To Simply say we should be having onyx data related to following keys account
, session
, credentials
, loginList
, user
, after clearing the keys we should do refresh
i.e window.location.reload
in browser and some native code for native devices to restart the app. Maybe using react-native-restart
.
-
View console
-
Since console might contain a big chunk of lines i.e in real scenario people can be using the application for days where logs can be in 1000s of lines. We have logs stored in
Log.logLines
which we can use to render logs. -
How, rendering well the logs are large list, which can block the low-end browsers to hang and doesn't give good UX. We should be using some virtualized list to render the logs, I prefer
react-virtual
for web andreact-native-fast-list
for native devices or we can simply use theSectionList
that we're using to render thereportActions
. -
Realtime logs rendering, well at the current code we cannot do with the current implementation, to resolve it we should be having some realtime i.e a variable that causes re-render with the code.
-
Saving, we should be generating a blob with type
text/plain
and download it using native tools, i.e for web we should be usingurl.createObjectURL
and same implementation orreact-native-fs
to create the file. -
Sharing, when we click on
Share
we should snapshot the current logs (not new logs, because we're thinking about the current logs) and use the basic logic to share the logs.
-
Implementation specifics / issues
Realtime rendering is the main issue here which will require some observable implementation.
Why? we're not saving logLines
in the react
specific code which doesn't let us use some react specific sections i.e useState
or so. While figuring out the solution, I found the following package https://jotai.org/docs/guides/react-native
which doesn't need any react specific code to create observable
and can also be used with react
that gives the ability to cause re-renders. So we should be creating a atom that stores the logs and should be exported and used in App
codebase.
We have a limit of 10000 characters inside composer, but we don't know that logLines
does contain the 10000
characters which will block the implementation of Share
feature. Maybe we can wrap the logs in a file and share it with the selected chat. @quinthar you're view on this?
What alternative solutions did you explore? (Optional)
NA
Hello Expensify Team
Your comprehensive plan demonstrates a deep understanding of the challenges at hand and offers a thoughtful and well-structured solution. I particularly appreciate the attention to detail in addressing each aspect of the problem. The proposed Troubleshooting section, along with the seamless integration of Test Preferences and the Reset and Refresh feature, reflects a strategic approach to streamline user support and enhance debugging processes.
The innovative use of observable atoms from the jotai package for real-time log rendering showcases a keen problem-solving mindset. Your consideration of potential issues, such as the character limit for sharing logs, reveals a thorough analysis of the implementation specifics.
This proposal not only tackles the immediate problem but also reflects a commitment to ensuring a robust, user-friendly application. I'm impressed by the holistic approach, covering everything from the user interface to the technical intricacies of real-time rendering.
I am ready and eager to collaborate on implementing these enhancements. Your feedback and guidance will be invaluable as we work together to create a more efficient and reliable workflow.
What's the next step? Who is doing it and when will it be done?
According to guidelines, All the proposals are reviewed by the Core Team from which one gets selected and then needs to do some follow up steps.
After that the contributor starts the work.
I haven't contributed to this project before so I would love to work on it.
@greg-schroeder, @fedirjh Uh oh! This issue is overdue by 2 days. Don't forget to update your issues!
Bump @fedirjh for proposal review
📣 It's been a week! Do we have any satisfactory proposals yet? Do we need to adjust the bounty for this issue? 💸
@greg-schroeder, @fedirjh Whoops! This issue is 2 days overdue. Let's get this updated quick!
@greg-schroeder, @fedirjh Huh... This is 4 days overdue. Who can take care of this?
Contributor details Your Expensify account email: [email protected] Upwork Profile Link: https://www.upwork.com/freelancers/~018e78ebb343f52495
✅ Contributor details stored successfully. Thank you for contributing to Expensify!
bump @fedirjh
Bump, what's the next step here? I think @TMisiukiewicz has already volunteered. Who are we waiting on? What's the next step, and when will it happen?