nodejs
Public User Feedback request - Promises
Following discussion in https://github.com/nodejs/user-feedback/issues/76 and https://github.com/nodejs/promise-use-cases/issues/26 we want to get some questions going for the Node.js promises usability and debuggability survey.
Going to try something - let me know if it's inappropriate :)
Post questions as comments - use 👍 and 👎 in order to indicate whether you think other questions should be included in the survey or not.
You are welcome to post question suggestions even if you are not Node.js project core. If you would like to consult me before posting a suggestion - feel free to contact me personally at [email protected]
What is your primary role in your workplace?
- Developer, I typically am not the person to debug production issues
- Developer, I typically am the person to debug production issues
- DevOps / Operations Engineer, I typically am not the person to debug production issues
- DevOps / Operations Engineer, I typically am the person to debug production issues
- Other (please specify)
Which do you use the most often in order to write async code:
- Promises without
async/await. - Promises with
async/await. - Node.js standard
(err, data)callbacks. - I don't use any of the above at all.
Which of the following tools do you use in order to debug Node.js code at least once a week:
- [ ] The Node.js inspector or an IDE debugger (
nodejs --inspect) - [ ] Long Stack traces with an external library like Bluebird
- [ ] Printed stack traces with native promises (for example
mochaerrors). - [ ] core dumps + lldb
- [ ]
gdb - [ ] Adding
console.logaround code and running it
What should the policy Node.js takes when an unhandled rejection occurs? For example - when an exception is thrown in an async function (or a then handler) but no code handles the error (.catch, then second param or catch in an async function).
- I would expect Node.js to abort the process like on uncaught exceptions.
- I would expect Node.js to stay silent about the error.
- I would expect Node.js to warn about the error. Note: this is the current behavior.
- I would expect something else.
Note: this is only the default and users would still have the ability of code to opt into different behaviour.
Do you use promises with async_hooks?
- I do not know what async_hooks are.
- I do know what async_hooks are but am not using them at all or with promises.
- I am using a third-party library that uses async_hooks with promises
- I don't know if I am using promises with async_hooks
- I am using promises with async_hooks in my code
If you answered "I am using promises with async_hooks in my code" to the previous question - please elaborate how?
_Enter answer here_
Is the performance of Node.js promises sufficient to you:
- Node.js promises are still slow (even on the latest version) and significant effort needs to be put in before I can use them.
- Node.js promises are slow - and while I am using them I think more time should be spent on optimizing them .
- Node.js promises are fast and more effort should be spent on making them more debuggable rather than faster.
- Node.js promises are fast enough now and I wouldn't mind them being slightly slower if that provides better debuggability features such as async stack traces in production apps.
- Node.js promises are not a limiting factor in my app's performance and I wouldn't mind them being slightly slower if that provides better debuggability features such as async stack traces in production apps.
- None of the above apply.
If you have answered that Node.js promises are slow, please write down why that factor limits your usage of promises:
__Enter answer here__
Note: see this note
@benjamingr
What is your primary role in your workplace?
What's the necessity of this?
A large proportion of Node.js users are using it for tooling; Node.js may only be used to bundle a site or run tests. Those users may say they aren't using Node.js in production (I would).
Which do you use the most often in order to write async code:
Is this "which do you use most often" or "which do you prefer?"
Do you use async iterators and for await( loops with streams?
Maybe this question could use a code example. :wink:
Which of the following tools do you use in order to debug Node.js code at least once a week:
Maybe "IDE or Editor" should be added to this list. I debug from VSCode wherever possible.
Is the performance of Node.js promises sufficient to you:
Users may not have an opinion, not care, or don't know.
But otherwise, unless the user is running Node.js 10, will the feedback be useful? If they complain it's slow, and they're using Node.js < 10, would it be an impetus to backport Promise performance improvements?
(Apologize if some of these have been answered already; I was not in attendance at relevant meeting)
@boneskull
What's the necessity of this?
A large proportion of Node.js users are using it for tooling; Node.js may only be used to bundle a site or run tests. Those users may say they aren't using Node.js in production (I would).
To validate that our survey reached a diverse set of developers - and to check the bias of various groups. For example in the meeting some participants mentioned they believe operations engineers use post-mortem debugging techniques differently from developers.
Is this "which do you use most often" or "which do you prefer?"
It is "which do you use most often".
Maybe this question could use a code example. 😉
Not really sure, it's basically "whether or not you have for await in your code or not.
Maybe "IDE or Editor" should be added to this list. I debug from VSCode wherever possible.
👍 added thanks
But otherwise, unless the user is running Node.js 10, will the feedback be useful? If they complain it's slow, and they're using Node.js < 10, would it be an impetus to backport Promise performance improvements?
Well - if people complain they're slow then sure - but there is a possibility they won't. If people complain about performance in older versions that's something to keep in mind as well.
More question suggestions welcome by the way.
Which of the following tools do you use in order to debug Node.js code at least once a week:
I believe you forgot:
- Regular stack traces with native promises
Which is what I use to debug my tests when I'm not running them under a debugger (looking at the stack trace helps me figure out what the problem is.)
@giltayar Regular stack traces with native promises can you show me an example of what you mean exactly? Just looking at hte output of unhandled rejections?
How about adding a question about the use of domain with promises? While I know they're deprecrated and async_hooks are replacing them, I believe a lot of people still use them as they are higher-level abstraction use than async_hooks.
This is a stack trace from my mocha test (which was run just now...). I don't use bluebird, and so don't have long stack traces (and I wish native promises had them!), but I still use these stack traces to debug my code.
@giltayar
I don't use bluebird, and so don't have long stack traces (and I wish native promises had them!)
Native promises have them with the inspector attached - as a "protip" you can run mocha with the node inspector running which enables long stack traces. Following the Berlin summit the V8 team are prioritizing and working on async stack traces in production. I will add the option - thanks for clarifying.
I thought bluebird's long stack trace support was largely from native functionality Error.stackTraceLimit = 50 or something similar?
On the async_hooks question, most people aren't aware that a tool like APM or a library is using hooks. A question more towards the goal of hooks with promises might give better results. Do you track (or wish to) metrics on the time each function in your promise chains runs.
As for the current question, it could use clarification whether it's about just using promises and hooks in the same codebase or something specific about using them together. I think you're aiming at getting examples of what needs to be supported in hooks, but I wouldn't know how to answer that question while I do plenty of promises and wrote blocked-at npm package using hooks.
@justsml no, we actually keep context. V8 does it on their side. stackTraceLimit helps with synchronous stack traces but not async ones.
@naugtur thanks for the suggestion - I agree that it's not optimal - I think "Do you track (or wish to) metrics on the time each function in your promise chains runs." will go above peoples' heads a little - I'm very open to suggestions though. Maybe we should ask about the most popular packages with async hooks? (That is, I use a tool such as ....)
For a number of them "Which of the following tools do you use in order to debug Node.js code at least once a week:"
I think adding an other where people can add their own would be good.
I think adding Chrome developer tools as an option to "Which of the following tools do you use in order to debug Node.js code at least once a week:" would be good.
@benjamingr what might also help is separately listing out what we want to understand based on the survey. That will help when looking at the questions to see if there are any gaps.
What is your primary role in your workplace?
At Netflix, everyone is full cycle engineers that are responsible for development and devops.
If your intent is to understand responsibilities, just separate out into explicit questions on each responsibility and whether that is part of their work/role. You could also provide a scale of how frequently or what percent of their role/time is that responsibility.
"Typically" is vague and open to many interpretations. Example: larger teams with longer on call rotations.
On primary role, I find your choices may have some possible implicit bias built in it. Again would focus on responsibilities not titles.
Importantly, what do you mean by "debug"? There are many ways to debug production issues and often depends on the issue. Is that meaningful to you? If so, be explicit, or your responses won't help you as much.
Example:
- I debug issues of Node.js services? Yes/No
- Do these run in production? Yes/No
- How frequently do you debug? Never, Daily, Weekly, Monthly, Yearly
- How do you debug? Multi checkbox: metrics, logs, flamegraphs, coredumps, other (writein)
Which do you use the most often in order to write async code
Should you have a other writein option? What about Generators or Observables? Is it useful to learn about those cases too?
Which of the following tools do you use in order to debug Node.js code at least once a week:
Why does it have to be "at least once a week"? Are my usages not valid otherwise? What about "which do you use frequently"
I would allow an other writein.
What should the policy Node.js takes when an unhandled rejection occurs? For example - when an exception is thrown in an async function (or a then handler) but no code handles the error (.catch, then second param or catch in an async function).
This text can be improved...
Example: What should be the default policy of Node.js when an unhandled rejection occurs?
Should you allow other writein, instead of I would expect something else?
Not really sure, it's basically "whether or not you have for await in your code or not.
Maybe just ask that then because the current question is hard to parse. I had to re-read few times.
See https://github.com/nodejs/promises-debugging/pull/12 for a survey I did recently at Netflix. There might be good questions/possible answers to borrow from it, but also probably good examples of mistakes we should avoid.
@benjamingr
we want to get some questions going
What do we want to get from the answers to these questions? Do we want to validate hypotheses, if so which ones? Do we want to get a general overview of some usage patterns, if so which ones? In general, it'd be useful to know what we want to achieve with those questions to inform the process of designing a survey.
Node.js promises usability and debuggability survey.
Is that scope potentially too broad? It seems usability and debuggability could be considered separately. It could even be possible to identify separate areas for those two topics (e.g. live vs post-mortem debugging, etc.). Intuitively it seems that the narrower the scope, the better and easier it'd be to design a survey.
In general, have we thought about collaborating with data collection/analysis/survey specialists? Surveys can be very challenging to design and analyze. Getting help from people skilled in this area could potentially increase the value of such a survey. Is that something the Node.js foundation could fund?
The foundation does have a data specialist that reviews the questions and will give us some suggestions.
@benjamingr
Would using Google docs be a better way to collaborate on the design of this survey?
it would probably remove the ability for GH users to add thumbs-up/thumbs-down to questions, but I don't think giving that ability is useful. It seems a thumbs-up or a thumbs-down without any specific comment is not very actionable.
On the other hand, using a Google document seems like it would allow for more collaboration (easier conversation threads per candidate question, easier to have collaborative editing, etc.).
Thoughts?