nginx
incorrect order of keys in Object.getOwnPropertyNames()
njs:
>> Object.getOwnPropertyNames([1,2,3].reduce((a,x,i,s) => { a[s.length - i] = x; a['s' + (s.length - i)] = x; return a; }, {}))
[
'3',
's3',
'2',
's2',
'1',
's1'
]
node:
> Object.getOwnPropertyNames([1,2,3].reduce((a,x,i,s) => { a[s.length - i] = x; a['s' + (s.length - i)] = x; return a; }, {}))
[ '1', '2', '3', 's3', 's2', 's1' ]
Also, while Object.keys doesn't guarantee the order shown, it widely used in test262 (https://github.com/tc39/test262/issues/2226).
For example: /language/computed-property-names/object/method/number.js
@ljharb
It does guarantee that the order is the same as in for in, but does not require sorted numbers go first.
I'm pretty certain that it does - Object.keys({ a: 1, '1': 1, b: 1 }) produces ["1", "a", "b"] in all browsers. If there's been an oversight and the spec doesn't mandate that, then web reality means that we need to fix it to do so.
But in the case of middle-ware there is no profit from the 'fix' only an unneeded overhead and inconsistency in number processing:
// node
> var o = {}, x = 10n**15n; while (x != 1) o[x = x/10n] = 1; Object.keys(o);
[ '1',
'10',
'100',
'1000',
'10000',
'100000',
'1000000',
'10000000',
'100000000',
'1000000000',
'100000000000000',
'10000000000000',
'1000000000000',
'100000000000',
'10000000000' ]
There's always profit in forcing every implementation of the spec to behave the same - if all the browsers are already applying that overhead the same, then that's what should be required for all JS engines.
So, Web Reality is forcing the browsers to waste computational resources, instead of forcing developers to fix their code. It's sad, but the reason is to make endusers happy, ok.
But why it should be extended to other ECMAScript implementations? I see no reason.
A more severe issue is that njs in some cases does not preserve the insertion order of the properties:
>> var obj = {}
>> var alphabet = 'abcdefghijklmnopqrst'
>> alphabet.split('').forEach(ch => { obj[ch] = ch })
>> obj
{
e: 'e',
d: 'd',
t: 't',
g: 'g',
f: 'f',
a: 'a',
q: 'q',
p: 'p',
c: 'c',
s: 's',
b: 'b',
r: 'r',
m: 'm',
l: 'l',
o: 'o',
n: 'n',
i: 'i',
h: 'h',
k: 'k',
j: 'j'
}
>> Object.keys(obj)
[
'e',
'd',
't',
'g',
'f',
'a',
'q',
'p',
'c',
's',
'b',
'r',
'm',
'l',
'o',
'n',
'i',
'h',
'k',
'j'
]
It’s interesting that it always ends up in this exact order, at least on my machine and with build njs-0.5.0-x86_64-linux.
@jirutka
A more severe issue is that njs in some cases does not preserve the insertion order of the properties:
In njs objects are simple hashes, they doesn't have an order :).
Take a look at Java HashMap vs. LinkedHashMap.
In njs objects are simple hashes, they doesn't have an order :). Take a look at Java HashMap vs. LinkedHashMap.
njs implements ECMAScript, not Java language. In ECMAScript, objects do have an order, it’s specified in the standard on the very same place that you linked: [[OwnPropertyKeys]]:
For each own property key P of O such that Type(P) is String and P is not an array index, in ascending chronological order of property creation, do Add P as the last element of keys.
And that’s how Node.js and all (?) modern browsers behave.
Maybe you misunderstood me because I used just >> obj instead of e.g. Object.keys in the example…? I updated it now.
njs implements ECMAScript, not Java language.
https://262.ecma-international.org/5.1/#sec-15.2.3.14 https://262.ecma-international.org/5.1/#sec-12.6.4
The mechanics and order of enumerating the properties (step 6.a in the first algorithm, step 7.a in the second) is not specified.
And, in my opinion, we should stay at es5.1 level there.
That doesn’t make any sense - especially since ecmascript is a living standard, and the only spec that matters is https://tc39.es/ecma262/ - but also because most people aren’t writing code to target a standard nobody sticks to anymore from a decade ago.
This properties shuffling seems to be related to some optimization, because objects do preserve the insertation order until you reach 11 properties:
>> var x = { a: 'a', b: 'b', c: 'c', d: 'd', e: 'e', f: 'f', g: 'g', h: 'h', i: 'i', j: 'j' }
>> x
{
a: 'a',
b: 'b',
c: 'c',
d: 'd',
e: 'e',
f: 'f',
g: 'g',
h: 'h',
i: 'i',
j: 'j'
}
>> var y = { a: 'a', b: 'b', c: 'c', d: 'd', e: 'e', f: 'f', g: 'g', h: 'h', i: 'i', j: 'j', k: 'k' }
>> y
{
e: 'e',
d: 'd',
g: 'g',
f: 'f',
a: 'a',
c: 'c',
b: 'b',
i: 'i',
h: 'h',
k: 'k',
j: 'j'
}
Which makes it a trap for the developers because they will not notice it on a very small data. Since all other implementations do preserve properties order for a very long time and ECMAScript standardizes this behaviour since ES2015 (released six years ago!), this is totally unexpected and it may be a serious problem for njs adoption in a wider community.
https://262.ecma-international.org/5.1/#sec-15.2.3.14 https://262.ecma-international.org/5.1/#sec-12.6.4
Did you notice the date when this version was published? June 2011, it’s 10 years ago! This version is obsolete. What sense does it make to implement a new ECMAScript interpreter according to already obsolete specification?
Moreover, you started this issue with a link to the current ECMAScript version where the order is defined. So why do you reference the obsolete version now?
this is totally unexpected and it may be a serious problem for njs adoption in a wider community.
To me it's totally unexpected, that one will rely on the order of hash map keys. It's just wrong.
> var y = { a: 'a', b: 'b', c: 'c', d: 'd', e: 'e', 1: 'f', g: 'g', h: 'h', 2: 'i', j: 'j', k: 'k' }
undefined
> Object.keys(y)
[
'1', '2', 'a', 'b',
'c', 'd', 'e', 'g',
'h', 'j', 'k'
]
Actually, I just don't want to pay a computational price that a proper ES7+ implementation require.
Consider JSON.parse'ing a 1M+ string, how many CPU cycles/memory will it waste...
So, to me the current implementation is a perfect, while been incompatible with the latest standard.
Yes, but your expectations are irrelevant. JavaScript dictates that one can, thus, one can.
why is anyone JSON parsing a 1MB string, if you’re taking about what “feels” wrong?
this project is named “njs” - not “n whatever drsm wishes js was”.
@ljharb
Note that Map and Set in JS are ordered as well.
yeah, but without integer footgun :)
why is anyone JSON parsing a 1MB string, if you’re taking about what “feels” wrong? this project is named “njs” - not “n whatever drsm wishes js was”.
This project is a lightweight implementation of ECMAScript script subset, it runs server-side. Some of most common usage patterns are:
- convert JSON request body to an object.
- do requests to external subsystems using JSON as a transport encoding.
And in JSON - An object is an unordered set of name/value pairs.
So, a "proper" ordering of object keys looks useless to me.
It’s not a footgun, and if you spread a Map or a Set to an array (as is common) you get integer indexes anyways.
Moddable runs xs with ES2020 on much lighter hardware than njs runs on, and they seem to be able to handle memory and performance just fine.
JSON.parse/stringify, and JS code using them, in fact relies on ordering of properties. The JSON spec also allows unquoted numbers that are larger than JS can represent - it does not matter what the JSON spec says to JS users.
It’s not a footguns, and if you spread a Map or a Set to an array (as is common) you get integer indexes anyways.
I mean unexpected integer keys when one relies on an insertion order as a footgun.
Moddable runs xs with ES2020 on much lighter hardware than njs runs on, and they seem to be able to handle memory and performance just fine.
A large number of parallel requests is a problem.
JSON.parse/stringify, and JS code using them, in fact relies on ordering of properties. The JSON spec also allows unquoted numbers that are larger than JS can represent - it does not matter what the JSON spec says to JS users.
But, if we do let x = await fetch('...'), y = await x.json(); is it correct to assume that Object.keys(y) has an arbitrary order?
I think no, like in <axml to="order" or="not to" order="?"/>.
Yes, it is correct to assume that if the keys are numeric. If they are not, then insertion order applies, and the correct assumption would be that they match the order of appearance in the original JSON string. More importantly, if you then did let z = await x.json(), y and z must have the same key ordering - ie, it must be deterministic.
@ljharb
Thanks.
If they are not, then insertion order applies, and the correct assumption would be that they match the order of appearance in the original JSON string.
So, there is a stable order over a wire and one can recover it in njs, if in need:
var good_json = '{"a":"a","b":"b","c":"c","d":"d","e":"e","f":"f","g":"g","h":"h","i":"i","j":"j","k":"k"}';
var bad_object = JSON.parse(good_json);
var recovered_data = Object.keys(bad_object)
.map((k) => [k, good_json.indexOf(k)]) // should quote k in prod
.sort((l, r) => l[1] - r[1])
.map((x) => ({ key: x[0], value: bad_object[x[0]]}));
console.log(recovered_data);
But, actually, do you consider relying on that order as a best practice for a use case of transmitting an ordered set of key-value pairs between some independent microservices?
BTW, insertion order doesn't preserved now even for a small number of properties:
>> var x = { a: 1, b: 2}; delete x.a; x.a = 1; Object.keys(x);
[
'a',
'b'
]
Anyway, it is just my personal opinion as njs user based on my usage patterns, YMMW. As I can see now, fixing this properly is a large complicated task, it's not just about some optimizations removal.
Yes, I absolutely do consider it a best practice (whereas any use of delete is a bad practice).
While I want to endorse @drsm that this is not how hashes are ment to be implemented, I want to point out that this is a significant addition to javascript to ensure consistent results. Consider the following example:
>> var i = 0; var x = {get b() { return i++ }, get a() { return i++ }, get 1() { return i++ }}; for(var a in x) { console.log(a, x[a]); } console.log(JSON.stringify(x));
b 0
a 1
1 2
{"b":3,"a":4,"1":5}
> var i = 0; var x = {get b() { return i++ }, get a() { return i++ }, get 1() { return i++ }}; for(var a in x) { console.log(a, x[a]); } console.log(JSON.stringify(x));
1 0
b 1
a 2
{"1":3,"b":4,"a":5}
Objects in JS are simply not hashes or hash maps, even if you can implement them that way.