Replace usage of pairs() with ipairs()

[aStonedPenguin]

ipairs is faster than pairs and it is good practice to use where applicable.

[ExtReMLapin]

GL to whoever is doing the QA

[Kefta]

I wonder if proving code correctness would get PRs accepted faster.

[neico]

One could add a CI with lua linting and all that stuff to make sure certain things stay the same across all changes.

At this point it's probably less time investment to make that happen rather than checking everything manually, and still let things slip trough. (as seen in recent merges) Of course a lot of things can't be handled by that like this PR for example, as lua's dynamic type variables make it harder to find out when something needs a pairs or ipairs.

One could fork luacheck and add glua specifics to it (actually someone already did, tough I think there's quite a few things missing: https://github.com/mpeterv/luacheck/compare/master...impulsh:master), but I still expect this tool to blow up given how inconsistent quite a few parts across gmod's code are.

[ExtReMLapin]

function pairs(...)
    return ipairs(...)
end

darkrp coder

[rcfai]

ipairs is faster than pairs

any source on this?

[ExtReMLapin]

are you fokin serious

[thegrb93]

iirc it's faster but only with lua jit; it's been a long while since I tested it though

[ExtReMLapin]

are we really having this conversation

[thegrb93]

are we really real bro

[thegrb93]

https://springrts.com/wiki/Lua_Performance#TEST_9:_for-loops

Like I said though, this could be outdated and I know it does better in lua-jit

[rcfai]

I ran a quick test and ipairs are indeed expectedly a bit slower than pairs - and I don't see why would it be the other way around (in any environment), that's why I'm asking for clarification.

edit: just tested again a bit more with a different setup, ipairs is faster with ordered tables (arrays) that were defined as such and/or populated with table.insert (but not tab[i] = val), also gives weird results for arrays returned by functions like player.GetAll (slower with smaller tables, faster with large ones).

are we really having this conversation

the only thing wrong with this conversation is your useless replies in it, pardon the attitude

[ExtReMLapin]

here we go

ipairs( table tab )

This will only iterate though numerical keys, and these must also be sequential; starting at 1 with no gaps.

local tbl = {}

print("filling table")
local i = 1
local time = SysTime();
while ( i < 50000000) do
	tbl[i] = 42
	i = i +1;
end

print("done in sec : ", SysTime()-time)
print("Checking pairs")
time = SysTime();
for k, v in pairs (tbl) do
	tbl[k] = 0
end
print("done in sec : ", SysTime()-time)
print("Checking ipairs")
time = SysTime();
for k, v in ipairs (tbl) do
	tbl[k] = 1
end
print("done in sec : ", SysTime()-time)

filling table
done in sec : 	0.3570148
Checking pairs
done in sec : 	0.34262840000002
Checking ipairs
done in sec : 	0.046578799999963

[thegrb93]

like i said, its faster in lua-jit. Do same test twice with jit.on() and jit.off() before each

[ExtReMLapin]

why would anyone even disable luajit

[thegrb93]

I'm not arguing against it, I'm just saying so everyone doesn't automatically assume its always faster.

[rcfai]

I guess the larger the table, the better ipairs() becomes then? Here are my results:

local function Benchmark(s, fn)
	local start = SysTime()

	for i = 1, 1000000 do
		fn()
	end

	print(s, SysTime() - start)
end

Benchmark("pairs", function()
	local tab = {}

	for i = 1, 100 do
		tab[i] = "abc"
	end

	for k, v in pairs(tab) do
		tab[k] = v .. "def"
	end
end)

Benchmark("ipairs", function()
	local tab = {}

	for i = 1, 100 do
		tab[i] = "abc"
	end

	for k, v in ipairs(tab) do
		tab[k] = v .. "def"
	end
end)


Benchmark("pairs array", function()
	local tab = {}

	for i = 1, 100 do
		table.insert(tab, i)
	end

	for k, v in pairs(tab) do
		tab[k] = v + 1
	end
end)

Benchmark("ipairs array", function()
	local tab = {}

	for i = 1, 100 do
		table.insert(tab, i)
	end

	for k, v in ipairs(tab) do
		tab[k] = v + 1
	end
end)


print(player.GetCount())

Benchmark("pairs players", function()
	for k, v in pairs(player.GetAll()) do
		v.someProp = 123
	end
end)

Benchmark("ipairs players", function()
	for k, v in pairs(player.GetAll()) do
		v.someProp = 123
	end
end)


print(ents.GetCount())

Benchmark("pairs ents", function()
	for k, v in pairs(ents.GetAll()) do
		v.someProp = 123
	end
end)

Benchmark("ipairs ents", function()
	for k, v in pairs(ents.GetAll()) do
		v.someProp = 123
	end
end)

jit.on()
pairs	4.0796654162872 -- winrar
ipairs	4.9667485460449
pairs array	2.9359403205518
ipairs array	2.3351906703064 -- winrar
8
pairs players	1.5856889561004
ipairs players	1.4271758479681 -- winrar
108
pairs ents	32.234398513788 -- winrar (barely)
ipairs ents	32.243435507719

jit.off()
pairs	4.5898933878526 -- winrar
ipairs	5.4330224654472
pairs array	5.2387944643681 -- winrar
ipairs array	6.0996491387908
8
pairs players	1.4524896966399 -- winrar (barely)
ipairs players	1.4544777069839
108
pairs ents	28.940780261178
ipairs ents	28.881680502325 -- winrar

[thegrb93]

Do you have ipairs overloaded somewhere? PrintTable(debug.getinfo(pairs, "s")) PrintTable(debug.getinfo(ipairs, "s"))

[rcfai]

Nope, I ran the tests without any external modifications

[thegrb93]

Run those functions and make sure

[rcfai]

i just did, both functions are fine

[thegrb93]

Idk why but it seems your jit isn't working.

[Kefta]

It's because your benchmarks are calling a bunch of other functions: only benchmark the loop itself. Also, localise pairs and ipairs to the file so you don't count global lookups.

[rcfai]

I would assume none of that really matters because any overhead created is equal for both functions?

Anyways:

local t = SysTime
local pairs = pairs
local ipairs = ipairs

local start = t()

for i = 1, 1000000 do
	local tab = {}

	for i = 1, 100 do
		tab[i] = "abc"
	end

	for k, v in pairs(tab) do
		tab[k] = v .. "def"
	end
end

print("pairs", t() - start)



start = t()

for i = 1, 1000000 do
	local tab = {}

	for i = 1, 100 do
		tab[i] = "abc"
	end

	for k, v in ipairs(tab) do
		tab[k] = v .. "def"
	end
end

print("ipairs", t() - start)



start = t()

for i = 1, 1000000 do
	local tab = {}

	for i = 1, 100 do
		tab[i] = "abc"
	end

	for i = 1, #tab do
		tab[i] = tab[i] .. "def"
	end
end

print("numeric for", t() - start)

with jit.off():

pairs	4.5386048382724
ipairs	5.456847150973
numeric for	4.6234451125326

with jit.on():

pairs	4.0024753086677
ipairs	4.9457950389245
numeric for	4.1129672361652

[thegrb93]

Just tried it out and jit.on/off seems to not work

[thegrb93]

Ok I get it, those have to be called before the function is defined.

require("xbench")

local t = {}
for i=1, 1e6 do
	t[i] = i
end

print("jit.on()")
jit.on()
xbench.Compare({
	["for k, v in ipairs(t) do end"] = function()
		for k, v in ipairs(t) do end
	end,
	["for k, v in pairs(t) do end"] = function()
		for k, v in pairs(t) do end
	end,
	["for i=1, #t do local x = t[i] end"] = function()
		for i=1, #t do local x = t[i] end
	end
},
10)

print("\njit.off()")
jit.off()
xbench.Compare({
	["for k, v in ipairs(t) do end"] = function()
		for k, v in ipairs(t) do end
	end,
	["for k, v in pairs(t) do end"] = function()
		for k, v in pairs(t) do end
	end,
	["for i=1, #t do local x = t[i] end"] = function()
		for i=1, #t do local x = t[i] end
	end
},
10)
jit.on()

jit.on()
for i=1, #t do local x = t[i] end: 0.0072522000000035
for k, v in ipairs(t) do end: 0.0074950000000058
for k, v in pairs(t) do end: 0.032577000000003

jit.off()
for k, v in pairs(t) do end: 0.032491200000003
for i=1, #t do local x = t[i] end: 0.052757999999997
for k, v in ipairs(t) do end: 0.1123873

[Kefta]

jit.* works fine. You're creating and filling the table as part of the benchmark loop - the majority of time is probably spent by the garbage collector and not the loops themselves.

I would assume none of that really matters because any overhead created is equal for both functions?

The overhead can differ completely on a number of inconsistent factors like JIT inlining, Lua->C->Lua table generation, etc.

Here's a test with just the loops and a simple passthrough function to make sure JIT doesn't optimise it out: https://pastebin.com/SJL5BFpB

Results:

numeric-for	1.1725788032439
ipairs	1.6107837944594
pairs	21.43975093841

[thegrb93]

Ya I found out you have to jit.on/off before defining the function to or not to be optimized. That's why his wasn't working too.

[rcfai]

Alright then, everything is clear. Thanks for the info.

[end360]

AFAIK the reason ipairs is faster than pairs is because ipairs is compiled in luajit but pairs is not. See: http://wiki.luajit.org/NYI https://blog.cloudflare.com/luajit-hacking-getting-next-out-of-the-nyi-list/

[Kefta]

Not so fast