Lua性能优化

table.new(narray, nhash)

This creates a pre-sized table, just like the C API equivalent lua_createtable().

This is useful for big tables if the final table size is known and automatic table resizing is too expensive.

预分配好数组和hash的大小，避免resize和rehash。

数组类型的table

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44


local table_new = table.new

local start_time
local used_time
local tableSize = 1000000


local stimer = function()
    ngx.update_time()
    start_time = ngx.now()
end

local etimer = function()
    ngx.update_time()
    used_time = ngx.now() - start_time
end

do
    local testTable = {}
    stimer()
    for i = 1, tableSize do
        local key = i
        local value = i
        testTable[key] = value
    end
    etimer()
    ngx.say("time used  : ", used_time, " sec")
end

do
    local testTable = table_new(tableSize, tableSize)
    stimer()
    for i = 1, tableSize do
        local key = i
        local value = i
        testTable[key] = value
    end
    etimer()
    ngx.say("time used  : ", used_time, " sec")
end


-- time used  : 0.010999917984009 sec
-- time used  : 0.00099992752075195 sec

对于数组类型的table，性能相差近20倍

hash类型的table

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43


math.randomseed(os.time())
local table_new = table.new

local start_time
local used_time
local tableSize = 1000000


local stimer = function()
    ngx.update_time()
    start_time = ngx.now()
end

local etimer = function()
    ngx.update_time()
    used_time = ngx.now() - start_time
end

do
    local testTable = {}
    stimer()
    for i = 1, tableSize do
        local key = "key_" .. i
        local value = i
        testTable[key] = value
    end
    etimer()
    ngx.say("time used  : ", used_time, " sec")
end

do
    local testTable = table_new(tableSize, tableSize)
    stimer()
    for i = 1, tableSize do
        local key = "key_" .. i
        local value = i
        testTable[key] = value
    end
    etimer()
    ngx.say("time used  : ", used_time, " sec")
end
-- time used  : 0.43399977684021 sec
-- time used  : 0.21200013160706 sec

差了2倍，不是很大。

pairs() vs ipairs()

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45


-- math.randomseed(os.time())
local table_new = table.new

local start_time
local used_time
local tableSize = 1000000


local stimer = function()
    ngx.update_time()
    start_time = ngx.now()
end

local etimer = function()
    ngx.update_time()
    used_time = ngx.now() - start_time
end

local testTable = table_new(tableSize, 0)
for i = 1, tableSize do
    local key = i
    local value = i
    testTable[key] = value
end


do
    stimer()
    for i,v in ipairs(testTable) do
    end
    etimer()
    ngx.say("time used  : ", used_time, " sec")
end

do
    stimer()
    for i,v in pairs(testTable) do
    end
    etimer()
    ngx.say("time used  : ", used_time, " sec")
end


-- time used  : 0.00099992752075195 sec
-- time used  : 0.0039999485015869 sec

数组类型的table，使用ipairs、pairs相差4倍。

table.clear(tab)

This clears all keys and values from a table, but preserves the allocated array/hash sizes. This is useful when a table, which is linked from multiple places, needs to be cleared and/or when recycling a table for use by the same context. This avoids managing backlinks, saves an allocation and the overhead of incremental array/hash part growth.

Please note, this function is meant for very specific situations. In most cases it’s better to replace the (usually single) link with a new table and let the GC do its work.

对于数组型的table，不需要有顾虑，用就行了，快。

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36


-- math.randomseed(os.time())
local table_new = table.new
local table_clear = table.clear

local start_time
local used_time
local tableSize = 1000000


local stimer = function()
    ngx.update_time()
    start_time = ngx.now()
end

local etimer = function()
    ngx.update_time()
    used_time = ngx.now() - start_time
end

local testTable = table_new(tableSize, 0)
for i = 1, tableSize do
    local key = i
    local value = i
    testTable[key] = value
end


do
    stimer()
    table_clear(testTable)
    etimer()
    ngx.say("time used  : ", used_time, " sec")
end


-- time used  : 0 sec

对于hash table，根据清除再赋值的key是否一样又分两种情况：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73


-- math.randomseed(os.time())
local table_new = table.new
local table_clear = table.clear

local start_time
local used_time
local tableSize = 1000000


local stimer = function()
    ngx.update_time()
    start_time = ngx.now()
end

local etimer = function()
    ngx.update_time()
    used_time = ngx.now() - start_time
end

local testTable = table_new(tableSize, 0)
do
    stimer()
    for i = 1, tableSize do
        local key = "key_" .. i
        local value = i
        testTable[key] = value
    end
    etimer()
    ngx.say("time used[创建表]  : ", used_time, " sec")
end

do
    stimer()
    table_clear(testTable)
    etimer()
    ngx.say("time used[清除表]  : ", used_time, " sec")
end

do
    stimer()
    for i = 1, tableSize do
        local key = "key_" .. i
        local value = i
        testTable[key] = value
    end
    etimer()
    ngx.say("time used[重新初始化表]  : ", used_time, " sec") 
end

do
    stimer()
    table_clear(testTable)
    etimer()
    ngx.say("time used[清除表]  : ", used_time, " sec")
end

do
    stimer()
    for i = 1, tableSize do
        -- key变了
        local key = "key_key" .. i
        local value = i
        testTable[key] = value
    end
    etimer()
    ngx.say("time used[重新初始化表，key变了]  : ", used_time, " sec") 
end

-- time used[创建表]  : 0.41700005531311 sec
-- time used[清除表]  : 0.002000093460083 sec
-- time used[重新初始化表]  : 0.16700005531311 sec
-- time used[清除表]  : 0.0019998550415039 sec
-- time used[重新初始化表，key变了]  : 0.34599995613098 sec

可以看到，如果表清除后，如果赋值的key不变，还是之前的那些那么使用clear要相比重新创建和初始化要快2倍。

如果新赋值的key变了，并没有很节省时间，因此官方文档中也强调了，此函数只适用很特定的场景。

除非清除前后key不变，否则大多情况还是推荐创建新表的方式。

table.concat

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41


local table_new = table.new
local table_concat = table.concat

local start_time
local used_time
local tableSize = 100000


local stimer = function()
    ngx.update_time()
    start_time = ngx.now()
end

local etimer = function()
    ngx.update_time()
    used_time = ngx.now() - start_time
end

local testStr = ""
do
    stimer()
    for i = 1, tableSize do
        testStr =  testStr .. "key_"
    end
    etimer()
    ngx.say("time used[..拼接]  : ", used_time, " sec")
end

do
    local testTable = table_new(tableSize, 0)
    stimer()
    for i = 1, tableSize do
        testTable[i] = "key_"
    end
    local testStr = table_concat(testTable, "")
    etimer()
    ngx.say("time used[concat]  : ", used_time, " sec")
end

-- time used[..拼接]  : 7.6070001125336 sec
-- time used[concat]  : 0.00099992752075195 sec

两种方式性能差别巨大。

resty.lrucache vs resty.lrucache.pureffi

This library offers two different implementations in the form of two classes: resty.lrucache and resty.lrucache.pureffi. Both implement the same API. The only difference is that the latter is a pure FFI implementation that also implements an FFI-based hash table for the cache lookup, while the former uses native Lua tables.

If the cache hit rate is relatively high, you should use the resty.lrucache class which is faster than resty.lrucache.pureffi.

However, if the cache hit rate is relatively low and there can be a lot of variations of keys inserted into and removed from the cache, then you should use the resty.lrucache.pureffi instead, because Lua tables are not good at removing keys frequently. You would likely see the resizetab function call in the LuaJIT runtime being very hot in on-CPU flame graphs if you use the resty.lrucache class instead of resty.lrucache.pureffi in such a use case.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93


local table_new = table.new
local table_concat = table.concat
local lrucache = require("lrucache.ll")
local lrucache_pureffi = require("lrucache.pureffi")

local start_time
local used_time
local tableSize = 100000


local stimer = function()
    ngx.update_time()
    start_time = ngx.now()
end

local etimer = function()
    ngx.update_time()
    used_time = ngx.now() - start_time
end

local testTable = table_new(tableSize, tableSize)
do
    stimer()
    for i = 1, tableSize do
        local key = "key_" .. i
        local value = i
        testTable[key] = value
    end
    etimer()
    ngx.say("time used[lua table set]  : ", used_time, " sec")

    do
        stimer()
        for i = 1, tableSize do
            local key = "key_" .. i
            local _ = testTable[key]
        end
        etimer()
        ngx.say("time used[lua table get]  : ", used_time, " sec")
    end
end

do
    local cache, err = lrucache.new(tableSize)
    stimer()
    for i = 1, tableSize do
        local key = "key_" .. i
        local value = i
        cache:set(key, value)
    end
    etimer()
    ngx.say("time used[lrucache local set]  : ", used_time, " sec")

    do
        stimer()
        for i = 1, tableSize do
            local key = "key_" .. i
            cache:get(key)
        end
        etimer()
        ngx.say("time used[lrucache local get]  : ", used_time, " sec")
    end

end


do
    local cache, err = lrucache_pureffi.new(tableSize)
    stimer()
    for i = 1, tableSize do
        local key = "key_" .. i
        local value = i
        cache:set(key, value)
    end
    etimer()
    ngx.say("time used[lrucache pureffi set]  : ", used_time, " sec")
    do
        stimer()
        for i = 1, tableSize do
            local key = "key_" .. i
            cache:get(key)
        end
        etimer()
        ngx.say("time used[lrucache pureffi get]  : ", used_time, " sec")
    end
end

-- time used[lua table set]  : 0.015999794006348 sec
-- time used[lua table get]  : 0.0060000419616699 sec
-- time used[lrucache local set]  : 0.045000076293945 sec
-- time used[lrucache local get]  : 0.0099999904632568 sec
-- time used[lrucache pureffi set]  : 0.016000032424927 sec
-- time used[lrucache pureffi get]  : 0.014999866485596 sec

原生lua table实现的版本插入要比纯ffi实现的要慢，读的速度则相反，所以读多插入少的场景应该用原生lua实现的版本，反之插入多读少的场景应该用纯ffi实现的版本。

参考

https://yxudong.github.io/%E3%80%8AOpenResty%E7%B2%BE%E5%8D%8E%E6%95%B4%E7%90%86%E3%80%8B6.%E6%80%A7%E8%83%BD%E4%BC%98%E5%8C%96/

http://www.lpq.design/2020/06/lua_table_spec/

https://zhuanlan.zhihu.com/p/295524176

文章目录

table.new(narray, nhash)

pairs() vs ipairs()

table.clear(tab)

table.concat

resty.lrucache vs resty.lrucache.pureffi

参考