|<<
<
>
>>|
/
Recent {{#i|developments}} in {{#x|nginx.conf}} scripting ☺{{#author|agentzh@gmail.com}}☺ {{#author|章亦春 (agentzh)}} {{#date|2010.6}} ---- {{#x|♡}} {{#x|Background}} reading {{http://agentzh.org/misc/slides/nginx-conf-scripting/}} {{img src="images/perlchina_org.gif" width="297" height="75"}} ---- {{#x|♡}} Handle {{#ci|POST request bodies}} painlessly in nginx.conf by means of {{#x|ngx_form_input}}! Thanks our new teammate, Calio Zhi! {{http://github.com/calio/form-input-nginx-module}} ---- {{img src="images/post-form.png" width="630" height="297"}} ---- <form method=\"{{#x|post}}\" action=\"{{#x|/login}}\"> <input name=\"{{#x|name}}\" type=\"text\" /> <input name=\"{{#x|pass}}\" type=\"password\" /> <input type=\"submit\" value=\"Login\" /> </form> ---- {{#kw|location}} = /login { {{#kw|set_form_input}} {{#v|$user}} {{#x|'user'}}; {{#kw|set_form_input}} {{#v|$password}} {{#x|'pass'}}; {{#kw|if}} ({{#v|$user}} != {{#x|'agentzh'}}) { {{#kw|return}} 403; {{#kw|break}}; } {{#kw|if}} ({{#v|$password}} != {{#x|'123456'}}) { {{#kw|return}} 403; {{#kw|break}}; } {{#kw|echo}} {{#x|"You're logged in!"}}; } ---- {{#x|♡}} {{#ci|ngx_postgres}} has {{#x|already}} landed. Thanks Piotr Sikora! {{http://github.com/FRiCKLE/ngx_postgres}} {{img src="images/elephant.png" width="143" height="147"}} ---- {{#cm|# configure the PostgreSQL upstream backend}} {{#kw|upstream}} my_pg_backend { {{#kw|postgres_server}} {{#x|10.62.136.3:5432}} dbname={{#x|test}} user={{#x|someone}} password={{#x|123456}}; } ---- {{#kw|location}} /cats { {{#kw|postgres_query}} {{#x|'select * from cats'}}; {{#kw|postgres_pass}} {{#x|my_pg_backend}}; {{#kw|rds_json}} on; } ---- {{#v|$}} curl {{#x|'localhost/cats'}} [{\"name\":\"Marry\",\"age\":32},{\"name\":\"Bob\",\"age\":12}] ---- {{#x|♡}} Everything is also {{#ci|non-blocking}} as ngx_drizzle. Thanks to {{#x|libpq}}'s nonblocking API! ---- {{#ci|☺}} Let's extract a specific {{#ci|cell}} in the result set! {{img src="images/excel-cell.jpg" width="454" height="353"}} ---- {{#cm|# How many cats are there?}} {{#kw|location}} /count { {{#kw|default_type}} {{#x|'text/plain'}}; {{#kw|postgres_query}} {{#x|'select count(*) from cats'}}; {{#kw|postgres_pass}} {{#x|my_pg_backend}}; {{#cm|# extract the cell in the result set}} {{#cm|# in the first column, first row.}} {{#kw|postgres_output}} {{#x|value 0 0}}; } ---- {{#v|$}} curl {{#x|'localhost/count'}} 2 ---- {{#ci|☺}} or even {{#ci|save}} a cell in the result set into an nginx {{#x|variable}} ---- {{#cm|# read a value from Pg and save it into}} {{#cm|# a cookie}} {{#kw|location}} /foo { {{#kw|postgres_query}} {{#x|"select signature}} {{#x|from users where user='foo'"}}; {{#kw|postgres_pass}} {{#x|my_pg_backend}}; {{#cm|# extract the cell in the result set}} {{#cm|# in the first column, first row.}} {{#kw|postgres_set}} {{#v|$signature}} 0 0; {{#kw|add_header}} Set-Cookie {{#x|"sig=}}{{#v|$signature}}{{#x|; path=/"}}; } ---- {{#x|♡}} Construct fully {{#ci|RESTful}} queries in a {{#x|single}} location ---- {{#kw|location}} ~ {{#x|'^/cat/(\d+)'}} { {{#kw|set}} {{#v|$id}} {{#v|$1}}; {{#kw|set_form_input}} {{#v|$name}}; {{#kw|set_quote_sql_str}} {{#v|$quoted_name}} {{#v|$name}}; {{#kw|postgres_query}} GET {{#x|"select * from cats where id=}}{{#v|$id}}{{#x|"}}; {{#kw|postgres_query}} DELETE {{#x|"delete from cats where id=}}{{#v|$id}}{{#x|"}}; {{#kw|postgres_query}} POST {{#x|"insert into cats (id, name) values(}}{{#v|$id}}{{#x|, }}{{#v|$quoted_name}}{{#x|)"}}; {{#kw|postgres_pass}} my_pg_backend; } ---- {{#x|♡}} Qunar.com is running {{#ci|ngx_postgres}} + {{#ci|ngx_rds_json}} in {{#x|production}}. Thanks Liseen Wan's promotion! ---- {{img src="images/qunar-hotel.png" width="928" height="590"}} ---- {{img src="images/qunar-pinyin.png" width="928" height="590"}} ---- {{#x|♡}} {{#ci|Caching}} database responses using {{#x|memcached}} via ngx_srcache and ngx_memc. {{http://github.com/agentzh/srcache-nginx-module}} ---- {{#cm|# enable the ngx_srcache and other}} {{#cm|# modules in your nginx build}} {{#v|$}} ./configure --prefix=/opt/nginx \ --add-module=/path/to/{{#x|srcache-nginx-module}} \ --add-module=/path/to/rds-json-nginx-module \ --add-module=/path/to/memc-nginx-module \ --add-module=/path/to/drizzle-nginx-module ---- {{#x|♡}} It's very important to put {{#x|ngx_srcache}} {{#ci|before}} {{#x|ngx_rds_json}} during nginx configure so that we cache the final JSON rather than RDS. ---- {{#cm|# configure the mysql upstream backend}} {{#kw|upstream}} mysql_backend { {{#kw|drizzle_server}} {{#x|127.0.0.1:3306}} dbname={{#x|test}} password={{#x|some_pass}} user={{#x|monty}} protocol={{#x|mysql}}; } ---- {{#cm|# configure the cache storage location}} {{#kw|location}} /memc { {{#kw|internal}}; {{#kw|set}} {{#v|$memc_key}} {{#v|$query_string}}; {{#kw|set}} {{#v|$memc_exptime}} 300; {{#kw|memc_pass}} 127.0.0.1:11211; } ---- {{#kw|location}} /cats { {{#kw|srcache_fetch}} {{#x|GET /memc}} {{#v|$uri}}; {{#kw|srcache_store}} {{#x|PUT /memc}} {{#v|$uri}}; {{#kw|default_type}} {{#x|application/json}}; {{#kw|drizzle_pass}} mysql_backend; {{#kw|drizzle_query}} 'select * from cats'; {{#kw|rds_json}} on; } ---- {{#v|$}} curl {{#x|'localhost/cats'}} [{\"name\":\"Marry\",\"age\":32},{\"name\":\"Bob\",\"age\":12}] ---- {{#cm|# if it is a cache miss}} {{#v|$}} memcached -vvv -p 11211 ... <10 new client connection <10 {{#x|get /cats}} > {{#c|NOT FOUND /cats}} >10 END <10 connection closed. <10 new client connection <10 {{#x|set /cats 0 300 44}} > NOT FOUND /cats >10 {{#c|STORED}} <10 connection closed. ---- {{#cm|# if it is a cache hit}} {{#v|$}} memcached -vvv -p 11211 ... <10 new client connection <10 {{#x|get /cats}} > {{#c|FOUND KEY /cats}} >10 sending key /cats >10 END <10 connection closed. ---- {{#x|♡}} You can even cache {{#ci|subrequests}}' responses out of the box! ---- {{img src="images/subrequests.png" width="856" height="391"}} ---- {{#kw|location}} = /main { {{#kw|echo}} -n {{#x|"[\n "}}; {{#kw|echo_location}} {{#x|/cats dir=asc}}; {{#kw|echo}} -n {{#x|",\n "}}; {{#kw|echo_location}} {{#x|/cats dir=desc}}; {{#kw|echo}} {{#x|"\n]"}}; } ---- {{#kw|location}} = /cats { {{#kw|internal}}; {{#kw|set}} {{#v|$key}} {{#x|"}}{{#v|$uri}}{{#x|-}}{{#v|$arg_dir}}{{#x|"}}; {{#kw|srcache_fetch}} {{#x|GET}} {{#x|/memc}} {{#v|$key}}; {{#kw|srcache_store}} {{#x|PUT}} {{#x|/memc}} {{#v|$key}}; {{#kw|default_type}} application/json; {{#kw|drizzle_query}} {{#x|'select * from cats order by id }}{{#v|$arg_dir}}{{#x|'}}; {{#kw|drizzle_pass}} backend; {{#kw|rds_json}} on; } ---- {{#v|$}} curl {{#x|'localhost/main'}} [ [{\"id\":2,\"name\":null},{\"id\":3,\"name\":\"bob\"}], [{\"id\":3,\"name\":\"bob\"},{\"id\":2,\"name\":null}] ] ---- {{#x|♡}} Theoretically speaking, you can cache {{#ci|arbitrary}} nginx locations using ngx_srcache! ---- {{#kw|location}} /blah { {{#kw|set}} {{#v|$key}} {{#x|"}}{{#v|$uri}}{{#x|-}}{{#v|$arg_foo}}{{#x|-}}{{#v|$arg_bar}}{{#x|"}}; {{#kw|srcache_fetch}} {{#x|GET /memc}} {{#v|$key}}; {{#kw|srcache_store}} {{#x|PUT /memc}} {{#v|$key}}; {{#cm|# proxy_pass/fastcgi_pass/postgres_pass/echo/...}} } ---- {{#x|♡}} {{#ci|Access control}} using {{#x|ngx_encrypted_session}} {{http://github.com/agentzh/encrypted-session-nginx-module}} ---- {{#cm|# private key used in AES (with MAC) encryption/decryption:}} {{#kw|encrypted_session_key}} {{#x|"abcdefghijklmnopqrstuvwxyz123456"}}; {{#cm|# init vector setting:}} {{#kw|encrypted_session_iv}} {{#x|"12345678"}}; {{#cm|# expiration time for generated sessions}} {{#cm|# (in seconds by default):}} {{#kw|encrypted_session_expires}} {{#x|1d}}; {{#cm|# here we specify 1 day}} ---- {{#kw|location}} = /login { {{#kw|eval_subrequest_in_memory}} {{#x|off}}; {{#kw|eval_override_content_type}} {{#x|text/plain}}; {{#kw|eval}} {{#v|$uid}} { {{#kw|postgres_query}} {{#x|"select id}} {{#x|from users}} {{#x|where name=}}{{#v|$arg_name}}{{#x| and pass=}}{{#v|$arg_pass}}{{#x|"}}; {{#kw|postgres_pass}} pg_backend; {{#kw|postgres_output}} value 0 0; } {{#kw|if}} ({{#v|$uid}} !~ {{#x|'^\d+$'}}) { {{#kw|rewrite}} ^ {{#x|/relogin}} {{#kw|redirect}}; {{#kw|break}}; } {{#kw|set_encrypt_session}} {{#v|$session}} {{#v|$uid}}; {{#kw|set_encode_base32}} {{#v|$session}}; {{#kw|add_header}} {{#x|Set-Cookie}} {{#x|"session=}}{{#v|$session}}{{#x|; path=/"}}; {{#kw|rewrite}} ^ {{#x|/main}} {{#kw|redirect}}; {{#kw|break}}; } ---- {{#kw|location}} = /some_api { {{#kw|include}} {{#x|conf/access.conf}}; {{#cm|# your normal content handler settings like}} {{#cm|# fastcgi_pass/postgres_pass/proxy_pass}} } ---- {{#cm|# conf/access.conf }} {{#kw|if}} ({{#v|$cookie_session}} = {{#x|''}}) { {{#kw|rds_json_ret}} 100 \"Login required\"; {{#kw|break}}; } {{#kw|set_unescape_uri}} {{#v|$session $cookie_session}}; {{#kw|set_decode_base32}} {{#v|$session}}; {{#kw|if}} ({{#v|$session}} = {{#x|''}}) { {{#kw|rds_json_ret}} 100 \"Login required\"; {{#kw|break}}; } {{#kw|set_decrypt_session}} {{#v|$uid $session}}; {{#kw|if}} ({{#v|$uid}} !~ {{#x|'^\d+$'}}) { {{#kw|rds_json_ret}} 100 \"Login required\"; {{#kw|break}}; } ---- {{#x|♡}} {{#x|ngx_lua}} is quite usable {{#ci|now}}! {{http://github.com/chaoslawful/lua-nginx-module}} {{img src="images/lua.jpg" width="118" height="118"}} ---- {{#x|♡}} chaoslawful is {{#ci|crazy}}! {{img src="images/chaoslawful.jpg" width="120" height="120"}} ---- {{#cm|# first install lua (or even luajit) into your system...}} {{#cm|# enable the ngx_lua module in your nginx build}} {{#v|$}} ./configure --prefix=/opt/nginx \ --add-module=/path/to/ngx_devel_kit \ --add-module=/path/to/echo-nginx-module \ --add-module=/path/to/{{#x|lua-nginx-module}} ---- {{#kw|location}} = /adder { {{#kw|set_by_lua}} {{#v|$res}} {{#x|"local a = tonumber(ngx.arg[1])}} {{#x|local b = tonumber(ngx.arg[2])}} {{#x|return a + b"}} {{#v|$arg_a}} {{#v|$arg_b}}; {{#kw|echo}} {{#v|$res}}; } ---- {{#v|$}} curl {{#x|'localhost/adder?a=25&b=75'}} 100 ---- {{#kw|location}} = /fib { {{#kw|set_by_lua}} {{#v|$res}} {{#x|"}} {{#x|function fib(n)}} {{#x|if n > 2 then}} {{#x|return fib(n-1) + fib(n-2)}} {{#x|else}} {{#x|return 1}} {{#x|end}} {{#x|end}} {{#x|local num = tonumber(ngx.arg[1])}} {{#x|return fib(num)}} {{#x|"}} {{#v|$arg_n}}; {{#kw|echo}} {{#v|$res}}; } ---- {{#v|$}} curl {{#x|'localhost/fib?n=10'}} 55 ---- {{#x|♡}} or use {{#ci|external}} Lua script file... ---- {{#kw|location}} = /fib { {{#kw|set_by_lua_file}} {{#v|$res}} {{#x|"conf/fib.lua"}} {{#v|$arg_n}}; {{#kw|echo}} {{#v|$res}}; } ---- {{#cm|-- conf/fib.lua file}} {{#kw|function}} fib(n) {{#kw|if}} n > 2 {{#kw|then}} {{#kw|return}} fib(n-1) + fib(n-2) {{#kw|else}} {{#kw|return}} 1 {{#kw|end}} {{#kw|end}} {{#kw|local}} num = {{#kw|tonumber}}(ngx.arg[1]) {{#kw|return}} fib(num) ---- {{#v|$}} {{#ci|Complex}} database cluster {{#x|hashing}} can also be done in Lua ---- {{#kw|http}} { {{#kw|upstream}} {{#x|A}} { {{#kw|drizzle_server}} ...; } {{#kw|upstream}} {{#x|B}} { {{#kw|drizzle_server}} ...; } {{#kw|upstream}} {{#x|C}} { {{#kw|drizzle_server}} ...; } ... } ---- {{#kw|location}} ~ {{#x|'^/user/(\d+)'}} { {{#kw|set}} {{#v|$uid}} {{#v|$1}}; {{#kw|set_by_lua_file}} {{#v|$backend}} {{#x|"conf/hash.lua"}} {{#v|$uid}}; {{#kw|if}} ({{#v|$backend}} = {{#x|''}}) { {{#kw|return}} 400; {{#kw|break}}; } {{#kw|drizzle_query}} {{#x|"select * from users}} {{#x|where uid=}}{{#v|$uid}}{{#x|"}}; {{#kw|drizzle_pass}} {{#v|$backend}}; {{#kw|rds_json}} on; } ---- {{#cm|-- hash.lua}} {{#kw|function}} hash(uid) {{#kw|if}} uid > 0 {{#kw|and}} uid <= 1200 {{#kw|then}} {{#kw|return}} {{#x|'A'}} {{#kw|end}} {{#kw|if}} uid > 1200 {{#kw|and}} uid <= 5300 {{#kw|then}} {{#kw|return}} {{#x|'B'}} {{#kw|end}} {{#kw|if}} uid > 5300 {{#kw|and}} uid <= 7100 {{#kw|then}} {{#kw|return}} {{#x|'C'}} {{#kw|end}} {{#kw|return}} '' {{#kw|end}} {{#kw|return}} hash({{#kw|tonumber}}(ngx.arg[1])) ---- {{#x|♡}} Use {{#x|Lua}} to code up nginx {{#ci|content handler}} directly ---- {{#kw|location}} = /lua { {{#kw|content_by_lua}} {{#x|'ngx.say("Hello, Lua!")'}}; } ---- {{#v|$}} curl 'localhost/lua' {{#x|Hello, Lua!}} ---- {{#x|♡}} ...and we can read arbitrary {{#ci|nginx variables}} from within our Lua {{#x|content}} handler! ---- {{#kw|location}} = /hello { {{#kw|content_by_lua}} {{#x|'local who = ngx.var.arg_who}} {{#x|ngx.say("Hello, ", who, "!")';}} } ---- {{#v|$}} curl 'localhost/hello?who={{#x|agentzh}}' Hello, {{#x|agentzh}}! ---- {{#x|♡}} We can also put Lua code into {{#ci|external}} .lua {{#x|file}} to eliminate escaping nightmare. ---- {{#kw|location}} /foo { ... {{#kw|content_by_lua_file}} {{#x|/path/to/your/lua-file.lua}}; } ---- {{#x|♡}} We can also issue nginx {{#ci|subrequests}} diredctly from within Lua content handler {{#i|now}}! ---- {{#kw|location}} /other { {{#kw|echo}} \"hello, world\"; } {{#cm|# transparent non-blocking I/O in Lua}} {{#kw|location}} /lua { {{#kw|content_by_lua}} {{#x|'}} {{#x|local res = ngx.location.capture("/other")}} {{#x|if res.status == 200 then}} {{#x|ngx.print(res.body)}} {{#x|end'}}; } ---- {{#v|$}} curl 'localhost/lua' {{#x|hello, world}} ---- {{#x|♡}} We'd call this whole set of nginx modules {{#ci|ngx_openresty}} and our work is heavily {{#x|funded}} by {{#i|Taobao.com}}. {{img src="images/taobao-logo.png" width="140" height="35"}} ---- {{#x|♡}} It is already {{#x|powering}} lz.taobao.com. {{img src="images/lz-logo.gif" width="234" height="47"}} ---- {{img src="images/lz-login.png" width="1059" height="723"}} ---- {{img src="images/lz-shopsummary.png" width="1059" height="723"}} ---- {{img src="images/lz-itemflow.png" width="1059" height="723"}} ---- {{img src="images/lz-srccomp.png" width="1059" height="723"}} ---- {{#x|♡}} {{#ci|Join}} us at the {{#x|OpenResty}} Google Group {{http://groups.google.com/group/OpenResty}} and the {{#x|nginx-devel}} mailing list {{http://nginx.org/mailman/listinfo/nginx-devel}} ---- {{#x|♡}} or just {{#ci|catch}} us on IRC: irc.freenode.net {{#x|#nginx}} {{#x|#openresty}} ---- {{#ci|☺}} {{#i|Any questions}}? {{#ci|☺}}