OpenResty: Your Relational Cloud ---- {{#x|OpenResty}}: Your {{#ci|Relational}} Cloud ☺{{#author|agentzh@yahoo.cn}}☺ {{#author|agentzh (章亦春)}} {{#date|2008.9}} ---- {{#cm|☺}} {{#x|Why}} OpenResty? ---- {{#v|agentzh}}'s {{#x|@boss}} have {{#ci|strong opinions...}} {{img src="#" width="0" height="0"}} {{img src="#" width="0" height="0"}} {{img src="images/boss.png" width="192" height="250"}} ---- {{#v|laser}}: I want a web OS for pure JavaScript sites! {{#v|ting}}: I want a thin web {{#ci|interface}} for our {{#x|Pg cluster}}! {{#v|agentzh}}: ... ---- {{#x|OpenResty}} was {{#ci|born}} since then... ---- {{#x|OpenResty}}'s {{#ci|birthday}}, as seen on the {{#kw|#eeee}} IRC channel... ---- {{#kw|(2007-11-17 01:59:33 PM)}}{{#v|agentzh}}: just created a project for {{#x|OpenAPI}} on openfoundry :) {{#kw|(2007-11-17 02:07:09 PM)}}{{#v|mbot}}: NOTICE: agentzh start to commit to /export/m1/svn/hack {{#kw|(2007-11-17 02:07:10 PM)}}{{#v|mbot}}: NOTICE: {{#ci|created}} the {{#x|openapi}} project {{#kw|(2007-11-17 02:08:44 PM)}}{{#v|mbot}}: NOTICE: [{{#x|openapi}}] initial checkin ---- {{#cm|☺}} Originally {{#ci|Inspired}} by {{#x|Jifty}}'s REST plugin {{img src="images/jifty-rest.png" width="600" height="400"}} ---- ...but have a {{#ci|BIGGER}} ambition. ---- {{#kw|(06:31:53 PM)}}{{#v|ting}}: ft.. 我怎么感觉我们在设计的是 一个{{#x|网络操作系统}}。。...赫赫。。 {{#kw|(06:40:36 PM)}}{{#v|agentzh}}: oh {{#kw|(06:40:42 PM)}}{{#v|agentzh}}: laser's {{#ci|vision}}? {{#kw|(06:40:44 PM)}}{{#v|agentzh}}: *grin* ---- {{#cm|☺}} So {{#x|What}} is {{#ci|OpenResty}} then? ---- {{#v|OpenResty}} is a {{#cm|✓}} {{#ci|general-purpose}} {{#x|RESTful}} web service platform {{#cm|✓}} REST {{#x|wrapper}} for {{#ci|relational}} databases {{#cm|✓}} Virtual web {{#ci|runtime}} for {{#x|RIA}} {{#cm|✓}} \"{{#ci|meta}} web site\" supporting {{#x|other sites}} via web services {{#cm|✓}} handy {{#ci|web database}} which can be accessed from {{#x|anywhere}} ---- {{img src="images/house.png" width="575" height="632"}} ---- {{#cm|☺}} {{#v|agentzh}} loves {{#x|combinations}}, so he designed his API {{#ci|that way}}. ---- {{img src="images/model-comb.png" width="850" height="400"}} ---- {{img src="images/view-comb.png" width="872" height="400"}} ---- {{img src="images/feed-comb.png" width="797" height="400"}} ---- {{img src="images/action-comb.png" width="808" height="400"}} ---- {{img src="images/role-comb.png" width="819" height="400"}} ---- {{#tag|☆}} {{#x|Models}}: Your database {{#ci|tables}} ---- {{#kw|(02:38:03 PM)}}{{#v|laser}}: {{#x|model}} means {{#ci|table}} in RDBMS world, right? {{#kw|(02:38:08 PM)}}{{#v|agentzh}}: yup {{#kw|(02:38:14 PM)}}{{#v|laser}}: and {{#x|column}} means {{#ci|column}} ---- {{img src="images/admin-models.png" width="770" height="632"}} ---- Let's {{#ci|create}} a Post {{#x|model}} for my blog... ---- {{#kw|POST}} {{#x|/=/model/Post}} HTTP/1.0 {{#v|Content-Type}}: {{#x|text/json}} {{#v|Content-Length}}: 111 { {{#kw|"description"}}:\"Blog post\", {{#kw|"columns"}}: [ { {{#kw|"name"}}:\"title\", {{#kw|"label"}}:\"Post title\", {{#kw|"type"}}:\"text\" }, { {{#kw|"name"}}:\"content\", {{#kw|"label"}}:\"Post content\", {{#kw|"type"}}:\"text\" }, { {{#kw|"name"}}:\"author\", {{#kw|"label"}}:\"Post author\", {{#kw|"type"}}:\"text\" }, { {{#kw|"name"}}:\"created\", {{#kw|"default"}}:[\"now()\"], {{#kw|"type"}}:\"timestamp (0) with time zone\", {{#kw|"label"}}:\"Post creation time\" } ], } ---- {{#ci|Insert}} some {{#x|rows}}? ---- {{#kw|POST}} {{#x|/=/model/Post/~/~}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 92 { {{#kw|"title"}}:\"My first post\", {{#kw|"content"}}:\"Blah blah blah...\", {{#kw|"author"}}:\"agentzh\" } ---- Get my {{#ci|rows}} back! ...with {{#x|paging}}... {{#kw|GET}} {{#x|/=/model/Post/~/~?_offset=0&_limit=100}} HTTP/1.0 ---- How about {{#ci|posts}} written by {{#x|agentzh}}? {{#kw|GET}} {{#x|/=/model/Post/author/agentzh}} HTTP/1.0 ---- {{#ci|Posts}} with the title {{#x|containing}} the word {{#v|"OpenResty"}}? {{#kw|GET}} {{#x|/=/model/Post/title/OpenResty?_op=contains}} HTTP/1.0 ---- Okay...I want to {{#ci|add}} a new {{#x|column}} named \"comments\" ---- {{#kw|POST}} {{#x|/=/model/Post/~}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 43 { {{#kw|"name"}}: \"{{#x|comments}}\", {{#kw|"label"}}: \"Number of comments to this post\", {{#kw|"type"}}: \"integer\", {{#kw|"default"}}: 0 } ---- Err...let me see {{#ci|all}} the {{#x|columns}} of the model? {{#kw|GET}} {{#x|/=/model/Post/~}} HTTP/1.0 ---- ...or {{#ci|just}} the definition of the {{#x|author}} column? {{#kw|GET}} {{#x|/=/model/Post/author}} HTTP/1.0 ---- {{#ci|Change}} the {{#x|column name}}? {{#kw|PUT}} {{#x|/=/model/Post/author}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 43 { {{#kw|"name"}}: \"sender\" } ---- Hmm...I'd {{#ci|remove}} this {{#x|column}} from the model! {{#kw|DELETE}} {{#x|/=/model/Post/sender}} HTTP/1.0 ---- {{#tag|☆}} {{#x|Views}}: Your custom {{#ci|SQL}} queries ---- {{#kw|(09:02:20 PM)}}{{#v|laser}}: {{#x|View}} is a very {{#ci|cool}} invention {{#kw|(09:02:30 PM)}}{{#v|laser}}: heh {{#kw|(09:02:52 PM)}}{{#v|laser}}: {{#x|SQL}} is a {{#ci|4th}} generation language ---- {{img src="images/admin-views.png" width="770" height="605"}} ---- {{#kw|POST}} {{#x|/=/view/~}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 312 { {{#kw|"name"}}: \"{{#x|CommentsForAuthor}}\", {{#kw|"description"}}: \"Comments for the specified author\", {{#kw|"definition"}}: \" {{#x|select Comment.sender as guest,}} {{#x|Comment.body as comment_body}} {{#x|from Comment, Post}} {{#x|where Comment.id = Post.id and Post.author = $author}}\" } ---- Get all the {{#ci|comments}} for the {{#x|author}} {{#v|agentzh}} {{#kw|GET}} {{#x|/=/view/CommentsForAuthor/author/agentzh}} HTTP/1.0 Or equivalently {{#kw|GET}} {{#x|/=/view/CommentsForAuthor/~/~?author=agentzh}} HTTP/1.0 ---- The HTTP {{#ci|response}} might be ---- HTTP/1.0 {{#kw|200 OK}} {{#v|Connection}}: close {{#v|Content-Type}}: text/json; charset=UTF-8 {{#v|Content-Length}}: 187 {{#v|Date}}: Mon, 21 Apr 2008 12:42:15 GMT [ { {{#kw|"guest"}}:\"laser\",{{#kw|"comment_body"}}:\"super cool!\" }, { {{#kw|"guest"}}:\"carriezh\",{{#kw|"comment_body"}}:\"hello?hello?\" }, { {{#kw|"guest"}}:\"agentzh\":{{#kw|"comment_body"}}:\"Thanks for commenting!\" } ] ---- {{#tag|☆}} {{#x|Feeds}}: Your {{#ci|RSS/Atom}} for views. ---- {{#kw|(10:38:28 AM)}}{{#v|agentzh}}: at present, {{#x|feed}} objexts are just {{#ci|XML wrappers}} for {{#x|views}}. {{#kw|(10:38:31 AM)}}{{#v|agentzh}}: heh {{#kw|(10:38:37 AM)}}{{#v|agentzh}}: i like this idea. (10:38:28 AM) agentzh: 目前，Feed 对象就是 View 对象的 XML 包裹 (10:38:31 AM) agentzh: 呵呵 (10:38:37 AM) agentzh: 这个概念我挺喜欢的 ---- {{img src="images/admin-feeds.png" width="770" height="605"}} ---- We first define a {{#ci|view}} to serve as the {{#x|data source}} ---- {{#kw|POST}} {{#x|/=/view/PostFeedView}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 312 { {{#kw|"description"}}: \"View for post feed\", {{#kw|"definition"}}: \" {{#x|select author, title,}} {{#x|content, created as published}} {{#x|from Post}} {{#x|order by created desc}} {{#x|limit $count | 20}} \" } ---- Then we create a {{#ci|feed object}} atop this view: ---- {{#kw|POST}} {{#x|/=/feed/Post}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 124 { {{#kw|"description"}}: \"Feed for blog posts\", {{#kw|"author"}}: \"agentzh\", {{#kw|"copyright"}}: \"Copyright 2008 by Yahoo! China EEEE Works\", {{#kw|"language"}}: \"zh-cn\", {{#kw|"title"}}: \"Posts for Human & Machine\", {{#kw|"link"}}: \"http://blog.agentzh.org\", {{#kw|"logo"}}: \"http://blog.agentzh.org/me.jpg\", {{#kw|"view"}}: {{#x|"PostFeedView"}} } ---- The {{#ci|feed}} is ready for {{#x|subscribers}}! {{#kw|GET}} {{#x|/=/feed/Post/~/~}} HTTP/1.0 ---- Or it can take {{#ci|arguments}} just as the underlying {{#x|view}}: {{#kw|GET}} {{#x|/=/feed/Post/count/20}} HTTP/1.0 ---- {{#tag|☆}} {{#x|Actions}}: {{#ci|Compose}} services the way you like ---- {{#kw|(03:17:11 PM)}}{{#v|agentzh}}: {{#x|action}} support is the next milestone {{#kw|(03:17:28 PM)}}{{#v|agentzh}}: so what is an action? {{#kw|(03:17:47 PM)}}{{#v|agentzh}}: an openresty action is a {{#ci|bunch}} of {{#x|restyscript commands}} with a {{#ci|name}} attached to it. ---- {{#ci|Compose}} several {{#x|view invocations}} into one action: {{#kw|POST}} {{#x|/=/action/Init}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 224 { {{#kw|"description"}}: \"Initialization for my site\", {{#kw|"parameters"}}: [], {{#kw|"definition"}}: \" {{#x|GET '/=/view/RecentComments/~/~';}} {{#x|GET '/=/view/RecentPosts/~/~';}} {{#x|GET '/=/model/Post/~/~?_offset=0&_limit=10';}} \" } ---- The server returns the {{#ci|result sets}} for all the {{#x|view calling}}: [ [ {{#ci|...}} ], {{#cm|// the results for the first view}} [ {{#ci|...}} ], {{#cm|// the results for the second view}} [ {{#ci|...}} ] {{#cm|// the results for the third view}} ] ---- {{#cm|♡}} You can also {{#ci|mix}} {{#x|SQL}}, {{#x|HTTP commands}}, and {{#x|JSON}} in one action! ---- {{#kw|POST}} {{#x|/=/action/~}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 189 { {{#kw|"name"}}: \"{{#x|PostComment}}\", {{#kw|"description"}}: \"Action for posting a comment\", {{#kw|"parameters"}}: [ { {{#kw|"name"}}:\"sender\", {{#kw|"type"}}:\"literal\" }, { {{#kw|"name"}}:\"body\", {{#kw|"type"}}:\"literal\" }, { {{#kw|"name"}}:\"post_id\", {{#kw|"type"}}:\"literal\" }, ], ---- {{#kw|"definition"}}: \" {{#x|update Post}} {{#x|set comments = comments + 1}} {{#x|where id = $post_id;}} {{#x|POST '/=/model/Comment/~/~'}} {{#x| { "sender": $sender, "body": $body, "post": $post_id } }} \" } ---- {{#tag|☆}} {{#x|Roles}}: Your custom {{#ci|access control}} (at service level) ---- {{img src="images/admin-roles.png" width="770" height="543"}} ---- Create a {{#ci|read-only}} role {{#kw|POST}} {{#x|/=/role/Reader}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 120 { {{#kw|"description"}}: \"My Read-only role\", {{#kw|"login"}}: \"anonymous\", {{#cm|// or password or captcha}} {{#kw|"password"}}: null } ---- The {{#x|Reader}} role {{#ci|cannot}} access any resources by default, let's {{#ci|add}} some {{#v|ACL rules}} to it. ---- {{#kw|POST}} {{#x|/=/role/Reader/~/~}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 82 [ {{#cm|// allow reading object lists:}} { {{#kw|"method"}}: \"GET\", {{#kw|"url"}}: \"/=/~\" }, {{#cm|// allow reading objects:}} { {{#kw|"method"}}: \"GET\", {{#kw|"url"}}: \"/=/~/~\" }, {{#cm|// allow reading object columns/parameters:}} { {{#kw|"method"}}: \"GET\", {{#kw|"url"}}: \"/=/~/~/~\" }, {{#cm|// allow reading object rows/rules:}} { {{#kw|"method"}}: \"GET\", {{#kw|"url"}}: \"/=/~/~/~/~\" } ] ---- {{#ci|Reader}} roles in action {{#x|(allowed)}}: {{#cm|✓}} {{#kw|GET}} {{#v|/=/model/Post/id/3?_user=agentzh.Reader}} {{#cm|✓}} {{#kw|GET}} {{#v|/=/view?_user=agentzh.Reader}} {{#cm|✓}} {{#kw|GET}} {{#v|/=/role/Reader/method/GET?_user=agentzh.Reader}} {{#cm|✓}} {{#kw|GET}} {{#v|/=/role/Reader?_user=agentzh.Reader}} ---- {{#ci|Reader}} roles in action {{#x|(denied)}}: {{#x|×}} {{#kw|DELETE}} {{#v|/=/model/Post?_user=agentzh.Reader}} {{#x|×}} {{#kw|PUT}} {{#v|/=/model/Post/author?_user=agentzh.Reader}} {{#x|×}} {{#kw|POST}} {{#v|/=/role/Reader/~/~?_user=agentzh.Reader}} ---- {{#tag|☆}} {{#x|Captchas}}: Make your services {{#ci|human-only}} ---- {{#kw|(10:34:50 AM)}}{{#v|Carrie}}: 恩。。。如果我想在{{#ci|发留言}}的时候 {{#kw|(10:34:56 AM)}}{{#v|Carrie}}: 增加个{{#ci|水印}}标志 {{#kw|(10:36:44 AM)}}{{#v|Carrie}}: 我想咱们可能得先实现这个水印的东西。。 。。。因为貌似很需要。。。。 {{#kw|(10:36:56 AM)}}{{#v|agentzh}}: {{#x|验证码图片}}属于一种特殊的{{#ci|角色登录}}方式 {{#kw|(10:37:14 AM)}}{{#v|agentzh}}: 该角色的\"登录密码\"将不放在应用的 JS 中 {{#kw|(10:37:21 AM)}}{{#v|agentzh}}: 而是指向一个验证码 ---- {{img src="images/yisou-captcha.png" width="744" height="479"}} ---- Create an ad-hoc {{#x|role}} with the {{#ci|captcha}} login method {{#kw|POST}} {{#x|/=/role/CommentPoster}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 64 { {{#kw|"description"}}:\"Role for posting comments\", {{#kw|"login"}}:{{#x|"captcha"}} } ---- Assign {{#ci|access}} to the {{#x|operations}} required solving captchas {{#kw|POST}} {{#x|/=/role/CommentPoster/~/~}} HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 48 { {{#kw|"method"}}:\"POST\", {{#kw|"url"}}:\"/=/model/Comment/~/~\" } ---- Then in the {{#ci|client-side}} JavaScript code... {{#kw|GET}} /=/captcha/id {{#cm|# get the captcha id}} {{#kw|GET}} /=/captcha/id/{{#x|[captcha_id]}} {{#cm|# get the captcha image}} {{#kw|POST}} /=/model/Comment/~/~?_user=agentzh.{{#x|CommentPoster}} \\ &_captcha={{#x|[captcha_id]}}:{{#x|[user_solution]}} \\ HTTP/1.0 {{#v|Content-Type}}: text/json {{#v|Content-Length}}: 52 {\"sender\":\"agentzh\",\"body\":\"Good post!\",\"post\":25} ---- {{#v|OpenResty}} offers {{#cm|✓}} (scalable) {{#ci|relational}} data {{#x|storage}} {{#cm|✓}} {{#ci|truely}} {{#x|RESTy}} interface {{#cm|✓}} {{#ci|JSON/YAML}} data transfer format {{#cm|✓}} {{#x|SQL}}-based reusable {{#ci|views}} {{#cm|✓}} a REST-oriented {{#ci|role system}} for {{#x|ACL}} {{#cm|✓}} {{#ci|view}}-based {{#x|RSS feeds}} {{#cm|✓}} user-defined {{#ci|actions}} in {{#x|RestyScript}} {{#cm|✓}} native {{#ci|captchas}} {{#cm|✓}} {{#ci|cross-site}} {{#x|AJAX}} support ---- {{#x|♡}} The OpenResty {{#ci|FastCGI}} server is currently written in {{#x|Perl 5}}. ---- {{img src="images/resty-guts.png" width="914" height="695"}} ---- {{#x|♡}} The {{#x|PgFarm}} backend of OpenResty is designed to be {{#ci|scalable}}. ---- {{img src="images/cluster-arch.png" width="712" height="503"}} ---- {{#cm|☺}} {{#x|OpenResty}} comes with a {{#ci|pony}} (just like {{#x|Jifty}}) {{img src="images/pony.jpg" width="300" height="200"}} ---- {{#kw|(04:49:43 PM)}}{{#v|Carrie}}: 可是你不觉得你现在应该先写出一个{{#x|admin page}}来么 {{#kw|(04:49:55 PM)}}{{#v|Carrie}}: 要不大家以后用你的api就跟{{#ci|背课文}}似的。。。。 ---- {{img src="images/admin-login.png" width="827" height="618"}} ---- {{img src="images/admin-index.png" width="827" height="618"}} ---- {{#x|☺}} It's written in {{#ci|100%}} {{#x|JavaScript}} as well. ---- CPAN modules {{#ci|born}} from {{#x|OpenResty}} {{#cm|✓}} CGI::Cookie::XS {{#cm|✓}} Filter::QuasiQuote ---- OpenResty {{#ci|handlers}} in the {{#x|future}}: {{#x|♡}} /=/mail/... {{#x|♡}} /=/attachment/... {{#x|♡}} /=/template/... {{#x|♡}} /=/git/...? {{#x|♡}} /=/prophet/...? ...{{#ci|Any one?}} ---- OpenResty {{#ci|backends}} in the {{#x|future}}: {{#x|♡}} MySQL {{#x|♡}} Oracle {{#x|♡}} MS SQL Server {{#x|♡}} {{#ci|Put your favorite RDBMS here...}} ---- Or {{#ci|peer-to-peer}} OpenResty {{#x|clusters}} in the future? A {{#ci|cluster}} of {{#ci|clusters}}? ---- {{img src="images/p2p.png" width="528" height="417"}} ---- {{#kw|☺}} {{#ci|Any questions}}? {{#kw|☺}}