|<<
<
>
>>|
/
Introduction to our VDOM.pm & vdom-webkit cluster ---- Introduction to our {{#x|VDOM.pm}} & {{#x|vdom-webkit}} cluster ☺{{#author|agentzh@yahoo.cn}}☺ {{#author|章亦春 (agentzh)}} {{#date|2009.9}} ---- {{#v|VDOM}} ➥ {{#x|Visual}} DOM ➥ DOMs with {{#ci|vision information}} ---- {{#kw|window}} {{#kw|location}}={{#x|"http://foo.bar.com/index.html"}} {{#kw|innerHeight}}=802 {{#kw|innerWidth}}=929 {{#kw|outerHeight}}=943 {{#kw|outerWidth}}=1272 { {{#kw|document}} {{#kw|width}}=914 {{#kw|height}}=5119 { {{#v|...}} } } ---- {{#kw|BODY}} {{#kw|x}}=0 {{#kw|y}}=0 {{#kw|w}}=914 {{#kw|h}}=5119 {{#kw|fontFamily}}=\"Helvetica,Arial,sans-serif\" {{#kw|fontSize}}=\"12px\" {{#kw|fontStyle}}=\"normal\" {{#kw|fontWeight}}=\"400\" {{#kw|color}}=\"rgb(0, 0, 0)\" {{#kw|backgroundColor}}=\"rgb(255, 255, 255)\" { {{#x|"\\n "}} w=0 { } {{#kw|DIV}} {{#kw|id}}=\"append_parent\" {{#kw|x}}=0 {{#kw|y}}=0 {{#kw|h}}=0 {{#kw|backgroundColor}}=\"transparent\" { {{#x|"首页\\n\\n"}} {{#kw|x}}=1 {{#kw|y}}=1 { {{#v|...}} } } {{#x|"\\n "}} w=0 { } } ---- {{#kw|FONT}} {{#kw|color}}=\"rgb(255, 0, 0)\" { {{#kw|B}} {{#kw|fontWeight}}=\"401\" { {{#x|"购物"}} {{#kw|h}}=32 {{#kw|w}}=56 { } } } ---- \"Why {{#ci|another}} language?\" \"Why {{#x|not}} just borrow HTML or XML's syntax?\" ---- {{#cm|✓}} We want to keep VDOM dump size {{#ci|small}}. {{#cm|✓}} We want to keep VDOM dump {{#ci|unambiguous}}. {{#cm|✓}} We want to make VDOM more {{#x|human-readable}} and more {{#x|human-writable}}. (Yeah, XML/HTML's syntax is very {{#i|cumbersome}}.) {{#cm|✓}} We want to make VDOM {{#i|parsers}} & {{#i|dumper}} {{#ci|trivial}} to implement and verify. (tens of lines of Perl for example ;)) {{#cm|✓}} Low level structures like {{#ci|text runs}} and {{#ci|text nodes}} are hard to express naturally in HTML or XML. ---- {{#x|☺}} We've already made both Mozilla {{#x|Gecko}} and Apple {{#x|WebKit}} {{#i|emit}} VDOMs ---- {{img src="images/gen-vdom.png" width="544" height="563"}} ---- {{#cm|# Generate VDOM from the command line:}} {{#v|$}} {{#ci|vdomkit}} --enable-js --proxy=proxy.cn:1080 \\ http://www.sina.com.cn > sina.vdom {{#cm|# Or access our vdomkit FastCGI server directly by HTTP:}} {{#v|$}} curl 'http://vdom.cn.yahoo.com/vdom?{{#x|url=http%3A%2F%2Fwww.sina.com.cn}}' \\ > sina.vdom ---- {{#cm|# The VDOM dump is much smaller than the original HTML:}} {{#v|$}} ls -lh {{#ci|sina.vdom}} -rw------- 1 agentz agentz {{#x|278K}} 2009-04-10 10:30 sina.vdom {{#v|$}} ls -lh {{#ci|sina.html}} -rw-r--r-- 1 agentz agentz {{#x|400K}} 2009-04-10 10:34 sina.html ---- {{#cm|✓}} Now {{#ci|Perl}} enjoys {{#x|very powerful DOMs}} as good as those in JavaScript. ---- {{#kw|use}} VDOM; {{#kw|open my}} {{#v|$in}}, {{#x|"sina.vdom"}} or die {{#v|$!}}; {{#kw|my}} {{#v|$win}} = VDOM::Window->new->parse_file({{#v|$in}}); {{#kw|my}} {{#v|$body}} = {{#v|$win}}->document->body; {{#kw|for my}} {{#v|$child}} ({{#v|$body}}->childNodes) { print {{#v|$child}}->tagName; print {{#v|$child}}->x; print {{#v|$child}}->h; print {{#v|$child}}->color; print {{#v|$child}}->fontFamily; {{#v|...}} } ---- print {{#v|$child}}->nextSibling; {{#v|$win}}->document->getElementById({{#x|"foo"}}); {{#cm|# These are Firefox 3.1 DOM methods, we have too ;)}} print {{#v|$child}}->previousElementSibling; print {{#v|$child}}->firstElementChild; print $child->parentNode; print {{#kw|join}} {{#x|' '}}, {{#kw|map}} { {{#v|$$_}}->href . {{#x|': '}} . {{#v|$$_}}->textContent } {{#v|$child}}->getElmenetsByTagName({{#x|"A"}}); ---- {{img src="images/vdom-pm.png" width="449" height="1176"}} ---- {{img src="images/vdom-pm2.png" width="373" height="285"}} ---- {{#cm|☺}} {{#i|Debug}} our Perl code from within {{#ci|Firefox}} via our {{#x|Visual DOM}} extension ---- {{img src="images/visualdom-ch.png" width="863" height="636"}} ---- {{img src="images/visualdom-ch-cfg.png" width="1016" height="762"}} ---- {{img src="images/between-ff-perl.png" width="726" height="648"}} ---- {{img src="images/visualdom-lh.png" width="1016" height="762"}} ---- {{img src="images/visualdom-lh-cfg.png" width="863" height="636"}} ---- {{#cm|☺}} The {{#ci|qt-webkit port}} of our {{#x|Visual DOM}} extension: {{#i|VDOM Browser}} ---- {{img src="images/vdom-browser-config.png" width="1280" height="1024"}} ---- {{img src="images/ch-eeee.png" width="1280" height="968"}} ---- {{img src="images/ch-bbs-big.png" width="1280" height="968"}} ---- {{img src="images/ch-bbs-guided.png" width="1034" height="658"}} ---- {{img src="images/between-vdombrowser-perl.png" width="611" height="648"}} ---- {{#cm|☺}} We can get geometry information of every {{#ci|text nodes}} in the DOM! ---- {{img src="images/vb-text-nodes.png" width="1047" height="797"}} ---- ...or even as small as {{#ci|text runs}}! (text run is the undividable component of a text node which has no line breaks in it) ---- {{img src="images/vb-text-runs.png" width="1047" height="797"}} ---- {{#cm|☺}} Put everything into a {{#ci|cluster}}. ---- {{img src="images/cluster-arch.png" width="938" height="501"}} ---- {{img src="images/vdomwebkit-farm.png" width="381" height="475"}} ---- {{img src="images/proxy-guts2.png" width="379" height="355"}} ---- {{img src="images/prefetcher-guts.png" width="700" height="600"}} ---- {{img src="images/memcacheq-guts.png" width="253" height="151"}} ---- {{#cm|☺}} Most of the components have been {{#ci|opensourced}} ---- QtWebKit with {{#ci|VDOM support}} ➥ {{http://github.com/agentzh/vdomwebkit/}} ---- vdomkit ({{#ci|command-line}} utility and {{#ci|web}} interface) ➥ {{http://github.com/agentzh/vdomkit/}} ---- VDOM Browser ➥ {{http://github.com/agentzh/vdombrowser/}} ---- VDOM.pm ➥ {{http://github.com/agentzh/vdompm/}} ---- queue-size-aware version of {{#ci|memcacheq}} ➥ {{http://github.com/agentzh/memcacheq/}} ---- Queue::Memcached::Buffered (a {{#ci|Perl client}} for memcacheq) ➥ {{http://github.com/agentzh/queue-memcached-buffered/}} ---- {{#x|Acknowledgements}} {{#x|☺}} haibo++ persuaded me to believe that the {{#ci|separation}} of browser rendering engines and our hunter extractors via VDOM dumping could give rise to {{#ci|lots}} of benefits. {{#x|☺}} jianingy++ effectively {{#ci|fired}} the great WebKit craze in our team. {{#x|☺}} xunxin++ {{#ci|ported}} Visual DOM extension's JavaScript VDOM dumper to qt-webkit C++ and did most of the hard work in {{#ci|vdom-webkit}}. {{#x|☺}} xunxin++ {{#ci|ported}} patched sina's memcacheq to make it aware of queue sizes. {{#x|☺}} mingyou++ shared a great deal of his {{#ci|knowledge}} of the WebKit internals with us and also gave very good suggestions for the slides you're browsing. ---- ☺ {{#ci|Any questions}}? ☺ ----