异步多线程C/S框架gko_pool
曾经在上家公司做了一个内网P2P的传输工具名曰gingko
当时太菜了，网络部分写的很差，各种阻塞式网络通信
后来离职把memcached的代码研究了一下，感觉里面的网络模型非常先进
我就把memcached的网络部分抽取出来和gingko做了结合，写成了一个网络框架叫gko_pool
大致特性是：
- 完全异步化（包括DNS解析）
- 可以listen一个端口当server，也可以主动发起连接写client
- worker是对等的多线程
- 内置一个简单的“内存池”
看代码发现memcached状态机有个逻辑上的bug 顺手给memcached提了个patch
后来这事就被淡忘了，过了两年被维护者发现合到master上了，维护者是这么说的：
Wow, two years old… and it looks correct to me. If that update_event fails the connection might zombie. It’s very hard for that to fail and it’s been that way forever.
Pushed a commit for the next release.
言归正传，可以毫不夸张的说memecached是多线程异步网络编程的一个丰碑（nginx是多进程异步网络编程的一个丰碑）
我就把memcached的网络部分抽取出来写成了一个网络框架叫gko_pool
https://github.com/auxten/gko_pool
举个栗子：http刷票器，在我的笔记本虚机上也能达到300KQPS，控制了一下速度，一不小心就把sina的服务器打挂了
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 |
/* * * Created on: May 18, 2012 * Author: auxten */ #include "gko.h" #include <sys/time.h> /// gingko global stuff s_gingko_global_t gko; pthread_mutex_t lock = PTHREAD_MUTEX_INITIALIZER; int cnt = 3; int counter = 0; void report_result(void * c, const char * msg) { GKOLOG(NOTICE, "%s", msg); pthread_mutex_lock(&lock); if (++counter == cnt) { printf("finished\n"); exit(0); } pthread_mutex_unlock(&lock); } int main(int argc, char** argv) { char cmd[2048]; gko.opt.to_debug = 0; gko.ready_to_serv = 1; gko.sig_flag = 0; gko.opt.worker_thread = 8; gko.opt.connlimit = SERV_POOL_SIZE; // gko.opt.to_debug = 1; gko_pool * gingko = gko_pool::getInstance(); gingko->setPort(-1); gingko->setOption(&gko.opt); // gingko->setProcessHandler(conn_send_data); gingko->setReportHandler(report_result); gingko->gko_run(); char ua[56][256] = { "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20100101 Firefox/15.0.1", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/536.26.14 (KHTML, like Gecko) Version/6.0.1 Safari/536.26.14", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)", "Mozilla/5.0 (Windows NT 5.1; rv:15.0) Gecko/20100101 Firefox/15.0.1", "Mozilla/5.0 (Windows NT 6.1; rv:15.0) Gecko/20100101 Firefox/15.0.1", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4", "Mozilla/5.0 (Linux; U; Android 2.2; fr-fr; Desire_A8181 Build/FRF91) App3leWebKit/53.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:5.0) Gecko/20100101 Firefox/5.0", "Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.112 Safari/535.1", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20100101 Firefox/16.0", "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.0) Opera 7.02 Bork-edition [en]", "Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.02", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.112 Safari/535.1", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) ; .NET CLR 3.5.30729)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322)", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; FunWebProducts; .NET CLR 1.1.4322; PeoplePal 6.2)", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; MRA 5.8 (build 4157); .NET CLR 2.0.50727; AskTbPTV/5.11.3.15590)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727)", "Mozilla/5.0 (Windows NT 6.1; rv:2.0b7pre) Gecko/20100921 Firefox/4.0b7pre", "Mozilla/5.0 (Windows NT 5.1; rv:5.0.1) Gecko/20100101 Firefox/5.0.1", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4", "Opera/9.80 (Windows NT 5.1; U; en) Presto/2.10.289 Version/12.01", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:15.0) Gecko/20100101 Firefox/15.0.1", "Mozilla/5.0 (iPhone; CPU iPhone OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.57.2 (KHTML, like Gecko) Version/5.1.7 Safari/534.57.2", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.92 Safari/537.4", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.26.14 (KHTML, like Gecko) Version/6.0.1 Safari/536.26.14", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4", "Mozilla/5.0 (Windows NT 5.1; rv:13.0) Gecko/20100101 Firefox/13.0.1", "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:15.0) Gecko/20100101 Firefox/15.0.1", "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.89 Safari/537.1", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:15.0) Gecko/20100101 Firefox/15.0.1", "Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10A403 Safari/8536.25", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/536.25 (KHTML, like Gecko) Version/6.0 Safari/536.25", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4", "Mozilla/5.0 (Windows NT 5.1; rv:16.0) Gecko/20100101 Firefox/16.0", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4", "Mozilla/5.0 (iPad; CPU OS 5_1_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B206 Safari/7534.48.3", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4", "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:15.0) Gecko/20100101 Firefox/15.0.1", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534+ (KHTML, like Gecko) BingPreview/1.0b", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20100101 Firefox/14.0.1", }; char srv[3][32] = { "202.108.43.189", "202.106.182.237", "home" }; int i = cnt; while (i--) { struct timeval tv; gettimeofday(&tv, NULL); long t = tv.tv_usec; sprintf(cmd, "GET /Poll.php?project_id=5168&id=49 HTTP/1.%ld\r\n" // "GET /Poll.php?project_id=5168&id=80 HTTP/1.%ld\r\n" "Host: hi.video.sina.com.cn\r\n" "User-Agent: %s\r\n" "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n" "Accept-Language: zh-cn,zh;q=0.8,en-us;q=0.5,en;q=0.3\r\n" "DNT: 1\r\n" "Client-IP: %ld.%ld.%ld.%ld\r\n" "Connection: keep-alive\r\n" "Referer: http://hi.video.sina.com.cn/you/2012banjiti/ph.php?dpc=1\r\n\r\n" , t % 2,ua[t % 56], 122 + t % 20, 121 + (i + t) % 128, 24 + (i + t) % 135, 24 + (i + t) % 213); gingko->make_active_connect(srv[i%3], 80, strlen(cmd), cmd, -1, -1, 0, 10); // gingko->make_active_connect("61.30.127.2", 80, strlen(cmd), cmd, -1, -1, 0, 10); } sleep(10); //由于发请求时异步的，这里也没去处理response，sleep秒再退出 return 0; } |
我为了刷票刷的真实，还写了这么多UserAgent，刷票界的良心啊，哈哈哈
更多例子，比如：一个自定义协议的client、server等请移步github
这套库的稳定性大家不用质疑，已经用在了我现在公司的上万台服务器的很多网络Daemon上
我擦这个牛，被合并到master上了都
最近在看mc，忘不吝赐教。
Narya scab©e::VribÃv, i bimbi hanno vinto bene bene. E manca un rigorino.:-) Mi hanno magnificato la prova di tutti ma particolarmente del buon Alfred che pare sia stato incontenibile vedremo che racconta luciano. Un abbraccio
GAC,Snow in Yosemite can only be blamed on the heavens. Lack of fuel in Canaima is poor management on behalf of the authorities. You can argue that 10 days is nothing. Regardless of that, any impact caused to fellow Venezuelans, small or large due to poor management is unexcusable.
What’s up all, here every person is sharing such experience, therefore it’s nice to
read this weblog, and I used to visit this weblog daily.
mc 的网络模型是 多线程+ libevent?