濡備綍鍋氫竴涓ソ鐨勫ぇ鏁版嵁骞冲彴鏋舵瀯 鍖椾含杩炲北绉戞妧鑲′唤鏈夐檺鍏徃-凯发娱乐

涓€銆丩ambda鏋舵瀯闇€姹?/h3>

lambda鏋舵瀯鑳屽悗鐨勯渶姹傛槸鐢变簬mr鏋舵瀯鐨勫欢杩熼棶棰樸€侻r铏界劧瀹炵幇浜嗗垎甯冨紡銆佸彲鎵╁睍鏁版嵁澶勭悊绯荤粺鐨勭洰鐨勶紝浣嗘槸鍦ㄥ鐞嗘暟鎹椂寤惰繜姣旇緝涓ラ噸銆傚疄闄呬笂濡傛灉鍐呭瓨鍜孋pu瓒冲寮哄ぇ锛孧r涔熷彲浠ュ疄鐜拌繎瀹炴椂杩愮畻锛屼絾瀹為檯涓氬姟鐜骞堕潪濡傛锛屽洜姝ゆ垜浠渶瑕佹潈琛★紝閫夋嫨瀹炴椂澶勭悊鍜屾壒澶勭悊鎵€闇€瑕佹暟鎹噺鍜屾伆褰撶殑璧勬簮銆?/span>
 

2012骞碨torm鐨勪綔鑰匩athan marz鎻愬嚭鐨凩ambda鏁版嵁澶勭悊妗嗘灦銆侺ambda鏋舵瀯鐨勭洰鏍囨槸璁捐鍑轰竴涓兘婊¤冻瀹炴椂澶ф暟鎹郴缁熷叧閿壒鎬х殑鏋舵瀯锛屽寘鎷湁锛氶珮瀹归敊銆佷綆寤舵椂鍜屽彲鎵╁睍绛夈€侺ambda鏋舵瀯鏁村悎绂荤嚎璁＄畻鍜屽疄鏃惰绠楋紝铻嶅悎涓嶅彲鍙樻€?immunability)锛岃鍐欏垎绂诲拰澶嶆潅鎬ч殧绂荤瓑涓€绯诲垪鏋舵瀯鍘熷垯锛屽彲闆嗘垚hadoop锛孠afka锛孲torm锛孲park锛孒base绛夊悇绫诲ぇ鏁版嵁缁勪欢銆?/span>
 

浜屻€丩ambda鏋舵瀯鐨勫叧閿?/h3>

 

 

妯悜鎵╁

鍙墿灞曟€ф剰鍛崇潃涓烘弧瓒虫棩鐩婂闀跨殑鐢ㄦ埛鏈嶅姟闇€姹傦紝鍚屾椂涓嶇敤瀵瑰簳灞傛灦鏋勬垨鑰呬唬鐮侊紝鍙互閫氳繃鐜版湁鏈哄櫒娣诲姞鍐呭瓨鎴栬€呯鐩樿祫婧愭潵瀹炵幇(鍨傜洿鎵╁睍)锛屾垨鑰呭彲浠ラ€氳繃鍦ㄩ泦缇や腑娣诲姞鏈哄櫒瀹炵幇(姘村钩鎵╁睍)銆傛棤璁烘槸瀹炴椂鎴栬€呮壒澶勭悊锛岄兘搴旇鑳藉涓嶅仠鏈嶅姟鐨勬儏鍐典笅锛屽彲浠ュ疄鏂芥按骞虫墿灞曘€?/span>
 

鏁呴殰瀹归敊

绯荤粺闇€瑕佸ε鍠勫鐞嗘晠闅滐紝纭繚绯荤粺鍦ㄦ煇浜涚粍浠跺彂鐢熸晠闅滅殑鎯呭喌涓嬶紝鏁翠釜绯荤粺鏈嶅姟鐨勫彲鐢ㄦ€с€傚彲鑳介儴鍒嗙粍浠舵晠闅滀細瀵艰嚧闆嗙兢涓儴鍒嗚妭鐐瑰畷鏈猴紝褰卞搷浜嗘暣鐞嗙殑sla锛屼絾鏄郴缁熻繕鏄彲浠ョ浉搴旂殑锛岀郴缁熶笉鑳芥湁鍗曠偣鏁呴殰銆?/span>
 

浣庡欢杩?/span>

寰堝搴旂敤瀵逛簬璇诲拰鍐欐搷浣滅殑寤舵椂瑕佹眰闈炲父楂橈紝瑕佹眰瀵规洿鏂板拰鏌ヨ鐨勫搷搴旀槸浣庡欢鏃剁殑銆?/span>
 

鍙墿灞?/span>

绯荤粺闇€瑕佽冻澶熺伒娲伙紝鑳藉瀹炵幇鏂板鍜屼慨鏀归渶姹傦紝鍙堜笉闇€瑕侀噸鏋勬暣涓郴缁熴€傚疄鏃跺鐞嗗拰鎵瑰鐞嗛殧绂诲紑锛岃兘澶熺伒娲讳慨鏀归渶姹傘€?/span>
 

鏄撶淮鎶?/span>

寮€鍙戦儴缃蹭笉鑳藉澶鏉傘€?/span>

涓夈€丩ambda鏋舵瀯鐨勫垎灞?/span>

鍦↙ambda鏋舵瀯涓柊鏁版嵁鍒拌揪鏃讹紝浼氳鍚屾椂鍒嗘淳鍒版壒澶勭悊灞傚拰蹇€熷鐞嗗眰銆備竴鏃︽暟鎹埌杈炬壒澶勭悊灞傦紝鎸夌収甯歌鎵瑰鐞嗘椂闂撮棿闅旓紝姣忔閮戒粠澶村紑濮嬮噸鏂拌绠楀苟鐢熸垚鎵瑰鐞嗚鍥俱€傜被浼煎湴锛屽彧瑕佹柊鏁版嵁鍒拌揪蹇€熷鐞嗗眰锛屽揩閫熷鐞嗗眰灏变細浣跨敤鏂版暟鎹敓鎴愬揩閫熻鍥俱€傚湪鏌ヨ鍒拌揪鏈嶅姟灞傛椂锛屽畠浼氬悎骞跺揩閫熻鍥惧拰鎵瑰鐞嗚鍥炬潵鐢熸垚閫傚綋鐨勬煡璇㈢粨鏋溿€傜敓鎴愭壒澶勭悊瑙嗗浘鍚庯紝蹇€熻鍥惧皢琚涪寮冿紝闄ら潪鏈夋柊鏁版嵁鎶佃揪锛屽惁鍒欏彧闇€瑕佹煡璇㈡壒澶勭悊瑙嗗浘锛屽洜涓烘鏃舵壒澶勭悊灞備腑鎷ユ湁鎵€鏈夌殑鏁版嵁銆?/span>
 

lambda鏋舵瀯瀹氫箟涓昏灞備互鍙婃瘡涓粍浠朵箣闂寸殑闆嗘垚銆傛敞鎰忓垎涓轰互涓嬪眰锛?/span>
 

鏁版嵁婧?/span>

鏁版嵁婧愭寚澶栭儴鐨勬暟鎹簱銆佹秷鎭槦鍒椼€佹枃浠剁瓑锛屽彲浠ュ紑鍙戞暟鎹秷璐瑰眰锛岄殣钘忔潵鑷笉鍚岃闂暟鎹殑澶嶆潅鎬э紝瀹氫箟濂芥暟鎹牸寮忋€?/span>
 

鏁版嵁娑堣垂灞?/span>

璐熻矗灏佽涓嶈兘鏁版嵁婧愯幏鍙栨暟鎹殑澶嶆潅鎬э紝灏嗗叾杞崲鍙敱鎵瑰鐞嗘垨鑰呮祦澶勭悊杩涗竴姝ヤ娇鐢ㄥ悓涓€鐨勬牸寮忚繘琛屾秷璐广€?/span>
 

鎵瑰鐞嗗眰

杩欐槸lambda鏋舵瀯鏍稿績灞備箣涓€锛屾壒澶勭悊鎺ュ彈鏁版嵁锛屾寔涔呭寲鍒扮敤鎴峰畾涔夊ソ鐨勬暟鎹粨鏋勪腑锛岀淮鎶ょ潃涓绘暟鎹€傛暟鎹粨鏋勪竴鑸笉鍋氭敼鍙橈紝鍙槸杩藉姞鏁版嵁銆傛壒澶勭悊杩樿礋璐ｅ垱寤哄拰缁存姢鎵瑰鐞嗚鍥俱€傛瘮濡傛垜浠父鍋氱殑hive etl ,缁熻涓€浜涙暟鎹紝鏈€鍚庡皢缁撴灉淇濆瓨鍦╤ive琛ㄤ腑锛屾垨鑰呮暟鎹簱涓紝灏卞睘浜庢壒澶勭悊灞傘€?/span>
 

瀹炴椂灞?/span>

杩欐槸lambda鍙︿竴涓牳蹇冨眰銆傛壒澶勭悊鍦ㄥ緢澶氬満鏅笅鑳藉婊¤冻闇€姹傦紝浣嗘槸闅忕潃涓氬姟闇€姹?ldquo;鑻涘埢鎬?rdquo;锛屼粬浠笇鏈涜兘澶熷強鏃剁湅鍒版暟鎹紝鑰屼笉鏄瓑鍒扮浜屽ぉ鎵嶇湅鎸囨爣鍙樺寲鍜屽垎鏋愮粨鏋溿€傛墍浠ュ紩鍏ヤ簡瀹炴椂澶勭悊銆傚疄鏃跺眰瑙ｅ喅浜嗕竴涓棶棰橈紝鍗冲彧瀛樺偍鍙珛鍗冲悜鐢ㄦ埛鎻愪緵鐨勪竴缁勬暟鎹紝杩欐牱灏变笉闇€瑕佸鍏ㄩ噺鏁版嵁杩涜澶勭悊锛屽ぇ澶ф彁渚涘鐞嗘晥鐜囥€傛瘮濡傛祦澶勭悊浠呬粎瀛樺偍鏈€杩?鍒嗛挓鐨勬暟鎹紝澶勭悊璁＄畻骞跺舰鎴愮粨鏋滐紝杩欏氨鏄垜浠敤spark streaming涓鏈夌殑鏃堕棿绐楀彛銆?/span>
 

鏈嶅姟灞?/span>

杩欐槸lambda鏋舵瀯鐨勬渶鍚庝竴灞傦紝鏈嶅姟灞傜殑鑱岃矗鏄幏鍙栨壒澶勭悊鍜屾祦澶勭悊鐨勭粨鏋滐紝鍚戠敤鎴锋彁渚涚粺涓€鏌ヨ瑙嗗浘鏈嶅姟銆?/span>

鍥涖€丩ambda鏋舵瀯鎬荤粨

lambda鏁版嵁鏋舵瀯鏇剧粡鎴愪负姣忎竴涓叕鍙稿ぇ鏁版嵁骞冲彴蹇呭鐨勬灦鏋勶紝瀹冭в鍐充簡涓€涓叕鍙稿ぇ鏁版嵁鎵归噺绂荤嚎澶勭悊鍜屽疄鏃舵暟鎹鐞嗙殑闇€姹傘€?/span>
 

鏁版嵁浠庡簳灞傜殑鏁版嵁婧愬紑濮嬶紝缁忚繃鍚勭鍚勬牱鐨勬牸寮忚繘鍏ュぇ鏁版嵁骞冲彴锛屽湪澶ф暟鎹钩鍙颁腑缁忚繃kafka銆丗lume绛夋暟鎹粍浠惰繘琛屾敹闆嗭紝鐒跺悗鍒嗘垚涓ゆ潯绾胯繘琛岃绠椼€備竴鏉＄嚎鏄繘鍏ユ祦寮忚绠楀钩鍙?渚嬪 storm銆丗link鎴栬€匰park streaming)锛屽幓璁＄畻瀹炴椂鐨勪竴浜涙寚鏍?鍙︿竴鏉＄嚎杩涘叆鎵归噺鏁版嵁澶勭悊绂荤嚎璁＄畻骞冲彴(渚嬪mapreduce銆丠ive锛孲park sql)锛屽幓璁＄畻t 1鐨勭浉鍏充笟鍔℃寚鏍囷紝杩欎簺鎸囨爣闇€瑕侀殧鏃ユ墠鑳界湅瑙併€?/span>
 

lambda鏋舵瀯缁忓巻澶氬勾鐨勫彂灞曪紝闈炲父绋冲畾锛屽浜庡疄鏃惰绠楅儴鍒嗙殑璁＄畻鎴愭湰鍙帶锛屾壒閲忓鐞嗗彲浠ョ敤鏅氫笂鐨勬椂闂存潵鏁翠綋鎵归噺璁＄畻锛岃繖鏍锋妸瀹炴椂璁＄畻鍜岀绾胯绠楅珮宄板垎寮€锛岃繖绉嶆灦鏋勬敮鎾戜簡鏁版嵁琛屼笟鐨勬棭鏈熷彂灞曪紝浣嗘槸瀹冧篃鏈変竴浜涜嚧鍛界己鐐癸細
 

瀹炴椂涓庢壒閲忚绠楃粨鏋滀笉涓€鑷?/span>

鍥犱负鎵归噺鍜屽疄鏃惰绠楄蛋鐨勬槸涓や釜璁＄畻妗嗘灦鍜岃绠楃▼搴忥紝绠楀嚭鐨勭粨鏋滃線寰€涓嶅悓锛岀粡甯哥湅鍒颁竴涓暟瀛楀綋澶╃湅鏄竴涓暟鎹紝绗簩澶╃湅鏄ㄥぉ鐨勬暟鎹弽鑰屽彂鐢熶簡鍙樺寲銆?/span>
 

鎵瑰鐞嗙殑鍋ュ．鎬?/span>

闅忕潃鏁版嵁閲忕骇瓒婃潵瓒婂ぇ锛岀粡甯稿彂鐜板闂村彧鏈?銆?涓皬鏃剁殑鏃堕棿绐楀彛锛屽凡缁忔棤娉曞畬鎴愮櫧澶?0澶氫釜灏忔椂绱鐨勬暟鎹紝淇濊瘉鏃╀笂涓婄彮鍓嶅噯鏃跺嚭鏁版嵁宸叉垚涓烘瘡涓ぇ鏁版嵁鍥㈤槦澶寸柤鐨勯棶棰橈紝鍚屾椂鍋氫釜浠诲姟骞惰鎵ц瀵逛簬澶ф暟鎹泦缇ょ殑绋冲畾鎬т篃鏄法澶х殑鑰冮獙锛岀粡甯镐細鏈変换鍔″洜涓鸿祫婧愪笉瓒虫病鏈夊畾鏃跺惎鍔ㄦ垨鑰呮姤閿欍€?/span>
 

寮€鍙戝拰缁存姢鐨勫鏉?/span>

lambda 鏋舵瀯涓鍚屾牱鐨勪笟鍔￠€昏緫杩涜涓ゆ缂栫▼锛氫竴娆′负鎵归噺璁＄畻鐨凟tl绯荤粺锛屼竴娆′负娴佸紡璁＄畻鐨凷treaming绯荤粺銆傞拡瀵瑰悓涓€涓笟鍔￠棶棰樹骇鐢熶簡涓や釜浠ｇ爜搴擄紝鍚勬湁涓嶅悓鐨勬紡娲炪€?/span>
 

瀛樺偍澧為暱蹇?/span>

鏁版嵁浠撳簱鐨勮璁′笉鍚堢悊锛屼細浜х敓澶ч噺鐨勪腑闂寸粨鏋滆〃锛岄€犳垚鏁版嵁鎬ラ€熻啫鑳€锛屽姞澶ф湇鍔″櫒瀛樺偍鍘嬪姏銆傛瘮濡傛垜浠粡甯哥籂缁撲簬鏁版嵁浠撳簱鍒板簳鎬庝箞鍒嗗眰锛屾槸鐩存帴ods灞傚埌搴旂敤鍛?杩樻槸ods灞傝鏅dws銆丏w绛夛紝鏈€鍚庢墠鍒板簲鐢ㄥ憿?
 

lambda鏋舵瀯铏界劧鏈夌己鐐癸紝浣嗘槸鍦ㄥ緢澶氬叕鍙镐緷鐒堕€傜敤锛屾湁鏃跺€欐垜浠病鏈夐偅涔堝ぇ鐨勪笟鍔￠噺锛屽疄鏃朵笟鍔￠渶姹傚苟娌℃湁閭ｄ箞鏄庢樉锛岀敤鐫€lambda鏋舵瀯渚濈劧寰堢埥銆傚浜庤秴澶ф暟鎹噺鐨勪笟鍔℃垨鑰呭疄鏃朵笟鍔″悓鏍峰鐨勬儏鍐碉紝鍙互鎺㈢储鏀硅壇lambda锛屼笟鍐呬篃鎻愬嚭浜咾appa鏋舵瀯锛屾劅鍏磋叮鐨勫皬浼欎即鍙互鎼滅储瀛︿範涓嬨€?/span>
 

浣滆€咃細crm鐮旂┒绀炬潵婧愶細浠婃棩澶存潯锛?/span>鍥剧墖婧愯嚜缃戠粶;

閮戦噸澹版槑锛氭湰绔欑郴鏈綉缂栬緫杞浇锛岃浆杞界洰鐨勫湪浜庝紶閫掓洿澶氫俊鎭€傚娑夊強浣滃搧鍐呭銆佺増鏉冨拰鍏跺畠闂锛岃鍦?0鏃ュ唴涓庢湰缃戣仈绯伙紝鎴戜滑灏嗗湪绗竴鏃堕棿鍒犻櫎鍐呭锛?/span>鏈珯鎷ユ湁瀵规澹版槑鐨勬渶缁堣в閲婃潈銆?/span>