Monday, April 21, 2008

WCF Memory Buffer Management

I have recently joined the Connected Framework Team at Microsoft - at least if 7 months ago still qualifies for "recently". The team is much less known by its name than by the product it created, i.e. Indigo, later charmingly renamed Windows Communication Foundation. As part of my job as a Program Manager I get to work on WCF performance, and many folks on the team are quite particular about it. Periodically, I monitor various related online forums and pick out questions related to performance. About a month ago I spotted a couple of questions about memory management in WCF. Since WCF provides a handful of different configuration parameters related to memory, people were curious what may be the impact of their various settings on performance. Kenny Wolf has a few posts on the subject in his blog: here and here.

However, curious as I am, I wanted to dive a little deeper and see how things work under the hood. A few hours of spelunking through source code and stepping through in the debugger (which you can also do now that the .Net Framework source code is publicly available - see here for more) I realized that WCF uses a pretty sophisticated memory buffer management mechanism in order to save on garbage collection and improve performance. The rest of this post describes just how it works, so if you're interested, read on.

The crux of the buffer managment algorithm rests with the class aptly named BufferManager. BufferManager has one static method CreateBufferManager which takes two parameters: maxBufferPoolSize and maxBufferSize. The first one indicates how much total memory the manager can hold on to, while the other determines the size of the largest individual buffer the manager can store. In reality, BufferManager's CreateBufferManager method delegates to one of two internal classes depending on the value of maxBufferPoolSize. If maxBufferPoolSize = 0, then a GCBufferManager gets created, otherwise you get a PooledBufferManager. The former is trivial, and in fact doesn't really do any management whatsoever, but simply allocates a new buffer for any request, and lets the garbage collector take care of disposing it. The latter is much more interesting.

The PooledBufferManager's constructor takes the same two parameters as the CreateBufferManager method, i.e. maxBufferPoolSize, maxBufferSize. The constructor creates a list of buffer pools of type SynchronizedPool. Each of them collects buffers of a certain size (bufferSize). The size of the buffers in the smallest pool is 128 bytes. Each subsequent pool doubles the size of the buffers it holds until the cap of maxBufferSize is reached. Each pool is also assigned a quota of buffers it can contain at any time. As the pools are created each is given a quota of a single buffer until the total allotment exceeds maxBufferPoolSize. Any subsequent pools get a quota of zero. Each pool also contains some bookkeeping data, such as the number of buffers currently in the pool (count), the maximum number of buffers in the pool (limit), as well as the peak number of buffers in the pool (peak) and the number of misses (misses), which will be explained a bit later.

Buffer managers are created by a number of components of WCF, most notably by the OnOpening method of the TransportChannelListener. It's the listener that receives messages sent to a WCF service. TransportChannelListener creates a buffer manager with the maximum buffer size equal to the MaxReceivedMessageSize. That's the value you can set on your binding in config (maxReceivedMessageSize) to prevent denial of service attacks by limiting the maximum amount of memory the system will allocate for any incoming message (see Kenny's post for more details). TransportChannelListener also sets the maximum buffer pool size according to the value of MaxBufferPoolSize property of the transport binding element.

The really interesting fact about buffer management in WCF is that BufferManager doesn't actually allocate any buffers up front. Instead, each of the buffer pools is allotted a quota of buffers it can hold (limit). If someone requests a buffer - say, for an incoming message - (by calling TakeBuffer) the manager locates the first buffer pool that holds buffers large enough to satisfy the request. For example, if the requested buffer size is 560 bytes, the manager will try 128, 256, 512, and finally settle on 1024. Now, the manager will ask the pool for a buffer. If one is available in the pool, it is returned to the requestor, the pool’s buffer count is decremented and that's the end of the story. On the other hand, if none is available the manager will allocate a new buffer according to the pool's buffer size (1024 bytes in my example), but before that happens, some bookkeeping takes place. The manager checks if the peak number of buffers in the pool has reached its allotted limit. If so, then we have a miss, which means that the total number of allocated buffers of this size currently in the system has exceeded the limit. The manager will bump up the number of misses for this pool. It will also increment its own counter of total misses for all buffer pools (totalMisses). Once this number reaches a predefined threshold (maxMissesBeforeTuning = 8), the allotment of buffer limits per pool must be tuned up. The manager will call TuneQuotas, but this will be described later. For now the request has been satisfied by either returning a buffer from a pool or allocating a new buffer.

Having obtained a buffer the requestor can now hold on to it for as long as is needed, but when it's done it must call ReturnBuffer to give the buffer back to the BufferManager. What happens upon buffer return is pretty much the reverse of what you saw for taking a buffer. The manager finds the matching pool. If none is found, or if the matching pool has no more room left (count = limit), the buffer is simply abandoned and left for the GC to collect (this is to ensure that the total pooled memory never exceeds maxBufferPoolSize). Otherwise, the buffer is returned to the pool and the pool’s buffer count is incremented. If the count exceeds peak, the latter is updated accordingly.

Now that you know how buffers are obtained and returned it’s time to take a look at quota tuning. First, the BufferManager finds the most starved buffer pool, defined as the pool missing the most memory, which in turn is calculated as the product of the number of misses in the pool and the buffer size. If there is enough memory left under the maxBufferPoolSize to accommodate one more buffer in the most starved pool, the pool’s quota is simply incremented by one, and remaining memory is adjusted. Otherwise, the manager must find a different pool from which to steal a buffer. The best candidate is the most underutilized pool, i.e. the one with the maximum product of (limit – peak) * bufferSize. Once such a pool is found its quota is decremented by one, and the remaining memory is adjusted accordingly. If there is enough memory now to add a buffer to the most starved pool, its quota will be bumped up by one. Otherwise, the tuning ends, and another attempted will be made in the next tuning cycle. Finally, the misses count for all pools and the managers totalMisses count are reset.

If all this sounds a bit complicated, it's because the WCF team has gone to great length to ensure this platform performs extremely well, and efficient buffer management contributes substantially to this goal. I should add that my description omits another aspect of buffer managment, which adds to its efficiency. I will only mention it here very briefly. Whenever multiple threads have to access a common pool of resources, you must obviously make sure that they do so without corrupting the state of the resource pool (here buffer manager), which, in short, involves thread synchronization and locking. Synchronization is expensive, and could potentially lead to such contention over the buffer manager that it would erase any benefit from buffer pooling. To avoid this problem BufferManager uses a SynchronizedPool for each of its buffer pools, which in turn employs some clever tricks to allocate dedicated buffers to the most active threads, so that they can obtain and return them without locking.

In summary, WCF comes with a powerful and efficient buffer pooling subsystem. It does not pre-allocate any memory, but instead creates and reuses memory buffers only as messages come in. In addition, it dynamically adapts the division of the pre-configured memory pool between buffers of various sizes to best match the size(s) of the most common messages your service receives.

41 comments:

alcherenga said...

this is really a good post on buffer management. i am writing a book on wcf and i looking for stuff as the buffer management is not all defined anywhere .

thanks indeed

投癢癢 said...

Joy often comes after sorrow, like morning after night...................................................

MicrosoftShit said...

Great description but I still have some questions.

If you have multiple endpoints with different bindings do you end up with more than one buffer pool? How about multiple endpoints with the same binding.

Does this apply to web service clients? If so, does it share the buffer pool with services in the same application?

Do you know any source of information about which of the ServiceModel settings apply to the client and which apply to the server and which should always match on client and server.

As an example, if bufferpoolsize applies to client and server then there should not be a need to have them the same on both sides.

Thanks!

微笑每一天 said...

Better say nothing than nothing to the purpose. ........................................

怡如 said...

請繼續發表好文!加油加油再加油!........................................

憲妤憲妤 said...

0401影音成人影片交流愛撫淫叫色情照片性愛圖片女生如何自慰激突成人論壇美女脫胸罩走光一夜正妹視訊情人性愛論壇85cc影城性伴侶85cc色情影片打手槍情色視訊聊天室色情a片色情色情圖片限制級照片色情電話520sex免費色情網情色性愛一夜情聊天網成人聊天網視訊成人台灣性網av一夜女情色av淫女免費視訊美女挑逗情色影音聊天作愛影片辣妹清涼秀美女做愛成人自拍貼圖成人資訊色論壇台灣色情成人網站色情視訊聊天露三點打砲情色性愛貼圖av成人網成人影片

嘉雯 said...

hello~welcome my world~<. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

AmiHopson瓊仁豪慧 said...

台灣本土性愛影片性愛遊性愛文性愛性交自拍走光貼色情片無碼色情片貼片色情片影片色情卡通貼圖色情光牒色情米克色情自拍圖貼色情自拍網站色情免費成人無碼電影片色情片gogo色情片色情文說色美眉寫真色美眉影片色站排行榜色情!色情DVD色情qvod色情人無碼影片色情上床色情女圖片色情介紹色情分享片色情天室色情文小說少年阿賓系列小說言情小說免費線上閱讀 貼圖片區女同志聊天室

ImogeneL22211 said...

生活總是起起伏伏,心情要保持快樂才好哦!!........................................

SungR_Auclair0佳亦 said...

sex女,ut,貼影,av,高潮,女優,做愛,手機成人影片,色遊戲,成人動漫,百分百貼圖區,85cc影片,成人影音,av色情影片,A片線上,a片,sex,777,三級線上看,美女的照片,視訊做愛,78論壇,打飛機,免費視訊,成人影院,辣妹視訊,視訊聊天,影片網,kiss911,a片,交友,聊天,做愛,免費影片,性交,線上成人,網路色情,聊天,美女自慰,免費a分享,免費短片,裸照,一夜情,女優,85cc成人片,美女寫真,偷拍a,情人視訊聊天室

茂慧 said...

go2av-視訊網 蕃薯藤交友 視訊聊天sexy girls get fucked辣妹 視訊交友天堂90739 高雄視訊34c正妹 免費視訊34c辣妹微風論壇 月宮貼圖色妹妹嘟嘟情人色網 免費視訊gogo 080豆豆出租情人視訊 999成人性站 拓網成人 視訊 純愛迷情成人綜合論壇 性感遊戲 辣妹自拍哈尼視訊 情色視訊 拓網交友情色視訊 520sex-情色視訊 拓網視訊交友 sexy girl色妹妹視訊 sex520貼片援交視訊聊天室 新浪視訊 色妹妹視訊高雄援交 34c視訊網愛聊天室 一對多視訊拓網辣妹視訊 176視訊聊天室 av最前線 視訊交友90739南台灣視訊 性愛貼圖 視訊聊天室90739 show-live視訊情色 網 美女米克綜合論壇視訊 日本avdvd女優美女寫真 完美女人辣妹鋼管脫衣秀 全裸美女圖片 辣妹視訊自拍美女聊天室 777美女dvd辣妹視訊 網路美女辣妹陪聊 台灣美女寫真集圖 成人卡通美女a片 視訊美女免費試玩 日本美女短片 色情漫畫777美女dvdav 亞洲藝人美女寫真 台灣亞洲名模美女 美女自拍走光貼圖 無碼女優美女圖光碟 護士美女自拍貼圖 巨乳美女寫真 自拍情趣美女 性感熱辣美女女優寫真

淑芳 said...

Beauty, unaccompanied by virtue, is as a flower without perfume..............................................

YuriS_New家愛 said...

enjoy your artical, thank you .............................................

智宜智宜 said...

做愛限制級波霸口交18禁貼圖寫真視訊援交露點爆乳潮吹裸體裸照裸女愛愛無碼尋夢視訊聊天a漫a片a圖一夜情一葉情人妻激情情色寫真美女自拍辣妹自拍正妹自拍美女走光辣妹走光正妹走光脫衣秀

子名子名 said...

安心亞寫真top1069拓網交友做愛自拍免費情色影片寫真集美女正妹照片正妹貼圖正妹視訊250av女優免費影片旺來出品辣妹寫真鋼管秀旺來風情寫真秀-辣妹過招旺來風情寫真秀旺來蓬萊仙山寫真集 vcd旺旺仙貝的狂想境地早洩韭南籽早期歐美a片早期范冰冰照片早春小老婆日本三性影片美女 視訊洪爺sex免費看a片論壇秘密情人影音視訊網 bt成人網av一葉情貼影色網18 禁一葉情貼影入口女生自衛影片免費聊天女同志聊天室成人聊天室做愛影片網交聊天室性愛姿勢免費av影片觀看拓峰交友plus論壇hbo論壇一夜情視訊聊天室五分鐘護半身視訊美女激情網愛聊天室臺灣情色網

思穎鄭宇柏 said...

第一次睇你blog,鐘意!........................................

josema said...

All roads lead to Rome. 堅持自己所選! .........................................

俊翔 said...

出遊不拘名勝,有景就是好的..............................................................

皇銘 said...

Many a little makes a mickle...................................................................

玫友 said...

Pen and ink is wits plough.................................................................

王瑞 said...

很棒的分享~~~來留個言囉~~~~.................................................................                           

靜宸靜宸 said...

成熟,就是有能力適應生活中的模糊。.................................................................

玉苓玉苓 said...

噴泉的高度,不會超過它的源頭。一個人的事業也是如此,它的成就絕不會超過自己的信念。.................................................................

玉苓玉苓 said...

Quietude is the crown of life.............................................................

柏強 said...

肯定與支持你!!!加油囉~..................................................................

懿綺懿綺 said...

Say not all that you know, believe not all that you hear.............................................................

嘉剛怡元 said...

在你一無所有的時候 是誰在陪伴你 他便是你最重要的人............................................................

芸茂芸茂 said...

生命的意義,是在於活的充實;而不是在於活得長久。............................................................

馥虹 said...

愛,拆開來是心和受兩個字。用心去接受對方的一切,用心去愛對方的所有。......................................................................

琬安琬安 said...

讚啦~~多謝分享!!>ˍ<............................................................

陳晏陳詩蓁雄 said...

世間事沒有一樣沒有困難,只要有信心去做,至少可以做出一些成績。..................................................

張怡萱張怡萱張怡萱 said...

好的開始並不代表會成功,壞的開始並不代表是失敗..................................................

蔡苡玄 said...

多謝美味的心靈雞湯......................................................................

黃子楊琬和和 said...

良好的開端,已是成功的一半。..................................................

玄王季玄王季玄王季 said...

Joy often comes after sorrow, like morning after night.. . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

蔡曼鄭美玉屏 said...

你的部落內容真棒,一定要持續下去! ..................................................................

佳幸雨幸雨幸雨俐 said...

活是一種鍛鍊靈魂的東西..................................................................

佳張張張張燕張張張張張 said...

每次看完你的文章,總是回味許久,要經常發表喔。..................................................

tongtong said...

令人心動的好文章~~............................................................

316 said...

值得一看再看的格子,請加油 ..................................................................

舒夏怡 said...

Many a little makes a mickle..................................................................