c# - Memory mapped file and memory consumption -

given following test code (x64 environment):

    static void test()     {         string filename = @"d:\map";         long length = new fileinfo(filename).length;         using (var file = memorymappedfile.createfromfile(filename, filemode.open, "mapfile", length, memorymappedfileaccess.readwrite))         {             byte* byteptr = (byte*)0;             var view = file.createviewaccessor(0, length, memorymappedfileaccess.readwrite);             view.safememorymappedviewhandle.acquirepointer(ref byteptr);              long count = (long)(length / sizeof(int));             long sum = 0;             long step =  count / 2000;              int* ptr = (int*)&byteptr[0];             long currentcount = 0 ;              parallel.for(0, count, new paralleloptions { maxdegreeofparallelism = 8 }, (i) =>             {                 interlocked.add(ref sum, ptr[i]);                 interlocked.increment(ref currentcount) ;                  if (currentcount % step == 0)                     console.write("\r{0:0.00}%", (100 * currentcount / (float)count));             });              console.writeline(sum);              view.dispose();         }     } 

given "d:\map" 40gb file, there odd behavior when there random access through pointer "ptr".

system physical memory used, , slowed, process taking more 2 hours complete.

when have sequential (and single threaded) access, physical memory used don't top 1gb , process takes 8 minutes.

my question is: when using memory mapped file "real" memory used? isn't virtual address space gets occupied?

i'm trying understand physical memory consumption when using memory mapped file.

memory mapped files use virtual memory. you'll have no trouble mapping many more gigabytes of vm space have ram on 64-bit operating system. point of demand-page virtual memory operating system, sum of memory required running processes exceeds amount of ram.

mapping ram costs money, that's demand comes in play. processor interrupts program , yells when tries access virtual memory address not mapped ram. called page fault.

if did not spend money on getting @ least 40 gb of ram you'll inevitably pay cost of os dealing these page faults. requires allocating ram page , filling content file. perf go south when has unmap mapped ram , save content file. followed re-using freed-up ram page , loading file content appropriate file offset. deer, disk slow. problem known "thrashing".

it less of issue when address memory sequentially, 1 page fault 4096 bytes of sequential access , you'll have odds disk reader head still in right spot when trip page fault.