c++ - Interpreting memory consumption of program using pugixml -

i have program parses xml file of ~50mb , extracts data internal object structure no links original xml file. when try estimate how memory need, reckon 40mb.

but program needs 350mb, , try find out happens. use boost::shared_ptr, i'm not dealing raw pointers , didn't produce memory leaks.

i try write did, , hope might point out problems in process, wrong assumptions , on.

first, how did measure? used htop find out memory full , processes using piece of code using of it. sum memory of different threads , more pretty output, used http://www.pixelbeat.org/scripts/ps_mem.py confirmed observation.

i estimated theoretical consumption idea factor lies between consumption , should @ least. it's 10. used valgrind --tool=massif analyze memory consumption. shows, @ peak of 350mb 250mb used called xml_allocator stems pugixml library. went section of code instantiate pugi::xml_document , put std::cout destructor of object confirm released happens pretty in program (at end sleep 20s have enough time measure memory consumption, stays 350mb after console output destructor appears).

now have no idea how interpret , hope can me make wrong assumptions or such.

the outermost code snippet using pugixml similar to:

void parse( std::string filename, my_data_structure& struc ) {     pugi::xml_document doc;     pugi::xml_parse_result result = doc.load_file(filename.c_str());      (pugi::xml_node node = doc.child("foo").child("bar"); node; node = node.next_sibling("bar"))     {         struc.hams.push_back( node.attribute("ham").value() );     } } 

and since in code don't store pugixml elements somewhere (only actual values pulled out of it), doc expect release resources when function parse left, looking on graph, cannot tell (on time axis) happens.

your assumptions incorrect.

here's how estimate pugixml memory consumption:

1. when load document, entire text of document gets loaded memory. that's 50 mb file. comes 1 allocation xml_document::load_file -> load_file_impl
2. in addition that, there's dom structure contains links other nodes, etc. size of node 32 bytes, size of attribute 20 bytes; that's 32-bit processes, multiply 2 64-bit processes. comes many allocations (each allocation 32kb) xml_allocator.

depending on density of nodes/attributes in document, memory consumption can range from, say, 110% of document size (i.e. 50 mb -> 55 mb) to, say, 600% (i.e. 50 mb -> 300 mb).

when destroy pugixml document (xml_document dtor gets called), data freed - however, depending on way os heap behaves, may not see returned system - may stay in process heap. verify can try doing parsing again , checking peak memory same after second parse.