Does malloc reserve more space while allocating memory?Memory usage doesn't decrease when free() usedWill malloc implementations return free-ed memory back to the system?How do I free memory in C?Improve INSERT-per-second performance of SQLite?Some memory seems to be left allocated after malloc() and free()Does dynamic memory allocation differ in C and C++ in popular implementations?Does Malloc only use the heap if requested memory space is large?Allocating more memory than there exists using mallocWhat does “Memory allocated at compile time” really mean?Why Linux Free command is not showing less free memory when I run a process which keeps on allocating memoryWhat are the contents of the memory just allocated by `malloc()`?Memory allocation and process memory useWhy is virtual memory allocated with malloc not released?

Which one is correct as adjective “protruding” or “protruded”?

Non-trope happy ending?

Freedom of speech and where it applies

250 Floor Tower

Biological Blimps: Propulsion

Is there a single word describing earning money through any means?

What is Cash Advance APR?

Yosemite Fire Rings - What to Expect?

Energy measurement from position eigenstate

Redundant comparison & "if" before assignment

WiFi Thermostat, No C Terminal on Furnace

Is it possible to have a strip of cold climate in the middle of a planet?

Why did the HMS Bounty go back to a time when whales are already rare?

Why is it that I can sometimes guess the next note?

C++ debug/print custom type with GDB : the case of nlohmann json library

Are the IPv6 address space and IPv4 address space completely disjoint?

Why did the EU agree to delay the Brexit deadline?

What should you do when eye contact makes your subordinate uncomfortable?

Approximating irrational number to rational number

When were female captains banned from Starfleet?

How should I respond when I lied about my education and the company finds out through background check?

How to indicate a cut out for a product window

Is there a name for this algorithm to calculate the concentration of a mixture of two solutions containing the same solute?

A social experiment. What is the worst that can happen?



Does malloc reserve more space while allocating memory?


Memory usage doesn't decrease when free() usedWill malloc implementations return free-ed memory back to the system?How do I free memory in C?Improve INSERT-per-second performance of SQLite?Some memory seems to be left allocated after malloc() and free()Does dynamic memory allocation differ in C and C++ in popular implementations?Does Malloc only use the heap if requested memory space is large?Allocating more memory than there exists using mallocWhat does “Memory allocated at compile time” really mean?Why Linux Free command is not showing less free memory when I run a process which keeps on allocating memoryWhat are the contents of the memory just allocated by `malloc()`?Memory allocation and process memory useWhy is virtual memory allocated with malloc not released?













23















I am observing the following behavior in my test program:



I am doing malloc() for 1 MB and then free() it after sleep(10). I am doing this five times. I am observing memory consumption in top while the program is running.



Once free()-d, I am expecting the program's virtual memory (VIRT) consumption to be down by 1 MB. But actually it isn't. It stays stable. What is the explanation for this behavior? Does malloc() do some reserve while allocating memory?










share|improve this question
























  • Related: How do I free memory in C?

    – Lundin
    yesterday






  • 2





    Possible duplicate of Memory usage doesn't decrease when free() used

    – Useless
    yesterday






  • 2





    @Useless This question has better answers than the older one so I've bucked convention and marked the old question a duplicate of this one.

    – John Kugelman
    yesterday











  • I think nearly all malloc/free implementations use some internal management which does request larger chunks and free them opportunistically. This may use brk(2) or mmap. It also means that pages might not actually get used before touched (and sometimes even uncommitted on free, so the virtual or data segment size is not so important)

    – eckes
    yesterday















23















I am observing the following behavior in my test program:



I am doing malloc() for 1 MB and then free() it after sleep(10). I am doing this five times. I am observing memory consumption in top while the program is running.



Once free()-d, I am expecting the program's virtual memory (VIRT) consumption to be down by 1 MB. But actually it isn't. It stays stable. What is the explanation for this behavior? Does malloc() do some reserve while allocating memory?










share|improve this question
























  • Related: How do I free memory in C?

    – Lundin
    yesterday






  • 2





    Possible duplicate of Memory usage doesn't decrease when free() used

    – Useless
    yesterday






  • 2





    @Useless This question has better answers than the older one so I've bucked convention and marked the old question a duplicate of this one.

    – John Kugelman
    yesterday











  • I think nearly all malloc/free implementations use some internal management which does request larger chunks and free them opportunistically. This may use brk(2) or mmap. It also means that pages might not actually get used before touched (and sometimes even uncommitted on free, so the virtual or data segment size is not so important)

    – eckes
    yesterday













23












23








23


1






I am observing the following behavior in my test program:



I am doing malloc() for 1 MB and then free() it after sleep(10). I am doing this five times. I am observing memory consumption in top while the program is running.



Once free()-d, I am expecting the program's virtual memory (VIRT) consumption to be down by 1 MB. But actually it isn't. It stays stable. What is the explanation for this behavior? Does malloc() do some reserve while allocating memory?










share|improve this question
















I am observing the following behavior in my test program:



I am doing malloc() for 1 MB and then free() it after sleep(10). I am doing this five times. I am observing memory consumption in top while the program is running.



Once free()-d, I am expecting the program's virtual memory (VIRT) consumption to be down by 1 MB. But actually it isn't. It stays stable. What is the explanation for this behavior? Does malloc() do some reserve while allocating memory?







c malloc free dynamic-memory-allocation






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited yesterday









Peter Mortensen

13.8k1987113




13.8k1987113










asked yesterday









user1228352user1228352

14011




14011












  • Related: How do I free memory in C?

    – Lundin
    yesterday






  • 2





    Possible duplicate of Memory usage doesn't decrease when free() used

    – Useless
    yesterday






  • 2





    @Useless This question has better answers than the older one so I've bucked convention and marked the old question a duplicate of this one.

    – John Kugelman
    yesterday











  • I think nearly all malloc/free implementations use some internal management which does request larger chunks and free them opportunistically. This may use brk(2) or mmap. It also means that pages might not actually get used before touched (and sometimes even uncommitted on free, so the virtual or data segment size is not so important)

    – eckes
    yesterday

















  • Related: How do I free memory in C?

    – Lundin
    yesterday






  • 2





    Possible duplicate of Memory usage doesn't decrease when free() used

    – Useless
    yesterday






  • 2





    @Useless This question has better answers than the older one so I've bucked convention and marked the old question a duplicate of this one.

    – John Kugelman
    yesterday











  • I think nearly all malloc/free implementations use some internal management which does request larger chunks and free them opportunistically. This may use brk(2) or mmap. It also means that pages might not actually get used before touched (and sometimes even uncommitted on free, so the virtual or data segment size is not so important)

    – eckes
    yesterday
















Related: How do I free memory in C?

– Lundin
yesterday





Related: How do I free memory in C?

– Lundin
yesterday




2




2





Possible duplicate of Memory usage doesn't decrease when free() used

– Useless
yesterday





Possible duplicate of Memory usage doesn't decrease when free() used

– Useless
yesterday




2




2





@Useless This question has better answers than the older one so I've bucked convention and marked the old question a duplicate of this one.

– John Kugelman
yesterday





@Useless This question has better answers than the older one so I've bucked convention and marked the old question a duplicate of this one.

– John Kugelman
yesterday













I think nearly all malloc/free implementations use some internal management which does request larger chunks and free them opportunistically. This may use brk(2) or mmap. It also means that pages might not actually get used before touched (and sometimes even uncommitted on free, so the virtual or data segment size is not so important)

– eckes
yesterday





I think nearly all malloc/free implementations use some internal management which does request larger chunks and free them opportunistically. This may use brk(2) or mmap. It also means that pages might not actually get used before touched (and sometimes even uncommitted on free, so the virtual or data segment size is not so important)

– eckes
yesterday












3 Answers
3






active

oldest

votes


















34















Once free()-d, I am expecting program's virtual memory (VIRT) consumption to be down by 1MB.




Well, this is not guaranteed by the C standard. It only says, once you free() the memory, you should not be accessing that any more.



Whether the memory block is actually returned to the available memory pool or kept aside for future allocations is decided by the memory manager.






share|improve this answer




















  • 1





    Is it possible to release the free()'d memory block back to OS?

    – user1228352
    yesterday






  • 7





    @user1228352 no, the C language doesn't allow this. If you want more control, you need to implement your own memory manager that relies on platform specific OS system calls.

    – Jabberwocky
    yesterday






  • 8





    @user1228352 I understand the feeling after this, let's say trickery, however - you really don't want to go that way, nor it makes sense in the long-term approach because it's just a waste of time for you to figure out how to make your own memory manager (if allowed by the OS) and debug it. Go by the C standard and you'll have more comfortable experience, while the OS does the thing it's made for. Well, unless your target is to make your own OS, but then you probably wouldn't ask this question.

    – KeyWeeUsr
    yesterday











  • @user1228352 Why would you want to? Virtual memory is effectively free.

    – David Schwartz
    yesterday











  • @DavidSchwartz My product works in a very resource-restrictive environment. I am aware of the fact that virtual memory is NOT actually memory in use, it's free. But it is definitely the memory my process CAN use. Just want to reduce the unnecessary consumption. I am fine with delay in response or processing but can't allow increase in VIRT memory consumption.

    – user1228352
    11 hours ago



















24














The C standard doesn't force on the implementer of malloc and free to return the memory to the OS directly. So different C library implementations will behave differently. Some of them might give it back directly and some might not. In fact, the same implementation will also behave differently depending on the allocation sizes and patterns.



This behavior, of course, is for good reasons:



  1. It is not always possible. OS-level memory allocations usually are done in pages (4KB, 4MB, or ... sizes at once). And if a small part of the page is still being used after freeing another part then the page cannot be given back to the operating system until that part is also freed.

  2. Efficiency. It is very likely that an application will ask for memory again. So why give it back to the OS and ask for it again soon after. (of course, there is probably a limit on the size of the memory kept.)

In most cases, you are not accountable for the memory you free if the implementation decided to keep it (assuming it is a good implementation). Sooner or later it will be reallocated or returned to the OS. Hence, optimizing for memory usage should be based on the amount you have malloc-ed and you haven't free-d. The case where you have to worry about this, is when your allocation patterns/sizes start causing memory fragmentation which is a very big topic on its own.



If you are, however, on an embedded system and the amount of memory available is limited and you need more control over when/how memory is allocated and freed then you need to ask for memory pages from the OS directly and manage it manually.



Edit: I did not explain why you are not accountable for memory you free.
The reason is, on a modern OS, allocated memory is virtual. Meaning if you allocate 512MB on 32-bit system or 10TB of 64-bit system, as long as you don't read or write to that memory, it will not reserve any physical space for it. Actually, it will only reserve physical memory for the pages you touch from that big block and not the entire block. And after "a while of not using that memory", its contents will be copied to disk and the underlying physical memory will be used for something else.






share|improve this answer




















  • 1





    Note that some allocators may avoid the possibility of copying data to disk by using OS specific calls that say "these pages aren't in use, so feel free to drop their contents, even though I'm not releasing the virtual memory itself". Example would be using the madvise call on Linux with MADV_DONTNEED.

    – ShadowRanger
    22 hours ago











  • @ShadowRanger very interesting to know! thank you.

    – Ameen
    22 hours ago


















10














This is very dependent on the actual malloc implementation in use.



Under Linux, there is a threshold (MMAP_THRESHOLD) to decide where the memory for a given malloc() request comes from.



If the requested amount is below or equal to MMAP_THRESHOLD, the request is satisfied by either taking it from the so-called "free list", if any memory blocks have already been free()d. Otherwise, the "break line" of the program (i. e. the end of the data segment) is increased and the memory made available to the program by this process is used for the request.



On free(), the freed memory block is added to the free list. If there is enough free memory at the very end of the data segment, the break line (mentionned above) is moved again to shrink the data segment, returning the excess memory to the OS.



If the requested amount exceeds MMAP_THRESHOLD, a separate memory block is requested by the OS and returned again during free().



See also https://linux.die.net/man/3/malloc for details.






share|improve this answer






















    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55294985%2fdoes-malloc-reserve-more-space-while-allocating-memory%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    3 Answers
    3






    active

    oldest

    votes








    3 Answers
    3






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    34















    Once free()-d, I am expecting program's virtual memory (VIRT) consumption to be down by 1MB.




    Well, this is not guaranteed by the C standard. It only says, once you free() the memory, you should not be accessing that any more.



    Whether the memory block is actually returned to the available memory pool or kept aside for future allocations is decided by the memory manager.






    share|improve this answer




















    • 1





      Is it possible to release the free()'d memory block back to OS?

      – user1228352
      yesterday






    • 7





      @user1228352 no, the C language doesn't allow this. If you want more control, you need to implement your own memory manager that relies on platform specific OS system calls.

      – Jabberwocky
      yesterday






    • 8





      @user1228352 I understand the feeling after this, let's say trickery, however - you really don't want to go that way, nor it makes sense in the long-term approach because it's just a waste of time for you to figure out how to make your own memory manager (if allowed by the OS) and debug it. Go by the C standard and you'll have more comfortable experience, while the OS does the thing it's made for. Well, unless your target is to make your own OS, but then you probably wouldn't ask this question.

      – KeyWeeUsr
      yesterday











    • @user1228352 Why would you want to? Virtual memory is effectively free.

      – David Schwartz
      yesterday











    • @DavidSchwartz My product works in a very resource-restrictive environment. I am aware of the fact that virtual memory is NOT actually memory in use, it's free. But it is definitely the memory my process CAN use. Just want to reduce the unnecessary consumption. I am fine with delay in response or processing but can't allow increase in VIRT memory consumption.

      – user1228352
      11 hours ago
















    34















    Once free()-d, I am expecting program's virtual memory (VIRT) consumption to be down by 1MB.




    Well, this is not guaranteed by the C standard. It only says, once you free() the memory, you should not be accessing that any more.



    Whether the memory block is actually returned to the available memory pool or kept aside for future allocations is decided by the memory manager.






    share|improve this answer




















    • 1





      Is it possible to release the free()'d memory block back to OS?

      – user1228352
      yesterday






    • 7





      @user1228352 no, the C language doesn't allow this. If you want more control, you need to implement your own memory manager that relies on platform specific OS system calls.

      – Jabberwocky
      yesterday






    • 8





      @user1228352 I understand the feeling after this, let's say trickery, however - you really don't want to go that way, nor it makes sense in the long-term approach because it's just a waste of time for you to figure out how to make your own memory manager (if allowed by the OS) and debug it. Go by the C standard and you'll have more comfortable experience, while the OS does the thing it's made for. Well, unless your target is to make your own OS, but then you probably wouldn't ask this question.

      – KeyWeeUsr
      yesterday











    • @user1228352 Why would you want to? Virtual memory is effectively free.

      – David Schwartz
      yesterday











    • @DavidSchwartz My product works in a very resource-restrictive environment. I am aware of the fact that virtual memory is NOT actually memory in use, it's free. But it is definitely the memory my process CAN use. Just want to reduce the unnecessary consumption. I am fine with delay in response or processing but can't allow increase in VIRT memory consumption.

      – user1228352
      11 hours ago














    34












    34








    34








    Once free()-d, I am expecting program's virtual memory (VIRT) consumption to be down by 1MB.




    Well, this is not guaranteed by the C standard. It only says, once you free() the memory, you should not be accessing that any more.



    Whether the memory block is actually returned to the available memory pool or kept aside for future allocations is decided by the memory manager.






    share|improve this answer
















    Once free()-d, I am expecting program's virtual memory (VIRT) consumption to be down by 1MB.




    Well, this is not guaranteed by the C standard. It only says, once you free() the memory, you should not be accessing that any more.



    Whether the memory block is actually returned to the available memory pool or kept aside for future allocations is decided by the memory manager.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited 21 hours ago









    Brian McCutchon

    4,76722136




    4,76722136










    answered yesterday









    Sourav GhoshSourav Ghosh

    111k15132191




    111k15132191







    • 1





      Is it possible to release the free()'d memory block back to OS?

      – user1228352
      yesterday






    • 7





      @user1228352 no, the C language doesn't allow this. If you want more control, you need to implement your own memory manager that relies on platform specific OS system calls.

      – Jabberwocky
      yesterday






    • 8





      @user1228352 I understand the feeling after this, let's say trickery, however - you really don't want to go that way, nor it makes sense in the long-term approach because it's just a waste of time for you to figure out how to make your own memory manager (if allowed by the OS) and debug it. Go by the C standard and you'll have more comfortable experience, while the OS does the thing it's made for. Well, unless your target is to make your own OS, but then you probably wouldn't ask this question.

      – KeyWeeUsr
      yesterday











    • @user1228352 Why would you want to? Virtual memory is effectively free.

      – David Schwartz
      yesterday











    • @DavidSchwartz My product works in a very resource-restrictive environment. I am aware of the fact that virtual memory is NOT actually memory in use, it's free. But it is definitely the memory my process CAN use. Just want to reduce the unnecessary consumption. I am fine with delay in response or processing but can't allow increase in VIRT memory consumption.

      – user1228352
      11 hours ago













    • 1





      Is it possible to release the free()'d memory block back to OS?

      – user1228352
      yesterday






    • 7





      @user1228352 no, the C language doesn't allow this. If you want more control, you need to implement your own memory manager that relies on platform specific OS system calls.

      – Jabberwocky
      yesterday






    • 8





      @user1228352 I understand the feeling after this, let's say trickery, however - you really don't want to go that way, nor it makes sense in the long-term approach because it's just a waste of time for you to figure out how to make your own memory manager (if allowed by the OS) and debug it. Go by the C standard and you'll have more comfortable experience, while the OS does the thing it's made for. Well, unless your target is to make your own OS, but then you probably wouldn't ask this question.

      – KeyWeeUsr
      yesterday











    • @user1228352 Why would you want to? Virtual memory is effectively free.

      – David Schwartz
      yesterday











    • @DavidSchwartz My product works in a very resource-restrictive environment. I am aware of the fact that virtual memory is NOT actually memory in use, it's free. But it is definitely the memory my process CAN use. Just want to reduce the unnecessary consumption. I am fine with delay in response or processing but can't allow increase in VIRT memory consumption.

      – user1228352
      11 hours ago








    1




    1





    Is it possible to release the free()'d memory block back to OS?

    – user1228352
    yesterday





    Is it possible to release the free()'d memory block back to OS?

    – user1228352
    yesterday




    7




    7





    @user1228352 no, the C language doesn't allow this. If you want more control, you need to implement your own memory manager that relies on platform specific OS system calls.

    – Jabberwocky
    yesterday





    @user1228352 no, the C language doesn't allow this. If you want more control, you need to implement your own memory manager that relies on platform specific OS system calls.

    – Jabberwocky
    yesterday




    8




    8





    @user1228352 I understand the feeling after this, let's say trickery, however - you really don't want to go that way, nor it makes sense in the long-term approach because it's just a waste of time for you to figure out how to make your own memory manager (if allowed by the OS) and debug it. Go by the C standard and you'll have more comfortable experience, while the OS does the thing it's made for. Well, unless your target is to make your own OS, but then you probably wouldn't ask this question.

    – KeyWeeUsr
    yesterday





    @user1228352 I understand the feeling after this, let's say trickery, however - you really don't want to go that way, nor it makes sense in the long-term approach because it's just a waste of time for you to figure out how to make your own memory manager (if allowed by the OS) and debug it. Go by the C standard and you'll have more comfortable experience, while the OS does the thing it's made for. Well, unless your target is to make your own OS, but then you probably wouldn't ask this question.

    – KeyWeeUsr
    yesterday













    @user1228352 Why would you want to? Virtual memory is effectively free.

    – David Schwartz
    yesterday





    @user1228352 Why would you want to? Virtual memory is effectively free.

    – David Schwartz
    yesterday













    @DavidSchwartz My product works in a very resource-restrictive environment. I am aware of the fact that virtual memory is NOT actually memory in use, it's free. But it is definitely the memory my process CAN use. Just want to reduce the unnecessary consumption. I am fine with delay in response or processing but can't allow increase in VIRT memory consumption.

    – user1228352
    11 hours ago






    @DavidSchwartz My product works in a very resource-restrictive environment. I am aware of the fact that virtual memory is NOT actually memory in use, it's free. But it is definitely the memory my process CAN use. Just want to reduce the unnecessary consumption. I am fine with delay in response or processing but can't allow increase in VIRT memory consumption.

    – user1228352
    11 hours ago














    24














    The C standard doesn't force on the implementer of malloc and free to return the memory to the OS directly. So different C library implementations will behave differently. Some of them might give it back directly and some might not. In fact, the same implementation will also behave differently depending on the allocation sizes and patterns.



    This behavior, of course, is for good reasons:



    1. It is not always possible. OS-level memory allocations usually are done in pages (4KB, 4MB, or ... sizes at once). And if a small part of the page is still being used after freeing another part then the page cannot be given back to the operating system until that part is also freed.

    2. Efficiency. It is very likely that an application will ask for memory again. So why give it back to the OS and ask for it again soon after. (of course, there is probably a limit on the size of the memory kept.)

    In most cases, you are not accountable for the memory you free if the implementation decided to keep it (assuming it is a good implementation). Sooner or later it will be reallocated or returned to the OS. Hence, optimizing for memory usage should be based on the amount you have malloc-ed and you haven't free-d. The case where you have to worry about this, is when your allocation patterns/sizes start causing memory fragmentation which is a very big topic on its own.



    If you are, however, on an embedded system and the amount of memory available is limited and you need more control over when/how memory is allocated and freed then you need to ask for memory pages from the OS directly and manage it manually.



    Edit: I did not explain why you are not accountable for memory you free.
    The reason is, on a modern OS, allocated memory is virtual. Meaning if you allocate 512MB on 32-bit system or 10TB of 64-bit system, as long as you don't read or write to that memory, it will not reserve any physical space for it. Actually, it will only reserve physical memory for the pages you touch from that big block and not the entire block. And after "a while of not using that memory", its contents will be copied to disk and the underlying physical memory will be used for something else.






    share|improve this answer




















    • 1





      Note that some allocators may avoid the possibility of copying data to disk by using OS specific calls that say "these pages aren't in use, so feel free to drop their contents, even though I'm not releasing the virtual memory itself". Example would be using the madvise call on Linux with MADV_DONTNEED.

      – ShadowRanger
      22 hours ago











    • @ShadowRanger very interesting to know! thank you.

      – Ameen
      22 hours ago















    24














    The C standard doesn't force on the implementer of malloc and free to return the memory to the OS directly. So different C library implementations will behave differently. Some of them might give it back directly and some might not. In fact, the same implementation will also behave differently depending on the allocation sizes and patterns.



    This behavior, of course, is for good reasons:



    1. It is not always possible. OS-level memory allocations usually are done in pages (4KB, 4MB, or ... sizes at once). And if a small part of the page is still being used after freeing another part then the page cannot be given back to the operating system until that part is also freed.

    2. Efficiency. It is very likely that an application will ask for memory again. So why give it back to the OS and ask for it again soon after. (of course, there is probably a limit on the size of the memory kept.)

    In most cases, you are not accountable for the memory you free if the implementation decided to keep it (assuming it is a good implementation). Sooner or later it will be reallocated or returned to the OS. Hence, optimizing for memory usage should be based on the amount you have malloc-ed and you haven't free-d. The case where you have to worry about this, is when your allocation patterns/sizes start causing memory fragmentation which is a very big topic on its own.



    If you are, however, on an embedded system and the amount of memory available is limited and you need more control over when/how memory is allocated and freed then you need to ask for memory pages from the OS directly and manage it manually.



    Edit: I did not explain why you are not accountable for memory you free.
    The reason is, on a modern OS, allocated memory is virtual. Meaning if you allocate 512MB on 32-bit system or 10TB of 64-bit system, as long as you don't read or write to that memory, it will not reserve any physical space for it. Actually, it will only reserve physical memory for the pages you touch from that big block and not the entire block. And after "a while of not using that memory", its contents will be copied to disk and the underlying physical memory will be used for something else.






    share|improve this answer




















    • 1





      Note that some allocators may avoid the possibility of copying data to disk by using OS specific calls that say "these pages aren't in use, so feel free to drop their contents, even though I'm not releasing the virtual memory itself". Example would be using the madvise call on Linux with MADV_DONTNEED.

      – ShadowRanger
      22 hours ago











    • @ShadowRanger very interesting to know! thank you.

      – Ameen
      22 hours ago













    24












    24








    24







    The C standard doesn't force on the implementer of malloc and free to return the memory to the OS directly. So different C library implementations will behave differently. Some of them might give it back directly and some might not. In fact, the same implementation will also behave differently depending on the allocation sizes and patterns.



    This behavior, of course, is for good reasons:



    1. It is not always possible. OS-level memory allocations usually are done in pages (4KB, 4MB, or ... sizes at once). And if a small part of the page is still being used after freeing another part then the page cannot be given back to the operating system until that part is also freed.

    2. Efficiency. It is very likely that an application will ask for memory again. So why give it back to the OS and ask for it again soon after. (of course, there is probably a limit on the size of the memory kept.)

    In most cases, you are not accountable for the memory you free if the implementation decided to keep it (assuming it is a good implementation). Sooner or later it will be reallocated or returned to the OS. Hence, optimizing for memory usage should be based on the amount you have malloc-ed and you haven't free-d. The case where you have to worry about this, is when your allocation patterns/sizes start causing memory fragmentation which is a very big topic on its own.



    If you are, however, on an embedded system and the amount of memory available is limited and you need more control over when/how memory is allocated and freed then you need to ask for memory pages from the OS directly and manage it manually.



    Edit: I did not explain why you are not accountable for memory you free.
    The reason is, on a modern OS, allocated memory is virtual. Meaning if you allocate 512MB on 32-bit system or 10TB of 64-bit system, as long as you don't read or write to that memory, it will not reserve any physical space for it. Actually, it will only reserve physical memory for the pages you touch from that big block and not the entire block. And after "a while of not using that memory", its contents will be copied to disk and the underlying physical memory will be used for something else.






    share|improve this answer















    The C standard doesn't force on the implementer of malloc and free to return the memory to the OS directly. So different C library implementations will behave differently. Some of them might give it back directly and some might not. In fact, the same implementation will also behave differently depending on the allocation sizes and patterns.



    This behavior, of course, is for good reasons:



    1. It is not always possible. OS-level memory allocations usually are done in pages (4KB, 4MB, or ... sizes at once). And if a small part of the page is still being used after freeing another part then the page cannot be given back to the operating system until that part is also freed.

    2. Efficiency. It is very likely that an application will ask for memory again. So why give it back to the OS and ask for it again soon after. (of course, there is probably a limit on the size of the memory kept.)

    In most cases, you are not accountable for the memory you free if the implementation decided to keep it (assuming it is a good implementation). Sooner or later it will be reallocated or returned to the OS. Hence, optimizing for memory usage should be based on the amount you have malloc-ed and you haven't free-d. The case where you have to worry about this, is when your allocation patterns/sizes start causing memory fragmentation which is a very big topic on its own.



    If you are, however, on an embedded system and the amount of memory available is limited and you need more control over when/how memory is allocated and freed then you need to ask for memory pages from the OS directly and manage it manually.



    Edit: I did not explain why you are not accountable for memory you free.
    The reason is, on a modern OS, allocated memory is virtual. Meaning if you allocate 512MB on 32-bit system or 10TB of 64-bit system, as long as you don't read or write to that memory, it will not reserve any physical space for it. Actually, it will only reserve physical memory for the pages you touch from that big block and not the entire block. And after "a while of not using that memory", its contents will be copied to disk and the underlying physical memory will be used for something else.







    share|improve this answer














    share|improve this answer



    share|improve this answer








    edited yesterday

























    answered yesterday









    AmeenAmeen

    7351519




    7351519







    • 1





      Note that some allocators may avoid the possibility of copying data to disk by using OS specific calls that say "these pages aren't in use, so feel free to drop their contents, even though I'm not releasing the virtual memory itself". Example would be using the madvise call on Linux with MADV_DONTNEED.

      – ShadowRanger
      22 hours ago











    • @ShadowRanger very interesting to know! thank you.

      – Ameen
      22 hours ago












    • 1





      Note that some allocators may avoid the possibility of copying data to disk by using OS specific calls that say "these pages aren't in use, so feel free to drop their contents, even though I'm not releasing the virtual memory itself". Example would be using the madvise call on Linux with MADV_DONTNEED.

      – ShadowRanger
      22 hours ago











    • @ShadowRanger very interesting to know! thank you.

      – Ameen
      22 hours ago







    1




    1





    Note that some allocators may avoid the possibility of copying data to disk by using OS specific calls that say "these pages aren't in use, so feel free to drop their contents, even though I'm not releasing the virtual memory itself". Example would be using the madvise call on Linux with MADV_DONTNEED.

    – ShadowRanger
    22 hours ago





    Note that some allocators may avoid the possibility of copying data to disk by using OS specific calls that say "these pages aren't in use, so feel free to drop their contents, even though I'm not releasing the virtual memory itself". Example would be using the madvise call on Linux with MADV_DONTNEED.

    – ShadowRanger
    22 hours ago













    @ShadowRanger very interesting to know! thank you.

    – Ameen
    22 hours ago





    @ShadowRanger very interesting to know! thank you.

    – Ameen
    22 hours ago











    10














    This is very dependent on the actual malloc implementation in use.



    Under Linux, there is a threshold (MMAP_THRESHOLD) to decide where the memory for a given malloc() request comes from.



    If the requested amount is below or equal to MMAP_THRESHOLD, the request is satisfied by either taking it from the so-called "free list", if any memory blocks have already been free()d. Otherwise, the "break line" of the program (i. e. the end of the data segment) is increased and the memory made available to the program by this process is used for the request.



    On free(), the freed memory block is added to the free list. If there is enough free memory at the very end of the data segment, the break line (mentionned above) is moved again to shrink the data segment, returning the excess memory to the OS.



    If the requested amount exceeds MMAP_THRESHOLD, a separate memory block is requested by the OS and returned again during free().



    See also https://linux.die.net/man/3/malloc for details.






    share|improve this answer



























      10














      This is very dependent on the actual malloc implementation in use.



      Under Linux, there is a threshold (MMAP_THRESHOLD) to decide where the memory for a given malloc() request comes from.



      If the requested amount is below or equal to MMAP_THRESHOLD, the request is satisfied by either taking it from the so-called "free list", if any memory blocks have already been free()d. Otherwise, the "break line" of the program (i. e. the end of the data segment) is increased and the memory made available to the program by this process is used for the request.



      On free(), the freed memory block is added to the free list. If there is enough free memory at the very end of the data segment, the break line (mentionned above) is moved again to shrink the data segment, returning the excess memory to the OS.



      If the requested amount exceeds MMAP_THRESHOLD, a separate memory block is requested by the OS and returned again during free().



      See also https://linux.die.net/man/3/malloc for details.






      share|improve this answer

























        10












        10








        10







        This is very dependent on the actual malloc implementation in use.



        Under Linux, there is a threshold (MMAP_THRESHOLD) to decide where the memory for a given malloc() request comes from.



        If the requested amount is below or equal to MMAP_THRESHOLD, the request is satisfied by either taking it from the so-called "free list", if any memory blocks have already been free()d. Otherwise, the "break line" of the program (i. e. the end of the data segment) is increased and the memory made available to the program by this process is used for the request.



        On free(), the freed memory block is added to the free list. If there is enough free memory at the very end of the data segment, the break line (mentionned above) is moved again to shrink the data segment, returning the excess memory to the OS.



        If the requested amount exceeds MMAP_THRESHOLD, a separate memory block is requested by the OS and returned again during free().



        See also https://linux.die.net/man/3/malloc for details.






        share|improve this answer













        This is very dependent on the actual malloc implementation in use.



        Under Linux, there is a threshold (MMAP_THRESHOLD) to decide where the memory for a given malloc() request comes from.



        If the requested amount is below or equal to MMAP_THRESHOLD, the request is satisfied by either taking it from the so-called "free list", if any memory blocks have already been free()d. Otherwise, the "break line" of the program (i. e. the end of the data segment) is increased and the memory made available to the program by this process is used for the request.



        On free(), the freed memory block is added to the free list. If there is enough free memory at the very end of the data segment, the break line (mentionned above) is moved again to shrink the data segment, returning the excess memory to the OS.



        If the requested amount exceeds MMAP_THRESHOLD, a separate memory block is requested by the OS and returned again during free().



        See also https://linux.die.net/man/3/malloc for details.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered yesterday









        glglglglglgl

        67.9k796169




        67.9k796169



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55294985%2fdoes-malloc-reserve-more-space-while-allocating-memory%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Sum ergo cogito? 1 nng

            三茅街道4182Guuntc Dn precexpngmageondP