Add custom memory allocator to MinGW and MacOS builds

+9

Makefile

··· 178 178 # 179 179 # Define NO_CROSS_DIRECTORY_HARDLINKS if you plan to distribute the installed 180 180 # programs as a tar, where bin/ and libexec/ might be on different file systems. 181 + # 182 + # Define USE_NED_ALLOCATOR if you want to replace the platforms default 183 + # memory allocators with the nedmalloc allocator written by Niall Douglas. 181 184 182 185 GIT-VERSION-FILE: .FORCE-GIT-VERSION-FILE 183 186 @$(SHELL_PATH) ./GIT-VERSION-GEN ··· 844 847 NO_ST_BLOCKS_IN_STRUCT_STAT = YesPlease 845 848 NO_NSEC = YesPlease 846 849 USE_WIN32_MMAP = YesPlease 850 + USE_NED_ALLOCATOR = YesPlease 847 851 UNRELIABLE_FSTAT = UnfortunatelyYes 848 852 OBJECT_CREATION_USES_RENAMES = UnfortunatelyNeedsTo 849 853 COMPAT_CFLAGS += -D__USE_MINGW_ACCESS -DNOGDI -Icompat -Icompat/regex -Icompat/fnmatch ··· 1128 1132 endif 1129 1133 ifdef UNRELIABLE_FSTAT 1130 1134 BASIC_CFLAGS += -DUNRELIABLE_FSTAT 1135 + endif 1136 + 1137 + ifdef USE_NED_ALLOCATOR 1138 + COMPAT_CFLAGS += -DUSE_NED_ALLOCATOR -DOVERRIDE_STRDUP -DNDEBUG -DREPLACE_SYSTEM_ALLOCATOR -Icompat/nedmalloc 1139 + COMPAT_OBJS += compat/nedmalloc/nedmalloc.o 1131 1140 endif 1132 1141 1133 1142 ifeq ($(TCLTK_PATH),)

+23

compat/nedmalloc/License.txt

··· 1 + Boost Software License - Version 1.0 - August 17th, 2003 2 + 3 + Permission is hereby granted, free of charge, to any person or organization 4 + obtaining a copy of the software and accompanying documentation covered by 5 + this license (the "Software") to use, reproduce, display, distribute, 6 + execute, and transmit the Software, and to prepare derivative works of the 7 + Software, and to permit third-parties to whom the Software is furnished to 8 + do so, all subject to the following: 9 + 10 + The copyright notices in the Software and this entire statement, including 11 + the above license grant, this restriction and the following disclaimer, 12 + must be included in all copies of the Software, in whole or in part, and 13 + all derivative works of the Software, unless such copies or derivative 14 + works are solely in the form of machine-executable object code generated by 15 + a source language processor. 16 + 17 + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 18 + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 19 + FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT 20 + SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE 21 + FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, 22 + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 23 + DEALINGS IN THE SOFTWARE.

+136

compat/nedmalloc/Readme.txt

··· 1 + nedalloc v1.05 15th June 2008: 2 + -=-=-=-=-=-=-=-=-=-=-=-=-=-=-= 3 + 4 + by Niall Douglas (http://www.nedprod.com/programs/portable/nedmalloc/) 5 + 6 + Enclosed is nedalloc, an alternative malloc implementation for multiple 7 + threads without lock contention based on dlmalloc v2.8.4. It is more 8 + or less a newer implementation of ptmalloc2, the standard allocator in 9 + Linux (which is based on dlmalloc v2.7.0) but also contains a per-thread 10 + cache for maximum CPU scalability. 11 + 12 + It is licensed under the Boost Software License which basically means 13 + you can do anything you like with it. This does not apply to the malloc.c.h 14 + file which remains copyright to others. 15 + 16 + It has been tested on win32 (x86), win64 (x64), Linux (x64), FreeBSD (x64) 17 + and Apple MacOS X (x86). It works very well on all of these and is very 18 + significantly faster than the system allocator on all of these platforms. 19 + 20 + By literally dropping in this allocator as a replacement for your system 21 + allocator, you can see real world improvements of up to three times in normal 22 + code! 23 + 24 + To use: 25 + -=-=-=- 26 + Drop in nedmalloc.h, nedmalloc.c and malloc.c.h into your project. 27 + Configure using the instructions in nedmalloc.h. Run and enjoy. 28 + 29 + To test, compile test.c. It will run a comparison between your system 30 + allocator and nedalloc and tell you how much faster nedalloc is. It also 31 + serves as an example of usage. 32 + 33 + Notes: 34 + -=-=-= 35 + If you want the very latest version of this allocator, get it from the 36 + TnFOX SVN repository at svn://svn.berlios.de/viewcvs/tnfox/trunk/src/nedmalloc 37 + 38 + Because of how nedalloc allocates an mspace per thread, it can cause 39 + severe bloating of memory usage under certain allocation patterns. 40 + You can substantially reduce this wastage by setting MAXTHREADSINPOOL 41 + or the threads parameter to nedcreatepool() to a fraction of the number of 42 + threads which would normally be in a pool at once. This will reduce 43 + bloating at the cost of an increase in lock contention. If allocated size 44 + is less than THREADCACHEMAX, locking is avoided 90-99% of the time and 45 + if most of your allocations are below this value, you can safely set 46 + MAXTHREADSINPOOL to one. 47 + 48 + You will suffer memory leakage unless you call neddisablethreadcache() 49 + per pool for every thread which exits. This is because nedalloc cannot 50 + portably know when a thread exits and thus when its thread cache can 51 + be returned for use by other code. Don't forget pool zero, the system pool. 52 + 53 + For C++ type allocation patterns (where the same sizes of memory are 54 + regularly allocated and deallocated as objects are created and destroyed), 55 + the threadcache always benefits performance. If however your allocation 56 + patterns are different, searching the threadcache may significantly slow 57 + down your code - as a rule of thumb, if cache utilisation is below 80% 58 + (see the source for neddisablethreadcache() for how to enable debug 59 + printing in release mode) then you should disable the thread cache for 60 + that thread. You can compile out the threadcache code by setting 61 + THREADCACHEMAX to zero. 62 + 63 + Speed comparisons: 64 + -=-=-=-=-=-=-=-=-= 65 + See Benchmarks.xls for details. 66 + 67 + The enclosed test.c can do two things: it can be a torture test or a speed 68 + test. The speed test is designed to be a representative synthetic 69 + memory allocator test. It works by randomly mixing allocations with frees 70 + with half of the allocation sizes being a two power multiple less than 71 + 512 bytes (to mimic C++ stack instantiated objects) and the other half 72 + being a simple random value less than 16Kb. 73 + 74 + The real world code results are from Tn's TestIO benchmark. This is a 75 + heavily multithreaded and memory intensive benchmark with a lot of branching 76 + and other stuff modern processors don't like so much. As you'll note, the 77 + test doesn't show the benefits of the threadcache mostly due to the saturation 78 + of the memory bus being the limiting factor. 79 + 80 + ChangeLog: 81 + -=-=-=-=-= 82 + v1.05 15th June 2008: 83 + * { 1042 } Added error check for TLSSET() and TLSFREE() macros. Thanks to 84 + Markus Elfring for reporting this. 85 + * { 1043 } Fixed a segfault when freeing memory allocated using 86 + nedindependent_comalloc(). Thanks to Pavel Vozenilek for reporting this. 87 + 88 + v1.04 14th July 2007: 89 + * Fixed a bug with the new optimised implementation that failed to lock 90 + on a realloc under certain conditions. 91 + * Fixed lack of thread synchronisation in InitPool() causing pool corruption 92 + * Fixed a memory leak of thread cache contents on disabling. Thanks to Earl 93 + Chew for reporting this. 94 + * Added a sanity check for freed blocks being valid. 95 + * Reworked test.c into being a torture test. 96 + * Fixed GCC assembler optimisation misspecification 97 + 98 + v1.04alpha_svn915 7th October 2006: 99 + * Fixed failure to unlock thread cache list if allocating a new list failed. 100 + Thanks to Dmitry Chichkov for reporting this. Futher thanks to Aleksey Sanin. 101 + * Fixed realloc(0, <size>) segfaulting. Thanks to Dmitry Chichkov for 102 + reporting this. 103 + * Made config defines #ifndef so they can be overriden by the build system. 104 + Thanks to Aleksey Sanin for suggesting this. 105 + * Fixed deadlock in nedprealloc() due to unnecessary locking of preferred 106 + thread mspace when mspace_realloc() always uses the original block's mspace 107 + anyway. Thanks to Aleksey Sanin for reporting this. 108 + * Made some speed improvements by hacking mspace_malloc() to no longer lock 109 + its mspace, thus allowing the recursive mutex implementation to be removed 110 + with an associated speed increase. Thanks to Aleksey Sanin for suggesting this. 111 + * Fixed a bug where allocating mspaces overran its max limit. Thanks to 112 + Aleksey Sanin for reporting this. 113 + 114 + v1.03 10th July 2006: 115 + * Fixed memory corruption bug in threadcache code which only appeared with >4 116 + threads and in heavy use of the threadcache. 117 + 118 + v1.02 15th May 2006: 119 + * Integrated dlmalloc v2.8.4, fixing the win32 memory release problem and 120 + improving performance still further. Speed is now up to twice the speed of v1.01 121 + (average is 67% faster). 122 + * Fixed win32 critical section implementation. Thanks to Pavel Kuznetsov 123 + for reporting this. 124 + * Wasn't locking mspace if all mspaces were locked. Thanks to Pavel Kuznetsov 125 + for reporting this. 126 + * Added Apple Mac OS X support. 127 + 128 + v1.01 24th February 2006: 129 + * Fixed multiprocessor scaling problems by removing sources of cache sloshing 130 + * Earl Chew <earl_chew <at> agilent <dot> com> sent patches for the following: 131 + 1. size2binidx() wasn't working for default code path (non x86) 132 + 2. Fixed failure to release mspace lock under certain circumstances which 133 + caused a deadlock 134 + 135 + v1.00 1st January 2006: 136 + * First release

+5750

compat/nedmalloc/malloc.c.h

··· 1 + /* 2 + This is a version (aka dlmalloc) of malloc/free/realloc written by 3 + Doug Lea and released to the public domain, as explained at 4 + http://creativecommons.org/licenses/publicdomain. Send questions, 5 + comments, complaints, performance data, etc to dl@cs.oswego.edu 6 + 7 + * Version pre-2.8.4 Mon Nov 27 11:22:37 2006 (dl at gee) 8 + 9 + Note: There may be an updated version of this malloc obtainable at 10 + ftp://gee.cs.oswego.edu/pub/misc/malloc.c 11 + Check before installing! 12 + 13 + * Quickstart 14 + 15 + This library is all in one file to simplify the most common usage: 16 + ftp it, compile it (-O3), and link it into another program. All of 17 + the compile-time options default to reasonable values for use on 18 + most platforms. You might later want to step through various 19 + compile-time and dynamic tuning options. 20 + 21 + For convenience, an include file for code using this malloc is at: 22 + ftp://gee.cs.oswego.edu/pub/misc/malloc-2.8.4.h 23 + You don't really need this .h file unless you call functions not 24 + defined in your system include files. The .h file contains only the 25 + excerpts from this file needed for using this malloc on ANSI C/C++ 26 + systems, so long as you haven't changed compile-time options about 27 + naming and tuning parameters. If you do, then you can create your 28 + own malloc.h that does include all settings by cutting at the point 29 + indicated below. Note that you may already by default be using a C 30 + library containing a malloc that is based on some version of this 31 + malloc (for example in linux). You might still want to use the one 32 + in this file to customize settings or to avoid overheads associated 33 + with library versions. 34 + 35 + * Vital statistics: 36 + 37 + Supported pointer/size_t representation: 4 or 8 bytes 38 + size_t MUST be an unsigned type of the same width as 39 + pointers. (If you are using an ancient system that declares 40 + size_t as a signed type, or need it to be a different width 41 + than pointers, you can use a previous release of this malloc 42 + (e.g. 2.7.2) supporting these.) 43 + 44 + Alignment: 8 bytes (default) 45 + This suffices for nearly all current machines and C compilers. 46 + However, you can define MALLOC_ALIGNMENT to be wider than this 47 + if necessary (up to 128bytes), at the expense of using more space. 48 + 49 + Minimum overhead per allocated chunk: 4 or 8 bytes (if 4byte sizes) 50 + 8 or 16 bytes (if 8byte sizes) 51 + Each malloced chunk has a hidden word of overhead holding size 52 + and status information, and additional cross-check word 53 + if FOOTERS is defined. 54 + 55 + Minimum allocated size: 4-byte ptrs: 16 bytes (including overhead) 56 + 8-byte ptrs: 32 bytes (including overhead) 57 + 58 + Even a request for zero bytes (i.e., malloc(0)) returns a 59 + pointer to something of the minimum allocatable size. 60 + The maximum overhead wastage (i.e., number of extra bytes 61 + allocated than were requested in malloc) is less than or equal 62 + to the minimum size, except for requests >= mmap_threshold that 63 + are serviced via mmap(), where the worst case wastage is about 64 + 32 bytes plus the remainder from a system page (the minimal 65 + mmap unit); typically 4096 or 8192 bytes. 66 + 67 + Security: static-safe; optionally more or less 68 + The "security" of malloc refers to the ability of malicious 69 + code to accentuate the effects of errors (for example, freeing 70 + space that is not currently malloc'ed or overwriting past the 71 + ends of chunks) in code that calls malloc. This malloc 72 + guarantees not to modify any memory locations below the base of 73 + heap, i.e., static variables, even in the presence of usage 74 + errors. The routines additionally detect most improper frees 75 + and reallocs. All this holds as long as the static bookkeeping 76 + for malloc itself is not corrupted by some other means. This 77 + is only one aspect of security -- these checks do not, and 78 + cannot, detect all possible programming errors. 79 + 80 + If FOOTERS is defined nonzero, then each allocated chunk 81 + carries an additional check word to verify that it was malloced 82 + from its space. These check words are the same within each 83 + execution of a program using malloc, but differ across 84 + executions, so externally crafted fake chunks cannot be 85 + freed. This improves security by rejecting frees/reallocs that 86 + could corrupt heap memory, in addition to the checks preventing 87 + writes to statics that are always on. This may further improve 88 + security at the expense of time and space overhead. (Note that 89 + FOOTERS may also be worth using with MSPACES.) 90 + 91 + By default detected errors cause the program to abort (calling 92 + "abort()"). You can override this to instead proceed past 93 + errors by defining PROCEED_ON_ERROR. In this case, a bad free 94 + has no effect, and a malloc that encounters a bad address 95 + caused by user overwrites will ignore the bad address by 96 + dropping pointers and indices to all known memory. This may 97 + be appropriate for programs that should continue if at all 98 + possible in the face of programming errors, although they may 99 + run out of memory because dropped memory is never reclaimed. 100 + 101 + If you don't like either of these options, you can define 102 + CORRUPTION_ERROR_ACTION and USAGE_ERROR_ACTION to do anything 103 + else. And if if you are sure that your program using malloc has 104 + no errors or vulnerabilities, you can define INSECURE to 1, 105 + which might (or might not) provide a small performance improvement. 106 + 107 + Thread-safety: NOT thread-safe unless USE_LOCKS defined 108 + When USE_LOCKS is defined, each public call to malloc, free, 109 + etc is surrounded with either a pthread mutex or a win32 110 + spinlock (depending on WIN32). This is not especially fast, and 111 + can be a major bottleneck. It is designed only to provide 112 + minimal protection in concurrent environments, and to provide a 113 + basis for extensions. If you are using malloc in a concurrent 114 + program, consider instead using nedmalloc 115 + (http://www.nedprod.com/programs/portable/nedmalloc/) or 116 + ptmalloc (See http://www.malloc.de), which are derived 117 + from versions of this malloc. 118 + 119 + System requirements: Any combination of MORECORE and/or MMAP/MUNMAP 120 + This malloc can use unix sbrk or any emulation (invoked using 121 + the CALL_MORECORE macro) and/or mmap/munmap or any emulation 122 + (invoked using CALL_MMAP/CALL_MUNMAP) to get and release system 123 + memory. On most unix systems, it tends to work best if both 124 + MORECORE and MMAP are enabled. On Win32, it uses emulations 125 + based on VirtualAlloc. It also uses common C library functions 126 + like memset. 127 + 128 + Compliance: I believe it is compliant with the Single Unix Specification 129 + (See http://www.unix.org). Also SVID/XPG, ANSI C, and probably 130 + others as well. 131 + 132 + * Overview of algorithms 133 + 134 + This is not the fastest, most space-conserving, most portable, or 135 + most tunable malloc ever written. However it is among the fastest 136 + while also being among the most space-conserving, portable and 137 + tunable. Consistent balance across these factors results in a good 138 + general-purpose allocator for malloc-intensive programs. 139 + 140 + In most ways, this malloc is a best-fit allocator. Generally, it 141 + chooses the best-fitting existing chunk for a request, with ties 142 + broken in approximately least-recently-used order. (This strategy 143 + normally maintains low fragmentation.) However, for requests less 144 + than 256bytes, it deviates from best-fit when there is not an 145 + exactly fitting available chunk by preferring to use space adjacent 146 + to that used for the previous small request, as well as by breaking 147 + ties in approximately most-recently-used order. (These enhance 148 + locality of series of small allocations.) And for very large requests 149 + (>= 256Kb by default), it relies on system memory mapping 150 + facilities, if supported. (This helps avoid carrying around and 151 + possibly fragmenting memory used only for large chunks.) 152 + 153 + All operations (except malloc_stats and mallinfo) have execution 154 + times that are bounded by a constant factor of the number of bits in 155 + a size_t, not counting any clearing in calloc or copying in realloc, 156 + or actions surrounding MORECORE and MMAP that have times 157 + proportional to the number of non-contiguous regions returned by 158 + system allocation routines, which is often just 1. In real-time 159 + applications, you can optionally suppress segment traversals using 160 + NO_SEGMENT_TRAVERSAL, which assures bounded execution even when 161 + system allocators return non-contiguous spaces, at the typical 162 + expense of carrying around more memory and increased fragmentation. 163 + 164 + The implementation is not very modular and seriously overuses 165 + macros. Perhaps someday all C compilers will do as good a job 166 + inlining modular code as can now be done by brute-force expansion, 167 + but now, enough of them seem not to. 168 + 169 + Some compilers issue a lot of warnings about code that is 170 + dead/unreachable only on some platforms, and also about intentional 171 + uses of negation on unsigned types. All known cases of each can be 172 + ignored. 173 + 174 + For a longer but out of date high-level description, see 175 + http://gee.cs.oswego.edu/dl/html/malloc.html 176 + 177 + * MSPACES 178 + If MSPACES is defined, then in addition to malloc, free, etc., 179 + this file also defines mspace_malloc, mspace_free, etc. These 180 + are versions of malloc routines that take an "mspace" argument 181 + obtained using create_mspace, to control all internal bookkeeping. 182 + If ONLY_MSPACES is defined, only these versions are compiled. 183 + So if you would like to use this allocator for only some allocations, 184 + and your system malloc for others, you can compile with 185 + ONLY_MSPACES and then do something like... 186 + static mspace mymspace = create_mspace(0,0); // for example 187 + #define mymalloc(bytes) mspace_malloc(mymspace, bytes) 188 + 189 + (Note: If you only need one instance of an mspace, you can instead 190 + use "USE_DL_PREFIX" to relabel the global malloc.) 191 + 192 + You can similarly create thread-local allocators by storing 193 + mspaces as thread-locals. For example: 194 + static __thread mspace tlms = 0; 195 + void* tlmalloc(size_t bytes) { 196 + if (tlms == 0) tlms = create_mspace(0, 0); 197 + return mspace_malloc(tlms, bytes); 198 + } 199 + void tlfree(void* mem) { mspace_free(tlms, mem); } 200 + 201 + Unless FOOTERS is defined, each mspace is completely independent. 202 + You cannot allocate from one and free to another (although 203 + conformance is only weakly checked, so usage errors are not always 204 + caught). If FOOTERS is defined, then each chunk carries around a tag 205 + indicating its originating mspace, and frees are directed to their 206 + originating spaces. 207 + 208 + ------------------------- Compile-time options --------------------------- 209 + 210 + Be careful in setting #define values for numerical constants of type 211 + size_t. On some systems, literal values are not automatically extended 212 + to size_t precision unless they are explicitly casted. You can also 213 + use the symbolic values MAX_SIZE_T, SIZE_T_ONE, etc below. 214 + 215 + WIN32 default: defined if _WIN32 defined 216 + Defining WIN32 sets up defaults for MS environment and compilers. 217 + Otherwise defaults are for unix. Beware that there seem to be some 218 + cases where this malloc might not be a pure drop-in replacement for 219 + Win32 malloc: Random-looking failures from Win32 GDI API's (eg; 220 + SetDIBits()) may be due to bugs in some video driver implementations 221 + when pixel buffers are malloc()ed, and the region spans more than 222 + one VirtualAlloc()ed region. Because dlmalloc uses a small (64Kb) 223 + default granularity, pixel buffers may straddle virtual allocation 224 + regions more often than when using the Microsoft allocator. You can 225 + avoid this by using VirtualAlloc() and VirtualFree() for all pixel 226 + buffers rather than using malloc(). If this is not possible, 227 + recompile this malloc with a larger DEFAULT_GRANULARITY. 228 + 229 + MALLOC_ALIGNMENT default: (size_t)8 230 + Controls the minimum alignment for malloc'ed chunks. It must be a 231 + power of two and at least 8, even on machines for which smaller 232 + alignments would suffice. It may be defined as larger than this 233 + though. Note however that code and data structures are optimized for 234 + the case of 8-byte alignment. 235 + 236 + MSPACES default: 0 (false) 237 + If true, compile in support for independent allocation spaces. 238 + This is only supported if HAVE_MMAP is true. 239 + 240 + ONLY_MSPACES default: 0 (false) 241 + If true, only compile in mspace versions, not regular versions. 242 + 243 + USE_LOCKS default: 0 (false) 244 + Causes each call to each public routine to be surrounded with 245 + pthread or WIN32 mutex lock/unlock. (If set true, this can be 246 + overridden on a per-mspace basis for mspace versions.) If set to a 247 + non-zero value other than 1, locks are used, but their 248 + implementation is left out, so lock functions must be supplied manually. 249 + 250 + USE_SPIN_LOCKS default: 1 iff USE_LOCKS and on x86 using gcc or MSC 251 + If true, uses custom spin locks for locking. This is currently 252 + supported only for x86 platforms using gcc or recent MS compilers. 253 + Otherwise, posix locks or win32 critical sections are used. 254 + 255 + FOOTERS default: 0 256 + If true, provide extra checking and dispatching by placing 257 + information in the footers of allocated chunks. This adds 258 + space and time overhead. 259 + 260 + INSECURE default: 0 261 + If true, omit checks for usage errors and heap space overwrites. 262 + 263 + USE_DL_PREFIX default: NOT defined 264 + Causes compiler to prefix all public routines with the string 'dl'. 265 + This can be useful when you only want to use this malloc in one part 266 + of a program, using your regular system malloc elsewhere. 267 + 268 + ABORT default: defined as abort() 269 + Defines how to abort on failed checks. On most systems, a failed 270 + check cannot die with an "assert" or even print an informative 271 + message, because the underlying print routines in turn call malloc, 272 + which will fail again. Generally, the best policy is to simply call 273 + abort(). It's not very useful to do more than this because many 274 + errors due to overwriting will show up as address faults (null, odd 275 + addresses etc) rather than malloc-triggered checks, so will also 276 + abort. Also, most compilers know that abort() does not return, so 277 + can better optimize code conditionally calling it. 278 + 279 + PROCEED_ON_ERROR default: defined as 0 (false) 280 + Controls whether detected bad addresses cause them to bypassed 281 + rather than aborting. If set, detected bad arguments to free and 282 + realloc are ignored. And all bookkeeping information is zeroed out 283 + upon a detected overwrite of freed heap space, thus losing the 284 + ability to ever return it from malloc again, but enabling the 285 + application to proceed. If PROCEED_ON_ERROR is defined, the 286 + static variable malloc_corruption_error_count is compiled in 287 + and can be examined to see if errors have occurred. This option 288 + generates slower code than the default abort policy. 289 + 290 + DEBUG default: NOT defined 291 + The DEBUG setting is mainly intended for people trying to modify 292 + this code or diagnose problems when porting to new platforms. 293 + However, it may also be able to better isolate user errors than just 294 + using runtime checks. The assertions in the check routines spell 295 + out in more detail the assumptions and invariants underlying the 296 + algorithms. The checking is fairly extensive, and will slow down 297 + execution noticeably. Calling malloc_stats or mallinfo with DEBUG 298 + set will attempt to check every non-mmapped allocated and free chunk 299 + in the course of computing the summaries. 300 + 301 + ABORT_ON_ASSERT_FAILURE default: defined as 1 (true) 302 + Debugging assertion failures can be nearly impossible if your 303 + version of the assert macro causes malloc to be called, which will 304 + lead to a cascade of further failures, blowing the runtime stack. 305 + ABORT_ON_ASSERT_FAILURE cause assertions failures to call abort(), 306 + which will usually make debugging easier. 307 + 308 + MALLOC_FAILURE_ACTION default: sets errno to ENOMEM, or no-op on win32 309 + The action to take before "return 0" when malloc fails to be able to 310 + return memory because there is none available. 311 + 312 + HAVE_MORECORE default: 1 (true) unless win32 or ONLY_MSPACES 313 + True if this system supports sbrk or an emulation of it. 314 + 315 + MORECORE default: sbrk 316 + The name of the sbrk-style system routine to call to obtain more 317 + memory. See below for guidance on writing custom MORECORE 318 + functions. The type of the argument to sbrk/MORECORE varies across 319 + systems. It cannot be size_t, because it supports negative 320 + arguments, so it is normally the signed type of the same width as 321 + size_t (sometimes declared as "intptr_t"). It doesn't much matter 322 + though. Internally, we only call it with arguments less than half 323 + the max value of a size_t, which should work across all reasonable 324 + possibilities, although sometimes generating compiler warnings. 325 + 326 + MORECORE_CONTIGUOUS default: 1 (true) if HAVE_MORECORE 327 + If true, take advantage of fact that consecutive calls to MORECORE 328 + with positive arguments always return contiguous increasing 329 + addresses. This is true of unix sbrk. It does not hurt too much to 330 + set it true anyway, since malloc copes with non-contiguities. 331 + Setting it false when definitely non-contiguous saves time 332 + and possibly wasted space it would take to discover this though. 333 + 334 + MORECORE_CANNOT_TRIM default: NOT defined 335 + True if MORECORE cannot release space back to the system when given 336 + negative arguments. This is generally necessary only if you are 337 + using a hand-crafted MORECORE function that cannot handle negative 338 + arguments. 339 + 340 + NO_SEGMENT_TRAVERSAL default: 0 341 + If non-zero, suppresses traversals of memory segments 342 + returned by either MORECORE or CALL_MMAP. This disables 343 + merging of segments that are contiguous, and selectively 344 + releasing them to the OS if unused, but bounds execution times. 345 + 346 + HAVE_MMAP default: 1 (true) 347 + True if this system supports mmap or an emulation of it. If so, and 348 + HAVE_MORECORE is not true, MMAP is used for all system 349 + allocation. If set and HAVE_MORECORE is true as well, MMAP is 350 + primarily used to directly allocate very large blocks. It is also 351 + used as a backup strategy in cases where MORECORE fails to provide 352 + space from system. Note: A single call to MUNMAP is assumed to be 353 + able to unmap memory that may have be allocated using multiple calls 354 + to MMAP, so long as they are adjacent. 355 + 356 + HAVE_MREMAP default: 1 on linux, else 0 357 + If true realloc() uses mremap() to re-allocate large blocks and 358 + extend or shrink allocation spaces. 359 + 360 + MMAP_CLEARS default: 1 except on WINCE. 361 + True if mmap clears memory so calloc doesn't need to. This is true 362 + for standard unix mmap using /dev/zero and on WIN32 except for WINCE. 363 + 364 + USE_BUILTIN_FFS default: 0 (i.e., not used) 365 + Causes malloc to use the builtin ffs() function to compute indices. 366 + Some compilers may recognize and intrinsify ffs to be faster than the 367 + supplied C version. Also, the case of x86 using gcc is special-cased 368 + to an asm instruction, so is already as fast as it can be, and so 369 + this setting has no effect. Similarly for Win32 under recent MS compilers. 370 + (On most x86s, the asm version is only slightly faster than the C version.) 371 + 372 + malloc_getpagesize default: derive from system includes, or 4096. 373 + The system page size. To the extent possible, this malloc manages 374 + memory from the system in page-size units. This may be (and 375 + usually is) a function rather than a constant. This is ignored 376 + if WIN32, where page size is determined using getSystemInfo during 377 + initialization. 378 + 379 + USE_DEV_RANDOM default: 0 (i.e., not used) 380 + Causes malloc to use /dev/random to initialize secure magic seed for 381 + stamping footers. Otherwise, the current time is used. 382 + 383 + NO_MALLINFO default: 0 384 + If defined, don't compile "mallinfo". This can be a simple way 385 + of dealing with mismatches between system declarations and 386 + those in this file. 387 + 388 + MALLINFO_FIELD_TYPE default: size_t 389 + The type of the fields in the mallinfo struct. This was originally 390 + defined as "int" in SVID etc, but is more usefully defined as 391 + size_t. The value is used only if HAVE_USR_INCLUDE_MALLOC_H is not set 392 + 393 + REALLOC_ZERO_BYTES_FREES default: not defined 394 + This should be set if a call to realloc with zero bytes should 395 + be the same as a call to free. Some people think it should. Otherwise, 396 + since this malloc returns a unique pointer for malloc(0), so does 397 + realloc(p, 0). 398 + 399 + LACKS_UNISTD_H, LACKS_FCNTL_H, LACKS_SYS_PARAM_H, LACKS_SYS_MMAN_H 400 + LACKS_STRINGS_H, LACKS_STRING_H, LACKS_SYS_TYPES_H, LACKS_ERRNO_H 401 + LACKS_STDLIB_H default: NOT defined unless on WIN32 402 + Define these if your system does not have these header files. 403 + You might need to manually insert some of the declarations they provide. 404 + 405 + DEFAULT_GRANULARITY default: page size if MORECORE_CONTIGUOUS, 406 + system_info.dwAllocationGranularity in WIN32, 407 + otherwise 64K. 408 + Also settable using mallopt(M_GRANULARITY, x) 409 + The unit for allocating and deallocating memory from the system. On 410 + most systems with contiguous MORECORE, there is no reason to 411 + make this more than a page. However, systems with MMAP tend to 412 + either require or encourage larger granularities. You can increase 413 + this value to prevent system allocation functions to be called so 414 + often, especially if they are slow. The value must be at least one 415 + page and must be a power of two. Setting to 0 causes initialization 416 + to either page size or win32 region size. (Note: In previous 417 + versions of malloc, the equivalent of this option was called 418 + "TOP_PAD") 419 + 420 + DEFAULT_TRIM_THRESHOLD default: 2MB 421 + Also settable using mallopt(M_TRIM_THRESHOLD, x) 422 + The maximum amount of unused top-most memory to keep before 423 + releasing via malloc_trim in free(). Automatic trimming is mainly 424 + useful in long-lived programs using contiguous MORECORE. Because 425 + trimming via sbrk can be slow on some systems, and can sometimes be 426 + wasteful (in cases where programs immediately afterward allocate 427 + more large chunks) the value should be high enough so that your 428 + overall system performance would improve by releasing this much 429 + memory. As a rough guide, you might set to a value close to the 430 + average size of a process (program) running on your system. 431 + Releasing this much memory would allow such a process to run in 432 + memory. Generally, it is worth tuning trim thresholds when a 433 + program undergoes phases where several large chunks are allocated 434 + and released in ways that can reuse each other's storage, perhaps 435 + mixed with phases where there are no such chunks at all. The trim 436 + value must be greater than page size to have any useful effect. To 437 + disable trimming completely, you can set to MAX_SIZE_T. Note that the trick 438 + some people use of mallocing a huge space and then freeing it at 439 + program startup, in an attempt to reserve system memory, doesn't 440 + have the intended effect under automatic trimming, since that memory 441 + will immediately be returned to the system. 442 + 443 + DEFAULT_MMAP_THRESHOLD default: 256K 444 + Also settable using mallopt(M_MMAP_THRESHOLD, x) 445 + The request size threshold for using MMAP to directly service a 446 + request. Requests of at least this size that cannot be allocated 447 + using already-existing space will be serviced via mmap. (If enough 448 + normal freed space already exists it is used instead.) Using mmap 449 + segregates relatively large chunks of memory so that they can be 450 + individually obtained and released from the host system. A request 451 + serviced through mmap is never reused by any other request (at least 452 + not directly; the system may just so happen to remap successive 453 + requests to the same locations). Segregating space in this way has 454 + the benefits that: Mmapped space can always be individually released 455 + back to the system, which helps keep the system level memory demands 456 + of a long-lived program low. Also, mapped memory doesn't become 457 + `locked' between other chunks, as can happen with normally allocated 458 + chunks, which means that even trimming via malloc_trim would not 459 + release them. However, it has the disadvantage that the space 460 + cannot be reclaimed, consolidated, and then used to service later 461 + requests, as happens with normal chunks. The advantages of mmap 462 + nearly always outweigh disadvantages for "large" chunks, but the 463 + value of "large" may vary across systems. The default is an 464 + empirically derived value that works well in most systems. You can 465 + disable mmap by setting to MAX_SIZE_T. 466 + 467 + MAX_RELEASE_CHECK_RATE default: 4095 unless not HAVE_MMAP 468 + The number of consolidated frees between checks to release 469 + unused segments when freeing. When using non-contiguous segments, 470 + especially with multiple mspaces, checking only for topmost space 471 + doesn't always suffice to trigger trimming. To compensate for this, 472 + free() will, with a period of MAX_RELEASE_CHECK_RATE (or the 473 + current number of segments, if greater) try to release unused 474 + segments to the OS when freeing chunks that result in 475 + consolidation. The best value for this parameter is a compromise 476 + between slowing down frees with relatively costly checks that 477 + rarely trigger versus holding on to unused memory. To effectively 478 + disable, set to MAX_SIZE_T. This may lead to a very slight speed 479 + improvement at the expense of carrying around more memory. 480 + */ 481 + 482 + /* Version identifier to allow people to support multiple versions */ 483 + #ifndef DLMALLOC_VERSION 484 + #define DLMALLOC_VERSION 20804 485 + #endif /* DLMALLOC_VERSION */ 486 + 487 + #ifndef WIN32 488 + #ifdef _WIN32 489 + #define WIN32 1 490 + #endif /* _WIN32 */ 491 + #ifdef _WIN32_WCE 492 + #define LACKS_FCNTL_H 493 + #define WIN32 1 494 + #endif /* _WIN32_WCE */ 495 + #endif /* WIN32 */ 496 + #ifdef WIN32 497 + #define WIN32_LEAN_AND_MEAN 498 + #define _WIN32_WINNT 0x403 499 + #include <windows.h> 500 + #define HAVE_MMAP 1 501 + #define HAVE_MORECORE 0 502 + #define LACKS_UNISTD_H 503 + #define LACKS_SYS_PARAM_H 504 + #define LACKS_SYS_MMAN_H 505 + #define LACKS_STRING_H 506 + #define LACKS_STRINGS_H 507 + #define LACKS_SYS_TYPES_H 508 + #define LACKS_ERRNO_H 509 + #ifndef MALLOC_FAILURE_ACTION 510 + #define MALLOC_FAILURE_ACTION 511 + #endif /* MALLOC_FAILURE_ACTION */ 512 + #ifdef _WIN32_WCE /* WINCE reportedly does not clear */ 513 + #define MMAP_CLEARS 0 514 + #else 515 + #define MMAP_CLEARS 1 516 + #endif /* _WIN32_WCE */ 517 + #endif /* WIN32 */ 518 + 519 + #if defined(DARWIN) || defined(_DARWIN) 520 + /* Mac OSX docs advise not to use sbrk; it seems better to use mmap */ 521 + #ifndef HAVE_MORECORE 522 + #define HAVE_MORECORE 0 523 + #define HAVE_MMAP 1 524 + /* OSX allocators provide 16 byte alignment */ 525 + #ifndef MALLOC_ALIGNMENT 526 + #define MALLOC_ALIGNMENT ((size_t)16U) 527 + #endif 528 + #endif /* HAVE_MORECORE */ 529 + #endif /* DARWIN */ 530 + 531 + #ifndef LACKS_SYS_TYPES_H 532 + #include <sys/types.h> /* For size_t */ 533 + #endif /* LACKS_SYS_TYPES_H */ 534 + 535 + /* The maximum possible size_t value has all bits set */ 536 + #define MAX_SIZE_T (~(size_t)0) 537 + 538 + #ifndef ONLY_MSPACES 539 + #define ONLY_MSPACES 0 /* define to a value */ 540 + #else 541 + #define ONLY_MSPACES 1 542 + #endif /* ONLY_MSPACES */ 543 + #ifndef MSPACES 544 + #if ONLY_MSPACES 545 + #define MSPACES 1 546 + #else /* ONLY_MSPACES */ 547 + #define MSPACES 0 548 + #endif /* ONLY_MSPACES */ 549 + #endif /* MSPACES */ 550 + #ifndef MALLOC_ALIGNMENT 551 + #define MALLOC_ALIGNMENT ((size_t)8U) 552 + #endif /* MALLOC_ALIGNMENT */ 553 + #ifndef FOOTERS 554 + #define FOOTERS 0 555 + #endif /* FOOTERS */ 556 + #ifndef ABORT 557 + #define ABORT abort() 558 + #endif /* ABORT */ 559 + #ifndef ABORT_ON_ASSERT_FAILURE 560 + #define ABORT_ON_ASSERT_FAILURE 1 561 + #endif /* ABORT_ON_ASSERT_FAILURE */ 562 + #ifndef PROCEED_ON_ERROR 563 + #define PROCEED_ON_ERROR 0 564 + #endif /* PROCEED_ON_ERROR */ 565 + #ifndef USE_LOCKS 566 + #define USE_LOCKS 0 567 + #endif /* USE_LOCKS */ 568 + #ifndef USE_SPIN_LOCKS 569 + #if USE_LOCKS && (defined(__GNUC__) && ((defined(__i386__) || defined(__x86_64__)))) || (defined(_MSC_VER) && _MSC_VER>=1310) 570 + #define USE_SPIN_LOCKS 1 571 + #else 572 + #define USE_SPIN_LOCKS 0 573 + #endif /* USE_LOCKS && ... */ 574 + #endif /* USE_SPIN_LOCKS */ 575 + #ifndef INSECURE 576 + #define INSECURE 0 577 + #endif /* INSECURE */ 578 + #ifndef HAVE_MMAP 579 + #define HAVE_MMAP 1 580 + #endif /* HAVE_MMAP */ 581 + #ifndef MMAP_CLEARS 582 + #define MMAP_CLEARS 1 583 + #endif /* MMAP_CLEARS */ 584 + #ifndef HAVE_MREMAP 585 + #ifdef linux 586 + #define HAVE_MREMAP 1 587 + #else /* linux */ 588 + #define HAVE_MREMAP 0 589 + #endif /* linux */ 590 + #endif /* HAVE_MREMAP */ 591 + #ifndef MALLOC_FAILURE_ACTION 592 + #define MALLOC_FAILURE_ACTION errno = ENOMEM; 593 + #endif /* MALLOC_FAILURE_ACTION */ 594 + #ifndef HAVE_MORECORE 595 + #if ONLY_MSPACES 596 + #define HAVE_MORECORE 0 597 + #else /* ONLY_MSPACES */ 598 + #define HAVE_MORECORE 1 599 + #endif /* ONLY_MSPACES */ 600 + #endif /* HAVE_MORECORE */ 601 + #if !HAVE_MORECORE 602 + #define MORECORE_CONTIGUOUS 0 603 + #else /* !HAVE_MORECORE */ 604 + #define MORECORE_DEFAULT sbrk 605 + #ifndef MORECORE_CONTIGUOUS 606 + #define MORECORE_CONTIGUOUS 1 607 + #endif /* MORECORE_CONTIGUOUS */ 608 + #endif /* HAVE_MORECORE */ 609 + #ifndef DEFAULT_GRANULARITY 610 + #if (MORECORE_CONTIGUOUS || defined(WIN32)) 611 + #define DEFAULT_GRANULARITY (0) /* 0 means to compute in init_mparams */ 612 + #else /* MORECORE_CONTIGUOUS */ 613 + #define DEFAULT_GRANULARITY ((size_t)64U * (size_t)1024U) 614 + #endif /* MORECORE_CONTIGUOUS */ 615 + #endif /* DEFAULT_GRANULARITY */ 616 + #ifndef DEFAULT_TRIM_THRESHOLD 617 + #ifndef MORECORE_CANNOT_TRIM 618 + #define DEFAULT_TRIM_THRESHOLD ((size_t)2U * (size_t)1024U * (size_t)1024U) 619 + #else /* MORECORE_CANNOT_TRIM */ 620 + #define DEFAULT_TRIM_THRESHOLD MAX_SIZE_T 621 + #endif /* MORECORE_CANNOT_TRIM */ 622 + #endif /* DEFAULT_TRIM_THRESHOLD */ 623 + #ifndef DEFAULT_MMAP_THRESHOLD 624 + #if HAVE_MMAP 625 + #define DEFAULT_MMAP_THRESHOLD ((size_t)256U * (size_t)1024U) 626 + #else /* HAVE_MMAP */ 627 + #define DEFAULT_MMAP_THRESHOLD MAX_SIZE_T 628 + #endif /* HAVE_MMAP */ 629 + #endif /* DEFAULT_MMAP_THRESHOLD */ 630 + #ifndef MAX_RELEASE_CHECK_RATE 631 + #if HAVE_MMAP 632 + #define MAX_RELEASE_CHECK_RATE 4095 633 + #else 634 + #define MAX_RELEASE_CHECK_RATE MAX_SIZE_T 635 + #endif /* HAVE_MMAP */ 636 + #endif /* MAX_RELEASE_CHECK_RATE */ 637 + #ifndef USE_BUILTIN_FFS 638 + #define USE_BUILTIN_FFS 0 639 + #endif /* USE_BUILTIN_FFS */ 640 + #ifndef USE_DEV_RANDOM 641 + #define USE_DEV_RANDOM 0 642 + #endif /* USE_DEV_RANDOM */ 643 + #ifndef NO_MALLINFO 644 + #define NO_MALLINFO 0 645 + #endif /* NO_MALLINFO */ 646 + #ifndef MALLINFO_FIELD_TYPE 647 + #define MALLINFO_FIELD_TYPE size_t 648 + #endif /* MALLINFO_FIELD_TYPE */ 649 + #ifndef NO_SEGMENT_TRAVERSAL 650 + #define NO_SEGMENT_TRAVERSAL 0 651 + #endif /* NO_SEGMENT_TRAVERSAL */ 652 + 653 + /* 654 + mallopt tuning options. SVID/XPG defines four standard parameter 655 + numbers for mallopt, normally defined in malloc.h. None of these 656 + are used in this malloc, so setting them has no effect. But this 657 + malloc does support the following options. 658 + */ 659 + 660 + #define M_TRIM_THRESHOLD (-1) 661 + #define M_GRANULARITY (-2) 662 + #define M_MMAP_THRESHOLD (-3) 663 + 664 + /* ------------------------ Mallinfo declarations ------------------------ */ 665 + 666 + #if !NO_MALLINFO 667 + /* 668 + This version of malloc supports the standard SVID/XPG mallinfo 669 + routine that returns a struct containing usage properties and 670 + statistics. It should work on any system that has a 671 + /usr/include/malloc.h defining struct mallinfo. The main 672 + declaration needed is the mallinfo struct that is returned (by-copy) 673 + by mallinfo(). The malloinfo struct contains a bunch of fields that 674 + are not even meaningful in this version of malloc. These fields are 675 + are instead filled by mallinfo() with other numbers that might be of 676 + interest. 677 + 678 + HAVE_USR_INCLUDE_MALLOC_H should be set if you have a 679 + /usr/include/malloc.h file that includes a declaration of struct 680 + mallinfo. If so, it is included; else a compliant version is 681 + declared below. These must be precisely the same for mallinfo() to 682 + work. The original SVID version of this struct, defined on most 683 + systems with mallinfo, declares all fields as ints. But some others 684 + define as unsigned long. If your system defines the fields using a 685 + type of different width than listed here, you MUST #include your 686 + system version and #define HAVE_USR_INCLUDE_MALLOC_H. 687 + */ 688 + 689 + /* #define HAVE_USR_INCLUDE_MALLOC_H */ 690 + 691 + #ifdef HAVE_USR_INCLUDE_MALLOC_H 692 + #include "/usr/include/malloc.h" 693 + #else /* HAVE_USR_INCLUDE_MALLOC_H */ 694 + #ifndef STRUCT_MALLINFO_DECLARED 695 + #define STRUCT_MALLINFO_DECLARED 1 696 + struct mallinfo { 697 + MALLINFO_FIELD_TYPE arena; /* non-mmapped space allocated from system */ 698 + MALLINFO_FIELD_TYPE ordblks; /* number of free chunks */ 699 + MALLINFO_FIELD_TYPE smblks; /* always 0 */ 700 + MALLINFO_FIELD_TYPE hblks; /* always 0 */ 701 + MALLINFO_FIELD_TYPE hblkhd; /* space in mmapped regions */ 702 + MALLINFO_FIELD_TYPE usmblks; /* maximum total allocated space */ 703 + MALLINFO_FIELD_TYPE fsmblks; /* always 0 */ 704 + MALLINFO_FIELD_TYPE uordblks; /* total allocated space */ 705 + MALLINFO_FIELD_TYPE fordblks; /* total free space */ 706 + MALLINFO_FIELD_TYPE keepcost; /* releasable (via malloc_trim) space */ 707 + }; 708 + #endif /* STRUCT_MALLINFO_DECLARED */ 709 + #endif /* HAVE_USR_INCLUDE_MALLOC_H */ 710 + #endif /* NO_MALLINFO */ 711 + 712 + /* 713 + Try to persuade compilers to inline. The most critical functions for 714 + inlining are defined as macros, so these aren't used for them. 715 + */ 716 + 717 + #ifndef FORCEINLINE 718 + #if defined(__GNUC__) 719 + #define FORCEINLINE __inline __attribute__ ((always_inline)) 720 + #elif defined(_MSC_VER) 721 + #define FORCEINLINE __forceinline 722 + #endif 723 + #endif 724 + #ifndef NOINLINE 725 + #if defined(__GNUC__) 726 + #define NOINLINE __attribute__ ((noinline)) 727 + #elif defined(_MSC_VER) 728 + #define NOINLINE __declspec(noinline) 729 + #else 730 + #define NOINLINE 731 + #endif 732 + #endif 733 + 734 + #ifdef __cplusplus 735 + extern "C" { 736 + #ifndef FORCEINLINE 737 + #define FORCEINLINE inline 738 + #endif 739 + #endif /* __cplusplus */ 740 + #ifndef FORCEINLINE 741 + #define FORCEINLINE 742 + #endif 743 + 744 + #if !ONLY_MSPACES 745 + 746 + /* ------------------- Declarations of public routines ------------------- */ 747 + 748 + #ifndef USE_DL_PREFIX 749 + #define dlcalloc calloc 750 + #define dlfree free 751 + #define dlmalloc malloc 752 + #define dlmemalign memalign 753 + #define dlrealloc realloc 754 + #define dlvalloc valloc 755 + #define dlpvalloc pvalloc 756 + #define dlmallinfo mallinfo 757 + #define dlmallopt mallopt 758 + #define dlmalloc_trim malloc_trim 759 + #define dlmalloc_stats malloc_stats 760 + #define dlmalloc_usable_size malloc_usable_size 761 + #define dlmalloc_footprint malloc_footprint 762 + #define dlmalloc_max_footprint malloc_max_footprint 763 + #define dlindependent_calloc independent_calloc 764 + #define dlindependent_comalloc independent_comalloc 765 + #endif /* USE_DL_PREFIX */ 766 + 767 + 768 + /* 769 + malloc(size_t n) 770 + Returns a pointer to a newly allocated chunk of at least n bytes, or 771 + null if no space is available, in which case errno is set to ENOMEM 772 + on ANSI C systems. 773 + 774 + If n is zero, malloc returns a minimum-sized chunk. (The minimum 775 + size is 16 bytes on most 32bit systems, and 32 bytes on 64bit 776 + systems.) Note that size_t is an unsigned type, so calls with 777 + arguments that would be negative if signed are interpreted as 778 + requests for huge amounts of space, which will often fail. The 779 + maximum supported value of n differs across systems, but is in all 780 + cases less than the maximum representable value of a size_t. 781 + */ 782 + void* dlmalloc(size_t); 783 + 784 + /* 785 + free(void* p) 786 + Releases the chunk of memory pointed to by p, that had been previously 787 + allocated using malloc or a related routine such as realloc. 788 + It has no effect if p is null. If p was not malloced or already 789 + freed, free(p) will by default cause the current program to abort. 790 + */ 791 + void dlfree(void*); 792 + 793 + /* 794 + calloc(size_t n_elements, size_t element_size); 795 + Returns a pointer to n_elements * element_size bytes, with all locations 796 + set to zero. 797 + */ 798 + void* dlcalloc(size_t, size_t); 799 + 800 + /* 801 + realloc(void* p, size_t n) 802 + Returns a pointer to a chunk of size n that contains the same data 803 + as does chunk p up to the minimum of (n, p's size) bytes, or null 804 + if no space is available. 805 + 806 + The returned pointer may or may not be the same as p. The algorithm 807 + prefers extending p in most cases when possible, otherwise it 808 + employs the equivalent of a malloc-copy-free sequence. 809 + 810 + If p is null, realloc is equivalent to malloc. 811 + 812 + If space is not available, realloc returns null, errno is set (if on 813 + ANSI) and p is NOT freed. 814 + 815 + if n is for fewer bytes than already held by p, the newly unused 816 + space is lopped off and freed if possible. realloc with a size 817 + argument of zero (re)allocates a minimum-sized chunk. 818 + 819 + The old unix realloc convention of allowing the last-free'd chunk 820 + to be used as an argument to realloc is not supported. 821 + */ 822 + 823 + void* dlrealloc(void*, size_t); 824 + 825 + /* 826 + memalign(size_t alignment, size_t n); 827 + Returns a pointer to a newly allocated chunk of n bytes, aligned 828 + in accord with the alignment argument. 829 + 830 + The alignment argument should be a power of two. If the argument is 831 + not a power of two, the nearest greater power is used. 832 + 8-byte alignment is guaranteed by normal malloc calls, so don't 833 + bother calling memalign with an argument of 8 or less. 834 + 835 + Overreliance on memalign is a sure way to fragment space. 836 + */ 837 + void* dlmemalign(size_t, size_t); 838 + 839 + /* 840 + valloc(size_t n); 841 + Equivalent to memalign(pagesize, n), where pagesize is the page 842 + size of the system. If the pagesize is unknown, 4096 is used. 843 + */ 844 + void* dlvalloc(size_t); 845 + 846 + /* 847 + mallopt(int parameter_number, int parameter_value) 848 + Sets tunable parameters The format is to provide a 849 + (parameter-number, parameter-value) pair. mallopt then sets the 850 + corresponding parameter to the argument value if it can (i.e., so 851 + long as the value is meaningful), and returns 1 if successful else 852 + 0. To workaround the fact that mallopt is specified to use int, 853 + not size_t parameters, the value -1 is specially treated as the 854 + maximum unsigned size_t value. 855 + 856 + SVID/XPG/ANSI defines four standard param numbers for mallopt, 857 + normally defined in malloc.h. None of these are use in this malloc, 858 + so setting them has no effect. But this malloc also supports other 859 + options in mallopt. See below for details. Briefly, supported 860 + parameters are as follows (listed defaults are for "typical" 861 + configurations). 862 + 863 + Symbol param # default allowed param values 864 + M_TRIM_THRESHOLD -1 2*1024*1024 any (-1 disables) 865 + M_GRANULARITY -2 page size any power of 2 >= page size 866 + M_MMAP_THRESHOLD -3 256*1024 any (or 0 if no MMAP support) 867 + */ 868 + int dlmallopt(int, int); 869 + 870 + /* 871 + malloc_footprint(); 872 + Returns the number of bytes obtained from the system. The total 873 + number of bytes allocated by malloc, realloc etc., is less than this 874 + value. Unlike mallinfo, this function returns only a precomputed 875 + result, so can be called frequently to monitor memory consumption. 876 + Even if locks are otherwise defined, this function does not use them, 877 + so results might not be up to date. 878 + */ 879 + size_t dlmalloc_footprint(void); 880 + 881 + /* 882 + malloc_max_footprint(); 883 + Returns the maximum number of bytes obtained from the system. This 884 + value will be greater than current footprint if deallocated space 885 + has been reclaimed by the system. The peak number of bytes allocated 886 + by malloc, realloc etc., is less than this value. Unlike mallinfo, 887 + this function returns only a precomputed result, so can be called 888 + frequently to monitor memory consumption. Even if locks are 889 + otherwise defined, this function does not use them, so results might 890 + not be up to date. 891 + */ 892 + size_t dlmalloc_max_footprint(void); 893 + 894 + #if !NO_MALLINFO 895 + /* 896 + mallinfo() 897 + Returns (by copy) a struct containing various summary statistics: 898 + 899 + arena: current total non-mmapped bytes allocated from system 900 + ordblks: the number of free chunks 901 + smblks: always zero. 902 + hblks: current number of mmapped regions 903 + hblkhd: total bytes held in mmapped regions 904 + usmblks: the maximum total allocated space. This will be greater 905 + than current total if trimming has occurred. 906 + fsmblks: always zero 907 + uordblks: current total allocated space (normal or mmapped) 908 + fordblks: total free space 909 + keepcost: the maximum number of bytes that could ideally be released 910 + back to system via malloc_trim. ("ideally" means that 911 + it ignores page restrictions etc.) 912 + 913 + Because these fields are ints, but internal bookkeeping may 914 + be kept as longs, the reported values may wrap around zero and 915 + thus be inaccurate. 916 + */ 917 + struct mallinfo dlmallinfo(void); 918 + #endif /* NO_MALLINFO */ 919 + 920 + /* 921 + independent_calloc(size_t n_elements, size_t element_size, void* chunks[]); 922 + 923 + independent_calloc is similar to calloc, but instead of returning a 924 + single cleared space, it returns an array of pointers to n_elements 925 + independent elements that can hold contents of size elem_size, each 926 + of which starts out cleared, and can be independently freed, 927 + realloc'ed etc. The elements are guaranteed to be adjacently 928 + allocated (this is not guaranteed to occur with multiple callocs or 929 + mallocs), which may also improve cache locality in some 930 + applications. 931 + 932 + The "chunks" argument is optional (i.e., may be null, which is 933 + probably the most typical usage). If it is null, the returned array 934 + is itself dynamically allocated and should also be freed when it is 935 + no longer needed. Otherwise, the chunks array must be of at least 936 + n_elements in length. It is filled in with the pointers to the 937 + chunks. 938 + 939 + In either case, independent_calloc returns this pointer array, or 940 + null if the allocation failed. If n_elements is zero and "chunks" 941 + is null, it returns a chunk representing an array with zero elements 942 + (which should be freed if not wanted). 943 + 944 + Each element must be individually freed when it is no longer 945 + needed. If you'd like to instead be able to free all at once, you 946 + should instead use regular calloc and assign pointers into this 947 + space to represent elements. (In this case though, you cannot 948 + independently free elements.) 949 + 950 + independent_calloc simplifies and speeds up implementations of many 951 + kinds of pools. It may also be useful when constructing large data 952 + structures that initially have a fixed number of fixed-sized nodes, 953 + but the number is not known at compile time, and some of the nodes 954 + may later need to be freed. For example: 955 + 956 + struct Node { int item; struct Node* next; }; 957 + 958 + struct Node* build_list() { 959 + struct Node** pool; 960 + int n = read_number_of_nodes_needed(); 961 + if (n <= 0) return 0; 962 + pool = (struct Node**)(independent_calloc(n, sizeof(struct Node), 0); 963 + if (pool == 0) die(); 964 + // organize into a linked list... 965 + struct Node* first = pool[0]; 966 + for (i = 0; i < n-1; ++i) 967 + pool[i]->next = pool[i+1]; 968 + free(pool); // Can now free the array (or not, if it is needed later) 969 + return first; 970 + } 971 + */ 972 + void** dlindependent_calloc(size_t, size_t, void**); 973 + 974 + /* 975 + independent_comalloc(size_t n_elements, size_t sizes[], void* chunks[]); 976 + 977 + independent_comalloc allocates, all at once, a set of n_elements 978 + chunks with sizes indicated in the "sizes" array. It returns 979 + an array of pointers to these elements, each of which can be 980 + independently freed, realloc'ed etc. The elements are guaranteed to 981 + be adjacently allocated (this is not guaranteed to occur with 982 + multiple callocs or mallocs), which may also improve cache locality 983 + in some applications. 984 + 985 + The "chunks" argument is optional (i.e., may be null). If it is null 986 + the returned array is itself dynamically allocated and should also 987 + be freed when it is no longer needed. Otherwise, the chunks array 988 + must be of at least n_elements in length. It is filled in with the 989 + pointers to the chunks. 990 + 991 + In either case, independent_comalloc returns this pointer array, or 992 + null if the allocation failed. If n_elements is zero and chunks is 993 + null, it returns a chunk representing an array with zero elements 994 + (which should be freed if not wanted). 995 + 996 + Each element must be individually freed when it is no longer 997 + needed. If you'd like to instead be able to free all at once, you 998 + should instead use a single regular malloc, and assign pointers at 999 + particular offsets in the aggregate space. (In this case though, you 1000 + cannot independently free elements.) 1001 + 1002 + independent_comallac differs from independent_calloc in that each 1003 + element may have a different size, and also that it does not 1004 + automatically clear elements. 1005 + 1006 + independent_comalloc can be used to speed up allocation in cases 1007 + where several structs or objects must always be allocated at the 1008 + same time. For example: 1009 + 1010 + struct Head { ... } 1011 + struct Foot { ... } 1012 + 1013 + void send_message(char* msg) { 1014 + int msglen = strlen(msg); 1015 + size_t sizes[3] = { sizeof(struct Head), msglen, sizeof(struct Foot) }; 1016 + void* chunks[3]; 1017 + if (independent_comalloc(3, sizes, chunks) == 0) 1018 + die(); 1019 + struct Head* head = (struct Head*)(chunks[0]); 1020 + char* body = (char*)(chunks[1]); 1021 + struct Foot* foot = (struct Foot*)(chunks[2]); 1022 + // ... 1023 + } 1024 + 1025 + In general though, independent_comalloc is worth using only for 1026 + larger values of n_elements. For small values, you probably won't 1027 + detect enough difference from series of malloc calls to bother. 1028 + 1029 + Overuse of independent_comalloc can increase overall memory usage, 1030 + since it cannot reuse existing noncontiguous small chunks that 1031 + might be available for some of the elements. 1032 + */ 1033 + void** dlindependent_comalloc(size_t, size_t*, void**); 1034 + 1035 + 1036 + /* 1037 + pvalloc(size_t n); 1038 + Equivalent to valloc(minimum-page-that-holds(n)), that is, 1039 + round up n to nearest pagesize. 1040 + */ 1041 + void* dlpvalloc(size_t); 1042 + 1043 + /* 1044 + malloc_trim(size_t pad); 1045 + 1046 + If possible, gives memory back to the system (via negative arguments 1047 + to sbrk) if there is unused memory at the `high' end of the malloc 1048 + pool or in unused MMAP segments. You can call this after freeing 1049 + large blocks of memory to potentially reduce the system-level memory 1050 + requirements of a program. However, it cannot guarantee to reduce 1051 + memory. Under some allocation patterns, some large free blocks of 1052 + memory will be locked between two used chunks, so they cannot be 1053 + given back to the system. 1054 + 1055 + The `pad' argument to malloc_trim represents the amount of free 1056 + trailing space to leave untrimmed. If this argument is zero, only 1057 + the minimum amount of memory to maintain internal data structures 1058 + will be left. Non-zero arguments can be supplied to maintain enough 1059 + trailing space to service future expected allocations without having 1060 + to re-obtain memory from the system. 1061 + 1062 + Malloc_trim returns 1 if it actually released any memory, else 0. 1063 + */ 1064 + int dlmalloc_trim(size_t); 1065 + 1066 + /* 1067 + malloc_stats(); 1068 + Prints on stderr the amount of space obtained from the system (both 1069 + via sbrk and mmap), the maximum amount (which may be more than 1070 + current if malloc_trim and/or munmap got called), and the current 1071 + number of bytes allocated via malloc (or realloc, etc) but not yet 1072 + freed. Note that this is the number of bytes allocated, not the 1073 + number requested. It will be larger than the number requested 1074 + because of alignment and bookkeeping overhead. Because it includes 1075 + alignment wastage as being in use, this figure may be greater than 1076 + zero even when no user-level chunks are allocated. 1077 + 1078 + The reported current and maximum system memory can be inaccurate if 1079 + a program makes other calls to system memory allocation functions 1080 + (normally sbrk) outside of malloc. 1081 + 1082 + malloc_stats prints only the most commonly interesting statistics. 1083 + More information can be obtained by calling mallinfo. 1084 + */ 1085 + void dlmalloc_stats(void); 1086 + 1087 + #endif /* ONLY_MSPACES */ 1088 + 1089 + /* 1090 + malloc_usable_size(void* p); 1091 + 1092 + Returns the number of bytes you can actually use in 1093 + an allocated chunk, which may be more than you requested (although 1094 + often not) due to alignment and minimum size constraints. 1095 + You can use this many bytes without worrying about 1096 + overwriting other allocated objects. This is not a particularly great 1097 + programming practice. malloc_usable_size can be more useful in 1098 + debugging and assertions, for example: 1099 + 1100 + p = malloc(n); 1101 + assert(malloc_usable_size(p) >= 256); 1102 + */ 1103 + size_t dlmalloc_usable_size(void*); 1104 + 1105 + 1106 + #if MSPACES 1107 + 1108 + /* 1109 + mspace is an opaque type representing an independent 1110 + region of space that supports mspace_malloc, etc. 1111 + */ 1112 + typedef void* mspace; 1113 + 1114 + /* 1115 + create_mspace creates and returns a new independent space with the 1116 + given initial capacity, or, if 0, the default granularity size. It 1117 + returns null if there is no system memory available to create the 1118 + space. If argument locked is non-zero, the space uses a separate 1119 + lock to control access. The capacity of the space will grow 1120 + dynamically as needed to service mspace_malloc requests. You can 1121 + control the sizes of incremental increases of this space by 1122 + compiling with a different DEFAULT_GRANULARITY or dynamically 1123 + setting with mallopt(M_GRANULARITY, value). 1124 + */ 1125 + mspace create_mspace(size_t capacity, int locked); 1126 + 1127 + /* 1128 + destroy_mspace destroys the given space, and attempts to return all 1129 + of its memory back to the system, returning the total number of 1130 + bytes freed. After destruction, the results of access to all memory 1131 + used by the space become undefined. 1132 + */ 1133 + size_t destroy_mspace(mspace msp); 1134 + 1135 + /* 1136 + create_mspace_with_base uses the memory supplied as the initial base 1137 + of a new mspace. Part (less than 128*sizeof(size_t) bytes) of this 1138 + space is used for bookkeeping, so the capacity must be at least this 1139 + large. (Otherwise 0 is returned.) When this initial space is 1140 + exhausted, additional memory will be obtained from the system. 1141 + Destroying this space will deallocate all additionally allocated 1142 + space (if possible) but not the initial base. 1143 + */ 1144 + mspace create_mspace_with_base(void* base, size_t capacity, int locked); 1145 + 1146 + /* 1147 + mspace_mmap_large_chunks controls whether requests for large chunks 1148 + are allocated in their own mmapped regions, separate from others in 1149 + this mspace. By default this is enabled, which reduces 1150 + fragmentation. However, such chunks are not necessarily released to 1151 + the system upon destroy_mspace. Disabling by setting to false may 1152 + increase fragmentation, but avoids leakage when relying on 1153 + destroy_mspace to release all memory allocated using this space. 1154 + */ 1155 + int mspace_mmap_large_chunks(mspace msp, int enable); 1156 + 1157 + 1158 + /* 1159 + mspace_malloc behaves as malloc, but operates within 1160 + the given space. 1161 + */ 1162 + void* mspace_malloc(mspace msp, size_t bytes); 1163 + 1164 + /* 1165 + mspace_free behaves as free, but operates within 1166 + the given space. 1167 + 1168 + If compiled with FOOTERS==1, mspace_free is not actually needed. 1169 + free may be called instead of mspace_free because freed chunks from 1170 + any space are handled by their originating spaces. 1171 + */ 1172 + void mspace_free(mspace msp, void* mem); 1173 + 1174 + /* 1175 + mspace_realloc behaves as realloc, but operates within 1176 + the given space. 1177 + 1178 + If compiled with FOOTERS==1, mspace_realloc is not actually 1179 + needed. realloc may be called instead of mspace_realloc because 1180 + realloced chunks from any space are handled by their originating 1181 + spaces. 1182 + */ 1183 + void* mspace_realloc(mspace msp, void* mem, size_t newsize); 1184 + 1185 + /* 1186 + mspace_calloc behaves as calloc, but operates within 1187 + the given space. 1188 + */ 1189 + void* mspace_calloc(mspace msp, size_t n_elements, size_t elem_size); 1190 + 1191 + /* 1192 + mspace_memalign behaves as memalign, but operates within 1193 + the given space. 1194 + */ 1195 + void* mspace_memalign(mspace msp, size_t alignment, size_t bytes); 1196 + 1197 + /* 1198 + mspace_independent_calloc behaves as independent_calloc, but 1199 + operates within the given space. 1200 + */ 1201 + void** mspace_independent_calloc(mspace msp, size_t n_elements, 1202 + size_t elem_size, void* chunks[]); 1203 + 1204 + /* 1205 + mspace_independent_comalloc behaves as independent_comalloc, but 1206 + operates within the given space. 1207 + */ 1208 + void** mspace_independent_comalloc(mspace msp, size_t n_elements, 1209 + size_t sizes[], void* chunks[]); 1210 + 1211 + /* 1212 + mspace_footprint() returns the number of bytes obtained from the 1213 + system for this space. 1214 + */ 1215 + size_t mspace_footprint(mspace msp); 1216 + 1217 + /* 1218 + mspace_max_footprint() returns the peak number of bytes obtained from the 1219 + system for this space. 1220 + */ 1221 + size_t mspace_max_footprint(mspace msp); 1222 + 1223 + 1224 + #if !NO_MALLINFO 1225 + /* 1226 + mspace_mallinfo behaves as mallinfo, but reports properties of 1227 + the given space. 1228 + */ 1229 + struct mallinfo mspace_mallinfo(mspace msp); 1230 + #endif /* NO_MALLINFO */ 1231 + 1232 + /* 1233 + malloc_usable_size(void* p) behaves the same as malloc_usable_size; 1234 + */ 1235 + size_t mspace_usable_size(void* mem); 1236 + 1237 + /* 1238 + mspace_malloc_stats behaves as malloc_stats, but reports 1239 + properties of the given space. 1240 + */ 1241 + void mspace_malloc_stats(mspace msp); 1242 + 1243 + /* 1244 + mspace_trim behaves as malloc_trim, but 1245 + operates within the given space. 1246 + */ 1247 + int mspace_trim(mspace msp, size_t pad); 1248 + 1249 + /* 1250 + An alias for mallopt. 1251 + */ 1252 + int mspace_mallopt(int, int); 1253 + 1254 + #endif /* MSPACES */ 1255 + 1256 + #ifdef __cplusplus 1257 + }; /* end of extern "C" */ 1258 + #endif /* __cplusplus */ 1259 + 1260 + /* 1261 + ======================================================================== 1262 + To make a fully customizable malloc.h header file, cut everything 1263 + above this line, put into file malloc.h, edit to suit, and #include it 1264 + on the next line, as well as in programs that use this malloc. 1265 + ======================================================================== 1266 + */ 1267 + 1268 + /* #include "malloc.h" */ 1269 + 1270 + /*------------------------------ internal #includes ---------------------- */ 1271 + 1272 + #ifdef WIN32 1273 + #pragma warning( disable : 4146 ) /* no "unsigned" warnings */ 1274 + #endif /* WIN32 */ 1275 + 1276 + #include <stdio.h> /* for printing in malloc_stats */ 1277 + 1278 + #ifndef LACKS_ERRNO_H 1279 + #include <errno.h> /* for MALLOC_FAILURE_ACTION */ 1280 + #endif /* LACKS_ERRNO_H */ 1281 + #if FOOTERS 1282 + #include <time.h> /* for magic initialization */ 1283 + #endif /* FOOTERS */ 1284 + #ifndef LACKS_STDLIB_H 1285 + #include <stdlib.h> /* for abort() */ 1286 + #endif /* LACKS_STDLIB_H */ 1287 + #ifdef DEBUG 1288 + #if ABORT_ON_ASSERT_FAILURE 1289 + #define assert(x) if(!(x)) ABORT 1290 + #else /* ABORT_ON_ASSERT_FAILURE */ 1291 + #include <assert.h> 1292 + #endif /* ABORT_ON_ASSERT_FAILURE */ 1293 + #else /* DEBUG */ 1294 + #ifndef assert 1295 + #define assert(x) 1296 + #endif 1297 + #define DEBUG 0 1298 + #endif /* DEBUG */ 1299 + #ifndef LACKS_STRING_H 1300 + #include <string.h> /* for memset etc */ 1301 + #endif /* LACKS_STRING_H */ 1302 + #if USE_BUILTIN_FFS 1303 + #ifndef LACKS_STRINGS_H 1304 + #include <strings.h> /* for ffs */ 1305 + #endif /* LACKS_STRINGS_H */ 1306 + #endif /* USE_BUILTIN_FFS */ 1307 + #if HAVE_MMAP 1308 + #ifndef LACKS_SYS_MMAN_H 1309 + #include <sys/mman.h> /* for mmap */ 1310 + #endif /* LACKS_SYS_MMAN_H */ 1311 + #ifndef LACKS_FCNTL_H 1312 + #include <fcntl.h> 1313 + #endif /* LACKS_FCNTL_H */ 1314 + #endif /* HAVE_MMAP */ 1315 + #ifndef LACKS_UNISTD_H 1316 + #include <unistd.h> /* for sbrk, sysconf */ 1317 + #else /* LACKS_UNISTD_H */ 1318 + #if !defined(__FreeBSD__) && !defined(__OpenBSD__) && !defined(__NetBSD__) 1319 + extern void* sbrk(ptrdiff_t); 1320 + #endif /* FreeBSD etc */ 1321 + #endif /* LACKS_UNISTD_H */ 1322 + 1323 + /* Declarations for locking */ 1324 + #if USE_LOCKS 1325 + #ifndef WIN32 1326 + #include <pthread.h> 1327 + #if defined (__SVR4) && defined (__sun) /* solaris */ 1328 + #include <thread.h> 1329 + #endif /* solaris */ 1330 + #else 1331 + #ifndef _M_AMD64 1332 + /* These are already defined on AMD64 builds */ 1333 + #ifdef __cplusplus 1334 + extern "C" { 1335 + #endif /* __cplusplus */ 1336 + #ifndef __MINGW32__ 1337 + LONG __cdecl _InterlockedCompareExchange(LONG volatile *Dest, LONG Exchange, LONG Comp); 1338 + LONG __cdecl _InterlockedExchange(LONG volatile *Target, LONG Value); 1339 + #endif 1340 + #ifdef __cplusplus 1341 + } 1342 + #endif /* __cplusplus */ 1343 + #endif /* _M_AMD64 */ 1344 + #ifndef __MINGW32__ 1345 + #pragma intrinsic (_InterlockedCompareExchange) 1346 + #pragma intrinsic (_InterlockedExchange) 1347 + #else 1348 + /* --[ start GCC compatibility ]---------------------------------------------- 1349 + * Compatibility <intrin_x86.h> header for GCC -- GCC equivalents of intrinsic 1350 + * Microsoft Visual C++ functions. Originally developed for the ReactOS 1351 + * (<http://www.reactos.org/>) and TinyKrnl (<http://www.tinykrnl.org/>) 1352 + * projects. 1353 + * 1354 + * Copyright (c) 2006 KJK::Hyperion <hackbunny@reactos.com> 1355 + * 1356 + * Permission is hereby granted, free of charge, to any person obtaining a 1357 + * copy of this software and associated documentation files (the "Software"), 1358 + * to deal in the Software without restriction, including without limitation 1359 + * the rights to use, copy, modify, merge, publish, distribute, sublicense, 1360 + * and/or sell copies of the Software, and to permit persons to whom the 1361 + * Software is furnished to do so, subject to the following conditions: 1362 + * 1363 + * The above copyright notice and this permission notice shall be included in 1364 + * all copies or substantial portions of the Software. 1365 + * 1366 + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 1367 + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 1368 + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 1369 + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 1370 + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 1371 + * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 1372 + * DEALINGS IN THE SOFTWARE. 1373 + */ 1374 + 1375 + /*** Atomic operations ***/ 1376 + #if (__GNUC__ * 10000 + __GNUC_MINOR__ * 100 + __GNUC_PATCHLEVEL__) > 40100 1377 + #define _ReadWriteBarrier() __sync_synchronize() 1378 + #else 1379 + static __inline__ __attribute__((always_inline)) long __sync_lock_test_and_set(volatile long * const Target, const long Value) 1380 + { 1381 + long res; 1382 + __asm__ __volatile__("xchg%z0 %2, %0" : "=g" (*(Target)), "=r" (res) : "1" (Value)); 1383 + return res; 1384 + } 1385 + static void __inline__ __attribute__((always_inline)) _MemoryBarrier(void) 1386 + { 1387 + __asm__ __volatile__("" : : : "memory"); 1388 + } 1389 + #define _ReadWriteBarrier() _MemoryBarrier() 1390 + #endif 1391 + /* BUGBUG: GCC only supports full barriers */ 1392 + static __inline__ __attribute__((always_inline)) long _InterlockedExchange(volatile long * const Target, const long Value) 1393 + { 1394 + /* NOTE: __sync_lock_test_and_set would be an acquire barrier, so we force a full barrier */ 1395 + _ReadWriteBarrier(); 1396 + return __sync_lock_test_and_set(Target, Value); 1397 + } 1398 + /* --[ end GCC compatibility ]---------------------------------------------- */ 1399 + #endif 1400 + #define interlockedcompareexchange _InterlockedCompareExchange 1401 + #define interlockedexchange _InterlockedExchange 1402 + #endif /* Win32 */ 1403 + #endif /* USE_LOCKS */ 1404 + 1405 + /* Declarations for bit scanning on win32 */ 1406 + #if defined(_MSC_VER) && _MSC_VER>=1300 1407 + #ifndef BitScanForward /* Try to avoid pulling in WinNT.h */ 1408 + #ifdef __cplusplus 1409 + extern "C" { 1410 + #endif /* __cplusplus */ 1411 + unsigned char _BitScanForward(unsigned long *index, unsigned long mask); 1412 + unsigned char _BitScanReverse(unsigned long *index, unsigned long mask); 1413 + #ifdef __cplusplus 1414 + } 1415 + #endif /* __cplusplus */ 1416 + 1417 + #define BitScanForward _BitScanForward 1418 + #define BitScanReverse _BitScanReverse 1419 + #pragma intrinsic(_BitScanForward) 1420 + #pragma intrinsic(_BitScanReverse) 1421 + #endif /* BitScanForward */ 1422 + #endif /* defined(_MSC_VER) && _MSC_VER>=1300 */ 1423 + 1424 + #ifndef WIN32 1425 + #ifndef malloc_getpagesize 1426 + # ifdef _SC_PAGESIZE /* some SVR4 systems omit an underscore */ 1427 + # ifndef _SC_PAGE_SIZE 1428 + # define _SC_PAGE_SIZE _SC_PAGESIZE 1429 + # endif 1430 + # endif 1431 + # ifdef _SC_PAGE_SIZE 1432 + # define malloc_getpagesize sysconf(_SC_PAGE_SIZE) 1433 + # else 1434 + # if defined(BSD) || defined(DGUX) || defined(HAVE_GETPAGESIZE) 1435 + extern size_t getpagesize(); 1436 + # define malloc_getpagesize getpagesize() 1437 + # else 1438 + # ifdef WIN32 /* use supplied emulation of getpagesize */ 1439 + # define malloc_getpagesize getpagesize() 1440 + # else 1441 + # ifndef LACKS_SYS_PARAM_H 1442 + # include <sys/param.h> 1443 + # endif 1444 + # ifdef EXEC_PAGESIZE 1445 + # define malloc_getpagesize EXEC_PAGESIZE 1446 + # else 1447 + # ifdef NBPG 1448 + # ifndef CLSIZE 1449 + # define malloc_getpagesize NBPG 1450 + # else 1451 + # define malloc_getpagesize (NBPG * CLSIZE) 1452 + # endif 1453 + # else 1454 + # ifdef NBPC 1455 + # define malloc_getpagesize NBPC 1456 + # else 1457 + # ifdef PAGESIZE 1458 + # define malloc_getpagesize PAGESIZE 1459 + # else /* just guess */ 1460 + # define malloc_getpagesize ((size_t)4096U) 1461 + # endif 1462 + # endif 1463 + # endif 1464 + # endif 1465 + # endif 1466 + # endif 1467 + # endif 1468 + #endif 1469 + #endif 1470 + 1471 + 1472 + 1473 + /* ------------------- size_t and alignment properties -------------------- */ 1474 + 1475 + /* The byte and bit size of a size_t */ 1476 + #define SIZE_T_SIZE (sizeof(size_t)) 1477 + #define SIZE_T_BITSIZE (sizeof(size_t) << 3) 1478 + 1479 + /* Some constants coerced to size_t */ 1480 + /* Annoying but necessary to avoid errors on some platforms */ 1481 + #define SIZE_T_ZERO ((size_t)0) 1482 + #define SIZE_T_ONE ((size_t)1) 1483 + #define SIZE_T_TWO ((size_t)2) 1484 + #define SIZE_T_FOUR ((size_t)4) 1485 + #define TWO_SIZE_T_SIZES (SIZE_T_SIZE<<1) 1486 + #define FOUR_SIZE_T_SIZES (SIZE_T_SIZE<<2) 1487 + #define SIX_SIZE_T_SIZES (FOUR_SIZE_T_SIZES+TWO_SIZE_T_SIZES) 1488 + #define HALF_MAX_SIZE_T (MAX_SIZE_T / 2U) 1489 + 1490 + /* The bit mask value corresponding to MALLOC_ALIGNMENT */ 1491 + #define CHUNK_ALIGN_MASK (MALLOC_ALIGNMENT - SIZE_T_ONE) 1492 + 1493 + /* True if address a has acceptable alignment */ 1494 + #define is_aligned(A) (((size_t)((A)) & (CHUNK_ALIGN_MASK)) == 0) 1495 + 1496 + /* the number of bytes to offset an address to align it */ 1497 + #define align_offset(A)\ 1498 + ((((size_t)(A) & CHUNK_ALIGN_MASK) == 0)? 0 :\ 1499 + ((MALLOC_ALIGNMENT - ((size_t)(A) & CHUNK_ALIGN_MASK)) & CHUNK_ALIGN_MASK)) 1500 + 1501 + /* -------------------------- MMAP preliminaries ------------------------- */ 1502 + 1503 + /* 1504 + If HAVE_MORECORE or HAVE_MMAP are false, we just define calls and 1505 + checks to fail so compiler optimizer can delete code rather than 1506 + using so many "#if"s. 1507 + */ 1508 + 1509 + 1510 + /* MORECORE and MMAP must return MFAIL on failure */ 1511 + #define MFAIL ((void*)(MAX_SIZE_T)) 1512 + #define CMFAIL ((char*)(MFAIL)) /* defined for convenience */ 1513 + 1514 + #if HAVE_MMAP 1515 + 1516 + #ifndef WIN32 1517 + #define MUNMAP_DEFAULT(a, s) munmap((a), (s)) 1518 + #define MMAP_PROT (PROT_READ|PROT_WRITE) 1519 + #if !defined(MAP_ANONYMOUS) && defined(MAP_ANON) 1520 + #define MAP_ANONYMOUS MAP_ANON 1521 + #endif /* MAP_ANON */ 1522 + #ifdef MAP_ANONYMOUS 1523 + #define MMAP_FLAGS (MAP_PRIVATE|MAP_ANONYMOUS) 1524 + #define MMAP_DEFAULT(s) mmap(0, (s), MMAP_PROT, MMAP_FLAGS, -1, 0) 1525 + #else /* MAP_ANONYMOUS */ 1526 + /* 1527 + Nearly all versions of mmap support MAP_ANONYMOUS, so the following 1528 + is unlikely to be needed, but is supplied just in case. 1529 + */ 1530 + #define MMAP_FLAGS (MAP_PRIVATE) 1531 + static int dev_zero_fd = -1; /* Cached file descriptor for /dev/zero. */ 1532 + #define MMAP_DEFAULT(s) ((dev_zero_fd < 0) ? \ 1533 + (dev_zero_fd = open("/dev/zero", O_RDWR), \ 1534 + mmap(0, (s), MMAP_PROT, MMAP_FLAGS, dev_zero_fd, 0)) : \ 1535 + mmap(0, (s), MMAP_PROT, MMAP_FLAGS, dev_zero_fd, 0)) 1536 + #endif /* MAP_ANONYMOUS */ 1537 + 1538 + #define DIRECT_MMAP_DEFAULT(s) MMAP_DEFAULT(s) 1539 + 1540 + #else /* WIN32 */ 1541 + 1542 + /* Win32 MMAP via VirtualAlloc */ 1543 + static FORCEINLINE void* win32mmap(size_t size) { 1544 + void* ptr = VirtualAlloc(0, size, MEM_RESERVE|MEM_COMMIT, PAGE_READWRITE); 1545 + return (ptr != 0)? ptr: MFAIL; 1546 + } 1547 + 1548 + /* For direct MMAP, use MEM_TOP_DOWN to minimize interference */ 1549 + static FORCEINLINE void* win32direct_mmap(size_t size) { 1550 + void* ptr = VirtualAlloc(0, size, MEM_RESERVE|MEM_COMMIT|MEM_TOP_DOWN, 1551 + PAGE_READWRITE); 1552 + return (ptr != 0)? ptr: MFAIL; 1553 + } 1554 + 1555 + /* This function supports releasing coalesed segments */ 1556 + static FORCEINLINE int win32munmap(void* ptr, size_t size) { 1557 + MEMORY_BASIC_INFORMATION minfo; 1558 + char* cptr = (char*)ptr; 1559 + while (size) { 1560 + if (VirtualQuery(cptr, &minfo, sizeof(minfo)) == 0) 1561 + return -1; 1562 + if (minfo.BaseAddress != cptr || minfo.AllocationBase != cptr || 1563 + minfo.State != MEM_COMMIT || minfo.RegionSize > size) 1564 + return -1; 1565 + if (VirtualFree(cptr, 0, MEM_RELEASE) == 0) 1566 + return -1; 1567 + cptr += minfo.RegionSize; 1568 + size -= minfo.RegionSize; 1569 + } 1570 + return 0; 1571 + } 1572 + 1573 + #define MMAP_DEFAULT(s) win32mmap(s) 1574 + #define MUNMAP_DEFAULT(a, s) win32munmap((a), (s)) 1575 + #define DIRECT_MMAP_DEFAULT(s) win32direct_mmap(s) 1576 + #endif /* WIN32 */ 1577 + #endif /* HAVE_MMAP */ 1578 + 1579 + #if HAVE_MREMAP 1580 + #ifndef WIN32 1581 + #define MREMAP_DEFAULT(addr, osz, nsz, mv) mremap((addr), (osz), (nsz), (mv)) 1582 + #endif /* WIN32 */ 1583 + #endif /* HAVE_MREMAP */ 1584 + 1585 + 1586 + /** 1587 + * Define CALL_MORECORE 1588 + */ 1589 + #if HAVE_MORECORE 1590 + #ifdef MORECORE 1591 + #define CALL_MORECORE(S) MORECORE(S) 1592 + #else /* MORECORE */ 1593 + #define CALL_MORECORE(S) MORECORE_DEFAULT(S) 1594 + #endif /* MORECORE */ 1595 + #else /* HAVE_MORECORE */ 1596 + #define CALL_MORECORE(S) MFAIL 1597 + #endif /* HAVE_MORECORE */ 1598 + 1599 + /** 1600 + * Define CALL_MMAP/CALL_MUNMAP/CALL_DIRECT_MMAP 1601 + */ 1602 + #if HAVE_MMAP 1603 + #define IS_MMAPPED_BIT (SIZE_T_ONE) 1604 + #define USE_MMAP_BIT (SIZE_T_ONE) 1605 + 1606 + #ifdef MMAP 1607 + #define CALL_MMAP(s) MMAP(s) 1608 + #else /* MMAP */ 1609 + #define CALL_MMAP(s) MMAP_DEFAULT(s) 1610 + #endif /* MMAP */ 1611 + #ifdef MUNMAP 1612 + #define CALL_MUNMAP(a, s) MUNMAP((a), (s)) 1613 + #else /* MUNMAP */ 1614 + #define CALL_MUNMAP(a, s) MUNMAP_DEFAULT((a), (s)) 1615 + #endif /* MUNMAP */ 1616 + #ifdef DIRECT_MMAP 1617 + #define CALL_DIRECT_MMAP(s) DIRECT_MMAP(s) 1618 + #else /* DIRECT_MMAP */ 1619 + #define CALL_DIRECT_MMAP(s) DIRECT_MMAP_DEFAULT(s) 1620 + #endif /* DIRECT_MMAP */ 1621 + #else /* HAVE_MMAP */ 1622 + #define IS_MMAPPED_BIT (SIZE_T_ZERO) 1623 + #define USE_MMAP_BIT (SIZE_T_ZERO) 1624 + 1625 + #define MMAP(s) MFAIL 1626 + #define MUNMAP(a, s) (-1) 1627 + #define DIRECT_MMAP(s) MFAIL 1628 + #define CALL_DIRECT_MMAP(s) DIRECT_MMAP(s) 1629 + #define CALL_MMAP(s) MMAP(s) 1630 + #define CALL_MUNMAP(a, s) MUNMAP((a), (s)) 1631 + #endif /* HAVE_MMAP */ 1632 + 1633 + /** 1634 + * Define CALL_MREMAP 1635 + */ 1636 + #if HAVE_MMAP && HAVE_MREMAP 1637 + #ifdef MREMAP 1638 + #define CALL_MREMAP(addr, osz, nsz, mv) MREMAP((addr), (osz), (nsz), (mv)) 1639 + #else /* MREMAP */ 1640 + #define CALL_MREMAP(addr, osz, nsz, mv) MREMAP_DEFAULT((addr), (osz), (nsz), (mv)) 1641 + #endif /* MREMAP */ 1642 + #else /* HAVE_MMAP && HAVE_MREMAP */ 1643 + #define CALL_MREMAP(addr, osz, nsz, mv) MFAIL 1644 + #endif /* HAVE_MMAP && HAVE_MREMAP */ 1645 + 1646 + /* mstate bit set if continguous morecore disabled or failed */ 1647 + #define USE_NONCONTIGUOUS_BIT (4U) 1648 + 1649 + /* segment bit set in create_mspace_with_base */ 1650 + #define EXTERN_BIT (8U) 1651 + 1652 + 1653 + /* --------------------------- Lock preliminaries ------------------------ */ 1654 + 1655 + /* 1656 + When locks are defined, there is one global lock, plus 1657 + one per-mspace lock. 1658 + 1659 + The global lock_ensures that mparams.magic and other unique 1660 + mparams values are initialized only once. It also protects 1661 + sequences of calls to MORECORE. In many cases sys_alloc requires 1662 + two calls, that should not be interleaved with calls by other 1663 + threads. This does not protect against direct calls to MORECORE 1664 + by other threads not using this lock, so there is still code to 1665 + cope the best we can on interference. 1666 + 1667 + Per-mspace locks surround calls to malloc, free, etc. To enable use 1668 + in layered extensions, per-mspace locks are reentrant. 1669 + 1670 + Because lock-protected regions generally have bounded times, it is 1671 + OK to use the supplied simple spinlocks in the custom versions for 1672 + x86. 1673 + 1674 + If USE_LOCKS is > 1, the definitions of lock routines here are 1675 + bypassed, in which case you will need to define at least 1676 + INITIAL_LOCK, ACQUIRE_LOCK, RELEASE_LOCK and possibly TRY_LOCK 1677 + (which is not used in this malloc, but commonly needed in 1678 + extensions.) 1679 + */ 1680 + 1681 + #if USE_LOCKS == 1 1682 + 1683 + #if USE_SPIN_LOCKS 1684 + #ifndef WIN32 1685 + 1686 + /* Custom pthread-style spin locks on x86 and x64 for gcc */ 1687 + struct pthread_mlock_t { 1688 + volatile unsigned int l; 1689 + volatile unsigned int c; 1690 + volatile pthread_t threadid; 1691 + }; 1692 + #define MLOCK_T struct pthread_mlock_t 1693 + #define CURRENT_THREAD pthread_self() 1694 + #define INITIAL_LOCK(sl) (memset(sl, 0, sizeof(MLOCK_T)), 0) 1695 + #define ACQUIRE_LOCK(sl) pthread_acquire_lock(sl) 1696 + #define RELEASE_LOCK(sl) pthread_release_lock(sl) 1697 + #define TRY_LOCK(sl) pthread_try_lock(sl) 1698 + #define SPINS_PER_YIELD 63 1699 + 1700 + static MLOCK_T malloc_global_mutex = { 0, 0, 0}; 1701 + 1702 + static FORCEINLINE int pthread_acquire_lock (MLOCK_T *sl) { 1703 + int spins = 0; 1704 + volatile unsigned int* lp = &sl->l; 1705 + for (;;) { 1706 + if (*lp != 0) { 1707 + if (sl->threadid == CURRENT_THREAD) { 1708 + ++sl->c; 1709 + return 0; 1710 + } 1711 + } 1712 + else { 1713 + /* place args to cmpxchgl in locals to evade oddities in some gccs */ 1714 + int cmp = 0; 1715 + int val = 1; 1716 + int ret; 1717 + __asm__ __volatile__ ("lock; cmpxchgl %1, %2" 1718 + : "=a" (ret) 1719 + : "r" (val), "m" (*(lp)), "0"(cmp) 1720 + : "memory", "cc"); 1721 + if (!ret) { 1722 + assert(!sl->threadid); 1723 + sl->c = 1; 1724 + sl->threadid = CURRENT_THREAD; 1725 + return 0; 1726 + } 1727 + if ((++spins & SPINS_PER_YIELD) == 0) { 1728 + #if defined (__SVR4) && defined (__sun) /* solaris */ 1729 + thr_yield(); 1730 + #else 1731 + #if defined(__linux__) || defined(__FreeBSD__) || defined(__APPLE__) 1732 + sched_yield(); 1733 + #else /* no-op yield on unknown systems */ 1734 + ; 1735 + #endif /* __linux__ || __FreeBSD__ || __APPLE__ */ 1736 + #endif /* solaris */ 1737 + } 1738 + } 1739 + } 1740 + } 1741 + 1742 + static FORCEINLINE void pthread_release_lock (MLOCK_T *sl) { 1743 + assert(sl->l != 0); 1744 + assert(sl->threadid == CURRENT_THREAD); 1745 + if (--sl->c == 0) { 1746 + sl->threadid = 0; 1747 + volatile unsigned int* lp = &sl->l; 1748 + int prev = 0; 1749 + int ret; 1750 + __asm__ __volatile__ ("lock; xchgl %0, %1" 1751 + : "=r" (ret) 1752 + : "m" (*(lp)), "0"(prev) 1753 + : "memory"); 1754 + } 1755 + } 1756 + 1757 + static FORCEINLINE int pthread_try_lock (MLOCK_T *sl) { 1758 + volatile unsigned int* lp = &sl->l; 1759 + if (*lp != 0) { 1760 + if (sl->threadid == CURRENT_THREAD) { 1761 + ++sl->c; 1762 + return 1; 1763 + } 1764 + } 1765 + else { 1766 + int cmp = 0; 1767 + int val = 1; 1768 + int ret; 1769 + __asm__ __volatile__ ("lock; cmpxchgl %1, %2" 1770 + : "=a" (ret) 1771 + : "r" (val), "m" (*(lp)), "0"(cmp) 1772 + : "memory", "cc"); 1773 + if (!ret) { 1774 + assert(!sl->threadid); 1775 + sl->c = 1; 1776 + sl->threadid = CURRENT_THREAD; 1777 + return 1; 1778 + } 1779 + } 1780 + return 0; 1781 + } 1782 + 1783 + 1784 + #else /* WIN32 */ 1785 + /* Custom win32-style spin locks on x86 and x64 for MSC */ 1786 + struct win32_mlock_t 1787 + { 1788 + volatile long l; 1789 + volatile unsigned int c; 1790 + volatile long threadid; 1791 + }; 1792 + 1793 + #define MLOCK_T struct win32_mlock_t 1794 + #define CURRENT_THREAD win32_getcurrentthreadid() 1795 + #define INITIAL_LOCK(sl) (memset(sl, 0, sizeof(MLOCK_T)), 0) 1796 + #define ACQUIRE_LOCK(sl) win32_acquire_lock(sl) 1797 + #define RELEASE_LOCK(sl) win32_release_lock(sl) 1798 + #define TRY_LOCK(sl) win32_try_lock(sl) 1799 + #define SPINS_PER_YIELD 63 1800 + 1801 + static MLOCK_T malloc_global_mutex = { 0, 0, 0}; 1802 + 1803 + static FORCEINLINE long win32_getcurrentthreadid() { 1804 + #ifdef _MSC_VER 1805 + #if defined(_M_IX86) 1806 + long *threadstruct=(long *)__readfsdword(0x18); 1807 + long threadid=threadstruct[0x24/sizeof(long)]; 1808 + return threadid; 1809 + #elif defined(_M_X64) 1810 + /* todo */ 1811 + return GetCurrentThreadId(); 1812 + #else 1813 + return GetCurrentThreadId(); 1814 + #endif 1815 + #else 1816 + return GetCurrentThreadId(); 1817 + #endif 1818 + } 1819 + 1820 + static FORCEINLINE int win32_acquire_lock (MLOCK_T *sl) { 1821 + int spins = 0; 1822 + for (;;) { 1823 + if (sl->l != 0) { 1824 + if (sl->threadid == CURRENT_THREAD) { 1825 + ++sl->c; 1826 + return 0; 1827 + } 1828 + } 1829 + else { 1830 + if (!interlockedexchange(&sl->l, 1)) { 1831 + assert(!sl->threadid); 1832 + sl->c=CURRENT_THREAD; 1833 + sl->threadid = CURRENT_THREAD; 1834 + sl->c = 1; 1835 + return 0; 1836 + } 1837 + } 1838 + if ((++spins & SPINS_PER_YIELD) == 0) 1839 + SleepEx(0, FALSE); 1840 + } 1841 + } 1842 + 1843 + static FORCEINLINE void win32_release_lock (MLOCK_T *sl) { 1844 + assert(sl->threadid == CURRENT_THREAD); 1845 + assert(sl->l != 0); 1846 + if (--sl->c == 0) { 1847 + sl->threadid = 0; 1848 + interlockedexchange (&sl->l, 0); 1849 + } 1850 + } 1851 + 1852 + static FORCEINLINE int win32_try_lock (MLOCK_T *sl) { 1853 + if(sl->l != 0) { 1854 + if (sl->threadid == CURRENT_THREAD) { 1855 + ++sl->c; 1856 + return 1; 1857 + } 1858 + } 1859 + else { 1860 + if (!interlockedexchange(&sl->l, 1)){ 1861 + assert(!sl->threadid); 1862 + sl->threadid = CURRENT_THREAD; 1863 + sl->c = 1; 1864 + return 1; 1865 + } 1866 + } 1867 + return 0; 1868 + } 1869 + 1870 + #endif /* WIN32 */ 1871 + #else /* USE_SPIN_LOCKS */ 1872 + 1873 + #ifndef WIN32 1874 + /* pthreads-based locks */ 1875 + 1876 + #define MLOCK_T pthread_mutex_t 1877 + #define CURRENT_THREAD pthread_self() 1878 + #define INITIAL_LOCK(sl) pthread_init_lock(sl) 1879 + #define ACQUIRE_LOCK(sl) pthread_mutex_lock(sl) 1880 + #define RELEASE_LOCK(sl) pthread_mutex_unlock(sl) 1881 + #define TRY_LOCK(sl) (!pthread_mutex_trylock(sl)) 1882 + 1883 + static MLOCK_T malloc_global_mutex = PTHREAD_MUTEX_INITIALIZER; 1884 + 1885 + /* Cope with old-style linux recursive lock initialization by adding */ 1886 + /* skipped internal declaration from pthread.h */ 1887 + #ifdef linux 1888 + #ifndef PTHREAD_MUTEX_RECURSIVE 1889 + extern int pthread_mutexattr_setkind_np __P ((pthread_mutexattr_t *__attr, 1890 + int __kind)); 1891 + #define PTHREAD_MUTEX_RECURSIVE PTHREAD_MUTEX_RECURSIVE_NP 1892 + #define pthread_mutexattr_settype(x,y) pthread_mutexattr_setkind_np(x,y) 1893 + #endif 1894 + #endif 1895 + 1896 + static int pthread_init_lock (MLOCK_T *sl) { 1897 + pthread_mutexattr_t attr; 1898 + if (pthread_mutexattr_init(&attr)) return 1; 1899 + if (pthread_mutexattr_settype(&attr, PTHREAD_MUTEX_RECURSIVE)) return 1; 1900 + if (pthread_mutex_init(sl, &attr)) return 1; 1901 + if (pthread_mutexattr_destroy(&attr)) return 1; 1902 + return 0; 1903 + } 1904 + 1905 + #else /* WIN32 */ 1906 + /* Win32 critical sections */ 1907 + #define MLOCK_T CRITICAL_SECTION 1908 + #define CURRENT_THREAD GetCurrentThreadId() 1909 + #define INITIAL_LOCK(s) (!InitializeCriticalSectionAndSpinCount((s), 0x80000000|4000)) 1910 + #define ACQUIRE_LOCK(s) (EnterCriticalSection(s), 0) 1911 + #define RELEASE_LOCK(s) LeaveCriticalSection(s) 1912 + #define TRY_LOCK(s) TryEnterCriticalSection(s) 1913 + #define NEED_GLOBAL_LOCK_INIT 1914 + 1915 + static MLOCK_T malloc_global_mutex; 1916 + static volatile long malloc_global_mutex_status; 1917 + 1918 + /* Use spin loop to initialize global lock */ 1919 + static void init_malloc_global_mutex() { 1920 + for (;;) { 1921 + long stat = malloc_global_mutex_status; 1922 + if (stat > 0) 1923 + return; 1924 + /* transition to < 0 while initializing, then to > 0) */ 1925 + if (stat == 0 && 1926 + interlockedcompareexchange(&malloc_global_mutex_status, -1, 0) == 0) { 1927 + InitializeCriticalSection(&malloc_global_mutex); 1928 + interlockedexchange(&malloc_global_mutex_status,1); 1929 + return; 1930 + } 1931 + SleepEx(0, FALSE); 1932 + } 1933 + } 1934 + 1935 + #endif /* WIN32 */ 1936 + #endif /* USE_SPIN_LOCKS */ 1937 + #endif /* USE_LOCKS == 1 */ 1938 + 1939 + /* ----------------------- User-defined locks ------------------------ */ 1940 + 1941 + #if USE_LOCKS > 1 1942 + /* Define your own lock implementation here */ 1943 + /* #define INITIAL_LOCK(sl) ... */ 1944 + /* #define ACQUIRE_LOCK(sl) ... */ 1945 + /* #define RELEASE_LOCK(sl) ... */ 1946 + /* #define TRY_LOCK(sl) ... */ 1947 + /* static MLOCK_T malloc_global_mutex = ... */ 1948 + #endif /* USE_LOCKS > 1 */ 1949 + 1950 + /* ----------------------- Lock-based state ------------------------ */ 1951 + 1952 + #if USE_LOCKS 1953 + #define USE_LOCK_BIT (2U) 1954 + #else /* USE_LOCKS */ 1955 + #define USE_LOCK_BIT (0U) 1956 + #define INITIAL_LOCK(l) 1957 + #endif /* USE_LOCKS */ 1958 + 1959 + #if USE_LOCKS 1960 + #define ACQUIRE_MALLOC_GLOBAL_LOCK() ACQUIRE_LOCK(&malloc_global_mutex); 1961 + #define RELEASE_MALLOC_GLOBAL_LOCK() RELEASE_LOCK(&malloc_global_mutex); 1962 + #else /* USE_LOCKS */ 1963 + #define ACQUIRE_MALLOC_GLOBAL_LOCK() 1964 + #define RELEASE_MALLOC_GLOBAL_LOCK() 1965 + #endif /* USE_LOCKS */ 1966 + 1967 + 1968 + /* ----------------------- Chunk representations ------------------------ */ 1969 + 1970 + /* 1971 + (The following includes lightly edited explanations by Colin Plumb.) 1972 + 1973 + The malloc_chunk declaration below is misleading (but accurate and 1974 + necessary). It declares a "view" into memory allowing access to 1975 + necessary fields at known offsets from a given base. 1976 + 1977 + Chunks of memory are maintained using a `boundary tag' method as 1978 + originally described by Knuth. (See the paper by Paul Wilson 1979 + ftp://ftp.cs.utexas.edu/pub/garbage/allocsrv.ps for a survey of such 1980 + techniques.) Sizes of free chunks are stored both in the front of 1981 + each chunk and at the end. This makes consolidating fragmented 1982 + chunks into bigger chunks fast. The head fields also hold bits 1983 + representing whether chunks are free or in use. 1984 + 1985 + Here are some pictures to make it clearer. They are "exploded" to 1986 + show that the state of a chunk can be thought of as extending from 1987 + the high 31 bits of the head field of its header through the 1988 + prev_foot and PINUSE_BIT bit of the following chunk header. 1989 + 1990 + A chunk that's in use looks like: 1991 + 1992 + chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1993 + | Size of previous chunk (if P = 0) | 1994 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1995 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P| 1996 + | Size of this chunk 1| +-+ 1997 + mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 1998 + | | 1999 + +- -+ 2000 + | | 2001 + +- -+ 2002 + | : 2003 + +- size - sizeof(size_t) available payload bytes -+ 2004 + : | 2005 + chunk-> +- -+ 2006 + | | 2007 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2008 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |1| 2009 + | Size of next chunk (may or may not be in use) | +-+ 2010 + mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2011 + 2012 + And if it's free, it looks like this: 2013 + 2014 + chunk-> +- -+ 2015 + | User payload (must be in use, or we would have merged!) | 2016 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2017 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |P| 2018 + | Size of this chunk 0| +-+ 2019 + mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2020 + | Next pointer | 2021 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2022 + | Prev pointer | 2023 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2024 + | : 2025 + +- size - sizeof(struct chunk) unused bytes -+ 2026 + : | 2027 + chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2028 + | Size of this chunk | 2029 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2030 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |0| 2031 + | Size of next chunk (must be in use, or we would have merged)| +-+ 2032 + mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2033 + | : 2034 + +- User payload -+ 2035 + : | 2036 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2037 + |0| 2038 + +-+ 2039 + Note that since we always merge adjacent free chunks, the chunks 2040 + adjacent to a free chunk must be in use. 2041 + 2042 + Given a pointer to a chunk (which can be derived trivially from the 2043 + payload pointer) we can, in O(1) time, find out whether the adjacent 2044 + chunks are free, and if so, unlink them from the lists that they 2045 + are on and merge them with the current chunk. 2046 + 2047 + Chunks always begin on even word boundaries, so the mem portion 2048 + (which is returned to the user) is also on an even word boundary, and 2049 + thus at least double-word aligned. 2050 + 2051 + The P (PINUSE_BIT) bit, stored in the unused low-order bit of the 2052 + chunk size (which is always a multiple of two words), is an in-use 2053 + bit for the *previous* chunk. If that bit is *clear*, then the 2054 + word before the current chunk size contains the previous chunk 2055 + size, and can be used to find the front of the previous chunk. 2056 + The very first chunk allocated always has this bit set, preventing 2057 + access to non-existent (or non-owned) memory. If pinuse is set for 2058 + any given chunk, then you CANNOT determine the size of the 2059 + previous chunk, and might even get a memory addressing fault when 2060 + trying to do so. 2061 + 2062 + The C (CINUSE_BIT) bit, stored in the unused second-lowest bit of 2063 + the chunk size redundantly records whether the current chunk is 2064 + inuse. This redundancy enables usage checks within free and realloc, 2065 + and reduces indirection when freeing and consolidating chunks. 2066 + 2067 + Each freshly allocated chunk must have both cinuse and pinuse set. 2068 + That is, each allocated chunk borders either a previously allocated 2069 + and still in-use chunk, or the base of its memory arena. This is 2070 + ensured by making all allocations from the the `lowest' part of any 2071 + found chunk. Further, no free chunk physically borders another one, 2072 + so each free chunk is known to be preceded and followed by either 2073 + inuse chunks or the ends of memory. 2074 + 2075 + Note that the `foot' of the current chunk is actually represented 2076 + as the prev_foot of the NEXT chunk. This makes it easier to 2077 + deal with alignments etc but can be very confusing when trying 2078 + to extend or adapt this code. 2079 + 2080 + The exceptions to all this are 2081 + 2082 + 1. The special chunk `top' is the top-most available chunk (i.e., 2083 + the one bordering the end of available memory). It is treated 2084 + specially. Top is never included in any bin, is used only if 2085 + no other chunk is available, and is released back to the 2086 + system if it is very large (see M_TRIM_THRESHOLD). In effect, 2087 + the top chunk is treated as larger (and thus less well 2088 + fitting) than any other available chunk. The top chunk 2089 + doesn't update its trailing size field since there is no next 2090 + contiguous chunk that would have to index off it. However, 2091 + space is still allocated for it (TOP_FOOT_SIZE) to enable 2092 + separation or merging when space is extended. 2093 + 2094 + 3. Chunks allocated via mmap, which have the lowest-order bit 2095 + (IS_MMAPPED_BIT) set in their prev_foot fields, and do not set 2096 + PINUSE_BIT in their head fields. Because they are allocated 2097 + one-by-one, each must carry its own prev_foot field, which is 2098 + also used to hold the offset this chunk has within its mmapped 2099 + region, which is needed to preserve alignment. Each mmapped 2100 + chunk is trailed by the first two fields of a fake next-chunk 2101 + for sake of usage checks. 2102 + 2103 + */ 2104 + 2105 + struct malloc_chunk { 2106 + size_t prev_foot; /* Size of previous chunk (if free). */ 2107 + size_t head; /* Size and inuse bits. */ 2108 + struct malloc_chunk* fd; /* double links -- used only if free. */ 2109 + struct malloc_chunk* bk; 2110 + }; 2111 + 2112 + typedef struct malloc_chunk mchunk; 2113 + typedef struct malloc_chunk* mchunkptr; 2114 + typedef struct malloc_chunk* sbinptr; /* The type of bins of chunks */ 2115 + typedef unsigned int bindex_t; /* Described below */ 2116 + typedef unsigned int binmap_t; /* Described below */ 2117 + typedef unsigned int flag_t; /* The type of various bit flag sets */ 2118 + 2119 + /* ------------------- Chunks sizes and alignments ----------------------- */ 2120 + 2121 + #define MCHUNK_SIZE (sizeof(mchunk)) 2122 + 2123 + #if FOOTERS 2124 + #define CHUNK_OVERHEAD (TWO_SIZE_T_SIZES) 2125 + #else /* FOOTERS */ 2126 + #define CHUNK_OVERHEAD (SIZE_T_SIZE) 2127 + #endif /* FOOTERS */ 2128 + 2129 + /* MMapped chunks need a second word of overhead ... */ 2130 + #define MMAP_CHUNK_OVERHEAD (TWO_SIZE_T_SIZES) 2131 + /* ... and additional padding for fake next-chunk at foot */ 2132 + #define MMAP_FOOT_PAD (FOUR_SIZE_T_SIZES) 2133 + 2134 + /* The smallest size we can malloc is an aligned minimal chunk */ 2135 + #define MIN_CHUNK_SIZE\ 2136 + ((MCHUNK_SIZE + CHUNK_ALIGN_MASK) & ~CHUNK_ALIGN_MASK) 2137 + 2138 + /* conversion from malloc headers to user pointers, and back */ 2139 + #define chunk2mem(p) ((void*)((char*)(p) + TWO_SIZE_T_SIZES)) 2140 + #define mem2chunk(mem) ((mchunkptr)((char*)(mem) - TWO_SIZE_T_SIZES)) 2141 + /* chunk associated with aligned address A */ 2142 + #define align_as_chunk(A) (mchunkptr)((A) + align_offset(chunk2mem(A))) 2143 + 2144 + /* Bounds on request (not chunk) sizes. */ 2145 + #define MAX_REQUEST ((-MIN_CHUNK_SIZE) << 2) 2146 + #define MIN_REQUEST (MIN_CHUNK_SIZE - CHUNK_OVERHEAD - SIZE_T_ONE) 2147 + 2148 + /* pad request bytes into a usable size */ 2149 + #define pad_request(req) \ 2150 + (((req) + CHUNK_OVERHEAD + CHUNK_ALIGN_MASK) & ~CHUNK_ALIGN_MASK) 2151 + 2152 + /* pad request, checking for minimum (but not maximum) */ 2153 + #define request2size(req) \ 2154 + (((req) < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(req)) 2155 + 2156 + 2157 + /* ------------------ Operations on head and foot fields ----------------- */ 2158 + 2159 + /* 2160 + The head field of a chunk is or'ed with PINUSE_BIT when previous 2161 + adjacent chunk in use, and or'ed with CINUSE_BIT if this chunk is in 2162 + use. If the chunk was obtained with mmap, the prev_foot field has 2163 + IS_MMAPPED_BIT set, otherwise holding the offset of the base of the 2164 + mmapped region to the base of the chunk. 2165 + 2166 + FLAG4_BIT is not used by this malloc, but might be useful in extensions. 2167 + */ 2168 + 2169 + #define PINUSE_BIT (SIZE_T_ONE) 2170 + #define CINUSE_BIT (SIZE_T_TWO) 2171 + #define FLAG4_BIT (SIZE_T_FOUR) 2172 + #define INUSE_BITS (PINUSE_BIT|CINUSE_BIT) 2173 + #define FLAG_BITS (PINUSE_BIT|CINUSE_BIT|FLAG4_BIT) 2174 + 2175 + /* Head value for fenceposts */ 2176 + #define FENCEPOST_HEAD (INUSE_BITS|SIZE_T_SIZE) 2177 + 2178 + /* extraction of fields from head words */ 2179 + #define cinuse(p) ((p)->head & CINUSE_BIT) 2180 + #define pinuse(p) ((p)->head & PINUSE_BIT) 2181 + #define chunksize(p) ((p)->head & ~(FLAG_BITS)) 2182 + 2183 + #define clear_pinuse(p) ((p)->head &= ~PINUSE_BIT) 2184 + #define clear_cinuse(p) ((p)->head &= ~CINUSE_BIT) 2185 + 2186 + /* Treat space at ptr +/- offset as a chunk */ 2187 + #define chunk_plus_offset(p, s) ((mchunkptr)(((char*)(p)) + (s))) 2188 + #define chunk_minus_offset(p, s) ((mchunkptr)(((char*)(p)) - (s))) 2189 + 2190 + /* Ptr to next or previous physical malloc_chunk. */ 2191 + #define next_chunk(p) ((mchunkptr)( ((char*)(p)) + ((p)->head & ~FLAG_BITS))) 2192 + #define prev_chunk(p) ((mchunkptr)( ((char*)(p)) - ((p)->prev_foot) )) 2193 + 2194 + /* extract next chunk's pinuse bit */ 2195 + #define next_pinuse(p) ((next_chunk(p)->head) & PINUSE_BIT) 2196 + 2197 + /* Get/set size at footer */ 2198 + #define get_foot(p, s) (((mchunkptr)((char*)(p) + (s)))->prev_foot) 2199 + #define set_foot(p, s) (((mchunkptr)((char*)(p) + (s)))->prev_foot = (s)) 2200 + 2201 + /* Set size, pinuse bit, and foot */ 2202 + #define set_size_and_pinuse_of_free_chunk(p, s)\ 2203 + ((p)->head = (s|PINUSE_BIT), set_foot(p, s)) 2204 + 2205 + /* Set size, pinuse bit, foot, and clear next pinuse */ 2206 + #define set_free_with_pinuse(p, s, n)\ 2207 + (clear_pinuse(n), set_size_and_pinuse_of_free_chunk(p, s)) 2208 + 2209 + #define is_mmapped(p)\ 2210 + (!((p)->head & PINUSE_BIT) && ((p)->prev_foot & IS_MMAPPED_BIT)) 2211 + 2212 + /* Get the internal overhead associated with chunk p */ 2213 + #define overhead_for(p)\ 2214 + (is_mmapped(p)? MMAP_CHUNK_OVERHEAD : CHUNK_OVERHEAD) 2215 + 2216 + /* Return true if malloced space is not necessarily cleared */ 2217 + #if MMAP_CLEARS 2218 + #define calloc_must_clear(p) (!is_mmapped(p)) 2219 + #else /* MMAP_CLEARS */ 2220 + #define calloc_must_clear(p) (1) 2221 + #endif /* MMAP_CLEARS */ 2222 + 2223 + /* ---------------------- Overlaid data structures ----------------------- */ 2224 + 2225 + /* 2226 + When chunks are not in use, they are treated as nodes of either 2227 + lists or trees. 2228 + 2229 + "Small" chunks are stored in circular doubly-linked lists, and look 2230 + like this: 2231 + 2232 + chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2233 + | Size of previous chunk | 2234 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2235 + `head:' | Size of chunk, in bytes |P| 2236 + mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2237 + | Forward pointer to next chunk in list | 2238 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2239 + | Back pointer to previous chunk in list | 2240 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2241 + | Unused space (may be 0 bytes long) . 2242 + . . 2243 + . | 2244 + nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2245 + `foot:' | Size of chunk, in bytes | 2246 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2247 + 2248 + Larger chunks are kept in a form of bitwise digital trees (aka 2249 + tries) keyed on chunksizes. Because malloc_tree_chunks are only for 2250 + free chunks greater than 256 bytes, their size doesn't impose any 2251 + constraints on user chunk sizes. Each node looks like: 2252 + 2253 + chunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2254 + | Size of previous chunk | 2255 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2256 + `head:' | Size of chunk, in bytes |P| 2257 + mem-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2258 + | Forward pointer to next chunk of same size | 2259 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2260 + | Back pointer to previous chunk of same size | 2261 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2262 + | Pointer to left child (child[0]) | 2263 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2264 + | Pointer to right child (child[1]) | 2265 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2266 + | Pointer to parent | 2267 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2268 + | bin index of this chunk | 2269 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2270 + | Unused space . 2271 + . | 2272 + nextchunk-> +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2273 + `foot:' | Size of chunk, in bytes | 2274 + +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 2275 + 2276 + Each tree holding treenodes is a tree of unique chunk sizes. Chunks 2277 + of the same size are arranged in a circularly-linked list, with only 2278 + the oldest chunk (the next to be used, in our FIFO ordering) 2279 + actually in the tree. (Tree members are distinguished by a non-null 2280 + parent pointer.) If a chunk with the same size an an existing node 2281 + is inserted, it is linked off the existing node using pointers that 2282 + work in the same way as fd/bk pointers of small chunks. 2283 + 2284 + Each tree contains a power of 2 sized range of chunk sizes (the 2285 + smallest is 0x100 <= x < 0x180), which is is divided in half at each 2286 + tree level, with the chunks in the smaller half of the range (0x100 2287 + <= x < 0x140 for the top nose) in the left subtree and the larger 2288 + half (0x140 <= x < 0x180) in the right subtree. This is, of course, 2289 + done by inspecting individual bits. 2290 + 2291 + Using these rules, each node's left subtree contains all smaller 2292 + sizes than its right subtree. However, the node at the root of each 2293 + subtree has no particular ordering relationship to either. (The 2294 + dividing line between the subtree sizes is based on trie relation.) 2295 + If we remove the last chunk of a given size from the interior of the 2296 + tree, we need to replace it with a leaf node. The tree ordering 2297 + rules permit a node to be replaced by any leaf below it. 2298 + 2299 + The smallest chunk in a tree (a common operation in a best-fit 2300 + allocator) can be found by walking a path to the leftmost leaf in 2301 + the tree. Unlike a usual binary tree, where we follow left child 2302 + pointers until we reach a null, here we follow the right child 2303 + pointer any time the left one is null, until we reach a leaf with 2304 + both child pointers null. The smallest chunk in the tree will be 2305 + somewhere along that path. 2306 + 2307 + The worst case number of steps to add, find, or remove a node is 2308 + bounded by the number of bits differentiating chunks within 2309 + bins. Under current bin calculations, this ranges from 6 up to 21 2310 + (for 32 bit sizes) or up to 53 (for 64 bit sizes). The typical case 2311 + is of course much better. 2312 + */ 2313 + 2314 + struct malloc_tree_chunk { 2315 + /* The first four fields must be compatible with malloc_chunk */ 2316 + size_t prev_foot; 2317 + size_t head; 2318 + struct malloc_tree_chunk* fd; 2319 + struct malloc_tree_chunk* bk; 2320 + 2321 + struct malloc_tree_chunk* child[2]; 2322 + struct malloc_tree_chunk* parent; 2323 + bindex_t index; 2324 + }; 2325 + 2326 + typedef struct malloc_tree_chunk tchunk; 2327 + typedef struct malloc_tree_chunk* tchunkptr; 2328 + typedef struct malloc_tree_chunk* tbinptr; /* The type of bins of trees */ 2329 + 2330 + /* A little helper macro for trees */ 2331 + #define leftmost_child(t) ((t)->child[0] != 0? (t)->child[0] : (t)->child[1]) 2332 + 2333 + /* ----------------------------- Segments -------------------------------- */ 2334 + 2335 + /* 2336 + Each malloc space may include non-contiguous segments, held in a 2337 + list headed by an embedded malloc_segment record representing the 2338 + top-most space. Segments also include flags holding properties of 2339 + the space. Large chunks that are directly allocated by mmap are not 2340 + included in this list. They are instead independently created and 2341 + destroyed without otherwise keeping track of them. 2342 + 2343 + Segment management mainly comes into play for spaces allocated by 2344 + MMAP. Any call to MMAP might or might not return memory that is 2345 + adjacent to an existing segment. MORECORE normally contiguously 2346 + extends the current space, so this space is almost always adjacent, 2347 + which is simpler and faster to deal with. (This is why MORECORE is 2348 + used preferentially to MMAP when both are available -- see 2349 + sys_alloc.) When allocating using MMAP, we don't use any of the 2350 + hinting mechanisms (inconsistently) supported in various 2351 + implementations of unix mmap, or distinguish reserving from 2352 + committing memory. Instead, we just ask for space, and exploit 2353 + contiguity when we get it. It is probably possible to do 2354 + better than this on some systems, but no general scheme seems 2355 + to be significantly better. 2356 + 2357 + Management entails a simpler variant of the consolidation scheme 2358 + used for chunks to reduce fragmentation -- new adjacent memory is 2359 + normally prepended or appended to an existing segment. However, 2360 + there are limitations compared to chunk consolidation that mostly 2361 + reflect the fact that segment processing is relatively infrequent 2362 + (occurring only when getting memory from system) and that we 2363 + don't expect to have huge numbers of segments: 2364 + 2365 + * Segments are not indexed, so traversal requires linear scans. (It 2366 + would be possible to index these, but is not worth the extra 2367 + overhead and complexity for most programs on most platforms.) 2368 + * New segments are only appended to old ones when holding top-most 2369 + memory; if they cannot be prepended to others, they are held in 2370 + different segments. 2371 + 2372 + Except for the top-most segment of an mstate, each segment record 2373 + is kept at the tail of its segment. Segments are added by pushing 2374 + segment records onto the list headed by &mstate.seg for the 2375 + containing mstate. 2376 + 2377 + Segment flags control allocation/merge/deallocation policies: 2378 + * If EXTERN_BIT set, then we did not allocate this segment, 2379 + and so should not try to deallocate or merge with others. 2380 + (This currently holds only for the initial segment passed 2381 + into create_mspace_with_base.) 2382 + * If IS_MMAPPED_BIT set, the segment may be merged with 2383 + other surrounding mmapped segments and trimmed/de-allocated 2384 + using munmap. 2385 + * If neither bit is set, then the segment was obtained using 2386 + MORECORE so can be merged with surrounding MORECORE'd segments 2387 + and deallocated/trimmed using MORECORE with negative arguments. 2388 + */ 2389 + 2390 + struct malloc_segment { 2391 + char* base; /* base address */ 2392 + size_t size; /* allocated size */ 2393 + struct malloc_segment* next; /* ptr to next segment */ 2394 + flag_t sflags; /* mmap and extern flag */ 2395 + }; 2396 + 2397 + #define is_mmapped_segment(S) ((S)->sflags & IS_MMAPPED_BIT) 2398 + #define is_extern_segment(S) ((S)->sflags & EXTERN_BIT) 2399 + 2400 + typedef struct malloc_segment msegment; 2401 + typedef struct malloc_segment* msegmentptr; 2402 + 2403 + /* ---------------------------- malloc_state ----------------------------- */ 2404 + 2405 + /* 2406 + A malloc_state holds all of the bookkeeping for a space. 2407 + The main fields are: 2408 + 2409 + Top 2410 + The topmost chunk of the currently active segment. Its size is 2411 + cached in topsize. The actual size of topmost space is 2412 + topsize+TOP_FOOT_SIZE, which includes space reserved for adding 2413 + fenceposts and segment records if necessary when getting more 2414 + space from the system. The size at which to autotrim top is 2415 + cached from mparams in trim_check, except that it is disabled if 2416 + an autotrim fails. 2417 + 2418 + Designated victim (dv) 2419 + This is the preferred chunk for servicing small requests that 2420 + don't have exact fits. It is normally the chunk split off most 2421 + recently to service another small request. Its size is cached in 2422 + dvsize. The link fields of this chunk are not maintained since it 2423 + is not kept in a bin. 2424 + 2425 + SmallBins 2426 + An array of bin headers for free chunks. These bins hold chunks 2427 + with sizes less than MIN_LARGE_SIZE bytes. Each bin contains 2428 + chunks of all the same size, spaced 8 bytes apart. To simplify 2429 + use in double-linked lists, each bin header acts as a malloc_chunk 2430 + pointing to the real first node, if it exists (else pointing to 2431 + itself). This avoids special-casing for headers. But to avoid 2432 + waste, we allocate only the fd/bk pointers of bins, and then use 2433 + repositioning tricks to treat these as the fields of a chunk. 2434 + 2435 + TreeBins 2436 + Treebins are pointers to the roots of trees holding a range of 2437 + sizes. There are 2 equally spaced treebins for each power of two 2438 + from TREE_SHIFT to TREE_SHIFT+16. The last bin holds anything 2439 + larger. 2440 + 2441 + Bin maps 2442 + There is one bit map for small bins ("smallmap") and one for 2443 + treebins ("treemap). Each bin sets its bit when non-empty, and 2444 + clears the bit when empty. Bit operations are then used to avoid 2445 + bin-by-bin searching -- nearly all "search" is done without ever 2446 + looking at bins that won't be selected. The bit maps 2447 + conservatively use 32 bits per map word, even if on 64bit system. 2448 + For a good description of some of the bit-based techniques used 2449 + here, see Henry S. Warren Jr's book "Hacker's Delight" (and 2450 + supplement at http://hackersdelight.org/). Many of these are 2451 + intended to reduce the branchiness of paths through malloc etc, as 2452 + well as to reduce the number of memory locations read or written. 2453 + 2454 + Segments 2455 + A list of segments headed by an embedded malloc_segment record 2456 + representing the initial space. 2457 + 2458 + Address check support 2459 + The least_addr field is the least address ever obtained from 2460 + MORECORE or MMAP. Attempted frees and reallocs of any address less 2461 + than this are trapped (unless INSECURE is defined). 2462 + 2463 + Magic tag 2464 + A cross-check field that should always hold same value as mparams.magic. 2465 + 2466 + Flags 2467 + Bits recording whether to use MMAP, locks, or contiguous MORECORE 2468 + 2469 + Statistics 2470 + Each space keeps track of current and maximum system memory 2471 + obtained via MORECORE or MMAP. 2472 + 2473 + Trim support 2474 + Fields holding the amount of unused topmost memory that should trigger 2475 + timming, and a counter to force periodic scanning to release unused 2476 + non-topmost segments. 2477 + 2478 + Locking 2479 + If USE_LOCKS is defined, the "mutex" lock is acquired and released 2480 + around every public call using this mspace. 2481 + 2482 + Extension support 2483 + A void* pointer and a size_t field that can be used to help implement 2484 + extensions to this malloc. 2485 + */ 2486 + 2487 + /* Bin types, widths and sizes */ 2488 + #define NSMALLBINS (32U) 2489 + #define NTREEBINS (32U) 2490 + #define SMALLBIN_SHIFT (3U) 2491 + #define SMALLBIN_WIDTH (SIZE_T_ONE << SMALLBIN_SHIFT) 2492 + #define TREEBIN_SHIFT (8U) 2493 + #define MIN_LARGE_SIZE (SIZE_T_ONE << TREEBIN_SHIFT) 2494 + #define MAX_SMALL_SIZE (MIN_LARGE_SIZE - SIZE_T_ONE) 2495 + #define MAX_SMALL_REQUEST (MAX_SMALL_SIZE - CHUNK_ALIGN_MASK - CHUNK_OVERHEAD) 2496 + 2497 + struct malloc_state { 2498 + binmap_t smallmap; 2499 + binmap_t treemap; 2500 + size_t dvsize; 2501 + size_t topsize; 2502 + char* least_addr; 2503 + mchunkptr dv; 2504 + mchunkptr top; 2505 + size_t trim_check; 2506 + size_t release_checks; 2507 + size_t magic; 2508 + mchunkptr smallbins[(NSMALLBINS+1)*2]; 2509 + tbinptr treebins[NTREEBINS]; 2510 + size_t footprint; 2511 + size_t max_footprint; 2512 + flag_t mflags; 2513 + #if USE_LOCKS 2514 + MLOCK_T mutex; /* locate lock among fields that rarely change */ 2515 + #endif /* USE_LOCKS */ 2516 + msegment seg; 2517 + void* extp; /* Unused but available for extensions */ 2518 + size_t exts; 2519 + }; 2520 + 2521 + typedef struct malloc_state* mstate; 2522 + 2523 + /* ------------- Global malloc_state and malloc_params ------------------- */ 2524 + 2525 + /* 2526 + malloc_params holds global properties, including those that can be 2527 + dynamically set using mallopt. There is a single instance, mparams, 2528 + initialized in init_mparams. Note that the non-zeroness of "magic" 2529 + also serves as an initialization flag. 2530 + */ 2531 + 2532 + struct malloc_params { 2533 + volatile size_t magic; 2534 + size_t page_size; 2535 + size_t granularity; 2536 + size_t mmap_threshold; 2537 + size_t trim_threshold; 2538 + flag_t default_mflags; 2539 + }; 2540 + 2541 + static struct malloc_params mparams; 2542 + 2543 + /* Ensure mparams initialized */ 2544 + #define ensure_initialization() (mparams.magic != 0 || init_mparams()) 2545 + 2546 + #if !ONLY_MSPACES 2547 + 2548 + /* The global malloc_state used for all non-"mspace" calls */ 2549 + static struct malloc_state _gm_; 2550 + #define gm (&_gm_) 2551 + #define is_global(M) ((M) == &_gm_) 2552 + 2553 + #endif /* !ONLY_MSPACES */ 2554 + 2555 + #define is_initialized(M) ((M)->top != 0) 2556 + 2557 + /* -------------------------- system alloc setup ------------------------- */ 2558 + 2559 + /* Operations on mflags */ 2560 + 2561 + #define use_lock(M) ((M)->mflags & USE_LOCK_BIT) 2562 + #define enable_lock(M) ((M)->mflags |= USE_LOCK_BIT) 2563 + #define disable_lock(M) ((M)->mflags &= ~USE_LOCK_BIT) 2564 + 2565 + #define use_mmap(M) ((M)->mflags & USE_MMAP_BIT) 2566 + #define enable_mmap(M) ((M)->mflags |= USE_MMAP_BIT) 2567 + #define disable_mmap(M) ((M)->mflags &= ~USE_MMAP_BIT) 2568 + 2569 + #define use_noncontiguous(M) ((M)->mflags & USE_NONCONTIGUOUS_BIT) 2570 + #define disable_contiguous(M) ((M)->mflags |= USE_NONCONTIGUOUS_BIT) 2571 + 2572 + #define set_lock(M,L)\ 2573 + ((M)->mflags = (L)?\ 2574 + ((M)->mflags | USE_LOCK_BIT) :\ 2575 + ((M)->mflags & ~USE_LOCK_BIT)) 2576 + 2577 + /* page-align a size */ 2578 + #define page_align(S)\ 2579 + (((S) + (mparams.page_size - SIZE_T_ONE)) & ~(mparams.page_size - SIZE_T_ONE)) 2580 + 2581 + /* granularity-align a size */ 2582 + #define granularity_align(S)\ 2583 + (((S) + (mparams.granularity - SIZE_T_ONE))\ 2584 + & ~(mparams.granularity - SIZE_T_ONE)) 2585 + 2586 + 2587 + /* For mmap, use granularity alignment on windows, else page-align */ 2588 + #ifdef WIN32 2589 + #define mmap_align(S) granularity_align(S) 2590 + #else 2591 + #define mmap_align(S) page_align(S) 2592 + #endif 2593 + 2594 + /* For sys_alloc, enough padding to ensure can malloc request on success */ 2595 + #define SYS_ALLOC_PADDING (TOP_FOOT_SIZE + MALLOC_ALIGNMENT) 2596 + 2597 + #define is_page_aligned(S)\ 2598 + (((size_t)(S) & (mparams.page_size - SIZE_T_ONE)) == 0) 2599 + #define is_granularity_aligned(S)\ 2600 + (((size_t)(S) & (mparams.granularity - SIZE_T_ONE)) == 0) 2601 + 2602 + /* True if segment S holds address A */ 2603 + #define segment_holds(S, A)\ 2604 + ((char*)(A) >= S->base && (char*)(A) < S->base + S->size) 2605 + 2606 + /* Return segment holding given address */ 2607 + static msegmentptr segment_holding(mstate m, char* addr) { 2608 + msegmentptr sp = &m->seg; 2609 + for (;;) { 2610 + if (addr >= sp->base && addr < sp->base + sp->size) 2611 + return sp; 2612 + if ((sp = sp->next) == 0) 2613 + return 0; 2614 + } 2615 + } 2616 + 2617 + /* Return true if segment contains a segment link */ 2618 + static int has_segment_link(mstate m, msegmentptr ss) { 2619 + msegmentptr sp = &m->seg; 2620 + for (;;) { 2621 + if ((char*)sp >= ss->base && (char*)sp < ss->base + ss->size) 2622 + return 1; 2623 + if ((sp = sp->next) == 0) 2624 + return 0; 2625 + } 2626 + } 2627 + 2628 + #ifndef MORECORE_CANNOT_TRIM 2629 + #define should_trim(M,s) ((s) > (M)->trim_check) 2630 + #else /* MORECORE_CANNOT_TRIM */ 2631 + #define should_trim(M,s) (0) 2632 + #endif /* MORECORE_CANNOT_TRIM */ 2633 + 2634 + /* 2635 + TOP_FOOT_SIZE is padding at the end of a segment, including space 2636 + that may be needed to place segment records and fenceposts when new 2637 + noncontiguous segments are added. 2638 + */ 2639 + #define TOP_FOOT_SIZE\ 2640 + (align_offset(chunk2mem(0))+pad_request(sizeof(struct malloc_segment))+MIN_CHUNK_SIZE) 2641 + 2642 + 2643 + /* ------------------------------- Hooks -------------------------------- */ 2644 + 2645 + /* 2646 + PREACTION should be defined to return 0 on success, and nonzero on 2647 + failure. If you are not using locking, you can redefine these to do 2648 + anything you like. 2649 + */ 2650 + 2651 + #if USE_LOCKS 2652 + 2653 + #define PREACTION(M) ((use_lock(M))? ACQUIRE_LOCK(&(M)->mutex) : 0) 2654 + #define POSTACTION(M) { if (use_lock(M)) RELEASE_LOCK(&(M)->mutex); } 2655 + #else /* USE_LOCKS */ 2656 + 2657 + #ifndef PREACTION 2658 + #define PREACTION(M) (0) 2659 + #endif /* PREACTION */ 2660 + 2661 + #ifndef POSTACTION 2662 + #define POSTACTION(M) 2663 + #endif /* POSTACTION */ 2664 + 2665 + #endif /* USE_LOCKS */ 2666 + 2667 + /* 2668 + CORRUPTION_ERROR_ACTION is triggered upon detected bad addresses. 2669 + USAGE_ERROR_ACTION is triggered on detected bad frees and 2670 + reallocs. The argument p is an address that might have triggered the 2671 + fault. It is ignored by the two predefined actions, but might be 2672 + useful in custom actions that try to help diagnose errors. 2673 + */ 2674 + 2675 + #if PROCEED_ON_ERROR 2676 + 2677 + /* A count of the number of corruption errors causing resets */ 2678 + int malloc_corruption_error_count; 2679 + 2680 + /* default corruption action */ 2681 + static void reset_on_error(mstate m); 2682 + 2683 + #define CORRUPTION_ERROR_ACTION(m) reset_on_error(m) 2684 + #define USAGE_ERROR_ACTION(m, p) 2685 + 2686 + #else /* PROCEED_ON_ERROR */ 2687 + 2688 + #ifndef CORRUPTION_ERROR_ACTION 2689 + #define CORRUPTION_ERROR_ACTION(m) ABORT 2690 + #endif /* CORRUPTION_ERROR_ACTION */ 2691 + 2692 + #ifndef USAGE_ERROR_ACTION 2693 + #define USAGE_ERROR_ACTION(m,p) ABORT 2694 + #endif /* USAGE_ERROR_ACTION */ 2695 + 2696 + #endif /* PROCEED_ON_ERROR */ 2697 + 2698 + /* -------------------------- Debugging setup ---------------------------- */ 2699 + 2700 + #if ! DEBUG 2701 + 2702 + #define check_free_chunk(M,P) 2703 + #define check_inuse_chunk(M,P) 2704 + #define check_malloced_chunk(M,P,N) 2705 + #define check_mmapped_chunk(M,P) 2706 + #define check_malloc_state(M) 2707 + #define check_top_chunk(M,P) 2708 + 2709 + #else /* DEBUG */ 2710 + #define check_free_chunk(M,P) do_check_free_chunk(M,P) 2711 + #define check_inuse_chunk(M,P) do_check_inuse_chunk(M,P) 2712 + #define check_top_chunk(M,P) do_check_top_chunk(M,P) 2713 + #define check_malloced_chunk(M,P,N) do_check_malloced_chunk(M,P,N) 2714 + #define check_mmapped_chunk(M,P) do_check_mmapped_chunk(M,P) 2715 + #define check_malloc_state(M) do_check_malloc_state(M) 2716 + 2717 + static void do_check_any_chunk(mstate m, mchunkptr p); 2718 + static void do_check_top_chunk(mstate m, mchunkptr p); 2719 + static void do_check_mmapped_chunk(mstate m, mchunkptr p); 2720 + static void do_check_inuse_chunk(mstate m, mchunkptr p); 2721 + static void do_check_free_chunk(mstate m, mchunkptr p); 2722 + static void do_check_malloced_chunk(mstate m, void* mem, size_t s); 2723 + static void do_check_tree(mstate m, tchunkptr t); 2724 + static void do_check_treebin(mstate m, bindex_t i); 2725 + static void do_check_smallbin(mstate m, bindex_t i); 2726 + static void do_check_malloc_state(mstate m); 2727 + static int bin_find(mstate m, mchunkptr x); 2728 + static size_t traverse_and_check(mstate m); 2729 + #endif /* DEBUG */ 2730 + 2731 + /* ---------------------------- Indexing Bins ---------------------------- */ 2732 + 2733 + #define is_small(s) (((s) >> SMALLBIN_SHIFT) < NSMALLBINS) 2734 + #define small_index(s) ((s) >> SMALLBIN_SHIFT) 2735 + #define small_index2size(i) ((i) << SMALLBIN_SHIFT) 2736 + #define MIN_SMALL_INDEX (small_index(MIN_CHUNK_SIZE)) 2737 + 2738 + /* addressing by index. See above about smallbin repositioning */ 2739 + #define smallbin_at(M, i) ((sbinptr)((char*)&((M)->smallbins[(i)<<1]))) 2740 + #define treebin_at(M,i) (&((M)->treebins[i])) 2741 + 2742 + /* assign tree index for size S to variable I. Use x86 asm if possible */ 2743 + #if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__)) 2744 + #define compute_tree_index(S, I)\ 2745 + {\ 2746 + unsigned int X = S >> TREEBIN_SHIFT;\ 2747 + if (X == 0)\ 2748 + I = 0;\ 2749 + else if (X > 0xFFFF)\ 2750 + I = NTREEBINS-1;\ 2751 + else {\ 2752 + unsigned int K;\ 2753 + __asm__("bsrl\t%1, %0\n\t" : "=r" (K) : "rm" (X));\ 2754 + I = (bindex_t)((K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1)));\ 2755 + }\ 2756 + } 2757 + 2758 + #elif defined (__INTEL_COMPILER) 2759 + #define compute_tree_index(S, I)\ 2760 + {\ 2761 + size_t X = S >> TREEBIN_SHIFT;\ 2762 + if (X == 0)\ 2763 + I = 0;\ 2764 + else if (X > 0xFFFF)\ 2765 + I = NTREEBINS-1;\ 2766 + else {\ 2767 + unsigned int K = _bit_scan_reverse (X); \ 2768 + I = (bindex_t)((K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1)));\ 2769 + }\ 2770 + } 2771 + 2772 + #elif defined(_MSC_VER) && _MSC_VER>=1300 2773 + #define compute_tree_index(S, I)\ 2774 + {\ 2775 + size_t X = S >> TREEBIN_SHIFT;\ 2776 + if (X == 0)\ 2777 + I = 0;\ 2778 + else if (X > 0xFFFF)\ 2779 + I = NTREEBINS-1;\ 2780 + else {\ 2781 + unsigned int K;\ 2782 + _BitScanReverse((DWORD *) &K, X);\ 2783 + I = (bindex_t)((K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1)));\ 2784 + }\ 2785 + } 2786 + 2787 + #else /* GNUC */ 2788 + #define compute_tree_index(S, I)\ 2789 + {\ 2790 + size_t X = S >> TREEBIN_SHIFT;\ 2791 + if (X == 0)\ 2792 + I = 0;\ 2793 + else if (X > 0xFFFF)\ 2794 + I = NTREEBINS-1;\ 2795 + else {\ 2796 + unsigned int Y = (unsigned int)X;\ 2797 + unsigned int N = ((Y - 0x100) >> 16) & 8;\ 2798 + unsigned int K = (((Y <<= N) - 0x1000) >> 16) & 4;\ 2799 + N += K;\ 2800 + N += K = (((Y <<= K) - 0x4000) >> 16) & 2;\ 2801 + K = 14 - N + ((Y <<= K) >> 15);\ 2802 + I = (K << 1) + ((S >> (K + (TREEBIN_SHIFT-1)) & 1));\ 2803 + }\ 2804 + } 2805 + #endif /* GNUC */ 2806 + 2807 + /* Bit representing maximum resolved size in a treebin at i */ 2808 + #define bit_for_tree_index(i) \ 2809 + (i == NTREEBINS-1)? (SIZE_T_BITSIZE-1) : (((i) >> 1) + TREEBIN_SHIFT - 2) 2810 + 2811 + /* Shift placing maximum resolved bit in a treebin at i as sign bit */ 2812 + #define leftshift_for_tree_index(i) \ 2813 + ((i == NTREEBINS-1)? 0 : \ 2814 + ((SIZE_T_BITSIZE-SIZE_T_ONE) - (((i) >> 1) + TREEBIN_SHIFT - 2))) 2815 + 2816 + /* The size of the smallest chunk held in bin with index i */ 2817 + #define minsize_for_tree_index(i) \ 2818 + ((SIZE_T_ONE << (((i) >> 1) + TREEBIN_SHIFT)) | \ 2819 + (((size_t)((i) & SIZE_T_ONE)) << (((i) >> 1) + TREEBIN_SHIFT - 1))) 2820 + 2821 + 2822 + /* ------------------------ Operations on bin maps ----------------------- */ 2823 + 2824 + /* bit corresponding to given index */ 2825 + #define idx2bit(i) ((binmap_t)(1) << (i)) 2826 + 2827 + /* Mark/Clear bits with given index */ 2828 + #define mark_smallmap(M,i) ((M)->smallmap |= idx2bit(i)) 2829 + #define clear_smallmap(M,i) ((M)->smallmap &= ~idx2bit(i)) 2830 + #define smallmap_is_marked(M,i) ((M)->smallmap & idx2bit(i)) 2831 + 2832 + #define mark_treemap(M,i) ((M)->treemap |= idx2bit(i)) 2833 + #define clear_treemap(M,i) ((M)->treemap &= ~idx2bit(i)) 2834 + #define treemap_is_marked(M,i) ((M)->treemap & idx2bit(i)) 2835 + 2836 + /* isolate the least set bit of a bitmap */ 2837 + #define least_bit(x) ((x) & -(x)) 2838 + 2839 + /* mask with all bits to left of least bit of x on */ 2840 + #define left_bits(x) ((x<<1) | -(x<<1)) 2841 + 2842 + /* mask with all bits to left of or equal to least bit of x on */ 2843 + #define same_or_left_bits(x) ((x) | -(x)) 2844 + 2845 + /* index corresponding to given bit. Use x86 asm if possible */ 2846 + 2847 + #if defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__)) 2848 + #define compute_bit2idx(X, I)\ 2849 + {\ 2850 + unsigned int J;\ 2851 + __asm__("bsfl\t%1, %0\n\t" : "=r" (J) : "rm" (X));\ 2852 + I = (bindex_t)J;\ 2853 + } 2854 + 2855 + #elif defined (__INTEL_COMPILER) 2856 + #define compute_bit2idx(X, I)\ 2857 + {\ 2858 + unsigned int J;\ 2859 + J = _bit_scan_forward (X); \ 2860 + I = (bindex_t)J;\ 2861 + } 2862 + 2863 + #elif defined(_MSC_VER) && _MSC_VER>=1300 2864 + #define compute_bit2idx(X, I)\ 2865 + {\ 2866 + unsigned int J;\ 2867 + _BitScanForward((DWORD *) &J, X);\ 2868 + I = (bindex_t)J;\ 2869 + } 2870 + 2871 + #elif USE_BUILTIN_FFS 2872 + #define compute_bit2idx(X, I) I = ffs(X)-1 2873 + 2874 + #else 2875 + #define compute_bit2idx(X, I)\ 2876 + {\ 2877 + unsigned int Y = X - 1;\ 2878 + unsigned int K = Y >> (16-4) & 16;\ 2879 + unsigned int N = K; Y >>= K;\ 2880 + N += K = Y >> (8-3) & 8; Y >>= K;\ 2881 + N += K = Y >> (4-2) & 4; Y >>= K;\ 2882 + N += K = Y >> (2-1) & 2; Y >>= K;\ 2883 + N += K = Y >> (1-0) & 1; Y >>= K;\ 2884 + I = (bindex_t)(N + Y);\ 2885 + } 2886 + #endif /* GNUC */ 2887 + 2888 + 2889 + /* ----------------------- Runtime Check Support ------------------------- */ 2890 + 2891 + /* 2892 + For security, the main invariant is that malloc/free/etc never 2893 + writes to a static address other than malloc_state, unless static 2894 + malloc_state itself has been corrupted, which cannot occur via 2895 + malloc (because of these checks). In essence this means that we 2896 + believe all pointers, sizes, maps etc held in malloc_state, but 2897 + check all of those linked or offsetted from other embedded data 2898 + structures. These checks are interspersed with main code in a way 2899 + that tends to minimize their run-time cost. 2900 + 2901 + When FOOTERS is defined, in addition to range checking, we also 2902 + verify footer fields of inuse chunks, which can be used guarantee 2903 + that the mstate controlling malloc/free is intact. This is a 2904 + streamlined version of the approach described by William Robertson 2905 + et al in "Run-time Detection of Heap-based Overflows" LISA'03 2906 + http://www.usenix.org/events/lisa03/tech/robertson.html The footer 2907 + of an inuse chunk holds the xor of its mstate and a random seed, 2908 + that is checked upon calls to free() and realloc(). This is 2909 + (probablistically) unguessable from outside the program, but can be 2910 + computed by any code successfully malloc'ing any chunk, so does not 2911 + itself provide protection against code that has already broken 2912 + security through some other means. Unlike Robertson et al, we 2913 + always dynamically check addresses of all offset chunks (previous, 2914 + next, etc). This turns out to be cheaper than relying on hashes. 2915 + */ 2916 + 2917 + #if !INSECURE 2918 + /* Check if address a is at least as high as any from MORECORE or MMAP */ 2919 + #define ok_address(M, a) ((char*)(a) >= (M)->least_addr) 2920 + /* Check if address of next chunk n is higher than base chunk p */ 2921 + #define ok_next(p, n) ((char*)(p) < (char*)(n)) 2922 + /* Check if p has its cinuse bit on */ 2923 + #define ok_cinuse(p) cinuse(p) 2924 + /* Check if p has its pinuse bit on */ 2925 + #define ok_pinuse(p) pinuse(p) 2926 + 2927 + #else /* !INSECURE */ 2928 + #define ok_address(M, a) (1) 2929 + #define ok_next(b, n) (1) 2930 + #define ok_cinuse(p) (1) 2931 + #define ok_pinuse(p) (1) 2932 + #endif /* !INSECURE */ 2933 + 2934 + #if (FOOTERS && !INSECURE) 2935 + /* Check if (alleged) mstate m has expected magic field */ 2936 + #define ok_magic(M) ((M)->magic == mparams.magic) 2937 + #else /* (FOOTERS && !INSECURE) */ 2938 + #define ok_magic(M) (1) 2939 + #endif /* (FOOTERS && !INSECURE) */ 2940 + 2941 + 2942 + /* In gcc, use __builtin_expect to minimize impact of checks */ 2943 + #if !INSECURE 2944 + #if defined(__GNUC__) && __GNUC__ >= 3 2945 + #define RTCHECK(e) __builtin_expect(e, 1) 2946 + #else /* GNUC */ 2947 + #define RTCHECK(e) (e) 2948 + #endif /* GNUC */ 2949 + #else /* !INSECURE */ 2950 + #define RTCHECK(e) (1) 2951 + #endif /* !INSECURE */ 2952 + 2953 + /* macros to set up inuse chunks with or without footers */ 2954 + 2955 + #if !FOOTERS 2956 + 2957 + #define mark_inuse_foot(M,p,s) 2958 + 2959 + /* Set cinuse bit and pinuse bit of next chunk */ 2960 + #define set_inuse(M,p,s)\ 2961 + ((p)->head = (((p)->head & PINUSE_BIT)|s|CINUSE_BIT),\ 2962 + ((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT) 2963 + 2964 + /* Set cinuse and pinuse of this chunk and pinuse of next chunk */ 2965 + #define set_inuse_and_pinuse(M,p,s)\ 2966 + ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\ 2967 + ((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT) 2968 + 2969 + /* Set size, cinuse and pinuse bit of this chunk */ 2970 + #define set_size_and_pinuse_of_inuse_chunk(M, p, s)\ 2971 + ((p)->head = (s|PINUSE_BIT|CINUSE_BIT)) 2972 + 2973 + #else /* FOOTERS */ 2974 + 2975 + /* Set foot of inuse chunk to be xor of mstate and seed */ 2976 + #define mark_inuse_foot(M,p,s)\ 2977 + (((mchunkptr)((char*)(p) + (s)))->prev_foot = ((size_t)(M) ^ mparams.magic)) 2978 + 2979 + #define get_mstate_for(p)\ 2980 + ((mstate)(((mchunkptr)((char*)(p) +\ 2981 + (chunksize(p))))->prev_foot ^ mparams.magic)) 2982 + 2983 + #define set_inuse(M,p,s)\ 2984 + ((p)->head = (((p)->head & PINUSE_BIT)|s|CINUSE_BIT),\ 2985 + (((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT), \ 2986 + mark_inuse_foot(M,p,s)) 2987 + 2988 + #define set_inuse_and_pinuse(M,p,s)\ 2989 + ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\ 2990 + (((mchunkptr)(((char*)(p)) + (s)))->head |= PINUSE_BIT),\ 2991 + mark_inuse_foot(M,p,s)) 2992 + 2993 + #define set_size_and_pinuse_of_inuse_chunk(M, p, s)\ 2994 + ((p)->head = (s|PINUSE_BIT|CINUSE_BIT),\ 2995 + mark_inuse_foot(M, p, s)) 2996 + 2997 + #endif /* !FOOTERS */ 2998 + 2999 + /* ---------------------------- setting mparams -------------------------- */ 3000 + 3001 + /* Initialize mparams */ 3002 + static int init_mparams(void) { 3003 + #ifdef NEED_GLOBAL_LOCK_INIT 3004 + if (malloc_global_mutex_status <= 0) 3005 + init_malloc_global_mutex(); 3006 + #endif 3007 + 3008 + ACQUIRE_MALLOC_GLOBAL_LOCK(); 3009 + if (mparams.magic == 0) { 3010 + size_t magic; 3011 + size_t psize; 3012 + size_t gsize; 3013 + 3014 + #ifndef WIN32 3015 + psize = malloc_getpagesize; 3016 + gsize = ((DEFAULT_GRANULARITY != 0)? DEFAULT_GRANULARITY : psize); 3017 + #else /* WIN32 */ 3018 + { 3019 + SYSTEM_INFO system_info; 3020 + GetSystemInfo(&system_info); 3021 + psize = system_info.dwPageSize; 3022 + gsize = ((DEFAULT_GRANULARITY != 0)? 3023 + DEFAULT_GRANULARITY : system_info.dwAllocationGranularity); 3024 + } 3025 + #endif /* WIN32 */ 3026 + 3027 + /* Sanity-check configuration: 3028 + size_t must be unsigned and as wide as pointer type. 3029 + ints must be at least 4 bytes. 3030 + alignment must be at least 8. 3031 + Alignment, min chunk size, and page size must all be powers of 2. 3032 + */ 3033 + if ((sizeof(size_t) != sizeof(char*)) || 3034 + (MAX_SIZE_T < MIN_CHUNK_SIZE) || 3035 + (sizeof(int) < 4) || 3036 + (MALLOC_ALIGNMENT < (size_t)8U) || 3037 + ((MALLOC_ALIGNMENT & (MALLOC_ALIGNMENT-SIZE_T_ONE)) != 0) || 3038 + ((MCHUNK_SIZE & (MCHUNK_SIZE-SIZE_T_ONE)) != 0) || 3039 + ((gsize & (gsize-SIZE_T_ONE)) != 0) || 3040 + ((psize & (psize-SIZE_T_ONE)) != 0)) 3041 + ABORT; 3042 + 3043 + mparams.granularity = gsize; 3044 + mparams.page_size = psize; 3045 + mparams.mmap_threshold = DEFAULT_MMAP_THRESHOLD; 3046 + mparams.trim_threshold = DEFAULT_TRIM_THRESHOLD; 3047 + #if MORECORE_CONTIGUOUS 3048 + mparams.default_mflags = USE_LOCK_BIT|USE_MMAP_BIT; 3049 + #else /* MORECORE_CONTIGUOUS */ 3050 + mparams.default_mflags = USE_LOCK_BIT|USE_MMAP_BIT|USE_NONCONTIGUOUS_BIT; 3051 + #endif /* MORECORE_CONTIGUOUS */ 3052 + 3053 + #if !ONLY_MSPACES 3054 + /* Set up lock for main malloc area */ 3055 + gm->mflags = mparams.default_mflags; 3056 + INITIAL_LOCK(&gm->mutex); 3057 + #endif 3058 + 3059 + #if (FOOTERS && !INSECURE) 3060 + { 3061 + #if USE_DEV_RANDOM 3062 + int fd; 3063 + unsigned char buf[sizeof(size_t)]; 3064 + /* Try to use /dev/urandom, else fall back on using time */ 3065 + if ((fd = open("/dev/urandom", O_RDONLY)) >= 0 && 3066 + read(fd, buf, sizeof(buf)) == sizeof(buf)) { 3067 + magic = *((size_t *) buf); 3068 + close(fd); 3069 + } 3070 + else 3071 + #endif /* USE_DEV_RANDOM */ 3072 + #ifdef WIN32 3073 + magic = (size_t)(GetTickCount() ^ (size_t)0x55555555U); 3074 + #else 3075 + magic = (size_t)(time(0) ^ (size_t)0x55555555U); 3076 + #endif 3077 + magic |= (size_t)8U; /* ensure nonzero */ 3078 + magic &= ~(size_t)7U; /* improve chances of fault for bad values */ 3079 + } 3080 + #else /* (FOOTERS && !INSECURE) */ 3081 + magic = (size_t)0x58585858U; 3082 + #endif /* (FOOTERS && !INSECURE) */ 3083 + 3084 + mparams.magic = magic; 3085 + } 3086 + 3087 + RELEASE_MALLOC_GLOBAL_LOCK(); 3088 + return 1; 3089 + } 3090 + 3091 + /* support for mallopt */ 3092 + static int change_mparam(int param_number, int value) { 3093 + size_t val = (value == -1)? MAX_SIZE_T : (size_t)value; 3094 + ensure_initialization(); 3095 + switch(param_number) { 3096 + case M_TRIM_THRESHOLD: 3097 + mparams.trim_threshold = val; 3098 + return 1; 3099 + case M_GRANULARITY: 3100 + if (val >= mparams.page_size && ((val & (val-1)) == 0)) { 3101 + mparams.granularity = val; 3102 + return 1; 3103 + } 3104 + else 3105 + return 0; 3106 + case M_MMAP_THRESHOLD: 3107 + mparams.mmap_threshold = val; 3108 + return 1; 3109 + default: 3110 + return 0; 3111 + } 3112 + } 3113 + 3114 + #if DEBUG 3115 + /* ------------------------- Debugging Support --------------------------- */ 3116 + 3117 + /* Check properties of any chunk, whether free, inuse, mmapped etc */ 3118 + static void do_check_any_chunk(mstate m, mchunkptr p) { 3119 + assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD)); 3120 + assert(ok_address(m, p)); 3121 + } 3122 + 3123 + /* Check properties of top chunk */ 3124 + static void do_check_top_chunk(mstate m, mchunkptr p) { 3125 + msegmentptr sp = segment_holding(m, (char*)p); 3126 + size_t sz = p->head & ~INUSE_BITS; /* third-lowest bit can be set! */ 3127 + assert(sp != 0); 3128 + assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD)); 3129 + assert(ok_address(m, p)); 3130 + assert(sz == m->topsize); 3131 + assert(sz > 0); 3132 + assert(sz == ((sp->base + sp->size) - (char*)p) - TOP_FOOT_SIZE); 3133 + assert(pinuse(p)); 3134 + assert(!pinuse(chunk_plus_offset(p, sz))); 3135 + } 3136 + 3137 + /* Check properties of (inuse) mmapped chunks */ 3138 + static void do_check_mmapped_chunk(mstate m, mchunkptr p) { 3139 + size_t sz = chunksize(p); 3140 + size_t len = (sz + (p->prev_foot & ~IS_MMAPPED_BIT) + MMAP_FOOT_PAD); 3141 + assert(is_mmapped(p)); 3142 + assert(use_mmap(m)); 3143 + assert((is_aligned(chunk2mem(p))) || (p->head == FENCEPOST_HEAD)); 3144 + assert(ok_address(m, p)); 3145 + assert(!is_small(sz)); 3146 + assert((len & (mparams.page_size-SIZE_T_ONE)) == 0); 3147 + assert(chunk_plus_offset(p, sz)->head == FENCEPOST_HEAD); 3148 + assert(chunk_plus_offset(p, sz+SIZE_T_SIZE)->head == 0); 3149 + } 3150 + 3151 + /* Check properties of inuse chunks */ 3152 + static void do_check_inuse_chunk(mstate m, mchunkptr p) { 3153 + do_check_any_chunk(m, p); 3154 + assert(cinuse(p)); 3155 + assert(next_pinuse(p)); 3156 + /* If not pinuse and not mmapped, previous chunk has OK offset */ 3157 + assert(is_mmapped(p) || pinuse(p) || next_chunk(prev_chunk(p)) == p); 3158 + if (is_mmapped(p)) 3159 + do_check_mmapped_chunk(m, p); 3160 + } 3161 + 3162 + /* Check properties of free chunks */ 3163 + static void do_check_free_chunk(mstate m, mchunkptr p) { 3164 + size_t sz = chunksize(p); 3165 + mchunkptr next = chunk_plus_offset(p, sz); 3166 + do_check_any_chunk(m, p); 3167 + assert(!cinuse(p)); 3168 + assert(!next_pinuse(p)); 3169 + assert (!is_mmapped(p)); 3170 + if (p != m->dv && p != m->top) { 3171 + if (sz >= MIN_CHUNK_SIZE) { 3172 + assert((sz & CHUNK_ALIGN_MASK) == 0); 3173 + assert(is_aligned(chunk2mem(p))); 3174 + assert(next->prev_foot == sz); 3175 + assert(pinuse(p)); 3176 + assert (next == m->top || cinuse(next)); 3177 + assert(p->fd->bk == p); 3178 + assert(p->bk->fd == p); 3179 + } 3180 + else /* markers are always of size SIZE_T_SIZE */ 3181 + assert(sz == SIZE_T_SIZE); 3182 + } 3183 + } 3184 + 3185 + /* Check properties of malloced chunks at the point they are malloced */ 3186 + static void do_check_malloced_chunk(mstate m, void* mem, size_t s) { 3187 + if (mem != 0) { 3188 + mchunkptr p = mem2chunk(mem); 3189 + size_t sz = p->head & ~(PINUSE_BIT|CINUSE_BIT); 3190 + do_check_inuse_chunk(m, p); 3191 + assert((sz & CHUNK_ALIGN_MASK) == 0); 3192 + assert(sz >= MIN_CHUNK_SIZE); 3193 + assert(sz >= s); 3194 + /* unless mmapped, size is less than MIN_CHUNK_SIZE more than request */ 3195 + assert(is_mmapped(p) || sz < (s + MIN_CHUNK_SIZE)); 3196 + } 3197 + } 3198 + 3199 + /* Check a tree and its subtrees. */ 3200 + static void do_check_tree(mstate m, tchunkptr t) { 3201 + tchunkptr head = 0; 3202 + tchunkptr u = t; 3203 + bindex_t tindex = t->index; 3204 + size_t tsize = chunksize(t); 3205 + bindex_t idx; 3206 + compute_tree_index(tsize, idx); 3207 + assert(tindex == idx); 3208 + assert(tsize >= MIN_LARGE_SIZE); 3209 + assert(tsize >= minsize_for_tree_index(idx)); 3210 + assert((idx == NTREEBINS-1) || (tsize < minsize_for_tree_index((idx+1)))); 3211 + 3212 + do { /* traverse through chain of same-sized nodes */ 3213 + do_check_any_chunk(m, ((mchunkptr)u)); 3214 + assert(u->index == tindex); 3215 + assert(chunksize(u) == tsize); 3216 + assert(!cinuse(u)); 3217 + assert(!next_pinuse(u)); 3218 + assert(u->fd->bk == u); 3219 + assert(u->bk->fd == u); 3220 + if (u->parent == 0) { 3221 + assert(u->child[0] == 0); 3222 + assert(u->child[1] == 0); 3223 + } 3224 + else { 3225 + assert(head == 0); /* only one node on chain has parent */ 3226 + head = u; 3227 + assert(u->parent != u); 3228 + assert (u->parent->child[0] == u || 3229 + u->parent->child[1] == u || 3230 + *((tbinptr*)(u->parent)) == u); 3231 + if (u->child[0] != 0) { 3232 + assert(u->child[0]->parent == u); 3233 + assert(u->child[0] != u); 3234 + do_check_tree(m, u->child[0]); 3235 + } 3236 + if (u->child[1] != 0) { 3237 + assert(u->child[1]->parent == u); 3238 + assert(u->child[1] != u); 3239 + do_check_tree(m, u->child[1]); 3240 + } 3241 + if (u->child[0] != 0 && u->child[1] != 0) { 3242 + assert(chunksize(u->child[0]) < chunksize(u->child[1])); 3243 + } 3244 + } 3245 + u = u->fd; 3246 + } while (u != t); 3247 + assert(head != 0); 3248 + } 3249 + 3250 + /* Check all the chunks in a treebin. */ 3251 + static void do_check_treebin(mstate m, bindex_t i) { 3252 + tbinptr* tb = treebin_at(m, i); 3253 + tchunkptr t = *tb; 3254 + int empty = (m->treemap & (1U << i)) == 0; 3255 + if (t == 0) 3256 + assert(empty); 3257 + if (!empty) 3258 + do_check_tree(m, t); 3259 + } 3260 + 3261 + /* Check all the chunks in a smallbin. */ 3262 + static void do_check_smallbin(mstate m, bindex_t i) { 3263 + sbinptr b = smallbin_at(m, i); 3264 + mchunkptr p = b->bk; 3265 + unsigned int empty = (m->smallmap & (1U << i)) == 0; 3266 + if (p == b) 3267 + assert(empty); 3268 + if (!empty) { 3269 + for (; p != b; p = p->bk) { 3270 + size_t size = chunksize(p); 3271 + mchunkptr q; 3272 + /* each chunk claims to be free */ 3273 + do_check_free_chunk(m, p); 3274 + /* chunk belongs in bin */ 3275 + assert(small_index(size) == i); 3276 + assert(p->bk == b || chunksize(p->bk) == chunksize(p)); 3277 + /* chunk is followed by an inuse chunk */ 3278 + q = next_chunk(p); 3279 + if (q->head != FENCEPOST_HEAD) 3280 + do_check_inuse_chunk(m, q); 3281 + } 3282 + } 3283 + } 3284 + 3285 + /* Find x in a bin. Used in other check functions. */ 3286 + static int bin_find(mstate m, mchunkptr x) { 3287 + size_t size = chunksize(x); 3288 + if (is_small(size)) { 3289 + bindex_t sidx = small_index(size); 3290 + sbinptr b = smallbin_at(m, sidx); 3291 + if (smallmap_is_marked(m, sidx)) { 3292 + mchunkptr p = b; 3293 + do { 3294 + if (p == x) 3295 + return 1; 3296 + } while ((p = p->fd) != b); 3297 + } 3298 + } 3299 + else { 3300 + bindex_t tidx; 3301 + compute_tree_index(size, tidx); 3302 + if (treemap_is_marked(m, tidx)) { 3303 + tchunkptr t = *treebin_at(m, tidx); 3304 + size_t sizebits = size << leftshift_for_tree_index(tidx); 3305 + while (t != 0 && chunksize(t) != size) { 3306 + t = t->child[(sizebits >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1]; 3307 + sizebits <<= 1; 3308 + } 3309 + if (t != 0) { 3310 + tchunkptr u = t; 3311 + do { 3312 + if (u == (tchunkptr)x) 3313 + return 1; 3314 + } while ((u = u->fd) != t); 3315 + } 3316 + } 3317 + } 3318 + return 0; 3319 + } 3320 + 3321 + /* Traverse each chunk and check it; return total */ 3322 + static size_t traverse_and_check(mstate m) { 3323 + size_t sum = 0; 3324 + if (is_initialized(m)) { 3325 + msegmentptr s = &m->seg; 3326 + sum += m->topsize + TOP_FOOT_SIZE; 3327 + while (s != 0) { 3328 + mchunkptr q = align_as_chunk(s->base); 3329 + mchunkptr lastq = 0; 3330 + assert(pinuse(q)); 3331 + while (segment_holds(s, q) && 3332 + q != m->top && q->head != FENCEPOST_HEAD) { 3333 + sum += chunksize(q); 3334 + if (cinuse(q)) { 3335 + assert(!bin_find(m, q)); 3336 + do_check_inuse_chunk(m, q); 3337 + } 3338 + else { 3339 + assert(q == m->dv || bin_find(m, q)); 3340 + assert(lastq == 0 || cinuse(lastq)); /* Not 2 consecutive free */ 3341 + do_check_free_chunk(m, q); 3342 + } 3343 + lastq = q; 3344 + q = next_chunk(q); 3345 + } 3346 + s = s->next; 3347 + } 3348 + } 3349 + return sum; 3350 + } 3351 + 3352 + /* Check all properties of malloc_state. */ 3353 + static void do_check_malloc_state(mstate m) { 3354 + bindex_t i; 3355 + size_t total; 3356 + /* check bins */ 3357 + for (i = 0; i < NSMALLBINS; ++i) 3358 + do_check_smallbin(m, i); 3359 + for (i = 0; i < NTREEBINS; ++i) 3360 + do_check_treebin(m, i); 3361 + 3362 + if (m->dvsize != 0) { /* check dv chunk */ 3363 + do_check_any_chunk(m, m->dv); 3364 + assert(m->dvsize == chunksize(m->dv)); 3365 + assert(m->dvsize >= MIN_CHUNK_SIZE); 3366 + assert(bin_find(m, m->dv) == 0); 3367 + } 3368 + 3369 + if (m->top != 0) { /* check top chunk */ 3370 + do_check_top_chunk(m, m->top); 3371 + /*assert(m->topsize == chunksize(m->top)); redundant */ 3372 + assert(m->topsize > 0); 3373 + assert(bin_find(m, m->top) == 0); 3374 + } 3375 + 3376 + total = traverse_and_check(m); 3377 + assert(total <= m->footprint); 3378 + assert(m->footprint <= m->max_footprint); 3379 + } 3380 + #endif /* DEBUG */ 3381 + 3382 + /* ----------------------------- statistics ------------------------------ */ 3383 + 3384 + #if !NO_MALLINFO 3385 + static struct mallinfo internal_mallinfo(mstate m) { 3386 + struct mallinfo nm = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 }; 3387 + ensure_initialization(); 3388 + if (!PREACTION(m)) { 3389 + check_malloc_state(m); 3390 + if (is_initialized(m)) { 3391 + size_t nfree = SIZE_T_ONE; /* top always free */ 3392 + size_t mfree = m->topsize + TOP_FOOT_SIZE; 3393 + size_t sum = mfree; 3394 + msegmentptr s = &m->seg; 3395 + while (s != 0) { 3396 + mchunkptr q = align_as_chunk(s->base); 3397 + while (segment_holds(s, q) && 3398 + q != m->top && q->head != FENCEPOST_HEAD) { 3399 + size_t sz = chunksize(q); 3400 + sum += sz; 3401 + if (!cinuse(q)) { 3402 + mfree += sz; 3403 + ++nfree; 3404 + } 3405 + q = next_chunk(q); 3406 + } 3407 + s = s->next; 3408 + } 3409 + 3410 + nm.arena = sum; 3411 + nm.ordblks = nfree; 3412 + nm.hblkhd = m->footprint - sum; 3413 + nm.usmblks = m->max_footprint; 3414 + nm.uordblks = m->footprint - mfree; 3415 + nm.fordblks = mfree; 3416 + nm.keepcost = m->topsize; 3417 + } 3418 + 3419 + POSTACTION(m); 3420 + } 3421 + return nm; 3422 + } 3423 + #endif /* !NO_MALLINFO */ 3424 + 3425 + static void internal_malloc_stats(mstate m) { 3426 + ensure_initialization(); 3427 + if (!PREACTION(m)) { 3428 + size_t maxfp = 0; 3429 + size_t fp = 0; 3430 + size_t used = 0; 3431 + check_malloc_state(m); 3432 + if (is_initialized(m)) { 3433 + msegmentptr s = &m->seg; 3434 + maxfp = m->max_footprint; 3435 + fp = m->footprint; 3436 + used = fp - (m->topsize + TOP_FOOT_SIZE); 3437 + 3438 + while (s != 0) { 3439 + mchunkptr q = align_as_chunk(s->base); 3440 + while (segment_holds(s, q) && 3441 + q != m->top && q->head != FENCEPOST_HEAD) { 3442 + if (!cinuse(q)) 3443 + used -= chunksize(q); 3444 + q = next_chunk(q); 3445 + } 3446 + s = s->next; 3447 + } 3448 + } 3449 + 3450 + fprintf(stderr, "max system bytes = %10lu\n", (unsigned long)(maxfp)); 3451 + fprintf(stderr, "system bytes = %10lu\n", (unsigned long)(fp)); 3452 + fprintf(stderr, "in use bytes = %10lu\n", (unsigned long)(used)); 3453 + 3454 + POSTACTION(m); 3455 + } 3456 + } 3457 + 3458 + /* ----------------------- Operations on smallbins ----------------------- */ 3459 + 3460 + /* 3461 + Various forms of linking and unlinking are defined as macros. Even 3462 + the ones for trees, which are very long but have very short typical 3463 + paths. This is ugly but reduces reliance on inlining support of 3464 + compilers. 3465 + */ 3466 + 3467 + /* Link a free chunk into a smallbin */ 3468 + #define insert_small_chunk(M, P, S) {\ 3469 + bindex_t I = small_index(S);\ 3470 + mchunkptr B = smallbin_at(M, I);\ 3471 + mchunkptr F = B;\ 3472 + assert(S >= MIN_CHUNK_SIZE);\ 3473 + if (!smallmap_is_marked(M, I))\ 3474 + mark_smallmap(M, I);\ 3475 + else if (RTCHECK(ok_address(M, B->fd)))\ 3476 + F = B->fd;\ 3477 + else {\ 3478 + CORRUPTION_ERROR_ACTION(M);\ 3479 + }\ 3480 + B->fd = P;\ 3481 + F->bk = P;\ 3482 + P->fd = F;\ 3483 + P->bk = B;\ 3484 + } 3485 + 3486 + /* Unlink a chunk from a smallbin */ 3487 + #define unlink_small_chunk(M, P, S) {\ 3488 + mchunkptr F = P->fd;\ 3489 + mchunkptr B = P->bk;\ 3490 + bindex_t I = small_index(S);\ 3491 + assert(P != B);\ 3492 + assert(P != F);\ 3493 + assert(chunksize(P) == small_index2size(I));\ 3494 + if (F == B)\ 3495 + clear_smallmap(M, I);\ 3496 + else if (RTCHECK((F == smallbin_at(M,I) || ok_address(M, F)) &&\ 3497 + (B == smallbin_at(M,I) || ok_address(M, B)))) {\ 3498 + F->bk = B;\ 3499 + B->fd = F;\ 3500 + }\ 3501 + else {\ 3502 + CORRUPTION_ERROR_ACTION(M);\ 3503 + }\ 3504 + } 3505 + 3506 + /* Unlink the first chunk from a smallbin */ 3507 + #define unlink_first_small_chunk(M, B, P, I) {\ 3508 + mchunkptr F = P->fd;\ 3509 + assert(P != B);\ 3510 + assert(P != F);\ 3511 + assert(chunksize(P) == small_index2size(I));\ 3512 + if (B == F)\ 3513 + clear_smallmap(M, I);\ 3514 + else if (RTCHECK(ok_address(M, F))) {\ 3515 + B->fd = F;\ 3516 + F->bk = B;\ 3517 + }\ 3518 + else {\ 3519 + CORRUPTION_ERROR_ACTION(M);\ 3520 + }\ 3521 + } 3522 + 3523 + 3524 + 3525 + /* Replace dv node, binning the old one */ 3526 + /* Used only when dvsize known to be small */ 3527 + #define replace_dv(M, P, S) {\ 3528 + size_t DVS = M->dvsize;\ 3529 + if (DVS != 0) {\ 3530 + mchunkptr DV = M->dv;\ 3531 + assert(is_small(DVS));\ 3532 + insert_small_chunk(M, DV, DVS);\ 3533 + }\ 3534 + M->dvsize = S;\ 3535 + M->dv = P;\ 3536 + } 3537 + 3538 + /* ------------------------- Operations on trees ------------------------- */ 3539 + 3540 + /* Insert chunk into tree */ 3541 + #define insert_large_chunk(M, X, S) {\ 3542 + tbinptr* H;\ 3543 + bindex_t I;\ 3544 + compute_tree_index(S, I);\ 3545 + H = treebin_at(M, I);\ 3546 + X->index = I;\ 3547 + X->child[0] = X->child[1] = 0;\ 3548 + if (!treemap_is_marked(M, I)) {\ 3549 + mark_treemap(M, I);\ 3550 + *H = X;\ 3551 + X->parent = (tchunkptr)H;\ 3552 + X->fd = X->bk = X;\ 3553 + }\ 3554 + else {\ 3555 + tchunkptr T = *H;\ 3556 + size_t K = S << leftshift_for_tree_index(I);\ 3557 + for (;;) {\ 3558 + if (chunksize(T) != S) {\ 3559 + tchunkptr* C = &(T->child[(K >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1]);\ 3560 + K <<= 1;\ 3561 + if (*C != 0)\ 3562 + T = *C;\ 3563 + else if (RTCHECK(ok_address(M, C))) {\ 3564 + *C = X;\ 3565 + X->parent = T;\ 3566 + X->fd = X->bk = X;\ 3567 + break;\ 3568 + }\ 3569 + else {\ 3570 + CORRUPTION_ERROR_ACTION(M);\ 3571 + break;\ 3572 + }\ 3573 + }\ 3574 + else {\ 3575 + tchunkptr F = T->fd;\ 3576 + if (RTCHECK(ok_address(M, T) && ok_address(M, F))) {\ 3577 + T->fd = F->bk = X;\ 3578 + X->fd = F;\ 3579 + X->bk = T;\ 3580 + X->parent = 0;\ 3581 + break;\ 3582 + }\ 3583 + else {\ 3584 + CORRUPTION_ERROR_ACTION(M);\ 3585 + break;\ 3586 + }\ 3587 + }\ 3588 + }\ 3589 + }\ 3590 + } 3591 + 3592 + /* 3593 + Unlink steps: 3594 + 3595 + 1. If x is a chained node, unlink it from its same-sized fd/bk links 3596 + and choose its bk node as its replacement. 3597 + 2. If x was the last node of its size, but not a leaf node, it must 3598 + be replaced with a leaf node (not merely one with an open left or 3599 + right), to make sure that lefts and rights of descendents 3600 + correspond properly to bit masks. We use the rightmost descendent 3601 + of x. We could use any other leaf, but this is easy to locate and 3602 + tends to counteract removal of leftmosts elsewhere, and so keeps 3603 + paths shorter than minimally guaranteed. This doesn't loop much 3604 + because on average a node in a tree is near the bottom. 3605 + 3. If x is the base of a chain (i.e., has parent links) relink 3606 + x's parent and children to x's replacement (or null if none). 3607 + */ 3608 + 3609 + #define unlink_large_chunk(M, X) {\ 3610 + tchunkptr XP = X->parent;\ 3611 + tchunkptr R;\ 3612 + if (X->bk != X) {\ 3613 + tchunkptr F = X->fd;\ 3614 + R = X->bk;\ 3615 + if (RTCHECK(ok_address(M, F))) {\ 3616 + F->bk = R;\ 3617 + R->fd = F;\ 3618 + }\ 3619 + else {\ 3620 + CORRUPTION_ERROR_ACTION(M);\ 3621 + }\ 3622 + }\ 3623 + else {\ 3624 + tchunkptr* RP;\ 3625 + if (((R = *(RP = &(X->child[1]))) != 0) ||\ 3626 + ((R = *(RP = &(X->child[0]))) != 0)) {\ 3627 + tchunkptr* CP;\ 3628 + while ((*(CP = &(R->child[1])) != 0) ||\ 3629 + (*(CP = &(R->child[0])) != 0)) {\ 3630 + R = *(RP = CP);\ 3631 + }\ 3632 + if (RTCHECK(ok_address(M, RP)))\ 3633 + *RP = 0;\ 3634 + else {\ 3635 + CORRUPTION_ERROR_ACTION(M);\ 3636 + }\ 3637 + }\ 3638 + }\ 3639 + if (XP != 0) {\ 3640 + tbinptr* H = treebin_at(M, X->index);\ 3641 + if (X == *H) {\ 3642 + if ((*H = R) == 0) \ 3643 + clear_treemap(M, X->index);\ 3644 + }\ 3645 + else if (RTCHECK(ok_address(M, XP))) {\ 3646 + if (XP->child[0] == X) \ 3647 + XP->child[0] = R;\ 3648 + else \ 3649 + XP->child[1] = R;\ 3650 + }\ 3651 + else\ 3652 + CORRUPTION_ERROR_ACTION(M);\ 3653 + if (R != 0) {\ 3654 + if (RTCHECK(ok_address(M, R))) {\ 3655 + tchunkptr C0, C1;\ 3656 + R->parent = XP;\ 3657 + if ((C0 = X->child[0]) != 0) {\ 3658 + if (RTCHECK(ok_address(M, C0))) {\ 3659 + R->child[0] = C0;\ 3660 + C0->parent = R;\ 3661 + }\ 3662 + else\ 3663 + CORRUPTION_ERROR_ACTION(M);\ 3664 + }\ 3665 + if ((C1 = X->child[1]) != 0) {\ 3666 + if (RTCHECK(ok_address(M, C1))) {\ 3667 + R->child[1] = C1;\ 3668 + C1->parent = R;\ 3669 + }\ 3670 + else\ 3671 + CORRUPTION_ERROR_ACTION(M);\ 3672 + }\ 3673 + }\ 3674 + else\ 3675 + CORRUPTION_ERROR_ACTION(M);\ 3676 + }\ 3677 + }\ 3678 + } 3679 + 3680 + /* Relays to large vs small bin operations */ 3681 + 3682 + #define insert_chunk(M, P, S)\ 3683 + if (is_small(S)) insert_small_chunk(M, P, S)\ 3684 + else { tchunkptr TP = (tchunkptr)(P); insert_large_chunk(M, TP, S); } 3685 + 3686 + #define unlink_chunk(M, P, S)\ 3687 + if (is_small(S)) unlink_small_chunk(M, P, S)\ 3688 + else { tchunkptr TP = (tchunkptr)(P); unlink_large_chunk(M, TP); } 3689 + 3690 + 3691 + /* Relays to internal calls to malloc/free from realloc, memalign etc */ 3692 + 3693 + #if ONLY_MSPACES 3694 + #define internal_malloc(m, b) mspace_malloc(m, b) 3695 + #define internal_free(m, mem) mspace_free(m,mem); 3696 + #else /* ONLY_MSPACES */ 3697 + #if MSPACES 3698 + #define internal_malloc(m, b)\ 3699 + (m == gm)? dlmalloc(b) : mspace_malloc(m, b) 3700 + #define internal_free(m, mem)\ 3701 + if (m == gm) dlfree(mem); else mspace_free(m,mem); 3702 + #else /* MSPACES */ 3703 + #define internal_malloc(m, b) dlmalloc(b) 3704 + #define internal_free(m, mem) dlfree(mem) 3705 + #endif /* MSPACES */ 3706 + #endif /* ONLY_MSPACES */ 3707 + 3708 + /* ----------------------- Direct-mmapping chunks ----------------------- */ 3709 + 3710 + /* 3711 + Directly mmapped chunks are set up with an offset to the start of 3712 + the mmapped region stored in the prev_foot field of the chunk. This 3713 + allows reconstruction of the required argument to MUNMAP when freed, 3714 + and also allows adjustment of the returned chunk to meet alignment 3715 + requirements (especially in memalign). There is also enough space 3716 + allocated to hold a fake next chunk of size SIZE_T_SIZE to maintain 3717 + the PINUSE bit so frees can be checked. 3718 + */ 3719 + 3720 + /* Malloc using mmap */ 3721 + static void* mmap_alloc(mstate m, size_t nb) { 3722 + size_t mmsize = mmap_align(nb + SIX_SIZE_T_SIZES + CHUNK_ALIGN_MASK); 3723 + if (mmsize > nb) { /* Check for wrap around 0 */ 3724 + char* mm = (char*)(CALL_DIRECT_MMAP(mmsize)); 3725 + if (mm != CMFAIL) { 3726 + size_t offset = align_offset(chunk2mem(mm)); 3727 + size_t psize = mmsize - offset - MMAP_FOOT_PAD; 3728 + mchunkptr p = (mchunkptr)(mm + offset); 3729 + p->prev_foot = offset | IS_MMAPPED_BIT; 3730 + (p)->head = (psize|CINUSE_BIT); 3731 + mark_inuse_foot(m, p, psize); 3732 + chunk_plus_offset(p, psize)->head = FENCEPOST_HEAD; 3733 + chunk_plus_offset(p, psize+SIZE_T_SIZE)->head = 0; 3734 + 3735 + if (mm < m->least_addr) 3736 + m->least_addr = mm; 3737 + if ((m->footprint += mmsize) > m->max_footprint) 3738 + m->max_footprint = m->footprint; 3739 + assert(is_aligned(chunk2mem(p))); 3740 + check_mmapped_chunk(m, p); 3741 + return chunk2mem(p); 3742 + } 3743 + } 3744 + return 0; 3745 + } 3746 + 3747 + /* Realloc using mmap */ 3748 + static mchunkptr mmap_resize(mstate m, mchunkptr oldp, size_t nb) { 3749 + size_t oldsize = chunksize(oldp); 3750 + if (is_small(nb)) /* Can't shrink mmap regions below small size */ 3751 + return 0; 3752 + /* Keep old chunk if big enough but not too big */ 3753 + if (oldsize >= nb + SIZE_T_SIZE && 3754 + (oldsize - nb) <= (mparams.granularity << 1)) 3755 + return oldp; 3756 + else { 3757 + size_t offset = oldp->prev_foot & ~IS_MMAPPED_BIT; 3758 + size_t oldmmsize = oldsize + offset + MMAP_FOOT_PAD; 3759 + size_t newmmsize = mmap_align(nb + SIX_SIZE_T_SIZES + CHUNK_ALIGN_MASK); 3760 + char* cp = (char*)CALL_MREMAP((char*)oldp - offset, 3761 + oldmmsize, newmmsize, 1); 3762 + if (cp != CMFAIL) { 3763 + mchunkptr newp = (mchunkptr)(cp + offset); 3764 + size_t psize = newmmsize - offset - MMAP_FOOT_PAD; 3765 + newp->head = (psize|CINUSE_BIT); 3766 + mark_inuse_foot(m, newp, psize); 3767 + chunk_plus_offset(newp, psize)->head = FENCEPOST_HEAD; 3768 + chunk_plus_offset(newp, psize+SIZE_T_SIZE)->head = 0; 3769 + 3770 + if (cp < m->least_addr) 3771 + m->least_addr = cp; 3772 + if ((m->footprint += newmmsize - oldmmsize) > m->max_footprint) 3773 + m->max_footprint = m->footprint; 3774 + check_mmapped_chunk(m, newp); 3775 + return newp; 3776 + } 3777 + } 3778 + return 0; 3779 + } 3780 + 3781 + /* -------------------------- mspace management -------------------------- */ 3782 + 3783 + /* Initialize top chunk and its size */ 3784 + static void init_top(mstate m, mchunkptr p, size_t psize) { 3785 + /* Ensure alignment */ 3786 + size_t offset = align_offset(chunk2mem(p)); 3787 + p = (mchunkptr)((char*)p + offset); 3788 + psize -= offset; 3789 + 3790 + m->top = p; 3791 + m->topsize = psize; 3792 + p->head = psize | PINUSE_BIT; 3793 + /* set size of fake trailing chunk holding overhead space only once */ 3794 + chunk_plus_offset(p, psize)->head = TOP_FOOT_SIZE; 3795 + m->trim_check = mparams.trim_threshold; /* reset on each update */ 3796 + } 3797 + 3798 + /* Initialize bins for a new mstate that is otherwise zeroed out */ 3799 + static void init_bins(mstate m) { 3800 + /* Establish circular links for smallbins */ 3801 + bindex_t i; 3802 + for (i = 0; i < NSMALLBINS; ++i) { 3803 + sbinptr bin = smallbin_at(m,i); 3804 + bin->fd = bin->bk = bin; 3805 + } 3806 + } 3807 + 3808 + #if PROCEED_ON_ERROR 3809 + 3810 + /* default corruption action */ 3811 + static void reset_on_error(mstate m) { 3812 + int i; 3813 + ++malloc_corruption_error_count; 3814 + /* Reinitialize fields to forget about all memory */ 3815 + m->smallbins = m->treebins = 0; 3816 + m->dvsize = m->topsize = 0; 3817 + m->seg.base = 0; 3818 + m->seg.size = 0; 3819 + m->seg.next = 0; 3820 + m->top = m->dv = 0; 3821 + for (i = 0; i < NTREEBINS; ++i) 3822 + *treebin_at(m, i) = 0; 3823 + init_bins(m); 3824 + } 3825 + #endif /* PROCEED_ON_ERROR */ 3826 + 3827 + /* Allocate chunk and prepend remainder with chunk in successor base. */ 3828 + static void* prepend_alloc(mstate m, char* newbase, char* oldbase, 3829 + size_t nb) { 3830 + mchunkptr p = align_as_chunk(newbase); 3831 + mchunkptr oldfirst = align_as_chunk(oldbase); 3832 + size_t psize = (char*)oldfirst - (char*)p; 3833 + mchunkptr q = chunk_plus_offset(p, nb); 3834 + size_t qsize = psize - nb; 3835 + set_size_and_pinuse_of_inuse_chunk(m, p, nb); 3836 + 3837 + assert((char*)oldfirst > (char*)q); 3838 + assert(pinuse(oldfirst)); 3839 + assert(qsize >= MIN_CHUNK_SIZE); 3840 + 3841 + /* consolidate remainder with first chunk of old base */ 3842 + if (oldfirst == m->top) { 3843 + size_t tsize = m->topsize += qsize; 3844 + m->top = q; 3845 + q->head = tsize | PINUSE_BIT; 3846 + check_top_chunk(m, q); 3847 + } 3848 + else if (oldfirst == m->dv) { 3849 + size_t dsize = m->dvsize += qsize; 3850 + m->dv = q; 3851 + set_size_and_pinuse_of_free_chunk(q, dsize); 3852 + } 3853 + else { 3854 + if (!cinuse(oldfirst)) { 3855 + size_t nsize = chunksize(oldfirst); 3856 + unlink_chunk(m, oldfirst, nsize); 3857 + oldfirst = chunk_plus_offset(oldfirst, nsize); 3858 + qsize += nsize; 3859 + } 3860 + set_free_with_pinuse(q, qsize, oldfirst); 3861 + insert_chunk(m, q, qsize); 3862 + check_free_chunk(m, q); 3863 + } 3864 + 3865 + check_malloced_chunk(m, chunk2mem(p), nb); 3866 + return chunk2mem(p); 3867 + } 3868 + 3869 + /* Add a segment to hold a new noncontiguous region */ 3870 + static void add_segment(mstate m, char* tbase, size_t tsize, flag_t mmapped) { 3871 + /* Determine locations and sizes of segment, fenceposts, old top */ 3872 + char* old_top = (char*)m->top; 3873 + msegmentptr oldsp = segment_holding(m, old_top); 3874 + char* old_end = oldsp->base + oldsp->size; 3875 + size_t ssize = pad_request(sizeof(struct malloc_segment)); 3876 + char* rawsp = old_end - (ssize + FOUR_SIZE_T_SIZES + CHUNK_ALIGN_MASK); 3877 + size_t offset = align_offset(chunk2mem(rawsp)); 3878 + char* asp = rawsp + offset; 3879 + char* csp = (asp < (old_top + MIN_CHUNK_SIZE))? old_top : asp; 3880 + mchunkptr sp = (mchunkptr)csp; 3881 + msegmentptr ss = (msegmentptr)(chunk2mem(sp)); 3882 + mchunkptr tnext = chunk_plus_offset(sp, ssize); 3883 + mchunkptr p = tnext; 3884 + int nfences = 0; 3885 + 3886 + /* reset top to new space */ 3887 + init_top(m, (mchunkptr)tbase, tsize - TOP_FOOT_SIZE); 3888 + 3889 + /* Set up segment record */ 3890 + assert(is_aligned(ss)); 3891 + set_size_and_pinuse_of_inuse_chunk(m, sp, ssize); 3892 + *ss = m->seg; /* Push current record */ 3893 + m->seg.base = tbase; 3894 + m->seg.size = tsize; 3895 + m->seg.sflags = mmapped; 3896 + m->seg.next = ss; 3897 + 3898 + /* Insert trailing fenceposts */ 3899 + for (;;) { 3900 + mchunkptr nextp = chunk_plus_offset(p, SIZE_T_SIZE); 3901 + p->head = FENCEPOST_HEAD; 3902 + ++nfences; 3903 + if ((char*)(&(nextp->head)) < old_end) 3904 + p = nextp; 3905 + else 3906 + break; 3907 + } 3908 + assert(nfences >= 2); 3909 + 3910 + /* Insert the rest of old top into a bin as an ordinary free chunk */ 3911 + if (csp != old_top) { 3912 + mchunkptr q = (mchunkptr)old_top; 3913 + size_t psize = csp - old_top; 3914 + mchunkptr tn = chunk_plus_offset(q, psize); 3915 + set_free_with_pinuse(q, psize, tn); 3916 + insert_chunk(m, q, psize); 3917 + } 3918 + 3919 + check_top_chunk(m, m->top); 3920 + } 3921 + 3922 + /* -------------------------- System allocation -------------------------- */ 3923 + 3924 + /* Get memory from system using MORECORE or MMAP */ 3925 + static void* sys_alloc(mstate m, size_t nb) { 3926 + char* tbase = CMFAIL; 3927 + size_t tsize = 0; 3928 + flag_t mmap_flag = 0; 3929 + 3930 + ensure_initialization(); 3931 + 3932 + /* Directly map large chunks */ 3933 + if (use_mmap(m) && nb >= mparams.mmap_threshold) { 3934 + void* mem = mmap_alloc(m, nb); 3935 + if (mem != 0) 3936 + return mem; 3937 + } 3938 + 3939 + /* 3940 + Try getting memory in any of three ways (in most-preferred to 3941 + least-preferred order): 3942 + 1. A call to MORECORE that can normally contiguously extend memory. 3943 + (disabled if not MORECORE_CONTIGUOUS or not HAVE_MORECORE or 3944 + or main space is mmapped or a previous contiguous call failed) 3945 + 2. A call to MMAP new space (disabled if not HAVE_MMAP). 3946 + Note that under the default settings, if MORECORE is unable to 3947 + fulfill a request, and HAVE_MMAP is true, then mmap is 3948 + used as a noncontiguous system allocator. This is a useful backup 3949 + strategy for systems with holes in address spaces -- in this case 3950 + sbrk cannot contiguously expand the heap, but mmap may be able to 3951 + find space. 3952 + 3. A call to MORECORE that cannot usually contiguously extend memory. 3953 + (disabled if not HAVE_MORECORE) 3954 + 3955 + In all cases, we need to request enough bytes from system to ensure 3956 + we can malloc nb bytes upon success, so pad with enough space for 3957 + top_foot, plus alignment-pad to make sure we don't lose bytes if 3958 + not on boundary, and round this up to a granularity unit. 3959 + */ 3960 + 3961 + if (MORECORE_CONTIGUOUS && !use_noncontiguous(m)) { 3962 + char* br = CMFAIL; 3963 + msegmentptr ss = (m->top == 0)? 0 : segment_holding(m, (char*)m->top); 3964 + size_t asize = 0; 3965 + ACQUIRE_MALLOC_GLOBAL_LOCK(); 3966 + 3967 + if (ss == 0) { /* First time through or recovery */ 3968 + char* base = (char*)CALL_MORECORE(0); 3969 + if (base != CMFAIL) { 3970 + asize = granularity_align(nb + SYS_ALLOC_PADDING); 3971 + /* Adjust to end on a page boundary */ 3972 + if (!is_page_aligned(base)) 3973 + asize += (page_align((size_t)base) - (size_t)base); 3974 + /* Can't call MORECORE if size is negative when treated as signed */ 3975 + if (asize < HALF_MAX_SIZE_T && 3976 + (br = (char*)(CALL_MORECORE(asize))) == base) { 3977 + tbase = base; 3978 + tsize = asize; 3979 + } 3980 + } 3981 + } 3982 + else { 3983 + /* Subtract out existing available top space from MORECORE request. */ 3984 + asize = granularity_align(nb - m->topsize + SYS_ALLOC_PADDING); 3985 + /* Use mem here only if it did continuously extend old space */ 3986 + if (asize < HALF_MAX_SIZE_T && 3987 + (br = (char*)(CALL_MORECORE(asize))) == ss->base+ss->size) { 3988 + tbase = br; 3989 + tsize = asize; 3990 + } 3991 + } 3992 + 3993 + if (tbase == CMFAIL) { /* Cope with partial failure */ 3994 + if (br != CMFAIL) { /* Try to use/extend the space we did get */ 3995 + if (asize < HALF_MAX_SIZE_T && 3996 + asize < nb + SYS_ALLOC_PADDING) { 3997 + size_t esize = granularity_align(nb + SYS_ALLOC_PADDING - asize); 3998 + if (esize < HALF_MAX_SIZE_T) { 3999 + char* end = (char*)CALL_MORECORE(esize); 4000 + if (end != CMFAIL) 4001 + asize += esize; 4002 + else { /* Can't use; try to release */ 4003 + (void) CALL_MORECORE(-asize); 4004 + br = CMFAIL; 4005 + } 4006 + } 4007 + } 4008 + } 4009 + if (br != CMFAIL) { /* Use the space we did get */ 4010 + tbase = br; 4011 + tsize = asize; 4012 + } 4013 + else 4014 + disable_contiguous(m); /* Don't try contiguous path in the future */ 4015 + } 4016 + 4017 + RELEASE_MALLOC_GLOBAL_LOCK(); 4018 + } 4019 + 4020 + if (HAVE_MMAP && tbase == CMFAIL) { /* Try MMAP */ 4021 + size_t rsize = granularity_align(nb + SYS_ALLOC_PADDING); 4022 + if (rsize > nb) { /* Fail if wraps around zero */ 4023 + char* mp = (char*)(CALL_MMAP(rsize)); 4024 + if (mp != CMFAIL) { 4025 + tbase = mp; 4026 + tsize = rsize; 4027 + mmap_flag = IS_MMAPPED_BIT; 4028 + } 4029 + } 4030 + } 4031 + 4032 + if (HAVE_MORECORE && tbase == CMFAIL) { /* Try noncontiguous MORECORE */ 4033 + size_t asize = granularity_align(nb + SYS_ALLOC_PADDING); 4034 + if (asize < HALF_MAX_SIZE_T) { 4035 + char* br = CMFAIL; 4036 + char* end = CMFAIL; 4037 + ACQUIRE_MALLOC_GLOBAL_LOCK(); 4038 + br = (char*)(CALL_MORECORE(asize)); 4039 + end = (char*)(CALL_MORECORE(0)); 4040 + RELEASE_MALLOC_GLOBAL_LOCK(); 4041 + if (br != CMFAIL && end != CMFAIL && br < end) { 4042 + size_t ssize = end - br; 4043 + if (ssize > nb + TOP_FOOT_SIZE) { 4044 + tbase = br; 4045 + tsize = ssize; 4046 + } 4047 + } 4048 + } 4049 + } 4050 + 4051 + if (tbase != CMFAIL) { 4052 + 4053 + if ((m->footprint += tsize) > m->max_footprint) 4054 + m->max_footprint = m->footprint; 4055 + 4056 + if (!is_initialized(m)) { /* first-time initialization */ 4057 + m->seg.base = m->least_addr = tbase; 4058 + m->seg.size = tsize; 4059 + m->seg.sflags = mmap_flag; 4060 + m->magic = mparams.magic; 4061 + m->release_checks = MAX_RELEASE_CHECK_RATE; 4062 + init_bins(m); 4063 + #if !ONLY_MSPACES 4064 + if (is_global(m)) 4065 + init_top(m, (mchunkptr)tbase, tsize - TOP_FOOT_SIZE); 4066 + else 4067 + #endif 4068 + { 4069 + /* Offset top by embedded malloc_state */ 4070 + mchunkptr mn = next_chunk(mem2chunk(m)); 4071 + init_top(m, mn, (size_t)((tbase + tsize) - (char*)mn) -TOP_FOOT_SIZE); 4072 + } 4073 + } 4074 + 4075 + else { 4076 + /* Try to merge with an existing segment */ 4077 + msegmentptr sp = &m->seg; 4078 + /* Only consider most recent segment if traversal suppressed */ 4079 + while (sp != 0 && tbase != sp->base + sp->size) 4080 + sp = (NO_SEGMENT_TRAVERSAL) ? 0 : sp->next; 4081 + if (sp != 0 && 4082 + !is_extern_segment(sp) && 4083 + (sp->sflags & IS_MMAPPED_BIT) == mmap_flag && 4084 + segment_holds(sp, m->top)) { /* append */ 4085 + sp->size += tsize; 4086 + init_top(m, m->top, m->topsize + tsize); 4087 + } 4088 + else { 4089 + if (tbase < m->least_addr) 4090 + m->least_addr = tbase; 4091 + sp = &m->seg; 4092 + while (sp != 0 && sp->base != tbase + tsize) 4093 + sp = (NO_SEGMENT_TRAVERSAL) ? 0 : sp->next; 4094 + if (sp != 0 && 4095 + !is_extern_segment(sp) && 4096 + (sp->sflags & IS_MMAPPED_BIT) == mmap_flag) { 4097 + char* oldbase = sp->base; 4098 + sp->base = tbase; 4099 + sp->size += tsize; 4100 + return prepend_alloc(m, tbase, oldbase, nb); 4101 + } 4102 + else 4103 + add_segment(m, tbase, tsize, mmap_flag); 4104 + } 4105 + } 4106 + 4107 + if (nb < m->topsize) { /* Allocate from new or extended top space */ 4108 + size_t rsize = m->topsize -= nb; 4109 + mchunkptr p = m->top; 4110 + mchunkptr r = m->top = chunk_plus_offset(p, nb); 4111 + r->head = rsize | PINUSE_BIT; 4112 + set_size_and_pinuse_of_inuse_chunk(m, p, nb); 4113 + check_top_chunk(m, m->top); 4114 + check_malloced_chunk(m, chunk2mem(p), nb); 4115 + return chunk2mem(p); 4116 + } 4117 + } 4118 + 4119 + MALLOC_FAILURE_ACTION; 4120 + return 0; 4121 + } 4122 + 4123 + /* ----------------------- system deallocation -------------------------- */ 4124 + 4125 + /* Unmap and unlink any mmapped segments that don't contain used chunks */ 4126 + static size_t release_unused_segments(mstate m) { 4127 + size_t released = 0; 4128 + int nsegs = 0; 4129 + msegmentptr pred = &m->seg; 4130 + msegmentptr sp = pred->next; 4131 + while (sp != 0) { 4132 + char* base = sp->base; 4133 + size_t size = sp->size; 4134 + msegmentptr next = sp->next; 4135 + ++nsegs; 4136 + if (is_mmapped_segment(sp) && !is_extern_segment(sp)) { 4137 + mchunkptr p = align_as_chunk(base); 4138 + size_t psize = chunksize(p); 4139 + /* Can unmap if first chunk holds entire segment and not pinned */ 4140 + if (!cinuse(p) && (char*)p + psize >= base + size - TOP_FOOT_SIZE) { 4141 + tchunkptr tp = (tchunkptr)p; 4142 + assert(segment_holds(sp, (char*)sp)); 4143 + if (p == m->dv) { 4144 + m->dv = 0; 4145 + m->dvsize = 0; 4146 + } 4147 + else { 4148 + unlink_large_chunk(m, tp); 4149 + } 4150 + if (CALL_MUNMAP(base, size) == 0) { 4151 + released += size; 4152 + m->footprint -= size; 4153 + /* unlink obsoleted record */ 4154 + sp = pred; 4155 + sp->next = next; 4156 + } 4157 + else { /* back out if cannot unmap */ 4158 + insert_large_chunk(m, tp, psize); 4159 + } 4160 + } 4161 + } 4162 + if (NO_SEGMENT_TRAVERSAL) /* scan only first segment */ 4163 + break; 4164 + pred = sp; 4165 + sp = next; 4166 + } 4167 + /* Reset check counter */ 4168 + m->release_checks = ((nsegs > MAX_RELEASE_CHECK_RATE)? 4169 + nsegs : MAX_RELEASE_CHECK_RATE); 4170 + return released; 4171 + } 4172 + 4173 + static int sys_trim(mstate m, size_t pad) { 4174 + size_t released = 0; 4175 + ensure_initialization(); 4176 + if (pad < MAX_REQUEST && is_initialized(m)) { 4177 + pad += TOP_FOOT_SIZE; /* ensure enough room for segment overhead */ 4178 + 4179 + if (m->topsize > pad) { 4180 + /* Shrink top space in granularity-size units, keeping at least one */ 4181 + size_t unit = mparams.granularity; 4182 + size_t extra = ((m->topsize - pad + (unit - SIZE_T_ONE)) / unit - 4183 + SIZE_T_ONE) * unit; 4184 + msegmentptr sp = segment_holding(m, (char*)m->top); 4185 + 4186 + if (!is_extern_segment(sp)) { 4187 + if (is_mmapped_segment(sp)) { 4188 + if (HAVE_MMAP && 4189 + sp->size >= extra && 4190 + !has_segment_link(m, sp)) { /* can't shrink if pinned */ 4191 + size_t newsize = sp->size - extra; 4192 + /* Prefer mremap, fall back to munmap */ 4193 + if ((CALL_MREMAP(sp->base, sp->size, newsize, 0) != MFAIL) || 4194 + (CALL_MUNMAP(sp->base + newsize, extra) == 0)) { 4195 + released = extra; 4196 + } 4197 + } 4198 + } 4199 + else if (HAVE_MORECORE) { 4200 + if (extra >= HALF_MAX_SIZE_T) /* Avoid wrapping negative */ 4201 + extra = (HALF_MAX_SIZE_T) + SIZE_T_ONE - unit; 4202 + ACQUIRE_MALLOC_GLOBAL_LOCK(); 4203 + { 4204 + /* Make sure end of memory is where we last set it. */ 4205 + char* old_br = (char*)(CALL_MORECORE(0)); 4206 + if (old_br == sp->base + sp->size) { 4207 + char* rel_br = (char*)(CALL_MORECORE(-extra)); 4208 + char* new_br = (char*)(CALL_MORECORE(0)); 4209 + if (rel_br != CMFAIL && new_br < old_br) 4210 + released = old_br - new_br; 4211 + } 4212 + } 4213 + RELEASE_MALLOC_GLOBAL_LOCK(); 4214 + } 4215 + } 4216 + 4217 + if (released != 0) { 4218 + sp->size -= released; 4219 + m->footprint -= released; 4220 + init_top(m, m->top, m->topsize - released); 4221 + check_top_chunk(m, m->top); 4222 + } 4223 + } 4224 + 4225 + /* Unmap any unused mmapped segments */ 4226 + if (HAVE_MMAP) 4227 + released += release_unused_segments(m); 4228 + 4229 + /* On failure, disable autotrim to avoid repeated failed future calls */ 4230 + if (released == 0 && m->topsize > m->trim_check) 4231 + m->trim_check = MAX_SIZE_T; 4232 + } 4233 + 4234 + return (released != 0)? 1 : 0; 4235 + } 4236 + 4237 + 4238 + /* ---------------------------- malloc support --------------------------- */ 4239 + 4240 + /* allocate a large request from the best fitting chunk in a treebin */ 4241 + static void* tmalloc_large(mstate m, size_t nb) { 4242 + tchunkptr v = 0; 4243 + size_t rsize = -nb; /* Unsigned negation */ 4244 + tchunkptr t; 4245 + bindex_t idx; 4246 + compute_tree_index(nb, idx); 4247 + if ((t = *treebin_at(m, idx)) != 0) { 4248 + /* Traverse tree for this bin looking for node with size == nb */ 4249 + size_t sizebits = nb << leftshift_for_tree_index(idx); 4250 + tchunkptr rst = 0; /* The deepest untaken right subtree */ 4251 + for (;;) { 4252 + tchunkptr rt; 4253 + size_t trem = chunksize(t) - nb; 4254 + if (trem < rsize) { 4255 + v = t; 4256 + if ((rsize = trem) == 0) 4257 + break; 4258 + } 4259 + rt = t->child[1]; 4260 + t = t->child[(sizebits >> (SIZE_T_BITSIZE-SIZE_T_ONE)) & 1]; 4261 + if (rt != 0 && rt != t) 4262 + rst = rt; 4263 + if (t == 0) { 4264 + t = rst; /* set t to least subtree holding sizes > nb */ 4265 + break; 4266 + } 4267 + sizebits <<= 1; 4268 + } 4269 + } 4270 + if (t == 0 && v == 0) { /* set t to root of next non-empty treebin */ 4271 + binmap_t leftbits = left_bits(idx2bit(idx)) & m->treemap; 4272 + if (leftbits != 0) { 4273 + bindex_t i; 4274 + binmap_t leastbit = least_bit(leftbits); 4275 + compute_bit2idx(leastbit, i); 4276 + t = *treebin_at(m, i); 4277 + } 4278 + } 4279 + 4280 + while (t != 0) { /* find smallest of tree or subtree */ 4281 + size_t trem = chunksize(t) - nb; 4282 + if (trem < rsize) { 4283 + rsize = trem; 4284 + v = t; 4285 + } 4286 + t = leftmost_child(t); 4287 + } 4288 + 4289 + /* If dv is a better fit, return 0 so malloc will use it */ 4290 + if (v != 0 && rsize < (size_t)(m->dvsize - nb)) { 4291 + if (RTCHECK(ok_address(m, v))) { /* split */ 4292 + mchunkptr r = chunk_plus_offset(v, nb); 4293 + assert(chunksize(v) == rsize + nb); 4294 + if (RTCHECK(ok_next(v, r))) { 4295 + unlink_large_chunk(m, v); 4296 + if (rsize < MIN_CHUNK_SIZE) 4297 + set_inuse_and_pinuse(m, v, (rsize + nb)); 4298 + else { 4299 + set_size_and_pinuse_of_inuse_chunk(m, v, nb); 4300 + set_size_and_pinuse_of_free_chunk(r, rsize); 4301 + insert_chunk(m, r, rsize); 4302 + } 4303 + return chunk2mem(v); 4304 + } 4305 + } 4306 + CORRUPTION_ERROR_ACTION(m); 4307 + } 4308 + return 0; 4309 + } 4310 + 4311 + /* allocate a small request from the best fitting chunk in a treebin */ 4312 + static void* tmalloc_small(mstate m, size_t nb) { 4313 + tchunkptr t, v; 4314 + size_t rsize; 4315 + bindex_t i; 4316 + binmap_t leastbit = least_bit(m->treemap); 4317 + compute_bit2idx(leastbit, i); 4318 + v = t = *treebin_at(m, i); 4319 + rsize = chunksize(t) - nb; 4320 + 4321 + while ((t = leftmost_child(t)) != 0) { 4322 + size_t trem = chunksize(t) - nb; 4323 + if (trem < rsize) { 4324 + rsize = trem; 4325 + v = t; 4326 + } 4327 + } 4328 + 4329 + if (RTCHECK(ok_address(m, v))) { 4330 + mchunkptr r = chunk_plus_offset(v, nb); 4331 + assert(chunksize(v) == rsize + nb); 4332 + if (RTCHECK(ok_next(v, r))) { 4333 + unlink_large_chunk(m, v); 4334 + if (rsize < MIN_CHUNK_SIZE) 4335 + set_inuse_and_pinuse(m, v, (rsize + nb)); 4336 + else { 4337 + set_size_and_pinuse_of_inuse_chunk(m, v, nb); 4338 + set_size_and_pinuse_of_free_chunk(r, rsize); 4339 + replace_dv(m, r, rsize); 4340 + } 4341 + return chunk2mem(v); 4342 + } 4343 + } 4344 + 4345 + CORRUPTION_ERROR_ACTION(m); 4346 + return 0; 4347 + } 4348 + 4349 + /* --------------------------- realloc support --------------------------- */ 4350 + 4351 + static void* internal_realloc(mstate m, void* oldmem, size_t bytes) { 4352 + if (bytes >= MAX_REQUEST) { 4353 + MALLOC_FAILURE_ACTION; 4354 + return 0; 4355 + } 4356 + if (!PREACTION(m)) { 4357 + mchunkptr oldp = mem2chunk(oldmem); 4358 + size_t oldsize = chunksize(oldp); 4359 + mchunkptr next = chunk_plus_offset(oldp, oldsize); 4360 + mchunkptr newp = 0; 4361 + void* extra = 0; 4362 + 4363 + /* Try to either shrink or extend into top. Else malloc-copy-free */ 4364 + 4365 + if (RTCHECK(ok_address(m, oldp) && ok_cinuse(oldp) && 4366 + ok_next(oldp, next) && ok_pinuse(next))) { 4367 + size_t nb = request2size(bytes); 4368 + if (is_mmapped(oldp)) 4369 + newp = mmap_resize(m, oldp, nb); 4370 + else if (oldsize >= nb) { /* already big enough */ 4371 + size_t rsize = oldsize - nb; 4372 + newp = oldp; 4373 + if (rsize >= MIN_CHUNK_SIZE) { 4374 + mchunkptr remainder = chunk_plus_offset(newp, nb); 4375 + set_inuse(m, newp, nb); 4376 + set_inuse(m, remainder, rsize); 4377 + extra = chunk2mem(remainder); 4378 + } 4379 + } 4380 + else if (next == m->top && oldsize + m->topsize > nb) { 4381 + /* Expand into top */ 4382 + size_t newsize = oldsize + m->topsize; 4383 + size_t newtopsize = newsize - nb; 4384 + mchunkptr newtop = chunk_plus_offset(oldp, nb); 4385 + set_inuse(m, oldp, nb); 4386 + newtop->head = newtopsize |PINUSE_BIT; 4387 + m->top = newtop; 4388 + m->topsize = newtopsize; 4389 + newp = oldp; 4390 + } 4391 + } 4392 + else { 4393 + USAGE_ERROR_ACTION(m, oldmem); 4394 + POSTACTION(m); 4395 + return 0; 4396 + } 4397 + 4398 + POSTACTION(m); 4399 + 4400 + if (newp != 0) { 4401 + if (extra != 0) { 4402 + internal_free(m, extra); 4403 + } 4404 + check_inuse_chunk(m, newp); 4405 + return chunk2mem(newp); 4406 + } 4407 + else { 4408 + void* newmem = internal_malloc(m, bytes); 4409 + if (newmem != 0) { 4410 + size_t oc = oldsize - overhead_for(oldp); 4411 + memcpy(newmem, oldmem, (oc < bytes)? oc : bytes); 4412 + internal_free(m, oldmem); 4413 + } 4414 + return newmem; 4415 + } 4416 + } 4417 + return 0; 4418 + } 4419 + 4420 + /* --------------------------- memalign support -------------------------- */ 4421 + 4422 + static void* internal_memalign(mstate m, size_t alignment, size_t bytes) { 4423 + if (alignment <= MALLOC_ALIGNMENT) /* Can just use malloc */ 4424 + return internal_malloc(m, bytes); 4425 + if (alignment < MIN_CHUNK_SIZE) /* must be at least a minimum chunk size */ 4426 + alignment = MIN_CHUNK_SIZE; 4427 + if ((alignment & (alignment-SIZE_T_ONE)) != 0) {/* Ensure a power of 2 */ 4428 + size_t a = MALLOC_ALIGNMENT << 1; 4429 + while (a < alignment) a <<= 1; 4430 + alignment = a; 4431 + } 4432 + 4433 + if (bytes >= MAX_REQUEST - alignment) { 4434 + if (m != 0) { /* Test isn't needed but avoids compiler warning */ 4435 + MALLOC_FAILURE_ACTION; 4436 + } 4437 + } 4438 + else { 4439 + size_t nb = request2size(bytes); 4440 + size_t req = nb + alignment + MIN_CHUNK_SIZE - CHUNK_OVERHEAD; 4441 + char* mem = (char*)internal_malloc(m, req); 4442 + if (mem != 0) { 4443 + void* leader = 0; 4444 + void* trailer = 0; 4445 + mchunkptr p = mem2chunk(mem); 4446 + 4447 + if (PREACTION(m)) return 0; 4448 + if ((((size_t)(mem)) % alignment) != 0) { /* misaligned */ 4449 + /* 4450 + Find an aligned spot inside chunk. Since we need to give 4451 + back leading space in a chunk of at least MIN_CHUNK_SIZE, if 4452 + the first calculation places us at a spot with less than 4453 + MIN_CHUNK_SIZE leader, we can move to the next aligned spot. 4454 + We've allocated enough total room so that this is always 4455 + possible. 4456 + */ 4457 + char* br = (char*)mem2chunk((size_t)(((size_t)(mem + 4458 + alignment - 4459 + SIZE_T_ONE)) & 4460 + -alignment)); 4461 + char* pos = ((size_t)(br - (char*)(p)) >= MIN_CHUNK_SIZE)? 4462 + br : br+alignment; 4463 + mchunkptr newp = (mchunkptr)pos; 4464 + size_t leadsize = pos - (char*)(p); 4465 + size_t newsize = chunksize(p) - leadsize; 4466 + 4467 + if (is_mmapped(p)) { /* For mmapped chunks, just adjust offset */ 4468 + newp->prev_foot = p->prev_foot + leadsize; 4469 + newp->head = (newsize|CINUSE_BIT); 4470 + } 4471 + else { /* Otherwise, give back leader, use the rest */ 4472 + set_inuse(m, newp, newsize); 4473 + set_inuse(m, p, leadsize); 4474 + leader = chunk2mem(p); 4475 + } 4476 + p = newp; 4477 + } 4478 + 4479 + /* Give back spare room at the end */ 4480 + if (!is_mmapped(p)) { 4481 + size_t size = chunksize(p); 4482 + if (size > nb + MIN_CHUNK_SIZE) { 4483 + size_t remainder_size = size - nb; 4484 + mchunkptr remainder = chunk_plus_offset(p, nb); 4485 + set_inuse(m, p, nb); 4486 + set_inuse(m, remainder, remainder_size); 4487 + trailer = chunk2mem(remainder); 4488 + } 4489 + } 4490 + 4491 + assert (chunksize(p) >= nb); 4492 + assert((((size_t)(chunk2mem(p))) % alignment) == 0); 4493 + check_inuse_chunk(m, p); 4494 + POSTACTION(m); 4495 + if (leader != 0) { 4496 + internal_free(m, leader); 4497 + } 4498 + if (trailer != 0) { 4499 + internal_free(m, trailer); 4500 + } 4501 + return chunk2mem(p); 4502 + } 4503 + } 4504 + return 0; 4505 + } 4506 + 4507 + /* ------------------------ comalloc/coalloc support --------------------- */ 4508 + 4509 + static void** ialloc(mstate m, 4510 + size_t n_elements, 4511 + size_t* sizes, 4512 + int opts, 4513 + void* chunks[]) { 4514 + /* 4515 + This provides common support for independent_X routines, handling 4516 + all of the combinations that can result. 4517 + 4518 + The opts arg has: 4519 + bit 0 set if all elements are same size (using sizes[0]) 4520 + bit 1 set if elements should be zeroed 4521 + */ 4522 + 4523 + size_t element_size; /* chunksize of each element, if all same */ 4524 + size_t contents_size; /* total size of elements */ 4525 + size_t array_size; /* request size of pointer array */ 4526 + void* mem; /* malloced aggregate space */ 4527 + mchunkptr p; /* corresponding chunk */ 4528 + size_t remainder_size; /* remaining bytes while splitting */ 4529 + void** marray; /* either "chunks" or malloced ptr array */ 4530 + mchunkptr array_chunk; /* chunk for malloced ptr array */ 4531 + flag_t was_enabled; /* to disable mmap */ 4532 + size_t size; 4533 + size_t i; 4534 + 4535 + ensure_initialization(); 4536 + /* compute array length, if needed */ 4537 + if (chunks != 0) { 4538 + if (n_elements == 0) 4539 + return chunks; /* nothing to do */ 4540 + marray = chunks; 4541 + array_size = 0; 4542 + } 4543 + else { 4544 + /* if empty req, must still return chunk representing empty array */ 4545 + if (n_elements == 0) 4546 + return (void**)internal_malloc(m, 0); 4547 + marray = 0; 4548 + array_size = request2size(n_elements * (sizeof(void*))); 4549 + } 4550 + 4551 + /* compute total element size */ 4552 + if (opts & 0x1) { /* all-same-size */ 4553 + element_size = request2size(*sizes); 4554 + contents_size = n_elements * element_size; 4555 + } 4556 + else { /* add up all the sizes */ 4557 + element_size = 0; 4558 + contents_size = 0; 4559 + for (i = 0; i != n_elements; ++i) 4560 + contents_size += request2size(sizes[i]); 4561 + } 4562 + 4563 + size = contents_size + array_size; 4564 + 4565 + /* 4566 + Allocate the aggregate chunk. First disable direct-mmapping so 4567 + malloc won't use it, since we would not be able to later 4568 + free/realloc space internal to a segregated mmap region. 4569 + */ 4570 + was_enabled = use_mmap(m); 4571 + disable_mmap(m); 4572 + mem = internal_malloc(m, size - CHUNK_OVERHEAD); 4573 + if (was_enabled) 4574 + enable_mmap(m); 4575 + if (mem == 0) 4576 + return 0; 4577 + 4578 + if (PREACTION(m)) return 0; 4579 + p = mem2chunk(mem); 4580 + remainder_size = chunksize(p); 4581 + 4582 + assert(!is_mmapped(p)); 4583 + 4584 + if (opts & 0x2) { /* optionally clear the elements */ 4585 + memset((size_t*)mem, 0, remainder_size - SIZE_T_SIZE - array_size); 4586 + } 4587 + 4588 + /* If not provided, allocate the pointer array as final part of chunk */ 4589 + if (marray == 0) { 4590 + size_t array_chunk_size; 4591 + array_chunk = chunk_plus_offset(p, contents_size); 4592 + array_chunk_size = remainder_size - contents_size; 4593 + marray = (void**) (chunk2mem(array_chunk)); 4594 + set_size_and_pinuse_of_inuse_chunk(m, array_chunk, array_chunk_size); 4595 + remainder_size = contents_size; 4596 + } 4597 + 4598 + /* split out elements */ 4599 + for (i = 0; ; ++i) { 4600 + marray[i] = chunk2mem(p); 4601 + if (i != n_elements-1) { 4602 + if (element_size != 0) 4603 + size = element_size; 4604 + else 4605 + size = request2size(sizes[i]); 4606 + remainder_size -= size; 4607 + set_size_and_pinuse_of_inuse_chunk(m, p, size); 4608 + p = chunk_plus_offset(p, size); 4609 + } 4610 + else { /* the final element absorbs any overallocation slop */ 4611 + set_size_and_pinuse_of_inuse_chunk(m, p, remainder_size); 4612 + break; 4613 + } 4614 + } 4615 + 4616 + #if DEBUG 4617 + if (marray != chunks) { 4618 + /* final element must have exactly exhausted chunk */ 4619 + if (element_size != 0) { 4620 + assert(remainder_size == element_size); 4621 + } 4622 + else { 4623 + assert(remainder_size == request2size(sizes[i])); 4624 + } 4625 + check_inuse_chunk(m, mem2chunk(marray)); 4626 + } 4627 + for (i = 0; i != n_elements; ++i) 4628 + check_inuse_chunk(m, mem2chunk(marray[i])); 4629 + 4630 + #endif /* DEBUG */ 4631 + 4632 + POSTACTION(m); 4633 + return marray; 4634 + } 4635 + 4636 + 4637 + /* -------------------------- public routines ---------------------------- */ 4638 + 4639 + #if !ONLY_MSPACES 4640 + 4641 + void* dlmalloc(size_t bytes) { 4642 + /* 4643 + Basic algorithm: 4644 + If a small request (< 256 bytes minus per-chunk overhead): 4645 + 1. If one exists, use a remainderless chunk in associated smallbin. 4646 + (Remainderless means that there are too few excess bytes to 4647 + represent as a chunk.) 4648 + 2. If it is big enough, use the dv chunk, which is normally the 4649 + chunk adjacent to the one used for the most recent small request. 4650 + 3. If one exists, split the smallest available chunk in a bin, 4651 + saving remainder in dv. 4652 + 4. If it is big enough, use the top chunk. 4653 + 5. If available, get memory from system and use it 4654 + Otherwise, for a large request: 4655 + 1. Find the smallest available binned chunk that fits, and use it 4656 + if it is better fitting than dv chunk, splitting if necessary. 4657 + 2. If better fitting than any binned chunk, use the dv chunk. 4658 + 3. If it is big enough, use the top chunk. 4659 + 4. If request size >= mmap threshold, try to directly mmap this chunk. 4660 + 5. If available, get memory from system and use it 4661 + 4662 + The ugly goto's here ensure that postaction occurs along all paths. 4663 + */ 4664 + 4665 + #if USE_LOCKS 4666 + ensure_initialization(); /* initialize in sys_alloc if not using locks */ 4667 + #endif 4668 + 4669 + if (!PREACTION(gm)) { 4670 + void* mem; 4671 + size_t nb; 4672 + if (bytes <= MAX_SMALL_REQUEST) { 4673 + bindex_t idx; 4674 + binmap_t smallbits; 4675 + nb = (bytes < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(bytes); 4676 + idx = small_index(nb); 4677 + smallbits = gm->smallmap >> idx; 4678 + 4679 + if ((smallbits & 0x3U) != 0) { /* Remainderless fit to a smallbin. */ 4680 + mchunkptr b, p; 4681 + idx += ~smallbits & 1; /* Uses next bin if idx empty */ 4682 + b = smallbin_at(gm, idx); 4683 + p = b->fd; 4684 + assert(chunksize(p) == small_index2size(idx)); 4685 + unlink_first_small_chunk(gm, b, p, idx); 4686 + set_inuse_and_pinuse(gm, p, small_index2size(idx)); 4687 + mem = chunk2mem(p); 4688 + check_malloced_chunk(gm, mem, nb); 4689 + goto postaction; 4690 + } 4691 + 4692 + else if (nb > gm->dvsize) { 4693 + if (smallbits != 0) { /* Use chunk in next nonempty smallbin */ 4694 + mchunkptr b, p, r; 4695 + size_t rsize; 4696 + bindex_t i; 4697 + binmap_t leftbits = (smallbits << idx) & left_bits(idx2bit(idx)); 4698 + binmap_t leastbit = least_bit(leftbits); 4699 + compute_bit2idx(leastbit, i); 4700 + b = smallbin_at(gm, i); 4701 + p = b->fd; 4702 + assert(chunksize(p) == small_index2size(i)); 4703 + unlink_first_small_chunk(gm, b, p, i); 4704 + rsize = small_index2size(i) - nb; 4705 + /* Fit here cannot be remainderless if 4byte sizes */ 4706 + if (SIZE_T_SIZE != 4 && rsize < MIN_CHUNK_SIZE) 4707 + set_inuse_and_pinuse(gm, p, small_index2size(i)); 4708 + else { 4709 + set_size_and_pinuse_of_inuse_chunk(gm, p, nb); 4710 + r = chunk_plus_offset(p, nb); 4711 + set_size_and_pinuse_of_free_chunk(r, rsize); 4712 + replace_dv(gm, r, rsize); 4713 + } 4714 + mem = chunk2mem(p); 4715 + check_malloced_chunk(gm, mem, nb); 4716 + goto postaction; 4717 + } 4718 + 4719 + else if (gm->treemap != 0 && (mem = tmalloc_small(gm, nb)) != 0) { 4720 + check_malloced_chunk(gm, mem, nb); 4721 + goto postaction; 4722 + } 4723 + } 4724 + } 4725 + else if (bytes >= MAX_REQUEST) 4726 + nb = MAX_SIZE_T; /* Too big to allocate. Force failure (in sys alloc) */ 4727 + else { 4728 + nb = pad_request(bytes); 4729 + if (gm->treemap != 0 && (mem = tmalloc_large(gm, nb)) != 0) { 4730 + check_malloced_chunk(gm, mem, nb); 4731 + goto postaction; 4732 + } 4733 + } 4734 + 4735 + if (nb <= gm->dvsize) { 4736 + size_t rsize = gm->dvsize - nb; 4737 + mchunkptr p = gm->dv; 4738 + if (rsize >= MIN_CHUNK_SIZE) { /* split dv */ 4739 + mchunkptr r = gm->dv = chunk_plus_offset(p, nb); 4740 + gm->dvsize = rsize; 4741 + set_size_and_pinuse_of_free_chunk(r, rsize); 4742 + set_size_and_pinuse_of_inuse_chunk(gm, p, nb); 4743 + } 4744 + else { /* exhaust dv */ 4745 + size_t dvs = gm->dvsize; 4746 + gm->dvsize = 0; 4747 + gm->dv = 0; 4748 + set_inuse_and_pinuse(gm, p, dvs); 4749 + } 4750 + mem = chunk2mem(p); 4751 + check_malloced_chunk(gm, mem, nb); 4752 + goto postaction; 4753 + } 4754 + 4755 + else if (nb < gm->topsize) { /* Split top */ 4756 + size_t rsize = gm->topsize -= nb; 4757 + mchunkptr p = gm->top; 4758 + mchunkptr r = gm->top = chunk_plus_offset(p, nb); 4759 + r->head = rsize | PINUSE_BIT; 4760 + set_size_and_pinuse_of_inuse_chunk(gm, p, nb); 4761 + mem = chunk2mem(p); 4762 + check_top_chunk(gm, gm->top); 4763 + check_malloced_chunk(gm, mem, nb); 4764 + goto postaction; 4765 + } 4766 + 4767 + mem = sys_alloc(gm, nb); 4768 + 4769 + postaction: 4770 + POSTACTION(gm); 4771 + return mem; 4772 + } 4773 + 4774 + return 0; 4775 + } 4776 + 4777 + void dlfree(void* mem) { 4778 + /* 4779 + Consolidate freed chunks with preceeding or succeeding bordering 4780 + free chunks, if they exist, and then place in a bin. Intermixed 4781 + with special cases for top, dv, mmapped chunks, and usage errors. 4782 + */ 4783 + 4784 + if (mem != 0) { 4785 + mchunkptr p = mem2chunk(mem); 4786 + #if FOOTERS 4787 + mstate fm = get_mstate_for(p); 4788 + if (!ok_magic(fm)) { 4789 + USAGE_ERROR_ACTION(fm, p); 4790 + return; 4791 + } 4792 + #else /* FOOTERS */ 4793 + #define fm gm 4794 + #endif /* FOOTERS */ 4795 + if (!PREACTION(fm)) { 4796 + check_inuse_chunk(fm, p); 4797 + if (RTCHECK(ok_address(fm, p) && ok_cinuse(p))) { 4798 + size_t psize = chunksize(p); 4799 + mchunkptr next = chunk_plus_offset(p, psize); 4800 + if (!pinuse(p)) { 4801 + size_t prevsize = p->prev_foot; 4802 + if ((prevsize & IS_MMAPPED_BIT) != 0) { 4803 + prevsize &= ~IS_MMAPPED_BIT; 4804 + psize += prevsize + MMAP_FOOT_PAD; 4805 + if (CALL_MUNMAP((char*)p - prevsize, psize) == 0) 4806 + fm->footprint -= psize; 4807 + goto postaction; 4808 + } 4809 + else { 4810 + mchunkptr prev = chunk_minus_offset(p, prevsize); 4811 + psize += prevsize; 4812 + p = prev; 4813 + if (RTCHECK(ok_address(fm, prev))) { /* consolidate backward */ 4814 + if (p != fm->dv) { 4815 + unlink_chunk(fm, p, prevsize); 4816 + } 4817 + else if ((next->head & INUSE_BITS) == INUSE_BITS) { 4818 + fm->dvsize = psize; 4819 + set_free_with_pinuse(p, psize, next); 4820 + goto postaction; 4821 + } 4822 + } 4823 + else 4824 + goto erroraction; 4825 + } 4826 + } 4827 + 4828 + if (RTCHECK(ok_next(p, next) && ok_pinuse(next))) { 4829 + if (!cinuse(next)) { /* consolidate forward */ 4830 + if (next == fm->top) { 4831 + size_t tsize = fm->topsize += psize; 4832 + fm->top = p; 4833 + p->head = tsize | PINUSE_BIT; 4834 + if (p == fm->dv) { 4835 + fm->dv = 0; 4836 + fm->dvsize = 0; 4837 + } 4838 + if (should_trim(fm, tsize)) 4839 + sys_trim(fm, 0); 4840 + goto postaction; 4841 + } 4842 + else if (next == fm->dv) { 4843 + size_t dsize = fm->dvsize += psize; 4844 + fm->dv = p; 4845 + set_size_and_pinuse_of_free_chunk(p, dsize); 4846 + goto postaction; 4847 + } 4848 + else { 4849 + size_t nsize = chunksize(next); 4850 + psize += nsize; 4851 + unlink_chunk(fm, next, nsize); 4852 + set_size_and_pinuse_of_free_chunk(p, psize); 4853 + if (p == fm->dv) { 4854 + fm->dvsize = psize; 4855 + goto postaction; 4856 + } 4857 + } 4858 + } 4859 + else 4860 + set_free_with_pinuse(p, psize, next); 4861 + 4862 + if (is_small(psize)) { 4863 + insert_small_chunk(fm, p, psize); 4864 + check_free_chunk(fm, p); 4865 + } 4866 + else { 4867 + tchunkptr tp = (tchunkptr)p; 4868 + insert_large_chunk(fm, tp, psize); 4869 + check_free_chunk(fm, p); 4870 + if (--fm->release_checks == 0) 4871 + release_unused_segments(fm); 4872 + } 4873 + goto postaction; 4874 + } 4875 + } 4876 + erroraction: 4877 + USAGE_ERROR_ACTION(fm, p); 4878 + postaction: 4879 + POSTACTION(fm); 4880 + } 4881 + } 4882 + #if !FOOTERS 4883 + #undef fm 4884 + #endif /* FOOTERS */ 4885 + } 4886 + 4887 + void* dlcalloc(size_t n_elements, size_t elem_size) { 4888 + void* mem; 4889 + size_t req = 0; 4890 + if (n_elements != 0) { 4891 + req = n_elements * elem_size; 4892 + if (((n_elements | elem_size) & ~(size_t)0xffff) && 4893 + (req / n_elements != elem_size)) 4894 + req = MAX_SIZE_T; /* force downstream failure on overflow */ 4895 + } 4896 + mem = dlmalloc(req); 4897 + if (mem != 0 && calloc_must_clear(mem2chunk(mem))) 4898 + memset(mem, 0, req); 4899 + return mem; 4900 + } 4901 + 4902 + void* dlrealloc(void* oldmem, size_t bytes) { 4903 + if (oldmem == 0) 4904 + return dlmalloc(bytes); 4905 + #ifdef REALLOC_ZERO_BYTES_FREES 4906 + if (bytes == 0) { 4907 + dlfree(oldmem); 4908 + return 0; 4909 + } 4910 + #endif /* REALLOC_ZERO_BYTES_FREES */ 4911 + else { 4912 + #if ! FOOTERS 4913 + mstate m = gm; 4914 + #else /* FOOTERS */ 4915 + mstate m = get_mstate_for(mem2chunk(oldmem)); 4916 + if (!ok_magic(m)) { 4917 + USAGE_ERROR_ACTION(m, oldmem); 4918 + return 0; 4919 + } 4920 + #endif /* FOOTERS */ 4921 + return internal_realloc(m, oldmem, bytes); 4922 + } 4923 + } 4924 + 4925 + void* dlmemalign(size_t alignment, size_t bytes) { 4926 + return internal_memalign(gm, alignment, bytes); 4927 + } 4928 + 4929 + void** dlindependent_calloc(size_t n_elements, size_t elem_size, 4930 + void* chunks[]) { 4931 + size_t sz = elem_size; /* serves as 1-element array */ 4932 + return ialloc(gm, n_elements, &sz, 3, chunks); 4933 + } 4934 + 4935 + void** dlindependent_comalloc(size_t n_elements, size_t sizes[], 4936 + void* chunks[]) { 4937 + return ialloc(gm, n_elements, sizes, 0, chunks); 4938 + } 4939 + 4940 + void* dlvalloc(size_t bytes) { 4941 + size_t pagesz; 4942 + ensure_initialization(); 4943 + pagesz = mparams.page_size; 4944 + return dlmemalign(pagesz, bytes); 4945 + } 4946 + 4947 + void* dlpvalloc(size_t bytes) { 4948 + size_t pagesz; 4949 + ensure_initialization(); 4950 + pagesz = mparams.page_size; 4951 + return dlmemalign(pagesz, (bytes + pagesz - SIZE_T_ONE) & ~(pagesz - SIZE_T_ONE)); 4952 + } 4953 + 4954 + int dlmalloc_trim(size_t pad) { 4955 + ensure_initialization(); 4956 + int result = 0; 4957 + if (!PREACTION(gm)) { 4958 + result = sys_trim(gm, pad); 4959 + POSTACTION(gm); 4960 + } 4961 + return result; 4962 + } 4963 + 4964 + size_t dlmalloc_footprint(void) { 4965 + return gm->footprint; 4966 + } 4967 + 4968 + size_t dlmalloc_max_footprint(void) { 4969 + return gm->max_footprint; 4970 + } 4971 + 4972 + #if !NO_MALLINFO 4973 + struct mallinfo dlmallinfo(void) { 4974 + return internal_mallinfo(gm); 4975 + } 4976 + #endif /* NO_MALLINFO */ 4977 + 4978 + void dlmalloc_stats() { 4979 + internal_malloc_stats(gm); 4980 + } 4981 + 4982 + int dlmallopt(int param_number, int value) { 4983 + return change_mparam(param_number, value); 4984 + } 4985 + 4986 + #endif /* !ONLY_MSPACES */ 4987 + 4988 + size_t dlmalloc_usable_size(void* mem) { 4989 + if (mem != 0) { 4990 + mchunkptr p = mem2chunk(mem); 4991 + if (cinuse(p)) 4992 + return chunksize(p) - overhead_for(p); 4993 + } 4994 + return 0; 4995 + } 4996 + 4997 + /* ----------------------------- user mspaces ---------------------------- */ 4998 + 4999 + #if MSPACES 5000 + 5001 + static mstate init_user_mstate(char* tbase, size_t tsize) { 5002 + size_t msize = pad_request(sizeof(struct malloc_state)); 5003 + mchunkptr mn; 5004 + mchunkptr msp = align_as_chunk(tbase); 5005 + mstate m = (mstate)(chunk2mem(msp)); 5006 + memset(m, 0, msize); 5007 + INITIAL_LOCK(&m->mutex); 5008 + msp->head = (msize|PINUSE_BIT|CINUSE_BIT); 5009 + m->seg.base = m->least_addr = tbase; 5010 + m->seg.size = m->footprint = m->max_footprint = tsize; 5011 + m->magic = mparams.magic; 5012 + m->release_checks = MAX_RELEASE_CHECK_RATE; 5013 + m->mflags = mparams.default_mflags; 5014 + m->extp = 0; 5015 + m->exts = 0; 5016 + disable_contiguous(m); 5017 + init_bins(m); 5018 + mn = next_chunk(mem2chunk(m)); 5019 + init_top(m, mn, (size_t)((tbase + tsize) - (char*)mn) - TOP_FOOT_SIZE); 5020 + check_top_chunk(m, m->top); 5021 + return m; 5022 + } 5023 + 5024 + mspace create_mspace(size_t capacity, int locked) { 5025 + mstate m = 0; 5026 + size_t msize; 5027 + ensure_initialization(); 5028 + msize = pad_request(sizeof(struct malloc_state)); 5029 + if (capacity < (size_t) -(msize + TOP_FOOT_SIZE + mparams.page_size)) { 5030 + size_t rs = ((capacity == 0)? mparams.granularity : 5031 + (capacity + TOP_FOOT_SIZE + msize)); 5032 + size_t tsize = granularity_align(rs); 5033 + char* tbase = (char*)(CALL_MMAP(tsize)); 5034 + if (tbase != CMFAIL) { 5035 + m = init_user_mstate(tbase, tsize); 5036 + m->seg.sflags = IS_MMAPPED_BIT; 5037 + set_lock(m, locked); 5038 + } 5039 + } 5040 + return (mspace)m; 5041 + } 5042 + 5043 + mspace create_mspace_with_base(void* base, size_t capacity, int locked) { 5044 + mstate m = 0; 5045 + size_t msize; 5046 + ensure_initialization(); 5047 + msize = pad_request(sizeof(struct malloc_state)); 5048 + if (capacity > msize + TOP_FOOT_SIZE && 5049 + capacity < (size_t) -(msize + TOP_FOOT_SIZE + mparams.page_size)) { 5050 + m = init_user_mstate((char*)base, capacity); 5051 + m->seg.sflags = EXTERN_BIT; 5052 + set_lock(m, locked); 5053 + } 5054 + return (mspace)m; 5055 + } 5056 + 5057 + int mspace_mmap_large_chunks(mspace msp, int enable) { 5058 + int ret = 0; 5059 + mstate ms = (mstate)msp; 5060 + if (!PREACTION(ms)) { 5061 + if (use_mmap(ms)) 5062 + ret = 1; 5063 + if (enable) 5064 + enable_mmap(ms); 5065 + else 5066 + disable_mmap(ms); 5067 + POSTACTION(ms); 5068 + } 5069 + return ret; 5070 + } 5071 + 5072 + size_t destroy_mspace(mspace msp) { 5073 + size_t freed = 0; 5074 + mstate ms = (mstate)msp; 5075 + if (ok_magic(ms)) { 5076 + msegmentptr sp = &ms->seg; 5077 + while (sp != 0) { 5078 + char* base = sp->base; 5079 + size_t size = sp->size; 5080 + flag_t flag = sp->sflags; 5081 + sp = sp->next; 5082 + if ((flag & IS_MMAPPED_BIT) && !(flag & EXTERN_BIT) && 5083 + CALL_MUNMAP(base, size) == 0) 5084 + freed += size; 5085 + } 5086 + } 5087 + else { 5088 + USAGE_ERROR_ACTION(ms,ms); 5089 + } 5090 + return freed; 5091 + } 5092 + 5093 + /* 5094 + mspace versions of routines are near-clones of the global 5095 + versions. This is not so nice but better than the alternatives. 5096 + */ 5097 + 5098 + 5099 + void* mspace_malloc(mspace msp, size_t bytes) { 5100 + mstate ms = (mstate)msp; 5101 + if (!ok_magic(ms)) { 5102 + USAGE_ERROR_ACTION(ms,ms); 5103 + return 0; 5104 + } 5105 + if (!PREACTION(ms)) { 5106 + void* mem; 5107 + size_t nb; 5108 + if (bytes <= MAX_SMALL_REQUEST) { 5109 + bindex_t idx; 5110 + binmap_t smallbits; 5111 + nb = (bytes < MIN_REQUEST)? MIN_CHUNK_SIZE : pad_request(bytes); 5112 + idx = small_index(nb); 5113 + smallbits = ms->smallmap >> idx; 5114 + 5115 + if ((smallbits & 0x3U) != 0) { /* Remainderless fit to a smallbin. */ 5116 + mchunkptr b, p; 5117 + idx += ~smallbits & 1; /* Uses next bin if idx empty */ 5118 + b = smallbin_at(ms, idx); 5119 + p = b->fd; 5120 + assert(chunksize(p) == small_index2size(idx)); 5121 + unlink_first_small_chunk(ms, b, p, idx); 5122 + set_inuse_and_pinuse(ms, p, small_index2size(idx)); 5123 + mem = chunk2mem(p); 5124 + check_malloced_chunk(ms, mem, nb); 5125 + goto postaction; 5126 + } 5127 + 5128 + else if (nb > ms->dvsize) { 5129 + if (smallbits != 0) { /* Use chunk in next nonempty smallbin */ 5130 + mchunkptr b, p, r; 5131 + size_t rsize; 5132 + bindex_t i; 5133 + binmap_t leftbits = (smallbits << idx) & left_bits(idx2bit(idx)); 5134 + binmap_t leastbit = least_bit(leftbits); 5135 + compute_bit2idx(leastbit, i); 5136 + b = smallbin_at(ms, i); 5137 + p = b->fd; 5138 + assert(chunksize(p) == small_index2size(i)); 5139 + unlink_first_small_chunk(ms, b, p, i); 5140 + rsize = small_index2size(i) - nb; 5141 + /* Fit here cannot be remainderless if 4byte sizes */ 5142 + if (SIZE_T_SIZE != 4 && rsize < MIN_CHUNK_SIZE) 5143 + set_inuse_and_pinuse(ms, p, small_index2size(i)); 5144 + else { 5145 + set_size_and_pinuse_of_inuse_chunk(ms, p, nb); 5146 + r = chunk_plus_offset(p, nb); 5147 + set_size_and_pinuse_of_free_chunk(r, rsize); 5148 + replace_dv(ms, r, rsize); 5149 + } 5150 + mem = chunk2mem(p); 5151 + check_malloced_chunk(ms, mem, nb); 5152 + goto postaction; 5153 + } 5154 + 5155 + else if (ms->treemap != 0 && (mem = tmalloc_small(ms, nb)) != 0) { 5156 + check_malloced_chunk(ms, mem, nb); 5157 + goto postaction; 5158 + } 5159 + } 5160 + } 5161 + else if (bytes >= MAX_REQUEST) 5162 + nb = MAX_SIZE_T; /* Too big to allocate. Force failure (in sys alloc) */ 5163 + else { 5164 + nb = pad_request(bytes); 5165 + if (ms->treemap != 0 && (mem = tmalloc_large(ms, nb)) != 0) { 5166 + check_malloced_chunk(ms, mem, nb); 5167 + goto postaction; 5168 + } 5169 + } 5170 + 5171 + if (nb <= ms->dvsize) { 5172 + size_t rsize = ms->dvsize - nb; 5173 + mchunkptr p = ms->dv; 5174 + if (rsize >= MIN_CHUNK_SIZE) { /* split dv */ 5175 + mchunkptr r = ms->dv = chunk_plus_offset(p, nb); 5176 + ms->dvsize = rsize; 5177 + set_size_and_pinuse_of_free_chunk(r, rsize); 5178 + set_size_and_pinuse_of_inuse_chunk(ms, p, nb); 5179 + } 5180 + else { /* exhaust dv */ 5181 + size_t dvs = ms->dvsize; 5182 + ms->dvsize = 0; 5183 + ms->dv = 0; 5184 + set_inuse_and_pinuse(ms, p, dvs); 5185 + } 5186 + mem = chunk2mem(p); 5187 + check_malloced_chunk(ms, mem, nb); 5188 + goto postaction; 5189 + } 5190 + 5191 + else if (nb < ms->topsize) { /* Split top */ 5192 + size_t rsize = ms->topsize -= nb; 5193 + mchunkptr p = ms->top; 5194 + mchunkptr r = ms->top = chunk_plus_offset(p, nb); 5195 + r->head = rsize | PINUSE_BIT; 5196 + set_size_and_pinuse_of_inuse_chunk(ms, p, nb); 5197 + mem = chunk2mem(p); 5198 + check_top_chunk(ms, ms->top); 5199 + check_malloced_chunk(ms, mem, nb); 5200 + goto postaction; 5201 + } 5202 + 5203 + mem = sys_alloc(ms, nb); 5204 + 5205 + postaction: 5206 + POSTACTION(ms); 5207 + return mem; 5208 + } 5209 + 5210 + return 0; 5211 + } 5212 + 5213 + void mspace_free(mspace msp, void* mem) { 5214 + if (mem != 0) { 5215 + mchunkptr p = mem2chunk(mem); 5216 + #if FOOTERS 5217 + mstate fm = get_mstate_for(p); 5218 + #else /* FOOTERS */ 5219 + mstate fm = (mstate)msp; 5220 + #endif /* FOOTERS */ 5221 + if (!ok_magic(fm)) { 5222 + USAGE_ERROR_ACTION(fm, p); 5223 + return; 5224 + } 5225 + if (!PREACTION(fm)) { 5226 + check_inuse_chunk(fm, p); 5227 + if (RTCHECK(ok_address(fm, p) && ok_cinuse(p))) { 5228 + size_t psize = chunksize(p); 5229 + mchunkptr next = chunk_plus_offset(p, psize); 5230 + if (!pinuse(p)) { 5231 + size_t prevsize = p->prev_foot; 5232 + if ((prevsize & IS_MMAPPED_BIT) != 0) { 5233 + prevsize &= ~IS_MMAPPED_BIT; 5234 + psize += prevsize + MMAP_FOOT_PAD; 5235 + if (CALL_MUNMAP((char*)p - prevsize, psize) == 0) 5236 + fm->footprint -= psize; 5237 + goto postaction; 5238 + } 5239 + else { 5240 + mchunkptr prev = chunk_minus_offset(p, prevsize); 5241 + psize += prevsize; 5242 + p = prev; 5243 + if (RTCHECK(ok_address(fm, prev))) { /* consolidate backward */ 5244 + if (p != fm->dv) { 5245 + unlink_chunk(fm, p, prevsize); 5246 + } 5247 + else if ((next->head & INUSE_BITS) == INUSE_BITS) { 5248 + fm->dvsize = psize; 5249 + set_free_with_pinuse(p, psize, next); 5250 + goto postaction; 5251 + } 5252 + } 5253 + else 5254 + goto erroraction; 5255 + } 5256 + } 5257 + 5258 + if (RTCHECK(ok_next(p, next) && ok_pinuse(next))) { 5259 + if (!cinuse(next)) { /* consolidate forward */ 5260 + if (next == fm->top) { 5261 + size_t tsize = fm->topsize += psize; 5262 + fm->top = p; 5263 + p->head = tsize | PINUSE_BIT; 5264 + if (p == fm->dv) { 5265 + fm->dv = 0; 5266 + fm->dvsize = 0; 5267 + } 5268 + if (should_trim(fm, tsize)) 5269 + sys_trim(fm, 0); 5270 + goto postaction; 5271 + } 5272 + else if (next == fm->dv) { 5273 + size_t dsize = fm->dvsize += psize; 5274 + fm->dv = p; 5275 + set_size_and_pinuse_of_free_chunk(p, dsize); 5276 + goto postaction; 5277 + } 5278 + else { 5279 + size_t nsize = chunksize(next); 5280 + psize += nsize; 5281 + unlink_chunk(fm, next, nsize); 5282 + set_size_and_pinuse_of_free_chunk(p, psize); 5283 + if (p == fm->dv) { 5284 + fm->dvsize = psize; 5285 + goto postaction; 5286 + } 5287 + } 5288 + } 5289 + else 5290 + set_free_with_pinuse(p, psize, next); 5291 + 5292 + if (is_small(psize)) { 5293 + insert_small_chunk(fm, p, psize); 5294 + check_free_chunk(fm, p); 5295 + } 5296 + else { 5297 + tchunkptr tp = (tchunkptr)p; 5298 + insert_large_chunk(fm, tp, psize); 5299 + check_free_chunk(fm, p); 5300 + if (--fm->release_checks == 0) 5301 + release_unused_segments(fm); 5302 + } 5303 + goto postaction; 5304 + } 5305 + } 5306 + erroraction: 5307 + USAGE_ERROR_ACTION(fm, p); 5308 + postaction: 5309 + POSTACTION(fm); 5310 + } 5311 + } 5312 + } 5313 + 5314 + void* mspace_calloc(mspace msp, size_t n_elements, size_t elem_size) { 5315 + void* mem; 5316 + size_t req = 0; 5317 + mstate ms = (mstate)msp; 5318 + if (!ok_magic(ms)) { 5319 + USAGE_ERROR_ACTION(ms,ms); 5320 + return 0; 5321 + } 5322 + if (n_elements != 0) { 5323 + req = n_elements * elem_size; 5324 + if (((n_elements | elem_size) & ~(size_t)0xffff) && 5325 + (req / n_elements != elem_size)) 5326 + req = MAX_SIZE_T; /* force downstream failure on overflow */ 5327 + } 5328 + mem = internal_malloc(ms, req); 5329 + if (mem != 0 && calloc_must_clear(mem2chunk(mem))) 5330 + memset(mem, 0, req); 5331 + return mem; 5332 + } 5333 + 5334 + void* mspace_realloc(mspace msp, void* oldmem, size_t bytes) { 5335 + if (oldmem == 0) 5336 + return mspace_malloc(msp, bytes); 5337 + #ifdef REALLOC_ZERO_BYTES_FREES 5338 + if (bytes == 0) { 5339 + mspace_free(msp, oldmem); 5340 + return 0; 5341 + } 5342 + #endif /* REALLOC_ZERO_BYTES_FREES */ 5343 + else { 5344 + #if FOOTERS 5345 + mchunkptr p = mem2chunk(oldmem); 5346 + mstate ms = get_mstate_for(p); 5347 + #else /* FOOTERS */ 5348 + mstate ms = (mstate)msp; 5349 + #endif /* FOOTERS */ 5350 + if (!ok_magic(ms)) { 5351 + USAGE_ERROR_ACTION(ms,ms); 5352 + return 0; 5353 + } 5354 + return internal_realloc(ms, oldmem, bytes); 5355 + } 5356 + } 5357 + 5358 + void* mspace_memalign(mspace msp, size_t alignment, size_t bytes) { 5359 + mstate ms = (mstate)msp; 5360 + if (!ok_magic(ms)) { 5361 + USAGE_ERROR_ACTION(ms,ms); 5362 + return 0; 5363 + } 5364 + return internal_memalign(ms, alignment, bytes); 5365 + } 5366 + 5367 + void** mspace_independent_calloc(mspace msp, size_t n_elements, 5368 + size_t elem_size, void* chunks[]) { 5369 + size_t sz = elem_size; /* serves as 1-element array */ 5370 + mstate ms = (mstate)msp; 5371 + if (!ok_magic(ms)) { 5372 + USAGE_ERROR_ACTION(ms,ms); 5373 + return 0; 5374 + } 5375 + return ialloc(ms, n_elements, &sz, 3, chunks); 5376 + } 5377 + 5378 + void** mspace_independent_comalloc(mspace msp, size_t n_elements, 5379 + size_t sizes[], void* chunks[]) { 5380 + mstate ms = (mstate)msp; 5381 + if (!ok_magic(ms)) { 5382 + USAGE_ERROR_ACTION(ms,ms); 5383 + return 0; 5384 + } 5385 + return ialloc(ms, n_elements, sizes, 0, chunks); 5386 + } 5387 + 5388 + int mspace_trim(mspace msp, size_t pad) { 5389 + int result = 0; 5390 + mstate ms = (mstate)msp; 5391 + if (ok_magic(ms)) { 5392 + if (!PREACTION(ms)) { 5393 + result = sys_trim(ms, pad); 5394 + POSTACTION(ms); 5395 + } 5396 + } 5397 + else { 5398 + USAGE_ERROR_ACTION(ms,ms); 5399 + } 5400 + return result; 5401 + } 5402 + 5403 + void mspace_malloc_stats(mspace msp) { 5404 + mstate ms = (mstate)msp; 5405 + if (ok_magic(ms)) { 5406 + internal_malloc_stats(ms); 5407 + } 5408 + else { 5409 + USAGE_ERROR_ACTION(ms,ms); 5410 + } 5411 + } 5412 + 5413 + size_t mspace_footprint(mspace msp) { 5414 + size_t result = 0; 5415 + mstate ms = (mstate)msp; 5416 + if (ok_magic(ms)) { 5417 + result = ms->footprint; 5418 + } 5419 + else { 5420 + USAGE_ERROR_ACTION(ms,ms); 5421 + } 5422 + return result; 5423 + } 5424 + 5425 + 5426 + size_t mspace_max_footprint(mspace msp) { 5427 + size_t result = 0; 5428 + mstate ms = (mstate)msp; 5429 + if (ok_magic(ms)) { 5430 + result = ms->max_footprint; 5431 + } 5432 + else { 5433 + USAGE_ERROR_ACTION(ms,ms); 5434 + } 5435 + return result; 5436 + } 5437 + 5438 + 5439 + #if !NO_MALLINFO 5440 + struct mallinfo mspace_mallinfo(mspace msp) { 5441 + mstate ms = (mstate)msp; 5442 + if (!ok_magic(ms)) { 5443 + USAGE_ERROR_ACTION(ms,ms); 5444 + } 5445 + return internal_mallinfo(ms); 5446 + } 5447 + #endif /* NO_MALLINFO */ 5448 + 5449 + size_t mspace_usable_size(void* mem) { 5450 + if (mem != 0) { 5451 + mchunkptr p = mem2chunk(mem); 5452 + if (cinuse(p)) 5453 + return chunksize(p) - overhead_for(p); 5454 + } 5455 + return 0; 5456 + } 5457 + 5458 + int mspace_mallopt(int param_number, int value) { 5459 + return change_mparam(param_number, value); 5460 + } 5461 + 5462 + #endif /* MSPACES */ 5463 + 5464 + /* -------------------- Alternative MORECORE functions ------------------- */ 5465 + 5466 + /* 5467 + Guidelines for creating a custom version of MORECORE: 5468 + 5469 + * For best performance, MORECORE should allocate in multiples of pagesize. 5470 + * MORECORE may allocate more memory than requested. (Or even less, 5471 + but this will usually result in a malloc failure.) 5472 + * MORECORE must not allocate memory when given argument zero, but 5473 + instead return one past the end address of memory from previous 5474 + nonzero call. 5475 + * For best performance, consecutive calls to MORECORE with positive 5476 + arguments should return increasing addresses, indicating that 5477 + space has been contiguously extended. 5478 + * Even though consecutive calls to MORECORE need not return contiguous 5479 + addresses, it must be OK for malloc'ed chunks to span multiple 5480 + regions in those cases where they do happen to be contiguous. 5481 + * MORECORE need not handle negative arguments -- it may instead 5482 + just return MFAIL when given negative arguments. 5483 + Negative arguments are always multiples of pagesize. MORECORE 5484 + must not misinterpret negative args as large positive unsigned 5485 + args. You can suppress all such calls from even occurring by defining 5486 + MORECORE_CANNOT_TRIM, 5487 + 5488 + As an example alternative MORECORE, here is a custom allocator 5489 + kindly contributed for pre-OSX macOS. It uses virtually but not 5490 + necessarily physically contiguous non-paged memory (locked in, 5491 + present and won't get swapped out). You can use it by uncommenting 5492 + this section, adding some #includes, and setting up the appropriate 5493 + defines above: 5494 + 5495 + #define MORECORE osMoreCore 5496 + 5497 + There is also a shutdown routine that should somehow be called for 5498 + cleanup upon program exit. 5499 + 5500 + #define MAX_POOL_ENTRIES 100 5501 + #define MINIMUM_MORECORE_SIZE (64 * 1024U) 5502 + static int next_os_pool; 5503 + void *our_os_pools[MAX_POOL_ENTRIES]; 5504 + 5505 + void *osMoreCore(int size) 5506 + { 5507 + void *ptr = 0; 5508 + static void *sbrk_top = 0; 5509 + 5510 + if (size > 0) 5511 + { 5512 + if (size < MINIMUM_MORECORE_SIZE) 5513 + size = MINIMUM_MORECORE_SIZE; 5514 + if (CurrentExecutionLevel() == kTaskLevel) 5515 + ptr = PoolAllocateResident(size + RM_PAGE_SIZE, 0); 5516 + if (ptr == 0) 5517 + { 5518 + return (void *) MFAIL; 5519 + } 5520 + // save ptrs so they can be freed during cleanup 5521 + our_os_pools[next_os_pool] = ptr; 5522 + next_os_pool++; 5523 + ptr = (void *) ((((size_t) ptr) + RM_PAGE_MASK) & ~RM_PAGE_MASK); 5524 + sbrk_top = (char *) ptr + size; 5525 + return ptr; 5526 + } 5527 + else if (size < 0) 5528 + { 5529 + // we don't currently support shrink behavior 5530 + return (void *) MFAIL; 5531 + } 5532 + else 5533 + { 5534 + return sbrk_top; 5535 + } 5536 + } 5537 + 5538 + // cleanup any allocated memory pools 5539 + // called as last thing before shutting down driver 5540 + 5541 + void osCleanupMem(void) 5542 + { 5543 + void **ptr; 5544 + 5545 + for (ptr = our_os_pools; ptr < &our_os_pools[MAX_POOL_ENTRIES]; ptr++) 5546 + if (*ptr) 5547 + { 5548 + PoolDeallocate(*ptr); 5549 + *ptr = 0; 5550 + } 5551 + } 5552 + 5553 + */ 5554 + 5555 + 5556 + /* ----------------------------------------------------------------------- 5557 + History: 5558 + V2.8.4 (not yet released) 5559 + * Add mspace_mmap_large_chunks; thanks to Jean Brouwers 5560 + * Fix insufficient sys_alloc padding when using 16byte alignment 5561 + * Fix bad error check in mspace_footprint 5562 + * Adaptations for ptmalloc, courtesy of Wolfram Gloger. 5563 + * Reentrant spin locks, courtesy of Earl Chew and others 5564 + * Win32 improvements, courtesy of Niall Douglas and Earl Chew 5565 + * Add NO_SEGMENT_TRAVERSAL and MAX_RELEASE_CHECK_RATE options 5566 + * Extension hook in malloc_state 5567 + * Various small adjustments to reduce warnings on some compilers 5568 + * Various configuration extensions/changes for more platforms. Thanks 5569 + to all who contributed these. 5570 + 5571 + V2.8.3 Thu Sep 22 11:16:32 2005 Doug Lea (dl at gee) 5572 + * Add max_footprint functions 5573 + * Ensure all appropriate literals are size_t 5574 + * Fix conditional compilation problem for some #define settings 5575 + * Avoid concatenating segments with the one provided 5576 + in create_mspace_with_base 5577 + * Rename some variables to avoid compiler shadowing warnings 5578 + * Use explicit lock initialization. 5579 + * Better handling of sbrk interference. 5580 + * Simplify and fix segment insertion, trimming and mspace_destroy 5581 + * Reinstate REALLOC_ZERO_BYTES_FREES option from 2.7.x 5582 + * Thanks especially to Dennis Flanagan for help on these. 5583 + 5584 + V2.8.2 Sun Jun 12 16:01:10 2005 Doug Lea (dl at gee) 5585 + * Fix memalign brace error. 5586 + 5587 + V2.8.1 Wed Jun 8 16:11:46 2005 Doug Lea (dl at gee) 5588 + * Fix improper #endif nesting in C++ 5589 + * Add explicit casts needed for C++ 5590 + 5591 + V2.8.0 Mon May 30 14:09:02 2005 Doug Lea (dl at gee) 5592 + * Use trees for large bins 5593 + * Support mspaces 5594 + * Use segments to unify sbrk-based and mmap-based system allocation, 5595 + removing need for emulation on most platforms without sbrk. 5596 + * Default safety checks 5597 + * Optional footer checks. Thanks to William Robertson for the idea. 5598 + * Internal code refactoring 5599 + * Incorporate suggestions and platform-specific changes. 5600 + Thanks to Dennis Flanagan, Colin Plumb, Niall Douglas, 5601 + Aaron Bachmann, Emery Berger, and others. 5602 + * Speed up non-fastbin processing enough to remove fastbins. 5603 + * Remove useless cfree() to avoid conflicts with other apps. 5604 + * Remove internal memcpy, memset. Compilers handle builtins better. 5605 + * Remove some options that no one ever used and rename others. 5606 + 5607 + V2.7.2 Sat Aug 17 09:07:30 2002 Doug Lea (dl at gee) 5608 + * Fix malloc_state bitmap array misdeclaration 5609 + 5610 + V2.7.1 Thu Jul 25 10:58:03 2002 Doug Lea (dl at gee) 5611 + * Allow tuning of FIRST_SORTED_BIN_SIZE 5612 + * Use PTR_UINT as type for all ptr->int casts. Thanks to John Belmonte. 5613 + * Better detection and support for non-contiguousness of MORECORE. 5614 + Thanks to Andreas Mueller, Conal Walsh, and Wolfram Gloger 5615 + * Bypass most of malloc if no frees. Thanks To Emery Berger. 5616 + * Fix freeing of old top non-contiguous chunk im sysmalloc. 5617 + * Raised default trim and map thresholds to 256K. 5618 + * Fix mmap-related #defines. Thanks to Lubos Lunak. 5619 + * Fix copy macros; added LACKS_FCNTL_H. Thanks to Neal Walfield. 5620 + * Branch-free bin calculation 5621 + * Default trim and mmap thresholds now 256K. 5622 + 5623 + V2.7.0 Sun Mar 11 14:14:06 2001 Doug Lea (dl at gee) 5624 + * Introduce independent_comalloc and independent_calloc. 5625 + Thanks to Michael Pachos for motivation and help. 5626 + * Make optional .h file available 5627 + * Allow > 2GB requests on 32bit systems. 5628 + * new WIN32 sbrk, mmap, munmap, lock code from <Walter@GeNeSys-e.de>. 5629 + Thanks also to Andreas Mueller <a.mueller at paradatec.de>, 5630 + and Anonymous. 5631 + * Allow override of MALLOC_ALIGNMENT (Thanks to Ruud Waij for 5632 + helping test this.) 5633 + * memalign: check alignment arg 5634 + * realloc: don't try to shift chunks backwards, since this 5635 + leads to more fragmentation in some programs and doesn't 5636 + seem to help in any others. 5637 + * Collect all cases in malloc requiring system memory into sysmalloc 5638 + * Use mmap as backup to sbrk 5639 + * Place all internal state in malloc_state 5640 + * Introduce fastbins (although similar to 2.5.1) 5641 + * Many minor tunings and cosmetic improvements 5642 + * Introduce USE_PUBLIC_MALLOC_WRAPPERS, USE_MALLOC_LOCK 5643 + * Introduce MALLOC_FAILURE_ACTION, MORECORE_CONTIGUOUS 5644 + Thanks to Tony E. Bennett <tbennett@nvidia.com> and others. 5645 + * Include errno.h to support default failure action. 5646 + 5647 + V2.6.6 Sun Dec 5 07:42:19 1999 Doug Lea (dl at gee) 5648 + * return null for negative arguments 5649 + * Added Several WIN32 cleanups from Martin C. Fong <mcfong at yahoo.com> 5650 + * Add 'LACKS_SYS_PARAM_H' for those systems without 'sys/param.h' 5651 + (e.g. WIN32 platforms) 5652 + * Cleanup header file inclusion for WIN32 platforms 5653 + * Cleanup code to avoid Microsoft Visual C++ compiler complaints 5654 + * Add 'USE_DL_PREFIX' to quickly allow co-existence with existing 5655 + memory allocation routines 5656 + * Set 'malloc_getpagesize' for WIN32 platforms (needs more work) 5657 + * Use 'assert' rather than 'ASSERT' in WIN32 code to conform to 5658 + usage of 'assert' in non-WIN32 code 5659 + * Improve WIN32 'sbrk()' emulation's 'findRegion()' routine to 5660 + avoid infinite loop 5661 + * Always call 'fREe()' rather than 'free()' 5662 + 5663 + V2.6.5 Wed Jun 17 15:57:31 1998 Doug Lea (dl at gee) 5664 + * Fixed ordering problem with boundary-stamping 5665 + 5666 + V2.6.3 Sun May 19 08:17:58 1996 Doug Lea (dl at gee) 5667 + * Added pvalloc, as recommended by H.J. Liu 5668 + * Added 64bit pointer support mainly from Wolfram Gloger 5669 + * Added anonymously donated WIN32 sbrk emulation 5670 + * Malloc, calloc, getpagesize: add optimizations from Raymond Nijssen 5671 + * malloc_extend_top: fix mask error that caused wastage after 5672 + foreign sbrks 5673 + * Add linux mremap support code from HJ Liu 5674 + 5675 + V2.6.2 Tue Dec 5 06:52:55 1995 Doug Lea (dl at gee) 5676 + * Integrated most documentation with the code. 5677 + * Add support for mmap, with help from 5678 + Wolfram Gloger (Gloger@lrz.uni-muenchen.de). 5679 + * Use last_remainder in more cases. 5680 + * Pack bins using idea from colin@nyx10.cs.du.edu 5681 + * Use ordered bins instead of best-fit threshhold 5682 + * Eliminate block-local decls to simplify tracing and debugging. 5683 + * Support another case of realloc via move into top 5684 + * Fix error occuring when initial sbrk_base not word-aligned. 5685 + * Rely on page size for units instead of SBRK_UNIT to 5686 + avoid surprises about sbrk alignment conventions. 5687 + * Add mallinfo, mallopt. Thanks to Raymond Nijssen 5688 + (raymond@es.ele.tue.nl) for the suggestion. 5689 + * Add `pad' argument to malloc_trim and top_pad mallopt parameter. 5690 + * More precautions for cases where other routines call sbrk, 5691 + courtesy of Wolfram Gloger (Gloger@lrz.uni-muenchen.de). 5692 + * Added macros etc., allowing use in linux libc from 5693 + H.J. Lu (hjl@gnu.ai.mit.edu) 5694 + * Inverted this history list 5695 + 5696 + V2.6.1 Sat Dec 2 14:10:57 1995 Doug Lea (dl at gee) 5697 + * Re-tuned and fixed to behave more nicely with V2.6.0 changes. 5698 + * Removed all preallocation code since under current scheme 5699 + the work required to undo bad preallocations exceeds 5700 + the work saved in good cases for most test programs. 5701 + * No longer use return list or unconsolidated bins since 5702 + no scheme using them consistently outperforms those that don't 5703 + given above changes. 5704 + * Use best fit for very large chunks to prevent some worst-cases. 5705 + * Added some support for debugging 5706 + 5707 + V2.6.0 Sat Nov 4 07:05:23 1995 Doug Lea (dl at gee) 5708 + * Removed footers when chunks are in use. Thanks to 5709 + Paul Wilson (wilson@cs.texas.edu) for the suggestion. 5710 + 5711 + V2.5.4 Wed Nov 1 07:54:51 1995 Doug Lea (dl at gee) 5712 + * Added malloc_trim, with help from Wolfram Gloger 5713 + (wmglo@Dent.MED.Uni-Muenchen.DE). 5714 + 5715 + V2.5.3 Tue Apr 26 10:16:01 1994 Doug Lea (dl at g) 5716 + 5717 + V2.5.2 Tue Apr 5 16:20:40 1994 Doug Lea (dl at g) 5718 + * realloc: try to expand in both directions 5719 + * malloc: swap order of clean-bin strategy; 5720 + * realloc: only conditionally expand backwards 5721 + * Try not to scavenge used bins 5722 + * Use bin counts as a guide to preallocation 5723 + * Occasionally bin return list chunks in first scan 5724 + * Add a few optimizations from colin@nyx10.cs.du.edu 5725 + 5726 + V2.5.1 Sat Aug 14 15:40:43 1993 Doug Lea (dl at g) 5727 + * faster bin computation & slightly different binning 5728 + * merged all consolidations to one part of malloc proper 5729 + (eliminating old malloc_find_space & malloc_clean_bin) 5730 + * Scan 2 returns chunks (not just 1) 5731 + * Propagate failure in realloc if malloc returns 0 5732 + * Add stuff to allow compilation on non-ANSI compilers 5733 + from kpv@research.att.com 5734 + 5735 + V2.5 Sat Aug 7 07:41:59 1993 Doug Lea (dl at g.oswego.edu) 5736 + * removed potential for odd address access in prev_chunk 5737 + * removed dependency on getpagesize.h 5738 + * misc cosmetics and a bit more internal documentation 5739 + * anticosmetics: mangled names in macros to evade debugger strangeness 5740 + * tested on sparc, hp-700, dec-mips, rs6000 5741 + with gcc & native cc (hp, dec only) allowing 5742 + Detlefs & Zorn comparison study (in SIGPLAN Notices.) 5743 + 5744 + Trial version Fri Aug 28 13:14:29 1992 Doug Lea (dl at g.oswego.edu) 5745 + * Based loosely on libg++-1.2X malloc. (It retains some of the overall 5746 + structure of old version, but most details differ.) 5747 + 5748 + */ 5749 + 5750 +

+966

compat/nedmalloc/nedmalloc.c

··· 1 + /* Alternative malloc implementation for multiple threads without 2 + lock contention based on dlmalloc. (C) 2005-2006 Niall Douglas 3 + 4 + Boost Software License - Version 1.0 - August 17th, 2003 5 + 6 + Permission is hereby granted, free of charge, to any person or organization 7 + obtaining a copy of the software and accompanying documentation covered by 8 + this license (the "Software") to use, reproduce, display, distribute, 9 + execute, and transmit the Software, and to prepare derivative works of the 10 + Software, and to permit third-parties to whom the Software is furnished to 11 + do so, all subject to the following: 12 + 13 + The copyright notices in the Software and this entire statement, including 14 + the above license grant, this restriction and the following disclaimer, 15 + must be included in all copies of the Software, in whole or in part, and 16 + all derivative works of the Software, unless such copies or derivative 17 + works are solely in the form of machine-executable object code generated by 18 + a source language processor. 19 + 20 + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 21 + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 22 + FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT 23 + SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE 24 + FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, 25 + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 26 + DEALINGS IN THE SOFTWARE. 27 + */ 28 + 29 + #ifdef _MSC_VER 30 + /* Enable full aliasing on MSVC */ 31 + /*#pragma optimize("a", on)*/ 32 + #endif 33 + 34 + /*#define FULLSANITYCHECKS*/ 35 + 36 + #include "nedmalloc.h" 37 + #if defined(WIN32) && !defined(__MINGW32__) 38 + #include <malloc.h> 39 + #endif 40 + #define MSPACES 1 41 + #define ONLY_MSPACES 1 42 + #ifndef USE_LOCKS 43 + #define USE_LOCKS 1 44 + #endif 45 + #define FOOTERS 1 /* Need to enable footers so frees lock the right mspace */ 46 + #undef DEBUG /* dlmalloc wants DEBUG either 0 or 1 */ 47 + #ifdef _DEBUG 48 + #define DEBUG 1 49 + #else 50 + #define DEBUG 0 51 + #endif 52 + #ifdef NDEBUG /* Disable assert checking on release builds */ 53 + #undef DEBUG 54 + #endif 55 + /* The default of 64Kb means we spend too much time kernel-side */ 56 + #ifndef DEFAULT_GRANULARITY 57 + #define DEFAULT_GRANULARITY (1*1024*1024) 58 + #endif 59 + /*#define USE_SPIN_LOCKS 0*/ 60 + 61 + 62 + /*#define FORCEINLINE*/ 63 + #include "malloc.c.h" 64 + #ifdef NDEBUG /* Disable assert checking on release builds */ 65 + #undef DEBUG 66 + #endif 67 + 68 + /* The maximum concurrent threads in a pool possible */ 69 + #ifndef MAXTHREADSINPOOL 70 + #define MAXTHREADSINPOOL 16 71 + #endif 72 + /* The maximum number of threadcaches which can be allocated */ 73 + #ifndef THREADCACHEMAXCACHES 74 + #define THREADCACHEMAXCACHES 256 75 + #endif 76 + /* The maximum size to be allocated from the thread cache */ 77 + #ifndef THREADCACHEMAX 78 + #define THREADCACHEMAX 8192 79 + #endif 80 + #if 0 81 + /* The number of cache entries for finer grained bins. This is (topbitpos(THREADCACHEMAX)-4)*2 */ 82 + #define THREADCACHEMAXBINS ((13-4)*2) 83 + #else 84 + /* The number of cache entries. This is (topbitpos(THREADCACHEMAX)-4) */ 85 + #define THREADCACHEMAXBINS (13-4) 86 + #endif 87 + /* Point at which the free space in a thread cache is garbage collected */ 88 + #ifndef THREADCACHEMAXFREESPACE 89 + #define THREADCACHEMAXFREESPACE (512*1024) 90 + #endif 91 + 92 + 93 + #ifdef WIN32 94 + #define TLSVAR DWORD 95 + #define TLSALLOC(k) (*(k)=TlsAlloc(), TLS_OUT_OF_INDEXES==*(k)) 96 + #define TLSFREE(k) (!TlsFree(k)) 97 + #define TLSGET(k) TlsGetValue(k) 98 + #define TLSSET(k, a) (!TlsSetValue(k, a)) 99 + #ifdef DEBUG 100 + static LPVOID ChkedTlsGetValue(DWORD idx) 101 + { 102 + LPVOID ret=TlsGetValue(idx); 103 + assert(S_OK==GetLastError()); 104 + return ret; 105 + } 106 + #undef TLSGET 107 + #define TLSGET(k) ChkedTlsGetValue(k) 108 + #endif 109 + #else 110 + #define TLSVAR pthread_key_t 111 + #define TLSALLOC(k) pthread_key_create(k, 0) 112 + #define TLSFREE(k) pthread_key_delete(k) 113 + #define TLSGET(k) pthread_getspecific(k) 114 + #define TLSSET(k, a) pthread_setspecific(k, a) 115 + #endif 116 + 117 + #if 0 118 + /* Only enable if testing with valgrind. Causes misoperation */ 119 + #define mspace_malloc(p, s) malloc(s) 120 + #define mspace_realloc(p, m, s) realloc(m, s) 121 + #define mspace_calloc(p, n, s) calloc(n, s) 122 + #define mspace_free(p, m) free(m) 123 + #endif 124 + 125 + 126 + #if defined(__cplusplus) 127 + #if !defined(NO_NED_NAMESPACE) 128 + namespace nedalloc { 129 + #else 130 + extern "C" { 131 + #endif 132 + #endif 133 + 134 + size_t nedblksize(void *mem) THROWSPEC 135 + { 136 + #if 0 137 + /* Only enable if testing with valgrind. Causes misoperation */ 138 + return THREADCACHEMAX; 139 + #else 140 + if(mem) 141 + { 142 + mchunkptr p=mem2chunk(mem); 143 + assert(cinuse(p)); /* If this fails, someone tried to free a block twice */ 144 + if(cinuse(p)) 145 + return chunksize(p)-overhead_for(p); 146 + } 147 + return 0; 148 + #endif 149 + } 150 + 151 + void nedsetvalue(void *v) THROWSPEC { nedpsetvalue(0, v); } 152 + void * nedmalloc(size_t size) THROWSPEC { return nedpmalloc(0, size); } 153 + void * nedcalloc(size_t no, size_t size) THROWSPEC { return nedpcalloc(0, no, size); } 154 + void * nedrealloc(void *mem, size_t size) THROWSPEC { return nedprealloc(0, mem, size); } 155 + void nedfree(void *mem) THROWSPEC { nedpfree(0, mem); } 156 + void * nedmemalign(size_t alignment, size_t bytes) THROWSPEC { return nedpmemalign(0, alignment, bytes); } 157 + #if !NO_MALLINFO 158 + struct mallinfo nedmallinfo(void) THROWSPEC { return nedpmallinfo(0); } 159 + #endif 160 + int nedmallopt(int parno, int value) THROWSPEC { return nedpmallopt(0, parno, value); } 161 + int nedmalloc_trim(size_t pad) THROWSPEC { return nedpmalloc_trim(0, pad); } 162 + void nedmalloc_stats() THROWSPEC { nedpmalloc_stats(0); } 163 + size_t nedmalloc_footprint() THROWSPEC { return nedpmalloc_footprint(0); } 164 + void **nedindependent_calloc(size_t elemsno, size_t elemsize, void **chunks) THROWSPEC { return nedpindependent_calloc(0, elemsno, elemsize, chunks); } 165 + void **nedindependent_comalloc(size_t elems, size_t *sizes, void **chunks) THROWSPEC { return nedpindependent_comalloc(0, elems, sizes, chunks); } 166 + 167 + struct threadcacheblk_t; 168 + typedef struct threadcacheblk_t threadcacheblk; 169 + struct threadcacheblk_t 170 + { /* Keep less than 16 bytes on 32 bit systems and 32 bytes on 64 bit systems */ 171 + #ifdef FULLSANITYCHECKS 172 + unsigned int magic; 173 + #endif 174 + unsigned int lastUsed, size; 175 + threadcacheblk *next, *prev; 176 + }; 177 + typedef struct threadcache_t 178 + { 179 + #ifdef FULLSANITYCHECKS 180 + unsigned int magic1; 181 + #endif 182 + int mymspace; /* Last mspace entry this thread used */ 183 + long threadid; 184 + unsigned int mallocs, frees, successes; 185 + size_t freeInCache; /* How much free space is stored in this cache */ 186 + threadcacheblk *bins[(THREADCACHEMAXBINS+1)*2]; 187 + #ifdef FULLSANITYCHECKS 188 + unsigned int magic2; 189 + #endif 190 + } threadcache; 191 + struct nedpool_t 192 + { 193 + MLOCK_T mutex; 194 + void *uservalue; 195 + int threads; /* Max entries in m to use */ 196 + threadcache *caches[THREADCACHEMAXCACHES]; 197 + TLSVAR mycache; /* Thread cache for this thread. 0 for unset, negative for use mspace-1 directly, otherwise is cache-1 */ 198 + mstate m[MAXTHREADSINPOOL+1]; /* mspace entries for this pool */ 199 + }; 200 + static nedpool syspool; 201 + 202 + static FORCEINLINE unsigned int size2binidx(size_t _size) THROWSPEC 203 + { /* 8=1000 16=10000 20=10100 24=11000 32=100000 48=110000 4096=1000000000000 */ 204 + unsigned int topbit, size=(unsigned int)(_size>>4); 205 + /* 16=1 20=1 24=1 32=10 48=11 64=100 96=110 128=1000 4096=100000000 */ 206 + 207 + #if defined(__GNUC__) 208 + topbit = sizeof(size)*__CHAR_BIT__ - 1 - __builtin_clz(size); 209 + #elif defined(_MSC_VER) && _MSC_VER>=1300 210 + { 211 + unsigned long bsrTopBit; 212 + 213 + _BitScanReverse(&bsrTopBit, size); 214 + 215 + topbit = bsrTopBit; 216 + } 217 + #else 218 + #if 0 219 + union { 220 + unsigned asInt[2]; 221 + double asDouble; 222 + }; 223 + int n; 224 + 225 + asDouble = (double)size + 0.5; 226 + topbit = (asInt[!FOX_BIGENDIAN] >> 20) - 1023; 227 + #else 228 + { 229 + unsigned int x=size; 230 + x = x | (x >> 1); 231 + x = x | (x >> 2); 232 + x = x | (x >> 4); 233 + x = x | (x >> 8); 234 + x = x | (x >>16); 235 + x = ~x; 236 + x = x - ((x >> 1) & 0x55555555); 237 + x = (x & 0x33333333) + ((x >> 2) & 0x33333333); 238 + x = (x + (x >> 4)) & 0x0F0F0F0F; 239 + x = x + (x << 8); 240 + x = x + (x << 16); 241 + topbit=31 - (x >> 24); 242 + } 243 + #endif 244 + #endif 245 + return topbit; 246 + } 247 + 248 + 249 + #ifdef FULLSANITYCHECKS 250 + static void tcsanitycheck(threadcacheblk **ptr) THROWSPEC 251 + { 252 + assert((ptr[0] && ptr[1]) || (!ptr[0] && !ptr[1])); 253 + if(ptr[0] && ptr[1]) 254 + { 255 + assert(nedblksize(ptr[0])>=sizeof(threadcacheblk)); 256 + assert(nedblksize(ptr[1])>=sizeof(threadcacheblk)); 257 + assert(*(unsigned int *) "NEDN"==ptr[0]->magic); 258 + assert(*(unsigned int *) "NEDN"==ptr[1]->magic); 259 + assert(!ptr[0]->prev); 260 + assert(!ptr[1]->next); 261 + if(ptr[0]==ptr[1]) 262 + { 263 + assert(!ptr[0]->next); 264 + assert(!ptr[1]->prev); 265 + } 266 + } 267 + } 268 + static void tcfullsanitycheck(threadcache *tc) THROWSPEC 269 + { 270 + threadcacheblk **tcbptr=tc->bins; 271 + int n; 272 + for(n=0; n<=THREADCACHEMAXBINS; n++, tcbptr+=2) 273 + { 274 + threadcacheblk *b, *ob=0; 275 + tcsanitycheck(tcbptr); 276 + for(b=tcbptr[0]; b; ob=b, b=b->next) 277 + { 278 + assert(*(unsigned int *) "NEDN"==b->magic); 279 + assert(!ob || ob->next==b); 280 + assert(!ob || b->prev==ob); 281 + } 282 + } 283 + } 284 + #endif 285 + 286 + static NOINLINE void RemoveCacheEntries(nedpool *p, threadcache *tc, unsigned int age) THROWSPEC 287 + { 288 + #ifdef FULLSANITYCHECKS 289 + tcfullsanitycheck(tc); 290 + #endif 291 + if(tc->freeInCache) 292 + { 293 + threadcacheblk **tcbptr=tc->bins; 294 + int n; 295 + for(n=0; n<=THREADCACHEMAXBINS; n++, tcbptr+=2) 296 + { 297 + threadcacheblk **tcb=tcbptr+1; /* come from oldest end of list */ 298 + /*tcsanitycheck(tcbptr);*/ 299 + for(; *tcb && tc->frees-(*tcb)->lastUsed>=age; ) 300 + { 301 + threadcacheblk *f=*tcb; 302 + size_t blksize=f->size; /*nedblksize(f);*/ 303 + assert(blksize<=nedblksize(f)); 304 + assert(blksize); 305 + #ifdef FULLSANITYCHECKS 306 + assert(*(unsigned int *) "NEDN"==(*tcb)->magic); 307 + #endif 308 + *tcb=(*tcb)->prev; 309 + if(*tcb) 310 + (*tcb)->next=0; 311 + else 312 + *tcbptr=0; 313 + tc->freeInCache-=blksize; 314 + assert((long) tc->freeInCache>=0); 315 + mspace_free(0, f); 316 + /*tcsanitycheck(tcbptr);*/ 317 + } 318 + } 319 + } 320 + #ifdef FULLSANITYCHECKS 321 + tcfullsanitycheck(tc); 322 + #endif 323 + } 324 + static void DestroyCaches(nedpool *p) THROWSPEC 325 + { 326 + if(p->caches) 327 + { 328 + threadcache *tc; 329 + int n; 330 + for(n=0; n<THREADCACHEMAXCACHES; n++) 331 + { 332 + if((tc=p->caches[n])) 333 + { 334 + tc->frees++; 335 + RemoveCacheEntries(p, tc, 0); 336 + assert(!tc->freeInCache); 337 + tc->mymspace=-1; 338 + tc->threadid=0; 339 + mspace_free(0, tc); 340 + p->caches[n]=0; 341 + } 342 + } 343 + } 344 + } 345 + 346 + static NOINLINE threadcache *AllocCache(nedpool *p) THROWSPEC 347 + { 348 + threadcache *tc=0; 349 + int n, end; 350 + ACQUIRE_LOCK(&p->mutex); 351 + for(n=0; n<THREADCACHEMAXCACHES && p->caches[n]; n++); 352 + if(THREADCACHEMAXCACHES==n) 353 + { /* List exhausted, so disable for this thread */ 354 + RELEASE_LOCK(&p->mutex); 355 + return 0; 356 + } 357 + tc=p->caches[n]=(threadcache *) mspace_calloc(p->m[0], 1, sizeof(threadcache)); 358 + if(!tc) 359 + { 360 + RELEASE_LOCK(&p->mutex); 361 + return 0; 362 + } 363 + #ifdef FULLSANITYCHECKS 364 + tc->magic1=*(unsigned int *)"NEDMALC1"; 365 + tc->magic2=*(unsigned int *)"NEDMALC2"; 366 + #endif 367 + tc->threadid=(long)(size_t)CURRENT_THREAD; 368 + for(end=0; p->m[end]; end++); 369 + tc->mymspace=tc->threadid % end; 370 + RELEASE_LOCK(&p->mutex); 371 + if(TLSSET(p->mycache, (void *)(size_t)(n+1))) abort(); 372 + return tc; 373 + } 374 + 375 + static void *threadcache_malloc(nedpool *p, threadcache *tc, size_t *size) THROWSPEC 376 + { 377 + void *ret=0; 378 + unsigned int bestsize; 379 + unsigned int idx=size2binidx(*size); 380 + size_t blksize=0; 381 + threadcacheblk *blk, **binsptr; 382 + #ifdef FULLSANITYCHECKS 383 + tcfullsanitycheck(tc); 384 + #endif 385 + /* Calculate best fit bin size */ 386 + bestsize=1<<(idx+4); 387 + #if 0 388 + /* Finer grained bin fit */ 389 + idx<<=1; 390 + if(*size>bestsize) 391 + { 392 + idx++; 393 + bestsize+=bestsize>>1; 394 + } 395 + if(*size>bestsize) 396 + { 397 + idx++; 398 + bestsize=1<<(4+(idx>>1)); 399 + } 400 + #else 401 + if(*size>bestsize) 402 + { 403 + idx++; 404 + bestsize<<=1; 405 + } 406 + #endif 407 + assert(bestsize>=*size); 408 + if(*size<bestsize) *size=bestsize; 409 + assert(*size<=THREADCACHEMAX); 410 + assert(idx<=THREADCACHEMAXBINS); 411 + binsptr=&tc->bins[idx*2]; 412 + /* Try to match close, but move up a bin if necessary */ 413 + blk=*binsptr; 414 + if(!blk || blk->size<*size) 415 + { /* Bump it up a bin */ 416 + if(idx<THREADCACHEMAXBINS) 417 + { 418 + idx++; 419 + binsptr+=2; 420 + blk=*binsptr; 421 + } 422 + } 423 + if(blk) 424 + { 425 + blksize=blk->size; /*nedblksize(blk);*/ 426 + assert(nedblksize(blk)>=blksize); 427 + assert(blksize>=*size); 428 + if(blk->next) 429 + blk->next->prev=0; 430 + *binsptr=blk->next; 431 + if(!*binsptr) 432 + binsptr[1]=0; 433 + #ifdef FULLSANITYCHECKS 434 + blk->magic=0; 435 + #endif 436 + assert(binsptr[0]!=blk && binsptr[1]!=blk); 437 + assert(nedblksize(blk)>=sizeof(threadcacheblk) && nedblksize(blk)<=THREADCACHEMAX+CHUNK_OVERHEAD); 438 + /*printf("malloc: %p, %p, %p, %lu\n", p, tc, blk, (long) size);*/ 439 + ret=(void *) blk; 440 + } 441 + ++tc->mallocs; 442 + if(ret) 443 + { 444 + assert(blksize>=*size); 445 + ++tc->successes; 446 + tc->freeInCache-=blksize; 447 + assert((long) tc->freeInCache>=0); 448 + } 449 + #if defined(DEBUG) && 0 450 + if(!(tc->mallocs & 0xfff)) 451 + { 452 + printf("*** threadcache=%u, mallocs=%u (%f), free=%u (%f), freeInCache=%u\n", (unsigned int) tc->threadid, tc->mallocs, 453 + (float) tc->successes/tc->mallocs, tc->frees, (float) tc->successes/tc->frees, (unsigned int) tc->freeInCache); 454 + } 455 + #endif 456 + #ifdef FULLSANITYCHECKS 457 + tcfullsanitycheck(tc); 458 + #endif 459 + return ret; 460 + } 461 + static NOINLINE void ReleaseFreeInCache(nedpool *p, threadcache *tc, int mymspace) THROWSPEC 462 + { 463 + unsigned int age=THREADCACHEMAXFREESPACE/8192; 464 + /*ACQUIRE_LOCK(&p->m[mymspace]->mutex);*/ 465 + while(age && tc->freeInCache>=THREADCACHEMAXFREESPACE) 466 + { 467 + RemoveCacheEntries(p, tc, age); 468 + /*printf("*** Removing cache entries older than %u (%u)\n", age, (unsigned int) tc->freeInCache);*/ 469 + age>>=1; 470 + } 471 + /*RELEASE_LOCK(&p->m[mymspace]->mutex);*/ 472 + } 473 + static void threadcache_free(nedpool *p, threadcache *tc, int mymspace, void *mem, size_t size) THROWSPEC 474 + { 475 + unsigned int bestsize; 476 + unsigned int idx=size2binidx(size); 477 + threadcacheblk **binsptr, *tck=(threadcacheblk *) mem; 478 + assert(size>=sizeof(threadcacheblk) && size<=THREADCACHEMAX+CHUNK_OVERHEAD); 479 + #ifdef DEBUG 480 + { /* Make sure this is a valid memory block */ 481 + mchunkptr p = mem2chunk(mem); 482 + mstate fm = get_mstate_for(p); 483 + if (!ok_magic(fm)) { 484 + USAGE_ERROR_ACTION(fm, p); 485 + return; 486 + } 487 + } 488 + #endif 489 + #ifdef FULLSANITYCHECKS 490 + tcfullsanitycheck(tc); 491 + #endif 492 + /* Calculate best fit bin size */ 493 + bestsize=1<<(idx+4); 494 + #if 0 495 + /* Finer grained bin fit */ 496 + idx<<=1; 497 + if(size>bestsize) 498 + { 499 + unsigned int biggerbestsize=bestsize+bestsize<<1; 500 + if(size>=biggerbestsize) 501 + { 502 + idx++; 503 + bestsize=biggerbestsize; 504 + } 505 + } 506 + #endif 507 + if(bestsize!=size) /* dlmalloc can round up, so we round down to preserve indexing */ 508 + size=bestsize; 509 + binsptr=&tc->bins[idx*2]; 510 + assert(idx<=THREADCACHEMAXBINS); 511 + if(tck==*binsptr) 512 + { 513 + fprintf(stderr, "Attempt to free already freed memory block %p - aborting!\n", tck); 514 + abort(); 515 + } 516 + #ifdef FULLSANITYCHECKS 517 + tck->magic=*(unsigned int *) "NEDN"; 518 + #endif 519 + tck->lastUsed=++tc->frees; 520 + tck->size=(unsigned int) size; 521 + tck->next=*binsptr; 522 + tck->prev=0; 523 + if(tck->next) 524 + tck->next->prev=tck; 525 + else 526 + binsptr[1]=tck; 527 + assert(!*binsptr || (*binsptr)->size==tck->size); 528 + *binsptr=tck; 529 + assert(tck==tc->bins[idx*2]); 530 + assert(tc->bins[idx*2+1]==tck || binsptr[0]->next->prev==tck); 531 + /*printf("free: %p, %p, %p, %lu\n", p, tc, mem, (long) size);*/ 532 + tc->freeInCache+=size; 533 + #ifdef FULLSANITYCHECKS 534 + tcfullsanitycheck(tc); 535 + #endif 536 + #if 1 537 + if(tc->freeInCache>=THREADCACHEMAXFREESPACE) 538 + ReleaseFreeInCache(p, tc, mymspace); 539 + #endif 540 + } 541 + 542 + 543 + 544 + 545 + static NOINLINE int InitPool(nedpool *p, size_t capacity, int threads) THROWSPEC 546 + { /* threads is -1 for system pool */ 547 + ensure_initialization(); 548 + ACQUIRE_MALLOC_GLOBAL_LOCK(); 549 + if(p->threads) goto done; 550 + if(INITIAL_LOCK(&p->mutex)) goto err; 551 + if(TLSALLOC(&p->mycache)) goto err; 552 + if(!(p->m[0]=(mstate) create_mspace(capacity, 1))) goto err; 553 + p->m[0]->extp=p; 554 + p->threads=(threads<1 || threads>MAXTHREADSINPOOL) ? MAXTHREADSINPOOL : threads; 555 + done: 556 + RELEASE_MALLOC_GLOBAL_LOCK(); 557 + return 1; 558 + err: 559 + if(threads<0) 560 + abort(); /* If you can't allocate for system pool, we're screwed */ 561 + DestroyCaches(p); 562 + if(p->m[0]) 563 + { 564 + destroy_mspace(p->m[0]); 565 + p->m[0]=0; 566 + } 567 + if(p->mycache) 568 + { 569 + if(TLSFREE(p->mycache)) abort(); 570 + p->mycache=0; 571 + } 572 + RELEASE_MALLOC_GLOBAL_LOCK(); 573 + return 0; 574 + } 575 + static NOINLINE mstate FindMSpace(nedpool *p, threadcache *tc, int *lastUsed, size_t size) THROWSPEC 576 + { /* Gets called when thread's last used mspace is in use. The strategy 577 + is to run through the list of all available mspaces looking for an 578 + unlocked one and if we fail, we create a new one so long as we don't 579 + exceed p->threads */ 580 + int n, end; 581 + for(n=end=*lastUsed+1; p->m[n]; end=++n) 582 + { 583 + if(TRY_LOCK(&p->m[n]->mutex)) goto found; 584 + } 585 + for(n=0; n<*lastUsed && p->m[n]; n++) 586 + { 587 + if(TRY_LOCK(&p->m[n]->mutex)) goto found; 588 + } 589 + if(end<p->threads) 590 + { 591 + mstate temp; 592 + if(!(temp=(mstate) create_mspace(size, 1))) 593 + goto badexit; 594 + /* Now we're ready to modify the lists, we lock */ 595 + ACQUIRE_LOCK(&p->mutex); 596 + while(p->m[end] && end<p->threads) 597 + end++; 598 + if(end>=p->threads) 599 + { /* Drat, must destroy it now */ 600 + RELEASE_LOCK(&p->mutex); 601 + destroy_mspace((mspace) temp); 602 + goto badexit; 603 + } 604 + /* We really want to make sure this goes into memory now but we 605 + have to be careful of breaking aliasing rules, so write it twice */ 606 + *((volatile struct malloc_state **) &p->m[end])=p->m[end]=temp; 607 + ACQUIRE_LOCK(&p->m[end]->mutex); 608 + /*printf("Created mspace idx %d\n", end);*/ 609 + RELEASE_LOCK(&p->mutex); 610 + n=end; 611 + goto found; 612 + } 613 + /* Let it lock on the last one it used */ 614 + badexit: 615 + ACQUIRE_LOCK(&p->m[*lastUsed]->mutex); 616 + return p->m[*lastUsed]; 617 + found: 618 + *lastUsed=n; 619 + if(tc) 620 + tc->mymspace=n; 621 + else 622 + { 623 + if(TLSSET(p->mycache, (void *)(size_t)(-(n+1)))) abort(); 624 + } 625 + return p->m[n]; 626 + } 627 + 628 + nedpool *nedcreatepool(size_t capacity, int threads) THROWSPEC 629 + { 630 + nedpool *ret; 631 + if(!(ret=(nedpool *) nedpcalloc(0, 1, sizeof(nedpool)))) return 0; 632 + if(!InitPool(ret, capacity, threads)) 633 + { 634 + nedpfree(0, ret); 635 + return 0; 636 + } 637 + return ret; 638 + } 639 + void neddestroypool(nedpool *p) THROWSPEC 640 + { 641 + int n; 642 + ACQUIRE_LOCK(&p->mutex); 643 + DestroyCaches(p); 644 + for(n=0; p->m[n]; n++) 645 + { 646 + destroy_mspace(p->m[n]); 647 + p->m[n]=0; 648 + } 649 + RELEASE_LOCK(&p->mutex); 650 + if(TLSFREE(p->mycache)) abort(); 651 + nedpfree(0, p); 652 + } 653 + 654 + void nedpsetvalue(nedpool *p, void *v) THROWSPEC 655 + { 656 + if(!p) { p=&syspool; if(!syspool.threads) InitPool(&syspool, 0, -1); } 657 + p->uservalue=v; 658 + } 659 + void *nedgetvalue(nedpool **p, void *mem) THROWSPEC 660 + { 661 + nedpool *np=0; 662 + mchunkptr mcp=mem2chunk(mem); 663 + mstate fm; 664 + if(!(is_aligned(chunk2mem(mcp))) && mcp->head != FENCEPOST_HEAD) return 0; 665 + if(!cinuse(mcp)) return 0; 666 + if(!next_pinuse(mcp)) return 0; 667 + if(!is_mmapped(mcp) && !pinuse(mcp)) 668 + { 669 + if(next_chunk(prev_chunk(mcp))!=mcp) return 0; 670 + } 671 + fm=get_mstate_for(mcp); 672 + if(!ok_magic(fm)) return 0; 673 + if(!ok_address(fm, mcp)) return 0; 674 + if(!fm->extp) return 0; 675 + np=(nedpool *) fm->extp; 676 + if(p) *p=np; 677 + return np->uservalue; 678 + } 679 + 680 + void neddisablethreadcache(nedpool *p) THROWSPEC 681 + { 682 + int mycache; 683 + if(!p) 684 + { 685 + p=&syspool; 686 + if(!syspool.threads) InitPool(&syspool, 0, -1); 687 + } 688 + mycache=(int)(size_t) TLSGET(p->mycache); 689 + if(!mycache) 690 + { /* Set to mspace 0 */ 691 + if(TLSSET(p->mycache, (void *)-1)) abort(); 692 + } 693 + else if(mycache>0) 694 + { /* Set to last used mspace */ 695 + threadcache *tc=p->caches[mycache-1]; 696 + #if defined(DEBUG) 697 + printf("Threadcache utilisation: %lf%% in cache with %lf%% lost to other threads\n", 698 + 100.0*tc->successes/tc->mallocs, 100.0*((double) tc->mallocs-tc->frees)/tc->mallocs); 699 + #endif 700 + if(TLSSET(p->mycache, (void *)(size_t)(-tc->mymspace))) abort(); 701 + tc->frees++; 702 + RemoveCacheEntries(p, tc, 0); 703 + assert(!tc->freeInCache); 704 + tc->mymspace=-1; 705 + tc->threadid=0; 706 + mspace_free(0, p->caches[mycache-1]); 707 + p->caches[mycache-1]=0; 708 + } 709 + } 710 + 711 + #define GETMSPACE(m,p,tc,ms,s,action) \ 712 + do \ 713 + { \ 714 + mstate m = GetMSpace((p),(tc),(ms),(s)); \ 715 + action; \ 716 + RELEASE_LOCK(&m->mutex); \ 717 + } while (0) 718 + 719 + static FORCEINLINE mstate GetMSpace(nedpool *p, threadcache *tc, int mymspace, size_t size) THROWSPEC 720 + { /* Returns a locked and ready for use mspace */ 721 + mstate m=p->m[mymspace]; 722 + assert(m); 723 + if(!TRY_LOCK(&p->m[mymspace]->mutex)) m=FindMSpace(p, tc, &mymspace, size);\ 724 + /*assert(IS_LOCKED(&p->m[mymspace]->mutex));*/ 725 + return m; 726 + } 727 + static FORCEINLINE void GetThreadCache(nedpool **p, threadcache **tc, int *mymspace, size_t *size) THROWSPEC 728 + { 729 + int mycache; 730 + if(size && *size<sizeof(threadcacheblk)) *size=sizeof(threadcacheblk); 731 + if(!*p) 732 + { 733 + *p=&syspool; 734 + if(!syspool.threads) InitPool(&syspool, 0, -1); 735 + } 736 + mycache=(int)(size_t) TLSGET((*p)->mycache); 737 + if(mycache>0) 738 + { 739 + *tc=(*p)->caches[mycache-1]; 740 + *mymspace=(*tc)->mymspace; 741 + } 742 + else if(!mycache) 743 + { 744 + *tc=AllocCache(*p); 745 + if(!*tc) 746 + { /* Disable */ 747 + if(TLSSET((*p)->mycache, (void *)-1)) abort(); 748 + *mymspace=0; 749 + } 750 + else 751 + *mymspace=(*tc)->mymspace; 752 + } 753 + else 754 + { 755 + *tc=0; 756 + *mymspace=-mycache-1; 757 + } 758 + assert(*mymspace>=0); 759 + assert((long)(size_t)CURRENT_THREAD==(*tc)->threadid); 760 + #ifdef FULLSANITYCHECKS 761 + if(*tc) 762 + { 763 + if(*(unsigned int *)"NEDMALC1"!=(*tc)->magic1 || *(unsigned int *)"NEDMALC2"!=(*tc)->magic2) 764 + { 765 + abort(); 766 + } 767 + } 768 + #endif 769 + } 770 + 771 + void * nedpmalloc(nedpool *p, size_t size) THROWSPEC 772 + { 773 + void *ret=0; 774 + threadcache *tc; 775 + int mymspace; 776 + GetThreadCache(&p, &tc, &mymspace, &size); 777 + #if THREADCACHEMAX 778 + if(tc && size<=THREADCACHEMAX) 779 + { /* Use the thread cache */ 780 + ret=threadcache_malloc(p, tc, &size); 781 + } 782 + #endif 783 + if(!ret) 784 + { /* Use this thread's mspace */ 785 + GETMSPACE(m, p, tc, mymspace, size, 786 + ret=mspace_malloc(m, size)); 787 + } 788 + return ret; 789 + } 790 + void * nedpcalloc(nedpool *p, size_t no, size_t size) THROWSPEC 791 + { 792 + size_t rsize=size*no; 793 + void *ret=0; 794 + threadcache *tc; 795 + int mymspace; 796 + GetThreadCache(&p, &tc, &mymspace, &rsize); 797 + #if THREADCACHEMAX 798 + if(tc && rsize<=THREADCACHEMAX) 799 + { /* Use the thread cache */ 800 + if((ret=threadcache_malloc(p, tc, &rsize))) 801 + memset(ret, 0, rsize); 802 + } 803 + #endif 804 + if(!ret) 805 + { /* Use this thread's mspace */ 806 + GETMSPACE(m, p, tc, mymspace, rsize, 807 + ret=mspace_calloc(m, 1, rsize)); 808 + } 809 + return ret; 810 + } 811 + void * nedprealloc(nedpool *p, void *mem, size_t size) THROWSPEC 812 + { 813 + void *ret=0; 814 + threadcache *tc; 815 + int mymspace; 816 + if(!mem) return nedpmalloc(p, size); 817 + GetThreadCache(&p, &tc, &mymspace, &size); 818 + #if THREADCACHEMAX 819 + if(tc && size && size<=THREADCACHEMAX) 820 + { /* Use the thread cache */ 821 + size_t memsize=nedblksize(mem); 822 + assert(memsize); 823 + if((ret=threadcache_malloc(p, tc, &size))) 824 + { 825 + memcpy(ret, mem, memsize<size ? memsize : size); 826 + if(memsize<=THREADCACHEMAX) 827 + threadcache_free(p, tc, mymspace, mem, memsize); 828 + else 829 + mspace_free(0, mem); 830 + } 831 + } 832 + #endif 833 + if(!ret) 834 + { /* Reallocs always happen in the mspace they happened in, so skip 835 + locking the preferred mspace for this thread */ 836 + ret=mspace_realloc(0, mem, size); 837 + } 838 + return ret; 839 + } 840 + void nedpfree(nedpool *p, void *mem) THROWSPEC 841 + { /* Frees always happen in the mspace they happened in, so skip 842 + locking the preferred mspace for this thread */ 843 + threadcache *tc; 844 + int mymspace; 845 + size_t memsize; 846 + assert(mem); 847 + GetThreadCache(&p, &tc, &mymspace, 0); 848 + #if THREADCACHEMAX 849 + memsize=nedblksize(mem); 850 + assert(memsize); 851 + if(mem && tc && memsize<=(THREADCACHEMAX+CHUNK_OVERHEAD)) 852 + threadcache_free(p, tc, mymspace, mem, memsize); 853 + else 854 + #endif 855 + mspace_free(0, mem); 856 + } 857 + void * nedpmemalign(nedpool *p, size_t alignment, size_t bytes) THROWSPEC 858 + { 859 + void *ret; 860 + threadcache *tc; 861 + int mymspace; 862 + GetThreadCache(&p, &tc, &mymspace, &bytes); 863 + { /* Use this thread's mspace */ 864 + GETMSPACE(m, p, tc, mymspace, bytes, 865 + ret=mspace_memalign(m, alignment, bytes)); 866 + } 867 + return ret; 868 + } 869 + #if !NO_MALLINFO 870 + struct mallinfo nedpmallinfo(nedpool *p) THROWSPEC 871 + { 872 + int n; 873 + struct mallinfo ret={0}; 874 + if(!p) { p=&syspool; if(!syspool.threads) InitPool(&syspool, 0, -1); } 875 + for(n=0; p->m[n]; n++) 876 + { 877 + struct mallinfo t=mspace_mallinfo(p->m[n]); 878 + ret.arena+=t.arena; 879 + ret.ordblks+=t.ordblks; 880 + ret.hblkhd+=t.hblkhd; 881 + ret.usmblks+=t.usmblks; 882 + ret.uordblks+=t.uordblks; 883 + ret.fordblks+=t.fordblks; 884 + ret.keepcost+=t.keepcost; 885 + } 886 + return ret; 887 + } 888 + #endif 889 + int nedpmallopt(nedpool *p, int parno, int value) THROWSPEC 890 + { 891 + return mspace_mallopt(parno, value); 892 + } 893 + int nedpmalloc_trim(nedpool *p, size_t pad) THROWSPEC 894 + { 895 + int n, ret=0; 896 + if(!p) { p=&syspool; if(!syspool.threads) InitPool(&syspool, 0, -1); } 897 + for(n=0; p->m[n]; n++) 898 + { 899 + ret+=mspace_trim(p->m[n], pad); 900 + } 901 + return ret; 902 + } 903 + void nedpmalloc_stats(nedpool *p) THROWSPEC 904 + { 905 + int n; 906 + if(!p) { p=&syspool; if(!syspool.threads) InitPool(&syspool, 0, -1); } 907 + for(n=0; p->m[n]; n++) 908 + { 909 + mspace_malloc_stats(p->m[n]); 910 + } 911 + } 912 + size_t nedpmalloc_footprint(nedpool *p) THROWSPEC 913 + { 914 + size_t ret=0; 915 + int n; 916 + if(!p) { p=&syspool; if(!syspool.threads) InitPool(&syspool, 0, -1); } 917 + for(n=0; p->m[n]; n++) 918 + { 919 + ret+=mspace_footprint(p->m[n]); 920 + } 921 + return ret; 922 + } 923 + void **nedpindependent_calloc(nedpool *p, size_t elemsno, size_t elemsize, void **chunks) THROWSPEC 924 + { 925 + void **ret; 926 + threadcache *tc; 927 + int mymspace; 928 + GetThreadCache(&p, &tc, &mymspace, &elemsize); 929 + GETMSPACE(m, p, tc, mymspace, elemsno*elemsize, 930 + ret=mspace_independent_calloc(m, elemsno, elemsize, chunks)); 931 + return ret; 932 + } 933 + void **nedpindependent_comalloc(nedpool *p, size_t elems, size_t *sizes, void **chunks) THROWSPEC 934 + { 935 + void **ret; 936 + threadcache *tc; 937 + int mymspace; 938 + size_t i, *adjustedsizes=(size_t *) alloca(elems*sizeof(size_t)); 939 + if(!adjustedsizes) return 0; 940 + for(i=0; i<elems; i++) 941 + adjustedsizes[i]=sizes[i]<sizeof(threadcacheblk) ? sizeof(threadcacheblk) : sizes[i]; 942 + GetThreadCache(&p, &tc, &mymspace, 0); 943 + GETMSPACE(m, p, tc, mymspace, 0, 944 + ret=mspace_independent_comalloc(m, elems, adjustedsizes, chunks)); 945 + return ret; 946 + } 947 + 948 + #ifdef OVERRIDE_STRDUP 949 + /* 950 + * This implementation is purely there to override the libc version, to 951 + * avoid a crash due to allocation and free on different 'heaps'. 952 + */ 953 + char *strdup(const char *s1) 954 + { 955 + char *s2 = 0; 956 + if (s1) { 957 + s2 = malloc(strlen(s1) + 1); 958 + strcpy(s2, s1); 959 + } 960 + return s2; 961 + } 962 + #endif 963 + 964 + #if defined(__cplusplus) 965 + } 966 + #endif

+180

compat/nedmalloc/nedmalloc.h

··· 1 + /* nedalloc, an alternative malloc implementation for multiple threads without 2 + lock contention based on dlmalloc v2.8.3. (C) 2005 Niall Douglas 3 + 4 + Boost Software License - Version 1.0 - August 17th, 2003 5 + 6 + Permission is hereby granted, free of charge, to any person or organization 7 + obtaining a copy of the software and accompanying documentation covered by 8 + this license (the "Software") to use, reproduce, display, distribute, 9 + execute, and transmit the Software, and to prepare derivative works of the 10 + Software, and to permit third-parties to whom the Software is furnished to 11 + do so, all subject to the following: 12 + 13 + The copyright notices in the Software and this entire statement, including 14 + the above license grant, this restriction and the following disclaimer, 15 + must be included in all copies of the Software, in whole or in part, and 16 + all derivative works of the Software, unless such copies or derivative 17 + works are solely in the form of machine-executable object code generated by 18 + a source language processor. 19 + 20 + THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 21 + IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 22 + FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT 23 + SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE 24 + FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE, 25 + ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER 26 + DEALINGS IN THE SOFTWARE. 27 + */ 28 + 29 + #ifndef NEDMALLOC_H 30 + #define NEDMALLOC_H 31 + 32 + 33 + /* See malloc.c.h for what each function does. 34 + 35 + REPLACE_SYSTEM_ALLOCATOR causes nedalloc's functions to be called malloc, 36 + free etc. instead of nedmalloc, nedfree etc. You may or may not want this. 37 + 38 + NO_NED_NAMESPACE prevents the functions from being defined in the nedalloc 39 + namespace when in C++ (uses the global namespace instead). 40 + 41 + EXTSPEC can be defined to be __declspec(dllexport) or 42 + __attribute__ ((visibility("default"))) or whatever you like. It defaults 43 + to extern. 44 + 45 + USE_LOCKS can be 2 if you want to define your own MLOCK_T, INITIAL_LOCK, 46 + ACQUIRE_LOCK, RELEASE_LOCK, TRY_LOCK, IS_LOCKED and NULL_LOCK_INITIALIZER. 47 + 48 + */ 49 + 50 + #include <stddef.h> /* for size_t */ 51 + 52 + #ifndef EXTSPEC 53 + #define EXTSPEC extern 54 + #endif 55 + 56 + #if defined(_MSC_VER) && _MSC_VER>=1400 57 + #define MALLOCATTR __declspec(restrict) 58 + #endif 59 + #ifdef __GNUC__ 60 + #define MALLOCATTR __attribute__ ((malloc)) 61 + #endif 62 + #ifndef MALLOCATTR 63 + #define MALLOCATTR 64 + #endif 65 + 66 + #ifdef REPLACE_SYSTEM_ALLOCATOR 67 + #define nedmalloc malloc 68 + #define nedcalloc calloc 69 + #define nedrealloc realloc 70 + #define nedfree free 71 + #define nedmemalign memalign 72 + #define nedmallinfo mallinfo 73 + #define nedmallopt mallopt 74 + #define nedmalloc_trim malloc_trim 75 + #define nedmalloc_stats malloc_stats 76 + #define nedmalloc_footprint malloc_footprint 77 + #define nedindependent_calloc independent_calloc 78 + #define nedindependent_comalloc independent_comalloc 79 + #ifdef _MSC_VER 80 + #define nedblksize _msize 81 + #endif 82 + #endif 83 + 84 + #ifndef NO_MALLINFO 85 + #define NO_MALLINFO 0 86 + #endif 87 + 88 + #if !NO_MALLINFO 89 + struct mallinfo; 90 + #endif 91 + 92 + #if defined(__cplusplus) 93 + #if !defined(NO_NED_NAMESPACE) 94 + namespace nedalloc { 95 + #else 96 + extern "C" { 97 + #endif 98 + #define THROWSPEC throw() 99 + #else 100 + #define THROWSPEC 101 + #endif 102 + 103 + /* These are the global functions */ 104 + 105 + /* Gets the usable size of an allocated block. Note this will always be bigger than what was 106 + asked for due to rounding etc. 107 + */ 108 + EXTSPEC size_t nedblksize(void *mem) THROWSPEC; 109 + 110 + EXTSPEC void nedsetvalue(void *v) THROWSPEC; 111 + 112 + EXTSPEC MALLOCATTR void * nedmalloc(size_t size) THROWSPEC; 113 + EXTSPEC MALLOCATTR void * nedcalloc(size_t no, size_t size) THROWSPEC; 114 + EXTSPEC MALLOCATTR void * nedrealloc(void *mem, size_t size) THROWSPEC; 115 + EXTSPEC void nedfree(void *mem) THROWSPEC; 116 + EXTSPEC MALLOCATTR void * nedmemalign(size_t alignment, size_t bytes) THROWSPEC; 117 + #if !NO_MALLINFO 118 + EXTSPEC struct mallinfo nedmallinfo(void) THROWSPEC; 119 + #endif 120 + EXTSPEC int nedmallopt(int parno, int value) THROWSPEC; 121 + EXTSPEC int nedmalloc_trim(size_t pad) THROWSPEC; 122 + EXTSPEC void nedmalloc_stats(void) THROWSPEC; 123 + EXTSPEC size_t nedmalloc_footprint(void) THROWSPEC; 124 + EXTSPEC MALLOCATTR void **nedindependent_calloc(size_t elemsno, size_t elemsize, void **chunks) THROWSPEC; 125 + EXTSPEC MALLOCATTR void **nedindependent_comalloc(size_t elems, size_t *sizes, void **chunks) THROWSPEC; 126 + 127 + /* These are the pool functions */ 128 + struct nedpool_t; 129 + typedef struct nedpool_t nedpool; 130 + 131 + /* Creates a memory pool for use with the nedp* functions below. 132 + Capacity is how much to allocate immediately (if you know you'll be allocating a lot 133 + of memory very soon) which you can leave at zero. Threads specifies how many threads 134 + will *normally* be accessing the pool concurrently. Setting this to zero means it 135 + extends on demand, but be careful of this as it can rapidly consume system resources 136 + where bursts of concurrent threads use a pool at once. 137 + */ 138 + EXTSPEC MALLOCATTR nedpool *nedcreatepool(size_t capacity, int threads) THROWSPEC; 139 + 140 + /* Destroys a memory pool previously created by nedcreatepool(). 141 + */ 142 + EXTSPEC void neddestroypool(nedpool *p) THROWSPEC; 143 + 144 + /* Sets a value to be associated with a pool. You can retrieve this value by passing 145 + any memory block allocated from that pool. 146 + */ 147 + EXTSPEC void nedpsetvalue(nedpool *p, void *v) THROWSPEC; 148 + /* Gets a previously set value using nedpsetvalue() or zero if memory is unknown. 149 + Optionally can also retrieve pool. 150 + */ 151 + EXTSPEC void *nedgetvalue(nedpool **p, void *mem) THROWSPEC; 152 + 153 + /* Disables the thread cache for the calling thread, returning any existing cache 154 + data to the central pool. 155 + */ 156 + EXTSPEC void neddisablethreadcache(nedpool *p) THROWSPEC; 157 + 158 + EXTSPEC MALLOCATTR void * nedpmalloc(nedpool *p, size_t size) THROWSPEC; 159 + EXTSPEC MALLOCATTR void * nedpcalloc(nedpool *p, size_t no, size_t size) THROWSPEC; 160 + EXTSPEC MALLOCATTR void * nedprealloc(nedpool *p, void *mem, size_t size) THROWSPEC; 161 + EXTSPEC void nedpfree(nedpool *p, void *mem) THROWSPEC; 162 + EXTSPEC MALLOCATTR void * nedpmemalign(nedpool *p, size_t alignment, size_t bytes) THROWSPEC; 163 + #if !NO_MALLINFO 164 + EXTSPEC struct mallinfo nedpmallinfo(nedpool *p) THROWSPEC; 165 + #endif 166 + EXTSPEC int nedpmallopt(nedpool *p, int parno, int value) THROWSPEC; 167 + EXTSPEC int nedpmalloc_trim(nedpool *p, size_t pad) THROWSPEC; 168 + EXTSPEC void nedpmalloc_stats(nedpool *p) THROWSPEC; 169 + EXTSPEC size_t nedpmalloc_footprint(nedpool *p) THROWSPEC; 170 + EXTSPEC MALLOCATTR void **nedpindependent_calloc(nedpool *p, size_t elemsno, size_t elemsize, void **chunks) THROWSPEC; 171 + EXTSPEC MALLOCATTR void **nedpindependent_comalloc(nedpool *p, size_t elems, size_t *sizes, void **chunks) THROWSPEC; 172 + 173 + #if defined(__cplusplus) 174 + } 175 + #endif 176 + 177 + #undef MALLOCATTR 178 + #undef EXTSPEC 179 + 180 + #endif