README for MEMWATCH 2.67

***** Purpose:

 MEMWATCH has been written to allow programmer to program 
 in C a memory error control product.
 The idea is that you use it during the development phase
 and then remove the MEMWATCH define to produce your final 
 product.MEMWATCH is distributed in source code form in order
 to allow you to compile it for your platform with your own 
 compiler.It's aim is to be 100% ANSI C, but some compilers 
 are more stingy than others. 

***** How it work:
 
 Using the C preprocessor, memwatch replaces all your programs
 calls to ANSI C memory allocation functions with calls to 
 it's own functions, which keeps a record of all allocations.

 Memwatch is very unobtrusive unless the define MEMWATCH is
 defined, memwatch removes all traces of itself from the code
 (using the preprocessor).Memwatch normally writes it's data
 to the file memwatch.log, but this can be overridden;
 
 You can capture output from MEMWATCH using mwSetOutFunc().           
 Just give it the adress of a "void myOutFunc(int c)" function,       
 and all characters to be output will be redirected there.            
                                                                     
 A failing ASSERT() or VERIFY() will normally always abort your       
 program. This can be changed using mwSetAriFunc(). Give it a         
 pointer to a "int myAriFunc(const char *)" function. Your function   
 must ask the user whether to Abort, Retry or Ignore the trap.        
 Return 2 to Abort, 1 to Retry or 0 to Ignore. Beware retry; it       
 causes the expression to be evaluated again! MEMWATCH has a          
 default ARI handler. It's disabled by default, but you can enable    
 it by calling 'mwDefaultAri()'. Note that this will STILL abort      
 to use the standard C I/O streams. Also, setting the ARI function    
 will cause MEMWATCH *NOT* to write the ARI error to stderr. The      
 error string is passed to the ARI function instead, as the           
 'const char *' parameter.                                            
                                                                     
 You can disable MEMWATCH's ASSERT/VERIFY and/or TRACE implementations
 This can be useful if you're using a debug terminal or smart debugger
 Disable them by defining MW_NOASSERT, MW_NOVERIFY or MW_NOTRACE.     
                                                                     
 MEMWATCH fills all allocated memory with the byte 0xFE, so if        
 you're looking at erroneous data which are all 0xFE:s, the           
 data probably was not initialized by you. The exception is           
 calloc(), which will fill with zero's. All freed buffers are         
 zapped with 0xFD. If this is what you look at, you're using          
 data that has been freed. If this is the case, be aware that         
 MEMWATCH places a 'free'd block info' structure immediately          
 before the freed data. This block contains info about where          
 the block was freed. The information is in readable text,            
 in the format "FBI<counter>filename(line)", for example:             
 "FBI<267>test.c(12)". Using FBI's slows down free(), so it's         
 disabled by default. Use mwFreeBufferInfo(1) to enable it.           
                                                                     
 To aid in tracking down wild pointer writes, MEMWATCH can perform    
 no-mans-land allocations. No-mans-land will contain the byte 0xFC.   
 MEMWATCH will, when this is enabled, convert recently free'd memory  
 into NML allocations.                                                
                                                                     
 MEMWATCH protects it's own data buffers with checksums. If you       
 get an internal error, it means you're overwriting wildly,           
 or using an uninitialized pointer.                                   

***** More detail:
 
C++ support
 
 You can use C++ source, but it's not recommended. C++ 
 allows individual classes to have their own memory management,
 and the preprocessor approach used by memwatch can cause 
 havoc with such class declarations if improperly used.

 If you have no such classes, or have them but still want
 to test it, you can give it a try.

 First, re-enable the C++ support code in memwatch.
 If you can't find it, you probably shouldn't be using
 it. Then, in your source code, after including ALL
 header files:

	#define new mwNew
	#define delete mwDelete

 This will cause calls to new and delete in that source file 
 to be directed to memwatch. Also, be sure to read all the
 text in memwatch.h regarding C++ support.

Initialization and cleanup

 In order to do it's work in a timely fashion, memwatch
 needs to do some startup and cleanup work. mwInit()
 initializes memwatch and mwTerm() terminates it. Memwatch
 can auto-initialize, and will do so if you don't call
 mwInit() yourself. If this is the case, memwatch will use
 atexit() to register mwTerm() to the atexit-queue.

 The auto-init technique has a caveat; if you are using
 atexit() yourself to do cleanup work, memwatch may
 terminate before your program is done. To be on the safe
 side, use mwInit() and mwTerm().

 mwInit() and mwTerm() is nestable, so you can call mwInit()
 several times, requiring mwTerm() to be called an equal
 number of times to terminate memwatch.

 In case of the program aborting in a controlled way, you
 may want to call mwAbort() instead of mwTerm(). mwAbort()
 will terminate memwatch even if there are outstanding calls
 to mwTerm().

I/O operations

 During normal operations, memwatch creates a file named
 memwatch.log. Sometimes, memwatch.log can't be created;
 then memwatch tries to create files name memwatNN.log,
 where NN is between 01 and 99. If that fails, no log will
 be produced.

 If you can't use a file log, or don't want to, no worry.
 Just call mwSetOutFunc() with the address of a "void
 func(int c)" function, and all output will be directed
 there, character by character.

 Memwatch also has an Abort/Retry/Ignore handler that is
 used when an ASSERT or VERIFY fails. The default handler
 does no I/O, but automatically aborts the program. You can
 use any handler you want; just send the address of a "int
 func(const char*)" to mwSetAriFunc(). For more details on
 that, see memwatch.h.

TRACE/ASSERT/VERIFY macros

 Memwatch defines (if not already defined) the macros TRACE,
 ASSERT and VERIFY. If you are already using macros with
 these names, memwatch 2.61 and later will not override
 them. Memwatch 2.61 and later will also always define the
 macros mwTRACE, mwASSERT and mwVERIFY, so you can use these
 to make sure you're talking to memwatch. Versions previous
 to 2.61 will *OVERRIDE* existing TRACE, ASSERT and VERIFY.

 To make sure that existing TRACE, ASSERT and VERIFY macros
 are preserved, you can define MW_NOTRACE, MW_NOASSERT and
 MW_NOVERIFY. All versions of memwatch will abide by these.

How slow can you go?

 Memwatch slows things down. Large allocations aren't
 affected so that you can measure it, but small allocations
 that would utilize a compilers small-allocator function
 suddenly cannot, and so gets slowed down alot. As a worst
 case, expect it to be 3-5 times slower.

 Free'ing gets hit worse, I'm afraid, as memwatch checks
 a lot of stuff when freeing. Expect it to be 5-7 times
 slower, no matter what the size of the allocation.

Stress-testing the application

 You can simulate low-memory conditions using mwLimit().
 mwLimit() takes the maximum number of bytes to be
 allocated; when the limit is hit, allocation requests will
 fail, and a "limit" message will be logged.

 If you hit a real low-memory situation, memwatch logs that
 too. Memwatch itself has some reserve memory tucked away so
 it should continue running even in the worst conditions.

Hunting down wild writes and other Nasty Things

 Wild writes are usually caused by using pointers that arent
 initialized, or that were initialized, but then the memory
 they points to is moved or freed. The best way to avoid
 these kind of problems is to ALWAYS initialize pointers to
 NULL, and after freeing a memory buffer, setting all
 pointers that pointed to it to NULL.

 To aid in tracking down uninitialized pointers memwatch
 zaps all memory with certain values. Recently allocated
 memory (unless calloc'd, of course), contains 0xFE.
 Recently freed memory contains 0xFD. So if your program
 crashes when using memwatch and not without memwatch, it's
 most likely because you are not initializing your allocated
 buffers, or using the buffers after they've been freed.

 In the event that a wild pointer should damage memwatch's
 internal data structures, memwatch employs checksums,
 multiple copies of some values, and can also repair it's
 own data structures.

 If you are a paranoid person, and as programmer you should
 be, you can use memwatch's mwIsReadAddr() and
 mwIsSafeAddr() functions to check the accessibility of
 memory. These are implemented for both ANSI C systems and
 Win32 systems. Just put an mwASSERT() around the check and
 forget about it.
 
***** Error type:
 
Format of error messages in MEMWATCH.LOG:
 
 message: <sequence-number> filename(linenumber), information
 
 Errors caught by MemWatch, when they are detected, and any
 actions taken besides writing to the log file MEMWATCH.LOG:

Double-freeing:
 
 A pointer that was recently freed and has not since been
 reused was freed again. The place where the previous free()
 was executed is displayed.
 
 Detect: delete or free() using the offending pointer.
 Action: The delete or free() is cancelled, execution continues.
 
Underflow:
      
 You have written just ahead of the allocated memory.
 The size and place of the allocation is displayed.
 
 Detect: delete or free() of the damaged buffer.
 Action: The buffer is freed, but there may be secondary damage.
 
Overflow:
 Like underflow, but you've written after the end of the buffer.
 
 Detect: see Underflow.
 Action: see Underflow.

WILD free:
 
 An unrecognized pointer was passed to delete or free().
 The pointer may have been returned from a library function;
 in that case, use mwFree_() to force free() of it.
 Also, this may be a double-free, but the previous free was
 too long ago, causing MEMWATCH to 'forget' it.
 
 Detect: delete or free() of the offending pointer.
 Action: The delete or free() is cancelled, execution continues.

NULL free:
 
 It's unclear to me whether or not freeing of NULL pointers
 is legal in ANSI C, therefore a warning is written to the log file,
 but the error counter remains the same. This is legal using C++,
 so the warning does not appear with delete.
 
 Detect: When you free(NULL).
 Action: The free() is cancelled.

Failed:
 
 A request to allocate memory failed. If the allocation is
 small, this may be due to memory depletion, but is more likely
 to be memory fragmentation problems. The amount of memory
 allocated so far is displayed also.
 
 Detect: When you new, malloc(), realloc() or calloc() memory.
 Action: NULL is returned.

Realloc:
 
 A request to re-allocate a memory buffer failed for reasons
 other than out-of-memory. The specific reason is shown.
     
 Detect: When you realloc()
 Action: realloc() is cancelled, NULL is returned

Limit fail:
 
 A request to allocate memory failed since it would violate
 the limit set using mwLimit(). mwLimit() is used to stress-test
 your code under simulated low memory conditions.
     
 Detect: At new, malloc(), realloc() or calloc().
 Action: NULL is returned.

Assert trap:
     
 An ASSERT() failed. The ASSERT() macro works like C's assert()
 macro/function, except that it's interactive. See your C manual.
     
 Detect: On the ASSERT().
 Action: Program ends with an advisory message to stderr, OR
         Program writes the ASSERT to the log and continues, OR
         Program asks Abort/Retry/Ignore? and takes that action.

Verify trap:
     
 A VERIFY() failed. The VERIFY() macro works like ASSERT(),
 but if MEMWATCH is not defined, it still evaluates the
 expression, but it does not act upon the result.
 
 Detect: On the VERIFY().
 Action: Program ends with an advisory message to stderr, OR
         Program writes the VERIFY to the log and continues, OR
         Program asks Abort/Retry/Ignore? and takes that action.
Wild pointer:
     
 A no-mans-land buffer has been written into. MEMWATCH can
 allocate and distribute chunks of memory solely for the
 purpose of trying to catch random writes into memory.
     
 Detect: Always on CHECK(), but can be detected in several places.
 Action: The error is logged, and if an ARI handler is installed,
         it is executed, otherwise, execution continues.

Unfreed:
    
 A memory buffer you allocated has not been freed.
 You are informed where it was allocated, and whether any
 over or underflow has occured. MemWatch also displays up to
 16 bytes of the data, as much as it can, in hex and text.
 
 Detect: When MemWatch terminates.
 Action: The buffer is freed.

Check:
    
 An error was detected during a CHECK() operation.
 The associated pointer is displayed along with
 the file and line where the CHECK() was executed.
 Followed immediately by a normal error message.
    
 Detect: When you CHECK()
 Action: Depends on the error

Relink:
 
 After a MEMWATCH internal control block has been trashed,
 MEMWATCH tries to repair the damage. If successful, program
 execution will continue instead of aborting. Some information
 about the block may be gone permanently, though.
    
 Detect: N/A
 Action: Relink successful: program continues.
         Relink fails: program aborts.

Internal:

 An internal error is flagged by MEMWATCH when it's control
 structures have been damaged. You are likely using an uninitialized
 pointer somewhere in your program, or are zapping memory all over.
 The message may give you additional diagnostic information.
 If possible, MEMWATCH will recover and continue execution.
    
 Detect: Various actions.
 Action: Whatever is needed

Mark:
    
 The program terminated without umarking all marked pointers. Marking
 can be used to track resources other than memory. mwMark(pointer,text,...)
 when the resource is allocated, and mwUnmark(pointer) when it's freed.
 The 'text' is displayed for still marked pointers when the program
 ends.
    
 Detect: When MemWatch terminates.
 Action: The error is logged.


***** Quick-start instructions:
 
 1. Make sure that memwatch.h is included in all of the
 source code files. If you have an include file that
 all of the source code uses, you might be able to include
 memwatch.h from there.
 
 2. Recompile the program with MEMWATCH defined. See your
 compiler's documentation if you don't know how to do this.
 The usual switch looks like "-DMEMWATCH". To have MEMWATCH
 use stderr for some output (like, "Abort, Retry, Ignore?"),
 please also define MW_STDIO (or MEMWATCH_STDIO, same thing).
 
 3. Run the program and examine the output in the
 log file "memwatch.log". If you didn't get a log file,
 you probably didn't do step 1 and 2 correctly, or your
 program crashed before memwatch flushed the file buffer.
 To have memwatch _always_ flush the buffer, add a call
 to "mwDoFlush(1)" at the top of your main function.
 
 4. There is no fourth step... but remember that there
 are limits to what memwatch can do, and that you need
 to be aware of them:
 
***** Limits to memwatch:
 
 Memwatch cannot catch all wild pointer writes. It can catch
 those it could make itself due to your program trashing
 memwatch's internal data structures. It can catch, sort of,
 wild writes into No Mans Land buffers (see the header file for
 more info). Anything else and you're going to get core dumped,
 or data corruption if you're lucky.
 
 There are other limits of course, but that one is the most
 serious one, and the one that you're likely to be suffering
 from.
 
***** Faq:
 
 Frequently Asked Questions for memwatch

Q. I'm not getting any log file! What's wrong??

A. Did you define MEMWATCH when compiling all files?
   Did you include memwatch.h in all the files?
   If you did, then...:

   Memwatch creates the file when it initializes. If you're not
   getting the log file, it's because a) memwatch is not
   initializing or b) it's initializing, but can't create the
   file.
   
   Memwatch has two functions, mwInit() and mwTerm(), that
   initialize and terminate memwatch, respectively. They are
   nestable. You USUALLY don't need to call mwInit() and
   mwTerm(), since memwatch will auto-initialize on the first
   call to a memory function, and then add mwTerm() to the
   atexit() list.
   
   You can call mwInit() and mwTerm() manually, if it's not
   initializing properly or if your system doesn't support
   atexit(). Call mwInit() as soon as you can, and mwTerm() at
   the logical no-error ending of your program. Call mwAbort()
   if the program is stopping due to an error; this will
   terminate memwatch even if more than one call to mwTerm() is
   outstanding.
   
   If you are using C++, remember that global and static C++
   objects constructors execute before main() when considering
   where to put mwInit(). Also, their destructors execute after
   main(). You may want to create a global object very early
   with mwInit() in the constructor and mwTerm() in the
   destructor. Too bad C++ does not guarantee initialization
   order for global objects.
   
   If this didn't help, try adding a call to mwDoFlush(1) after
   mwInit(). If THAT didn't help, then memwatch is unable to
   create the log file. Check write permissions.
   
   If you can't use a log file, you can still use memwatch by
   redirecting the output to a function of your choice. See the
   next question.

Q. I'd like memwatch's output to pipe to my fave debugger! How?

A. Call mwSetOutFunc() with the addres of a "void func(int c)"
   function. You should also consider doing something about
   the ARI handler, see memwatch.h for more details about that.

Q. Why isn't there any C++ support?

A. Because C++ is for sissies! =) Just kidding.
   C++ comes with overridable allocation/deallocation
   built-in. You can define your own new/delete operators
   for any class, and thus circumvent memwatch, or confuse
   it to no end. Also, the keywords "new" and "delete" may
   appear in declarations in C++, making the preprocessor
   replacement approach shaky. You can do it, but it's not
   very stable.
   
   If someone were to write a rock solid new/delete checker
   for C++, there is no conflict with memwatch; use them both.

Q. I'm getting "WILD free" errors, but the code is bug-free!

A. If memwatch's free() recieves a pointer that wasn't allocated
   by memwatch, a "WILD free" message appears. If the source of
   the memory buffer is outside of memwatch (a non-standard
   library function, for instance), you can use mwFree_() to
   release it. mwFree_() calls free() on the pointer given if
   memwatch can't recognize it, instead of blocking it.
   
   Another source of "WILD free" messages is that if memwatch
   is terminated before all memory allocated is freed, memwatch
   will have forgotten about it, and thus generate the errors.
   This is commonly caused by having memwatch auto-initialize,
   and then using atexit() to clean up. When auto-initializing,
   memwatch registers mwTerm() with atexit(), but if mwTerm()
   runs before all memory is freed, then you will get "unfreed"
   and "WILD free" messages when your own atexit()-registered
   cleanup code runs, and frees the memory.

Q. I'm getting "unfreed" errors, but the code is bug-free!

A. You can get erroneous "unfreed" messages if memwatch 
   terminates before all memory has been freed. Try using
   mwInit() and mwTerm() instead of auto-initialization.
   
   If you _are_ using mwInit() and mwTerm(), it may be that
   some code in your program executes before mwInit() or
   after mwTerm(). Make sure that mwInit() is the first thing
   executed, and mwTerm() the last.

Q. When compiling memwatch I get these 'might get clobbered'
   errors, and something about a longjmp() inside memwatch.

A. When using gcc or egcs with the optimization to inline
   functions, this warning occurs. This is because gcc and
   egcs inlines memwatch's functions with setjmp/longjmp,
   causing the calling functions to become unstable.
   
   The gcc/egcs maintainers have been informed of this
   problem, but until they modify the inline optimization
   so that it leaves setjmp functions alone, make sure to 
   compile memwatch without inline function optimizations.
   
   gcc 2.95.2 can be patched for this, and I have been told
   it will be fixed in an upcoming version.

Q. My program crashes with SIGSEGV or alignment errors, but
   only when I compile with memwatch enabled!

A. You are using a 64-bit (or higher) platform, and memwatch
   was unable to detect and adjust for this. You'll have to
   either compile with a suitable define for mwROUNDALLOC,
   I suggest (number of bits / 8), or define mwROUNDALLOC 
   directly in memwatch.c.
   
   Also, please check your limits.h file for the relevant
   #defines, and tell me what they are.

Q. When I include string.h after memwatch.h, I get errors
   related to strdup(), what gives?

A. Most, but probably not all, platforms are nice about
   including files multiple times, so I could probably
   avoid these errors by including string.h from memwatch.h.
   But since I want to be on the safe side, I don't.
   
   To fix this, simply include string.h before memwatch.h, 
   or modify memwatch.h to include string.h.


 
 
 