structure, layout and automation

node_modules/node-sass/src/libsass/docs/dev-ast-memory.md

@@ -0,0 +1,223 @@
+# LibSass smart pointer implementation
+
+LibSass uses smart pointers very similar to `shared_ptr` known
+by Boost or C++11. Implementation is a bit less modular since
+it was not needed. Various compile time debug options are
+available if you need to debug memory life-cycles.
+
+
+## Memory Classes
+
+### SharedObj
+
+Base class for the actual node implementations. This ensures
+that every object has a reference counter and other values.
+
+```c++
+class AST_Node : public SharedObj { ... };
+```
+
+### SharedPtr (base class for SharedImpl)
+
+Base class that holds on to the pointer. The reference counter
+is stored inside the pointer object directly (`SharedObj`).
+
+### SharedImpl (inherits from SharedPtr)
+
+This is the main base class for objects you use in your code. It
+will make sure that the memory it points at will be deleted once
+all copies to the same object/memory go out of scope.
+
+```c++
+Class* pointer = new Class(...);
+SharedImpl<Class> obj(pointer);
+```
+
+To spare the developer of typing the templated class every time,
+we created typedefs for each available AST Node specialization.
+
+```c++
+typedef SharedImpl<Number> Number_Obj;
+Number_Obj number = SASS_MEMORY_NEW(...);
+```
+
+
+## Memory life-cycles
+
+### Pointer pickups
+
+I often use the terminology of "pickup". This means the moment when
+a raw pointer not under any control is assigned to a reference counted
+object (`XYZ_Obj = XYZ_Ptr`). From that point on memory will be
+automatically released once the object goes out of scope (but only
+if the reference counter reaches zero). Main point beeing, you don't
+have to worry about memory management yourself.
+
+### Object detach
+
+Sometimes we can't return reference counted objects directly (see
+invalid covariant return types problems below). But we often still
+need to use reference objects inside a function to avoid leaks when
+something throws. For this you can use `detach`, which basically
+detaches the pointer memory from the reference counted object. So
+when the reference counted object goes out of scope, it will not
+free the attached memory. You are now again in charge of freeing
+the memory (just assign it to a reference counted object again).
+
+
+## Circular references
+
+Reference counted memory implementations are prone to circular references.
+This can be addressed by using a multi generation garbage collector. But
+for our use-case that seems overkill. There is no way so far for users
+(sass code) to create circular references. Therefore we can code around
+this possible issue. But developers should be aware of this limitation.
+
+There are AFAIR two places where circular references could happen. One is
+the `sources` member on every `Selector`. The other one can happen in the
+extend code (Node handling). The easy way to avoid this is to only assign
+complete object clones to these members. If you know the objects lifetime
+is longer than the reference you create, you can also just store the raw
+pointer. Once needed this could be solved with weak pointers.
+
+
+## Addressing the invalid covariant return types problems
+
+If you are not familiar with the mentioned problem, you may want
+to read up on covariant return types and virtual functions, i.e.
+
+- http://stackoverflow.com/questions/6924754/return-type-covariance-with-smart-pointers
+- http://stackoverflow.com/questions/196733/how-can-i-use-covariant-return-types-with-smart-pointers
+- http://stackoverflow.com/questions/2687790/how-to-accomplish-covariant-return-types-when-returning-a-shared-ptr
+
+We hit this issue at least with the CRTP visitor pattern (eval, expand,
+listize and so forth). This means we cannot return reference counted
+objects directly. We are forced to return raw pointers or we would need
+to have a lot of explicit and expensive upcasts by callers/consumers.
+
+### Simple functions that allocate new AST Nodes
+
+In the parser step we often create new objects and can just return a
+unique pointer (meaning ownership clearly shifts back to the caller).
+The caller/consumer is responsible that the memory is freed.
+
+```c++
+typedef Number* Number_Ptr;
+int parse_integer() {
+  ... // do the parsing
+  return 42;
+}
+Number_Ptr parse_number() {
+  Number_Ptr p_nr = SASS_MEMORY_NEW(...);
+  p_nr->value(parse_integer());
+  return p_nr;
+}
+Number_Obj nr = parse_number();
+```
+
+The above would be the encouraged pattern for such simple cases.
+
+### Allocate new AST Nodes in functions that can throw
+
+There is a major caveat with the previous example, considering this
+more real-life implementation that throws an error. The throw may
+happen deep down in another function. Holding raw pointers that
+we need to free would leak in this case.
+
+```c++
+int parse_integer() {
+  ... // do the parsing
+  if (error) throw(error);
+  return 42;
+}
+```
+
+With this `parse_integer` function the previous example would leak memory.
+I guess it is pretty obvious, as the allocated memory will not be freed,
+as it was never assigned to a SharedObj value. Therefore the above code
+would better be written as:
+
+```c++
+typedef Number* Number_Ptr;
+int parse_integer() {
+  ... // do the parsing
+  if (error) throw(error);
+  return 42;
+}
+// this leaks due to pointer return
+// should return Number_Obj instead
+// though not possible for virtuals!
+Number_Ptr parse_number() {
+  Number_Obj nr = SASS_MEMORY_NEW(...);
+  nr->value(parse_integer()); // throws
+  return &nr; // Ptr from Obj
+}
+Number_Obj nr = parse_number();
+// will now be freed automatically
+```
+
+The example above unfortunately will not work as is, since we return a
+`Number_Ptr` from that function. Therefore the object allocated inside
+the function is already gone when it is picked up again by the caller.
+The easy fix for the given simplified use case would be to change the
+return type of `parse_number` to `Number_Obj`. Indeed we do it exactly
+this way in the parser. But as stated above, this will not work for
+virtual functions due to invalid covariant return types!
+
+### Return managed objects from virtual functions
+
+The easy fix would be to just create a new copy on the heap and return
+that. But this seems like a very inelegant solution to this problem. I
+mean why can't we just tell the object to treat it like a newly allocated
+object? And indeed we can. I've added a `detach` method that will tell
+the object to survive deallocation until the next pickup. This means
+that it will leak if it is not picked up by consumer.
+
+```c++
+typedef Number* Number_Ptr;
+int parse_integer() {
+  ... // do the parsing
+  if (error) throw(error);
+  return 42;
+}
+Number_Ptr parse_number() {
+  Number_Obj nr = SASS_MEMORY_NEW(...);
+  nr->value(parse_integer()); // throws
+  return nr.detach();
+}
+Number_Obj nr = parse_number();
+// will now be freed automatically
+```
+
+
+## Compile time debug options
+
+To enable memory debugging you need to define `DEBUG_SHARED_PTR`.
+This can i.e. be done in `include/sass/base.h`
+
+```c++
+define DEBUG_SHARED_PTR
+```
+
+This will print lost memory on exit to stderr. You can also use
+`setDbg(true)` on sepecific variables to emit reference counter
+increase, decrease and other events.
+
+
+## Why reinvent the wheel when there is `shared_ptr` from C++11
+
+First, implementing a smart pointer class is not really that hard. It
+was indeed also a learning experience for myself. But there are more
+profound advantages:
+
+- Better GCC 4.4 compatibility (which most code still has OOTB)
+- Not thread safe (give us some free performance on some compiler)
+- Beeing able to track memory allocations for debugging purposes
+- Adding additional features if needed (as seen in `detach`)
+- Optional: optimized weak pointer implementation possible
+
+### Thread Safety
+
+As said above, this is not thread safe currently. But we don't need
+this ATM anyway. And I guess we probably never will share AST Nodes
+across different threads.