Dbstl persistence
Prev | Chapter 7. Standard Template Library API | Next

## Dbstl persistence

The following sections provide information on how to achieve persistence using dbstl.

### Direct database get

Each container has a begin() method which produces an iterator. These begin methods take a boolean parameter, directdb_get, which controls the caching behavior of the iterator. The default value of this parameter is `true`.

If directdb_get is `true`, then the persistent object is fetched anew from the database each time the iterator is dereferenced as a pointer by use of the star-operator (*iterator) or by use of the arrow-operator (iterator->member). If directdb_get is `false`, then the first dereferencing of the iterator fetches the object from the database, but later dereferences can return cached data.

With directdb_get set to `true`, if you call:

    (*iterator).datamember1=new-value1;
    (*iterator).datamember2=new-value2;

then the assignment to `datamember1` will be lost, because the second dereferencing of the iterator would cause the cached copy of the object to be overwritten by the object's persistent data from the database.

You also can use the arrow operator like this:

    iterator->datamember1=new-value1;
    iterator->datamember2=new-value2;

This works exactly the same way as iterator::operator*. For this reason, the same caching rules apply to arrow operators as they do for star operators.

One way to avoid this problem is to create a reference to the object, and use it to access the object:

    container::value_type &ref = *iterator;
    ref.datamember1=new-value1;
    ref.datamember2=new-value2;
    ...// more member function calls and datamember assignments
    ref._DB_STL_StoreElement();

The above code will not lose the newly assigned value of `ref.datamember1` in the way that the previous example did.

In order to avoid these complications, you can assign to the object referenced by an iterator with another object of the same type like this:

    container::value_type obj2;
    obj2.datamember1 = new-value1;
    obj2.datamember2 = new-value2;
    *itr = obj2;

This code snippet causes the new values in `obj2` to be stored into the underlying database.

If you have two iterators going through the same container like this:

    for (iterator1 = v.begin(), iterator2 = v.begin();
         iterator1 != v.end();
         ++iterator1, ++iterator2) {
            *iterator1 = new_value;
            print(*iterator2);
    }

then the printed value will depend on the value of directdb_get with which the iterator had been created. If directdb_get is `false`, then the original, persistent value is printed; otherwise the newly assigned value is returned from the cache when `iterator2` is dereferenced. This happens because each iterator has its own cached copy of the persistent object, and the dereferencing of `iterator2` refreshes `iterator2`'s copy from the database, retrieving the value stored by the assignment to `*iterator1`.

Alternatively, you can set directdb_get to `false` and call `iterator2->refresh()` immediately before the dereferencing of `iterator2`, so that `iterator2`'s cached value is refreshed.

If directdb_get is `false`, a few of the tests in dbstl's test kit will fail. This is because the above contrived case appears in several of C++ STL tests. Consequently, the default value of the directdb_get parameter in the `container::begin()` methods is `true`. If your use cases avoid such bizarre usage of iterators, you can set it to `false`, which makes the iterator read operation faster.

### Change persistence

If you modify the object to which an iterator refers by using one of the following:

    (*iterator).member_function_call()

or

    (*iterator).data_member = new_value

then you should call `iterator->_DB_STL_StoreElement()` to store the change. Otherwise the change is lost after the iterator moves on to other elements.

If you are storing a sequence, and you modified some part of it, you should also call `iterator->_DB_STL_StoreElement()` before moving the iterator.

And in both cases, if directdb_get is `true` (this is the default value), you should call `_DB_STL_StoreElement()` after the change and before the next iterator movement OR the next dereferencing of the iterator by the star or arrow operators (`iterator::operator*` or `iterator::operator->`). Otherwise, you will lose the change.

If you update the element by assigning to a dereferenced iterator like this:

    *iterator = new_element;

then you never have to call `_DB_STL_StoreElement()` because the change is stored in the database automatically.

### Object life time and persistence

Dbstl is an interface to Berkeley DB, so it is used to store data persistently. This is really a different purpose from that of regular C++ STL. This difference in their goals has implications on expected object lifetime: In standard STL, when you store an object A of type ID into C++ stl vector V using V.push_back(A), if a proper copy constructor is provided in A's class type, then the copy of A (call it B) and everything in B, such as another object C pointed to by B's data member B.c_ptr, will be stored in V and will live as long as B is still in V and V is alive. B will be destroyed when V is destroyed or B is erased from V.

This is not true for dbstl, which will copy A's data and store it in the underlying database. The copy is by default a shallow copy, but users can register their object marshalling and unmarshalling functions using the `DbstlElemTraits` class template. So if A is passed to a `db_vector` container, `dv`, by using `dv.push_back(A)`, then dbstl copies A's data using the registered functions, and stores data into the underlying database. Consequently, A will be valid, even if the container is destroyed, because it is stored into the database.

If the copy is simply a shallow copy, and A is later destroyed, then the pointer stored in the database will become invalid. The next time we use the retrieved object, we will be using an invalid pointer, which probably will result in errors. To avoid this, store the referred object C rather than the pointer member A.c_ptr itself, by registering the right marshalling/unmarshalling function with `DbstlElemTraits`.

For example, consider the following example class declaration:

    class ID
    {
    public:
        string Name;
        int Score;
    };

Here, the class ID has a data member Name, which refers to a memory address of the actual characters in the string. If we simply shallow copy an object, `id`, of class ID to store it, then the stored data, `idd`, is invalid when `id` is destroyed. This is because `idd` and `id` refer to a common memory address which is the base address of the memory space storing all characters in the string, and this memory space is released when `id` is destroyed. So `idd` will be referring to an invalid address. The next time we retrieve `idd` and use it, there will probably be memory corruption.

The way to store `id` is to write a marshal/unmarshal function pair like this:

    void copy_id(void *dest, const ID&elem)
    {
    	memcpy(dest, &elem.Score, sizeof(elem.Score));
    	char *p = ((char *)dest) + sizeof(elem.Score);
    	strcpy(p, elem.Name.c_str());
    }

    void restore_id(ID& dest, const void *srcdata)
    {
    	memcpy(&dest.Score, srcdata, sizeof(dest.Score));
    	const char *p = ((char *)srcdata) + sizeof(dest.Score);
    	dest.Name = p;
    }

    size_t size_id(const ID& elem)
    {
    	return sizeof(elem.Score) + elem.Name.size() +
    	    1;// store the '\0' char.
    }

Then register the above functions before storing any instance of `ID`:

    DbstlElemTraits<ID>::instance()->set_copy_function(copy_id);
    DbstlElemTraits<ID>::instance()->set_size_function(size_id);
    DbstlElemTraits<ID>::instance()->set_restore_function(restore_id);

This way, the actual data of instances of ID are stored, and so the data will persist even if the container itself is destroyed.

Prev | Up | Next
Store and Retrieve data or objects of complex types | Home | Dbstl container specific notes