I've written another library, Catalina. It started as an example for using the threading library Iris and turned into what I think is a useful library. Catalina is an object data-store for glib and gobject. It provides access through a natural key/value pair interface.
Transparent serialization is supported to and from storage for types that can be stored in GValue's. A tight binary format is provided with the library. It supports basic types such as integers, doubles, floats and strings as well as GObjects in an endian-safe manner. However someone should go double check to call my bluff (and verify its correctness). A JSON serializer would be a quick hack if someone was interested.
In addition to serialization, Catalina supports buffer transformations to and from storage. Included is CatalinaZlibTransform which can apply compression using zlib. It will avoid compression on buffers smaller than the watermark property. This will help on data-sets that are occasionally small and compression would in fact enlarge them.
Catalina is an asynchronous data-store by design. The optimal way of accessing it is the same.
Everything is built upon Trivial DB (TDB) from the samba project. It was chosen over Berkeley DB because of its license. Like Catalina, it is LGPL and does not impose extra restrictions on linking applications such as BDB.
However, the one downside to using TDB is its lack of concurrent transactions. This means that if you have multiple threads doing work and updating storage the transactions would interleave. Since we are using iris, we can use message passing as a way to manage concurrent transactions. (This is done by queuing messages until the commit phase.)
Here is a short example using Vala to asynchronously open, serialize and store a bunch of "Person" GObjects. All the while compressing each buffer with zlib. Don't be scared by the mutex/cond, it's there to negate the need of a main loop.
I intend to add indexes soon, however that is going to take a bit of planning.
So there you have it, my newest hack.
git clone git://git.dronelabs.com/catalina
Comments (13)
Might want to consider a rename; the name “Catalina” is used by Apache Tomcat.
they are quite different, so i think ill keep it for now.
Is there any relation between glib-couchDB and your own library ?
@davidcl
There is no relation. CouchDB glib really doesn’t do much for you at this point other than handle getting/setting data elements (no serialization management or anything).
Surely tdb is GPL, not LGPL.
@James
TDB userspace utilities are GPL’d. However the library itself is LGPL.
Hmm – well, silly me, I was looking here:
http://tdb.cvs.sourceforge.net/viewvc/tdb/tdb/tdb.h?revision=1.27&view=markup
But the sources on Samba do indeed seem to have been relicensed. That’s good news!
These are really great libraries. I have FLOSS use cases for ethos and now this library. Keep it up, but don’t overstretch yourself
.
Anyone know of something similar for .NET (Mono)? I have a Gtk# app that gets [lots] of data from a server and keeps a local cache. A thread-safe data-store of some kind would be immensely useful; currently I just use a Hashtable and LINQ (for searching). I don’t want to require a SQL database and that would also mean I’d have to map to an SQL schema which really isn’t optimal. I tried DB4o but found it gets REALLY slow, a big Hashtable and LINQ is much faster.
Oh but I _am_ scared by the mutex/cond. Next time write a main loop. See if you can get it right.
Use IDisposable for storage resources.
Use try finally for unlocking mutexes.
*ONLY* use cond.wait(…) in a loop _and_ check the exit code.
Your code is not threadsafe (or at least relies on known calling patterns – robust code comes from adhering to good standard practice at all times. Always wear a hard hat at construction sites).
It is not exception safe (by a mile). That’s ok, but maybe you should not be creating the illusion by having exceptionhandling in the code.
I, however, am _not_ tempted to try Catalina for anything storage related if the sample itself is … sketchy like this.
Important correction: I should have made it clear I was referring to the *SAMPLE* code. I haven’t looked at the library.
@Adam
I wrote something similar for people using .NET last winter.
http://github.com/chergert/adroit/tree/src/Adroit.Data/Adroit.Data
@Seth
Thanks for the warm and fuzzy comments.
There is no IDisposable in Vala. How would locking/unlocking in a finally block help. They don’t throw exceptions (in Vala). Perhaps you have confused this with a runtime such as .NET.
For more information on Vala and what it is, see http://live.gnome.org/Vala.
Please point out what you believe is not thread-safe rather than just saying it.
Trackback/Pingback (1)
[...] available information related to your search Christian Hergert: Catalina is now available in this link…: News [...]