What is LMDB? Lightning Memory-Mapped Database (LMDB) is a software library that provides a high-performance embedded transactional database in the form of a key-value store. LMDB is written in C with API bindings for several programming languages.
LMDB is a Btree-based database management library modeled loosely on the BerkeleyDB API, but much simplified. The entire database is exposed in a memory map, and all data fetches return data directly from the mapped memory, so no malloc's or memcpy's occur during data fetches. As such, the library is extremely simple because it requires no page caching layer of its own, and it is extremely high performance and memory-efficient. It is also fully transactional with full ACID semantics, and when the memory map is read-only, the database integrity cannot be corrupted by stray pointer writes from application code.
The library is fully thread-aware and supports concurrent read/write access from multiple processes and threads. Data pages use a copy-on- write strategy so no active data pages are ever overwritten, which also provides resistance to corruption and eliminates the need of any special recovery procedures after a system crash. Writes are fully serialized; only one write transaction may be active at a time, which guarantees that writers can never deadlock. The database structure is multi-versioned so readers run with no locks; writers cannot block readers, and readers don't block writers.
Unlike other well-known database mechanisms which use either write-ahead transaction logs or append-only data writes, LMDB requires no maintenance during operation. Both write-ahead loggers and append-only databases require periodic checkpointing and/or compaction of their log or database files otherwise they grow without bound. LMDB tracks free pages within the database and re-uses them for new write operations, so the database size does not grow without bound in normal use
LMDB uses memory-mapped files, giving much better I/O performance.
Works well with really large datasets. The HDF5 files are always read entirely into memory, so you can’t have any HDF5 file exceed your memory capacity. You can easily split your data into several HDF5 files though (just put several paths to h5 files in your text file).
Aerospike is a key-value, in-memory, operational NoSQL database with ACID properties which support complex objects and easy to scale. But I have already used something which does absolutely the same.
Aerospike is designed to be the premier high speed, scalable, and reliable NoSQL database. Every line of Aerospike code, every architectural decision focuses on high performance and easy scaling and operations:
Indexes in RAM
Threaded transaction models
Cache-line optimization transaction and data replication
Seamless auto-rebalance scaling
Use of Linux
Storage and failover reliability.
Aerospike’s distributed Shared-Nothing NoSQL database architecture is designed and built to reliably store data with high availability.
MemcacheDB is a distributed key-value storage system designed for persistent. It is NOT a cache solution, but a persistent storage engine for fast and reliable key-value based object storage and retrieval. It conforms to memcache protocol(not completed, see below), so any memcached client can have connectivity with it. MemcacheDB uses Berkeley DB as a storing backend, so lots of features including transaction and replication are supported.
Free & open source, high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load.
Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.
Memcached is simple yet powerful. Its simple design promotes quick deployment, ease of development, and solves many problems facing large data caches. Its API is available for most popular languages.
MemcacheDB is compatible with memcache protocol, so any clients that support memcache protocol have connectivity with it.
MemcacheDB Commands supports:
get(also mutiple get)
set, add, replace
A version of MemcacheDB using Lightning Memory-Mapped Database (LMDB) is also available, offering greater performance. MemcacheDB is accessed through the same protocol as memcached, so applications may use any memcached API as a means of accessing a MemcacheDB database.
MemcacheQ is a MemcacheDB variant that provides a simple message queue service.
Scylla is an open-source distributed NoSQL data store. It was designed to be compatible with Apache Cassandra while achieving significantly higher throughputs and lower latencies. It supports the same protocols as Cassandra (CQL and Thrift) and the same file formats (SSTable), but is a completely rewritten implementation, using the C++14 language replacing Cassandra's Java, and the Seastar asynchronous programming library replacing threads, shared memory, mapped files, and other classic Linux programming techniques.
Scylla uses a shared design on each node, meaning that each CPU core handles a different subset of data. Cores do not share data, but rather communicate explicitly when they need to. The Scylla authors claim that this design allows Scylla to achieve much better performance on modern NUMA SMP machines, and to scale very well with the number of cores. They have measured as much as 2 million requests per second on a single machine, and also claim that a Scylla cluster can serve as many requests as a Cassandra cluster 10 times its size - and do so with lower latencies.
OrientDB is an open source NoSQL database management system written in Java. It is a multi-model database, supporting graph, document, key/value, and object models, but the relationships are managed as in graph databases with direct connections between records.
OrientDB - The World's First Distributed Multi-Model NoSQL Database with a Graph Database Engine
OrientDB’s has a Native Graph Database engine compliant with the Apache TinkerPop standard. It supports schema-less, schema-full and schema-mixed modes and includes SQL among its query languages which reduces the learning curve for those new to OrientDB.
MongoDB is an open-source database used by companies of all sizes, across all industries and for a wide variety of applications. It is an agile database that allows schemas to change quickly as applications evolve, while still providing the functionality developers expect from traditional databases, such as secondary indexes, a full query language and strict consistency.
MongoDB is a document database that provides high performance, high availability, and easy scalability.
MongoDB is the leading NoSQL database, empowering businesses to be more agile and scalable. Fortune 500 companies and startups alike are using MongoDB to create new types of applications, improve customer experience, accelerate time to market and reduce costs.
Features for MongoDB
Json style documents
Index on any attribute
GridFS to store files
Advantages of MongoDB over RDBMS
Schema less : MongoDB is document database in which one collection holds different different documents. Number of fields, content and size of the document can be differ from one document to another.
Structure of a single object is clear
No complex joins
Deep query-ability. MongoDB supports dynamic queries on documents using a document-based query** language that's nearly as powerful as SQL
Ease of scale-out: MongoDB is easy to scale
Why should use MongoDB
Document Oriented Storage : Data is stored in the form of JSON style documents
Index on any attribute
Multi-Model: Documents, graphs and key-value pairs — model your data as you see fit for your application. Joins: Conveniently join what belongs together for flexible ad-hoc querying, less data redundancy. Transactions: Easy application development keeping your data consistent and safe. No hassle in your client.
ArangoDB is to use the arangoimp command-line tool. arangoimp allows you to import data records from a file into an existing database collection.
ArangoDB provides scalable, highly efficient queries when working with graph data.The database uses JSON as a default storage format, but internally it uses ArangoDB's VelocyPack - a fast and compact binary format for serialization and storage.ArangoDB can natively store a nested JSON object as a data entry inside a collection. Therefore, there is no need to disassemble the resulting JSON objects.