top button
Flag Notify
Site Registration

Designing a Pythonic search DSL for SQL and NoSQL databases

0 votes
232 views

I am analysing designing an abstraction layer over a select few NoSQL and SQL databases.

Specifically:

  • Redis, Neo4j, MongoDB, CouchDB
  • PostgreSQL

Being inexperienced; it is hard to know a nice way of abstracting search. For conciseness in my explanation, think of Table as being table, object, entity or key; and name as being name or type.

Maybe res = Table.name.search()

Or on multiple Table:
`res = AbstractDB().AbstractSearch(

)`

Then: res.paginate(limit=25, offset=5)

Or if you want all: res.all()

And additionally borrow/alias from a relevant subset of PEP249, e.g.: fetchone and fetchmany

Will open-source this once it's of sufficient functionality. Any suggestion

posted Jul 19, 2013 by anonymous

Share this question
Facebook Share Button Twitter Share Button LinkedIn Share Button

1 Answer

0 votes

Before you even begin to think about "how to do this in Python", you need to think about "how to do this at all". You've got a huge range of storage paradigms there. Redis is basically a key-value store. Mongo does documents. Postgres does relations. I'm not familiar with Neo and Couch, but I assume they also have their own interesting ways of storing things.

You need to figure out what it means to abstract search over such a diverse range of technologies. I honestly don't think it's possible, but maybe you've got some good ideas. In any case, you need to figure that part out before you even begin to think about how to implement it in any particular language.

answer Jul 19, 2013 by anonymous
Similar Questions
+1 vote

What is the Pythonic way to determine the type of an object? Are there multiple valid ways, and when should each be used?

We have obj.__class__, an attribute bound to the object's class. Or is it? When is that true, and when should we not rely on it?

We have type(obj), calling the constructor for the type type in order to get a reference to the type of obj. Or is it? When is that true, and when should we not rely on it?

Are there other ways to get at the type of a Python object? What reasons are there to choose or avoid them?

+3 votes

Does anyone have experience with or awareness of the use of NoSQL databases for 4G LTE EPC network elements such as the HSS or PCRF? These network elements might deal with huge amounts of subscriber data (depending on the subscriber base). These days, the use of NoSQL databases are common among the Internet companies. However, being a relatively newer technology underlining a paradigm shift, as compared to RDBMS, there might not be a large number of instances of the use of NoSQL technology in the Telecommunications industry.

I am looking for feedback on this aspect, and if NoSQL technology is being used, which kind of store (out of Key-Value, Document-based, Column-based, Graph-based) is being used, especially for such elements as HSS or PCRF.

Thank you!

+1 vote

I have about 500 search queries, and about 52000 files in which I have to find all matches for each of the 500 queries.

How should I approach this? Seems like the straightforward way to do it would be to loop through each of the files and go line by line comparing all the terms to the query, but this seems like it would take too long.

Can someone give me a suggestion as to how to minimize the search time?

...