It's scales so perfectly we are deploying it across 3 datacenters, each with four 384 core machines with 3 terabytes of RAM. I can't speak highly enough about sqlite and the team behind it.
quinthar - 19 hours ago
It may be a bit out-of-date, but BerkeleyDB provides a SQLite front-end that supports page-level locking -- and thus has better TPS when multiple threads are writing. It's basically SQLite with the btree ripped out and replaced with BerkeleyDB. More information here [0].
[0] http://www.oracle.com/technetwork/database/database-technolo...
coleifer - 17 hours ago
If this requires turning off pragma synchronous, doesn't that disqualify it for anything but read only tasks? Power loss etc would corrupt the database...
charleslmunger - 19 hours ago
Expensify is powered entirely using sqlite (inside bedrockdb.com) and it is freaking amazing.
quinthar - 19 hours ago
What I wasn't able to understand is if this follow the main SQLite develop efforts or of it is just a frozen branch.
Is there any reason why it is not incorporated into the main branch and activated by compile time flags? Just complexity?
siscia - 19 hours ago
This is good news! Already storing over 7PB of data in sqlite, and have already needed to make many work arounds with readers/writers.
mbrumlow - 15 hours ago
we have been using this page locking technology for quite a while now and it works incredibly well. Having both read and write concurrency is super powerful and scales fantastically.
quinthar - 19 hours ago
I don't understand why SQLite would be used instead of writing a data structure. To me going through SQL (& ORMs) is a hindrance.
Why use a database if you only have one process?
rb808 - 10 hours ago
The collision rate (percentage of attempted transactions that failed due to a page-level locking conflict) in all tests was between 1 and 2%. Failed transactions are not included in the TPS counts below. I read this as: retry logic required. That's a bit onerous IMO. Wrapping a single writer in a mutex gives near rw=1 performance in the cases I tested.
le-mark - 18 hours ago