UTXO is under active development.
Are you not planning on porting over Huntercoin to Libcoin? I'm pretty ignorant about the detail here, but I know that Micheal G. of Libcoin is planning on removing BDB entirely. Although, I'm guessing you already knew this : )
Berkeley DB
-
- Posts: 801
- Joined: Sun Aug 18, 2013 8:26 pm
- os: mac
Re: Berkeley DB
Last edited by indolering on Mon Apr 28, 2014 7:49 am, edited 1 time in total.
DNS is much more than a key->value datastore.
Re: Berkeley DB
We were planning this, but I'm not really sure any more. For me, running Namecoin with libcoin (although in persistence=full mode, so not using his UTXO implementation) uses much more disk space and is even more painful to sync than with namecoind. I get the impression that BDB is indeed more efficient for storage than his SQLite database. It is just my personal impression, though ... it would be good to get other opinions.indolering wrote:UXTO is under active development.
Are you not planning on porting over Huntercoin to Libcoin? I'm pretty ignorant about the detail here, but I know that Micheal G. of Libcoin is planning on removing BDB entirely. Although, I'm guessing you already knew this : )
I think it is worthwhile to add the missing Namecoin RPC commands in any case, in order to get at least an alternative client. But porting over Huntercoin would be much more work, since we would have to port also the game engine and other protocol extensions necessary. It is not the current priority, at least.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/
-
- Posts: 801
- Joined: Sun Aug 18, 2013 8:26 pm
- os: mac
Re: Berkeley DB
Is the Bitcoin codebase so divergent that we could not switch using LevelDB?domob wrote:We were planning this, but I'm not really sure any more. For me, running Namecoin with libcoin (although in persistence=full mode, so not using his UTXO implementation) uses much more disk space and is even more painful to sync than with namecoind. I get the impression that BDB is indeed more efficient for storage than his SQLite database. It is just my personal impression, though ... it would be good to get other opinions.indolering wrote:UXTO is under active development.
Are you not planning on porting over Huntercoin to Libcoin? I'm pretty ignorant about the detail here, but I know that Micheal G. of Libcoin is planning on removing BDB entirely. Although, I'm guessing you already knew this : )
I think it is worthwhile to add the missing Namecoin RPC commands in any case, in order to get at least an alternative client. But porting over Huntercoin would be much more work, since we would have to port also the game engine and other protocol extensions necessary. It is not the current priority, at least.
We could ask Micheal to weigh in, I had an email exchange with him about the use of BDB. For an exchange, well, the creation of relational databases were largely driven by banks needing to balance the books! There are adapters for SQL queries for both LevelDB and BDB. Of course, my experience in this area is skin deep.
What I really worried about is how servicing lite-clients using UTXO, SCIP, and SPV impact the mainline architecture. I used CouchDB because it lets me preprocess their queries, but the security model relies on public key crypto and the choice of CouchDB was based on clients having flaky storage and data connections. I'm guessing that the CouchDB model will be put into maintenance mode within a year or two. But I should just dive into the code if I want an answer to that question....
DNS is much more than a key->value datastore.
Re: Berkeley DB
BTW: https://stackoverflow.com/questions/234 ... e/23475187
For me, shutting down Namecoin (and in particular also Huntercoin) take a very long time. Most of this time seems to be spent in the DbEnv::lsn_reset call during DBFlush(). According to the answer to my question on StackOverflow, this call completely rewrites the database. (Not sure why exactly that would be necessary, but it seems to be like that.) On the other hand, if we don't use lsn_reset, it seems we need to keep the DB log files and are also not able to securely move the blockchain files form one machine to another.
Should we try to make it configurable whether or not lsn_reset should be called to make the DB portable and remove the logs? Or maybe only call it from time to time when the logs grow too large, but not always? I think that it is a very bad idea to completely rewrite all databases (except the blk000?.dat files which are not in BDB) on each shut down - I really don't understand why BDB needs to do that.
For me, shutting down Namecoin (and in particular also Huntercoin) take a very long time. Most of this time seems to be spent in the DbEnv::lsn_reset call during DBFlush(). According to the answer to my question on StackOverflow, this call completely rewrites the database. (Not sure why exactly that would be necessary, but it seems to be like that.) On the other hand, if we don't use lsn_reset, it seems we need to keep the DB log files and are also not able to securely move the blockchain files form one machine to another.
Should we try to make it configurable whether or not lsn_reset should be called to make the DB portable and remove the logs? Or maybe only call it from time to time when the logs grow too large, but not always? I think that it is a very bad idea to completely rewrite all databases (except the blk000?.dat files which are not in BDB) on each shut down - I really don't understand why BDB needs to do that.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/
Re: Berkeley DB
I don't see a particular reason for the reset on shutdown (assuming the wallet is always flushed!). But the DB should be cleaned up at some point.domob wrote:BTW: https://stackoverflow.com/questions/234 ... e/23475187
For me, shutting down Namecoin (and in particular also Huntercoin) take a very long time. Most of this time seems to be spent in the DbEnv::lsn_reset call during DBFlush(). According to the answer to my question on StackOverflow, this call completely rewrites the database. (Not sure why exactly that would be necessary, but it seems to be like that.) On the other hand, if we don't use lsn_reset, it seems we need to keep the DB log files and are also not able to securely move the blockchain files form one machine to another.
Should we try to make it configurable whether or not lsn_reset should be called to make the DB portable and remove the logs? Or maybe only call it from time to time when the logs grow too large, but not always? I think that it is a very bad idea to completely rewrite all databases (except the blk000?.dat files which are not in BDB) on each shut down - I really don't understand why BDB needs to do that.
Re: Berkeley DB
We could implement a more flexible approach in DBFlush(), that, for instance, calls lsn_reset for a full flush for the wallet but not the blkindex or nameindex. For those, we could only flush from time to time if the log files become "too large" in some sense. I'll have to investigate how the DB behaves without the lsn_reset to find out how to gauge the parameters and how it will do with these changes applied.phelix wrote:I don't see a particular reason for the reset on shutdown (assuming the wallet is always flushed!). But the DB should be cleaned up at some point.domob wrote:BTW: https://stackoverflow.com/questions/234 ... e/23475187
For me, shutting down Namecoin (and in particular also Huntercoin) take a very long time. Most of this time seems to be spent in the DbEnv::lsn_reset call during DBFlush(). According to the answer to my question on StackOverflow, this call completely rewrites the database. (Not sure why exactly that would be necessary, but it seems to be like that.) On the other hand, if we don't use lsn_reset, it seems we need to keep the DB log files and are also not able to securely move the blockchain files form one machine to another.
Should we try to make it configurable whether or not lsn_reset should be called to make the DB portable and remove the logs? Or maybe only call it from time to time when the logs grow too large, but not always? I think that it is a very bad idea to completely rewrite all databases (except the blk000?.dat files which are not in BDB) on each shut down - I really don't understand why BDB needs to do that.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/