Page 2 of 3

Re: Module Request: Keep a Namespace in Sync

Posted: Thu Jan 28, 2016 12:37 am
by hla
I wrote a patch for Namecoin Core which adds a name_sync command, which is used to implement namesync.
It's in a subdirectory of this repository but github is down so: https://github.com/hlandau/namesync

Re: Module Request: Keep a Namespace in Sync

Posted: Thu Jan 28, 2016 9:51 am
by phelix
hla wrote:I wrote a patch for Namecoin Core which adds a name_sync command, which is used to implement namesync.
It's in a subdirectory of this repository but github is down so: https://github.com/hlandau/namesync
Hit! Once again I am impressed by the thorough specification and general sophistication of your work. Sorry this was wasted on me earlier, I am mostly looking at the things on the forum with their own topic.

I was thinking about a solution without changes to the client but your solution looks more efficient and elegant. What are the advantages vs. a polling/blocknotify solution besides not having to parse blocks (which is great)?

The changes are almost completely within the namecoin specific files of the codebase so should we merge this into Namecoin Core? Domob?

Digging out previous discussions:
# from namecoin-dev meeting notes, 2015 02 21
Hugo proposes a name_sync RPC command for allowing external databases to sync with Namecoin Core's name database.
https://github.com/hlandau/namesync.t/b ... me_sync.md
Daniel suggests that this would potentially be useful for Bitcoin as well, maybe merge upstream.
Daniel mentions that Bitcoin Core supports a notify command feature for new blocks
Jeremy suggests that NMControl should add support for this, so that the name cache refresh rate is more intelligent than every 30 minutes.
Jeremy suggests a namenotify command
Hugo says that a transactional method is more robust than the current Bitcoin Core notify commands.
Hugo will write a proposal for Bitcoin Core.
Any progress on that?

Re: Module Request: Keep a Namespace in Sync

Posted: Fri Jan 29, 2016 4:55 am
by hla
Sorry, I never attended to any of that. :/

I will confess I have a dislike of polling probably to an almost irrational degree. It just exudes inefficiency to me. No programmer could defend putting a sleep() statement in a program to fix a bug occurring due to a race condition except as the most desparate, last-minute hack... polling is basically the network equivalent. I don't see an issue with my long-polling system though, even if long polling is a bit of a hack from an HTTP perspective. It's not the first RPC command to use it.

I'd love to see name_sync merged. As far as I can tell the notify functionality of Bitcoin is based on executing a command. This seems a) inefficient, b) I'm wary of the security of executing external programs based on blockchain data. More generally, it doesn't seem like something that lends itself to a robust implementation. Whereas my name_sync system tries to be pretty comprehensive.

The principal weakness of name_sync I think, besides the fixed rollback capacity (not really an issue since we're assuming there will never be more than 12 blocks rolled back anyway) is that after a rollback, as mentioned in the spec, the name database is patched with current names rather than names for the current synced height. This is a pretty small weakness, and given how quickly namesync syncs (as far as I can remember), it's basically an academic consideration, I think.

Re: Module Request: Keep a Namespace in Sync

Posted: Sat Jan 30, 2016 9:30 am
by biolizard89
hla wrote:I'd love to see name_sync merged. As far as I can tell the notify functionality of Bitcoin is based on executing a command. This seems a) inefficient, b) I'm wary of the security of executing external programs based on blockchain data. More generally, it doesn't seem like something that lends itself to a robust implementation. Whereas my name_sync system tries to be pretty comprehensive.
I think there have been some changes to upstream Bitcoin's notify stuff, including ZeroMQ support. See https://github.com/bitcoin/bitcoin/blob ... doc/zmq.md

This seems to address your efficiency and security concerns. Do you have concerns about using Bitcoin's ZeroMQ for name notifications?

Re: Module Request: Keep a Namespace in Sync

Posted: Mon Feb 01, 2016 1:18 am
by hla
That seems like a one-way notification interface, not a two-way synchronization interface. So it can't really satisfy the use case of name_sync, since it can't know the state of the synchronized database and thus know what to send. It would only work if the synchronization daemon was attached to namecoind from the beginning, as it synchronized, etc. and the mechanism never failed.

Re: Module Request: Keep a Namespace in Sync

Posted: Sun Feb 21, 2016 12:28 am
by biolizard89
hla wrote:That seems like a one-way notification interface, not a two-way synchronization interface. So it can't really satisfy the use case of name_sync, since it can't know the state of the synchronized database and thus know what to send. It would only work if the synchronization daemon was attached to namecoind from the beginning, as it synchronized, etc. and the mechanism never failed.
Well, if the goal is to stay synchronized to the name database, the client could use name_scan to get the current state, and then use ZeroMQ to stay synced after that.

That said, I agree with Daniel that while an explicit sync interface might have some useful use cases, it should probably be merged upstream in Bitcoin.

Re: Module Request: Keep a Namespace in Sync

Posted: Fri Feb 26, 2016 4:01 am
by hla
How does that work with block rollbacks? And you're introducing more moving parts there, making the whole thing more fragile.

The name_sync patch is surprisingly small. Even if I hadn't written it myself, I'd trust it more than the solution you propose...

Re: Module Request: Keep a Namespace in Sync

Posted: Sat Feb 27, 2016 1:25 pm
by biolizard89
hla wrote:How does that work with block rollbacks? And you're introducing more moving parts there, making the whole thing more fragile.

The name_sync patch is surprisingly small. Even if I hadn't written it myself, I'd trust it more than the solution you propose...
I have no objection to Bitcoin merging a sync patch. It would be useful for some use cases. Feel free to talk to the Bitcoin Core devs about it.

Re: Module Request: Keep a Namespace in Sync

Posted: Tue Mar 01, 2016 1:20 pm
by hla
That makes no sense. The patch only exports name data. It's useless for a blockchain without names.

Re: Module Request: Keep a Namespace in Sync

Posted: Tue Mar 01, 2016 5:00 pm
by domob
hla wrote:That makes no sense. The patch only exports name data. It's useless for a blockchain without names.
This is true, but it would be useful for UTXO's, for instance.