Module Request: Keep a Namespace in Sync

Forum rules
Warning !
Avoid using binary softwares from untrusted users.
Prefer compiling it yourself and verify sources.
hla
Posts: 46
Joined: Mon Nov 10, 2014 12:01 am
os: linux
Contact:

Re: Module Request: Keep a Namespace in Sync

Post by hla » Thu Jan 28, 2016 12:37 am

I wrote a patch for Namecoin Core which adds a name_sync command, which is used to implement namesync.
It's in a subdirectory of this repository but github is down so: https://github.com/hlandau/namesync

phelix
Posts: 1631
Joined: Thu Aug 18, 2011 6:59 am

Re: Module Request: Keep a Namespace in Sync

Post by phelix » Thu Jan 28, 2016 9:51 am

hla wrote:I wrote a patch for Namecoin Core which adds a name_sync command, which is used to implement namesync.
It's in a subdirectory of this repository but github is down so: https://github.com/hlandau/namesync
Hit! Once again I am impressed by the thorough specification and general sophistication of your work. Sorry this was wasted on me earlier, I am mostly looking at the things on the forum with their own topic.

I was thinking about a solution without changes to the client but your solution looks more efficient and elegant. What are the advantages vs. a polling/blocknotify solution besides not having to parse blocks (which is great)?

The changes are almost completely within the namecoin specific files of the codebase so should we merge this into Namecoin Core? Domob?

Digging out previous discussions:
# from namecoin-dev meeting notes, 2015 02 21
Hugo proposes a name_sync RPC command for allowing external databases to sync with Namecoin Core's name database.
https://github.com/hlandau/namesync.t/b ... me_sync.md
Daniel suggests that this would potentially be useful for Bitcoin as well, maybe merge upstream.
Daniel mentions that Bitcoin Core supports a notify command feature for new blocks
Jeremy suggests that NMControl should add support for this, so that the name cache refresh rate is more intelligent than every 30 minutes.
Jeremy suggests a namenotify command
Hugo says that a transactional method is more robust than the current Bitcoin Core notify commands.
Hugo will write a proposal for Bitcoin Core.
Any progress on that?
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

hla
Posts: 46
Joined: Mon Nov 10, 2014 12:01 am
os: linux
Contact:

Re: Module Request: Keep a Namespace in Sync

Post by hla » Fri Jan 29, 2016 4:55 am

Sorry, I never attended to any of that. :/

I will confess I have a dislike of polling probably to an almost irrational degree. It just exudes inefficiency to me. No programmer could defend putting a sleep() statement in a program to fix a bug occurring due to a race condition except as the most desparate, last-minute hack... polling is basically the network equivalent. I don't see an issue with my long-polling system though, even if long polling is a bit of a hack from an HTTP perspective. It's not the first RPC command to use it.

I'd love to see name_sync merged. As far as I can tell the notify functionality of Bitcoin is based on executing a command. This seems a) inefficient, b) I'm wary of the security of executing external programs based on blockchain data. More generally, it doesn't seem like something that lends itself to a robust implementation. Whereas my name_sync system tries to be pretty comprehensive.

The principal weakness of name_sync I think, besides the fixed rollback capacity (not really an issue since we're assuming there will never be more than 12 blocks rolled back anyway) is that after a rollback, as mentioned in the spec, the name database is patched with current names rather than names for the current synced height. This is a pretty small weakness, and given how quickly namesync syncs (as far as I can remember), it's basically an academic consideration, I think.

biolizard89
Posts: 1977
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Module Request: Keep a Namespace in Sync

Post by biolizard89 » Sat Jan 30, 2016 9:30 am

hla wrote:I'd love to see name_sync merged. As far as I can tell the notify functionality of Bitcoin is based on executing a command. This seems a) inefficient, b) I'm wary of the security of executing external programs based on blockchain data. More generally, it doesn't seem like something that lends itself to a robust implementation. Whereas my name_sync system tries to be pretty comprehensive.
I think there have been some changes to upstream Bitcoin's notify stuff, including ZeroMQ support. See https://github.com/bitcoin/bitcoin/blob ... doc/zmq.md

This seems to address your efficiency and security concerns. Do you have concerns about using Bitcoin's ZeroMQ for name notifications?
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

hla
Posts: 46
Joined: Mon Nov 10, 2014 12:01 am
os: linux
Contact:

Re: Module Request: Keep a Namespace in Sync

Post by hla » Mon Feb 01, 2016 1:18 am

That seems like a one-way notification interface, not a two-way synchronization interface. So it can't really satisfy the use case of name_sync, since it can't know the state of the synchronized database and thus know what to send. It would only work if the synchronization daemon was attached to namecoind from the beginning, as it synchronized, etc. and the mechanism never failed.

biolizard89
Posts: 1977
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Module Request: Keep a Namespace in Sync

Post by biolizard89 » Sun Feb 21, 2016 12:28 am

hla wrote:That seems like a one-way notification interface, not a two-way synchronization interface. So it can't really satisfy the use case of name_sync, since it can't know the state of the synchronized database and thus know what to send. It would only work if the synchronization daemon was attached to namecoind from the beginning, as it synchronized, etc. and the mechanism never failed.
Well, if the goal is to stay synchronized to the name database, the client could use name_scan to get the current state, and then use ZeroMQ to stay synced after that.

That said, I agree with Daniel that while an explicit sync interface might have some useful use cases, it should probably be merged upstream in Bitcoin.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

hla
Posts: 46
Joined: Mon Nov 10, 2014 12:01 am
os: linux
Contact:

Re: Module Request: Keep a Namespace in Sync

Post by hla » Fri Feb 26, 2016 4:01 am

How does that work with block rollbacks? And you're introducing more moving parts there, making the whole thing more fragile.

The name_sync patch is surprisingly small. Even if I hadn't written it myself, I'd trust it more than the solution you propose...

biolizard89
Posts: 1977
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Module Request: Keep a Namespace in Sync

Post by biolizard89 » Sat Feb 27, 2016 1:25 pm

hla wrote:How does that work with block rollbacks? And you're introducing more moving parts there, making the whole thing more fragile.

The name_sync patch is surprisingly small. Even if I hadn't written it myself, I'd trust it more than the solution you propose...
I have no objection to Bitcoin merging a sync patch. It would be useful for some use cases. Feel free to talk to the Bitcoin Core devs about it.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

hla
Posts: 46
Joined: Mon Nov 10, 2014 12:01 am
os: linux
Contact:

Re: Module Request: Keep a Namespace in Sync

Post by hla » Tue Mar 01, 2016 1:20 pm

That makes no sense. The patch only exports name data. It's useless for a blockchain without names.

domob
Posts: 1119
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Module Request: Keep a Namespace in Sync

Post by domob » Tue Mar 01, 2016 5:00 pm

hla wrote:That makes no sense. The patch only exports name data. It's useless for a blockchain without names.
This is true, but it would be useful for UTXO's, for instance.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

Post Reply