[Brainstorming] Namecoin SPV

dolphin
Posts: 16
Joined: Wed Jul 02, 2014 6:52 pm
os: windows

Re: [Brainstorming] Namecoin SPV

Post by dolphin » Thu Oct 09, 2014 6:23 pm

@biolizard89 Hmmm... I didn't know that anyone is actively working on this. Are there threads or discussions that you could point to? Thanks for the catch though! I'd certainly not want multiple people working on the same.
NameCoin and Security News
Chrome extension for Gmail encryption - protect your identity and E-mails using NameCoin

biolizard89
Posts: 1971
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: [Brainstorming] Namecoin SPV

Post by biolizard89 » Wed Oct 15, 2014 7:20 am

dolphin wrote:@biolizard89 Hmmm... I didn't know that anyone is actively working on this. Are there threads or discussions that you could point to? Thanks for the catch though! I'd certainly not want multiple people working on the same.
Here's maaku's blog: http://utxo.tumblr.com/

Last I heard he's trying to get some pull requests merged into Bitcoin Core.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

indolering
Posts: 800
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Re: [Brainstorming] Namecoin SPV

Post by indolering » Mon Oct 20, 2014 5:54 am

AFIAK, we will need an SPV client to manage while the client downloads the UTXO set. We should probably do this in C, C++ or JavaScript. We need to shove this into the browser at some point : /
DNS is much more than a key->value datastore.

biolizard89
Posts: 1971
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: [Brainstorming] Namecoin SPV

Post by biolizard89 » Mon Oct 20, 2014 6:04 am

indolering wrote:AFIAK, we will need an SPV client to manage while the client downloads the UTXO set. We should probably do this in C, C++ or JavaScript. We need to shove this into the browser at some point : /
2 things here:

(1) It would be more effective to adapt an existing SPV Bitcoin client than to build one from scratch.
(2) Bitcoin Core is planning to implement this themselves. Our time might be better spent merging their code after rebasing, instead of duplicating their work.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

dolphin
Posts: 16
Joined: Wed Jul 02, 2014 6:52 pm
os: windows

Re: [Brainstorming] Namecoin SPV

Post by dolphin » Mon Oct 20, 2014 5:27 pm

indolering wrote:AFIAK, we will need an SPV client to manage while the client downloads the UTXO set. We should probably do this in C, C++ or JavaScript. We need to shove this into the browser at some point : /
I agree and, in fact, incorporating it in the browser is the specific reason why I posted my comments. We're working on end-to-end E-mail encryption plugin for Chrome that utilizes the NameCoin for key management.
Not having UTXO reduces the robustness of the key lookup.

It would seem that in absence of UTXO NameCoin adoption is pretty much limited to server world. Client side adoption is highly problematic. It comes to "need to download the whole chain even for simple name lookup". Compare that with DNS where all you need is just a client library and access to the server. Sure, NameCoin is fundamentally more secure. The problem is that security and other benefits come at cost of few GB download. We need to address that.
NameCoin and Security News
Chrome extension for Gmail encryption - protect your identity and E-mails using NameCoin

biolizard89
Posts: 1971
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: [Brainstorming] Namecoin SPV

Post by biolizard89 » Mon Oct 20, 2014 6:00 pm

dolphin wrote:
indolering wrote:AFIAK, we will need an SPV client to manage while the client downloads the UTXO set. We should probably do this in C, C++ or JavaScript. We need to shove this into the browser at some point : /
I agree and, in fact, incorporating it in the browser is the specific reason why I posted my comments. We're working on end-to-end E-mail encryption plugin for Chrome that utilizes the NameCoin for key management.
Not having UTXO reduces the robustness of the key lookup.

It would seem that in absence of UTXO NameCoin adoption is pretty much limited to server world. Client side adoption is highly problematic. It comes to "need to download the whole chain even for simple name lookup". Compare that with DNS where all you need is just a client library and access to the server. Sure, NameCoin is fundamentally more secure. The problem is that security and other benefits come at cost of few GB download. We need to address that.
First off, I agree that we need SPV+UTXO clients, for the reasons you state.

I'm not familiar with how Chrome handles extensions, but Firefox lets extensions bundle native binaries. FreeSpeechMe uses this to bundle namecoind and NMControl. So, having the code written in Javascript probably isn't necessary. If we did want it to run as JS, using Emscripten would probably be better than hand-writing a bunch of Javascript code.

Generally, it's easier to modify Bitcoin code than to write Namecoin code from scratch. Given that maaku and others have been working on UTXO code, I think it's a good idea to let them handle those details. The worst possible outcome would be if we adopted something on our own, Bitcoin adopted something better 3 months later, and we had to hardfork to be able to merge Bitcoin's version.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

dolphin
Posts: 16
Joined: Wed Jul 02, 2014 6:52 pm
os: windows

Re: [Brainstorming] Namecoin SPV

Post by dolphin » Mon Oct 20, 2014 9:11 pm

Thanks for sharing your thoughts, good pointers and very interesting discussion.
biolizard89 wrote:So, having the code written in Javascript probably isn't necessary. If we did want it to run as JS, using Emscripten would probably be better than hand-writing a bunch of Javascript code.
I agree. No need to implement a full NameCoin client in JavaScript. That would be silly. That being said, altering the NameCoin protocol in such a way that lightweight clients written in JavaScript can reliably verify names using a trustless intermediary would open enormous possibilities.

I was thinking for a while and get increasingly convinced as I think more that the concept NameCoin client software being hard dependency for any application utilizing NameCoin severely restricts the use.

Pure JavaScript applications are easy to deliver through extension stores. I imagine some creative uses of reliable name lookup even in a context of HTML/JS pages/apps. All of that requires either pure JavaScript client or support for "lookup" in the browser. I'm not sure that mainstream browser support for NameCoin lookup is possible without mass adoption, which, in turn, seems to be best attainable through web apps and extensions.

biolizard89 wrote: Generally, it's easier to modify Bitcoin code than to write Namecoin code from scratch. Given that maaku and others have been working on UTXO code, I think it's a good idea to let them handle those details. The worst possible outcome would be if we adopted something on our own, Bitcoin adopted something better 3 months later, and we had to hardfork to be able to merge Bitcoin's version.
Yes, it makes lots of sense... I wish the BitCoin UTXO was progressing faster. After I've looked at maaku blog I realized that I've seen it some time ago. It seems like the progress is quite slow. I should say that BitCoin UTXO is substantially more complex and pursues slightly different goal than the ultimate NameCoin UTXO:
a. from bitcoin perspective no transaction is ever expired, i.e. all unspent transactions need to be part of UTXO tree. That's not the case with NameCoin.
b. NameCoin has much higher standard of verification robustness. BitCoin SPV client can afford to be fooled assuming the potential loss is acceptable or if there are other, out-of-band mechanisms to recover from failure. Example: BitCoin SPV accepts payment of X coins to sell a video card. It is ok for SPV+UTXO client to mistakenly accept a spent output, assuming full validation is done later, before the card is shipped. With NameCoin the mistake is almost always not recoverable. Example: there is nothing I can do to recover from using the wrong PGP key to send an E-mail to someone due to SPV+UTXO weakness exploited by malicious third party. My main point is that what BitCoin community may adopt for SPV+UTXO is not necessarily and, in fact, likely, not to be "THE SOLUTION" for NameCoin. Would you agree?

Lastly, I should say that I did not mean to put up a proposal for someone else to implement. The idea was rather to build the proposed solution if the community is willing to take it in, assuming satisfactory quality and robustness.
NameCoin and Security News
Chrome extension for Gmail encryption - protect your identity and E-mails using NameCoin

biolizard89
Posts: 1971
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: [Brainstorming] Namecoin SPV

Post by biolizard89 » Tue Oct 21, 2014 4:53 am

dolphin wrote:Thanks for sharing your thoughts, good pointers and very interesting discussion.
biolizard89 wrote:So, having the code written in Javascript probably isn't necessary. If we did want it to run as JS, using Emscripten would probably be better than hand-writing a bunch of Javascript code.
I agree. No need to implement a full NameCoin client in JavaScript. That would be silly. That being said, altering the NameCoin protocol in such a way that lightweight clients written in JavaScript can reliably verify names using a trustless intermediary would open enormous possibilities.
Agreed.
dolphin wrote:I was thinking for a while and get increasingly convinced as I think more that the concept NameCoin client software being hard dependency for any application utilizing NameCoin severely restricts the use.

Pure JavaScript applications are easy to deliver through extension stores. I imagine some creative uses of reliable name lookup even in a context of HTML/JS pages/apps. All of that requires either pure JavaScript client or support for "lookup" in the browser. I'm not sure that mainstream browser support for NameCoin lookup is possible without mass adoption, which, in turn, seems to be best attainable through web apps and extensions.
I don't see a huge difference between installing a browser addon and running an installer program. Assuming that both are well-designed in terms of UI, they seem to me to have similar difficulty. I think the blockchain download time is by far the biggest issue involved here, long term. (Short term it doesn't matter; a 1.1 GB download of about an hour isn't a big deal for most users. Just tell them to let it run over their lunch break.)
dolphin wrote:
biolizard89 wrote: Generally, it's easier to modify Bitcoin code than to write Namecoin code from scratch. Given that maaku and others have been working on UTXO code, I think it's a good idea to let them handle those details. The worst possible outcome would be if we adopted something on our own, Bitcoin adopted something better 3 months later, and we had to hardfork to be able to merge Bitcoin's version.
Yes, it makes lots of sense... I wish the BitCoin UTXO was progressing faster. After I've looked at maaku blog I realized that I've seen it some time ago. It seems like the progress is quite slow. I should say that BitCoin UTXO is substantially more complex and pursues slightly different goal than the ultimate NameCoin UTXO:
a. from bitcoin perspective no transaction is ever expired, i.e. all unspent transactions need to be part of UTXO tree. That's not the case with NameCoin.
b. NameCoin has much higher standard of verification robustness. BitCoin SPV client can afford to be fooled assuming the potential loss is acceptable or if there are other, out-of-band mechanisms to recover from failure. Example: BitCoin SPV accepts payment of X coins to sell a video card. It is ok for SPV+UTXO client to mistakenly accept a spent output, assuming full validation is done later, before the card is shipped. With NameCoin the mistake is almost always not recoverable. Example: there is nothing I can do to recover from using the wrong PGP key to send an E-mail to someone due to SPV+UTXO weakness exploited by malicious third party. My main point is that what BitCoin community may adopt for SPV+UTXO is not necessarily and, in fact, likely, not to be "THE SOLUTION" for NameCoin. Would you agree?
My understanding is that the whole point of UTXO is that it's immediately provable whether an output is unspent. So I think Bitcoin and Namecoin are going for the same threat model here.
dolphin wrote:Lastly, I should say that I did not mean to put up a proposal for someone else to implement. The idea was rather to build the proposed solution if the community is willing to take it in, assuming satisfactory quality and robustness.
The Namecoin client code is in the middle of a rewrite effort (the current release is based on Bitcoin 0.3.x, which isn't maintainable). My opinion (which may or may not be shared by other devs) is that something as complex as SPV+UTXO should be implemented based on the rewritten client rather than the current one. It's also possible that when that is finished, Bitcoin will have produced some working code. Would you be interested in helping to port Namecoin to the current Bitcoin Core codebase? We'd love to have some help with that. :)
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

domob
Posts: 1100
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: [Brainstorming] Namecoin SPV

Post by domob » Tue Oct 21, 2014 5:25 am

biolizard89 wrote:The Namecoin client code is in the middle of a rewrite effort (the current release is based on Bitcoin 0.3.x, which isn't maintainable). My opinion (which may or may not be shared by other devs) is that something as complex as SPV+UTXO should be implemented based on the rewritten client rather than the current one. It's also possible that when that is finished, Bitcoin will have produced some working code. Would you be interested in helping to port Namecoin to the current Bitcoin Core codebase? We'd love to have some help with that. :)
Let me chime in here only briefly - and note that I have only skimmed the proposal for Namecoin UTXO posted above. While I agree with biolizard89 that the best course of action is to finish the rebase on Bitcoin Core and hope they have implemented their UTXO stuff then, I also believe that it could make sense to build a simpler model that just facilitates name lookups (could be used in addition and be easier to handle for clients that just need names - but that depends on the details).

For instance, blocks could include the full name-value database (i. e., basically the name_show result for each name) in the Merkle tree. (This should be done in a deterministic way so that this data is not transmitted together with the block but instead created by each node and verified as part of the block validation. Something like that is done for Huntercoin already (for a different purpose) and works well.) With this, an untrusted name resolver could prove its result by providing a branch in the Merkle tree to the light client. Since each name that is registered or expired can only appear once in the Merkle tree, the client could be sure that the result is indeed the correct value (and also up-to-date, including its expiration status). The only thing a resolver could do is pretend that a name doesn't exist when it in fact does. Not sure if that's a problem. I'm definitely open for discussion about including something like this even in the old codebase (especially if you want to do it) and then porting it also to the new one. IMHO this is a complementary thing to Bitcoin's UTXO support (whenever it comes).
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

dolphin
Posts: 16
Joined: Wed Jul 02, 2014 6:52 pm
os: windows

Re: [Brainstorming] Namecoin SPV

Post by dolphin » Tue Oct 21, 2014 6:20 pm

biolizard89 wrote: I don't see a huge difference between installing a browser addon and running an installer program. Assuming that both are well-designed in terms of UI, they seem to me to have similar difficulty. I think the blockchain download time is by far the biggest issue involved here, long term. (Short term it doesn't matter; a 1.1 GB download of about an hour isn't a big deal for most users. Just tell them to let it run over their lunch break.)
You have a good point there. That being said... I can't stop my mind from wondering about uses of reliable name lookup in web only apps, i.e. HTML/JS pages that communicate with trustless servers that expose JSON/REST API. In fact, I find myself thinking more and more of HTTP/CDN as the ultimate block distribution mechanism. What if blocks were published as JSON files indexed by block number/hash/etc and delivered through CDN? That would greatly simplify access at the cost of inflated data volume. However, that wouldn't be a concern if UTXO+SPV or other form of secure lightweight lookup was implemented. Anyhow, just thoughts...
biolizard89 wrote: The Namecoin client code is in the middle of a rewrite effort (the current release is based on Bitcoin 0.3.x, which isn't maintainable). My opinion (which may or may not be shared by other devs) is that something as complex as SPV+UTXO should be implemented based on the rewritten client rather than the current one. It's also possible that when that is finished, Bitcoin will have produced some working code. Would you be interested in helping to port Namecoin to the current Bitcoin Core codebase? We'd love to have some help with that. :)
Sure, absolutely. I didn't find any task list around. I guess I must have missed it somehow. Would you, please, direct me to where to look? I'd much appreciate that!
NameCoin and Security News
Chrome extension for Gmail encryption - protect your identity and E-mails using NameCoin

Post Reply