Lite Client - for names only

[snailbrain]

just random thought

it "should" be possible for clients to hash nameindex.dat and miners(?) to include the hash in the chain every x blocks.
The nameindex.dat of block XXXX can be distributed in "NMC Torrent Client" automatically (or some other way).
Clients can check if the file matches the hash in the block chain (they may need to wait a few blocks to get it, but don't need to store the whole chain).

I speak with my "colleague" about it

Nameindex.dat -- The file will be different, needs to be normalized. When orphan blocks happen, entries in nameindex can go in different order. So probably need a text file (with a list of all names/values) instead. But may be slow to generate.

so looks do-able? -- may need some efficient way of generating nameindex.dat to distribute

not thought about it properly, and probably some flaws, but probably workeable some way?

[domob]

That's an interesting idea! I also think that we probably shouldn't hash nameindex.dat (BDB data file?) directly, but rather think about a simple, clean (and explicitly specified/documented somewhere!) way to perform the hash of all current names. What about using the same idea about Merkle trees that is used already for transactions in a block? Not sure if that makes sense for the list of names itself, but possibly this could be used for the namespaces. That way, it would even be possible to develop light clients that only know about names in d/ (for instance) and ignore all others. Basically it could work like this:

For all names in a namespace:

For each (non-expired?) name, hash its value; then produce a file of them form:

Code: Select all

d/myname: HASH-OF-VALUE
d/xyz: HASH-OF-VALUE
...

where the lines appear in a well-defined order (e. g., sorted lexically by name). Hash this file to produce a "master hash" for the d/ namespace.

For all namespaces:

Again write up a file of a form like this:

Code: Select all

d: HASH-OF-NAMESPACE
id: HASH-OF-NAMESPACE
...

Hash this file, too.

Of course, the actual forms of how things are hashed in detail are not important and could be changed to something else. The final hash of the namespace file will then be included by miners in some block, or even every block (as part of the block header or simply the tx merkle tree - would not even require a hard fork?). Then every node that has a current list of name-value pairs for some namespace plus a list of all namespaces with their master hashes can easily verify that this list is indeed correct as of a certain block height.

If we do a full protocol extension, we may even make full nodes serve this list upon request to other nodes, so that light clients don't even need an additional source for the data (like a Torrent). I think the name index should be relatively small, shouldn't it? Of course, if it grows too large, one can also have each node only honour requests for a limited number of names at a time, so that a light client must query multiple nodes over time to get a full list. This shouldn't be a big deal, though.

Finally, we could (at least I think so) even produce a tree of some kind (e. g., balanced binary tree) from the names and create a "real" Merkle tree of all names. That way, a light client need only query some other node for a single name plus its tree branch in order to get a (provably correct) value for it! (I think this is exactly how SVP clients in Bitcoin handle transactions; I don't see why it wouldn't work also for querying names.)

[phelix]

Hmm... all blockheaders are currently about 12mb. Merkle tree with name_ops only should also only be a couple of megs. Maybe we could just use that.

Also with a chain of say 5 blockheaders you can be pretty sure it is the real thing all by itself.

[biolizard89]

Gregory Maxwell talks about this in his proposal.

https://en.bitcoin.it/wiki/User:Gmaxwel ... sucks_less
https://bitcointalk.org/index.php?topic=21995.0

If I understand correctly, this would allow lite clients to securely obtain the latest content of any name, and only have to download block headers (which are trivially small, even for mobile clients).

[domob]

If one is only interested in a single name, one could also retrieve the corresponding name_update transaction alone together with its Merkle branch, based only on the same structure as Bitcoin SPV clients in the already existing Merkle trees. Note though, that this only allows you to know that the value was the specific one at the time of this transaction; it doesn't prove that the value is still up-to-date. (This is what gmaxwell addresses in his thread, I think.)

As I understand it, snailbrain's idea also allows light clients to keep track of the full index of names (which, at least at the moment, seems like a reasonable load even for mobile clients, I think), so that they can resolve any name without needing network connection to some full node, as long as they update their index (and check it with the name index hash in the blockchain) from time to time. I like this idea, and think it is not directly what gmaxwell suggested (which is more about single transactions?).

[biolizard89]

If one is only interested in a single name, one could also retrieve the corresponding name_update transaction alone together with its Merkle branch, based only on the same structure as Bitcoin SPV clients in the already existing Merkle trees. Note though, that this only allows you to know that the value was the specific one at the time of this transaction; it doesn't prove that the value is still up-to-date. (This is what gmaxwell addresses in his thread, I think.)

As I understand it, snailbrain's idea also allows light clients to keep track of the full index of names (which, at least at the moment, seems like a reasonable load even for mobile clients, I think), so that they can resolve any name without needing network connection to some full node, as long as they update their index (and check it with the name index hash in the blockchain) from time to time. I like this idea, and think it is not directly what gmaxwell suggested (which is more about single transactions?).

Hmm... what would happen if a name updates in the blockchain after the index was downloaded with this lite client? I think the client would have no way of knowing, and would have to redownload the full index? This could be a security issue (and also a usability issue) if clients are loading websites with outdated IP or TLS info.

gmaxwell does address the use case of getting all the data:

Next, degrade the security model to SPV for the past history. Unlikely a currency, naming doesn't really require strong evidence that in the past the name authorities weren't cheating (inflating names? who gives a ?!#@). On syncup a node would grab headers, then pull the (say) last month of blocks and the UTXO from a month ago then apply forward and only check the recent updates.

This does require a network connection to another node, but I'm not sure why that would be a problem -- am I missing something?

[snailbrain]

If one is only interested in a single name, one could also retrieve the corresponding name_update transaction alone together with its Merkle branch, based only on the same structure as Bitcoin SPV clients in the already existing Merkle trees. Note though, that this only allows you to know that the value was the specific one at the time of this transaction; it doesn't prove that the value is still up-to-date. (This is what gmaxwell addresses in his thread, I think.)

As I understand it, snailbrain's idea also allows light clients to keep track of the full index of names (which, at least at the moment, seems like a reasonable load even for mobile clients, I think), so that they can resolve any name without needing network connection to some full node, as long as they update their index (and check it with the name index hash in the blockchain) from time to time. I like this idea, and think it is not directly what gmaxwell suggested (which is more about single transactions?).

Hmm... what would happen if a name updates in the blockchain after the index was downloaded with this lite client? I think the client would have no way of knowing, and would have to redownload the full index? This could be a security issue (and also a usability issue) if clients are loading websites with outdated IP or TLS info.

gmaxwell does address the use case of getting all the data:

Next, degrade the security model to SPV for the past history. Unlikely a currency, naming doesn't really require strong evidence that in the past the name authorities weren't cheating (inflating names? who gives a ?!#@). On syncup a node would grab headers, then pull the (say) last month of blocks and the UTXO from a month ago then apply forward and only check the recent updates.

This does require a network connection to another node, but I'm not sure why that would be a problem -- am I missing something?

you would be out of date sometimes, but websites probably won't change so often? maybe even in the real world someone may change nameservers and it's not up-to-date for upto 48hours?

And you can update when you want (as long as you have the hash).. if you want on the fly guaranteed update, you could at least get nameindex up to date when you turn your PC on, then you download all the blocks from that "block onwards" while client is running and append any new name changes (and disregard the blocks), this way would always be up to date?

The NMC-Instant-Message-Torrent-Client System (lol) could say "it's been 5 days since your namedb was last updated, you should update"

the namedb could even be stored on some site (although not in a decentralized fashion) it can at least be trusted if checks ok against the hash.

[biolizard89]

If one is only interested in a single name, one could also retrieve the corresponding name_update transaction alone together with its Merkle branch, based only on the same structure as Bitcoin SPV clients in the already existing Merkle trees. Note though, that this only allows you to know that the value was the specific one at the time of this transaction; it doesn't prove that the value is still up-to-date. (This is what gmaxwell addresses in his thread, I think.)

As I understand it, snailbrain's idea also allows light clients to keep track of the full index of names (which, at least at the moment, seems like a reasonable load even for mobile clients, I think), so that they can resolve any name without needing network connection to some full node, as long as they update their index (and check it with the name index hash in the blockchain) from time to time. I like this idea, and think it is not directly what gmaxwell suggested (which is more about single transactions?).

Hmm... what would happen if a name updates in the blockchain after the index was downloaded with this lite client? I think the client would have no way of knowing, and would have to redownload the full index? This could be a security issue (and also a usability issue) if clients are loading websites with outdated IP or TLS info.

gmaxwell does address the use case of getting all the data:

Next, degrade the security model to SPV for the past history. Unlikely a currency, naming doesn't really require strong evidence that in the past the name authorities weren't cheating (inflating names? who gives a ?!#@). On syncup a node would grab headers, then pull the (say) last month of blocks and the UTXO from a month ago then apply forward and only check the recent updates.

This does require a network connection to another node, but I'm not sure why that would be a problem -- am I missing something?

you would be out of date sometimes, but websites probably won't change so often? maybe even in the real world someone may change nameservers and it's not up-to-date for upto 48hours?

And you can update when you want (as long as you have the hash).. if you want on the fly guaranteed update, you could at least get nameindex up to date when you turn your PC on, then you download all the blocks from that "block onwards" while client is running and append any new name changes (and disregard the blocks), this way would always be up to date?

The NMC-Instant-Message-Torrent-Client System (lol) could say "it's been 5 days since your namedb was last updated, you should update"

the namedb could even be stored on some site (although not in a decentralized fashion) it can at least be trusted if checks ok against the hash.

Honestly, one of the reasons Namecoin is so appealing to me is that updates are effectively instant. The 48-hour delay with standard DNS is extremely frustrating, and I'd rather Namecoin not emulate it.

In any event, what you're talking about sounds like a more hacky / less elegant way of doing what Greg is discussing. Why not just use UTXO as Greg suggests? Is there something I'm missing?

[snailbrain]

not sure if the last bit is the same as what i meant, only you get all the ancient names downloaded in one go, instead of downloading the whole chain from scratch which is what you need?
i guess, actually, for a lite client you actually only need to download the last 36k blocks anyway? - i think..

as i said I've not thought about it properly, and only thinking out loud (with a logical common sense thought process).. the reason for the post was i read in the reddit post that namecoin is "failure" because it can't have a lite client, which i think is wrong, i just made something up in my head quick.. i'd also heard people mention this before too

i'm sure we can come up with something, and maybe gmaxwells proposal works, have just not read it (assuming i'll be able to understand it)

[phelix]

gmaxwell has some good points for sure but it would require quite some drastic changes.

Maybe we can make a lite client from the normal client with some small changes like this:
* don't check TXs
* regard all name_op TXs six blocks burried in the longest chain as valid
* discard all TXs, only save headers