[RPC] Remove auxpow from the block index

[cointra]

I see in your pull request that you are removing the getheaders message, I am working on an Android Wallet for Namecoin and it uses getheaders to get headers since it doesn't really maintain complete blockchain. Currently namecoin returns headers with auxpow and so we are parsing the auxpow at the client but that can be removed once your auxpow removal gets integrated in the main client. getheaders seems to work fine during our minimal minimal tests and I would like to suggest that we keep it.

Nice to hear you are working on an Android wallet! Actually, it is not too hard to implement getheaders even with a full header (including the auxpow) - if you need it, we can re-enable it as soon as you will be using it in practice. A previous version of the patch even had the message still supported, but I decided to disable it because I had no way of testing that everything works and I didn't want to leave some untested code in there. But as soon as you have a client to test with and which makes actually use of the feature, I can put it back in.

The current version (0.3.73) has a working implementation of getheaders (only the server side since the client never calls getheaders). It already includes auxpow (which should be removed though) and the Android wallet we're working on already uses the headers without issues. The wallet will be released to public soon (somewhere around next week) and I will post here the github links for both the wallet and namecoinj (java implementation of namecoin forked from bitcoinj).

[virtual_master]

the Android wallet we're working on already uses the headers without issues. The wallet will be released to public soon (somewhere around next week) and I will post here the github links for both the wallet and namecoinj (java implementation of namecoin forked from bitcoinj).

Fantastic news.

[domob]

The current version (0.3.73) has a working implementation of getheaders (only the server side since the client never calls getheaders). It already includes auxpow (which should be removed though) and the Android wallet we're working on already uses the headers without issues. The wallet will be released to public soon (somewhere around next week) and I will post here the github links for both the wallet and namecoinj (java implementation of namecoin forked from bitcoinj).

If you don't use the auxpow, for what exactly do you use the block headers? You can't verify which chain has the strongest PoW, can you? Of course, it would be simplest to remove the auxpow for getblockheaders - but I don't see (yet) how block headers without PoW verification can be useful.

[cointra]

The current version (0.3.73) has a working implementation of getheaders (only the server side since the client never calls getheaders). It already includes auxpow (which should be removed though) and the Android wallet we're working on already uses the headers without issues. The wallet will be released to public soon (somewhere around next week) and I will post here the github links for both the wallet and namecoinj (java implementation of namecoin forked from bitcoinj).

If you don't use the auxpow, for what exactly do you use the block headers? You can't verify which chain has the strongest PoW, can you? Of course, it would be simplest to remove the auxpow for getblockheaders - but I don't see (yet) how block headers without PoW verification can be useful.

Currently we do use auxpow for pow verification (for blocks after 19200). Once auxpow is removed, I was planning to use the block hash and check against the difficulty target for some basic pow verification. Also note that the android wallet doesn't only use getheaders, it uses getheaders only upto the timestamp when the wallet was first created, since the user is most likely not interested in anything before his wallet creation. After wallet creation, it gets complete blocks from peers and verifies the transactions relevant to it. One thing missing in Namecoin currently is bloom filters which causes the client to get all transactions and not just the ones it is interested in, but at this stage we can live with it I believe. If we (or maybe you?) get the chance to look into the possibility of adding bloom filters https://github.com/bitcoin/bips/blob/ma ... .mediawiki that would be great for SPV.

[domob]

The current version (0.3.73) has a working implementation of getheaders (only the server side since the client never calls getheaders). It already includes auxpow (which should be removed though) and the Android wallet we're working on already uses the headers without issues. The wallet will be released to public soon (somewhere around next week) and I will post here the github links for both the wallet and namecoinj (java implementation of namecoin forked from bitcoinj).

If you don't use the auxpow, for what exactly do you use the block headers? You can't verify which chain has the strongest PoW, can you? Of course, it would be simplest to remove the auxpow for getblockheaders - but I don't see (yet) how block headers without PoW verification can be useful.

Currently we do use auxpow for pow verification (for blocks after 19200). Once auxpow is removed, I was planning to use the block hash and check against the difficulty target for some basic pow verification. Also note that the android wallet doesn't only use getheaders, it uses getheaders only upto the timestamp when the wallet was first created, since the user is most likely not interested in anything before his wallet creation. After wallet creation, it gets complete blocks from peers and verifies the transactions relevant to it. One thing missing in Namecoin currently is bloom filters which causes the client to get all transactions and not just the ones it is interested in, but at this stage we can live with it I believe. If we (or maybe you?) get the chance to look into the possibility of adding bloom filters https://github.com/bitcoin/bips/blob/ma ... .mediawiki that would be great for SPV.

Makes sense - but I still don't get why the block hash will allow for a "basic PoW verification" when you don't have the auxpow field any more. Does it tell you anything about the PoW? But on the other hand, I have no problem to leave the auxpow field in getheaders - nodes that process the request would simply have to load the block from disk instead of using the in-memory block index. But since that would only be done when a getheaders command is processed, it shouldn't affect performance too much. I just didn't want to have that code there as long as it is untested, but I can re-enable it if you test it for us and tell me that everything is fine.

[cointra]

Makes sense - but I still don't get why the block hash will allow for a "basic PoW verification" when you don't have the auxpow field any more. Does it tell you anything about the PoW? But on the other hand, I have no problem to leave the auxpow field in getheaders - nodes that process the request would simply have to load the block from disk instead of using the in-memory block index. But since that would only be done when a getheaders command is processed, it shouldn't affect performance too much. I just didn't want to have that code there as long as it is untested, but I can re-enable it if you test it for us and tell me that everything is fine.

The way it should work once auxpow is removed is by comparing block hash against difficulty target and if block hash is greater lower than difficulty target we accept the block, again its not really fool proof and depends on the block itself claiming its own difficulty which can be faked by a bad peer, but as I mentioned earlier that should not be an issue for us since this method of PoW checking is only done for headers before the wallet creation, anything after wallet creation which is what matters in our specific case (android wallet) is done using the complete block and not just headers. The check without auxpow mentioned above has to be implemented/tested once (and if) auxpow is removed from headers.

[domob]

Makes sense - but I still don't get why the block hash will allow for a "basic PoW verification" when you don't have the auxpow field any more. Does it tell you anything about the PoW? But on the other hand, I have no problem to leave the auxpow field in getheaders - nodes that process the request would simply have to load the block from disk instead of using the in-memory block index. But since that would only be done when a getheaders command is processed, it shouldn't affect performance too much. I just didn't want to have that code there as long as it is untested, but I can re-enable it if you test it for us and tell me that everything is fine.

The way it should work once auxpow is removed is by comparing block hash against difficulty target and if block hash is greater lower than difficulty target we accept the block, again its not really fool proof and depends on the block itself claiming its own difficulty which can be faked by a bad peer, but as I mentioned earlier that should not be an issue for us since this method of PoW checking is only done for headers before the wallet creation, anything after wallet creation which is what matters in our specific case (android wallet) is done using the complete block and not just headers. The check without auxpow mentioned above has to be implemented/tested once (and if) auxpow is removed from headers.

My personal opinion is that getblockheaders, if implemented, actually should return the full block headers (except if others agree that it makes sense to have it send the headers without auxpow). It won't be a big deal to load the auxpow from disk for the (rare) cases of getblockheaders. As said, just give me a "go" and I can create a patch that re-enables getblockheaders without testing it, and you can then tell me whether it works or not.