Changing merge-mining format for BIP9

domob
Posts: 1083
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Changing merge-mining format for BIP9

Post by domob » Mon Dec 21, 2015 7:10 pm

It seems that Bitcoin is getting serious about using the individual bits of nVersion for softforks (BIP9). This will severly interfer with the current way merge-mining is handled through nVersion. Thus, I think we should push for a fork that changes the merge-mining format so that it does no longer mess with nVersion, and allows us to keep up with upstream development.

Currently, merge-mined blocks signal this fact through a bit in nVersion, and also set the merge-mining "chain id" in higher bits of this number. I propose the following change (that is a hardfork): Starting with the fork block height (or possibly a certain block time, so that this can be deduced without context as nTime is part of the header already), we change the serialisation format of block headers. nVersion will no longer be "abused" and will just match Bitcoin's version field to 100%, but the initial 80 bytes of the current block header will always be followed by data indicating the merge-mining information - also for non-merge-mined blocks. This additional data could be a new byte that contains the merge-mining information and chain ID. If this byte signals merge-mining, the auxpow follows as before.

I believe it is no issue that this new data is added after the block header and thus no longer (opposed to nVersion) part of the block hash (for instance). The auxpow is only there to "certify" the PoW done on the block, and so I see no reason why this would need to be secured itself by PoW.

Please let me know what you think about this proposal - I think that it should be executed in "due time", say within a few months at the latest (before Bitcoin pushes through with BIP9). I think that I should be able to work on the patch after Christmas, unless there are any issues with my idea.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

biolizard89
Posts: 1932
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Changing merge-mining format for BIP9

Post by biolizard89 » Mon Dec 21, 2015 9:24 pm

Hmm. I'm not really qualified to talk about the merits of this particular change, but a few things come to mind.
  1. How are the other merged mining projects handling this? (In particular, Blockstream and P2Pool.) If they already have a solution to this issue, maybe it would be a good opportunity to switch to their merged mining protocol, or at least inform our changes based on what they've done.
  2. Hardforking before we have a full port of the Namecoin-Qt name management GUI would be problematic, because we currently recommend that end users who want a name management GUI should use 0.3.80. (Maybe we should look seriously at a bounty for porting, with maintenance?)
EDIT: For the benefit of anyone unfamiliar, BIP 9 is at https://github.com/bitcoin/bips/blob/ma ... .mediawiki .
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

phelix
Posts: 1625
Joined: Thu Aug 18, 2011 6:59 am

Re: Changing merge-mining format for BIP9

Post by phelix » Tue Dec 22, 2015 12:38 pm

domob wrote:It seems that Bitcoin is getting serious about using the individual bits of nVersion for softforks (BIP9). This will severly interfer with the current way merge-mining is handled through nVersion. Thus, I think we should push for a fork that changes the merge-mining format so that it does no longer mess with nVersion, and allows us to keep up with upstream development.
Good thing you have an eye on this :)

Do I see it correctly that BIP9 in Bitcoin will break merge mining with current Namecoin Core and v0.3.81 as we do not currently allow the parent block to be an auxPOW block?
Currently, merge-mined blocks signal this fact through a bit in nVersion, and also set the merge-mining "chain id" in higher bits of this number. I propose the following change (that is a hardfork): Starting with the fork block height (or possibly a certain block time, so that this can be deduced without context as nTime is part of the header already),
+1 to nTime
we change the serialisation format of block headers. nVersion will no longer be "abused" and will just match Bitcoin's version field to 100%, but the initial 80 bytes of the current block header will always be followed by data indicating the merge-mining information - also for non-merge-mined blocks. This additional data could be a new byte that contains the merge-mining information and chain ID. If this byte signals merge-mining, the auxpow follows as before.
Sounds good.
I believe it is no issue that this new data is added after the block header and thus no longer (opposed to nVersion) part of the block hash (for instance). The auxpow is only there to "certify" the PoW done on the block, and so I see no reason why this would need to be secured itself by PoW.
Could messing with the chain ID cause problems? I would feel a bit safer about this if it would also be hashed.
Please let me know what you think about this proposal - I think that it should be executed in "due time", say within a few months at the latest (before Bitcoin pushes through with BIP9). I think that I should be able to work on the patch after Christmas, unless there are any issues with my idea.
That would be great :)
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

johnc
Posts: 89
Joined: Sun Dec 28, 2014 10:03 am

Re: Changing merge-mining format for BIP9

Post by johnc » Tue Dec 22, 2015 3:49 pm

From what i can understand bitcoin now will be able to signal multiple softforks simultaneously based on % of hashrate accepting the change or upgrading.

Namecoin on the other hand relies on block-height for the soft/hard forks.

Logic indicates that namecoin can have forks based on changes in bitcoin code that we can do nothing about and namecoin related changes based on namecoin code that for the sake of simplicity should continuing to be based on block-height.

Now, it seems that the new bitcoin code interferes with how Auxpow is indicated.

It seems that a hard fork at a certain height will be inevitable.

As long as you can change the auxpow in a way that can be understood later, it will be fine, you are one of the biggest expert on this. IDK. paging lukejr?

Last, if you announce the imminent demise of v0.3.81 support, people will finally start to look at nameGUI ;)

biolizard89
Posts: 1932
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Changing merge-mining format for BIP9

Post by biolizard89 » Tue Dec 22, 2015 8:12 pm

phelix wrote:
domob wrote:It seems that Bitcoin is getting serious about using the individual bits of nVersion for softforks (BIP9). This will severly interfer with the current way merge-mining is handled through nVersion. Thus, I think we should push for a fork that changes the merge-mining format so that it does no longer mess with nVersion, and allows us to keep up with upstream development.
Good thing you have an eye on this :)

Do I see it correctly that BIP9 in Bitcoin will break merge mining with current Namecoin Core and v0.3.81 as we do not currently allow the parent block to be an auxPOW block?
As far as I can tell, it has nothing to do with the parent block being an auxPOW block (BIP 9 doesn't propose making Bitcoin blocks use auxPOW). The issue is that BIP 9 will permit use of the nVersion field for purposes that collide with Namecoin's usage.
phelix wrote:
Currently, merge-mined blocks signal this fact through a bit in nVersion, and also set the merge-mining "chain id" in higher bits of this number. I propose the following change (that is a hardfork): Starting with the fork block height (or possibly a certain block time, so that this can be deduced without context as nTime is part of the header already),
+1 to nTime
Generally I would prefer having miners vote for the change via either nVersion or a coinbase string (same as how Bitcoin does it). That way we don't get any surprises due to miners failing to upgrade. Is there a strong reason to deviate from Bitcoin's behavior here? It is noteworthy that Wang Chun from F2Pool suggested that we decrease the threshold for consensus rule forks from 95% to circa 90%.
phelix wrote:
I believe it is no issue that this new data is added after the block header and thus no longer (opposed to nVersion) part of the block hash (for instance). The auxpow is only there to "certify" the PoW done on the block, and so I see no reason why this would need to be secured itself by PoW.
Could messing with the chain ID cause problems? I would feel a bit safer about this if it would also be hashed.
I think I agree with Phelix. If there is no compelling reason to omit the auxpow data from the block hash, I would prefer to keep it hashed as it is now.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

domob
Posts: 1083
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Changing merge-mining format for BIP9

Post by domob » Wed Dec 23, 2015 9:48 am

biolizard89 wrote:
phelix wrote:
domob wrote:It seems that Bitcoin is getting serious about using the individual bits of nVersion for softforks (BIP9). This will severly interfer with the current way merge-mining is handled through nVersion. Thus, I think we should push for a fork that changes the merge-mining format so that it does no longer mess with nVersion, and allows us to keep up with upstream development.
Good thing you have an eye on this :)

Do I see it correctly that BIP9 in Bitcoin will break merge mining with current Namecoin Core and v0.3.81 as we do not currently allow the parent block to be an auxPOW block?
As far as I can tell, it has nothing to do with the parent block being an auxPOW block (BIP 9 doesn't propose making Bitcoin blocks use auxPOW). The issue is that BIP 9 will permit use of the nVersion field for purposes that collide with Namecoin's usage.
Exactly this.
biolizard89 wrote:
phelix wrote:
Currently, merge-mined blocks signal this fact through a bit in nVersion, and also set the merge-mining "chain id" in higher bits of this number. I propose the following change (that is a hardfork): Starting with the fork block height (or possibly a certain block time, so that this can be deduced without context as nTime is part of the header already),
+1 to nTime
Generally I would prefer having miners vote for the change via either nVersion or a coinbase string (same as how Bitcoin does it). That way we don't get any surprises due to miners failing to upgrade. Is there a strong reason to deviate from Bitcoin's behavior here? It is noteworthy that Wang Chun from F2Pool suggested that we decrease the threshold for consensus rule forks from 95% to circa 90%.
My reason is simplicity. I don't really believe that the fork will be so complicated to push through that it is justified to implement a complex signalling system. As far as I know, Bitcoin doesn't really have one, either - they never did a well-coordinated hardfork before. There is only the softfork-signalling system based on nVersion.
phelix wrote:
phelix wrote:
I believe it is no issue that this new data is added after the block header and thus no longer (opposed to nVersion) part of the block hash (for instance). The auxpow is only there to "certify" the PoW done on the block, and so I see no reason why this would need to be secured itself by PoW.
Could messing with the chain ID cause problems? I would feel a bit safer about this if it would also be hashed.
I think I agree with Phelix. If there is no compelling reason to omit the auxpow data from the block hash, I would prefer to keep it hashed as it is now.
I think changing the hashing of the block header is quite an invasive change and that we should try to avoid it where possible. The chain ID is fixed anyway (and checked against the fixed one when validating a block), so I don't really see where an issue could arise. But of course, if you see a concrete problem, let me know!
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

phelix
Posts: 1625
Joined: Thu Aug 18, 2011 6:59 am

Re: Changing merge-mining format for BIP9

Post by phelix » Wed Dec 23, 2015 10:13 pm

domob wrote:
biolizard89 wrote:
phelix wrote: Do I see it correctly that BIP9 in Bitcoin will break merge mining with current Namecoin Core and v0.3.81 as we do not currently allow the parent block to be an auxPOW block?
As far as I can tell, it has nothing to do with the parent block being an auxPOW block (BIP 9 doesn't propose making Bitcoin blocks use auxPOW). The issue is that BIP 9 will permit use of the nVersion field for purposes that collide with Namecoin's usage.
Exactly this.
There seems to have been a misunderstanding, let me rephrase. If I understand correctly Namecoin does not allow the parent block chain to be merge mined. Might BIP9 Bitcoin block header nVersion data be misinterpreted as auxPOW blocks by the Namecoin client, resulting in rejection?
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

domob
Posts: 1083
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Changing merge-mining format for BIP9

Post by domob » Fri Dec 25, 2015 7:49 am

phelix wrote:There seems to have been a misunderstanding, let me rephrase. If I understand correctly Namecoin does not allow the parent block chain to be merge mined. Might BIP9 Bitcoin block header nVersion data be misinterpreted as auxPOW blocks by the Namecoin client, resulting in rejection?
I see - very good point! Yes, that is possible and another reason for us to fix this as soon as possible.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

phelix
Posts: 1625
Joined: Thu Aug 18, 2011 6:59 am

Re: Changing merge-mining format for BIP9

Post by phelix » Sat Dec 26, 2015 11:13 pm

domob wrote:
phelix wrote:There seems to have been a misunderstanding, let me rephrase. If I understand correctly Namecoin does not allow the parent block chain to be merge mined. Might BIP9 Bitcoin block header nVersion data be misinterpreted as auxPOW blocks by the Namecoin client, resulting in rejection?
I see - very good point! Yes, that is possible and another reason for us to fix this as soon as possible.
Ok. So we might have to hardfork as early as March...

Is the nonce used at all in Namecoin (auxPOW child) blocks? Maybe it could do the job in combination with the highest bit of nVersion which Bitcoin will not use anytime soon. Or we just test for a valid auxPOW if the header is not valid without.
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

domob
Posts: 1083
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Changing merge-mining format for BIP9

Post by domob » Sun Dec 27, 2015 7:45 pm

phelix wrote:
domob wrote:
phelix wrote:There seems to have been a misunderstanding, let me rephrase. If I understand correctly Namecoin does not allow the parent block chain to be merge mined. Might BIP9 Bitcoin block header nVersion data be misinterpreted as auxPOW blocks by the Namecoin client, resulting in rejection?
I see - very good point! Yes, that is possible and another reason for us to fix this as soon as possible.
Ok. So we might have to hardfork as early as March...

Is the nonce used at all in Namecoin (auxPOW child) blocks? Maybe it could do the job in combination with the highest bit of nVersion which Bitcoin will not use anytime soon. Or we just test for a valid auxPOW if the header is not valid without.
Hm, yes, the nonce could probably be used - but I see no point, TBH. This may just create a similar problem in the future. I'd like to avoid "abusing" any other data fields; I think we can just do what I suggested, namely adding the auxpow information unconditionally with a change in the block header serialisation. This is basically what you suggest (somewhat, at least).
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

Post Reply