Maximum P2P message size

[domob]

As I've just found out (somewhat "the hard way"), Bitcoin recently introduced a maximum size for P2P messages: https://github.com/bitcoin/bitcoin/comm ... c3626eb7a7

For Namecoin Core, this needs to be changed. 2 MiB is too little for 2,000 block headers with auxpow. We can either increase this message size limit (or remove the check again), or decrease the maximum number of block headers sent. I have to admit that I don't really feel experienced enough about the networking side of things to decide which one is "better". Any opinions?

[domob]

No opinions? I think this should be fixed quickly, since it currently prevents a fresh syncing with Namecoin Core (staying up-to-date is not affected).

Let me quickly estimate how large a block header with auxpow can be:

Block header itself: 80 bytes
Auxpow coinbase tx: 1 KiB (guessing a probable upper bound)
Merkle Tx block hash: 32 bytes
Tx Merkle branch: 1 KiB (roughly; 1 + 30 * 32 + 4 = 965 bytes - assuming max. 2^30 tx per Bitcoin block which seems more than future-proof)
Auxpow Merkle branch: 1 KiB (roughly the same, 30 is hard upper limit for length of auxpow Merkle branch)
Parent header: 80 bytes

So in total, if we assume 4 KiB for the block header plus auxpow, we should have a very conservative estimate. (It is however not a strict upper bound, as the Bitcoin coinbase tx could have lots of outputs - think P2Pool.) Does this sound reasonable?

So we would need to increase the maximum message size to 8 MiB (currently set by the Bitcoin devs to 2 MiB). While I agree that it makes sense to not allow very large messages, 8 MiB seems fine to me. Particularly since Bitcoin is playing with the idea of increasing the block size to 20 MiB anyway. I suggest to simply set the maximum message size to 8 MiB and not reduce the number of block headers sent. Any objections to that? I'll check that this indeed allows to sync against the existing chain, and I'll also check how large the real 2,000-block headers-messages are.

[domob]

Some more information: All 2,000 block header messages sent by syncing to main net are below 3 MiB. However, on testnet there appear some messages of almost 32 MiB size. Possibly someone experimented there with artificially large coinbase transactions in auxpows or something. I'll take a further look.

In order to get a hard limit for the maximum size of a block header (which would be useful), should we propose a softfork that restricts the maximum size of an auxpow coinbase tx to some reasonable value? (Like 1 KiB or 2 KiB, which should be more than enough unless someone does fishy stuff.) This would, however, restrict merge-mining pools also in how they construct their Bitcoin coinbase. Not sure if this would generate opposition to such a proposal.

[phelix]

Do you know why they introduced the limit in Bitcoin? I can ask sipa.

Besides that, increasing the maximum message size to 8 MiB sounds fine to me.

Also I agree that limiting the auxpow coinbase tx size would make things easier.

[biolizard89]

Some more information: All 2,000 block header messages sent by syncing to main net are below 3 MiB. However, on testnet there appear some messages of almost 32 MiB size. Possibly someone experimented there with artificially large coinbase transactions in auxpows or something. I'll take a further look.

In order to get a hard limit for the maximum size of a block header (which would be useful), should we propose a softfork that restricts the maximum size of an auxpow coinbase tx to some reasonable value? (Like 1 KiB or 2 KiB, which should be more than enough unless someone does fishy stuff.) This would, however, restrict merge-mining pools also in how they construct their Bitcoin coinbase. Not sure if this would generate opposition to such a proposal.

If it affects pools, I think we should check with the major pool operators and see if this would cause problems for them.

[cassini]

All 2,000 block header messages sent by syncing to main net are below 3 MiB.

Same findings here.

(It is however not a strict upper bound,

Does this mean 2000 headers could theoretically be larger than 8 MiB in the future? In this case we'd have to make sure the sender, too, limits the message size to 8 MiB, i.e. sends less than 2000 messages in this situation. Otherwise synchronizing would come to a halt. With sipa's commit a sender gets disconnected if it violates MAX_PROTOCOL_MESSAGE_LENGTH. However, if there is a 2000 header's message > 8 MiB to be sent then a synchronising client disconnects all peers, doesn't it?

If it affects pools, I think we should check with the major pool operators and see if this would cause problems for them.

From an irc namecoin-dev meeting:
Luke-Jr notes BTCGuild mines empty blocks and wouldn't go along with a softfork or hardfork

[biolizard89]

All 2,000 block header messages sent by syncing to main net are below 3 MiB.

Same findings here.

(It is however not a strict upper bound,

Does this mean 2000 headers could theoretically be larger than 8 MiB in the future? In this case we'd have to make sure the sender, too, limits the message size to 8 MiB, i.e. sends less than 2000 messages in this situation. Otherwise synchronizing would come to a halt. With sipa's commit a sender gets disconnected if it violates MAX_PROTOCOL_MESSAGE_LENGTH. However, if there is a 2000 header's message > 8 MiB to be sent then a synchronising client disconnects all peers, doesn't it?

Sending fewer than 2000 headers when 2000 exceeds the limit seems like a decent workaround. Would this be a particularly invasive change?

If it affects pools, I think we should check with the major pool operators and see if this would cause problems for them.

From an irc namecoin-dev meeting:
Luke-Jr notes BTCGuild mines empty blocks and wouldn't go along with a softfork or hardfork

Is BTCGuild still a major pool?

[phelix]

If it affects pools, I think we should check with the major pool operators and see if this would cause problems for them.

+1 In any case we should not set the limits too tight.

Is BTCGuild still a major pool?

4% for Bitcoin, probably twice that much or so for Namecoin. https://blockchain.info/de/pools?timespan=4days

IMHO we should not wait for them and their nasty business practice (not including Namecoin transactions).

[domob]

I've committed a temporary patch that increases the size to 32 MiB. This is necessary to sync testnet, and is (AFAIK) what we had before. 32 MiB is/was the maximum size for serialising anything (or at least something like that).

The client uses the number of returned headers as a criterion whether or not to ask the peer for more. (I. e., if 2,000 headers are returned, more are requested. Otherwise not.) So this probably needs some change if we want to allow returning less headers. (We could do something like: Ask for more if 2,000 headers are returned or the size is >50% the maximum message size. If possible I would prefer to mess as little as possible with the Bitcoin code, though.)

I think that, in theory, the auxpow size is not restricted at all. The coinbase tx can be whatever the miner decides (no need for it to be valid as Bitcoin tx, BTW, just have a valid serialisation format), and also the Merkle branch can be arbitrarily large if the auxblock is not actually a valid Bitcoin block. I guess this is what someone tried out on testnet - but I'll investigate further. (I have already the scripts ready, they just need some time to run.)

Of course, it is not economical for a miner to work on an auxblock that is not a valid Bitcoin block - but it is a possible source of disruption nevertheless. I think that there should be at least some limit imposed - even if it is, say, length 30 for the Merkle branch (with corresponds to 2^30 tx in a Bitcoin block) and 10 KiB for the coinbase tx. (Which should be more than plenty, probably even for things like P2Pool. Does anyone know how large their coinbases are usually?)

[domob]

Ok, here are some stats. If you want to recreate them, use "contrib/auxpow-sizes/auxpow-sizes.py" in the Namecoin Core Github. Main net:

Code: Select all

Merkle lengths:
   0:   1683
   1:    293
   2:    978
   3:   3706
   4:  10041
   5:  12695
   6:  18997
   7:  27975
   8:  37832
   9:  46782
  10:  35519
  11:   7951
  12:    569

Coinbase tx sizes:
          0 - 999: 205021
      1000 - 1999:  15580
      2000 - 4999:  10481
      5000 - 9999:   3319
    10000 - 19999:    377
    20000 - 49999:      1
         >= 50000:      0

Testnet:

Code: Select all

Merkle lengths:
   0:  20036
   1:   8250
   2:  12788
   3:  15500
   4:  15157
   5:  10161
   6:   3886
   7:   2603
   8:   2793
   9:   3339
  10:   3449
  11:    794
  12:     43

Coinbase tx sizes:
          0 - 999:  98799
      1000 - 1999:  11130
      2000 - 4999:  11129
      5000 - 9999:  11123
    10000 - 19999:  11121
    20000 - 49999:      0
         >= 50000:      0

So it seems that the issue is not the Merkle branch length (although it makes sense to restrict it as well, IMHO) but excessively large coinbase tx with hundreds of outputs. Not sure where they come from. At least on testnet, it seems that someone was experimenting with them on purpose. Based on these numbers, it seems reasonable to restrict the coinbase tx size to 5 KiB or even 1 KiB. The vast majority of miners would not be affected. If anyone has contact to P2Pool miners, it would be interesting to hear their thoughts on the issue!