Hardfork 2016 Value Size Limit

phelix
Posts: 1625
Joined: Thu Aug 18, 2011 6:59 am

Hardfork 2016 Value Size Limit

Post by phelix » Wed Jan 27, 2016 10:31 am

Currently the limit of the value size for name_update is 520 bytes. This is a bit tight for even a bunch of hashes and links. It can be worked around by spanning multiple names using "import" or similar. For simple use cases this might become a bit of a hassle, though. With the workaround it can bloat the blockchain as it needs an additional 400-500 bytes for the additional transaction data. This has already happened with e.g. the u/ namespace.

Originally the system was planned to have a value size limit of 1023 bytes as can still be seen in a lot of places, e.g. here. Due to an oversight it became 520 bytes (for name_update, name_firstupdate is possible with 1023 bytes though it is not allowed by the user interface currently). Note that this happened already five years ago and storage and bandwith prices have dropped significantly in the meantime.

There have been endless discussions about raising the value size here in 2012 (consensus was 9000bytes), here in 2013 (consensus was 5000 bytes) and a poll in 2014 (resulting in a compromise of 2500 bytes).

These earlier discussions were under the premise that it should not get cheaper to flood the blockchain. Turns out that the way fees are currently calculated (for Bitcoin, Namecoin Core uses the same formula) with a value size of up to 1100 bytes it is always at least 30% more expensive to flood the blockchain.

With a hardfork upcoming soon it is clear to me that we should take the opportunity and raise the value size limit a bit. It is little effort and safe. As an additional benefit we get a positive marketing/psychological effect (1000 just sounds better) and free attention to our project.
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

johnc
Posts: 89
Joined: Sun Dec 28, 2014 10:03 am

Re: Hardfork 2016 Value Size Limit

Post by johnc » Wed Jan 27, 2016 1:12 pm

I'm all in for an increase to 2kb. people are paying money to store a tiny amount of info on his/her domain, the least we can do is allow to store the whole set of data in one step, linking many names to store a tiny bit of data is redundant and causes overhead and maintenance problems.
phelix wrote:Currently the limit of the value size for name_update is 520 bytes. This is a bit tight for even a bunch of hashes and links. It can be worked around by spanning multiple names using "import" or similar. For simple use cases this might become a bit of a hassle, though. With the workaround it can bloat the blockchain as it needs an additional 400-500 bytes for the additional transaction data.
This... To store 520bytes you cause another 500 of overhead. The relationship rate between name and value array length is 1:2 and should be something like 1:10

Someone could argue that spamming is cheap now but it all depends on how many bitcoins did you spend to buy your namecoins. So i could argue the opposite.

I will be worried if so few people used namecoin that no-one bothers to spam it. :LOL:

phelix
Posts: 1625
Joined: Thu Aug 18, 2011 6:59 am

Re: Hardfork 2016 Value Size Limit

Post by phelix » Wed Jan 27, 2016 4:32 pm

johnc wrote:I'm all in for an increase to 2kb. people are paying money to store a tiny amount of info on his/her domain, the least we can do is allow to store the whole set of data in one step, linking many names to store a tiny bit of data is redundant and causes overhead and maintenance problems.
Well, I would prefer 2kb, too, but it might become way more complicated politically as we might have to think about the fee structure giving the right incentives. From experience I guess this would take a long time and also diverts us a bit further from the Bitcoin code. All in all I think 1kb is an ok compromise for the urgent hardfork.
phelix wrote:Currently the limit of the value size for name_update is 520 bytes. This is a bit tight for even a bunch of hashes and links. It can be worked around by spanning multiple names using "import" or similar. For simple use cases this might become a bit of a hassle, though. With the workaround it can bloat the blockchain as it needs an additional 400-500 bytes for the additional transaction data.
This... To store 520bytes you cause another 500 of overhead. The relationship rate between name and value array length is 1:2 and should be something like 1:10

Someone could argue that spamming is cheap now but it all depends on how many bitcoins did you spend to buy your namecoins. So i could argue the opposite.

I will be worried if so few people used namecoin that no-one bothers to spam it. :LOL:
Hopefully we will find a niche to grow a proper userbase from... that should increase price which in turn should make spamming and squatting harder and also bring more pools into the boat.
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

biolizard89
Posts: 1932
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Hardfork 2016 Value Size Limit

Post by biolizard89 » Wed Jan 27, 2016 9:13 pm

I'll post a more thorough analysis of this later, once I try to analyze the economic arguments. But, we should consider whether it's possible to implement value size changes without a hardfork. In particular, how do the Namecoin consensus rules view a name_update that has more than 2 PUSHDATA ops? Right now the first PUSHDATA is the identifier, and the second is the value. If a name_update is valid with more than 2 PUSHDATA ops (i.e. the extra data is discarded), we could probably do this without a hardfork, it would just be a change to how we interpret the extra PUSHDATA ops. If, instead, the consensus rules don't update the name index for such name_update operations, then this doesn't work.

Credit goes to Hugo for noticing this possible trick.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

biolizard89
Posts: 1932
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Hardfork 2016 Value Size Limit

Post by biolizard89 » Wed Jan 27, 2016 9:37 pm

biolizard89 wrote:I'll post a more thorough analysis of this later, once I try to analyze the economic arguments. But, we should consider whether it's possible to implement value size changes without a hardfork. In particular, how do the Namecoin consensus rules view a name_update that has more than 2 PUSHDATA ops? Right now the first PUSHDATA is the identifier, and the second is the value. If a name_update is valid with more than 2 PUSHDATA ops (i.e. the extra data is discarded), we could probably do this without a hardfork, it would just be a change to how we interpret the extra PUSHDATA ops. If, instead, the consensus rules don't update the name index for such name_update operations, then this doesn't work.

Credit goes to Hugo for noticing this possible trick.
Here's the relevant consensus code:

https://github.com/namecoin/namecoin-co ... mes.cpp#L9

From looking at the code for a few minutes, I *think* I see a way to do this without a consensus fork. You'll notice that OP_NAME_UPDATE must have an args.size of 2. args.size is calculated by the number of PUSHDATA ops following OP_NAME_UPDATE before any OP_DROP, OP_2DROP, or OP_NOP. The OP_NAME_UPDATE, any following PUSHDATA ops, and any following OP_DROP/OP_2DROP/OP_NOP ops, are dropped, and what's left is interpreted as a Bitcoin script.

So what happens if you do something much like buildNameUpdate but like this instead:

Code: Select all

CScript
CNameScript::buildNameUpdateExt (const CScript& addr, const valtype& name,
                              const valtype& value)
{
  CScript prefix;
  prefix << OP_NAME_UPDATE << name << value << OP_2DROP << OP_DROP << value << OP_DROP;

  return prefix + addr;
}
Note the last 2 things I stuck on the end of the prefix. The current consensus rules will ignore the extra value and OP_DROP, because they're part of the Bitcoin script. But we can easily change the semantics so that the extra value is interpreted as concatenating to the first value.

You could do this today (assuming you find someone willing to mine this nonstandard transaction), and anyone with an appropriately modified Namecoin client will see the full value. Anyone with a stock Namecoin client will only see the first 520 bytes.

Also note that this effectively removes the value size limit completely, except for the limits that we inherit from Bitcoin (I don't know what the limits are on script length and transaction size, but I assume Bitcoin has some).

Have I missed some other part of the consensus code that disallows this trick?

Disclaimer: I make no claims about whether it is good or bad that this is allowed. I'm just looking at what the current code allows.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

johnc
Posts: 89
Joined: Sun Dec 28, 2014 10:03 am

Re: Hardfork 2016 Value Size Limit

Post by johnc » Fri Jan 29, 2016 11:13 am

biolizard89 wrote:(I don't know what the limits are on script length and transaction size, but I assume Bitcoin has some).
Yes 1mb . :mrgreen:

biolizard89
Posts: 1932
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Hardfork 2016 Value Size Limit

Post by biolizard89 » Sat Jan 30, 2016 9:13 am

For anyone who cares, I coded up Hugo's idea for experimental purposes. Hoping to test it this weekend, but I might be distracted with some "research" which I hope will drop the stock price and/or profits of a large evil company. ;)
johnc wrote:
biolizard89 wrote:(I don't know what the limits are on script length and transaction size, but I assume Bitcoin has some).
Yes 1mb . :mrgreen:
Actually it appears that the maximum size of a combined scriptSig and scriptPubKey is 10 kilobytes. (For anyone illiterate about the metric system who's reading this, that's kilobytes, not kibibytes.)

Also, full disclosure: I am strongly against increasing the value size at all unless we do it at the same time as implementing Segregated Name Values. Segregated values only need to stick around for a year or two (less for some validation models). Under the current system, values stick around forever. I implemented Hugo's idea mainly to check that it's not a fork, and therefore not urgent. I don't think my changes should be merged to Namecoin Core.

If new arguments are made regarding either Hugo's idea's validity, or segregated values, then I might change my opinion.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

phelix
Posts: 1625
Joined: Thu Aug 18, 2011 6:59 am

Re: Hardfork 2016 Value Size Limit

Post by phelix » Fri Feb 05, 2016 10:36 am

biolizard89 wrote:I'll post a more thorough analysis of this later, once I try to analyze the economic arguments. But, we should consider whether it's possible to implement value size changes without a hardfork. In particular, how do the Namecoin consensus rules view a name_update that has more than 2 PUSHDATA ops? Right now the first PUSHDATA is the identifier, and the second is the value. If a name_update is valid with more than 2 PUSHDATA ops (i.e. the extra data is discarded), we could probably do this without a hardfork, it would just be a change to how we interpret the extra PUSHDATA ops. If, instead, the consensus rules don't update the name index for such name_update operations, then this doesn't work.

Credit goes to Hugo for noticing this possible trick.
Thecoder already suggested this a couple of years ago and Domob indepentently a bit later. It is interesting and should have it's own thread. However, it is mostly irrelevant for this discussion as the old consensus rules will break anyway once Bitcoin implements BIP 9 - old clients will probably not even be able to recognize a block at all.
biolizard89 wrote: Actually it appears that the maximum size of a combined scriptSig and scriptPubKey is 10 kilobytes. (For anyone illiterate about the metric system who's reading this, that's kilobytes, not kibibytes.)
Very interesting. It means that the value size limit we discuss here is only a "recommendation" for legit users. It does not work as a protection against malicious blockchain inflation at all (as far as I am aware this has not yet happened). One more argument to increase the limit.
Also, full disclosure: I am strongly against increasing the value size at all unless we do it at the same time as implementing Segregated Name Values.
https://en.wikipedia.org/wiki/Nirvana_fallacy This is just some vague idea at this point with plenty of risk and effort to handle.
Segregated values only need to stick around for a year or two (less for some validation models). Under the current system, values stick around forever.
They might be pruned at some point in the future just like old Bitcoin blocks and transactions. Note Namecoin Core already prunes them from the name database without the history flag.


Reopening previous discussions is bad practice but I can't help it:
(from https://forum.namecoin.info/viewtopic.php?p=6574#p6574)
biolizard89 wrote:
[...]

Assuming that a single large-value name is more expensive than multiple small-value names, then users will only use long-valued names for use cases such as the 4k GPG key.

5k is reasonable in my opinion. But I don't feel particularly strongly.
What made you change your mind by a factor of 10?
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

biolizard89
Posts: 1932
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Hardfork 2016 Value Size Limit

Post by biolizard89 » Sun Feb 07, 2016 12:14 am

phelix wrote:
biolizard89 wrote:I'll post a more thorough analysis of this later, once I try to analyze the economic arguments. But, we should consider whether it's possible to implement value size changes without a hardfork. In particular, how do the Namecoin consensus rules view a name_update that has more than 2 PUSHDATA ops? Right now the first PUSHDATA is the identifier, and the second is the value. If a name_update is valid with more than 2 PUSHDATA ops (i.e. the extra data is discarded), we could probably do this without a hardfork, it would just be a change to how we interpret the extra PUSHDATA ops. If, instead, the consensus rules don't update the name index for such name_update operations, then this doesn't work.

Credit goes to Hugo for noticing this possible trick.
Thecoder already suggested this a couple of years ago and Domob indepentently a bit later. It is interesting and should have it's own thread. However, it is mostly irrelevant for this discussion as the old consensus rules will break anyway once Bitcoin implements BIP 9 - old clients will probably not even be able to recognize a block at all.
No, it's totally relevant, because what I'm arguing is that if Hugo's idea (which I guess is really also Mikhail's and Daniel's idea; I didn't know that) is valid, then the value size discussion can be completely decoupled from the BIP 9 discussion, which grants us the ability to evaluate solutions to those things independently. BIP 9 is an imminent issue and the solution is relatively uncontroversial. If we make a more controversial change in the same consensus rules fork, then there is a serious risk that some users will refuse to upgrade. Imagine what would have happened if Bitcoin Core's devs had implemented a block size increase as part of the same fork as the LevelDB consensus hardfork. The result would have been that many users would refuse to upgrade to LevelDB, which would have caused chaos. In our case it's even worse, because BDB still worked for Bitcoin, while our current consensus rules are likely to completely break as a result of BIP 9.
phelix wrote:
biolizard89 wrote: Actually it appears that the maximum size of a combined scriptSig and scriptPubKey is 10 kilobytes. (For anyone illiterate about the metric system who's reading this, that's kilobytes, not kibibytes.)
Very interesting. It means that the value size limit we discuss here is only a "recommendation" for legit users. It does not work as a protection against malicious blockchain inflation at all (as far as I am aware this has not yet happened). One more argument to increase the limit.
You appear to be forgetting the isStandard check, as well as social pressure. Both of these are why Bitcoin doesn't have malicious users storing ridiculous amounts of data in the blockchain. (Yes, there are a few people who have done this, but for both Bitcoin and Namecoin this is only a real scalability threat if it becomes widespread.)
phelix wrote:
Also, full disclosure: I am strongly against increasing the value size at all unless we do it at the same time as implementing Segregated Name Values.
https://en.wikipedia.org/wiki/Nirvana_fallacy This is just some vague idea at this point with plenty of risk and effort to handle.
That's exactly why I don't want to be making decisions at this point. The entire point of why I brought up using additional PUSHDATA ops is that Segregated Name Values needs a bunch of additional review before it's deployed, and therefore I think we should let the review process happen instead of rushing things by abusing the BIP 9 hardfork.
phelix wrote:
Segregated values only need to stick around for a year or two (less for some validation models). Under the current system, values stick around forever.
They might be pruned at some point in the future just like old Bitcoin blocks and transactions. Note Namecoin Core already prunes them from the name database without the history flag.
That's entirely orthogonal and has nothing whatsoever to do with the issues that Segregated Name Values addresses.
phelix wrote:Reopening previous discussions is bad practice but I can't help it:
(from https://forum.namecoin.info/viewtopic.php?p=6574#p6574)
biolizard89 wrote:
[...]

Assuming that a single large-value name is more expensive than multiple small-value names, then users will only use long-valued names for use cases such as the 4k GPG key.

5k is reasonable in my opinion. But I don't feel particularly strongly.
What made you change your mind by a factor of 10?
Been quite a while since that discussion, and I haven't read through that thread recently. However, my weak opinion at the time was based on some assumptions, which may not be true. Also, since that time, I did calculations of blockchain size, UTXO size, block size, etc., which suggest to me that supporting use cases like 4k GPG keys is not obviously a good idea.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

johnc
Posts: 89
Joined: Sun Dec 28, 2014 10:03 am

Re: Hardfork 2016 Value Size Limit

Post by johnc » Sun Feb 07, 2016 12:36 am

Not having to depend on "import" for space so much would also simplify the implementation of Namecoin services.
If more data is allowed on a name doesn't mean that people will use it to spam.
Just look the namesquatters, they just post a bitmessage address.

Services like Onename were not scalable, we can't have everyone's id on chain, eventually it becomes expensive.

However if Namecoin only allows to store a tiny amount of data what is the point? Bitcoin does that already!

Well, to store all data used on a regular DNS service. You can point to dns servers or include all the info onchain and replace them. Making impossible-to-shutdown domains a reality, which is the point of Namecoin.

Unlike the onename case, not everyone needs a domain name, so there will always be enough namecoins for domains.

Unlike bitcoin, we don't expect everyone to register domains, but just to be able to access them, read them. Only a few need or care to create unbreakable-domains, but everyone should be able to access them.

Bitcoin=/=Namecoin
Last edited by johnc on Sun Feb 07, 2016 12:49 am, edited 1 time in total.

Post Reply