Hardfork 2016 Value Size Limit
Hardfork 2016 Value Size Limit
Currently the limit of the value size for name_update is 520 bytes. This is a bit tight for even a bunch of hashes and links. It can be worked around by spanning multiple names using "import" or similar. For simple use cases this might become a bit of a hassle, though. With the workaround it can bloat the blockchain as it needs an additional 400-500 bytes for the additional transaction data. This has already happened with e.g. the u/ namespace.
Originally the system was planned to have a value size limit of 1023 bytes as can still be seen in a lot of places, e.g. here. Due to an oversight it became 520 bytes (for name_update, name_firstupdate is possible with 1023 bytes though it is not allowed by the user interface currently). Note that this happened already five years ago and storage and bandwith prices have dropped significantly in the meantime.
There have been endless discussions about raising the value size here in 2012 (consensus was 9000bytes), here in 2013 (consensus was 5000 bytes) and a poll in 2014 (resulting in a compromise of 2500 bytes).
These earlier discussions were under the premise that it should not get cheaper to flood the blockchain. Turns out that the way fees are currently calculated (for Bitcoin, Namecoin Core uses the same formula) with a value size of up to 1100 bytes it is always at least 30% more expensive to flood the blockchain.
With a hardfork upcoming soon it is clear to me that we should take the opportunity and raise the value size limit a bit. It is little effort and safe. As an additional benefit we get a positive marketing/psychological effect (1000 just sounds better) and free attention to our project.
Originally the system was planned to have a value size limit of 1023 bytes as can still be seen in a lot of places, e.g. here. Due to an oversight it became 520 bytes (for name_update, name_firstupdate is possible with 1023 bytes though it is not allowed by the user interface currently). Note that this happened already five years ago and storage and bandwith prices have dropped significantly in the meantime.
There have been endless discussions about raising the value size here in 2012 (consensus was 9000bytes), here in 2013 (consensus was 5000 bytes) and a poll in 2014 (resulting in a compromise of 2500 bytes).
These earlier discussions were under the premise that it should not get cheaper to flood the blockchain. Turns out that the way fees are currently calculated (for Bitcoin, Namecoin Core uses the same formula) with a value size of up to 1100 bytes it is always at least 30% more expensive to flood the blockchain.
With a hardfork upcoming soon it is clear to me that we should take the opportunity and raise the value size limit a bit. It is little effort and safe. As an additional benefit we get a positive marketing/psychological effect (1000 just sounds better) and free attention to our project.
Re: Hardfork 2016 Value Size Limit
I'm all in for an increase to 2kb. people are paying money to store a tiny amount of info on his/her domain, the least we can do is allow to store the whole set of data in one step, linking many names to store a tiny bit of data is redundant and causes overhead and maintenance problems.
Someone could argue that spamming is cheap now but it all depends on how many bitcoins did you spend to buy your namecoins. So i could argue the opposite.
I will be worried if so few people used namecoin that no-one bothers to spam it. :LOL:
This... To store 520bytes you cause another 500 of overhead. The relationship rate between name and value array length is 1:2 and should be something like 1:10phelix wrote:Currently the limit of the value size for name_update is 520 bytes. This is a bit tight for even a bunch of hashes and links. It can be worked around by spanning multiple names using "import" or similar. For simple use cases this might become a bit of a hassle, though. With the workaround it can bloat the blockchain as it needs an additional 400-500 bytes for the additional transaction data.
Someone could argue that spamming is cheap now but it all depends on how many bitcoins did you spend to buy your namecoins. So i could argue the opposite.
I will be worried if so few people used namecoin that no-one bothers to spam it. :LOL:
Re: Hardfork 2016 Value Size Limit
Well, I would prefer 2kb, too, but it might become way more complicated politically as we might have to think about the fee structure giving the right incentives. From experience I guess this would take a long time and also diverts us a bit further from the Bitcoin code. All in all I think 1kb is an ok compromise for the urgent hardfork.johnc wrote:I'm all in for an increase to 2kb. people are paying money to store a tiny amount of info on his/her domain, the least we can do is allow to store the whole set of data in one step, linking many names to store a tiny bit of data is redundant and causes overhead and maintenance problems.
Hopefully we will find a niche to grow a proper userbase from... that should increase price which in turn should make spamming and squatting harder and also bring more pools into the boat.This... To store 520bytes you cause another 500 of overhead. The relationship rate between name and value array length is 1:2 and should be something like 1:10phelix wrote:Currently the limit of the value size for name_update is 520 bytes. This is a bit tight for even a bunch of hashes and links. It can be worked around by spanning multiple names using "import" or similar. For simple use cases this might become a bit of a hassle, though. With the workaround it can bloat the blockchain as it needs an additional 400-500 bytes for the additional transaction data.
Someone could argue that spamming is cheap now but it all depends on how many bitcoins did you spend to buy your namecoins. So i could argue the opposite.
I will be worried if so few people used namecoin that no-one bothers to spam it. :LOL:
-
- Posts: 2001
- Joined: Tue Jun 05, 2012 6:25 am
- os: linux
Re: Hardfork 2016 Value Size Limit
I'll post a more thorough analysis of this later, once I try to analyze the economic arguments. But, we should consider whether it's possible to implement value size changes without a hardfork. In particular, how do the Namecoin consensus rules view a name_update that has more than 2 PUSHDATA ops? Right now the first PUSHDATA is the identifier, and the second is the value. If a name_update is valid with more than 2 PUSHDATA ops (i.e. the extra data is discarded), we could probably do this without a hardfork, it would just be a change to how we interpret the extra PUSHDATA ops. If, instead, the consensus rules don't update the name index for such name_update operations, then this doesn't work.
Credit goes to Hugo for noticing this possible trick.
Credit goes to Hugo for noticing this possible trick.
-
- Posts: 2001
- Joined: Tue Jun 05, 2012 6:25 am
- os: linux
Re: Hardfork 2016 Value Size Limit
Here's the relevant consensus code:biolizard89 wrote:I'll post a more thorough analysis of this later, once I try to analyze the economic arguments. But, we should consider whether it's possible to implement value size changes without a hardfork. In particular, how do the Namecoin consensus rules view a name_update that has more than 2 PUSHDATA ops? Right now the first PUSHDATA is the identifier, and the second is the value. If a name_update is valid with more than 2 PUSHDATA ops (i.e. the extra data is discarded), we could probably do this without a hardfork, it would just be a change to how we interpret the extra PUSHDATA ops. If, instead, the consensus rules don't update the name index for such name_update operations, then this doesn't work.
Credit goes to Hugo for noticing this possible trick.
https://github.com/namecoin/namecoin-co ... mes.cpp#L9
From looking at the code for a few minutes, I *think* I see a way to do this without a consensus fork. You'll notice that OP_NAME_UPDATE must have an args.size of 2. args.size is calculated by the number of PUSHDATA ops following OP_NAME_UPDATE before any OP_DROP, OP_2DROP, or OP_NOP. The OP_NAME_UPDATE, any following PUSHDATA ops, and any following OP_DROP/OP_2DROP/OP_NOP ops, are dropped, and what's left is interpreted as a Bitcoin script.
So what happens if you do something much like buildNameUpdate but like this instead:
Code: Select all
CScript
CNameScript::buildNameUpdateExt (const CScript& addr, const valtype& name,
const valtype& value)
{
CScript prefix;
prefix << OP_NAME_UPDATE << name << value << OP_2DROP << OP_DROP << value << OP_DROP;
return prefix + addr;
}
You could do this today (assuming you find someone willing to mine this nonstandard transaction), and anyone with an appropriately modified Namecoin client will see the full value. Anyone with a stock Namecoin client will only see the first 520 bytes.
Also note that this effectively removes the value size limit completely, except for the limits that we inherit from Bitcoin (I don't know what the limits are on script length and transaction size, but I assume Bitcoin has some).
Have I missed some other part of the consensus code that disallows this trick?
Disclaimer: I make no claims about whether it is good or bad that this is allowed. I'm just looking at what the current code allows.
Re: Hardfork 2016 Value Size Limit
Yes 1mb .biolizard89 wrote:(I don't know what the limits are on script length and transaction size, but I assume Bitcoin has some).
![Mr. Green :mrgreen:](./images/smilies/icon_mrgreen.gif)
-
- Posts: 2001
- Joined: Tue Jun 05, 2012 6:25 am
- os: linux
Re: Hardfork 2016 Value Size Limit
For anyone who cares, I coded up Hugo's idea for experimental purposes. Hoping to test it this weekend, but I might be distracted with some "research" which I hope will drop the stock price and/or profits of a large evil company.
Also, full disclosure: I am strongly against increasing the value size at all unless we do it at the same time as implementing Segregated Name Values. Segregated values only need to stick around for a year or two (less for some validation models). Under the current system, values stick around forever. I implemented Hugo's idea mainly to check that it's not a fork, and therefore not urgent. I don't think my changes should be merged to Namecoin Core.
If new arguments are made regarding either Hugo's idea's validity, or segregated values, then I might change my opinion.
![Wink ;)](./images/smilies/icon_e_wink.gif)
Actually it appears that the maximum size of a combined scriptSig and scriptPubKey is 10 kilobytes. (For anyone illiterate about the metric system who's reading this, that's kilobytes, not kibibytes.)johnc wrote:Yes 1mb .biolizard89 wrote:(I don't know what the limits are on script length and transaction size, but I assume Bitcoin has some).
Also, full disclosure: I am strongly against increasing the value size at all unless we do it at the same time as implementing Segregated Name Values. Segregated values only need to stick around for a year or two (less for some validation models). Under the current system, values stick around forever. I implemented Hugo's idea mainly to check that it's not a fork, and therefore not urgent. I don't think my changes should be merged to Namecoin Core.
If new arguments are made regarding either Hugo's idea's validity, or segregated values, then I might change my opinion.
Re: Hardfork 2016 Value Size Limit
Thecoder already suggested this a couple of years ago and Domob indepentently a bit later. It is interesting and should have it's own thread. However, it is mostly irrelevant for this discussion as the old consensus rules will break anyway once Bitcoin implements BIP 9 - old clients will probably not even be able to recognize a block at all.biolizard89 wrote:I'll post a more thorough analysis of this later, once I try to analyze the economic arguments. But, we should consider whether it's possible to implement value size changes without a hardfork. In particular, how do the Namecoin consensus rules view a name_update that has more than 2 PUSHDATA ops? Right now the first PUSHDATA is the identifier, and the second is the value. If a name_update is valid with more than 2 PUSHDATA ops (i.e. the extra data is discarded), we could probably do this without a hardfork, it would just be a change to how we interpret the extra PUSHDATA ops. If, instead, the consensus rules don't update the name index for such name_update operations, then this doesn't work.
Credit goes to Hugo for noticing this possible trick.
Very interesting. It means that the value size limit we discuss here is only a "recommendation" for legit users. It does not work as a protection against malicious blockchain inflation at all (as far as I am aware this has not yet happened). One more argument to increase the limit.biolizard89 wrote: Actually it appears that the maximum size of a combined scriptSig and scriptPubKey is 10 kilobytes. (For anyone illiterate about the metric system who's reading this, that's kilobytes, not kibibytes.)
https://en.wikipedia.org/wiki/Nirvana_fallacy This is just some vague idea at this point with plenty of risk and effort to handle.Also, full disclosure: I am strongly against increasing the value size at all unless we do it at the same time as implementing Segregated Name Values.
They might be pruned at some point in the future just like old Bitcoin blocks and transactions. Note Namecoin Core already prunes them from the name database without the history flag.Segregated values only need to stick around for a year or two (less for some validation models). Under the current system, values stick around forever.
Reopening previous discussions is bad practice but I can't help it:
(from https://forum.namecoin.info/viewtopic.php?p=6574#p6574)
What made you change your mind by a factor of 10?biolizard89 wrote:
[...]
Assuming that a single large-value name is more expensive than multiple small-value names, then users will only use long-valued names for use cases such as the 4k GPG key.
5k is reasonable in my opinion. But I don't feel particularly strongly.
-
- Posts: 2001
- Joined: Tue Jun 05, 2012 6:25 am
- os: linux
Re: Hardfork 2016 Value Size Limit
No, it's totally relevant, because what I'm arguing is that if Hugo's idea (which I guess is really also Mikhail's and Daniel's idea; I didn't know that) is valid, then the value size discussion can be completely decoupled from the BIP 9 discussion, which grants us the ability to evaluate solutions to those things independently. BIP 9 is an imminent issue and the solution is relatively uncontroversial. If we make a more controversial change in the same consensus rules fork, then there is a serious risk that some users will refuse to upgrade. Imagine what would have happened if Bitcoin Core's devs had implemented a block size increase as part of the same fork as the LevelDB consensus hardfork. The result would have been that many users would refuse to upgrade to LevelDB, which would have caused chaos. In our case it's even worse, because BDB still worked for Bitcoin, while our current consensus rules are likely to completely break as a result of BIP 9.phelix wrote:Thecoder already suggested this a couple of years ago and Domob indepentently a bit later. It is interesting and should have it's own thread. However, it is mostly irrelevant for this discussion as the old consensus rules will break anyway once Bitcoin implements BIP 9 - old clients will probably not even be able to recognize a block at all.biolizard89 wrote:I'll post a more thorough analysis of this later, once I try to analyze the economic arguments. But, we should consider whether it's possible to implement value size changes without a hardfork. In particular, how do the Namecoin consensus rules view a name_update that has more than 2 PUSHDATA ops? Right now the first PUSHDATA is the identifier, and the second is the value. If a name_update is valid with more than 2 PUSHDATA ops (i.e. the extra data is discarded), we could probably do this without a hardfork, it would just be a change to how we interpret the extra PUSHDATA ops. If, instead, the consensus rules don't update the name index for such name_update operations, then this doesn't work.
Credit goes to Hugo for noticing this possible trick.
You appear to be forgetting the isStandard check, as well as social pressure. Both of these are why Bitcoin doesn't have malicious users storing ridiculous amounts of data in the blockchain. (Yes, there are a few people who have done this, but for both Bitcoin and Namecoin this is only a real scalability threat if it becomes widespread.)phelix wrote:Very interesting. It means that the value size limit we discuss here is only a "recommendation" for legit users. It does not work as a protection against malicious blockchain inflation at all (as far as I am aware this has not yet happened). One more argument to increase the limit.biolizard89 wrote: Actually it appears that the maximum size of a combined scriptSig and scriptPubKey is 10 kilobytes. (For anyone illiterate about the metric system who's reading this, that's kilobytes, not kibibytes.)
That's exactly why I don't want to be making decisions at this point. The entire point of why I brought up using additional PUSHDATA ops is that Segregated Name Values needs a bunch of additional review before it's deployed, and therefore I think we should let the review process happen instead of rushing things by abusing the BIP 9 hardfork.phelix wrote:https://en.wikipedia.org/wiki/Nirvana_fallacy This is just some vague idea at this point with plenty of risk and effort to handle.Also, full disclosure: I am strongly against increasing the value size at all unless we do it at the same time as implementing Segregated Name Values.
That's entirely orthogonal and has nothing whatsoever to do with the issues that Segregated Name Values addresses.phelix wrote:They might be pruned at some point in the future just like old Bitcoin blocks and transactions. Note Namecoin Core already prunes them from the name database without the history flag.Segregated values only need to stick around for a year or two (less for some validation models). Under the current system, values stick around forever.
Been quite a while since that discussion, and I haven't read through that thread recently. However, my weak opinion at the time was based on some assumptions, which may not be true. Also, since that time, I did calculations of blockchain size, UTXO size, block size, etc., which suggest to me that supporting use cases like 4k GPG keys is not obviously a good idea.phelix wrote:Reopening previous discussions is bad practice but I can't help it:
(from https://forum.namecoin.info/viewtopic.php?p=6574#p6574)What made you change your mind by a factor of 10?biolizard89 wrote:
[...]
Assuming that a single large-value name is more expensive than multiple small-value names, then users will only use long-valued names for use cases such as the 4k GPG key.
5k is reasonable in my opinion. But I don't feel particularly strongly.
Re: Hardfork 2016 Value Size Limit
If more data is allowed on a name doesn't mean that people will use it to spam.Not having to depend on "import" for space so much would also simplify the implementation of Namecoin services.
Just look the namesquatters, they just post a bitmessage address.
Services like Onename were not scalable, we can't have everyone's id on chain, eventually it becomes expensive.
However if Namecoin only allows to store a tiny amount of data what is the point? Bitcoin does that already!
Well, to store all data used on a regular DNS service. You can point to dns servers or include all the info onchain and replace them. Making impossible-to-shutdown domains a reality, which is the point of Namecoin.
Unlike the onename case, not everyone needs a domain name, so there will always be enough namecoins for domains.
Unlike bitcoin, we don't expect everyone to register domains, but just to be able to access them, read them. Only a few need or care to create unbreakable-domains, but everyone should be able to access them.
Bitcoin=/=Namecoin
Last edited by johnc on Sun Feb 07, 2016 12:49 am, edited 1 time in total.