Calculating storage requirements for UTXO

Post Reply
indolering
Posts: 801
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Calculating storage requirements for UTXO

Post by indolering »

From Aarron Schwartz's [Square Triangle FAQ](https://squaretriangle.jottit.com/faq),
Daniel Colascione wrote:Total size doesn't seem to actually be a problem, surprisingly. We have 1.9e8 domains right now; if we suppose each name is 20 bytes long, has a 20 byte nonce, a 20 byte revocation key, an 8-bit timestamp, and maps to an IPv6 address, we're looking at a per-node total database size of about 16GB --- big, but not unthinkable.
20 bytes * 3 + 8 bits = 61 bytes. However, this doesn't line up with the numbers above.

Code: Select all

domains * bytes = totalBytes/bytesPerGig = totalGigs
which is

Code: Select all

190000000 (domains * 61 bytes) / 1073741824 bytesPerGig = 10.8 Gb
Working backwards from Colascione's 16GB estimation should be

Code: Select all

totalGigs * bytesPerGig / domains
or

Code: Select all

16GB * 1073741824 bytes / 190000000 domains
which is ~ 90 bytes per domain.

Am I just fucking up the algebra or is something else wrong here?
DNS is much more than a key->value datastore.

midnightmagic
Posts: 18
Joined: Tue Sep 13, 2011 6:50 am

Re: Calculating storage requirements for UTXO

Post by midnightmagic »

By *his* algebra we have:

20 bytes(name) + 20 bytes(nonce) + 20 bytes(revoke) + 1 bytes(stamp) + 16 bytes(IPv6) = 77 bytes

77 * 1.9e8 = 14630000000 bytes, or 13.625GB for the *raw data*. He's estimating up for database overhead perhaps, or he was just doing the math kind of in his head and coming up with a raw figure.

He's right, it's not that much, not really. The magnitude is really the important thing anyway.

indolering
Posts: 801
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Re: Calculating storage requirements for UTXO

Post by indolering »

midnightmagic wrote:By *his* algebra we have:

20 bytes(name) + 20 bytes(nonce) + 20 bytes(revoke) + 1 bytes(stamp) + 16 bytes(IPv6) = 77 bytes

77 * 1.9e8 = 14630000000 bytes, or 13.625GB for the *raw data*. He's estimating up for database overhead perhaps, or he was just doing the math kind of in his head and coming up with a raw figure.

He's right, it's not that much, not really. The magnitude is really the important thing anyway.
Duh, thanks.
DNS is much more than a key->value datastore.

indolering
Posts: 801
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Re: Calculating storage requirements for UTXO

Post by indolering »

One more question, would the UXTO data structure require holding the entire /d space or can I choose a subset? The Zipf distribution would enable skimming the to 20% most important sites in client side storage.
DNS is much more than a key->value datastore.

indolering
Posts: 801
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Re: Calculating storage requirements for UTXO

Post by indolering »

From IRC:
midnightmagic wrote:It depends on how they do it. They could verify back just to merkel root of the generation tx.
:)
DNS is much more than a key->value datastore.

Post Reply