[RFC] Signing files with the namecoin blockchain

[domob]

Today some of us namecoiners in Munich had a discussion on how to use namecoin to sign files (so that a binary / source release can be verified by users downloading it). I always was of the opinion that you had to supply a signature file along side the original one similarly to the .asc / .sig files commonly supplied with free software sources for GPG signatures. Namecoin can be handy because you can use it to be sure about the "correct" public signing key (for instance with the developers' id/ names or even the project's domain). However, we came up with an idea where you can sign arbitrary files even without the need to supply either a signed and a non-signed version or the plain file and a second signature file. Please let us know what you think about it!

How signing a file works:

You calculate some kind of hash value from the file's content, H. It doesn't really matter what hash you use (could be SHA-2 or maybe even the same procedure used for Bitcoin addresses), this hash is only used to distinguish the file from all other signed files and it need not really be secure against a malicious collision attack (although it doesn't hurt if it is of course). Then you register a name "hash/H", which should still be free because H is basically random. It is not possible for anyone to "squat" this name to prevent you signing your next release because it can't be predicted without the actual file what it will be. As value of this name you then store signatures for the file, possibly with a different / more secure hash (could be SHA-512 or SHA-3, while I think it is good if H is not too long and thus not SHA-512 itself; I don't know if there's a maximal name length). The value could look like this:

Code: Select all

{
  "id/domob": {"method": "sha256", "signature": "HDRw84xtECK24iia+yyQBoASkcUsFPUHKG+Iou5oXXdTc14g5Cp+TZcNTi2qVjdN444vBS3We98r9+h5iiBDJLA="},
  "d/domob": {"method": "sha512", "signature": "G8IODHIfUkoQ8r2ScxzNmqcnmQFhHLGPYk7mlOblxmaam4LWBXhYZ5VVLnxplopStPvwCMEC7WrttVPNnu68tiU="}
}

(This would be for the empty file. Signature is "namecoind signmessage ADDRESS HASH", and the addresses in the example are the ones currently holding those names, NFppu8bRjGVYTjyVrFZE9cGmjvzD6VUo5m and N2twvKxy7LrUubm7J4HZFAdTLjq8ffq24e.)

Apart from adding this entry in the blockchain, you don't have to change the file in any way and can just put it on your download page as you would without signature. (Or do whatever you want.)

How to check a signature:

Given a downloaded file, calculate the hash H by the same method as before. (If in the future multiple hashing methods for this first step are introduced because of technological evolution, just try all of them in some order.) See if "hash/H" exists, and if not, there's no valid signature. If it exists, fetch the value and check the signatures there against your file. Thus you get (if successful) a list of namecoin names (identities or domains in the probably most common case) which can be displayed to the user, so that he/she can decide whether or not to trust them.

EDIT: It seems not necessary to require that the hash/H name is updated over time since conflicts are very unlikely anyway. Instead I suggest that when validating a file, the whole history of the name is searched and every valid signature found for any update is reported. That way, one can add signatures by more people over time if that should be wanted.

[domob]

I took the liberty to reserve some possible domain names for a future website of this project (should we want to set up one):

• namesig(nature)
• filesecure
• signfile
• authfile
• filevalidator
• dlcheck
• dlsecure
• checkfile

If someone else takes over the responsibility for managing it and we indeed set up a site, I'll send over the names. You can also see the list as my own suggestions for names this project could have.

[moa]

Cool idea. Nice work domob.

Just to see if I have grapsed the concept;

1) any file can have a hash

2) store that hash in the nmc blockchain as a name, along with signatures of that hash using nmc pub-keys associated with human readable names

So it is a way to publish that the file has been generated by (or 'belongs' to) you in some way?

[biolizard89]

Nice work, this looks like a good application of Namecoin. I don't see any problems at first glance with the specification you've posted. Are you planning to try to implement this?

[phelix]

ftr: https://bitcointalk.org/index.php?topic=210876 [NMC] Brainstorm: File Signing


I gave it some more thought and IMHO we will have to verify the filename, too. Imagine the filename format.exe renamed to checkdisk.exe...

possibilities:
* filename included in the hash / in another hash round
"a2b3ca0dd00a3b95ee4772379336e963ebbe2341d69ba1839739893a5edd5c80"
* filename plus full hash
filename_a2b3ca0dd00a3b95ee4772379336e963ebbe2341d69ba1839739893a5edd5c80
* filename plus partial hash
filename_a2b3ca0dd0

I am not sure if there would be any advantage/disadvantage of having the filename included in clear.

We could also include the file size in the hash (like a salt).


The actual filename should be marked so the user knows he can verify it with our system: filename_#nsig.tar.bz


I am still pondering about the suggested update mechanism. Via SPV and some blocks around name_new / name_update we could prove that the information was in the blockchain at a specified point of time (e.g. when queried from a server).
All information should be contained in the latest update so that the blockchain does not have to be scanned (and the data can still be extended, also be changed).

We could also include a link and an info field.

What about abbreviating "method" and "signature" to save room in the blockchain? With 520 bytes there is room for three signatures.

Code: Select all

{
  fn : filename,
  i : info,
  url : link,
  sigs : [ {
      n : "id/domob",
      m : "sha256",
      sig: "HDRw84xtECK24iia+yyQBoASkcUsFPUHKG+Iou5oXXdTc14g5Cp+TZcNTi2qVjdN444vBS3We98r9+h5iiBDJLA="
    },
  ]
  import : name,
}

[biolizard89]

ftr: https://bitcointalk.org/index.php?topic=210876 [NMC] Brainstorm: File Signing


I gave it some more thought and IMHO we will have to verify the filename, too. Imagine the filename format.exe renamed to checkdisk.exe...

possibilities:
* filename included in the hash / in another hash round
"a2b3ca0dd00a3b95ee4772379336e963ebbe2341d69ba1839739893a5edd5c80"
* filename plus full hash
filename_a2b3ca0dd00a3b95ee4772379336e963ebbe2341d69ba1839739893a5edd5c80
* filename plus partial hash
filename_a2b3ca0dd0

I am not sure if there would be any advantage/disadvantage of having the filename included in clear.

I think having the filename in the clear would be a privacy leak; some users might not want the whole world to know that their Namecoin address signed TheFederalistPapers.pdf. Including an unsalted hash wouldn't be much better, assuming that an adversary might know which filenames to look for (which is plausible in some cases). I think hashing both the file contents and the filename into a single hash would be the best way to do it, so that an adversary wouldn't be able to identify the file unless he/she already possesses the entire file.

We could also include the file size in the hash (like a salt).


The actual filename should be marked so the user knows he can verify it with our system: filename_#nsig.tar.bz


I am still pondering about the suggested update mechanism. Via SPV and some blocks around name_new / name_update we could prove that the information was in the blockchain at a specified point of time (e.g. when queried from a server).
All information should be contained in the latest update so that the blockchain does not have to be scanned (and the data can still be extended, also be changed).

Agreed, relying on expired values is not a good way to do it IMHO, particularly since in the future it is likely that expired values wil be pruned from storage.

We could also include a link and an info field.

What about abbreviating "method" and "signature" to save room in the blockchain? With 520 bytes there is room for three signatures.

Code: Select all

{
  fn : filename,
  i : info,
  url : link,
  sigs : [ {
      n : "id/domob",
      m : "sha256",
      sig: "HDRw84xtECK24iia+yyQBoASkcUsFPUHKG+Iou5oXXdTc14g5Cp+TZcNTi2qVjdN444vBS3We98r9+h5iiBDJLA="
    },
  ]
  import : name,
}

The import field is definitely necessary to support. The info/url fields are also a good idea, although presumably a lot of people wouldn't use them for privacy reasons.

[phelix]

ftr: https://bitcointalk.org/index.php?topic=210876 [NMC] Brainstorm: File Signing


I gave it some more thought and IMHO we will have to verify the filename, too. Imagine the filename format.exe renamed to checkdisk.exe...

possibilities:
* filename included in the hash / in another hash round
"a2b3ca0dd00a3b95ee4772379336e963ebbe2341d69ba1839739893a5edd5c80"
* filename plus full hash
filename_a2b3ca0dd00a3b95ee4772379336e963ebbe2341d69ba1839739893a5edd5c80
* filename plus partial hash
filename_a2b3ca0dd0

I am not sure if there would be any advantage/disadvantage of having the filename included in clear.

I think having the filename in the clear would be a privacy leak; some users might not want the whole world to know that their Namecoin address signed TheFederalistPapers.pdf. Including an unsalted hash wouldn't be much better, assuming that an adversary might know which filenames to look for (which is plausible in some cases). I think hashing both the file contents and the filename into a single hash would be the best way to do it, so that an adversary wouldn't be able to identify the file unless he/she already possesses the entire file.

Yeah, privacy is a strong argument. With the optional fn field everyone can have their way.

spec
What about:

name : base58(sha256(strutf8(filename) + str(sizeinbytes(file) + sha256(fileopenedasbinary))[:20])

{
sigs : [ {
n : "id/domob", # n for name
a : "sha256", # a for algorithm as in shasum -a 256 ?
sig: "HDRw84xtECK24iia+yyQBoASkcUsFPUHKG+Iou5oXXdTc14g5Cp+TZcNTi2qVjdN444vBS3We98r9+h5iiBDJLA="
},
]
fn : filename, # optional
i : "info about the file", # optional
t : "tags,tag1,tag2", # optional comma separated, no spaces bet
dl : "link to download the file", # optional https:/asdasdf.com/sdafdasdf.zip
w : "link to website with download / info on the file", # optional https:/asdasdf.com/download
imp : "name to import", # optional
}

Spaces should be removed from JSON.

Namespace
file/ ?

[domob]

Yeah, privacy is a strong argument. With the optional fn field everyone can have their way.

I also don't want the filename to be (mandatorily) in the clear and support the "hash filename + content together" option.

name : base58(sha256(strutf8(filename) + str(sizeinbytes(file) + sha256(fileopenedasbinary))[:20])

Sounds good. Just why do you want to include the size into the hash? Isn't this redundant when the full binary content is hashed anyway? If you think it still is useful, I'm not opposed to it either. (Just not sure whether it is necessary or would just complicate things.)

{
sigs : [ {
n : "id/domob", # n for name
a : "sha256", # a for algorithm as in shasum -a 256 ?
sig: "HDRw84xtECK24iia+yyQBoASkcUsFPUHKG+Iou5oXXdTc14g5Cp+TZcNTi2qVjdN444vBS3We98r9+h5iiBDJLA="
},
]
fn : filename, # optional
i : "info about the file", # optional
t : "tags,tag1,tag2", # optional comma separated, no spaces bet
dl : "link to download the file", # optional https:/asdasdf.com/sdafdasdf.zip
w : "link to website with download / info on the file", # optional https:/asdasdf.com/download
imp : "name to import", # optional
}

I don't know the details about import, but from what I grasped in the discussion yesterday it seems useful - not sure whether or not it is supposed to be always called "import" (if that's some kind of a "standardised" feature) and abbreviating it is ok? I'm also not sure about "dl" and "w" - I would prefer a "uri" field which is much clearer, which would be the link to download the file. You can add an additional optional field like your "w" for a descriptional page. But that is just a suggestion, I'm also fine with "dl" and "w".

If you want to further reduce the size, what about this structure for the "sigs" part:

Code: Select all

{
  "id/domob": { ... a / sig part ... },
  "d/domob": { ... a / sig part ...}
}

I. e., turning it into an associative array / JSON object where the signing names are the keys? This would make it also easier to check just whether a particular name I'm interested in has signed the file. (Although I don't think this will help much in practice.)

Namespace
file/ ?

Sounds good, but maybe we could also use the more generic "hash/" I suggested. I think I read already that the "Holy Grail" guys discussed about putting hashes into the blockchain (supposedly for a similar feature but not to verify files but some OT datastructures or so), and then it may make sense to have some kind of "generic" namespace and specification for how to do this to verify "things". What do you think?

Regarding implementation: Phelix mentioned interest in working on it yesterday - are you still interested? If so, I'll leave it to you and try my hands on Pidgin-OTR integration as next project. (The OTR guys seemed to be interested in the general idea, although they don't want to include it into the "stock" OTR plugin - but am willing to adapt it so it can work when the user installs a "namecoin" plugin alongside.) I can work on a Gtk / GNU/Linux desktop integration UI when the "core" is done.

[domob]

Something else to think about: Signing is not actually done with a name itself (how would it) but rather the address the name is (currently) associated with. This is no problem because the name owner of course also owns that address. However, when in the future a name_update is done, presumably the address changes (at least it can change).

For NameID for instance, this is no problem because the name/address signatures used there are only used temporarily to log in and checked right-away. But with this file verification architecture, it may happen that someone wants to verify an old signature but the signing name is in the mean time at a different address, right? So we probably need extra rules to look at when the signature was made (the block height of the corresponding hash/ name update) and retrieve the address the signing name was at then to verify. (?)

[phelix]

Yeah, privacy is a strong argument. With the optional fn field everyone can have their way.

I also don't want the filename to be (mandatorily) in the clear and support the "hash filename + content together" option.

name : base58(sha256(strutf8(filename) + str(sizeinbytes(file) + sha256(fileopenedasbinary))[:20])

Sounds good. Just why do you want to include the size into the hash? Isn't this redundant when the full binary content is hashed anyway? If you think it still is useful, I'm not opposed to it either. (Just not sure whether it is necessary or would just complicate things.)

To be useful as something like a salt it would also be necessary to add the size in clear. I have no argument how it would be a benefit so away with it - simple is better.

Code: Select all

name : base58(sha256(strutf8(filename) + sha256(fileopenedasbinary))[:20])

What about the length of the name? I think 20 characters of base58 should be enough, do you guys agree? A Bitcoin address is about 30 characters long but in this case a collision would be harmless.

note: fileopenedasbinary - this can make a difference for text files on windows I think

{
sigs : [ {
n : "id/domob", # n for name
a : "sha256", # a for algorithm as in shasum -a 256 ?
sig: "HDRw84xtECK24iia+yyQBoASkcUsFPUHKG+Iou5oXXdTc14g5Cp+TZcNTi2qVjdN444vBS3We98r9+h5iiBDJLA="
},
]
fn : filename, # optional
i : "info about the file", # optional
t : "tags,tag1,tag2", # optional comma separated, no spaces bet
dl : "link to download the file", # optional https:/asdasdf.com/sdafdasdf.zip
w : "link to website with download / info on the file", # optional https:/asdasdf.com/download
imp : "name to import", # optional
}

I don't know the details about import, but from what I grasped in the discussion yesterday it seems useful - not sure whether or not it is supposed to be always called "import" (if that's some kind of a "standardised" feature) and abbreviating it is ok?

It has never been actually used but it is defined as "import" in two other namespaces. Ok let's stick to it. What a waste.

I'm also not sure about "dl" and "w" - I would prefer a "uri" field which is much clearer, which would be the link to download the file. You can add an additional optional field like your "w" for a descriptional page. But that is just a suggestion, I'm also fine with "dl" and "w".

ok, "uri".

If you want to further reduce the size, what about this structure for the "sigs" part:

Code: Select all

{
  "id/domob": { ... a / sig part ... },
  "d/domob": { ... a / sig part ...}
}

I. e., turning it into an associative array / JSON object where the signing names are the keys? This would make it also easier to check just whether a particular name I'm interested in has signed the file. (Although I don't think this will help much in practice.)

What about one domain signing with several algorithms? OK, probably overkill, they should simply issue an update should an algorithm really be weakened.

Code: Select all

{
  sigs : {
      "id/domob" : {
        a : "sha256",    #  a for algorithm as in shasum -a 256     optional, default is sha256
        s: "HDRw84xtECK24iia+yyQBoASkcUsFPUHKG+Iou5oXXdTc14g5Cp+TZcNTi2qVjdN444vBS3We98r9+h5iiBDJLA="
      }
    },
  ]
  fn : filename,  #  optional
  i : "info about the file",   # optional
  t : "tags,tag1,tag2",  #  optional    comma separated, no spaces
  uri : "link to download the file",   # optional   https:/asdasdf.com/sdafdasdf.zip
  w : "link to website with download / info on the file",   # optional   https:/asdasdf.com/download
  import : "name to import",  #  optional
}

Namespace
file/ ?

Sounds good, but maybe we could also use the more generic "hash/" I suggested. I think I read already that the "Holy Grail" guys discussed about putting hashes into the blockchain (supposedly for a similar feature but not to verify files but some OT datastructures or so), and then it may make sense to have some kind of "generic" namespace and specification for how to do this to verify "things". What do you think?

I think they should use ot/ . hash/ is too general imho Would like to hear more opinions on this.

Regarding implementation: Phelix mentioned interest in working on it yesterday - are you still interested? If so, I'll leave it to you and try my hands on Pidgin-OTR integration as next project. (The OTR guys seemed to be interested in the general idea, although they don't want to include it into the "stock" OTR plugin - but am willing to adapt it so it can work when the user installs a "namecoin" plugin alongside.) I can work on a Gtk / GNU/Linux desktop integration UI when the "core" is done.

Yup, I would like to do it but would be glad about help on the Linux side.