NMControl and "import"

[domob]

I think this has already been discussed at some points in time before, but since I currently have to update a bunch of names because my VPS provider changed my server's IP address, I'd like to bring it up again: It would be really great if we had "import" support for the d/ namespace. The spec on the wiki describes lots of complicated things (importing for subdomains, overriding some imported variables with specific ones, "delegate" as well as "import", ...) which I haven't yet fully understood, and which I believe are not really necessary for most use cases.

What I would love to have is simple: Define my server's IP address plus its TLS fingerprint in a name (see dd/domob), and then have multiple .bit domains resolve to the same server (d/nameid, d/domob, d/id and so on - see d/kraftware for a test I did). As far as I can tell, NMControl doesn't support this yet. Is this correct, or did I simply do something wrong with d/kraftware and/or dd/domob?

While I haven't looked at the NMControl source yet (and I wasn't really able to find my way through it the last time I tried since it seems to be spread out a lot - and I'm no Python expert in addition), I think that it shouldn't be hard to implement at least this simple use of "import": Just one level to prevent DoS via recursion and things like this, and whenever a JSON object contains an import field, just replace it with that other name's JSON object (or mix the variables in). What do the NMControl experts think about this, am I completely off here or could this indeed be done easily? It would be of great use, I believe, and at least very handy to me in the current situation, for instance. If it really is as "easy" as it seems to me and you agree on the plan, I can try to write a patch. I think that in that case I should be able to get it done.

[khal]

The import feature should not be too difficult.

For the non-recursion part, I'm not sure there is a "simple" way to do it.

You could give it a try and if you don't find your way in my labyrinth I'll do it

[domob]

I did already: https://github.com/namecoin/nmcontrol/pull/5 But I like the suggestion to move things to a central location, will do that and resubmit as soon as possible.

[biolizard89]

I think this has already been discussed at some points in time before, but since I currently have to update a bunch of names because my VPS provider changed my server's IP address, I'd like to bring it up again: It would be really great if we had "import" support for the d/ namespace. The spec on the wiki describes lots of complicated things (importing for subdomains, overriding some imported variables with specific ones, "delegate" as well as "import", ...) which I haven't yet fully understood, and which I believe are not really necessary for most use cases.

What I would love to have is simple: Define my server's IP address plus its TLS fingerprint in a name (see dd/domob), and then have multiple .bit domains resolve to the same server (d/nameid, d/domob, d/id and so on - see d/kraftware for a test I did). As far as I can tell, NMControl doesn't support this yet. Is this correct, or did I simply do something wrong with d/kraftware and/or dd/domob?

While I haven't looked at the NMControl source yet (and I wasn't really able to find my way through it the last time I tried since it seems to be spread out a lot - and I'm no Python expert in addition), I think that it shouldn't be hard to implement at least this simple use of "import": Just one level to prevent DoS via recursion and things like this, and whenever a JSON object contains an import field, just replace it with that other name's JSON object (or mix the variables in). What do the NMControl experts think about this, am I completely off here or could this indeed be done easily? It would be of great use, I believe, and at least very handy to me in the current situation, for instance. If it really is as "easy" as it seems to me and you agree on the plan, I can try to write a patch. I think that in that case I should be able to get it done.

Sorry I'm late to the party. (Are e-mail notifications broken on the forum?)

Here's my thoughts on "import".

Any "import" implementation should have the ability to specify a subset of dd/ fields which get imported into a d/ name, and anything in the d/ name should override the dd/ name. There are security reasons for this. E.g. I may want my TLS fingerprint to be in cold storage (d/), while pulling my IP from hot wallet (dd/), but specifically disallowing anything else e.g. an Onion addresses from being imported from hot wallet.

Here's an example spec:

"import" : [["dd/myhotwallet", "ip", 1], ["dd/myhotwallet", "ip6"], ["dd/mycoldwallet", "i2p", "b32"], ["dd/mycoldwallet", "tls"]]

This imports the 2nd IPv4 address and all IPv6 addresses from a hot wallet, and an I2P base32 address and TLS configuration from a cold wallet. All other fields are not imported. To import an entire name's contents instead of one field, just specify the name by itself without any fields, e.g. the last entry in this:

"import" : [["dd/myhotwallet", "ip", 1], ["dd/myhotwallet", "ip6"], ["dd/mycoldwallet"]]

This spec can be a little bit verbose in a few cases, but it's flexible and has simple syntax, and if we wanted to save space we would be using gzip encoding which would eliminate any redundancy.

Comments?

[domob]

Any "import" implementation should have the ability to specify a subset of dd/ fields which get imported into a d/ name, and anything in the d/ name should override the dd/ name. There are security reasons for this. E.g. I may want my TLS fingerprint to be in cold storage (d/), while pulling my IP from hot wallet (dd/), but specifically disallowing anything else e.g. an Onion addresses from being imported from hot wallet.

Here's an example spec:

"import" : [["dd/myhotwallet", "ip", 1], ["dd/myhotwallet", "ip6"], ["dd/mycoldwallet", "i2p", "b32"], ["dd/mycoldwallet", "tls"]]

This imports the 2nd IPv4 address and all IPv6 addresses from a hot wallet, and an I2P base32 address and TLS configuration from a cold wallet. All other fields are not imported. To import an entire name's contents instead of one field, just specify the name by itself without any fields, e.g. the last entry in this:

"import" : [["dd/myhotwallet", "ip", 1], ["dd/myhotwallet", "ip6"], ["dd/mycoldwallet"]]

This spec can be a little bit verbose in a few cases, but it's flexible and has simple syntax, and if we wanted to save space we would be using gzip encoding which would eliminate any redundancy.

Comments?

I understand the concerns here, and I think the points are valid. In addition, I think it should be not too hard to implement (my patch already should do the "d/ overrides dd/" thing, but I have not tested it), and can be done easily as an extension when I've finished the basic support.

I also believe that I roughly understand how your system works, but there are still some open questions I have. Do you want to write the spec in more details so everything is clear and explicitly specified?

[biolizard89]

Any "import" implementation should have the ability to specify a subset of dd/ fields which get imported into a d/ name, and anything in the d/ name should override the dd/ name. There are security reasons for this. E.g. I may want my TLS fingerprint to be in cold storage (d/), while pulling my IP from hot wallet (dd/), but specifically disallowing anything else e.g. an Onion addresses from being imported from hot wallet.

Here's an example spec:

"import" : [["dd/myhotwallet", "ip", 1], ["dd/myhotwallet", "ip6"], ["dd/mycoldwallet", "i2p", "b32"], ["dd/mycoldwallet", "tls"]]

This imports the 2nd IPv4 address and all IPv6 addresses from a hot wallet, and an I2P base32 address and TLS configuration from a cold wallet. All other fields are not imported. To import an entire name's contents instead of one field, just specify the name by itself without any fields, e.g. the last entry in this:

"import" : [["dd/myhotwallet", "ip", 1], ["dd/myhotwallet", "ip6"], ["dd/mycoldwallet"]]

This spec can be a little bit verbose in a few cases, but it's flexible and has simple syntax, and if we wanted to save space we would be using gzip encoding which would eliminate any redundancy.

Comments?

I understand the concerns here, and I think the points are valid. In addition, I think it should be not too hard to implement (my patch already should do the "d/ overrides dd/" thing, but I have not tested it), and can be done easily as an extension when I've finished the basic support.

I also believe that I roughly understand how your system works, but there are still some open questions I have. Do you want to write the spec in more details so everything is clear and explicitly specified?

Sure, I'll try to be a bit more specific. The "import" field will take an array of things to import. Each item in this array will be its own array, with at least 1 item (the name to import from). Subsequent items choose smaller parts to import.

So, this:

"import": [["dd/name1"]]

Will take the JSON in dd/name1, and import all of it (except things that are already defined in the main name.

This:

"import": [["dd/name1", "ip"]]

Will take the JSON, and will apply ["ip"] to it (selecting just the JSON of the "ip" field) before importing.

"import": [["dd/name1", "ip", 2]]

Is similar; it takes the JSON, and applies ["ip"][2] to it (so it selects the 3rd entry in the array that makes up the content of the "ip" field).

Multiple imports can be specified by putting more than 1 entry in the array. Earlier entries have priority. If an import entry's target does not exist in the blockchain (either because the name doesn't exist or because the name doesn't have the specified JSON field in it), it is ignored. It would be nice to have a getDataImportErrors RPC call which lists targets that don't exist.

[domob]

Thanks for the clarification; I already supposed it was something like this (specifying the path elements in the JSON tree). What you do not specify, however, is where the entries are put in the scope into which they are imported. Presumably, you want to import the full "JSON tree branch" specified and put it into the current scope. Is this correct? For arrays, you need an extra rule (so that ["dd/name", "ip", 1] gets imported into ip[0] at the parent scope, since ip[1] makes no sense if there's no ip[0]). I was mainly thinking also about these details when asking for more info. Do you think this is alreay ripe for a full spec, or does it need more discussions?

And I would prefer to have the possibility to import a single name like this:

Code: Select all

{"import":"dd/data"}

Just for simplicity, it would be equal to enclosing the string into two layers of arrays. Do you think there's a problem with that?

[biolizard89]

Thanks for the clarification; I already supposed it was something like this (specifying the path elements in the JSON tree). What you do not specify, however, is where the entries are put in the scope into which they are imported. Presumably, you want to import the full "JSON tree branch" specified and put it into the current scope. Is this correct? For arrays, you need an extra rule (so that ["dd/name", "ip", 1] gets imported into ip[0] at the parent scope, since ip[1] makes no sense if there's no ip[0]). I was mainly thinking also about these details when asking for more info. Do you think this is alreay ripe for a full spec, or does it need more discussions?

And I would prefer to have the possibility to import a single name like this:

Code: Select all

{"import":"dd/data"}

Just for simplicity, it would be equal to enclosing the string into two layers of arrays. Do you think there's a problem with that?

Yeah, I'm envisioning placing it into the current scope. It just occured to me that this could lead to interesting issues with "map", since what do you do if "import" is inside a "map"? My opinion is that to solve "map" and "import" coexisting, you don't remove "import" fields which couldn't be resolved to an item in a name. So since the contents of an "import.domain.bit" subdomain won't be equal to a Namecoin name/path, the "import" field wouldn't be removed by nmcontrol when it's being used as a subdomain name.

Right, for arrays you need to take all imported indices and collapse them into a new array (as you state).

We've been trying to discuss the "import" field for months and haven't made a lot of progress, I would vote for implementing something but leaving details open to change if someone finds something broken in the spec.

[domob]

We've been trying to discuss the "import" field for months and haven't made a lot of progress, I would vote for implementing something but leaving details open to change if someone finds something broken in the spec.

I'm all for this. I'll at least continue my patch for the most basic import support in nmcontrol until it can be pushed, since I want to use it myself right now and not update all names to the new IP and still duplicate the data in all of them. If we later decide on more features to allow more detailed control, it shouldn't be hard to extend the import logic. (And I even volunteer to do it if I'm not too busy.)