"import" specification

phelix
Posts: 1634
Joined: Thu Aug 18, 2011 6:59 am

"import" specification

Post by phelix »

http://dot-bit.org/Namespace:Domain_nam ... delegation

Should it be recursive?

Can we use "import" or should we use something new (shorter)? From the wiki it seems to be specific to domains (it is automatically going down "map" - or is it an error?). I would prefer it to be general and to work with general json entries.

Maybe we can lean on something existing like the behavior of the python "import" statement?

The ability of the current suggestion in the wiki to be able to only import specific parts looks elegant but is is really useful?


I would expect it to work like this: Update/overwrite the current dictionary at the current level in the order of appearance of import entries.

What should happen if there are several import statements on one level?
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

domob
Posts: 1127
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: "import" specification

Post by domob »

phelix wrote:http://dot-bit.org/Namespace:Domain_nam ... delegation

Should it be recursive?

Can we use "import" or should we use something new (shorter)? From the wiki it seems to be specific to domains (it is automatically going down "map" - or is it an error?). I would prefer it to be general and to work with general json entries.

Maybe we can lean on something existing like the behavior of the python "import" statement?

The ability of the current suggestion in the wiki to be able to only import specific parts looks elegant but is is really useful?


I would expect it to work like this: Update/overwrite the current dictionary at the current level in the order of appearance of import entries.

What should happen if there are several import statements on one level?
I would suggest to add any keys present in the imported name, but have them overwritten by keys explicitly specified in the value. Multiple imports could be done using an array of names as value of "import", and having them processed in this order. I think we should make import non-recursive - both for performance/security reasons and because it is not really necessary (IMHO - but with non-recursive import the effective value size is again restricted by a finite bound; maybe we should restrict the maximum recursion depth instead?).

General JSON entries seems like what we should do. Which actual use-cases of Namecoin interpret import is up to them, I think we should just define the general behaviour of import and let "everyone" (DNS / NMControl, NameID, ...) decide for themselves whether or not to make use of this feature. (This means, I'm not in favour of specifying that *every* Namecoin value should be interpreted as if import as processed, only how importing should work if it is wanted. Like the "signer" field we recently discussed.)

Several import statements shouldn't be possible, at least I think that JSON disallows that. (Although of course the textual value could include them, but wouldn't this be invalid JSON then and thus something we should disallow?) Use name arrays instead.

Another question: Should import work only on the top-level JSON scope of a value, or for any object within the JSON value? (Like the "map" entry of a DNS name.) But I think this should be defined by the actual use-case specification for each namespace.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: "import" specification

Post by biolizard89 »

A few points.

1. "import" should not assume the "map" field; it should be generic JSON (so works with other namespaces than d/).

2. Fields added by "import" should be overridden by the existing fields. This is critical to allow cold storage to work.

3. Allowing partial import (only import a subset of fields) is critical to allow cold storage to work.

4. Recursive import is necessary to allow a logical name to span large numbers of raw names. If someone is willing to pay the fees for 50 names, they should be allowed to use them as subdomains instead of requiring them to be separate 2nd-level domains. It might be useful to include a limit on recursive depth, and recursive loops should be ignored.

And yes, we should get this spec figured out. Thanks for bringing it up. :)
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

agorism1
Posts: 3
Joined: Sun Oct 13, 2013 5:26 pm
os: linux

Re: "import" specification

Post by agorism1 »

I have attempted to implement the "import" feature. It is all in python.
Import is (should be) recursive in my design.
It has not been tested much yet, but I also added the ability to leave posts on an your ID. or to leave comments on your own or other people's posts.

I wrote it for namecoind (it uses your browser as the GUI) , and will be updating for namecoinq soon.

https://rlevykyswlg7ibqi.onion.to/

Hopefully this will make namecoin's value go up.

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: "import" specification

Post by biolizard89 »

agorism1 wrote:I have attempted to implement the "import" feature. It is all in python.
Import is (should be) recursive in my design.
It has not been tested much yet, but I also added the ability to leave posts on an your ID. or to leave comments on your own or other people's posts.

I wrote it for namecoind (it uses your browser as the GUI) , and will be updating for namecoinq soon.

https://rlevykyswlg7ibqi.onion.to/

Hopefully this will make namecoin's value go up.
Seeing as the spec is not decided on, I'm not sure why you implemented something without being involved in the spec design process. Also, my opinion is that this should be implemented within nmcontrol.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

domob
Posts: 1127
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: "import" specification

Post by domob »

biolizard89 wrote:Seeing as the spec is not decided on, I'm not sure why you implemented something without being involved in the spec design process. Also, my opinion is that this should be implemented within nmcontrol.
+1 for nmcontrol. However, I think there will be alternative implementations, too, as not everyone might want to use nmcontrol. (Not sure about NameID, for instance.) But for them, I think it should be rather easy to implement "import" anyway (once the specification is defined precisely).
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

phelix
Posts: 1634
Joined: Thu Aug 18, 2011 6:59 am

Re: "import" specification

Post by phelix »

domob wrote: I would suggest to add any keys present in the imported name, but have them overwritten by keys explicitly specified in the value.
This seems counter intuitive to me. I would expect it to work like the python import, overwriting everything that is present by what is being imported. Any particular reason for this behaviour?
Multiple imports could be done using an array of names as value of "import", and having them processed in this order.
+1
I think we should make import non-recursive - both for performance/security reasons and because it is not really necessary (IMHO - but with non-recursive import the effective value size is again restricted by a finite bound; maybe we should restrict the maximum recursion depth instead?).
If recursive depth certainly would have to be restricted tightly to make ddos attacks less easy. 0, 1, 2 levels?
General JSON entries seems like what we should do. Which actual use-cases of Namecoin interpret import is up to them, I think we should just define the general behaviour of import and let "everyone" (DNS / NMControl, NameID, ...) decide for themselves whether or not to make use of this feature. (This means, I'm not in favour of specifying that *every* Namecoin value should be interpreted as if import as processed, only how importing should work if it is wanted. Like the "signer" field we recently discussed.)
+1
Several import statements shouldn't be possible, at least I think that JSON disallows that. (Although of course the textual value could include them, but wouldn't this be invalid JSON then and thus something we should disallow?) Use name arrays instead.
Python json only seems to use the last occurrence.
Another question: Should import work only on the top-level JSON scope of a value, or for any object within the JSON value? (Like the "map" entry of a DNS name.) But I think this should be defined by the actual use-case specification for each namespace.
Hmm... this would allow for several imports per name. Should we limit the total number of allowed imports? Or the size of the resulting object?
biolizard89 wrote: 1. "import" should not assume the "map" field; it should be generic JSON (so works with other namespaces than d/).
Check, no "map".
2. Fields added by "import" should be overridden by the existing fields. This is critical to allow cold storage to work.
As domob suggested... but I don't get it. Why would I use import but also have the field present in the actual value? Could you please elaborate?
3. Allowing partial import (only import a subset of fields) is critical to allow cold storage to work.
...
4. Recursive import is necessary to allow a logical name to span large numbers of raw names. If someone is willing to pay the fees for 50 names, they should be allowed to use them as subdomains instead of requiring them to be separate 2nd-level domains. It might be useful to include a limit on recursive depth, and recursive loops should be ignored.
Ignoring loops sure is a good idea.
agorism1 wrote:I have attempted to implement the "import" feature. It is all in python.
Import is (should be) recursive in my design.
It has not been tested much yet, but I also added the ability to leave posts on an your ID. or to leave comments on your own or other people's posts.

I wrote it for namecoind (it uses your browser as the GUI) , and will be updating for namecoinq soon.

https://rlevykyswlg7ibqi.onion.to/

Hopefully this will make namecoin's value go up.
Is the code available somewhere? For this application there should not be a difference between classic namecoind and namecoinq daemon/qt.
domob wrote:
biolizard89 wrote:Seeing as the spec is not decided on, I'm not sure why you implemented something without being involved in the spec design process. Also, my opinion is that this should be implemented within nmcontrol.
+1 for nmcontrol. However, I think there will be alternative implementations, too, as not everyone might want to use nmcontrol. (Not sure about NameID, for instance.) But for them, I think it should be rather easy to implement "import" anyway (once the specification is defined precisely).
It needs to be in NMControl. Maybe in a modular way so that the code can easily be integrated into other software.

Should we use "import" or should we use something new (shorter)? What about "imp" instead of import? It would also avoid any potential conflict with the existing "import" (e.g. the "map" stuff).
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: "import" specification

Post by biolizard89 »

phelix wrote:
2. Fields added by "import" should be overridden by the existing fields. This is critical to allow cold storage to work.
As domob suggested... but I don't get it. Why would I use import but also have the field present in the actual value? Could you please elaborate?
You may be right that this doesn't matter as long as users are competent.
phelix wrote:
3. Allowing partial import (only import a subset of fields) is critical to allow cold storage to work.
...
Okay, here's an example:

d/wikileaks: {"fingerprint": "[WIKILEAKS-FINGERPRINT]", import: [["dd/wikileaks-ip", "ip"]] }
dd/wikileaks-ip: {"ip": "123.45.67.89"}

(I made up a syntax for import for the purposes of this example.)

d/wikileaks is held in cold storage. dd/wikileaks-ip is more easily accessible in case the feds raid the server farm and WikiLeaks needs to switch IP's. Now, what happens if the feds hijack dd/wikileaks-ip, and change its value to this: {"ip": "456.78.91.23", "fingerprint": "[MALICIOUS-FINGERPRINT]"}

Basically, we want to make sure that the malicious fingerprint doesn't override the fingerprint in cold storage. Disallowing overrides by itself isn't enough (what if the hijacked name uses the tls field instead of fingerprint?)
phelix wrote:
domob wrote:
biolizard89 wrote:Seeing as the spec is not decided on, I'm not sure why you implemented something without being involved in the spec design process. Also, my opinion is that this should be implemented within nmcontrol.
+1 for nmcontrol. However, I think there will be alternative implementations, too, as not everyone might want to use nmcontrol. (Not sure about NameID, for instance.) But for them, I think it should be rather easy to implement "import" anyway (once the specification is defined precisely).
It needs to be in NMControl. Maybe in a modular way so that the code can easily be integrated into other software.

Should we use "import" or should we use something new (shorter)? What about "imp" instead of import? It would also avoid any potential conflict with the existing "import" (e.g. the "map" stuff).
I think "import" is fine. As far as I know there aren't any widely used implementations that use map, so I wouldn't worry about backward compatibility. The spec is labeled a draft for a reason. :)
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

phelix
Posts: 1634
Joined: Thu Aug 18, 2011 6:59 am

Re: "import" specification

Post by phelix »

OK, makes sense. Imported data shall be "weaker" than existing data. Actually the current spec says so, too: "data is imported to the current context and then overridden by explicitly provided entries"

"import"/"imp"... ok, clarity over saving space. :|
Limit to one name recursion? name0:import name:1import name2
Arrays - who is stronger, first or last? I would say first is stronger but only to be compliant.
With two "import"s on the same level only the last one counts (seems to be JSON behavior)
No "map" stuff
self import is ignored

Loops shall be ignored - how precisely?

Should we limit the total number of allowed imports? Or the size of the resulting object? (imagine recursive imports on 20 levels --> 400 imports...). What about only allowing "import" on the root level?

"import": [ ["s/shareddata", "www"], ["s/shareddata", "ftp"] ]


Which actual use-cases of Namecoin interpret import is up to them, I think we should just define the general behaviour of import and let "everyone" (DNS / NMControl, NameID, ...) decide for themselves whether or not to make use of this feature. (This means, I'm not in favour of specifying that *every* Namecoin value should be interpreted as if import as processed, only how importing should work if it is wanted.
But I guess NMControl should always import...




Anything else?
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: "import" specification

Post by biolizard89 »

So, a few follow up questions:
  1. How do we handle cases where a namespace spec desires "import" to mean something namespace-specific? For example, the d/ spec allows having a subdomain called "import"; we don't want a conflict. We could try renaming the import operation to something more unique, e.g. "#import". Any symbol that we use should be generally unused by base64, DNS, or email addresses, to name a few; a # seems reasonable by first glance.
  2. Right now "import" assumes that the imported name's contents are placed into a dict. This make sense for most things, but what about the use case where we're importing two strings or two arrays and want to concatenate them? Define a "#concat" field as well? What would be the syntax?
  3. Hugo has some concerns about doing this on the JSON level rather than namespace/map level... I'll let him describe his concerns here.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

Post Reply