Preferred language for NMControl-like things

hla
Posts: 46
Joined: Mon Nov 10, 2014 12:01 am
os: linux
Contact:

Re: Preferred language for NMControl-like things

Post by hla » Tue Sep 15, 2015 7:11 pm

Code: Select all

        hl | indolering: Alright. Can I record your vote as abstain then? 
indolering | hl: sure.
Now in favour: brand0, Joseph, me, sugarpuff, Jeremy Rand, ryan-c
No strong preference: indolering, Domob
Now against: phelix (I infer)
Yet to express a preference: cassini

biolizard89
Posts: 1933
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Preferred language for NMControl-like things

Post by biolizard89 » Wed Sep 16, 2015 3:15 am

FWIW, my preference is to do a "trial run". I.e., try to develop with the ncdns codebase for somewhere between a few weeks and a couple months, and see how things go. If there are showstoppers, we can reverse course and use NMControl. That will also help the end users (and developers) decide.

Reproducible builds are very important, and I don't think Joseph or I want the project to be completely dependent something that Tor devs were unable to do with several orders of magnitude more funding.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

domob
Posts: 1083
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Preferred language for NMControl-like things

Post by domob » Wed Sep 16, 2015 5:46 am

biolizard89 wrote:Reproducible builds are very important, and I don't think Joseph or I want the project to be completely dependent something that Tor devs were unable to do with several orders of magnitude more funding.
I agree that deterministic builds are important, but what is really stopping us from simply running the source through a Python interpreter? I thought that's how it is supposed to be done anyway. With respect to a reproducible Python interpreter: What's the point if you can simply use the one bundled and signed by a major distro? You have to trust "some" component of your OS anyway.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

phelix
Posts: 1626
Joined: Thu Aug 18, 2011 6:59 am

Re: Preferred language for NMControl-like things

Post by phelix » Wed Sep 16, 2015 9:16 am

biolizard89 wrote: Reproducible builds are very important, and I don't think Joseph or I want the project to be completely dependent something that Tor devs were unable to do with several orders of magnitude more funding.
At this moment reproducible builds are of secondary priority. What is important is that we get out a working product.
hla wrote:
phelix wrote:IMHO we should not discard all the work we have put into NMControl when we are just about to have a usable product. It simply does not make sense to restart now.
ncdns (which provides DNS resolution) works today. It is the only currently available implementation of domain-names.md, as far as I'm aware. So if we compare the completion of the two implementations, ncdns wins, unless I'm mistaken.
domain-names.md is not the benchmark. Is ncdns ready to be deployed as a product (on Windows)?
phelix wrote:Why not have a competing second program? I think Go is a very interesting language but I for myself would have to dig into Go first which would take time and effort I would rather spend on other things. If you and others think they can create something better than current NMControl it's fine with me. But we should let the users decide and not force a single "official" software on them.
Who's suggesting we do that?
Well, as far as I understand the still vague question at hand it is suggested to "switch" from NMControl to something new based on ncdns.
People can use what they like - but it doesn't make sense to divide the limited energies available between two middleware stacks within the context of the project.
It does not make sense to have all developers switch from one language to another in the middle of a project either. Particularly not so as most other code and scripts are in Python, too. This will forbid some synergy effects (we are only partially using so far) and will divide the team.
phelix wrote:Whereas Go might be easier to create reproducible builds it is possible to simply run the python source directly (e.g. with Bitmessage this is pretty common).
I believe the issue pertains to creating reproducible builds of the Python interpreter itself.
Huh? At some point you will need to have a compiler for Go, too.
phelix wrote:DNSSEC is low priority from my current point of view, if not even a dead end: https://forum.namecoin.info/posting.php ... 70&p=15526
DNSSEC is essential to allowing d/ names to securely delegate to nameservers, which is necessary for large zones.
Who cares about large zones?
Moreover, the issues with getting DNSSEC support into browsers are basically the same issues as getting Namecoin TLSA verification support into browsers. They are essentially the same problem.
For TLS support it is not necessary for the browser to support anything but a (socks) proxy.


As far as I see it the two main arguments for Go, reproducible builds and a DNSSEC library are void.

phelix wrote:
biolizard89 wrote:in Python it is difficult to know which exceptions to catch.
I don't see how you could get around this in any language. It was never a problem for though.
If the fallibility of a given function is explicitly declared, callers know they need to handle any errors which it returns. Unlike Python, Go is statically typed and uses returned error values. Therefore, it can be determined from the documentation whether the function can fail, and, when writing Go, all such failures are generally handled appropriately.
Wow, not having to worry about error handling is indeed an advantage. ;)


This has been stated before but I would like to repeat it here for the record: Decision finding in NMControl development has already been slow and sometimes difficult in the past. I expect this to become way worse with three projects combined.
If you can find a good solution to this I will be interested.

I will continue working on NMControl as time permits and I hope others join me. But you are free to do your thing (btw please give it a proper name soon) and have it compete with NMControl. If you can pull it off and create something really better I will be happy to have one thing less on my list.
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

hla
Posts: 46
Joined: Mon Nov 10, 2014 12:01 am
os: linux
Contact:

Re: Preferred language for NMControl-like things

Post by hla » Wed Sep 16, 2015 4:38 pm

phelix wrote:domain-names.md is not the benchmark. Is ncdns ready to be deployed as a product (on Windows)?
Yes.
phelix wrote:Well, as far as I understand the still vague question at hand it is suggested to "switch" from NMControl to something new based on ncdns.
I understand the proposal is to switch to ncdns, not something based on it. Of course the codebase can be evolved from its current state to provide whatever functionality is required, but ncdns is complete.
phelix wrote:It does not make sense to have all developers switch from one language to another in the middle of a project either. Particularly not so as most other code and scripts are in Python, too. This will forbid some synergy effects (we are only partially using so far) and will divide the team.
But I think it's been carefully explained exactly why this switch makes sense.
phelix wrote:Who cares about large zones?
:roll:

Since you clearly don't care about this use case, I would suggest you retract your proposal to ban delegation to external nameservers, and thus make Namecoin unusable for anyone who does care about this use case.
phelix wrote:For TLS support it is not necessary for the browser to support anything but a (socks) proxy.
I am assuming you are advocating MitM-based support for custom validation. This is a highly undesirable hack and certainly should not be an end aspiration. Jeremy Rand and I are both opposed to the use of MitM-based methods if at all possible.
phelix wrote:As far as I see it the two main arguments for Go, reproducible builds and a DNSSEC library are void.
Do you want more? Python is lacking a mature DNS library. It is very unclear to me that tightly coupling NMControl with Unbound using Unbound's python extension interface will particularly solve this issue, and it creates undesirable tight coupling. Go has a mature, widely-used DNS library on which ncdns builds.
phelix wrote:Wow, not having to worry about error handling is indeed an advantage. ;)
Not sure what your position here is. "Writing correct programs is hard, screw that"?
phelix wrote:But you are free to do your thing (btw please give it a proper name soon) and have it compete with NMControl.
Huh? It already has a name, ncdns.

biolizard89
Posts: 1933
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Preferred language for NMControl-like things

Post by biolizard89 » Wed Sep 16, 2015 7:24 pm

hla wrote:
phelix wrote:domain-names.md is not the benchmark. Is ncdns ready to be deployed as a product (on Windows)?
Yes.
phelix wrote:Well, as far as I understand the still vague question at hand it is suggested to "switch" from NMControl to something new based on ncdns.
I understand the proposal is to switch to ncdns, not something based on it. Of course the codebase can be evolved from its current state to provide whatever functionality is required, but ncdns is complete.
I think Phelix may be talking about ncdns not having a REST interface. That said, REST is relatively straightforward to implement in Go, correct? Phelix may also be talking about the lack of identity-based functionality, which to be fair NMControl doesn't do a great job of either. My take is that the functionality that is missing from ncdns will probably be easier to implement than the functionality that's missing in NMControl. But, this is secondary to my concern about reproducible builds.
hla wrote:
phelix wrote:It does not make sense to have all developers switch from one language to another in the middle of a project either. Particularly not so as most other code and scripts are in Python, too. This will forbid some synergy effects (we are only partially using so far) and will divide the team.
But I think it's been carefully explained exactly why this switch makes sense.
phelix wrote:Who cares about large zones?
:roll:

Since you clearly don't care about this use case, I would suggest you retract your proposal to ban delegation to external nameservers, and thus make Namecoin unusable for anyone who does care about this use case.
phelix wrote:For TLS support it is not necessary for the browser to support anything but a (socks) proxy.
I am assuming you are advocating MitM-based support for custom validation. This is a highly undesirable hack and certainly should not be an end aspiration. Jeremy Rand and I are both opposed to the use of MitM-based methods if at all possible.
I would prefer to avoid intercepting proxies where feasible. I am not certain that we can completely avoid it for all use cases, but we should definitely be trying to implement TLS without intercepting proxies. Hugo and I have been hacking around with Firefox -- things were working until Firefox 39, at which point Mozilla changed the behavior of the API we were using. I think it's possible to work around this, but not sure. If Mozilla supported TLSA records, that would make a lot of the pain just go away.

DNSSEC support is not just for the purpose of delegating to ICANN domains (of which I'm not a huge fan), but also for making DNSSEC-aware applications understand Namecoin domains. E.g. TLSA and SSHFP records are only valid for DNSSEC-secured domains. Running a DNSSEC-aware server will in theory make TLSA and SSHFP records work out of the box -- note that OpenSSH already supports SSHFP, so this is not purely theoretical.
hla wrote:
phelix wrote:As far as I see it the two main arguments for Go, reproducible builds and a DNSSEC library are void.
Do you want more? Python is lacking a mature DNS library. It is very unclear to me that tightly coupling NMControl with Unbound using Unbound's python extension interface will particularly solve this issue, and it creates undesirable tight coupling. Go has a mature, widely-used DNS library on which ncdns builds.
Technically we could use madns with NMControl -- indeed I started on that but didn't get very far. Using ncdns is cleaner, though.
hla wrote:
phelix wrote:Wow, not having to worry about error handling is indeed an advantage. ;)
Not sure what your position here is. "Writing correct programs is hard, screw that"?
phelix wrote:But you are free to do your thing (btw please give it a proper name soon) and have it compete with NMControl.
Huh? It already has a name, ncdns.
I would favor renaming whatever we end up with to make it more clear that (1) it's not just for DNS, i.e. identities are covered too, and (2) it's not just for Namecoin, i.e. you could use it with other naming systems if you wanted to. This kind of depends on what architecture we want to go with -- do we put everything in one program, or have lots of little programs that call common libraries? (That's an open question that can be settled separately.) Also, sugarpuff really wants it to be clear that the code works with any naming system, if he is to be involved. (I still need to talk to sugarpuff about that.... I seem to have too much to do.)
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

hla
Posts: 46
Joined: Mon Nov 10, 2014 12:01 am
os: linux
Contact:

Re: Preferred language for NMControl-like things

Post by hla » Wed Sep 16, 2015 9:00 pm

I'm open to renaming. My only real preference is that the name be in some way functionality-derived rather than abstract (I don't particularly care for naming projects after arbitrary things, calling them "Whiskey" or something... it feels rather pretentious). Though this is hardly important if there are no good alternatives. OpenBSD rather exemplifies this practice. Does what it says on the tin and all that.

I was thinking, in order to deassociate ncdns from Namecoin and from DNS, some sort of contraction of 'byzantine name daemon' could work (e.g. bynad, byname, byzandae). But I'm open to suggestions, if anyone cares to make them.

(That said, eliminating the Namecoin association might be a big case of YAGNI due to the lack of other distributed key-value stores providing byzantine fault tolerance. Bitshares might be an option if they recover funding. GNUnet's name system is apparently DHT-based and not globally unique.)

phelix
Posts: 1626
Joined: Thu Aug 18, 2011 6:59 am

Re: Preferred language for NMControl-like things

Post by phelix » Thu Sep 17, 2015 2:11 pm

hla wrote:
phelix wrote:domain-names.md is not the benchmark. Is ncdns ready to be deployed as a product (on Windows)?
Yes.
Interesting. I'd like to give it a try.
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

hla
Posts: 46
Joined: Mon Nov 10, 2014 12:01 am
os: linux
Contact:

Re: Preferred language for NMControl-like things

Post by hla » Thu Sep 17, 2015 7:33 pm

phelix wrote:
hla wrote:
phelix wrote:domain-names.md is not the benchmark. Is ncdns ready to be deployed as a product (on Windows)?
Yes.
Interesting. I'd like to give it a try.
1. Install Go (1.4 or later should do fine).
2. git clone https://github.com/hlandau/ncdns.t
3. cd ncdns.t
4. make
5. mkdir etc
6. cp doc/ncdns.conf.example etc/ncdns.conf
7. Edit the configuration file and set at least bind, namecoinrpcaddress, namecoinrpcusername, namecoinrpcpassword. You may set httplistenaddr to enable the HTTP server (if the tpl directory cannot be located automatically, set tplpath).
If you don't care about DNSSEC, don't set the DNSSEC settings. If you want to use DNSSEC, see the README.

Run ncdns by running ./bin/ncdns from the ncdns.t directory. Do not run as root. (Due to a regression in Go 1.5 involving obscure interactions between glibc, the Linux kernel and setuid calls, it is not possible to bind to low ports using Go1.5. This bug will be fixed in Go1.5.2.)

I recommend you put ncdns on a high, unprivileged port and point to it from Unbound. Instructions for configuring Unbound are in the README. You can also test it by issuing queries to it directly. For example, if you place it on port 1153:
$ dig @127.0.0.1 -p1153 nf.bit.

Obviously, namecore needs to be running and accessible.

biolizard89
Posts: 1933
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Preferred language for NMControl-like things

Post by biolizard89 » Sat Sep 19, 2015 6:34 am

domob wrote:
biolizard89 wrote:Reproducible builds are very important, and I don't think Joseph or I want the project to be completely dependent something that Tor devs were unable to do with several orders of magnitude more funding.
I agree that deterministic builds are important, but what is really stopping us from simply running the source through a Python interpreter? I thought that's how it is supposed to be done anyway. With respect to a reproducible Python interpreter: What's the point if you can simply use the one bundled and signed by a major distro? You have to trust "some" component of your OS anyway.
Whoops, totally missed this post, sorry Daniel.

Major Linux distros are working on making all their packages reproducible (particularly Debian, though also Fedora). Armory is basically working in Debian's reproducible build toolchain (unless I'm misremembering what Joseph said), so Python on Debian-based OS's is reproducible. So for Linux users, this is less of an issue. The bigger problem is making reproducible builds for non-Linux distros. If you're a Windows user, you inherently trust Microsoft, but you may not trust a Python interpreter that you download from the Python website, and you definitely shouldn't trust a Python interpreter that's embedded in a PyInstaller-generated .exe file that a random software vendor (such as us) provides. Python is near-impossible to build for Windows reproducibly, while Go is trivially easy from looking at Tor's Gitian scripts.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

Post Reply