sugarpuff wrote:I'll go through the list, with the answers for
DNSChain:
phelix wrote:This would allow for lightweight applications: Firefox browser plugin with TLS, file signatures, name browsing
* Should use Nmcontrol (integrated or query) or maybe a custom Python script
Uses JSON-RPC2 to talk to namecoind, no need for nmcontrol.
Agreed, while expanding on an existing library is a nice idea, this does not do that. We need to pipe data between the database in Namecoind and an HTTP server, arbitrarily forcing the use of nmcontrol is just adds an additional layer of abstraction to the name_show RPC.
sugarpuff wrote:
phelix wrote:
* Server should run on nginx, tornado, supervisord
I have nginx proxying requests to it, and DNSChain, like nginx, is an asynchronous, high-performance server written on top of NodeJS.
Specifying the use of any single technology only makes sense if that technology reduces the amount of work needed to perform the task. What we need to do is serve up the same exact requests for each query, there is no state to maintain or users to keep track of. We simply serve the same dumb response over and over again.
We need to preprocess the data and serve static copies. That is all.
High-performance python servers exist but we should just use the right tool for the right job. Since we are NOT handling complex queries there is NO need for complex processing on the server side.
sugarpuff wrote:
phelix wrote:
* Should implement some kind of query time / size limit and DDOS protection in Nmcontrol
Don't use NMControl, and this is easily done using nginx's rate limiting options.
Agreed, this is a read only service! nginx is a great reverse proxy.
sugarpuff wrote:
phelix wrote:* Should have an ICANN domain name or subdomain (not on namecoin.org)
dns.dnschain.net (as examples above show), but that's just my personal server I'm letting others use for free.
Wait, so no namecoin.info domain? Anyway, yeah, I've got endpoints on speech.is and at Cloudant.
sugarpuff wrote:
phelix wrote:
* All files need to be open source on github with a free license
Check.
Ditto, although I'm basically just using CouchDB. The JS for a scraper is online and FOSS. I'll probably just reimpliment the updating mechanism directly in Libcoin's SQLite and remove the scraper all-together.
sugarpuff wrote:
phelix wrote:
* Should come with an example query file in python
Not sure what you need this for... or why it needs to be in python? It's just HTTP (language agnostic).
I think he means just example python code. CouchDB has libraries for every major language, including Python. There are several embeddable databases (For Android, iOS, and JavaScript) and there is at least one implementation of the replication API in Python (meaning we can sync databases in Python if need be).
Config files should be in a standard key=value list, YAML, coffee, or JSON.
sugarpuff wrote:
phelix wrote:* a .bit domain with TLS support would be nice (should be very simple with the config files above)
DNSChain is a DNS server, and so you can load .bit domains over it. To make HTTPS warnings go away, something like a browser extension is required.
We should use Namecoin's .bit website.
sugarpuff wrote:
phelix wrote:* should be reasonably performant (cached name list)
It is. Caching can be done in many ways (PowerDNS, built-in memory, using memcached, etc.).
Here is a general refutation of any complaints you could have about CouchDB's performance characteristics:
If you want to try and build specific benchmarks to show CouchDB being slower than some other database/server combo, go for it. But when it comes to serving static JSON requests it's very, very hard to beat CouchDB. The hype around couchDB was misguided at first: it is good at one thing but it is not a great general-purpose DB. The map/reduce syntax is weird to work with. It doesn't do well on latency. It's not a Javascript object-database. That's why MongoDB "won" over CouchDB.
However, all of those complaints have little to do with CouchDB. We want a read-only datastore, if you are comparing map/reduce to SQL you should just use SQL because you're task probably doesn't need map/reduce preprocessing, nothing is faster than local reads, and it isn't meant to serve HTTP queries. Use a Javascript object DB if you want to store generic Javascript data objects which include functions.
It was made using Erlang so has great parallelism, the hot spots have been rewritten in C, and the performance comes from using an HTTP protocol that leverages the vast number of HTTP caches between the server and the client. Your local ISP doesn't cache binary data used by most databases, it does, however, cache all of the HTTP calls.
Finally, this is all based on full-node, master-to-master replication. The wireline latency doesn't matter because I can just shove the entire DB into the client. We can scale to the size of the .biz domain on only 5 megs of storage space, so just fucking do full replication and store everything on the client.
You can't beat CouchDB in this very specific niche without spending more time and effort than it would take to just use CouchDB.
sugarpuff wrote:
phelix wrote:* For much later: offer some security by adding the original block + a couple of following blockheaders
Not sure what you mean by this, but DNSChain will soon sign all its replies.
Right, I will add signing as well. Blockchain validation stuff will come with UXTO, so let's not do a crappy implementation of that here.
sugarpuff wrote:
phelix wrote:API (json):
* name_show (just like namecoind or nmcontrol)
* name_show at previous height, return failure if older than e.g. 100k blocks (nice to have)
Just load the URLs, it's a RESTful API (examples given already).
I've limited records to current, non-expired domains and included the block at which they expire. I will eventually get to creating a map/reduce job (AKA view) that filters expired domains. This is trivial, indeed, I will have a cleaned view which will serve as the "master database" clients copy from and I will add in all of the other restrictions we have (like valid JSON, etc).
sugarpuff wrote:
phelix wrote:Edit:
You will receive 0.3BTC now and 0.2BTC after one year.
Query names could be hashed and results encrypted with the name or something to protect the server operator from liabilities.
edited 2014-01-19
DNSChain supports dnscrypt! See the GitHub page (I'm using it now!).
As for encrypting HTTPS, that can also be added easily using NodeJS (and will be added soon). Currently HTTP is supported.
That is, sadly, not the type of encryption Phelix is referring to. I will integrate this when mainline does or sooner if it becomes an issue.
Okay, I should stop saying "CouchDB" and just start saying REST, because that's what makes CouchDB sp performant. Check out this
PhD thesis which explains why REST scales so well, it has some 3K citations. CouchDB is the only database that uses REST as it's communication protocol, that's what makes it scale so well. You can do the same thing by sharding databases and throwing reverse proxies in front of them and doing off-line map/reduce jobs to preprocess queries. CouchDB simply encapsulates a known architectural pattern into a single package.
I'm solving all of the problems one needs to care about in an API by reusing a system that is purpose built for that task. Additional API features are just map/reduce jobs which create a new view. As a bonus, we get to freeload off of the libraries, the embeddable implementations, replication, and validation (read code signing) with close to zero maintenance. That's the hallmark of good engineering.
Sugarpuff is totally correct when he says that it is not hard to implement the backend. He is a much more competent coder than I am and wrote a good one in 600 lines of Coffeescript. I wrote one up a fairly crappy PHP version which used Namecoind directly as well, then I wrote client side software and then I said, "Fuck this, I'm just going to reimplement everything using industrial grade components because it will be better than anything I can write."
As I said before, arguing about the backend is rather silly, all of this goes away once we have UXTO. I probably will toss CouchDB entirely and try to reuse BitTorrent or another P2P protocol in the future. UXTO does not replace the problems which Speech.is is trying to solve (web interoperability) nor what OKTurtles is trying to solve (authentication and encryption over existing services, basically transparent PGP for Facebook).
I really wish we had collaborated early on because I really need to work on the front-end, not the backend. I honestly would have just used Sug's backend had I understood what he was doing early on. However, retooling for either one will take time and I don't think either of us really want to redo that work again. I think mine can be beefed up more quickly and require less work in the long run, but I suggest that we both get to work on the front-end and start providing real value before we worry about existential engineering choices.
I would like the money to be used productively and I think covering hosting costs and setting up bounties for additional features are a good way to do that. I would still strongly suggest using my API setup, but I don't really care.
DNS is much more than a key->value datastore.