How much work?

Post Reply
somename
Posts: 80
Joined: Mon Sep 15, 2014 3:12 pm
os: windows

How much work?

Post by somename »

As discussed on this board, Zeronet has a Namecoin domain plugin.

1. Can someone estimate the amount of time required to adjust these for u/ ?
https://github.com/HelloZeroNet/ZeroNet ... tPlugin.py
and
https://github.com/HelloZeroNet/ZeroNet ... rPlugin.py

2. Do you think it's be good to add support for u/ as well?

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: How much work?

Post by biolizard89 »

somename wrote:As discussed on this board, Zeronet has a Namecoin domain plugin.

1. Can someone estimate the amount of time required to adjust these for u/ ?
https://github.com/HelloZeroNet/ZeroNet ... tPlugin.py
and
https://github.com/HelloZeroNet/ZeroNet ... rPlugin.py

2. Do you think it's be good to add support for u/ as well?
First off, u/ doesn't exist. It's something that a for-profit company (with an undisclosed business model) created to spam the Namecoin blockchain, and they then abandoned it. The right way to do identities is id/.

Second, DNSChain is abandoned code; the author is not actively working on any middleware at the moment but is backing ncdns.

So, it's not at all clear to me what you're asking for. Maybe you could clarify?
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

somename
Posts: 80
Joined: Mon Sep 15, 2014 3:12 pm
os: windows

Re: How much work?

Post by somename »

1) Right, id/. I had spent too much time reading about the other approach..
2) dnschain.info: didn't know that, the site looks alive and ZeroNet docs mention DNSchain as supported.

Okay, so what I'm interested in is what approach could be used to fetch d/ and id/ details the way DNSchain makes/made it possible (Web API).
I was hoping the existing DNSchain plugin for Zeronet could be modified to also query id/ details, but if that is not maintained, then something would have to be done from scratch (at least as far as Zeronet is concerned).

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: How much work?

Post by biolizard89 »

somename wrote:1) Right, id/. I had spent too much time reading about the other approach..
2) dnschain.info: didn't know that, the site looks alive and ZeroNet docs mention DNSchain as supported.

Okay, so what I'm interested in is what approach could be used to fetch d/ and id/ details the way DNSchain makes/made it possible (Web API).
I was hoping the existing DNSchain plugin for Zeronet could be modified to also query id/ details, but if that is not maintained, then something would have to be done from scratch (at least as far as Zeronet is concerned).
So, sounds like what you're looking for is basically a REST API implementation for retrieving name data.

A few questions about what you want:

Do you want the API to handle imports for you, or are you planning to be responsible for this yourself?
Do you expect untrusted users to use this API?
Do you expect users to trust the server, or do you want the server to be untrusted?
Do you actually need it to be REST-based? E.g. would an Electrum server that can serve name data meet your use case? (Electrum protocol would require less trust of the server, since it supplies a DMMS for each result.)
What kind of load are you expecting to be put on the API server?

I might have additional questions later, but the above should help get some discussion going. :)
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

somename
Posts: 80
Joined: Mon Sep 15, 2014 3:12 pm
os: windows

Re: How much work?

Post by somename »

biolizard89 wrote:
somename wrote:1) Right, id/. I had spent too much time reading about the other approach..
2) dnschain.info: didn't know that, the site looks alive and ZeroNet docs mention DNSchain as supported.

Okay, so what I'm interested in is what approach could be used to fetch d/ and id/ details the way DNSchain makes/made it possible (Web API).
I was hoping the existing DNSchain plugin for Zeronet could be modified to also query id/ details, but if that is not maintained, then something would have to be done from scratch (at least as far as Zeronet is concerned).
So, sounds like what you're looking for is basically a REST API implementation for retrieving name data.

A few questions about what you want:

Do you want the API to handle imports for you, or are you planning to be responsible for this yourself?
Do you expect untrusted users to use this API?
Do you expect users to trust the server, or do you want the server to be untrusted?
Do you actually need it to be REST-based? E.g. would an Electrum server that can serve name data meet your use case? (Electrum protocol would require less trust of the server, since it supplies a DMMS for each result.)
What kind of load are you expecting to be put on the API server?

I might have additional questions later, but the above should help get some discussion going. :)
Thanks.

Q1: Do you want the API to handle imports for you, or are you planning to be responsible for this yourself?

I'd have to plan more to answer this precisely, but I suppose a middle of the road answer could be: I'd be happy for the API or script to start with a list of id's, query the server for me, and store matches results in CSV file(s) (say, several fields from each matching ID, such as bitcoin, bitmessage, namecoin, otr).

Q2: Do you expect untrusted users to use this API?

No, for now it's assumed trusted user will run the API/batch script.

Q3: Do you expect users to trust the server, or do you want the server to be untrusted?

I'd pick the easier variant, assume the server serving NMC data is trusted, as this is merely for reference rather than for transactional use of Namecoin id/ namespace.

Q4: Do you actually need it to be REST-based? E.g. would an Electrum server that can serve name data meet your use case? (Electrum protocol would require less trust of the server, since it supplies a DMMS for each result.)

It can be anything based, even the NMC CLI (Bash, Python, Ruby, or JS), as long as it doesn't take a lot of resources (but if it runs once a day, then it can take up to 500 MB of RAM when it does run).
Note: yesterday I still didn't know the exact size of the NameCoin data, but in the meantime I installed and caught up and I see I'd need less than 5 GB for the client, so queries could be made to the local NMC server which I can run 1 hour per day, just to let it catch up before I run this script/program.
Something like: fetch ID's from the user DB table, query those IDs on the NMC blockchain, INSERT fields like PGP & Bitcoin to matching ID records.

Q5: What kind of load are you expecting to be put on the API server?

A small community. Imagine if you wanted this forum's members profiles to be populated based on "id/" they entered.
The sole purpose is that users don't have to populate and maintain their personal data (say, you change your bitmessage address in NMC, it gets updated on the forum).

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: How much work?

Post by biolizard89 »

somename wrote:Thanks.

Q1: Do you want the API to handle imports for you, or are you planning to be responsible for this yourself?

I'd have to plan more to answer this precisely, but I suppose a middle of the road answer could be: I'd be happy for the API or script to start with a list of id's, query the server for me, and store matches results in CSV file(s) (say, several fields from each matching ID, such as bitcoin, bitmessage, namecoin, otr).

Q2: Do you expect untrusted users to use this API?

No, for now it's assumed trusted user will run the API/batch script.

Q3: Do you expect users to trust the server, or do you want the server to be untrusted?

I'd pick the easier variant, assume the server serving NMC data is trusted, as this is merely for reference rather than for transactional use of Namecoin id/ namespace.

Q4: Do you actually need it to be REST-based? E.g. would an Electrum server that can serve name data meet your use case? (Electrum protocol would require less trust of the server, since it supplies a DMMS for each result.)

It can be anything based, even the NMC CLI (Bash, Python, Ruby, or JS), as long as it doesn't take a lot of resources (but if it runs once a day, then it can take up to 500 MB of RAM when it does run).
Note: yesterday I still didn't know the exact size of the NameCoin data, but in the meantime I installed and caught up and I see I'd need less than 5 GB for the client, so queries could be made to the local NMC server which I can run 1 hour per day, just to let it catch up before I run this script/program.
Something like: fetch ID's from the user DB table, query those IDs on the NMC blockchain, INSERT fields like PGP & Bitcoin to matching ID records.

Q5: What kind of load are you expecting to be put on the API server?

A small community. Imagine if you wanted this forum's members profiles to be populated based on "id/" they entered.
The sole purpose is that users don't have to populate and maintain their personal data (say, you change your bitmessage address in NMC, it gets updated on the forum).
Okay, so anyone reading this is welcome to suggest architectures that might address your needs, but here are some initial thoughts:

Namecoin Core's REST API might meet your needs. I did a quick Google search for Bitcoin Core's default RAM usage, and I'm getting figures between 200 MB and 1 GB. (I'm not sure why the discrepency.) Looks like recent Bitcoin Core clients allow you to restrict the mempool size. Bitcoin Core also allows restricting the database cache size. Both of these are likely to reduce the above figures. Namecoin has a smaller blockchain than Bitcoin, but has a name database that Bitcoin doesn't have, so I'm not sure how exactly Namecoin will end up differing from Bitcoin in RAM usage.

The Namecoin Core REST API does need Namecoin Core to be running 24/7 though. Can you comment on exactly what your requirements are regarding the subject of something running 24/7?

Another option would be to use libcoind. It has the advantage of storing its name database using standard sqlite, so you could run libcoind for a short period of time per day, and then just use whatever sqlite tools you like in order to check the status of names. It should be pretty easy to customize the cache size of libcoind (I've done it before, it's changing one line of code and a recompile). The downside here is that libcoind hasn't gotten as much peer review as Namecoin Core, and since its database differs from Namecoin Core, there is always the chance of a consensus difference between libcoind and Namecoin Core. The degree to which this matters depends on your security needs. libcoind is by Michael Gronager, who's well-respected in terms of writing secure code. (He reported the CoinSlayer bug in Namecoin, which he found while adding Namecoin support to libcoin.) Last I heard, libcoin is used by Kraken for a subset of their operations, so they seem to consider it trustworthy at least for some use cases.

You could also maybe use something based on BitcoinJ or Electrum in order to do SPV verification of name lookups. This is likely to have much lower RAM and storage requirements than Namecoin Core or libcoind, but you would need to have BitcoinJ or Electrum running 24/7 (unlike libcoind). If 100% of the network nodes you connect to are malicious, they could hide updates to names, or make it appear that the names you look up don't exist. The names that you look up would also be revealed to the network nodes you connect to, although in theory you could use Tor (perhaps with stream isolation) to reduce this risk. BitcoinJ and Electrum have both received substantially more peer review than libcoind.

In terms of public benefit to the Namecoin community, I'd say that Namecoin name support in BitcoinJ or Electrum would probably be of substantially higher benefit than something based on libcoind.

Anyway, these are my thoughts. Feel free to suggest modifications, or other ideas, or ask questions about the above, etc. Everyone else reading is encouraged to do the same.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

somename
Posts: 80
Joined: Mon Sep 15, 2014 3:12 pm
os: windows

Re: How much work?

Post by somename »

Thanks for these great suggestions.

I'm not opposed to running something 24x7, it's just that hosted sites have plenty of space for the entire Namecoin blockchain (if need be), but RAM is usually scarce.
At the same time this info doesn't have to be completely up to date (it's not critical to the functioning of the site), so if for example I could use a 1GB instance and in off-hours fire up Namecoin, let it sync up and then execute my batch job, I wouldn't need a 2GB instance (I'd rather save the difference and donate part of it for a bounty here).
I figure it's reasonable to put somewhere a note and say "we'll populate these 4 fields with NMC data based on your user ID, usually within 24 hours of registration or update".
(A funny scenario would be someone registering a site ID of a real NMC id who may also want to join the same site. Solution: send email to support@site.com).

The libcoind idea is very much along those lines.
But last night I started namecoind and by today all 3.8 GB of data were up to date, so I think daily incremental updates can complete within 5-10 mins (and in fact they do, I have a Windows version installed and when update it it's very quick). That's why I am thinking something proven and familiar to more devs, like namecoind, is probably better.

> In terms of public benefit to the Namecoin community, I'd say that Namecoin name support in BitcoinJ or Electrum would probably be of substantially higher benefit than something based on libcoind.

That's a good idea. I asked about SPV on the other thread because I thought that would be the best and most secure long term approach, but I saw that it's not ready, but it will be ready soon-ish.

For my own purpose, it would be enough to have a Python or similar script that would get username details, query the id/ namespace and produce a CSV file (which I would then pass to mySQL or other client to update user data).

But if someone could produce that as an additional requirement for NMC BitcoinJ support bounty (would that be a bad idea?), for example, I'd gladly donate to that bounty and get my script once BitcoinJ support is delivered. (I don't mind Electrum, but it sounds like it requires comparatively more resources and management).

One reason why I'd like to have the script done is it doesn't have to be made specifically for my table or DB, but generic. That way anyone who likes this approach could leverage it for various forums and other sites. There are many sites that have a SQL DB (almost all do, actually) but they can't run (or doesn't know how to use) this or that language, but nearly everyone can execute a SQL insert query against their database (be it Sqlite or mySQL or whatever) or even flat files that some static Web sites use.

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: How much work?

Post by biolizard89 »

If you're okay with retrieving name JSON data from a BitcoinJ client each time you need to do a lookup (e.g. over an RPC or REST API served by the BitcoinJ client), that's something that would probably be useful to a lot of people.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

Post Reply