Scripts or library to crawl the namechain

Forum rules
Warning !
Avoid using binary softwares from untrusted users.
Prefer compiling it yourself and verify sources.
Post Reply
JZA
Posts: 18
Joined: Sat Mar 19, 2016 1:30 pm

Scripts or library to crawl the namechain

Post by JZA »

I am interesting in having a namechain scrapper, but I wonder if I would need a copy of the namechain on my local drive, or can I use a third party (public chain) and what would be needed. I am still not very well versed on blockchain technologies, so a blockchain scrapper might be the solution.

I want something like this:

Code: Select all

getnamechain mydomain.bit
>> found :: { name: 'my domain: welcome', ip: '192.168.0.32', ... }
Is there something like that already?

cassini
Posts: 336
Joined: Sun May 26, 2013 6:36 pm

Re: Scripts or library to crawl the namechain

Post by cassini »

Try https://namecoin.webbtc.com/
Use the Search field, or form the query URL yourself, e.g.
https://namecoin.webbtc.com/name/d/mydomain
displays the current value and the history of mydomain.bit (which is d/mydomain on the blockchain).

JZA
Posts: 18
Joined: Sat Mar 19, 2016 1:30 pm

Re: Scripts or library to crawl the namechain

Post by JZA »

I know, I know the explorers, the point was more about, do I have a library that can crawl third party explorers. Like a python or npm module, that I can just call on my terminal withou needing a local blockchain.
Keywords Scripts or libraries not webapps.

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Scripts or library to crawl the namechain

Post by biolizard89 »

First off, it's unclear to me what you mean by "the namechain". "The namechain" is not a term that I've ever heard used. Are you asking about the set of name transactions in the blockchain?

Second, assuming that you trust the 3rd party who's running the explorer, you might have tried clicking the "API" link in the link that Cassini provided. Assuming that you're competent enough to perform HTTP lookups and parse JSON with Python, that should presumably do what you're talking about.

Third, it's unclear to me what application you're trying to target, but if it's anything remotely serious, you shouldn't be trusting a blockchain explorer. In such cases, either use Namecoin Core, or (if SPV security is safe for your threat model) wait for the SPV client to get an official release.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

JZA
Posts: 18
Joined: Sat Mar 19, 2016 1:30 pm

Re: Scripts or library to crawl the namechain

Post by JZA »

namechain = namecoin's blockchain, (odd you couldnt make that up in your head).

I understand the API model, but this needs an api by the explorer, I was more looking for raw open nodes blockchain, similar to now ntp open nodes works, for example.

Just like ntp, I can create scripts like ntpdate, ntptrace, ntptime.

My idea were namecoin scripts that could act like SPV and had access to different blockchains and is able to do things like getrawtransaction or find name history and such.

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Scripts or library to crawl the namechain

Post by biolizard89 »

JZA wrote:namechain = namecoin's blockchain, (odd you couldnt make that up in your head).

I understand the API model, but this needs an api by the explorer, I was more looking for raw open nodes blockchain, similar to now ntp open nodes works, for example.

Just like ntp, I can create scripts like ntpdate, ntptrace, ntptime.

My idea were namecoin scripts that could act like SPV and had access to different blockchains and is able to do things like getrawtransaction or find name history and such.
Just for the record, I have no idea what you're asking for. The wire protocol doesn't support getrawtransaction or name_history, for the same reason that Bitcoin Core doesn't support that.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

johnc
Posts: 89
Joined: Sun Dec 28, 2014 10:03 am

Re: Scripts or library to crawl the namechain

Post by johnc »

JZA, no, but you can do nslookup mydomain.bit and if your computer is pointed to a dns server that supports namecoin you will be able to get the regular dns info.

I get your point, you want a command that returns all the json info, or even to be able to query like you are on the namecoin-core commandline.

It will be great thing to have one, but no, i think there is aren't, (unless nmcontrol works for you!? but that is a program not a library) and that will require that you pass a few parameters like IP of the machine and possibly username/password and i am not even sure this will work in the same way in different versions of blockexplorers (not supported by nmcontrol) or nodes.

So, someone could create such a library but it will be most likely hardcoded to query a specific website, because different websites have different api-s.

To query random nodes is out of the question because it will be totally insecure unless SPV verification were implemented on such a library, and that is not supported by bitcoin yet.

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Scripts or library to crawl the namechain

Post by biolizard89 »

johnc wrote:JZA, no, but you can do nslookup mydomain.bit and if your computer is pointed to a dns server that supports namecoin you will be able to get the regular dns info.

I get your point, you want a command that returns all the json info, or even to be able to query like you are on the namecoin-core commandline.

It will be great thing to have one, but no, i think there is aren't, (unless nmcontrol works for you!? but that is a program not a library) and that will require that you pass a few parameters like IP of the machine and possibly username/password and i am not even sure this will work in the same way in different versions of blockexplorers (not supported by nmcontrol) or nodes.

So, someone could create such a library but it will be most likely hardcoded to query a specific website, because different websites have different api-s.

To query random nodes is out of the question because it will be totally insecure unless SPV verification were implemented on such a library, and that is not supported by bitcoin yet.
What are you alleging that JZA wants? It's entirely unclear to me from his post what he's asking for; on IRC we came up with 4 different explanations of what he's asking for, none of which make much sense.

It's not at all clear to me what NMControl or nslookup has to do with this.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

JZA
Posts: 18
Joined: Sat Mar 19, 2016 1:30 pm

Re: Scripts or library to crawl the namechain

Post by JZA »

Taking the SPV electrum wallet for example, it uses a pool of servers to get all the data to operate the wallet including, transactions, accounts, etc. Electrum can be used as a daemon in order to perform operations on the commandline, and be able to retrieve data like transactions from the blockchain and even sign messgages with the bitcoin address.

Electrum even includes a directory with random scripts to perform miscelaneous tasks like transaction radar, follow addresses and such.

What I was looking is for a bootstrap version of the network library that hold the pool and connect to them and provides an interface to interact with the namecoin blockchain.

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Scripts or library to crawl the namechain

Post by biolizard89 »

JZA wrote:Taking the SPV electrum wallet for example, it uses a pool of servers to get all the data to operate the wallet including, transactions, accounts, etc. Electrum can be used as a daemon in order to perform operations on the commandline, and be able to retrieve data like transactions from the blockchain and even sign messgages with the bitcoin address.

Electrum even includes a directory with random scripts to perform miscelaneous tasks like transaction radar, follow addresses and such.

What I was looking is for a bootstrap version of the network library that hold the pool and connect to them and provides an interface to interact with the namecoin blockchain.
libdohj and a block explorer sound like roughly what you want. Note that libdohj is still experimental. I vaguely recall that someone also made a Namecoin port of PyBitcoinTools, but I'm not certain of that.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

Post Reply