Page 1 of 1

Adopting and running gombadi/dnsseeder from the repo at Github

Posted: Sat Feb 27, 2021 9:18 pm
by jonasbits
This thread is mostly to document the steps needed to run "dnsseeder" for Mainnet and Testnet.

I will try to write about:
  • Building with Golang
  • Running dnsseeder with verbose flags
  • Testing with "dig" or "drill"
  • Monitor with "links"
  • Explain a little about DNS
  • How to create a systemd service file
  • Follow the packets with wireshark

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Posted: Sat Feb 27, 2021 9:47 pm
by jonasbits
Reserved

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Posted: Sat Feb 27, 2021 11:05 pm
by jonasbits
We need Go to build dnsseeder from source. Its pretty straight forward. Download from golang.org and check the hash with something like "shasum -a 256". Follow the short instruction for your OS and maybe export GOPATH to a path of your liking.

We will "go get" from the repo at https://github.com/gombadi/dnsseeder/ and the instructions there are quite good, the only addition I want to make is an alternative to using port 53 as non root, "setcap" with the flag "cap_net_bind_service=+ep" will give the binary permission to use port 53 (and infact any lower port).

I use tmux to run "dnsseeder" with all verbose flags active. That way its possible to attach and detach from the tmux session at will. Its possible to split the window and also run "links" to monitor the stats webgui. Testing of the DNS output can be performed with "dig" or "drill" like this:

for ipv4

Code: Select all

dig @localhost -p 5353 subdomain.example.org a
and for ipv6

Code: Select all

dig @localhost -p 5353 subdomain.example.org aaaa
For a production server we will need to have some knowledge of how DNS works and to run "dnsseeder" as a service. That will come in a later post. My main concern is to find the bottleneck (with wireshark) that is responsible for my crawlers not finding more nodes.

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Posted: Fri Mar 05, 2021 7:39 pm
by jonasbits
The quest continues to find the real bottleneck. One strange thing is that using an initial node that is a legacy node with protocol version 38000 ( https://github.com/namecoin/namecoin-le ... lize.h#L38 ) gets me quickly to about 15 nodes found.

I use Wireshark to listen to the traffic from/to port 8334 (wire) and try not to mix it up with port 8336 (rpc). My advice is to save both rpc and wire ports as a "capture filter" like so:

Code: Select all

tcp port 8334
and for RPC:

Code: Select all

tcp port 8336
Then you can select port 8334 without mixing it up with the RPC port.

with the Display filter:

Code: Select all

bitcoin.addr
you can get a overview of how many nodes answer with a simple "addr" message back that only contains self.

When bootstraping dnsseeder with nodes from another dnsseeder (Seeder1,Seeder2,Seeder3) I quickly get 37 packets with "addr" messages and only three nodes report back more than 1 address.

To help display more relevant info, I have added more columes with:

Code: Select all

bitcoin.addr.count
bitcoin.string.value
bitcoin.version.version
Will post more later

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Posted: Sun Mar 07, 2021 1:09 am
by jonasbits
Next up is explaining NS records and how to check if anything already listens on port 53 on your server.

On debian you can use this command:

Code: Select all

sudo lsof -i
That will output a list of processes listening and will get you a clue what ports are already taken.
Read more about it at https://packages.debian.org/sid/lsof

Lets skim the surface of how DNS works. Its possible to use only one NS that will resolve every subdomain at every level, in this case foo.bar.example.org will be resolved by ns1.example.org.

The other possibility is divide the responsibility, usually by location, for example ns1 could be resolving everything behind *.npole.example.org and ns2 could take care of *.spole.example.org

Lets look at a NS Record for seed.example.org

Code: Select all

seed.example.org. IN NS ns4seed.example.org.
The key is to leave out the A and AAAA record for seed.example.org in the NS Record. Please notice the last dot at "org."
DNS querys goes from the right to the left, like so:

Code: Select all

seed.example.org.
________________^
dot, please get me the NS for org
org, please get me the NS for example
example, please get me the ip of seed
im sorry, dont have that, ask ns4seed
ns4seed, please get me ips of seed
To get a feel for this, check out "drill", its very simular to "dig" and have a trace flag:
https://manpages.debian.org/stretch/ldn ... .1.en.html

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Posted: Sun Mar 07, 2021 9:33 pm
by jonasbits
x-posting this from chat
Image
this is a possible explanation for why we have so many "addr" wire messages only containing IP of self.

It would not be logical if bitcoind/namecoind reported back nodes behind NAT without port forwarding active (manual or UPNP)