Adopting and running gombadi/dnsseeder from the repo at Github

[jonasbits]

This thread is mostly to document the steps needed to run "dnsseeder" for Mainnet and Testnet.

I will try to write about:

• Building with Golang
• Running dnsseeder with verbose flags
• Testing with "dig" or "drill"
• Monitor with "links"
• Explain a little about DNS
• How to create a systemd service file
• Follow the packets with wireshark

[jonasbits]

Reserved

[jonasbits]

We need Go to build dnsseeder from source. Its pretty straight forward. Download from golang.org and check the hash with something like "shasum -a 256". Follow the short instruction for your OS and maybe export GOPATH to a path of your liking.

We will "go get" from the repo at https://github.com/gombadi/dnsseeder/ and the instructions there are quite good, the only addition I want to make is an alternative to using port 53 as non root, "setcap" with the flag "cap_net_bind_service=+ep" will give the binary permission to use port 53 (and infact any lower port).

I use tmux to run "dnsseeder" with all verbose flags active. That way its possible to attach and detach from the tmux session at will. Its possible to split the window and also run "links" to monitor the stats webgui. Testing of the DNS output can be performed with "dig" or "drill" like this:

for ipv4

Code: Select all

dig @localhost -p 5353 subdomain.example.org a

and for ipv6

Code: Select all

dig @localhost -p 5353 subdomain.example.org aaaa

For a production server we will need to have some knowledge of how DNS works and to run "dnsseeder" as a service. That will come in a later post. My main concern is to find the bottleneck (with wireshark) that is responsible for my crawlers not finding more nodes.

[jonasbits]

The quest continues to find the real bottleneck. One strange thing is that using an initial node that is a legacy node with protocol version 38000 ( https://github.com/namecoin/namecoin-le ... lize.h#L38 ) gets me quickly to about 15 nodes found.

I use Wireshark to listen to the traffic from/to port 8334 (wire) and try not to mix it up with port 8336 (rpc). My advice is to save both rpc and wire ports as a "capture filter" like so:

Code: Select all

tcp port 8334

and for RPC:

Code: Select all

tcp port 8336

Then you can select port 8334 without mixing it up with the RPC port.

with the Display filter:

Code: Select all

bitcoin.addr

you can get a overview of how many nodes answer with a simple "addr" message back that only contains self.

When bootstraping dnsseeder with nodes from another dnsseeder (Seeder1,Seeder2,Seeder3) I quickly get 37 packets with "addr" messages and only three nodes report back more than 1 address.

To help display more relevant info, I have added more columes with:

Code: Select all

bitcoin.addr.count
bitcoin.string.value
bitcoin.version.version

Will post more later

[jonasbits]

Next up is explaining NS records and how to check if anything already listens on port 53 on your server.

On debian you can use this command:

Code: Select all

sudo lsof -i

That will output a list of processes listening and will get you a clue what ports are already taken.
Read more about it at https://packages.debian.org/sid/lsof

Lets skim the surface of how DNS works. Its possible to use only one NS that will resolve every subdomain at every level, in this case foo.bar.example.org will be resolved by ns1.example.org.

The other possibility is divide the responsibility, usually by location, for example ns1 could be resolving everything behind *.npole.example.org and ns2 could take care of *.spole.example.org

Lets look at a NS Record for seed.example.org

Code: Select all

seed.example.org. IN NS ns4seed.example.org.

The key is to leave out the A and AAAA record for seed.example.org in the NS Record. Please notice the last dot at "org."
DNS querys goes from the right to the left, like so:

Code: Select all

seed.example.org.
________________^
dot, please get me the NS for org
org, please get me the NS for example
example, please get me the ip of seed
im sorry, dont have that, ask ns4seed
ns4seed, please get me ips of seed

To get a feel for this, check out "drill", its very simular to "dig" and have a trace flag:
https://manpages.debian.org/stretch/ldn ... .1.en.html

[jonasbits]

x-posting this from chat

this is a possible explanation for why we have so many "addr" wire messages only containing IP of self.

It would not be logical if bitcoind/namecoind reported back nodes behind NAT without port forwarding active (manual or UPNP)

[jonasbits]

I have a seed running and reachable at seed.namecoin.libreisp.se and this moment there is 133 reachable nodes

[jonasbits]

golang dnsseeder is running at seed.namecoin.libreisp.se and at this moment there is 117 reachable nodes of 419 total found.

[jonasbits]

This image is wrong, but illustrates a backward compatible issue

x-posting this from chat

this is a possible explanation for why we have so many "addr" wire messages only containing IP of self.

It would not be logical if bitcoind/namecoind reported back nodes behind NAT without port forwarding active (manual or UPNP)

Jeremy found the weird behavior,

Code: Select all

                                // Bitcoin nodes typically return two Addr messages: one with
				// only one peer, and another with many peers.  This is
				// probably because ancient protocol versions (pver < 209) only
				// allowed one peer per Addr mesage, so returning a one-peer
				// Addr message first improves backward-compatibility.  Anyway,
				// this means we need to wait for the second Addr message.

This is now fixed in the latest version of golang dnsseeder