Adopting and running gombadi/dnsseeder from the repo at Github

Forum rules
Warning !
Avoid using binary softwares from untrusted users.
Prefer compiling it yourself and verify sources.
Post Reply
jonasbits
Posts: 47
Joined: Tue Mar 04, 2014 4:47 pm
os: linux

Adopting and running gombadi/dnsseeder from the repo at Github

Post by jonasbits »

This thread is mostly to document the steps needed to run "dnsseeder" for Mainnet and Testnet.

I will try to write about:
  • Building with Golang
  • Running dnsseeder with verbose flags
  • Testing with "dig" or "drill"
  • Monitor with "links"
  • Explain a little about DNS
  • How to create a systemd service file
  • Follow the packets with wireshark
My Namecoin address: NC3HGHk527xuWZBgMdGJ2GxjpRSw8D4oA6

jonasbits
Posts: 47
Joined: Tue Mar 04, 2014 4:47 pm
os: linux

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Post by jonasbits »

Reserved
My Namecoin address: NC3HGHk527xuWZBgMdGJ2GxjpRSw8D4oA6

jonasbits
Posts: 47
Joined: Tue Mar 04, 2014 4:47 pm
os: linux

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Post by jonasbits »

We need Go to build dnsseeder from source. Its pretty straight forward. Download from golang.org and check the hash with something like "shasum -a 256". Follow the short instruction for your OS and maybe export GOPATH to a path of your liking.

We will "go get" from the repo at https://github.com/gombadi/dnsseeder/ and the instructions there are quite good, the only addition I want to make is an alternative to using port 53 as non root, "setcap" with the flag "cap_net_bind_service=+ep" will give the binary permission to use port 53 (and infact any lower port).

I use tmux to run "dnsseeder" with all verbose flags active. That way its possible to attach and detach from the tmux session at will. Its possible to split the window and also run "links" to monitor the stats webgui. Testing of the DNS output can be performed with "dig" or "drill" like this:

for ipv4

Code: Select all

dig @localhost -p 5353 subdomain.example.org a
and for ipv6

Code: Select all

dig @localhost -p 5353 subdomain.example.org aaaa
For a production server we will need to have some knowledge of how DNS works and to run "dnsseeder" as a service. That will come in a later post. My main concern is to find the bottleneck (with wireshark) that is responsible for my crawlers not finding more nodes.

jonasbits
Posts: 47
Joined: Tue Mar 04, 2014 4:47 pm
os: linux

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Post by jonasbits »

The quest continues to find the real bottleneck. One strange thing is that using an initial node that is a legacy node with protocol version 38000 ( https://github.com/namecoin/namecoin-le ... lize.h#L38 ) gets me quickly to about 15 nodes found.

I use Wireshark to listen to the traffic from/to port 8334 (wire) and try not to mix it up with port 8336 (rpc). My advice is to save both rpc and wire ports as a "capture filter" like so:

Code: Select all

tcp port 8334
and for RPC:

Code: Select all

tcp port 8336
Then you can select port 8334 without mixing it up with the RPC port.

with the Display filter:

Code: Select all

bitcoin.addr
you can get a overview of how many nodes answer with a simple "addr" message back that only contains self.

When bootstraping dnsseeder with nodes from another dnsseeder (Seeder1,Seeder2,Seeder3) I quickly get 37 packets with "addr" messages and only three nodes report back more than 1 address.

To help display more relevant info, I have added more columes with:

Code: Select all

bitcoin.addr.count
bitcoin.string.value
bitcoin.version.version
Will post more later
My Namecoin address: NC3HGHk527xuWZBgMdGJ2GxjpRSw8D4oA6

jonasbits
Posts: 47
Joined: Tue Mar 04, 2014 4:47 pm
os: linux

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Post by jonasbits »

Next up is explaining NS records and how to check if anything already listens on port 53 on your server.

On debian you can use this command:

Code: Select all

sudo lsof -i
That will output a list of processes listening and will get you a clue what ports are already taken.
Read more about it at https://packages.debian.org/sid/lsof

Lets skim the surface of how DNS works. Its possible to use only one NS that will resolve every subdomain at every level, in this case foo.bar.example.org will be resolved by ns1.example.org.

The other possibility is divide the responsibility, usually by location, for example ns1 could be resolving everything behind *.npole.example.org and ns2 could take care of *.spole.example.org

Lets look at a NS Record for seed.example.org

Code: Select all

seed.example.org. IN NS ns4seed.example.org.
The key is to leave out the A and AAAA record for seed.example.org in the NS Record. Please notice the last dot at "org."
DNS querys goes from the right to the left, like so:

Code: Select all

seed.example.org.
________________^
dot, please get me the NS for org
org, please get me the NS for example
example, please get me the ip of seed
im sorry, dont have that, ask ns4seed
ns4seed, please get me ips of seed
To get a feel for this, check out "drill", its very simular to "dig" and have a trace flag:
https://manpages.debian.org/stretch/ldn ... .1.en.html
My Namecoin address: NC3HGHk527xuWZBgMdGJ2GxjpRSw8D4oA6

jonasbits
Posts: 47
Joined: Tue Mar 04, 2014 4:47 pm
os: linux

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Post by jonasbits »

x-posting this from chat
Image
this is a possible explanation for why we have so many "addr" wire messages only containing IP of self.

It would not be logical if bitcoind/namecoind reported back nodes behind NAT without port forwarding active (manual or UPNP)
My Namecoin address: NC3HGHk527xuWZBgMdGJ2GxjpRSw8D4oA6

jonasbits
Posts: 47
Joined: Tue Mar 04, 2014 4:47 pm
os: linux

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Post by jonasbits »

I have a seed running and reachable at seed.namecoin.libreisp.se and this moment there is 133 reachable nodes
My Namecoin address: NC3HGHk527xuWZBgMdGJ2GxjpRSw8D4oA6

jonasbits
Posts: 47
Joined: Tue Mar 04, 2014 4:47 pm
os: linux

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Post by jonasbits »

golang dnsseeder is running at seed.namecoin.libreisp.se and at this moment there is 117 reachable nodes of 419 total found.
My Namecoin address: NC3HGHk527xuWZBgMdGJ2GxjpRSw8D4oA6

jonasbits
Posts: 47
Joined: Tue Mar 04, 2014 4:47 pm
os: linux

Re: Adopting and running gombadi/dnsseeder from the repo at Github

Post by jonasbits »

This image is wrong, but illustrates a backward compatible issue
jonasbits wrote:
Sun Mar 07, 2021 9:33 pm
x-posting this from chat
Image
this is a possible explanation for why we have so many "addr" wire messages only containing IP of self.

It would not be logical if bitcoind/namecoind reported back nodes behind NAT without port forwarding active (manual or UPNP)
Jeremy found the weird behavior,

Code: Select all

                                // Bitcoin nodes typically return two Addr messages: one with
				// only one peer, and another with many peers.  This is
				// probably because ancient protocol versions (pver < 209) only
				// allowed one peer per Addr mesage, so returning a one-peer
				// Addr message first improves backward-compatibility.  Anyway,
				// this means we need to wait for the second Addr message.
This is now fixed in the latest version of golang dnsseeder
My Namecoin address: NC3HGHk527xuWZBgMdGJ2GxjpRSw8D4oA6

Post Reply