Let's build some merged mining pools!

Post Reply
nodemaster
Posts: 172
Joined: Wed Jun 15, 2011 12:46 pm
os: linux

Let's build some merged mining pools!

Post by nodemaster »

This thread is for all people who are willing to start a merged mining pool and need input on this issue or want to ask some question:

We started here: http://dot-bit.org/forum/viewtopic.php?p=1887#p1887 but felt, that this thread would be more appropriate in the technical support subforum.
Access .bit domains with Firefox in 4 easy steps: https://masterpool.eu/proxy
MasterPool Namecoin Mining Pool

twobits
Posts: 26
Joined: Fri Sep 02, 2011 10:10 pm
os: bsd

Re: Let's build some merged mining pools!

Post by twobits »

nodemaster wrote:This thread is for all people who are willing to start a merged mining pool and need input on this issue or want to ask some question:

We started here: http://dot-bit.org/forum/viewtopic.php?p=1887#p1887 but felt, that this thread would be more appropriate in the technical support subforum.

Hmmm.... I was hoping to get support from someone who already had a merged mining pool for an idea.

In theory though isn't it just a matter of running the daemons, running the proxy, and pointing pushpool at the proxy? Then you have to make the front end, and that takes more time then we have I would guess.

nodemaster
Posts: 172
Joined: Wed Jun 15, 2011 12:46 pm
os: linux

Re: Let's build some merged mining pools!

Post by nodemaster »

Davincij15 wrote: I can do it however I need to fix the pool one problem is you have half the hash rate and get WAY more blocks than I do. I am not using v 3.24.60 yet so that may be the problem.
Well, never really checked that. But it seems you prepared yourself good for your "May the best pool do very well."-Quest :lol: SCNR

First of all: I'm a systems architect, not a programmer. Thus it might be, we are talking past each other some time thus please tell me , if we are drifting apart too much. Okay lets see. From what I have read you have a fundamentally different setup than me. MasterPool was build with lowest possible latency in mind. I made this decision because lower latency means you can inform the network earlier about found blocks and you lower the stales, which means your users have lower blend. However latency is not only important on network layer but also on server hardware layer.

From what I have read your setup look like follows:

You have several Amazon micro instances. All running a namecoind, blkmond and a pushpoold. I'm not sure about your database. Would be interesting if you have one on each node or one central. You are using gbyte.dk's pool software and thus I guess you are using MySQL as database. Not sure abuout your webfrontends, I assume you are running it on separate servers otherwise the micro instances would go nuts.

Some suggestons to lower your latency significantly:

Don't use DNS round robin. You already recognized its a big problems as miners are hopping from node to node loosing their session. Don't use an additional loadbalancer. I tried a level three loadbalancer at first too, but it adds unnessecary overhead and is a single point of failure.

Namecoin is a bitch regarding I/O on harddisk. From what I have tested Bitcoin is even worse (more transactions). You need a hell lot of RAM and CPU cores. You need RAM in order to cache harddisk stuff. More CPU cores ensure that other tasks (that don't need I/O) aren't blocked during HDD transfers. If you want your server to stop lagging consider a SSD for the folder where you store your blockchains or put them into RAM disk using an overlay. This is not possible with Amazon virtualized mashines.

Make sure you have a good and direct connection to the P2P networks. Don't rely on NAT if possible. This makes sure you get informed about new blocks as fast as possible. If one of your miners have found a possible solution this is you ticket to let the rest of the network know ASAP.

Consider offloading pushpoold (and merged mine proxy) from the server running MySQL and the blockchain daemons. In general don't put all daemons on all server, but connect them smart using IPSEC and let them share ressources where possible.

Have you solved your problem about namecoind dying again and again? If you monitor your server have a look on the open connections. Have you raised the values for max open files for the user running the daemons?
Last edited by nodemaster on Mon Sep 12, 2011 6:30 pm, edited 1 time in total.
Access .bit domains with Firefox in 4 easy steps: https://masterpool.eu/proxy
MasterPool Namecoin Mining Pool

nodemaster
Posts: 172
Joined: Wed Jun 15, 2011 12:46 pm
os: linux

Re: Let's build some merged mining pools!

Post by nodemaster »

twobits wrote: In theory though isn't it just a matter of running the daemons, running the proxy, and pointing pushpool at the proxy? Then you have to make the front end, and that takes more time then we have I would guess.
In theory the most simplest setup looks like:

Code: Select all

pushpoold -> merged-mine-proxy ---> namecoind
    |                           \-> bitcoind
    v
database <- webfrontend

           
This is without long polling (you need to monitor the blockchains for new blocks and send SIGUSR1 to pushppold) and of course you need some logic that pays out the coins. either via webinterface accessing the blockchain daemons directly or done as a backend task.

In theory with pushpoold, merged-mine-proxy and simplecoin webfrontend most nessecary software is available as open source. The only thing that is missing is like you said the second blockchain. however its logic is the same as for the other blockchain.
Access .bit domains with Firefox in 4 easy steps: https://masterpool.eu/proxy
MasterPool Namecoin Mining Pool

doublec
Posts: 149
Joined: Mon May 23, 2011 12:47 am
os: linux
Location: Auckland, New Zealand
Contact:

Re: Let's build some merged mining pools!

Post by doublec »

You can just run two blkmond instances for the long polling, each monitoring the different chains.

shads
Posts: 26
Joined: Sun Jul 31, 2011 6:36 am
os: linux

Re: Let's build some merged mining pools!

Post by shads »

When the mm proxy is preparing the aux root it has to call buildmerkletree on the parent daemon. The doco for this is very sparse and it isn't clear of this should include the hash from the parent chain or not?

if I try it without... e.g.
namecoind-mm getauxblock
{
"target" : "00000000000000000000000000000000000000000000000069b2000000000000",
"hash" : "fbdb162a594676dae28ff08ea28dacffb576b13013c432768fc6a45a4b3a9d47",
"chainid" : 1
}

bitcoind-mm buildmerkletree fbdb162a594676dae28ff08ea28dacffb576b13013c432768fc6a45a4b3a9d47
[
"fbdb162a594676dae28ff08ea28dacffb576b13013c432768fc6a45a4b3a9d47"
]

It doesn't even appear to hash the input.

If I add the bitcoind hash to the request:
bitcoind-mm getauxblock{
"target" : "0000000000000000000000000000000000000000000000a58e09000000000000",
"hash" : "e7d51a780adecb8a5727f295eda3bf8fad82ba83b28397dac61aaffdcb1f4e19",
"chainid" : 0
}

I get something that makes a little more sense (I know haven't calculated the order of inputs properly)
bitcoind-mm buildmerkletree fbdb162a594676dae28ff08ea28dacffb576b13013c432768fc6a45a4b3a9d47 e7d51a780adecb8a5727f295eda3bf8fad82ba83b28397dac61aaffdcb1f4e19
[
"fbdb162a594676dae28ff08ea28dacffb576b13013c432768fc6a45a4b3a9d47",
"e7d51a780adecb8a5727f295eda3bf8fad82ba83b28397dac61aaffdcb1f4e19",
"b8dab153781b329b7fd512b90be2f3b942122a6499a3e570db8e16ffcb086fdb"
]

so we've got, namecoind-hash, bitcoind-hash, merkleroot.

so if we have to make a round trip to bitcoind to get the hash to send to buildmerkletree I don't see the point of another round trip to bitcoind to get the merkleroot. Is there any reason it can't be calculated locally? If you didn't have to include the parent hash in the buildmerkletree call I could understand since you'd still have to make one rpc call anyway...

shads
Posts: 26
Joined: Sun Jul 31, 2011 6:36 am
os: linux

Re: Let's build some merged mining pools!

Post by shads »

I've taken the previous question and a few others to this thread: http://dot-bit.org/forum/viewtopic.php?f=7&t=316

I think perhaps the guts of mm proxy is probably not the focus of this thread...

Post Reply