[ANN] libcoin rebase attempt

Namecoin, NMControl
domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

[ANN] libcoin rebase attempt

Post by domob »

I'm attempting to start a re-implementation of namecoind based on libcoin. I've taken a slight look at libcoin's sources, and it seems that it should be trivial to get at least a very simple wallet running. This is even an example in their code, see https://github.com/libcoin/libcoin/blob ... wallet.cpp. I will try to do that as a first step, and then continue to evaluate what else needs to be added in order to build a full replacement for namecoind. My code will be at https://github.com/domob1812/nmc-libcoin.

Question: Is a fully compatible replacement for namecoind what we want, or should we use the opportunity to discuss possible changes to the Namecoin-specific RPC interface? Possibly also to facilitate future implementation of a UI on top of the RPC interface.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: [ANN] libcoin rebase attempt

Post by biolizard89 »

domob wrote:I'm attempting to start a re-implementation of namecoind based on libcoin. I've taken a slight look at libcoin's sources, and it seems that it should be trivial to get at least a very simple wallet running. This is even an example in their code, see https://github.com/libcoin/libcoin/blob ... wallet.cpp. I will try to do that as a first step, and then continue to evaluate what else needs to be added in order to build a full replacement for namecoind. My code will be at https://github.com/domob1812/nmc-libcoin.

Question: Is a fully compatible replacement for namecoind what we want, or should we use the opportunity to discuss possible changes to the Namecoin-specific RPC interface? Possibly also to facilitate future implementation of a UI on top of the RPC interface.
Awesome, thanks for working on this.

I'm pretty sure Armory doesn't actually use the RPC interface, so if we're planning on porting Armory, RPC changes shouldn't be needed for that. I can't think of any specific RPC changes I'd like... anyone else have suggestions?
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: [ANN] libcoin rebase attempt

Post by domob »

Quick update so far: There seems to be also an "actual" main command-line tool (called libcoind) included, which is somewhat similar to the example file I linked in the OP but includes connecting to an RPC server or setting it up optionally, just like namecoind does (not sure what the example really does in this direction). Apart from that, it looks somewhat similar to the example.

I've tried it out, and it seems to be fine running as Bitcoin client (at least it happily connects to the network and downloads the blockchain - I've not yet tried other things). The Namecoin client doesn't work, though. It doesn't get any connections, even when I point it explicitly to my node. Not sure if there's a stupid mistake somewhere (my node having reached maxconnections and thus not responding or something) or if libcoin isn't (yet) able to run a full node in Namecoin. Since it reports 70300 as protocol version for getinfo, my wild guess is that this may be why it can't find nodes to connect to. From a quick glance, the PROTOCOL_VERSION used in the code is hardcoded "somewhat" and looks a bit hackish in general. It would be interesting to know how Kraken runs their Namecoin wallet based on libcoin.

I'll experiment a bit more, and maybe I can find out what the problem is. It seems to me that we should be able to get a straight-forward RPC client out of libcoin at least for all the wallet stuff if things work out, and from the comments of Michael I've read, it seems that also a nameindex is already there and could be used easily. This would make the rebase almost trivial, at least in theory.... Let's see how this works out. :)
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

khal
Site Admin
Posts: 708
Joined: Mon May 09, 2011 5:09 pm
os: linux

Re: [ANN] libcoin rebase attempt

Post by khal »

domob wrote:Quick update so far: There seems to be also an "actual" main command-line tool (called libcoind) included, which is somewhat similar to the example file I linked in the OP but includes connecting to an RPC server or setting it up optionally, just like namecoind does (not sure what the example really does in this direction). Apart from that, it looks somewhat similar to the example.
domob wrote:I've tried it out, and it seems to be fine running as Bitcoin client (at least it happily connects to the network and downloads the blockchain - I've not yet tried other things). The Namecoin client doesn't work, though. It doesn't get any connections, even when I point it explicitly to my node. Not sure if there's a stupid mistake somewhere (my node having reached maxconnections and thus not responding or something) or if libcoin isn't (yet) able to run a full node in Namecoin. Since it reports 70300 as protocol version for getinfo, my wild guess is that this may be why it can't find nodes to connect to. From a quick glance, the PROTOCOL_VERSION used in the code is hardcoded "somewhat" and looks a bit hackish in general. It would be interesting to know how Kraken runs their Namecoin wallet based on libcoin.
Do you have something in the logs ?
At a first thought, I would say that pchMessage is wrong and you are rejected from the network. But I remember checking it and it was ok.
Anyway, getting some info from an active user (kraken) is a good idea.

domob wrote:I'll experiment a bit more, and maybe I can find out what the problem is. It seems to me that we should be able to get a straight-forward RPC client out of libcoin at least for all the wallet stuff if things work out, and from the comments of Michael I've read, it seems that also a nameindex is already there and could be used easily. This would make the rebase almost trivial, at least in theory.... Let's see how this works out. :)
I'll also do some tests on my side. Good new :p
Is there specific params to build namecoind ?
NamecoinID: id/khal
GPG : 9CC5B92E965D69A9
NMC: N1KHAL5C1CRzy58NdJwp1tbLze3XrkFxx9
BTC: 1KHAL8bUjnkMRMg9yd2dNrYnJgZGH8Nj6T

Register Namecoin domains with BTC
My bitcoin Identity - Send messages to bitcoin users
Charity Ad - Make a good deed without paying a cent

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: [ANN] libcoin rebase attempt

Post by domob »

khal wrote:
domob wrote:I've tried it out, and it seems to be fine running as Bitcoin client (at least it happily connects to the network and downloads the blockchain - I've not yet tried other things). The Namecoin client doesn't work, though. It doesn't get any connections, even when I point it explicitly to my node. Not sure if there's a stupid mistake somewhere (my node having reached maxconnections and thus not responding or something) or if libcoin isn't (yet) able to run a full node in Namecoin. Since it reports 70300 as protocol version for getinfo, my wild guess is that this may be why it can't find nodes to connect to. From a quick glance, the PROTOCOL_VERSION used in the code is hardcoded "somewhat" and looks a bit hackish in general. It would be interesting to know how Kraken runs their Namecoin wallet based on libcoin.
Do you have something in the logs ?
At a first thought, I would say that pchMessage is wrong and you are rejected from the network. But I remember checking it and it was ok.
Anyway, getting some info from an active user (kraken) is a good idea.
Do you mean the logs of libcoind or the logs of my vanilla client I tried to addnode? The libcoind logs show nothing really interesting, IIRC, but I haven't yet looked at the logs of the client I try to connect to. In case it rejects an incoming connection because of a wrong handshake, would I get a debug.log message for this event? I'll try that tomorrow.
khal wrote:I'll also do some tests on my side. Good new :p
Is there specific params to build namecoind ?
Not that I'm aware of. You can just build it (except that one of the examples didn't build for me and I had to comment it out in the makefiles), and then use the generated libcoind binary. It can (presumably) act as all kinds of different clients depending on a command-line flag you give it to select the chain you want to use. For our rebased namecoind client, we can of course also "hardcode" the Namecoin blockchain from within the code.

BTW, there's a whole bunch of things in the "applications" folder. I really have no idea what each of those are supposed to do (it seems there are not really very many meaningful comments, at least not on a high level), so it may be something stupid I'm missing. But at least the code of libcoind looks like it is supposed to be a general client application suitable for different chains.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: [ANN] libcoin rebase attempt

Post by domob »

Update: It seems to work now when I explicitly addnode my server (I removed its maxconnections restriction). I can download the blockchain just fine but stay at one connection. So possibly there are some problems with peer discovery. Also, it doesn't work with "--verify FULL --validate FULL" so far (I get a segfault), and I presume that we want these options for a full node (although I have no idea what the details are behind other settings here).

EDIT: Now it seems to work just fine without addnode and I get other connections as well. Maybe because already discovered peers (from my addnode attempt) are cached and now tried right away? At least it seems as if I got a running node now. Also, it creates a SQLite3 database of the blockchain that also includes what looks like a name_history table. I feel fairly confident that it would be a simple matter to implement name_list, name_history and name_show based on that if it is not yet available inside libcoin already.

Maybe I'll try to set up a wallet and transfer coins to it and back later to check out whether that works fully or not. getwork also yields some results (but I have no idea whether they are as they should be), and according to a statement from Michael, getauxwork would not be too hard to add.

I'm not sure what the "rebase" then actually would be ... just extending libcoin itself so that libcoind can be used as a full namecoin replacement (or such that only a trivial main program can replace namecoind but everything happens inside of libcoin already), or using libcoin as-is and adding the missing features (name querying?) in a (still presumably relatively simple and small) main program? Anything else I'm missing at the moment?

When I've done a few more tests, I'll try to contact Michael about specific things to get his opinion and decide how to go further.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

khal
Site Admin
Posts: 708
Joined: Mon May 09, 2011 5:09 pm
os: linux

Re: [ANN] libcoin rebase attempt

Post by khal »

domob wrote:Update: It seems to work now when I explicitly addnode my server (I removed its maxconnections restriction). I can download the blockchain just fine but stay at one connection. So possibly there are some problems with peer discovery. Also, it doesn't work with "--verify FULL --validate FULL" so far (I get a segfault), and I presume that we want these options for a full node (although I have no idea what the details are behind other settings here).
Same behavior: addnode seems to behave like --connect (limit to 1 connection, for now), but after a restart, still with addnode, I now have several connections.
"--validate FULL" segfault for all chains I tested (and I don't know how to enable symbols in the binaries with ./configure & make to find more details with gdb except that it segfaults in BlockChain::validation_depth).
domob wrote:EDIT: Now it seems to work just fine without addnode and I get other connections as well. Maybe because already discovered peers (from my addnode attempt) are cached and now tried right away? At least it seems as if I got a running node now. Also, it creates a SQLite3 database of the blockchain that also includes what looks like a name_history table. I feel fairly confident that it would be a simple matter to implement name_list, name_history and name_show based on that if it is not yet available inside libcoin already.
Yes, those 3 rpc commands should be easy to add (name_filter too, if sqlite supports regexps).
name_list will require to search in the wallet.
name_new, name_firstupdate and name_update will require a bit more work and integration with libcoin code to create custom tx.
domob wrote:Maybe I'll try to set up a wallet and transfer coins to it and back later to check out whether that works fully or not. getwork also yields some results (but I have no idea whether they are as they should be), and according to a statement from Michael, getauxwork would not be too hard to add.
Should we start a new testnet and work on this new chain ?
domob wrote:I'm not sure what the "rebase" then actually would be ... just extending libcoin itself so that libcoind can be used as a full namecoin replacement (or such that only a trivial main program can replace namecoind but everything happens inside of libcoin already), or using libcoin as-is and adding the missing features (name querying?) in a (still presumably relatively simple and small) main program? Anything else I'm missing at the moment?
I guess it would be more easy to integrate name_new/update_list, etc in libcoind than having an external software.
For name_scan/filter, the requests can be done externally, on the DB. But having them integrated would simplify usage too, and not add a lot of complex code.
In any case, it will allow lighter block explorer (most data will be in the existing sqlite DB. You can also enable the search by address with --searchable=1, even less job for the explorer)

There will be some work to :
- manage name tx (edit: it seems name values are already decoded and stored, maybe less work required than I initially thought)
- not count locked coins in all balances and rpc calls related to amounts (or to avoid modifying a lot of code, maybe add a getlockedcoins rpc call and leave other rpc calls as they are). But, in all cases, when creating a normal tx, locked coins should not be used (this must be checked).
- implement getauxwork
- find bitcoin features & patches that should not be used in namecoin
- add support for namecoin testnet blockchain
domob wrote:When I've done a few more tests, I'll try to contact Michael about specific things to get his opinion and decide how to go further.
Good :p
NamecoinID: id/khal
GPG : 9CC5B92E965D69A9
NMC: N1KHAL5C1CRzy58NdJwp1tbLze3XrkFxx9
BTC: 1KHAL8bUjnkMRMg9yd2dNrYnJgZGH8Nj6T

Register Namecoin domains with BTC
My bitcoin Identity - Send messages to bitcoin users
Charity Ad - Make a good deed without paying a cent

indolering
Posts: 801
Joined: Sun Aug 18, 2013 8:26 pm
os: mac

Re: [ANN] libcoin rebase attempt

Post by indolering »

Yes, those 3 rpc commands should be easy to add (name_filter too, if sqlite supports regexps).
There is an alias that requires an external library but it's fairly standard.

Could you guys push a branch or repo on github/namecoin? I would like to build Libcoin but I don't understand what's wrong with the build config.

**update** I need to stop writing posts at 2 AM.
Last edited by indolering on Sun Feb 23, 2014 4:05 am, edited 2 times in total.
DNS is much more than a key->value datastore.

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: [ANN] libcoin rebase attempt

Post by domob »

I've contacted Michael now with some questions about things mentiond above as well as a possible inconsistency between libcoind and namecoind (privately because of this), but not yet heard back. I'll keep you updated.

@indolering: I've already created a test-bed repository at https://github.com/domob1812/nmc-libcoin. It doesn't yet contain any code since I'm experimenting with the already included main program (libcoind) for now. When we get forward, I will create a custom one based on this code, though, so that it works for Namecoin out-of-the-box without the --namecoin option. I do not really understand what you want to say, though.

@khal: It would be great to have a functioning test-net (ideally one reset based on a fresh genesis block and possibly with difficulty-resetting that should already be available in libcoin). I will try to add it to libcoin if I get lucky enough to manage it. :)
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

khal
Site Admin
Posts: 708
Joined: Mon May 09, 2011 5:09 pm
os: linux

Re: [ANN] libcoin rebase attempt

Post by khal »

I've added a first version of namecoin testnet on this repo :
https://github.com/khalahan/libcoin

To launch it, use --namecointest

Here is the full command I use to launch it :

Code: Select all

bin/libcoind --namecointest --verification=FULL --validation=NONE --persistance=MINIMAL --portmap=0 --debug --addnode 94.23.252.190 --port=18335 --rpcport=18337

Code: Select all

<full_command> getinfo
{
    "version" : 70300,
    "protocolversion" : 70300,
    "blocks" : 4031,
    "connections" : 3,
    "difficulty" : 0.06249911,
    "testnet" : false
}
Block 4031 is not the last one, I guess I've missed something (chainID of merged mining, or something like that)

Edit: it is related to difficulty adjust of testnet :

Code: Select all

12:20:21 DEBUG(node) [BlockFilter.cpp:243 void BlockFilter::process(const Block&, Peers)] ProcessBlock: ACCEPTED
12:20:21 DEBUG(node) [BlockFilter.cpp:40 virtual bool BlockFilter::operator()(Peer*, Message&)] received block 00000003fbc13a48b8de
12:20:21 DEBUG(node) [Chain.cpp:721 virtual unsigned int NamecoinTestnetChain::nextWorkRequired(BlockIterator) const]   nActualTimespan = 1093913  before bounds
12:20:21 INFO(node) [Chain.cpp:737 nextWorkRequired] GetNextWorkRequired RETARGET
12:20:21 INFO(node) [Chain.cpp:738 nextWorkRequired]    nTargetTimespan = 1209600    nActualTimespan = 1093913
12:20:21 INFO(node) [Chain.cpp:739 nextWorkRequired]    Before: 1d0fffff  0000000fffff0000000000000000000000000000000000000000000000000000
12:20:21 INFO(node) [Chain.cpp:740 nextWorkRequired]    After:  1d0e7840  0000000e78408b82082082082082082082082082082082082082082082082082
12:20:21 DEBUG(node) [Chain.cpp:721 virtual unsigned int NamecoinTestnetChain::nextWorkRequired(BlockIterator) const]   nActualTimespan = 1093913  before bounds
12:20:21 INFO(node) [Chain.cpp:737 nextWorkRequired] GetNextWorkRequired RETARGET
12:20:21 INFO(node) [Chain.cpp:738 nextWorkRequired]    nTargetTimespan = 1209600    nActualTimespan = 1093913
12:20:21 INFO(node) [Chain.cpp:739 nextWorkRequired]    Before: 1d0fffff  0000000fffff0000000000000000000000000000000000000000000000000000
12:20:21 INFO(node) [Chain.cpp:740 nextWorkRequired]    After:  1d0e7840  0000000e78408b82082082082082082082082082082082082082082082082082
12:20:21 ERROR(node) [BlockFilter.cpp:246 process] append(Block) failed: Incorrect proof of work: 487487558 versus 487487552
NamecoinID: id/khal
GPG : 9CC5B92E965D69A9
NMC: N1KHAL5C1CRzy58NdJwp1tbLze3XrkFxx9
BTC: 1KHAL8bUjnkMRMg9yd2dNrYnJgZGH8Nj6T

Register Namecoin domains with BTC
My bitcoin Identity - Send messages to bitcoin users
Charity Ad - Make a good deed without paying a cent

Post Reply