Libs for NMControl and documentation for developers.

[indolering]

Before we can attract donations and developers, we need a solid development platform such that a developer can just install the backend stuff in one step and have libraries to communicate with it.

Jeremy and I chatted on IRC about cleaning up NMControl and auto-generating documentation from inline commenting of the API. However, I think that might have been a misstep. NMControl needs to be cleaned up so others can hack on it: better inline commenting, basic documentation, etc.

However, if we are going to spend our time documenting NMControl's JSON-RPC API, we might was well just write a spec using an Interface Definition Language from which we can generate documentation and JSON-RPC client libraries for communicating with NMControl.

While toying with creating a new API for Namecoind last quarter* I gained experience with two solutions for auto-generating JSON-RPC libraries: Barrister-RPC and Haxe.

Barrister-RPC

• Clients in Java, Node, PHP, Python, and Ruby.
• Natively generates Docco-style docs.
• I don't believe it generates unit tests for each target language.
• Probably the most robust solution since it is purpose built for this task.

Haxe with jsonhx

• Clients in Java, C#, PHP, Python, Ruby, and C++.
• Since the IDL is written in JSON, the documentation would only be semi-automated.
• Should generate native unit tests for all of the target languages.
• NOT purpose built for generating RPC libraries, probably involves more QA.
• Poor tooling for Haxe and the relative immaturity of the language probably negates any of the fancy automated unit testing.
• Would require adding the HTTP transport to NMControls JSON-RPC interface.

When I was working on this last quarter, I got stuck on jsonhx's lack of support for Node.js's async execution (which took a long time to track down because it works fine in the browser). However, it should just work for every other language.

I've got a lot of work ahead of me this summer (8-hours of intensive spanish/day, finding a job, moving, etc) so I'm actually leaning towards Barrister-RPC because I think it would be easier to recruit new developers for the task. However, this is a really crucial area of development and I will commit to job once NMControl is to the point that I can test against it.

*I abandoned the Haxe-based API (and the abstractions I wanted) because I figured out a clever way to load everything from the blockchain directly into CouchDB without having to handle each record individually ... thus negating the utility of a generic JS library for Namecoin for the immediate task. That and I decided I should work on other Namecoin stuff and circle back after Haxe's tooling improves.

[phelix]

Before we can attract donations and developers, we need a solid development platform such that a developer can just install the backend stuff in one step and have libraries to communicate with it.

Jeremy and I chatted on IRC about cleaning up NMControl and auto-generating documentation from inline commenting of the API. However, I think that might have been a misstep. NMControl needs to be cleaned up so others can hack on it: better inline commenting, basic documentation, etc.

However, if we are going to spend our time documenting NMControl's JSON-RPC API, we might was well just write a spec using an Interface Definition Language from which we can generate documentation and JSON-RPC client libraries for communicating with NMControl.

While toying with creating a new API for Namecoind last quarter* I gained experience with two solutions for auto-generating JSON-RPC libraries: Barrister-RPC and Haxe.

Barrister-RPC

• Clients in Java, Node, PHP, Python, and Ruby.
• Natively generates Docco-style docs.
• I don't believe it generates unit tests for each target language.
• Probably the most robust solution since it is purpose built for this task.

Haxe with jsonhx

• Clients in Java, C#, PHP, Python, Ruby, and C++.
• Since the IDL is written in JSON, the documentation would only be semi-automated.
• Should generate native unit tests for all of the target languages.
• NOT purpose built for generating RPC libraries, probably involves more QA.
• Poor tooling for Haxe and the relative immaturity of the language probably negates any of the fancy automated unit testing.
• Would require adding the HTTP transport to NMControls JSON-RPC interface.

When I was working on this last quarter, I got stuck on jsonhx's lack of support for Node.js's async execution (which took a long time to track down because it works fine in the browser). However, it should just work for every other language.

I've got a lot of work ahead of me this summer (8-hours of intensive spanish/day, finding a job, moving, etc) so I'm actually leaning towards Barrister-RPC because I think it would be easier to recruit new developers for the task. However, this is a really crucial area of development and I will commit to job once NMControl is to the point that I can test against it.

*I abandoned the Haxe-based API (and the abstractions I wanted) because I figured out a clever way to load everything from the blockchain directly into CouchDB without having to handle each record individually ... thus negating the utility of a generic JS library for Namecoin for the immediate task. That and I decided I should work on other Namecoin stuff and circle back after Haxe's tooling improves.

http://blockchained.com/stuff/namerpc.py_ for Python. Still somewhat experimental but it's really quite simple.

[biolizard89]

After chatting with indolering about this, I still don't understand the purpose of dedicated NMControl libs. It's standard JSON-RPC over TCP; most languages already have libs for this protocol. What's there for us to do?

I do think we should document which libs support JSON-RPC over TCP (this isn't always obvious)... but that's way easier+simpler than writing new libs.

[domob]

After chatting with indolering about this, I still don't understand the purpose of dedicated NMControl libs. It's standard JSON-RPC over TCP; most languages already have libs for this protocol. What's there for us to do?

I do think we should document which libs support JSON-RPC over TCP (this isn't always obvious)... but that's way easier+simpler than writing new libs.

I agree. IMHO, JSON-RPC is totally fine and in some sense already provides the API / "library" by itself. We need documentation at best, and yes, a list of code ready-to-use seems also like a good idea. (For Python, my Bitmessage patch includes this, for instance.)

[biolizard89]

After chatting with indolering about this, I still don't understand the purpose of dedicated NMControl libs. It's standard JSON-RPC over TCP; most languages already have libs for this protocol. What's there for us to do?

I do think we should document which libs support JSON-RPC over TCP (this isn't always obvious)... but that's way easier+simpler than writing new libs.

I agree. IMHO, JSON-RPC is totally fine and in some sense already provides the API / "library" by itself. We need documentation at best, and yes, a list of code ready-to-use seems also like a good idea. (For Python, my Bitmessage patch includes this, for instance.)

FYI, a few months ago I needed to access NMControl from Python3, and I hacked together a quick library (based on NMControl's rpcClient.py) which makes querying NMControl very simple (1 line of code per query, plus 1 line of code at the beginning for providing the host/port of NMControl; the queries can call any NMControl plugin and they return JSON equivalent to what the NMControl command line client outputs). Would it be useful it I started a GitHub repo for the Python2/3 versions of that lib, so that example libs in other languages can also be added there? Seems a central repo for NMControl examples would be quite useful. (For example, I could also post the Javascript version from FreeSpeechMe.)

[domob]

FYI, a few months ago I needed to access NMControl from Python3, and I hacked together a quick library (based on NMControl's rpcClient.py) which makes querying NMControl very simple (1 line of code per query, plus 1 line of code at the beginning for providing the host/port of NMControl; the queries can call any NMControl plugin and they return JSON equivalent to what the NMControl command line client outputs). Would it be useful it I started a GitHub repo for the Python2/3 versions of that lib, so that example libs in other languages can also be added there? Seems a central repo for NMControl examples would be quite useful. (For example, I could also post the Javascript version from FreeSpeechMe.)

I think such a thing would be useful - it could also include code for namecoind itself, and be a general "client code snippets" thing. I also have JS code for namecoind (but easily adaptable for NMControl, I think, it does HTTP manually so just leaving that away should do the trick), and I think it uses a simpler (more high-level) API of Mozilla than FreeSpeechMe's. libnmcrpc provides the functionality (plus much more high-level constructs) in C++ with libcurl, and I've also a version that uses pure C and GLib. And, of course, the PHP code from NameID. All these could be added there.

[biolizard89]

FYI, a few months ago I needed to access NMControl from Python3, and I hacked together a quick library (based on NMControl's rpcClient.py) which makes querying NMControl very simple (1 line of code per query, plus 1 line of code at the beginning for providing the host/port of NMControl; the queries can call any NMControl plugin and they return JSON equivalent to what the NMControl command line client outputs). Would it be useful it I started a GitHub repo for the Python2/3 versions of that lib, so that example libs in other languages can also be added there? Seems a central repo for NMControl examples would be quite useful. (For example, I could also post the Javascript version from FreeSpeechMe.)

I think such a thing would be useful - it could also include code for namecoind itself, and be a general "client code snippets" thing. I also have JS code for namecoind (but easily adaptable for NMControl, I think, it does HTTP manually so just leaving that away should do the trick), and I think it uses a simpler (more high-level) API of Mozilla than FreeSpeechMe's. libnmcrpc provides the functionality (plus much more high-level constructs) in C++ with libcurl, and I've also a version that uses pure C and GLib. And, of course, the PHP code from NameID. All these could be added there.

https://github.com/JeremyRand/NMC-Client-API-Examples

Licensed under CC0 (since these are examples and nothing more).

Feel free to make a fork under the namecoin GitHub account.

[phelix]

FYI, a few months ago I needed to access NMControl from Python3, and I hacked together a quick library (based on NMControl's rpcClient.py) which makes querying NMControl very simple (1 line of code per query, plus 1 line of code at the beginning for providing the host/port of NMControl; the queries can call any NMControl plugin and they return JSON equivalent to what the NMControl command line client outputs). Would it be useful it I started a GitHub repo for the Python2/3 versions of that lib, so that example libs in other languages can also be added there? Seems a central repo for NMControl examples would be quite useful. (For example, I could also post the Javascript version from FreeSpeechMe.)

I think such a thing would be useful - it could also include code for namecoind itself, and be a general "client code snippets" thing. I also have JS code for namecoind (but easily adaptable for NMControl, I think, it does HTTP manually so just leaving that away should do the trick), and I think it uses a simpler (more high-level) API of Mozilla than FreeSpeechMe's. libnmcrpc provides the functionality (plus much more high-level constructs) in C++ with libcurl, and I've also a version that uses pure C and GLib. And, of course, the PHP code from NameID. All these could be added there.

https://github.com/JeremyRand/NMC-Client-API-Examples

Licensed under CC0 (since these are examples and nothing more).

Feel free to make a fork under the namecoin GitHub account.

http://blockchained.com/stuff/namerpc.py_ based on domob's Bitmessage integration probably can do the same. It can connect to both namecoind / nmcontrol. Indolering certainly is right in that we should merge all these efforts into one standard lib.

[indolering]

http://blockchained.com/stuff/namerpc.py_ based on domob's Bitmessage integration probably can do the same. It can connect to both namecoind / nmcontrol. Indolering certainly is right in that we should merge all these efforts into one standard lib.

Thanks for the vote of confidence, it's frustrating to see all this work being duplicated. It's also just confusing because our own pet-projects are generally good enough for our own purposes leave much to be desired for anyone else: scant documentation, no testing, etc, etc.