Namecoin Torrent tracking

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Namecoin Torrent tracking

Post by domob »

In recent posts, the idea of a Namecoin based Torrent tracker has come up some times. I don't remember seeing any 'formal' brainstorming about how this could/should work (sorry if there exists already a thread somewhere which I missed). I like the idea very much - Namecoin together with Bittorrent as a P2P data distribution scheme would allow for a censorship resistant Wikileaks (and other things of varying degrees of usefulness). This sounds very much like an ideal Namecoin application to me.

Are there already ideas about how the tracking (plus a decentralised Torrent directory, I guess) could work? What is the difference to finding peers by a DHT, which seems to be already implemented in Bittorrent clients? What would the advantage be, and how could it be leveraged efficiently? (I don't know much about how DHT tracking works at the moment, to be honest.)
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

moa
Posts: 255
Joined: Mon May 23, 2011 6:13 am

Re: Namecoin Torrent tracking

Post by moa »

It has been discussed several times on IRC is only place I know of.

Several features ;

- storage of magnet links http://en.wikipedia.org/wiki/Magnet_URI_scheme in the blockhain making direct verifable look-ups possible, since file hash is the link

- magnet links provided by a specific NMC_ADDRESS could incorporate some kind of financial reward (donations) scheme using nmc payments to the link providing address ... and reputation of known good NMC_ADDRESS in providing reliable magnet links can be accrued

- other more complex variations on above points

snailbrain
Posts: 309
Joined: Tue Jul 19, 2011 9:33 pm

Re: Namecoin Torrent tracking

Post by snailbrain »

i think what we are waiting for is pruning?

but I "believe", it is not a difficult task at all to create NMC-Torrent Client

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Namecoin Torrent tracking

Post by domob »

snailbrain wrote:i think what we are waiting for is pruning?

but I "believe", it is not a difficult task at all to create NMC-Torrent Client
I don't think either, that's why I find it to be an interesting idea. I usually don't hang around on IRC, so that's probably why I didn't know of anything already discussed. Maybe we could set up a thread or so to do a "public brainstorming"?

BTW, if we provide magnet links, wouldn't this be more like a "Torrent directory" than "tracker"? Isn't a tracker the server that knows about nodes providing a particular Torrent, and telling others where to connect to? As I understand it, this is no longer necessary with the DHT approach, though. (And also, I think TPB operates both a directory (which is their website) and a tracker (which is an additional service provided on their servers), which is why they are still often called "tracker".)

Thinking about the proposal to have encrypted values in order to shield against law enforcement problems: In general, I like the idea, especially if it is optional. However, in particular for this use-case, I don't know whether it is a good thing to have. Since we want to have a directory of magnet links, it should be possible to view and browse without knowing the names already. Otherwise, if someone needs to know the "secret name" of a Torrent, doesn't this make the whole system obsolete? They could just as well know the magnet link itself already.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Namecoin Torrent tracking

Post by biolizard89 »

domob wrote:
snailbrain wrote:i think what we are waiting for is pruning?

but I "believe", it is not a difficult task at all to create NMC-Torrent Client
I don't think either, that's why I find it to be an interesting idea. I usually don't hang around on IRC, so that's probably why I didn't know of anything already discussed. Maybe we could set up a thread or so to do a "public brainstorming"?

BTW, if we provide magnet links, wouldn't this be more like a "Torrent directory" than "tracker"? Isn't a tracker the server that knows about nodes providing a particular Torrent, and telling others where to connect to? As I understand it, this is no longer necessary with the DHT approach, though. (And also, I think TPB operates both a directory (which is their website) and a tracker (which is an additional service provided on their servers), which is why they are still often called "tracker".)

Thinking about the proposal to have encrypted values in order to shield against law enforcement problems: In general, I like the idea, especially if it is optional. However, in particular for this use-case, I don't know whether it is a good thing to have. Since we want to have a directory of magnet links, it should be possible to view and browse without knowing the names already. Otherwise, if someone needs to know the "secret name" of a Torrent, doesn't this make the whole system obsolete? They could just as well know the magnet link itself already.
I said this via PM, but I'll say it here for anyone who didn't receive that. If encryption is added, I think collision detection should be made optional. d/ and id/ names will typically be queried with collision detection turned on (so that any duplicate d/ names which were registered don't have any effect on .bit). Torrent tracking would have collision detection disabled. This basically is equivalent to a Bitmessage chan (anyone who knows the chan name can access all content within that chan), except with the persistence and editing capabilities of Namecoin. Here's how I view that being used in practice: a torrent community of some kind can choose a shared name, which could be anything from a name of a group "torrent/wikileaks" to an interest area "torrent/movies". Anyone with the name can access all content within that name (much like anyone who knows the URL of The Pirate Bay can access all of its content). The key is that the miners and the users who don't use a particular name aren't able to see its contents, which gives them plausible deniability with respect to "knowingly facilitating infringement".

It occurs to me that collision detection may be useful even in these cases to avoid impostor torrents. So, how about this proposal: the portion of a name which appears after the 2nd forward slash will be encrypted using the rest of the name as the key, and values are also encrypted with that key. Collisions are still prevented as currently. Example:

Full name: "torrent/wikileaks/insurance2010"
Full value: "magnet:xyz"
Stored in blockchain as:
Name: hash("torrent/wikileaks/") CONCAT encrypt(data="insurance2010", key="torrent/wikileaks/")
Value: encrypt(data="magnet:xyz", key="torrent/wikileaks/")

This allows only anyone who knows the name portion "torrent/wikileaks/" to see all the names which start with that, while maintaining name collision detection. This is basically as private or public as people want, in that you can choose a guessable or nonguessable 2nd-level namespace (what I'm calling the portion before the 2nd forward slash). But it allows miners and other entities handling the blockchain to legitimately argue that they don't know what's in their blocks.
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Namecoin Torrent tracking

Post by domob »

biolizard89 wrote:It occurs to me that collision detection may be useful even in these cases to avoid impostor torrents. So, how about this proposal: the portion of a name which appears after the 2nd forward slash will be encrypted using the rest of the name as the key, and values are also encrypted with that key. Collisions are still prevented as currently. Example:

Full name: "torrent/wikileaks/insurance2010"
Full value: "magnet:xyz"
Stored in blockchain as:
Name: hash("torrent/wikileaks/") CONCAT encrypt(data="insurance2010", key="torrent/wikileaks/")
Value: encrypt(data="magnet:xyz", key="torrent/wikileaks/")

This allows only anyone who knows the name portion "torrent/wikileaks/" to see all the names which start with that, while maintaining name collision detection. This is basically as private or public as people want, in that you can choose a guessable or nonguessable 2nd-level namespace (what I'm calling the portion before the 2nd forward slash). But it allows miners and other entities handling the blockchain to legitimately argue that they don't know what's in their blocks.
I haven't thought this through in detail, but it looks like a very nice idea at first glance. This would be a good balance between "public directory" and "private names/values", IMHO. BTW, is there already some "broad" consensus that we want / will implement name/value encryption in the future for Namecoin? I think it should be done (after careful thinking, but possibly with something like your idea above) and can try to help with it, but I haven't got the feeling yet that all others in the community also want this and will support such an effort. (Which would again lead to a hard fork, I guess.)
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

phelix
Posts: 1634
Joined: Thu Aug 18, 2011 6:59 am

Re: Namecoin Torrent tracking

Post by phelix »

domob wrote:
biolizard89 wrote:It occurs to me that collision detection may be useful even in these cases to avoid impostor torrents. So, how about this proposal: the portion of a name which appears after the 2nd forward slash will be encrypted using the rest of the name as the key, and values are also encrypted with that key. Collisions are still prevented as currently. Example:

Full name: "torrent/wikileaks/insurance2010"
Full value: "magnet:xyz"
Stored in blockchain as:
Name: hash("torrent/wikileaks/") CONCAT encrypt(data="insurance2010", key="torrent/wikileaks/")
Value: encrypt(data="magnet:xyz", key="torrent/wikileaks/")

This allows only anyone who knows the name portion "torrent/wikileaks/" to see all the names which start with that, while maintaining name collision detection. This is basically as private or public as people want, in that you can choose a guessable or nonguessable 2nd-level namespace (what I'm calling the portion before the 2nd forward slash). But it allows miners and other entities handling the blockchain to legitimately argue that they don't know what's in their blocks.
I haven't thought this through in detail, but it looks like a very nice idea at first glance. This would be a good balance between "public directory" and "private names/values", IMHO. BTW, is there already some "broad" consensus that we want / will implement name/value encryption in the future for Namecoin? I think it should be done (after careful thinking, but possibly with something like your idea above) and can try to help with it, but I haven't got the feeling yet that all others in the community also want this and will support such an effort. (Which would again lead to a hard fork, I guess.)
+1

https://bitcointalk.org/index.php?topic ... msg1490169
crazy_rabbit wrote:
doublec wrote:
crazy_rabbit wrote: That is quite short- magnet links for example, for movies at popular torrent sites however are many times longer then that. I'd post here an example except for obvious reasons. :-)
As I test I found a long magnet link from a movie site. I took just the hash from that link (the string of hex digits after '?xt=urn:btih:') and searched for that in the DHT network via btdigg.org. This pulled up the correct file information, including a shorter magnet link (without the 'tr' tracker links). This seems to imply that just requiring the hash is fine. I did the same for searching for the archlinux iso (hash e940a7a57294e4c98f62514b32611e38181b6cae) and it came up.

This does seem to indicate however that namecoin isn't needed for storing information since the DHT already has it. Maybe the namecoin provides the "unique identity that can be followed to provide files I'm interested in" aspect.
I didn't know about BTdigg, indeed that sounds like the've already solved the question we were asking. Although perhaps the namecoin blockchain- being merge mined with Bitcoin is more robust?
It seems to be possible to search the torrent dht just like that which somewhat took the momentum from the discussion...
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

domob
Posts: 1129
Joined: Mon Jun 24, 2013 11:27 am
Contact:

Re: Namecoin Torrent tracking

Post by domob »

@phelix: Yes, I think the hash of a file is enough to locate it and carrying nodes using the DHT, making traditional Torrent trackers already obsolete. (I tried to express that already in my first post, but probably it was not formulated clearly.)

However, I think what Namecoin could really provide is the equivalent of TBP's search engine, i. e., a directory. Where anyone can post Torrent hashes together with titles, keywords and things like this, and others can simply query for "collateral murder" and have the corresponding Torrent's hash returned. I don't think the DHT provides this, although I may be mistaken.

Another thought, along the lines of biolizard89's encryption idea: What if Torrent data would be encrypted, with a decryption key stored also together with the Torrent hash in Namecoin? Would this remove legal liability for nodes serving the Torrent data? (Because, I think, most legal complaints / DMCA notices targeted at Tor exit relays are for copyright infringement with Bittorrent.) I'm not a lawyer, though, so I can't answer this. However, I think this is exactly what Mega does to avoid legal problems, IIRC.
BTC: 1domobKsPZ5cWk2kXssD8p8ES1qffGUCm | NMC: NCdomobcmcmVdxC5yxMitojQ4tvAtv99pY
BM-GtQnWM3vcdorfqpKXsmfHQ4rVYPG5pKS
Use your Namecoin identity as OpenID: https://nameid.org/

phelix
Posts: 1634
Joined: Thu Aug 18, 2011 6:59 am

Re: Namecoin Torrent tracking

Post by phelix »

Could be that I misunderstood what BTDigg is doing.

Anyway, trust would be a good selling point here. You could easily create something like a youtube channel. I guess with Bittorrent's DHT alone anybody can use your name - this could easily be prevented with Namecoin. Also you can receive donations.

Actually it is somewhat odd this has not been used so far - Namecoin could be warez paradise. Sigh.
nx.bit - some namecoin stats
nf.bit - shortcut to this forum

biolizard89
Posts: 2001
Joined: Tue Jun 05, 2012 6:25 am
os: linux

Re: Namecoin Torrent tracking

Post by biolizard89 »

domob wrote:@phelix: Yes, I think the hash of a file is enough to locate it and carrying nodes using the DHT, making traditional Torrent trackers already obsolete. (I tried to express that already in my first post, but probably it was not formulated clearly.)

However, I think what Namecoin could really provide is the equivalent of TBP's search engine, i. e., a directory. Where anyone can post Torrent hashes together with titles, keywords and things like this, and others can simply query for "collateral murder" and have the corresponding Torrent's hash returned. I don't think the DHT provides this, although I may be mistaken.

Another thought, along the lines of biolizard89's encryption idea: What if Torrent data would be encrypted, with a decryption key stored also together with the Torrent hash in Namecoin? Would this remove legal liability for nodes serving the Torrent data? (Because, I think, most legal complaints / DMCA notices targeted at Tor exit relays are for copyright infringement with Bittorrent.) I'm not a lawyer, though, so I can't answer this. However, I think this is exactly what Mega does to avoid legal problems, IIRC.
My understanding of Mega's situation is that as long as a service provider can honestly say that they have no way of checking the content that they're serving, they appear to have less liability for that content. This is the principle of the WikiLeaks insurance files as well. So yes, my guess is that if you stored a decryption key in an encrypted Namecoin value along with the magnet URL, any entity sharing the data who is able to honestly say that they don't possess the decryption key (i.e. that they don't know the Namecoin name) should have less liability. (Not a lawyer here either.)
Jeremy Rand, Lead Namecoin Application Engineer
NameID: id/jeremy
DyName: Dynamic DNS update client for .bit domains.

Donations: BTC 1EcUWRa9H6ZuWPkF3BDj6k4k1vCgv41ab8 ; NMC NFqbaS7ReiQ9MBmsowwcDSmp4iDznjmEh5

Post Reply