#!/usr/bin/perl -w
use strict;
use lib "/home/ton/lib";
use POE;
use POE::Component::BitTorrent;
use HTTP::Request::Common qw(GET POST);
use HTTP::Cookies;
my $VERSION = 0.01;
sub tstp_handler {
"POE::Session"->create(inline_states => {
tstp => sub {
my $kernel = $_[KERNEL];
$kernel->sig_handled;
# SIGTSTP hack, convert to SIGSTOP
kill "STOP", $$;
return;
},
_start => sub {
my $kernel = $_[KERNEL];
$kernel->sig("TSTP", "tstp");
$kernel->alias_set("tstp_handler");
}
});
}
@ARGV = ("http://localhost/bt/crop.05:00.jpg.torrent") unless
@ARGV;
tstp_handler();
POE::Component::BitTorrent->spawn
(Alias => 'bt',
Agent => "TorrentClient/$VERSION",
CookieJar => HTTP::Cookies->new(file => "lwpcookies.txt", autosave => 1),
);
"POE::Session"->create
(inline_states => {
_start => \&start,
response => \&response,
_default => sub {
die("calling non existant state ", $_[ARG0]) unless
substr($_[ARG0], 0, 1) eq "_";
return;
},
});
sub start {
my $kernel = $_[KERNEL];
$poe_kernel->post('bt',
# state on alias we trigger
'request',
# state on current session expecting response
'response',
# .torrent URL
GET($_),
# user info
"waf",
) for @ARGV;
}
sub response {
# print STDERR "bt response @_\n";
my ($request, $response) = @_[ARG0, ARG1];
print STDERR "Request: $request->[0]\n";
print STDERR "User Info: $request->[1]\n";
print STDERR "Response: $response->[0]\n";
}
eval {
$poe_kernel->run();
};
if ($@) {
print $@;
exit 1;
}
__END__
server side:
web server presents metainfo file, contains Server url, file etc.
runs a TorrentServer, answers http requests
runs one client with the start content
Notes:
--------------------------------------------------------------
I also just checked in endgame mode. Previously, there was a very common
case that you'd download a whole file just fine and spend forever
downloading the last little bit off some loser on a hosed modem line. That
problem has now been fixed. It requests the last few bits from everybody
it can, and sends out cancels as pieces come in.
--------------------------------------------------------------
Downloads are currently going well, the main problem people are
experiencing is that there's often considerable slowness at the end of a
download. This is caused by a backlog of requests to a slow link, which
you don't request from anyone else because you have a policy of only
having one request for a specific piece at a time.
My current plan for how to fix this is to make it so that when a
downloader has pending requests for everything remaining, they go into
another download state where they always keep a full backlog with each
peer, even if it means having duplicate requests pending. To cut down on
redundant bandwidth usage I'm going to add a 'cancel request' message as
well, which won't get rid of all duplicate sends, but should mitigate the
problem considerably.
Of course, the most natural way of implementing that would wind up
requesting in the same order from all peers, which would result in tons of
duplicate piece sends. Implementing it properly promises to be a royal
pain, but the problem should go away once that's done.
So planned protocol changes at this point are -
Get rid of 'type' key - it's just plain unnecessary.
Add 'cancel' message - this will have the same payload as request messages
do.
-------------------------------------------------------------------------
The way nonat mode works is that the tracker tries to connect back to the
downloader on the port they claim to be listening on, and if it doesn't
connect or gets an invalid handshake, they're sent an error message. If it
works they're sent a list of peers.
It's off by default, you can turn it on with --nonat. It's meant for
downloads which are expected to get mostly people behind NAT, so it's
really a choice between bad performance for everyone or good performance
for people who can accept incoming connections and no performance for
people behind NAT.
----------------------------------------------------------------------------
Yes. BitTorrent makes two connections to each counterparty - an upload and
a download. Which side initiated the connection doesn't affect how it
functions.
The advantage to having separate upload and download connections, aside
from ease of implementation, is that control codes for one aren't held up
by bulk data from the other.
Publishers spend a lot of time trying to give stuff to downloaders on slow
connections. This can be fixed by making them prefer downloaders based on
their download rate, rather than balance.
Downloaders spend a lot of time uploading to counterparties who they've
downloaded a lot from before but are so in debt to now it's hopeless. This
can be fixed by making them prefer based on their current rate of
downloada from each peer, rather than balances.
Hopefully this won't cost me any libertarian end users.
Both downloaders and publishers suffer from getting 'locked' into a set of
uploads when better ones are available. This can be fixed by giving them,
in addition to their 'permanent' uploads, a single 'floater' upload which
rotates through all their connections changing once every 20 seconds. If
you're wondering where I got 20 seconds from, it was picked via the same
highly scientific and thus far successful method I've used for all other
constants in BitTorrent, which is to say, I pulled it outta my ass.
---------------------------------------------------------------------
Guide to Fast Downloads
-----------------------
The name of the game is connecting to as many peers as possible. If you are
behind a NAT or firewall, the single thing that will make the biggest
difference in your download speed is to reconfigure your NAT/firewall to allow
incoming connections on the ports that BT is listening to (it uses a new port
for every download, starting at the minimum you specify.) Then all the other
peers behind a NAT or firewall will connect to you so that you can download
from them.
BitTorrent uses "tit for tat" for deciding which peer to upload to. In general
terms, the client uploads to the peers that it is downloading from the fastest.
This is why there can be a delay after connecting to peers before downloading
begins; you have nothing to upload to other peers. The torrent typically
bursts to life once your client gets a complete piece or two. If there is
excess bandwidth available, perhaps because many peers left their window open,
then you can get good download rates without uploading much. If you are on a
very fast connection and think you could be downloading faster, try increasing
the maximum number of uploads; by uploading to more peers you may end up
downloading from more peers. Give the client a few minutes to "settle" after
tweaking it. The client uses one upload "slot" to cycle through peers looking
for fast downloads and only changes this slot every 30 seconds.
------------------------------------------------------------------
Currently BitTorrent doesn't use getopt but does use basically the same
format - arguments are passed in as -name=value pairs before the ordered
args. Here's a list of what they are -
For the downloader -
unthrottle_diff - how much a peer's balance must exceed that of the
lowest balance current downloader before they get unthrottled, currently
defaults to 2 ** 23, will be removed after the switch from balances to
transfer rates.
rethrottle_diff - the point at which unthrottle_diff is undone - defaults
to 2 ** 20, will be removed after the switch to transfer rates.
max_uploads - the maximum number of uploads to allow at once, dafault is
2, will be changed to 3.
max_downloads - the maximum number of downloads to do at once, default is
4, will be increased.
download_chunk_size - how many bytes to query for per requests, defaults
to 2 ** 15
request_backlog - how many requests to keep in a single pipe at once,
defaults to 5
min_fast_reconnect, max_fast_reconnect - range in seconds of how long to
wait to try to reconnect a locally initiated connection which dropped
(there used to be a slow_reconnect, but it's gone now - it's unclear than
even keeping these is a good idea).
max_message_length - maximum length prefix encoding you'll accept
over the wire - larger values get the connection dropped.
port - port to listen on, defaults to 6880 - should actually be made to
default to a random port, like the windows build does.
socket_poll_period - number of milliseconds to block in calls to poll()
myip - ip to report you have to the publicist, the name of this should be
changed to 'ip'
For the publisher -
rethrottle_diff, unthrottle_diff, max_uploads, max_downloads - same as in
downloader, these will be eliminated since publisher throttling is
inherently very different from downloader throttling.
piece_size - how big the individually hashed pieces which the files are
broken into should be, defaults to 2 ** 20
max_message_length - same as in downloader
port - port to listen on, defaults to 6881
socket_poll_period - same as in downloader
ip - ip of self to report to the publicist, optional - useful if, for
example, the publicist is on localhost and will detect the ip as
127.0.0.1, which others can't use to connect.
location - the prefix url for announcing to the publicist
postlocation - the post url for announcing to the publicist - optional
the publicist -
port - the port to listen on, defaults to 8080
ip - your own port, for sending announcement urls to downloaders (sorry, I
couldn't let them be relative urls because MSIE is a big stinking piece of
crap and won't pass along the original url).
-------------------------------------------------------------------
People have been commenting about relative performance between 3.1 and
3.0.2 a lot. This is a very complicated issue, so I'll explain in
detail. At the bottom I'll explain what I'm going to do about the new
problems, they're all quite fixable.
There are four things which 3.1 has over 3.0.2 which are likely to alter
performance.
(1) Better rate estimation
The old method of calculating transfer rates had some nasty artifacts and
tended to inflate transfer rates. The new algorithm is much better, and
also the default slice size was reduced from 32k to 16k, which doubled the
effective sampling rate. These changes will produce subtly better overall
performance, but somewhat lower (although much more accurate!) estimated
transfer rates.
(2) rarest first
Previously, clients downloaded pieces in random order. This caused several
problems. For one thing, in the case of a deployment which was
bottlenecking on the upload rate of the origin, the download would go much
slower than necessary because the origin would frequently send out the
same piece several times. For another, the likelihood of some piece
getting lost was rather high, so the origin going down frequently made a
file unavailable.
Rarest first downloads pieces starting at the least common. These are the
pieces which both are most likely to be unavailable later and are most
likely to be useful for uploading to peers.
After 3.1 was released, there was a sudden and dramatic improvement in the
robustness and longevity of BitTorrent deployments. I think this is mostly
due to rarest first (and (4) below, which rarest first enabled).
Unfortunately rarest first exacerbates a problem which was happening
before. When a downloader first connects, especially at the very start of
a very popular file, they have nothing to upload, so they sit there choked
by everyone until they manage to cobble together a whole piece from random
optimistic unchokes and then can start uploading, at which tit-for-tat
works its magic and they start getting a decent transfer rate.
Rarest first is arguably the worst possible strategy at the very
beginning, because it maximizes the amount of time it will take to
download that one piece. This is somewhat mitigated by (4) below, which
increases the chances that a whole piece will be transferred during a
single optimistic unchoke period, but it's still a serious problem,
especially at the beginning of a very popular download.
Of course, random sometimes happens to pick the rarest first, so the old
behavior wasn't especially good at the beginning either.
(3) anti-snubbing
Watching transfers happen with diagnostics turned on, I noticed that
occasionally all peers in 69 happen to choke at the same time, resulting
in a very slow download rate until the optimistic unchoke finds better
ones. To mitigate this, I added anti-snubbing, which checks to see if one
of the peers it would like to reciprocate with hasn't sent anything in
over a minute, and in that case dedicate that slot to an optimistic
unchoke.
This technique doesn't kick in unless poor download rates are being
experienced, and in those cases it does at least twice as much optimistic
unchoking as it does normally. The result is much more consistent download
rates for everyone.
Due to the evening out effect this produces, some people may experience
reduced download rates because the idiosynchracies of their connection
happened to make them get the better of the somewhat more arbitrary load
distribution which was happening before. It's hard to say when this might
happen though.
A much more serious problem this produces has to do with how I implemented
it. Previously, once a downloader finished they preferred peers which they
uploaded to the fastest. Now once a peer is finished it simply figures
it's snubbed by everyone, and unchokes four different peers every
period. This results in greatly reduced upload rate because connections
have to slow start once every 30 seconds, and also because peers on very
fast connections may happen to unchoke only peers on much slower
connections in any given period, bottlenecking on the download
side. Setting max_uploads to 20 helps with this problem but doesn't
completely make it go away.
I think this is the largest performance regression to happen in the latest
release, since it drastically reduces upload capacity utilization.
(4) reduced default piece size
The default piece size was reduced in the latest release from a megabyte
to a quarter megabyte. This won't make any difference for .torrents which
were generated with the old release, but for ones generated with the new
one it will result in a much shorter slow period at the very beginning and
subtly better performance at other times.
This increases on the wire overhead somewhat, but it's still well below
one percent. The main danger is that the risk of a piece getting lost is
much greater if all the clients are 3.0.2, but with rarest first widely
implemented that isn't a concern.
Those are the new problems, here are the planned solutions -
(1) Rarest first will be kept, but when no pieces have been downloaded yet
instead of picking pieces which only one peer has, it will try to pick
pieces which about half of all of its peers have, these should both be
reasonably quick to download and useful for uploading, since a fair number
of peers have them and a fair number of peers want them.
(2) Upload behavior after completion will be changed to favor greater
upload rates again.
(3) A new feature will be added to complete uploaders that they give each
new downloader which connects at least a full piece before choking them.
This will help speed up the initial slow phase a lot. This is obviously
gameable, but it's always been the case that you can get more out of a
complete downloader by making lots of connections to it, and the way to
deal with that is to make complete downloaders view peers with the same IP
address as effectively the same connection for unchoking purposes.
That should clean up all the new artifacts nicely. The next release will
in all ways be clearly better than both 3.0.2 and 3.1.
It occured to me that the first piece selection algorithm I mentioned
before sucks in a very common and important case.
Specifically, when there's a single seeder and many downloaders who have
nothing, it will cause dowloaders to all want the piece which one other
downloader has, since two peers will have that, which is closer to half
than one, which is how many peers have every other piece.
To avoid a meltdown in which deployments bottlenecked on the seeder
completely waste all of their bottleneck's capacity, I'm planning on
making it revert to random piece selection before a single piece is done
being downloaded, and rarest first after that. This should result in
decent performance under almost all circumstances.
Amazing how hard it is to come up with algorithms for this stuff which
don't have really nasty artifacts. My whole approach to improving the
algorithms has focused on ignoring any vague concepts of qualitative
measures of efficiency and instead focused on getting rid of clear
artifacts without intruducing new ones. At least this way I know I'm
------------------------------------------------------------------------
> Actually, the tracker doesn't even know what pieces everyone needs to
> download. All that it really needs to know is where the current
> downloaders are, the data about download amounts sent back to it is just
> for statistics gathering.
>
> -Bram
-------------------------------------------------------------------------
From reading the source of track.py, the '/scrape' information seems to
be as follows:
When a request for the url '/scrape' is made on the BitTorrent tracker,
a bEncoded dictionary with one key 'files'. The value for this key is a
bEncoded dictionary. The key for each item is the info_hash of a
torrent, and the value is a bEncoded dictionary with keys as follows:
complete: The integer encoded number of complete downloaders *currently
connected* for this file
incomplete: The integer encoded number of *currently connected* downloaders
that have not yet finished downloading this file
name: An optional bEncoded string containing the filename of this .torrent
-----------------------------------------------------------
I've gotten exactly one comment on it, and have had the chance to mull it
over for a few days, so here is my official pronouncement on where to find
the /scrape url
Take the tracker url. Find the last '/' in it. If the text immediately
following that '/' isn't 'announce' it will be taken as a sign that that
tracker doesn't support the scrape convention. If it does, substitute
'scrape' for 'announce' to find the scrape page.
Some examples
http://spam.com/announce
http://spam.com/scrape
http://spam.com/a
None
http://spam.com/announce.php
http://spam.com/scrape.php
http://spam.com/announce?x=2%0644
http://spam.com/scrape?x=2%0644
http://spam.com/announce?x=2/4
None
http://spam.com/x%064announce
None
http://spam.com/x/announce
http://spam.com/x/scrape
Note especially that entity unquoting is *not* to be done.
Everyone please make your tools comply with this standard.
-----------------------------------------------------------------
I just added some new parameters to the tracker protocol.
Downloaders can now send a 'numwant' parameter, which is the number of
peers they want. Currently it's only used for a downloader to say 0 when
they already have enough.
New parameter 'tracker id' sent back from tracker and 'trackerid' sent to
tracker. This is for trackers which are worried about losing their state
information. It's echoed back if sent.
Downloaders now accept a 'min internval' parameter, which is an optional
advisory piece of information for how often they should rerequest if they
don't have enough peers. I don't advise using this yet through, for some
rather complicated reasons.
New parameters 'last' and 'num peers' are sent back from the tracker, and
'last' is also a parameter back to the tracker. These are both
integers. The purpose of 'last' is so that the tracker can report to the
downloader what the last update number on the peer list was if there was a
small enough number of peers that the downloader got everything, and the
downloader can report that back so the tracker only sends back new
stuff. This should cut down a lot on tracker bandwidth usage.
'num peers' is the number of non-natted peers available, not including the
one connecting in. It's sent back so that the downloader can check if the
number of peers returned is too out of kilter and stop sending in 'last'
temporarily, in case old connections to peers which are still up somehow
got killed.
'last' and 'num peers' aren't supported by trackers yet. I've decided to
push off implementing them until after the new release, to avoid
destabilizing things for the time being.
New parameters will probably be added later as part of making seeders be
more intelligent about picking what files to seed, but that's all for now.
--------------------------------------------------------------------
(user suggestion)
When downloading more than one file at a time (such as a whole directory tree),
please add a preference for any individual file that is near completion.
Although the current policy "any block is just as good as any other" tends
to maximize the efficiency of the swarm, there is value in the policy
"shortest time to finish". For instance, I downloaded all three .iso
of redhat9 (638MB+646MB+485MB = 1769MB) in about 13 hours, yet I had no
complete file until a few minutes before the entire batch finished.
Here are some possible policies; there are others.
1) If there is an incomplete file for which remaining <= sqrt(length) ,
then don't request blocks from any file which doesn't meet that test.
[A 650MB .iso would get priority when there is less than 25KB remaining.
This is weak, except when there are many individual files.]
2) Or, give any incomplete file a relative preference of
1.0/ceiling(log2(remaining)) . [Shorter files get a boost from
the beginning.]
3) Or, double the preference of any incomplete file with
remaining <= length/ceiling(log2(length)) . [A 650MB .iso would get
priority with 22MB remaining.]
----------------------------------------------------------------
http://groups.yahoo.com/group/btports/
---------------------------------------------------------------
There is an IRC channel on server irc.freenode.net channel #btports if you
want to go there.
---------------------------------------------------------------
Currently BitTorrent has a strong tendency to fragment file systems, since
it writes out data in random order. I've considered several methods of
fixing this, and concluded that the best one is to do background
allocation, as suggested by Taral (who does file systems for a living).
My first pass at implementing this will be to spawn off a thread which
allocates the entire file with 0xFF. Until that's done, all data to be
written out will be cached in memeory. Once allocation is complete, data
stored in memory will be written out and file access will proceed as
normal.
This has the disadvantage that LAN transfers might use up a whole lot of
memory at the beginning. That problem is unusual and fixable, but tricky,
so I'll deal with it after the first pass.
-Bram
------------------------------------------------------------------------
Currently doing resumes with BitTorrent requires scanning the entire file
first. It would obviously be beneficial to not have to wait for that.
The problem with making resumes faster is that the information about what
pieces are downloaded already has to be stored somewhere, and putting it
into another file is very klunky. The approach which I actually like is to
put a bitfield at the end of the file which contains data about what's
been downloaded, then truncate the file when the download is complete.
This way resumes can happen very quickly. An incomplete file can be
immediately recognized based on its length (size + bitfield size) and a
complete one can be recognized similarly.
I'm not sure what the best thing to do is with a resume of a too-small
file. These might be resumes of ftp transfers, or they might be mistaken
overwrites. Perhaps the best thing to do is check the first piece, and if
it matches assume it's an ftp resume, and if not figure it's a mistake.
I'm not sure what to do if there isn't a single piece's worth of data in
the file to resume.
-------------------------------------------------------------------
Also changed write characted to be 0x1 instead of 0x0 - I suspect some file
systems don't force an allocation when you write a 0 after the end of the file.
> Just noticed this value in the tracker log. What is it?
>
> numwant=0
That tells the tracker 'I have enough peers, don't send me any'. It's to
save bandwidth.
------------------------------------------------------------------
I think I've figured out how BitTorrent will behave on startup for the
next release. There are two completely separate cases - fresh download and
resume.
In the fresh download case, it will start by pre-allocating the file
starting from the beginning. It will download pieces and store them in
already pre-allocated parts of the file. When preallocation is complete,
the file will be rearranged to have pieces written in the correct places,
and download will continue as normal.
In the resume case, it will start immediately and check parts of the file
in random order. Parts of the file which don't hash check will be
downloaded as normal, and parts of the file which do hash check will be
announced to all peers that they're present when that fact is discovered.
-------------------------------------------------------------------
Turn on word wrapping to read this document.
BitTorrent tit-for-tat reworked by David Player. SECOND DRAFT COPY
Works just like BitTorrent currently does except that the tit-for-tat
and optimistic unchoke algorithms are replaced.
The problem:
Currently BT unchokes N peers and has an optional or practical
bandwidth limit of B. This bandwidth is divided amongst these peers
in a completely uncontrolled manner. Currently BT unchokes the (N-1)
peers who are sending the most throughput at the moment, and also
unchokes one random peer. In practice, several peers will have a high
B/(N-1) ratio R, and others will have a low rate. Thus for a fast
peer, increasing R will result in a few unchokes from fast high R
peers, while decreasing R will result in more unchokes from slower or
low R peers but a loss of trade with the fast peers. Unfortunately
the number of peers is not infinity and this leads to inefficient
bandwidth matchups.
N Number of unchoked peers
B Amount of bandwidth in use
R B/(N-1)
For example, a very fast peer has enough bandwidth to become
preferred by the 3 other high speed high R peers, as well as more
left over. Tweaking the numbers by hand, this peer engages in 3
somewhat equal exchanges with these fast peers, plus 1 very uneven
exchange with a slower / low R peer. Instead of spending all that
bandwidth on a single slow peer, it would be more efficient to break
that bandwidth into 2 additional connections to engage in 3 somewhat
even exchanges with slow / low R peers. Unfortunately that is not
possible currently.
A flawed solution:
One solution that has been publicly mentioned several times is for
each peer to broadcast a 'preferred list minimum' number to all
peers, indicating how much bandwidth the slowest peer on the
preferred list is currently sending. Other peers could then allocate
that much bandwidth to make the list. Unfortunately bandwidth is not
a controlled resource that can be allocated. Even in single user
environments users will run other network software and the amount of
available bandwidth will change unexpectedly. Another problem is that
sending frequent minimum announcements will cause fibrillation
amongst peers as they repeatedly vie for one peer only to lose
interest in favor of a now neglected peer.
A dynamic solution:
Language notes:
'may' indicates that the feature is optional
'should' or 'will' indicates that the feature is mandatory
The problem thus far has been the focus on throughput instead of
data. Throughput rates are difficult to control, but data is not. The
key is to focus on the total amount of data transferred instead of
the instantaneous rate. Each peer in an exchange will try to maintain
a 1:1 exchange of data. A peer that has sent you more data than
you've sent it is a preferred partner, and vice versa. To keep
exchanges active and quick, each peer will try to stay slightly ahead
of its partner, and may also give partners that are currently sending
data some leeway before choking it. This does not imply that the
throughput rate on each connection must be precisely managed, since
as long as you are ahead on any exchange your partner should
continue. Bursts and or trickles are fine, though efficient clients
may prefer to keep sending to active partners and slowly increase the
bandwidth over time as reasonable. Excess bandwidth can be spent in
any way a client wants, but an efficient client may spread bandwidth
over many peers so that they become indebted and will (hopefully)
start sending data in return. Clients will set aside some bandwidth
for Optimistic Unchoke, I suggest the square root of your current
bandwidth in kilobytes per second. This may be sent to many peers or
only one.
Note: Optimistic Unchoke
In the old system, one key (though unstated) purpose of Optimistic
Unchoke was to find better trading partners. In this new system, that
is no longer the case in any way. However optimistic unchoke is still
included so that new peers will have something to trade with other
members of the swarm.
Note: Impatient clients and minimum actual data throughput rates
One obvious outcome of this algorithm is that clients may have an
infinite number of active exchanges going at once. This causes
several problems including clients waiting on trickles to reach 100%
and pieces not being transmitted quickly enough to facilitate
efficient trading. As such, clients will attempt to maintain a
minimum bandwidth on active connections of 2 kilobytes per second,
and under no circumstances less than 0.75 kilobytes per second. Thus,
the 16kB chunks which will each be transmitted in at most 8 to 22
seconds. Peers transmitting at less than these rates should be
considered inactive, and chunks may be cancelled and requested from
other peers.
Note: Concurrent connections
Note: Bandwidth calculations
Bandwidth is not instantaneous. Maximum outgoing bandwidth should be
calculated as an average over 2 minutes. Incoming bandwidth from a
single peer should be calculated as an average over 1 minute (to
allow for bursty peers).
Note: peer memory
This system lends itself to long term peer memory and debt.
Unfortunately BT does not currently have any sort of unique PEER ID
system. Until BT has this feature, and hopefully some security
regarding it, peer memory is difficult to get right. Once a reliable
unique PEER ID system is in place, clients should keep peer memory
(though records may expire).
Note: Hacked clients
Or, the exploitability of this algorithm. This is a data ratio
exchange algorithm and as such is not inherintly vulnerable to
attack. However, clients could try the following:
Send data and then stop.
Peers will remove you from their preferred list and stop sending
data. They will probably send you slightly more than you sent them,
but this number will probably be small enough to be useless.
Send bogus data.
Peers should remember who sent what chunks. As a suggestion, when a
piece fails a hash check, a CRC32 value of each chunk should be
generated using a semi-random lookup table. When the piece later
passes the hash check, the client may determine which peer send the
bad data and debit them double that amount. Peers that consistantly
send bad data may be blacklisted. The CRC32 table may be randomized
at startup; the only point to this is that malicious peers can't send
CRC32 matching bogus data.
Trickle data to nearly peers to prevent piece / torrent completion.
Active sends that average less than 0.75 kilobytes per second
should be considered inactive. Any chunks being sent by that peer
should be cancelled and requested from another peer. If clients
repeatedly send at less than minimum throughput without first sending
a choke message they should be less preferred for trading and
requests.
Last mile
At some point a hacked client may switch tit-for-tat algorithms in
an effort to reach completion more quickly. If the client is clever
enough this will work fine. The only protection against this is peer
memory.
Note: Two peers monopolize each other.
Two peers send data to each other at their maximum throughput rates,
minus the optimistic unchoke. These peers effectively monopolize each
other, and while they get good throughput briefly they are not acting
in the best interest of the swarm or, most likely, even themselves.
For this reason, clients should prefer not to use more than a third
of their bandwidth on a single peer. There are cases where this rule
should not be followed, but I leave determination of that up to
specific clients.
Note: Leechers.
A peer joins a torrent and uploads nothing or nearly nothing. They
will never be preferred and will only get optimistic unchoke and
excess bandwidth.
Note: Download faster than upload?
My intention in writing this algorithm is to reduce inefficiencies
and unfairness in bittorrent. Making your download rate more closely
based on your upload rate is the major goal. Currently many peers are
able to download faster than they upload due to excess bandwidth
caused by peers, mismatches in other exchanges, and imperfectly set N
values (this is also made possible by the fact that they have
asynchronous connections, but that is not the cause). This system
will be more fair to all peers regardless of their bandwidth, and any
excess bandwidth will be spread more evenly throughout the swarm.
Thus using bittorrent in an efficient manner will require and reward
less fine tweaking.
Note: Compatibility
Although these changes do not require any change in the protocol, it
will not work perfectly with older clients because the algorithms
don't match.
Note: Seeds
Seeds should only allow oportunistic unchoking, instead of sending
to the N fastest peers. The number of unchokes, in the case of seeds
N, should vary depending on the bandwidth being used compared to the
estimated total available bandwidth.
------------------------------------------------------
> The problem thus far has been the focus on throughput instead of
> data. Throughput rates are difficult to control, but data is not. The
> key is to focus on the total amount of data transferred instead of
> the instantaneous rate.
BitTorrent worked that way a long, long time ago. It's a disaster. You
wind up using a lot of completely irrelevant historical data which doesn't
matter because network conditions have changed since.
The first time I did a trial deployment after basic one machine to another
transfers started working well it was using straight long term ratios. It
fell on its face after about a half-dozen peers entered.
------------------------------------------------------------------------------
There is a major problem with the new rarest-first prioritization for
the PiecePicker algorithm, which I discovered after taking some
statistics of transfers. Apparently what is happening is that partial
transfers of rares get "stranded" as piece priorities change, and
spurned in favor of a different rare piece. On a torrent of about 15
clients and about 450 pieces I saw better than 50 pieces partially
downloaded as the torrent entered the endgame phase.
This poses problems both for the individual and for the torrent at
large. For the individual, because the more incomplete pieces
retrieved, the more data that has to be discarded if a client needs to
be restarted. For the entire torrent, because it decreases the number
of complete pieces and therefore the torrent's redundancy.
I modified my client so that instead of switching behavior based on the
number of pieces completed, it used rarest-first only on seeds, and
otherwise used the basic try-to-complete-first from 3.2.1b. The result
seems to be encouraging, reducing the number of incomplete pieces at any
time by about 2/3rds. I haven't finished testing yet, though.
--------------------------------------------------------------------
The bug which caused restarted trackers to reject re-publication after
they were restarted has been fixed.
A work-around for filling the TCP cache was added. When a connection first
starts, the upload code thinks it's sending very fast because a bunch of
pieces get quickly shoved into the kernel's TCP cache. I also put in the
work-around for downloading - it might make sense to get rid of that.
Immediate reciprocation has been added! With this latest change, I can
finally say that the choking algorithms are fairly mature. You guys have
no idea what a long road this has been.
--------------------------------------------------------------------------
Immediate reciprocation when done right is pretty easy - 15 seconds after
someone unchokes you, check if their upload rate is greater than two of
the current downloaders, and if so put them in first priority and
rechoke. It's two instead of one to avoid the situation where you put
someone in first place because they beat the old floater then immediately
put them in last place because they didn't beat the old long-term
connections. ('floater' refers to the peer you're currently giving a
free trial run to, to see if they reciprocate.)
Rate limiting isn't too bad, just gotta use token buckets, and doing it in
Uploader will make distributing 'fairly' across multiple connections
pretty easy.
------------------------------------------------------------------
When BitTorrent first starts or loses a connection, it waits between
min_fast_reconnect and max_fast_reconnect seconds to try to connect
again. The defaults are 5 and 25 minutes, which can be boring to wait for,
so I have those set to 1 minute and 4 minutes (60 and 240).
min_fast_reconnect should not be zero because your counterparty may be in
the process of cycling their process.
When BitTorrent tries to connect but gets connection refused, it waits
between min_slow_reconnect and max_slow_reconnect seconds to try to
connect again. The defaults are three and nine hours, respectively, which
you may want to shorten a bit.
----------------------------------------------------------------------
A bunch of people have noticed that partial completion isn't working very
well in 3.3. This is because I did a poor job of striking a balance
between the following two behaviors
1) A torrent as a whole performing well when there's only one slow seeder
2) Each individual downloader completing pieces as much as possible.
3.3 is much better at 1, at the expense of 2.
To fix this, I'm going to (yet again) change how completing partials is
prioritized vs. rarest first. The new strategy is as follows -
When downloading from non-seeders, always prefer completion of partials.
When downloading from seeders, prefer partials which were already partway
downloaded from a seed. Do not prefer partials which haven't had any
downloading done from a seed. However, being a partial acts as a tiebreak
within rarest first, and if a partial which wasn't previously part
downloaded by a seed is part downloaded by a seed it goes into the
partials from seeds category.
When downloading from non-seeders, partials from seeds take priority over
partials from non-seeds.
That should do a good job of meeting both criteria. It's based on a
suggestion someone had of only preferring rarest first from seeders, but I
just today thought of the dual categorization to keep too many partials
from building up. It's somewhat based on another heuristic I had in CVS
before but tore out because it didn't work well.
Hopefully after the next release I'll be done with headaches related to
piece picking.
--------------------------------------------------------------------------
Choke cancels the outstanding requests outbound. You should clear
any buffered requests when you choke, and the remote should assume
that the buffered requests have been cleared, and request them via
another peer.
You will also want to reassign that block in preference over most
other blocks in the file since it is already partially complete. If
you don't then it will take you forever to get your first complete
block as peers often choke before transferring a complete block.
--------------------------------------------------------------------------
The mainline client rejects non-ascii filenames, for security
reasons. So the standard is ascii, because nothing else works.
It's actually a lot more stringent than just ascii - I decided
to be on the safe side and whitelist rather than blacklist.
File name canonicalization is a security disaster.
-Bram
--------------------------------------------------------------------------
Proposed multitracker extension:
http://bittornado.com/docs/multitracker-spec.txt
--------------------------------------------------------------------------
Kevin, this is not correct behavior. The correct behavior is to send
all zeros back to the other client. The protocol handshake is meant so
that, if a client has an extension that corresponds to a bit in that
handshake, if it receives a zero for that bit for a client it will not
utilize that extension with that client, but instead will operate in a
backwards-compatible fashion.
--------------------------------------------------------------------------
When I originally made BitTorrent protocol, I used expanded base 10 with
period delimiters notation for IP addresses on the grounds that it could
be easily extended to IPv6. Since it's now clear that special casing for
IPv6 will be necessary anyway, I'm going to go even farther than
no_peer_id and put in a compact, which will cause not only peer ids to not
be returned but make the whole peer list be a single string in binary
format. The format will be that each peer's info is six bytes long, the
first four being the ip and the last two being the port.
I'm going to do this for 2.4.1 and junk no_peer_id, since it's a mostly
useless intermediate optimization. The code for doing compact is mostly
the same though.
-Bram
--------------------------------------------------------------------------
http://wiki.theory.org/index.php/BitTorrentSpecification
--------------------------------------------------------------------------
I'm planning on building ipv6 support into BitTorrent. A crucial feature
of this support is that there should be overlap between the two systems
within a single torrent. Previously I stated that I didn't understand the
issues well enough to make any comments about ipv6. Having spoken about it
with shadow, who's already dealt with and gotten burnt by the issues, I
think I understand it well enough to have a plan.
When a client connects to a tracker, it checks if it (the client) supports
ipv6. If it doesn't, it forgets about ipv6 completely and proceeds to do
everything with ipv4. If it does, then it checks if the tracker has an
ipv6 dns entry. If not, it proceeds with ipv4 as before (I'm making the
assumption that any tracker which is able to support ipv6 will be able to
have its dns records updated, which might or might not be reasonable). If
the tracker has both an ipv6 dns entry and an ipv4 dns entry, then the
client connects to the tracker via ipv4 and sends an ipv6= parameter in
addition to all the usual parameters.
The tracker remembers for each peer whether it supports ipv4 only, ipv6
only, or both ipv4 and ipv6. If a peer connects with ipv4 without an
ipv6= parameter, then it's assumed to be ipv4 only. If a peer connects via
ipv6, it's assumed to be ipv6 only. If a peer connects via ipv4 with an
ipv6= parameter, then it's assumed to support both, although NAT check
(really firewall check, since natting ipv6 is ludicrous) is kicked off for
both of them.
Does anyone who is ipv6-knowledgeable have any opinions about this?
-Bram
--------------------------------------------------------------------------
I thought about ipv6 some more, and went over feedback from this list and
some other places, and have a new plan.
The new plan is that when a client connects via ipv4 it's assumed to
support ipv4 only. If it then connects via ipv6 later it's assumed to
support both ipv4 and ipv6. Typical client behavior will be to connect
first via ipv4, then if that works make another connection via ipv6.
I think this is much simpler than the previous plan, both conceptually and
in terms of implementation.
-Bram
In the new plan, ipv6 only is supported just fine. Clients simply connect
via ipv6, and the tracker assumes that they only speak ipv6 unless they at
some point later connect via ipv4.
-Bram
--------------------------------------------------------------------------
Snoop:
Start client
Phase 1: Get torrent file (packet 11)
52397 connects to localhost:www
Does request:
GET /bt/crop.05:00.jpg.torrent HTTP/1.0\r\n
Request Method: GET
Host: localhost\r\n
User-Agent: Python-urllib/2.0a1\r\n
Accept-encoding: gzip\r\n
User-agent: BitTorrent/3.3\r\n
\r\n
www answers:
HTTP/1.1 200 OK\r\n
Date: Tue, 30 Sep 2003 17:54:40 GMT\r\n
Server: Apache/1.3.27 (Unix) mod_fastcgi/2.4.0 mod_ssl/2.8.14 OpenSSL/0.9.6c\r\n
Last-Modified: Sun, 28 Sep 2003 14:41:08 GMT\r\n
ETag: "8286-764-3f76f304"\r\n
Accept-Ranges: bytes\r\n
Content-Length: 1892\r\n
Connection: close\r\n
Content-Type: application/x-bittorrent\r\n
\r\n
d8:announce30:http://127.0.0.1:6969/announce13:creation datei1064759973e4:infod6:lengthi22169e4:name14:crop.05:00.jpg12:piece lengthi256e6:pieces1740:k\221\302\332yL\255\003_\303N\202\3709\205\351\212\224\n
www closes connection
52397 acks
Phase 2: port 52398 connects to tracker (packet 21)
GET /announce?info_hash=-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2&peer_id=%00%00%00%00%00%00%00%00%00%00%00%00%F5%1A%E4%E8Z%87X%08&port=6883&uploaded=0&downloaded=0&left=22169&event=started HTTP/1.0\r\n
Request Method: GET
Host: 127.0.0.1:6969\r\n
User-Agent: Python-urllib/2.0a1\r\n
Accept-encoding: gzip\r\n
User-agent: BitTorrent/3.3\r\n
\r\n
tracker responds:
HTTP/1.0 200 OK\r\n
Response Code: 200
Content-Length: 108\r\n
Content-Type: text/plain\r\n
Pragma: no-cache\r\n
Content-Encoding: gzip\r\n
\r\n
Data (108 bytes)
0000 1f 8b 08 00 60 c3 79 3f 02 ff 4b b1 b0 ca cc 2b ....`.y?..K....+
0010 49 2d 2a 4b cc c9 34 b4 30 30 48 35 b5 2a 48 4d I-*K..4.00H5.*HM
0020 2d 2a ce 49 31 b2 ca 2c b0 b4 32 34 32 d7 33 00 -*.I1..,..242.3.
0030 42 43 73 b0 b8 42 66 8a 91 81 15 03 12 a8 d1 7b BCs..Bf........{
0040 28 fb 5e 7d 71 9a 89 55 41 7e 51 49 a6 99 85 85 (.^}q..UA~QI....
0050 61 6a 2a b1 9a bf 4a 3d 79 11 d5 1e c1 81 d0 6c aj*...J=y......l
0060 9c 0a 04 00 f9 20 77 5c 95 00 00 00 ..... w\....
decodes to:
{ 'peers' => [
{ 'peer id' => '|.áï\'£f',
'ip' => '127.0.0.1',
'port' => '6881'
},
{ 'peer id' => 'õäèZ',
'ip' => '127.0.0.1',
'port' => '6883'
}
],
'interval' => '1800' };
tracker closes connection
52398 acks
Phase 3:
(Before answering the GET, the tracker itself seems to contact the client)
52399 connects to 6883 (the client listener) (frame 26)
(When the connect succeeds, the tracker GET is answered)
52399 sends
0000 13 42 69 74 54 6f 72 72 65 6e 74 20 70 72 6f 74 .BitTorrent prot
0010 6f 63 6f 6c 00 00 00 00 00 00 00 00 2d 04 4f c0 ocol........-.O.
0020 db 1a 61 75 39 be 4d 72 c2 a5 45 a0 fd 48 03 e2 ..au9.Mr..E..H..
19 'BitTorrent protocol' 8*0x00,
sha1: -.O...au9.Mr..E..H.. (-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2)
(this fixes the file that we want)
a bit later client sends greeting to tracker:
0000 13 42 69 74 54 6f 72 72 65 6e 74 20 70 72 6f 74 .BitTorrent prot
0010 6f 63 6f 6c 00 00 00 00 00 00 00 00 2d 04 4f c0 ocol........-.O.
0020 db 1a 61 75 39 be 4d 72 c2 a5 45 a0 fd 48 03 e2 ..au9.Mr..E..H..
0030 00 00 00 00 00 00 00 00 00 00 00 00 f5 1a e4 e8 ................
0040 5a 87 58 08 Z.X.
19 'BitTorrent protocol' 8*0x00,
sha1: -.O...au9.Mr..E..H.. (-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2)
(this fixes the file that we want)
peer id: 00 00 00 00 00 00 00 00 00 00 00 00 f5 1a e4 e8 5a 87 58 08
Tracker closes connection
Phase 4:
Client on port 52400 contacts one of the announced peers:
Client sends greeting to peer (frame 39)
0000 13 42 69 74 54 6f 72 72 65 6e 74 20 70 72 6f 74 .BitTorrent prot
0010 6f 63 6f 6c 00 00 00 00 00 00 00 00 2d 04 4f c0 ocol........-.O.
0020 db 1a 61 75 39 be 4d 72 c2 a5 45 a0 fd 48 03 e2 ..au9.Mr..E..H..
0030 00 00 00 00 00 00 00 00 00 00 00 00 f5 1a e4 e8 ................
0040 5a 87 58 08 Z.X.
19 'BitTorrent protocol' 8*0x00,
sha1: -.O...au9.Mr..E..H.. (-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2)
(this fixes the file that we want)
peer id: 00 00 00 00 00 00 00 00 00 00 00 00 f5 1a e4 e8 5a 87 58 08
(peer_id=%00%00%00%00%00%00%00%00%00%00%00%00%F5%1A%E4%E8Z%87X%08)
Peer sends greeting to client (frame 41):
0000 13 42 69 74 54 6f 72 72 65 6e 74 20 70 72 6f 74 .BitTorrent prot
0010 6f 63 6f 6c 00 00 00 00 00 00 00 00 2d 04 4f c0 ocol........-.O.
0020 db 1a 61 75 39 be 4d 72 c2 a5 45 a0 fd 48 03 e2 ..au9.Mr..E..H..
0030 00 00 00 00 00 00 00 00 00 00 00 00 7c 2e e1 1d ............|...
0040 ef 27 a3 66 .'.f
19 'BitTorrent protocol' 8*0x00,
sha1: -.O...au9.Mr..E..H.. (-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2)
(this fixes the file that we want)
peer id: 00 00 00 00 00 00 00 00 00 00 00 00 7c 2e e1 1d ef 27 a3 66
Peer sends messages to client:
0000 00 00 00 0c 05 ff ff ff ff ff ff ff ff ff ff fe ................
0010 00 00 00 01 01 .....
one length 12 message:
05: bitfield (telling what he has) for 87 pieces (extra bit set to 0)
ff ff ff ff ff ff ff ff ff ff fe
one length 1 message:
01: unchoke
client sends message to peer (frame 48):
00 00 00 01 01 .....
unchoke
Client sends message to peer (frame 52):
0000 00 00 00 01 02 00 00 00 0d 06 00 00 00 27 00 00 .............'..
0010 00 00 00 00 01 00 00 00 00 0d 06 00 00 00 3b 00 ..............;.
0020 00 00 00 00 00 01 00 00 00 00 0d 06 00 00 00 46 ...............F
0030 00 00 00 00 00 00 01 00 00 00 00 0d 06 00 00 00 ................
0040 49 00 00 00 00 00 00 01 00 00 00 00 0d 06 00 00 I...............
0050 00 38 00 00 00 00 00 00 01 00 .8........
1 interested (2)
length 13 request: index 0x27, begin 0x00, length 0x100
length 13 request: index 0x3b, begin 0x00, length 0x100
length 13 request: index 0x46, begin 0x00, length 0x100
length 13 request: index 0x49, begin 0x00, length 0x100
length 13 request: index 0x38, begin 0x00, length 0x100
Peer sends to client (frame 54):
length 0x109 Piece 0x27 starting at 0x00 +256 bytes
Client sends to peer:
length 13 request: index 0x40, begin 0x00, length 0x100
Peer sends to client (frame 54):
length 0x109 Piece 0x3b starting at 0x00 +256 bytes
length 0x109 Piece 0x46 starting at 0x00 +256 bytes
length 0x109 Piece 0x49 starting at 0x00 +256 bytes
length 0x109 Piece 0x38 starting at 0x00 +256 bytes
client sends to peer (frame 57):
have 0x27
Peer sends to client (frame 58):
length 0x109 Piece 0x40 starting at 0x00 +256 bytes
Client sends to peer (frame 59):
length 13 request: index 0x3e starting at 0x00, length 0x100
Peer sends to client (frame 60):
length 0x109 Piece 0x3e starting at 0x00 +256 bytes
Client sends to peer (frame 61):
have 0x3b
Client sends to peer (frame 63):
length 13 request: index 0x4e starting at 0x00, length 0x100
have 0x46
request: index 0x2e starting at 0x00, length 0x100
have 0x49
request: index 0x07 starting at 0x00, length 0x100
have 0x38
request: index 0x39 starting at 0x00, length 0x100
have 0x40
request 0x28
have 0x3e
....bla bla bla...
52401 gets tracker status again (frame 193)
GET /announce?info_hash=-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2&peer_id=%00%00%00%00%00%00%00%00%00%00%00%00%F5%1A%E4%E8Z%87X%08&port=6883&uploaded=0&downloaded=22169&left=0&event=completed HTTP/1.0\r\n
Request Method: GET
Host: 127.0.0.1:6969\r\n
User-Agent: Python-urllib/2.0a1\r\n
Accept-encoding: gzip\r\n
User-agent: BitTorrent/3.3\r\n
\r\n
(no tracker probe here)
Tracker answers:
HTTP/1.0 200 OK\r\n
Response Code: 200
Content-Length: 108\r\n
Content-Type: text/plain\r\n
Pragma: no-cache\r\n
Content-Encoding: gzip\r\n
\r\n
Data (108 bytes)
0000 1f 8b 08 00 61 c3 79 3f 02 ff 4b b1 b0 ca cc 2b ....a.y?..K....+
0010 49 2d 2a 4b cc c9 34 b4 30 30 48 35 b5 2a 48 4d I-*K..4.00H5.*HM
0020 2d 2a ce 49 31 b2 ca 2c b0 b4 32 34 32 d7 33 00 -*.I1..,..242.3.
0030 42 43 73 b0 b8 42 66 8a 91 81 15 03 12 a8 d1 7b BCs..Bf........{
0040 28 fb 5e 7d 71 9a 89 55 41 7e 51 49 a6 99 85 85 (.^}q..UA~QI....
0050 61 6a 2a b1 9a bf 4a 3d 79 11 d5 1e c1 81 d0 6c aj*...J=y......l
0060 9c 0a 04 00 f9 20 77 5c 95 00 00 00 ..... w\....
{ 'peers' => [ { 'peer id' => '|.áï\'£f',
'ip' => '127.0.0.1',
'port' => '6881'
},
{ 'peer id' => 'õäèZ',
'ip' => '127.0.0.1',
'port' => '6883'
}
],
'interval' => '1800'}
tracker closes request
Client closes connection to peer
Phase 5:
Client on port 52402 connects to peer (6881)
Client sends greeting (frame 206):
0000 13 42 69 74 54 6f 72 72 65 6e 74 20 70 72 6f 74 .BitTorrent prot
0010 6f 63 6f 6c 00 00 00 00 00 00 00 00 2d 04 4f c0 ocol........-.O.
0020 db 1a 61 75 39 be 4d 72 c2 a5 45 a0 fd 48 03 e2 ..au9.Mr..E..H..
0030 00 00 00 00 00 00 00 00 00 00 00 00 f5 1a e4 e8 ................
0040 5a 87 58 08 Z.X.
19 'BitTorrent protocol' 8*0x00,
sha1: -.O...au9.Mr..E..H.. (-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2)
(this fixes the file that we want)
peer id: 00 00 00 00 00 00 00 00 00 00 00 00 f5 1a e4 e8 5a 87 58 08
(peer_id=%00%00%00%00%00%00%00%00%00%00%00%00%F5%1A%E4%E8Z%87X%08)
Peer sends greeting (frame 207):
0000 13 42 69 74 54 6f 72 72 65 6e 74 20 70 72 6f 74 .BitTorrent prot
0010 6f 63 6f 6c 00 00 00 00 00 00 00 00 2d 04 4f c0 ocol........-.O.
0020 db 1a 61 75 39 be 4d 72 c2 a5 45 a0 fd 48 03 e2 ..au9.Mr..E..H..
0030 00 00 00 00 00 00 00 00 00 00 00 00 7c 2e e1 1d ............|...
0040 ef 27 a3 66 .'.f
19 'BitTorrent protocol' 8*0x00,
sha1: -.O...au9.Mr..E..H.. (-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2)
(this fixes the file that we want)
peer id: 00 00 00 00 00 00 00 00 00 00 00 00 7c 2e e1 1d ef 27 a3 66
Peer sends bitmap (have all), Choked
0000 00 00 00 0c 05 ff ff ff ff ff ff ff ff ff ff fe ................
0010 00 00 00 01 01 .....
Client sends bitmap (have all)
0000 00 00 00 0c 05 ff ff ff ff ff ff ff ff ff ff fe ................
Peer closes connection
(Long pause, user presses ^C)
Client connects to tracker:
GET /announce?info_hash=-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2&peer_id=%00%00%00%00%00%00%00%00%00%00%00%00%F5%1A%E4%E8Z%87X%08&port=6883&uploaded=0&downloaded=22169&left=0&event=stopped HTTP/1.0\r\n
Request Method: GET
Host: 127.0.0.1:6969\r\n
User-Agent: Python-urllib/2.0a1\r\n
Accept-encoding: gzip\r\n
User-agent: BitTorrent/3.3\r\n
\r\n
tracker responds:
HTTP/1.0 200 OK\r\n
Response Code: 200
Content-Length: 88\r\n
Content-Type: text/plain\r\n
Pragma: no-cache\r\n
\r\n
d8:intervali1800e5:peersld2:ip9:127.0.0.17:peer id20:\000\000\000\000\000\000\000\000\000\000\000\000|.\341\035\357'\243f4:porti6881eeee
=========================================================================
Now a tracker starts that already has the complete file:
Client connects to www:
GET /bt/crop.05:00.jpg.torrent HTTP/1.0\r\n
Request Method: GET
Host: localhost\r\n
User-Agent: Python-urllib/2.0a1\r\n
Accept-encoding: gzip\r\n
User-agent: BitTorrent/3.3\r\n
\r\n
Www returns .torrent to Client:
HTTP/1.1 200 OK\r\n
Response Code: 200
Date: Tue, 30 Sep 2003 21:05:20 GMT\r\n
Server: Apache/1.3.27 (Unix) mod_fastcgi/2.4.0 mod_ssl/2.8.14 OpenSSL/0.9.6c\r\n
Last-Modified: Sun, 28 Sep 2003 14:41:08 GMT\r\n
ETag: "8286-764-3f76f304"\r\n
Accept-Ranges: bytes\r\n
Content-Length: 1892\r\n
Connection: close\r\n
Content-Type: application/x-bittorrent\r\n
\r\n
d8:announce30:http://127.0.0.1:6969/announce13:creation datei1064759973e4:infod6:lengthi22169e4:name14:crop.05:00.jpg12:piece lengthi256e6:pieces1740:k\221\302\332yL\255\003_\303N\202\3709\205\351\212\224\n
Data (1723 bytes)
Client connects to tracker:
GET /announce?info_hash=-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2&peer_id=%00%00%00%00%00%00%00%00%00%00%00%00A%D3%BD%AE%3B%09Z%5E&port=6883&uploaded=0&downloaded=0&left=0&event=started HTTP/1.0\r\n
Request Method: GET
Host: 127.0.0.1:6969\r\n
User-Agent: Python-urllib/2.0a1\r\n
Accept-encoding: gzip\r\n
User-agent: BitTorrent/3.3\r\n
\r\n
Tracker connects to client
Tracker does track answer:
HTTP/1.0 200 OK\r\n
Response Code: 200
Content-Length: 109\r\n
Content-Type: text/plain\r\n
Pragma: no-cache\r\n
Content-Encoding: gzip\r\n
\r\n
Data (109 bytes)
{ 'peers' => [ {
'peer id' => '^@^@^@^@^@^@^@^@^@^@^@^@«0!Ë^Nj²-',
'ip' => '127.0.0.1',
'port' => '6881'
}, { 'peer id' => '^@^@^@^@^@^@^@^@^@^@^@^@é9ýÚ\\±¹!',
'ip' => '127.0.0.1',
'port' => '6883'
}
],
'interval' => '1800' };
tracker sends up to sha1 (48 bytes) to client.
tracker closes tracker connection
Client connects to peer
Client does handshake
Peer does handshake
Peer sends bitmap (have all) and unchoke
Client sends handshake to tracker probe
Tracker closes tracker probe
Client sends to Peer: bitmap (have all)
Peer closes connection
(later, ^C)
Client connects to tracker
GET /announce?info_hash=-%04O%C0%DB%1Aau9%BEMr%C2%A5E%A0%FDH%03%E2&peer_id=%00%00%00%00%00%00%00%00%00%00%00%00A%D3%BD%AE%3B%09Z%5E&port=6883&uploaded=0&downloaded=0&left=0&event=stopped HTTP/1.0\r\n
Request Method: GET
Host: 127.0.0.1:6969\r\n
User-Agent: Python-urllib/2.0a1\r\n
Accept-encoding: gzip\r\n
User-agent: BitTorrent/3.3\r\n
\r\n
Tracker answers:
HTTP/1.0 200 OK\r\n
Response Code: 200
Content-Length: 88\r\n
Content-Type: text/plain\r\n
Pragma: no-cache\r\n
\r\n
d8:intervali1800e5:peersld2:ip9:127.0.0.17:peer id20:\000\000\000\000\000\000\000\000\000\000\000\000\2530!\313\016j\262-4:porti6881eeee
tracker closes.