Discussion:
[tor-relays] Slow relay speeds for Australian geographic location(s)
Mathew
2014-10-11 19:19:16 UTC
Permalink
Hello all,

I run a non-exit relay in Australia. My relay has been running for almost
15 days and has seen very little traffic.
I have a 100/40 fibre connection and bandwidth is set at 2MB/s and 2.5MB/s
burst.

The mean read/write is 3.22kb/s and the advertised bandwidth constantly
varies between 100-800kb/s which is obviously a fraction of my available
bandwidth.

Do relays located in less tor frequented countries see much less traffic or
something? I have ports set at 443 and dirport 9030, is this an optimal
port setting?

Anyone with helpful information would be appreciated. My relay is 1337m8 if
anyone wants to look at the traffic.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141012/e282f489/attachment.html>
Lukas Erlacher
2014-10-11 19:40:02 UTC
Permalink
As always, https://blog.torproject.org/blog/lifecycle-of-a-new-relay is relevant.

Best,
Luke
teor
2014-10-11 23:34:48 UTC
Permalink
Date: Sun, 12 Oct 2014 06:19:16 +1100
From: Mathew <wired.kid at gmail.com>
To: tor-relays at lists.torproject.org
Subject: [tor-relays] Slow relay speeds for Australian geographic
location(s)
Hello all,
I run a non-exit relay in Australia. My relay has been running for almost
15 days and has seen very little traffic.
I have a 100/40 fibre connection and bandwidth is set at 2MB/s and 2.5MB/s
burst.
The mean read/write is 3.22kb/s and the advertised bandwidth constantly
varies between 100-800kb/s which is obviously a fraction of my available
bandwidth.
Do relays located in less tor frequented countries see much less traffic or
something? I have ports set at 443 and dirport 9030, is this an optimal
port setting?
Anyone with helpful information would be appreciated. My relay is 1337m8 if
anyone wants to look at the traffic.
Thanks
A few hints:

1. The Tor network currently has an oversupply of non-exit relays. Therefore, your relay will never be fully utilised unless it is incredibly fast and stable over the longer term (months).

2. Your relay doesn't have the Guard flag yet. When it gets it, there will be a drop in traffic as your relay changes roles from middle to guard, then an increase overall. [-1]

3. If you look at the graphs for your relay on Atlas [0], there is a gradual increase in bandwidth. This should keep gradually increasing, apart from the guard transition in point #2.

4. It's unlikely your relay is compute-bound (at 100% CPU) or network-bound (saturating any network links). But it might be resource-bound (again, this seems unlikely at the levels you're seeing).

Can your NAT box handle ~5000 simultaneous TCP connections/open files/NAT translations?
(Mine can't, it reboots occasionally.)
Can your Windows server handle ~5000 simultaneous TCP connections?

5. Most of the Tor network is outside Australia.

How is TPG's international bandwidth for the NBN? Are the coping? Or are they shaping international transfers? (Please feel free to correct my ISP and network assumptions.)

What do you get if you use an online bandwidth test server located in the US or Europe? What are your typical upload speeds on the link?

6. The bandwidth authorities are outside Australia. This appears to produce greater bandwidth measurement variance. To measure bandwidth, the authorities build a circuit between it, your router, and another router with similar bandwidth. More than likely, the authority and other router are a long way away from Australia. My experience has been variance between 50 - 120 KB/s on a 120KB/s set bandwidth. (Which is a little too low for a Tor relay, by the way.)

And to answer your questions:

A. Tor is country-agnostic, but network-aware by default. Faster, stable relays get priority.

B. In my experience, 443 gets more bandwidth than the Tor default port of 9001. You could try setting up another instance on 9001, and see if it experiences the same issues after a few weeks.


You could also try paste-binning your torrc, and we'll look for specifics.


[-1]: https://blog.torproject.org/blog/lifecycle-of-a-new-relay
[0]: https://atlas.torproject.org/


teor
pgp 0xABFED1AC
hkp://pgp.mit.edu/
https://gist.github.com/teor2345/d033b8ce0a99adbc89c5
http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7wtmU66Mx



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141012/8b383063/attachment.sig>
teor
2014-10-12 01:04:50 UTC
Permalink
Date: Sat, 11 Oct 2014 23:25:47 +0100
From: Tor externet co uk <tor at externet.co.uk>
To: tor-relays at lists.torproject.org
Subject: [tor-relays] Question on running bridge nodes
Message-ID: <49c1abc0aa88e1bf8425fdc8e482402d at nodataavailable.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed
Hi,
I've set up a bridge node in the previous few weeks, but have had to put
a bandwidth limit on, as I only have 10TB of traffic per month before my
ISP will start throttling me to 100k/sec.
I wondered whether it was more helpful to the Tor network as a whole to
have have a very fast node which hibernated every 12-15 hours, or if I
throttled Tor traffic, so that the node was more stable.
I'll confess that I'm far more au fait with the politics of Tor than I
am of the exact ins and outs of how the technology works. Any help would
be gratefully received.
Thanks
L
For relays, where pathing is quite dynamic, we recommend speed + hibernation over uptime.

But for bridges, users obtain only 3 bridge descriptors at a time, usually via some difficult or dangerous method. We'd want to make sure at least 1 stays up at all times (2 for reliability), which would favour throttling.


teor
pgp 0xABFED1AC
hkp://pgp.mit.edu/
https://gist.github.com/teor2345/d033b8ce0a99adbc89c5
http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7wtmU66Mx



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141012/01dc036f/attachment.sig>
Tor externet co uk
2014-10-12 08:30:37 UTC
Permalink
Thanks, that's what I thought, but wasn't sure.

I'll play around for the next few days to see how fast I can get it
without triggering hibernation.

L
Post by teor
Date: Sat, 11 Oct 2014 23:25:47 +0100
From: Tor externet co uk <tor at externet.co.uk>
To: tor-relays at lists.torproject.org
Subject: [tor-relays] Question on running bridge nodes
Message-ID: <49c1abc0aa88e1bf8425fdc8e482402d at nodataavailable.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed
Hi,
I've set up a bridge node in the previous few weeks, but have had to put
a bandwidth limit on, as I only have 10TB of traffic per month before my
ISP will start throttling me to 100k/sec.
I wondered whether it was more helpful to the Tor network as a whole to
have have a very fast node which hibernated every 12-15 hours, or if I
throttled Tor traffic, so that the node was more stable.
I'll confess that I'm far more au fait with the politics of Tor than I
am of the exact ins and outs of how the technology works. Any help would
be gratefully received.
Thanks
L
For relays, where pathing is quite dynamic, we recommend speed + hibernation over uptime.
But for bridges, users obtain only 3 bridge descriptors at a time,
usually via some difficult or dangerous method. We'd want to make sure
at least 1 stays up at all times (2 for reliability), which would
favour throttling.
teor
pgp 0xABFED1AC
hkp://pgp.mit.edu/
https://gist.github.com/teor2345/d033b8ce0a99adbc89c5
http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7wtmU66Mx
teor
2014-10-12 08:43:15 UTC
Permalink
Post by Tor externet co uk
Post by teor
Date: Sat, 11 Oct 2014 23:25:47 +0100
From: Tor externet co uk <tor at externet.co.uk>
To: tor-relays at lists.torproject.org
Subject: [tor-relays] Question on running bridge nodes
Message-ID: <49c1abc0aa88e1bf8425fdc8e482402d at nodataavailable.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed
Hi,
I've set up a bridge node in the previous few weeks, but have had to put
a bandwidth limit on, as I only have 10TB of traffic per month before my
ISP will start throttling me to 100k/sec.
I wondered whether it was more helpful to the Tor network as a whole to
have have a very fast node which hibernated every 12-15 hours, or if I
throttled Tor traffic, so that the node was more stable.
I'll confess that I'm far more au fait with the politics of Tor than I
am of the exact ins and outs of how the technology works. Any help would
be gratefully received.
Thanks
L
For relays, where pathing is quite dynamic, we recommend speed + hibernation over uptime.
But for bridges, users obtain only 3 bridge descriptors at a time,
usually via some difficult or dangerous method. We'd want to make sure
at least 1 stays up at all times (2 for reliability), which would
favour throttling.
teor
Thanks, that's what I thought, but wasn't sure.
I'll play around for the next few days to see how fast I can get it without triggering hibernation.
L
A little hibernation isn't that bad - going down for half an hour every few days isn't as much of an issue for reliability as 18 hours every day.

(Fixed mixed top- and bottom-posting.)

teor
pgp 0xABFED1AC
hkp://pgp.mit.edu/
https://gist.github.com/teor2345/d033b8ce0a99adbc89c5
http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7wtmU66Mx



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141012/4e738e1e/attachment.sig>
Mathew
2014-10-12 06:48:51 UTC
Permalink
Test
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141012/0702e72a/attachment.html>
Mathew
2014-10-12 07:06:42 UTC
Permalink
Sorry about that test.

Thanks for the in depth post, Teor. I had read the lifecycle article but
was concerned when people were posting that their relay was soaking all
their bandwidth after a day or two. This makes sense now, given the
oversupply and location. It was also worrying when the advertised bandwidth
was fluctuating so much and was only advertising a fraction of what
actually is available.

The Windows server is using very minimal resources at the moment, and TCP
connections are around 250. The router will definitely be able to handle
the load, I run a pfSense APU.

TPG doesn't have any bandwidth constraints specifically. They run a lot of
their own infrastructure including their own undersea PPC-1 cable
https://www.tpg.com.au/about/networks.php. No connection is ever shaped by
TPG and I have unlimited bandwidth.

Amsterdam - http://www.speedtest.net/my-result/3825961515
LA - http://www.speedtest.net/my-result/3825963543
Boston - http://www.speedtest.net/my-result/3825964878
London - http://www.speedtest.net/my-result/3825966771

So you think it isn't detecting all of my available bandwidth due to
distance and location and such? Is it just one of those things?

torrc http://pastebin.com/GcTswhQx


Thanks again.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141012/23aeaa86/attachment-0001.html>
teor
2014-10-12 08:11:55 UTC
Permalink
Date: Sun, 12 Oct 2014 18:06:42 +1100
From: Mathew <wired.kid at gmail.com>
To: tor-relays at lists.torproject.org
Subject: Re: [tor-relays] Slow relay speeds for Australian geographic
location(s)
Thanks for the in depth post, Teor. I had read the lifecycle article but
was concerned when people were posting that their relay was soaking all
their bandwidth after a day or two. This makes sense now, given the
oversupply and location. It was also worrying when the advertised bandwidth
was fluctuating so much and was only advertising a fraction of what
actually is available.
The Windows server is using very minimal resources at the moment, and TCP
connections are around 250. The router will definitely be able to handle
the load, I run a pfSense APU.
TPG doesn't have any bandwidth constraints specifically. They run a lot of
their own infrastructure including their own undersea PPC-1 cable
https://www.tpg.com.au/about/networks.php. No connection is ever shaped by
TPG and I have unlimited bandwidth.
Amsterdam - http://www.speedtest.net/my-result/3825961515
LA - http://www.speedtest.net/my-result/3825963543
Boston - http://www.speedtest.net/my-result/3825964878
London - http://www.speedtest.net/my-result/3825966771
So you think it isn't detecting all of my available bandwidth due to
distance and location and such? Is it just one of those things?
torrc http://pastebin.com/GcTswhQx
Actually, I think it's a matter of waiting on a stable IP and network connection for a month or two.
(Your torrc looks fine.)

I was just trying to list all possible factors, however unlikely.

The instability is likely due to the bandwidth authorities (there's only a small number, which doesn't help stability). It will stabilise over time. But I don't think your overall level is caused by the authorities.

Now that I think about it, as far as I recall, the "observed bandwidth" is a notional figure, not an actual bandwidth measurement. Because middle relays are in oversupply, their bandwidths get weighted down. (My middle relays are listed at about 50% of their advertised/link bandwidth.)

Australian middle relays that are getting similar results to yours are:
Aquinas
DC5E2202E0148A53379F68A04207E04FFA7B4B2D (default) - Windows 8
terranullius - Windows 8
CoD

Australian middle relays that are getting almost their exact advertised bandwidth:
Serversaurus

If you can pick the difference, I'm sure we'd be glad to know why!


teor
pgp 0xABFED1AC
hkp://pgp.mit.edu/
https://gist.github.com/teor2345/d033b8ce0a99adbc89c5
http://0bin.net/paste/Mu92kPyphK0bqmbA#Zvt3gzMrSCAwDN6GKsUk7Q8G-eG+Y+BLpe7wtmU66Mx



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141012/96d2a8ce/attachment.sig>
Jon Daniels
2014-10-13 15:36:16 UTC
Permalink
Hi Mathew,

I run multiple exit nodes in the US and have what could be the same
problem. My first node 'apexio' has been running with 99.99% uptime for
four months and the bandwidth usage is minimal and dropping. I'm using
Linux with very fast hardware and ample resources.

I mentioned all the specifics on this list a couple weeks ago and no one
had a solution.

https://globe.torproject.org/#/search/query=apexio

A customer of mine also has the same issue.

Something is going on that's NOT related to hardware or network speed, but
is a symptom of some issue in the Tor network.

Recently I turned up nineteen additional nodes on that server and they're
averaging 60Mbps of overall throughput. CPU load is still 0.00.

-Jon
Post by Mathew
Hello all,
I run a non-exit relay in Australia. My relay has been running for almost
15 days and has seen very little traffic.
I have a 100/40 fibre connection and bandwidth is set at 2MB/s and 2.5MB/s
burst.
The mean read/write is 3.22kb/s and the advertised bandwidth constantly
varies between 100-800kb/s which is obviously a fraction of my available
bandwidth.
Do relays located in less tor frequented countries see much less traffic
or something? I have ports set at 443 and dirport 9030, is this an optimal
port setting?
Anyone with helpful information would be appreciated. My relay is 1337m8
if anyone wants to look at the traffic.
Thanks
_______________________________________________
tor-relays mailing list
tor-relays at lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141013/6f9e367a/attachment.html>
Jeremy Olexa
2014-10-13 22:55:22 UTC
Permalink
Hi Jon,
Post by Jon Daniels
Recently I turned up nineteen additional nodes on that server and they're
averaging 60Mbps of overall throughput. CPU load is still 0.00.
While I can't speak for the Australian problem, I do want to highlight
that you can only have two TOR processes per IP.

"Note that running more than two tor processes per IP address will
result in those other nodes not being used on the network. You'll see
the following message in your logs:

[notice] Heartbeat: It seems like we are not in the cached consensus."

(source: https://www.torservers.net/wiki/setup/server - It probably is
in the spec somewhere but I didn't have the right search term)

-Jeremy
Jon Daniels
2014-10-13 23:25:41 UTC
Permalink
Jeremy,

Yea I noticed that too. So, I ended up putting them all on different
IP's. Working well so far.

Cheers,
Jon
Post by Jeremy Olexa
Hi Jon,
Post by Jon Daniels
Recently I turned up nineteen additional nodes on that server and they're
averaging 60Mbps of overall throughput. CPU load is still 0.00.
While I can't speak for the Australian problem, I do want to highlight
that you can only have two TOR processes per IP.
"Note that running more than two tor processes per IP address will
result in those other nodes not being used on the network. You'll see
[notice] Heartbeat: It seems like we are not in the cached consensus."
(source: https://www.torservers.net/wiki/setup/server - It probably is
in the spec somewhere but I didn't have the right search term)
-Jeremy
_______________________________________________
tor-relays mailing list
tor-relays at lists.torproject.org
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.torproject.org/pipermail/tor-relays/attachments/20141013/cfc87e1b/attachment.html>
Loading...