"octave.sf.net bandwidth exceeded"

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

"octave.sf.net bandwidth exceeded"

Philip Nienhuis
Hmm... just got this when I tried to surf to the OF developers page:

SourceForge.net Logo

This project has been temporarily blocked for exceeding its bandwidth threshold


Projects and users who host on SourceForge.net are given web space to augment their communities and interact with their users. In cases where a project or user's web space exceeds pre-determined thresholds, this message will be displayed. This block is typically short lived (10-15 minutes), unless the cause of the bandwidth usage is not mitigated. We recommend that the administrator of the site take action to reduce bandwidth usage. If you feel you are receiving this message erroneously, please contact SourceForge.net Support for assistance. Typically this message is displayed when files that belong in the FRS are uploaded into project web, or someone tried to host some pr0n or w4r3z. SourceForge.net Support is willing and able to assist in resolving the cause of this error.
In a way this is good news (in that OF gets quite a bit of attention)...

But maybe also something to watch out for as soon as we put binary installers for 3.8.0 there.

Philip
Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

John Swensen-3
On Wed, Jan 8, 2014 at 4:08 PM, Philip Nienhuis <[hidden email]> wrote:
Hmm... just got this when I tried to surf to the OF developers page:


> SourceForge.net Logo
>
> This project has been temporarily blocked for exceeding its bandwidth
> threshold
>
>
> Projects and users who host on SourceForge.net are given web space to
> augment their communities and interact with their users. In cases where a
> project or user's web space exceeds pre-determined thresholds, this
> message will be displayed. This block is typically short lived (10-15
> minutes), unless the cause of the bandwidth usage is not mitigated. We
> recommend that the administrator of the site take action to reduce
> bandwidth usage. If you feel you are receiving this message erroneously,
> please contact SourceForge.net Support for assistance. Typically this
> message is displayed when files that belong in the FRS are uploaded into
> project web, or someone tried to host some pr0n or w4r3z. SourceForge.net
> Support is willing and able to assist in resolving the cause of this
> error.

In a way this is good news (in that OF gets quite a bit of attention)...

But maybe also something to watch out for as soon as we put binary
installers for 3.8.0 there.

Philip



--
View this message in context: http://octave.1599824.n4.nabble.com/octave-sf-net-bandwidth-exceeded-tp4660813.html
Sent from the Octave - Maintainers mailing list archive at Nabble.com.

A quick look at download stats shows the following for the past 7 days:
OSX binary = ~1000 x 682MB = 682GB
mingw binary = ~1130 x 156MB = 176GB
MSVC binary = ~3000 x 71.5MB = 214GB
Total: 1.072 TB in 7 days

I suspect this may be more than they like to host? I also expect that one the newest Windows installer with the GUI goes up that it will be slammed even more.

John Swensen

P.S. Philip, sorry for the double email, I forgot to reply-all on my original response.
Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

Torsten
In reply to this post by Philip Nienhuis
On 08.01.2014 22:08, Philip Nienhuis wrote:

> Hmm... just got this when I tried to surf to the OF developers page:
>
>
>> SourceForge.net Logo
>>
>> This project has been temporarily blocked for exceeding its bandwidth
>> threshold
>>
>>
>> Projects and users who host on SourceForge.net are given web space to
>> augment their communities and interact with their users. In cases where a
>> project or user's web space exceeds pre-determined thresholds, this
>> message will be displayed. This block is typically short lived (10-15
>> minutes), unless the cause of the bandwidth usage is not mitigated. We
>> recommend that the administrator of the site take action to reduce
>> bandwidth usage. If you feel you are receiving this message erroneously,
>> please contact SourceForge.net Support for assistance. Typically this
>> message is displayed when files that belong in the FRS are uploaded into
>> project web, or someone tried to host some pr0n or w4r3z. SourceForge.net
>> Support is willing and able to assist in resolving the cause of this
>> error.
>
> In a way this is good news (in that OF gets quite a bit of attention)...

Maybe good news but maybe someone (ab)uses the site for other (not so
good) stuff.

> But maybe also something to watch out for as soon as we put binary
> installers for 3.8.0 there.

I am sure but to my knowledge the binaries are hosted on

http://sourceforge.net/projects/octave/files/Octave%20Windows%20binaries/

which is  is available. The blocked site is only the project site which
is not supposed to produce a huge amount of traffic.

Torsten

Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

John W. Eaton
Administrator
In reply to this post by Philip Nienhuis
On 01/08/2014 04:08 PM, Philip Nienhuis wrote:

> Hmm... just got this when I tried to surf to the OF developers page:
>
>
>> SourceForge.net Logo
>>
>> This project has been temporarily blocked for exceeding its bandwidth
>> threshold
>>
>>
>> Projects and users who host on SourceForge.net are given web space to
>> augment their communities and interact with their users. In cases where a
>> project or user's web space exceeds pre-determined thresholds, this
>> message will be displayed. This block is typically short lived (10-15
>> minutes), unless the cause of the bandwidth usage is not mitigated. We
>> recommend that the administrator of the site take action to reduce
>> bandwidth usage. If you feel you are receiving this message erroneously,
>> please contact SourceForge.net Support for assistance. Typically this
>> message is displayed when files that belong in the FRS are uploaded into
>> project web, or someone tried to host some pr0n or w4r3z. SourceForge.net
>> Support is willing and able to assist in resolving the cause of this
>> error.
>
> In a way this is good news (in that OF gets quite a bit of attention)...

Is there a way that we can find what files are responsible for most of
the traffic, how large they are, how many times they have been
downloaded, and the limit that triggers this warning?

> But maybe also something to watch out for as soon as we put binary
> installers for 3.8.0 there.

I'm not opposed to paying for some storage (i.e., google, amazon,
whatever) for distributing the bigger binary files, but I'd like to
know how much data transfer we might have to pay for.  Typically,
these storage systems cost around $.05-.09 per GB per month for the
storage space and another $.08-.22 per GB for transfers out (may vary
by location and total volume).  So for the Windows installer at around
100MB, it would cost something like $.01 per transfer.  Can we afford
that?  I don't know.  It would cost around $100 for 10,000 transfers.
Will 10,000 transfers generate at least $100 in donations to pay the
transfer fees?

The other option is to just distribute from ftp.gnu.org (and the
mirrors).  For comparison, the Windows binary distribution of Emacs is
about 50MB.  I'd guess that's the largest single file that's
distributed from ftp.gnu.org.  If we go this route, I think we should
be asking for permission first.

jwe
Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

Philip Nienhuis
In reply to this post by John Swensen-3
John Swensen-3 wrote
On Wed, Jan 8, 2014 at 4:08 PM, Philip Nienhuis <[hidden email]>wrote:

> Hmm... just got this when I tried to surf to the OF developers page:
>
>
> > SourceForge.net Logo
> >
> > This project has been temporarily blocked for exceeding its bandwidth
> > threshold
<snip>
> In a way this is good news (in that OF gets quite a bit of attention)...

<snip>

A quick look at download stats shows the following for the past 7 days:
OSX binary = ~1000 x 682MB = 682GB
mingw binary = ~1130 x 156MB = 176GB
MSVC binary = ~3000 x 71.5MB = 214GB
Total: 1.072 TB in 7 days
That confirms that Octave does get quite some attention.... 5130 downloads in a week is not bad I'd say :-)

Anyway I find it a bit ridiculous that the Octave project would need to cough up (net that is) for hosting very good programs that get downloaded for free.

This also sheds some light on what to include in a binary installer.
I have one that also includes the required spreadsheet I/O Java class libraries (11 pcs), some 22 MB altogether.
Makes me think that we'd need to push more to get donations for more complete / luxury binary installers.

Philip
Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

Carnë Draug
In reply to this post by Philip Nienhuis
On 8 January 2014 21:08, Philip Nienhuis <[hidden email]> wrote:

> Hmm... just got this when I tried to surf to the OF developers page:
>
>
>> SourceForge.net Logo
>>
>> This project has been temporarily blocked for exceeding its bandwidth
>> threshold
>>
>>
>> Projects and users who host on SourceForge.net are given web space to
>> augment their communities and interact with their users. In cases where a
>> project or user's web space exceeds pre-determined thresholds, this
>> message will be displayed. This block is typically short lived (10-15
>> minutes), unless the cause of the bandwidth usage is not mitigated. We
>> recommend that the administrator of the site take action to reduce
>> bandwidth usage. If you feel you are receiving this message erroneously,
>> please contact SourceForge.net Support for assistance. Typically this
>> message is displayed when files that belong in the FRS are uploaded into
>> project web, or someone tried to host some pr0n or w4r3z. SourceForge.net
>> Support is willing and able to assist in resolving the cause of this
>> error.
>
> In a way this is good news (in that OF gets quite a bit of attention)...
>
> But maybe also something to watch out for as soon as we put binary
> installers for 3.8.0 there.

The packages and binaries are independent of the web pages. This was
caused because someone was crawling the doxygen documentation for the
new release which is 2.3GB in size. If however is doing it is reading
this, please stop. Just download the Octave source, move into the doc
directory and run "make doxyhtml".

The site is back online. Don't forget to reload the webpage, since the
browser can cache the warning.

Carnë
Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

Michael Goffioul
In reply to this post by Philip Nienhuis
On Wed, Jan 8, 2014 at 5:04 PM, Philip Nienhuis <[hidden email]> wrote:
John Swensen-3 wrote
> On Wed, Jan 8, 2014 at 4:08 PM, Philip Nienhuis &lt;

> prnienhuis@.sf

> &gt;wrote:
>
>> Hmm... just got this when I tried to surf to the OF developers page:
>>
>>
>> > SourceForge.net Logo
>> >
>> > This project has been temporarily blocked for exceeding its bandwidth
>> > threshold
> <snip>
>> In a way this is good news (in that OF gets quite a bit of attention)...
> <snip>
> A quick look at download stats shows the following for the past 7 days:
> OSX binary = ~1000 x 682MB = 682GB
> mingw binary = ~1130 x 156MB = 176GB
> MSVC binary = ~3000 x 71.5MB = 214GB
> Total: 1.072 TB in 7 days

That confirms that Octave does get quite some attention.... 5130 downloads
in a week is not bad I'd say :-)

Anyway I find it a bit ridiculous that the Octave project would need to
cough up (net that is) for hosting very good programs that get downloaded
for free.

This also sheds some light on what to include in a binary installer.
I have one that also includes the required spreadsheet I/O Java class
libraries (11 pcs), some 22 MB altogether.
Makes me think that we'd need to push more to get donations for more
complete / luxury binary installers.

Actually, if you look at the trends for the Windows binaries (3.6.4) over the last year, we're in the decreasing phase. And download rate has been much higher, for instance last fall. OTOH what has changed recently is the addition of the OSX binary (3.8.0), which is quite large (about 10x the size of the MSVC installer). So maybe SF is choking on distributing such large file.

Michael.

Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

c.-2
In reply to this post by Carnë Draug

On 8 Jan 2014, at 23:08, Carnë Draug <[hidden email]> wrote:

> On 8 January 2014 21:08, Philip Nienhuis <[hidden email]> wrote:
>> Hmm... just got this when I tried to surf to the OF developers page:
>>
>>
>>> SourceForge.net Logo
>>>
>>> This project has been temporarily blocked for exceeding its bandwidth
>>> threshold
>>>
>>>
>>> Projects and users who host on SourceForge.net are given web space to
>>> augment their communities and interact with their users. In cases where a
>>> project or user's web space exceeds pre-determined thresholds, this
>>> message will be displayed. This block is typically short lived (10-15
>>> minutes), unless the cause of the bandwidth usage is not mitigated. We
>>> recommend that the administrator of the site take action to reduce
>>> bandwidth usage. If you feel you are receiving this message erroneously,
>>> please contact SourceForge.net Support for assistance. Typically this
>>> message is displayed when files that belong in the FRS are uploaded into
>>> project web, or someone tried to host some pr0n or w4r3z. SourceForge.net
>>> Support is willing and able to assist in resolving the cause of this
>>> error.
>>
>> In a way this is good news (in that OF gets quite a bit of attention)...
>>
>> But maybe also something to watch out for as soon as we put binary
>> installers for 3.8.0 there.
>
> The packages and binaries are independent of the web pages. This was
> caused because someone was crawling the doxygen documentation for the
> new release which is 2.3GB in size. If however is doing it is reading
> this, please stop. Just download the Octave source, move into the doc
> directory and run "make doxyhtml".
>
> The site is back online. Don't forget to reload the webpage, since the
> browser can cache the warning.
>
> Carnë

Carnë,
Maybe we should also distribute the doxygen docs in html and pdf to be read offline?
c.


Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

c.-2
In reply to this post by Michael Goffioul

On 8 Jan 2014, at 23:09, Michael Goffioul <[hidden email]> wrote:

> Actually, if you look at the trends for the Windows binaries (3.6.4) over the last year, we're in the decreasing phase. And download rate has been much higher, for instance last fall. OTOH what has changed recently is the addition of the OSX binary (3.8.0), which is quite large (about 10x the size of the MSVC installer). So maybe SF is choking on distributing such large file.

No fortunately that is not the case, as Carnë explained the bandwidth limit is on the web pages only (which do not go through the mirroring system), not on the distribution files.
Anyway reducing the size of the OSX binay is on my todo list, but that will cost me time while the sourceforge.net bandwidth doe not cost me anything so the priority is currently low ...

> Michael.
c.
Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

Michael Goffioul
On Wed, Jan 8, 2014 at 5:46 PM, c. <[hidden email]> wrote:

On 8 Jan 2014, at 23:09, Michael Goffioul <[hidden email]> wrote:

> Actually, if you look at the trends for the Windows binaries (3.6.4) over the last year, we're in the decreasing phase. And download rate has been much higher, for instance last fall. OTOH what has changed recently is the addition of the OSX binary (3.8.0), which is quite large (about 10x the size of the MSVC installer). So maybe SF is choking on distributing such large file.

No fortunately that is not the case, as Carnë explained the bandwidth limit is on the web pages only (which do not go through the mirroring system), not on the distribution files.
Anyway reducing the size of the OSX binay is on my todo list, but that will cost me time while the sourceforge.net bandwidth doe not cost me anything so the priority is currently low ...

Fair enough. I also found it strange that it only choked now, while the download rate is much lower than a couple of months earlier.

Michael.

Reply | Threaded
Open this post in threaded view
|

Re: "octave.sf.net bandwidth exceeded"

Carnë Draug
In reply to this post by c.-2


On 8 Jan 2014 22:43, "c." <[hidden email]> wrote:
>
>
> On 8 Jan 2014, at 23:08, Carnë Draug <[hidden email]> wrote:
>
> > On 8 January 2014 21:08, Philip Nienhuis <[hidden email]> wrote:
> >> Hmm... just got this when I tried to surf to the OF developers page:
> >>
> >>
> >>> SourceForge.net Logo
> >>>
> >>> This project has been temporarily blocked for exceeding its bandwidth
> >>> threshold
> >>>
> >>>
> >>> Projects and users who host on SourceForge.net are given web space to
> >>> augment their communities and interact with their users. In cases where a
> >>> project or user's web space exceeds pre-determined thresholds, this
> >>> message will be displayed. This block is typically short lived (10-15
> >>> minutes), unless the cause of the bandwidth usage is not mitigated. We
> >>> recommend that the administrator of the site take action to reduce
> >>> bandwidth usage. If you feel you are receiving this message erroneously,
> >>> please contact SourceForge.net Support for assistance. Typically this
> >>> message is displayed when files that belong in the FRS are uploaded into
> >>> project web, or someone tried to host some pr0n or w4r3z. SourceForge.net
> >>> Support is willing and able to assist in resolving the cause of this
> >>> error.
> >>
> >> In a way this is good news (in that OF gets quite a bit of attention)...
> >>
> >> But maybe also something to watch out for as soon as we put binary
> >> installers for 3.8.0 there.
> >
> > The packages and binaries are independent of the web pages. This was
> > caused because someone was crawling the doxygen documentation for the
> > new release which is 2.3GB in size. If however is doing it is reading
> > this, please stop. Just download the Octave source, move into the doc
> > directory and run "make doxyhtml".
> >
> > The site is back online. Don't forget to reload the webpage, since the
> > browser can cache the warning.
> >
> > Carnë
>
> Carnë,
> Maybe we should also distribute the doxygen docs in html and pdf to be read offline?

I doubt that whoever decided to do this looked much for such solution or would have found it easier to build it themselves. Of so, having a PDF somewhere wouldn't have prevented it. Plus, the interface is not really comparable. The HTML are 2.3GB which would be problematic to distribute.

If this happens again, the IP address can be blacklisted which hopefully will lead to find this thread with right way to get it.

Carnë

Reply | Threaded
Open this post in threaded view
|

Distributing large files (was: Re: "octave.sf.net bandwidth exceeded")

John W. Eaton
Administrator
In reply to this post by John W. Eaton
On 01/08/2014 04:36 PM, John W. Eaton wrote:

> The other option is to just distribute from ftp.gnu.org (and the
> mirrors).  For comparison, the Windows binary distribution of Emacs is
> about 50MB.  I'd guess that's the largest single file that's
> distributed from ftp.gnu.org.  If we go this route, I think we should
> be asking for permission first.

I was wrong.  The gcc-4.8.2.tar.bz2 file is around 83MB and the tar.gz
file is close to 100MB...

jwe