.Org needs a new hosting company

[captain]

I need to get the site out of my house... Anyone out there in Busaland connected... We will need about 500 / 1000 gig of bandwidth on a linux server... let me know we are getting close to crisis mode here...

CAp

 

[SLEEPERBUSA]

You check with Kevin?
He's got the other ORGs doin good.

 

[monsterspeedfreak]

Cap...just sell the site to me

 

[justintime2]

This doesn't sound good.

 

[jwcfbd]

What's up Captain?

 

[captain]

whaddya offer for it...

cap

 

[monsterspeedfreak]

PM me the stats, what yer sponsors cover, and yer asking price if yer serious Cap.
Have ya kinda had enough......they are a boatload a work aren't they!!!

 

[captain]

We are getting killed by folks surfing the site... Thats a good thing but we have so many that we are saturating both T1's here at the house. The web server has about 200 continious connections all day and they are coming from all over the world. There are links to pics and post from other countries and those folks are just looking at the pics and not offering anything back to the site just eating bandwidth. This board moved 80 gig of data in November alone and Dec is right on track for that. I have the site spread out over 4 servers and 3 circuits and we are getting bigger and bigger so I need to do some pruning..... I changed the site today to require members to log in so that we can minimize any extra traffic from those outside the English reading world. I figure if you can't read english your going to have a hard time logging in..... The site as a whole, message board, vids and gallery put out just under 20 million hits a month with a total around 500 gig of data transfer...... Look at the gallery sometime... 863829 times the gallery has been accessed and there is just over 19,000 pics... You do the math....... I pay a pretty healthy price for the two T1's at the house not to mention the servers and firewalls. Watchguard the best firewall money can buy......  Anyway I love that the site is spreading out and reaching all over the world and I like control of having it here at my fingertips however Im as frustrated as everyone else when the site is slow... We used to have 50 to 80 members surfing at the same time and now we are lucky to see 35... I want this place to be the best all around not just a message board.... Outside Hosting is expensive when you talking about the traffic and server loads that we put out.

Here is the coutries served in Nov.
# Hits Files KBytes Country

1 4344774 40.37% 3476216 38.99% 34855133 41.17% Network
2 3287004 30.54% 2763848 31.00% 26769221 31.62% US Commercial
3 1537940 14.29% 1296217 14.54% 11444101 13.52% Unresolved/Unknown
4 428335 3.98% 365948 4.10% 2911457 3.44% US Military
5 144603 1.34% 99943 1.12% 1126864 1.33% US Government
6 137758 1.28% 125325 1.41% 1049910 1.24% Australia
7 111056 1.03% 96746 1.09% 859624 1.02% Canada
8 71904 0.67% 60886 0.68% 478861 0.57% Italy
9 61485 0.57% 50669 0.57% 495096 0.58% United States
10 43996 0.41% 39290 0.44% 434112 0.51% US Educational
11 43416 0.40% 38121 0.43% 392267 0.46% United Kingdom
12 41446 0.39% 38287 0.43% 348469 0.41% Netherlands
13 41230 0.38% 36631 0.41% 314695 0.37% Non-Profit Organization
14 40310 0.37% 35111 0.39% 227651 0.27% Greece
15 36344 0.34% 34917 0.39% 286674 0.34% Singapore
16 30661 0.28% 30373 0.34% 156532 0.18% Brazil
17 28909 0.27% 24836 0.28% 226123 0.27% Japan
18 27296 0.25% 26540 0.30% 182640 0.22% Finland
19 26520 0.25% 26066 0.29% 128107 0.15% Poland
20 25318 0.24% 24583 0.28% 160685 0.19% France
21 21900 0.20% 20460 0.23% 122225 0.14% Denmark
22 20899 0.19% 18289 0.21% 185874 0.22% Sweden
23 15782 0.15% 15175 0.17% 104189 0.12% New Zealand (Aotearoa)
24 15503 0.14% 14913 0.17% 145916 0.17% Belgium
25 14080 0.13% 13015 0.15% 98218 0.12% Thailand
26 13966 0.13% 13703 0.15% 62882 0.07% Czech Republic
27 12960 0.12% 12153 0.14% 107604 0.13% Germany
28 12647 0.12% 11544 0.13% 70346 0.08% Turkey
29 11924 0.11% 11012 0.12% 62843 0.07% Hungary
30 11867 0.11% 11703 0.13% 68991 0.08% India
31 9102 0.08% 7706 0.09% 112171 0.13% Portugal
32 7028 0.07% 6135 0.07% 57264 0.07% Qatar
33 6605 0.06% 6207 0.07% 45045 0.05% Old style Arpanet (arpa)
34 6021 0.06% 5641 0.06% 47916 0.06% Switzerland
35 5917 0.05% 5780 0.06% 41987 0.05% Mexico
36 4937 0.05% 4860 0.05% 36358 0.04% Romania
37 4783 0.04% 4501 0.05% 36790 0.04% Austria
38 4628 0.04% 4256 0.05% 36079 0.04% Norway
39 4361 0.04% 4320 0.05% 31149 0.04% Estonia
40 3823 0.04% 3769 0.04% 19886 0.02% Morocco
41 3531 0.03% 2489 0.03% 27587 0.03% Bosnia and Herzegovina
42 3519 0.03% 3388 0.04% 21698 0.03% South Africa
43 3462 0.03% 2910 0.03% 44298 0.05% Taiwan
44 2706 0.03% 2601 0.03% 15648 0.02% Spain
45 2291 0.02% 2235 0.03% 9474 0.01% Bulgaria
46 2241 0.02% 2136 0.02% 14635 0.02% Cyprus
47 2239 0.02% 2196 0.02% 14439 0.02% Malaysia
48 2014 0.02% 1482 0.02% 9001 0.01% Namibia
49 1915 0.02% 1718 0.02% 13686 0.02% Aruba
50 1743 0.02% 1690 0.02% 15892 0.02% Philippines
51 1704 0.02% 1680 0.02% 7570 0.01% Croatia (Hrvatska)
52 1481 0.01% 1472 0.02% 10159 0.01% Israel
53 1287 0.01% 1276 0.01% 9041 0.01% Argentina
54 1276 0.01% 1234 0.01% 10883 0.01% Hong Kong
55 1186 0.01% 1177 0.01% 8749 0.01% Slovak Republic
56 1156 0.01% 1135 0.01% 5526 0.01% Yugoslavia
57 940 0.01% 747 0.01% 4435 0.01% Guatemala
58 872 0.01% 863 0.01% 5005 0.01% Saudi Arabia
59 692 0.01% 669 0.01% 6722 0.01% Iceland
60 691 0.01% 643 0.01% 5148 0.01% Lithuania
61 674 0.01% 657 0.01% 3045 0.00% Dominican Republic
62 571 0.01% 565 0.01% 3388 0.00% Peru
63 564 0.01% 564 0.01% 2528 0.00% Venezuela
64 529 0.00% 527 0.01% 4211 0.00% Luxembourg
65 511 0.00% 511 0.01% 2486 0.00% Pakistan
66 431 0.00% 430 0.00% 5958 0.01% Ireland
67 411 0.00% 371 0.00% 3704 0.00% Russian Federation
68 401 0.00% 390 0.00% 3118 0.00% Uruguay
69 323 0.00% 323 0.00% 2822 0.00% Malta
70 297 0.00% 296 0.00% 720 0.00% Latvia
71 286 0.00% 286 0.00% 3481 0.00% Mauritius
72 264 0.00% 261 0.00% 1919 0.00% Indonesia
73 254 0.00% 250 0.00% 2617 0.00% Colombia
74 249 0.00% 249 0.00% 423 0.00% Nepal
75 175 0.00% 175 0.00% 1195 0.00% Cayman Islands
76 169 0.00% 155 0.00% 1246 0.00% Netherlands Antilles
77 149 0.00% 141 0.00% 2949 0.00% Slovenia
78 143 0.00% 142 0.00% 784 0.00% Chile
79 109 0.00% 109 0.00% 63 0.00% United Arab Emirates
80 66 0.00% 63 0.00% 463 0.00% New Caledonia
81 53 0.00% 53 0.00% 218 0.00% Tuvalu
82 45 0.00% 45 0.00% 473 0.00% Costa Rica
83 40 0.00% 33 0.00% 797 0.00% Niue
84 38 0.00% 37 0.00% 181 0.00% Egypt
85 22 0.00% 20 0.00% 133 0.00% Trinidad and Tobago
86 14 0.00% 14 0.00% 171 0.00% Lebanon
87 10 0.00% 10 0.00% 94 0.00% Gibraltar
88 10 0.00% 5 0.00% 12 0.00% Seychelles
89 9 0.00% 9 0.00% 154 0.00% Cote D'Ivoire (Ivory Coast)
90 9 0.00% 6 0.00% 59 0.00% Syria
91 9 0.00% 6 0.00% 396 0.00% Ukraine
92 4 0.00% 4 0.00% 365 0.00% Cocos (Keeling) Islands
93 2 0.00% 2 0.00% 161 0.00% Iran
94 2 0.00% 2 0.00% 54 0.00% Kenya
95 2 0.00% 2 0.00% 20 0.00% Oman
96 2 0.00% 2 0.00% 54 0.00% Puerto Rico
97 1 0.00% 1 0.00% 1 0.00% International (int)
98 1 0.00% 1 0.00% 151 0.00% Albania
99 1 0.00% 1 0.00% 10 0.00% Jordan
100 1 0.00% 1 0.00% 10 0.00% Kuwait


CAp

 

[WWJD]

Ain't I.T. a blast? ? ?

If this were my contract, I would evaluate the use fo the bandwidth: is it pictures, videos, graphics whatever and figure out how to scale that back first and tame that beast, perhaps slap some caps on pic size, check out the architecture of the hosting software: is IKONboard efficient at savign bandwidth or a hog by design? Would the site benefit by utilizing multiple host locations: one for pics, one for vids, one for boards etc,. splitting the bandwidth.

I can guess you are paying a couple hundred a month for your T1 setup. Could you squeeze the site down into one of those cheapo hosting sites for $14.95?

Maybe segment across some free hosts services for the home page and messages boards.

Good luck with whatever direction you go, cap.

 

[Revlis]

I thought we had cut out the damn videos and Gallary Months and months ago...

This doesn't make any sense... I Never see more than 20 or so Members logged in, and guests maybe another 30 or so? Hell I dunno where the Bandwidth is going... Doesn't add up...

Videos gotta go, Gallery Can go. IMHO....

Once again though... I'm Baffled... No Donations accepted, still wrestling with the finances, catch 22...

 

[captain]

Ummmm lets just say the vids aren't on my servers anymore and leave that one alone....... (thanks Frank) The Gallery is on a T1 along with a few other sites that hosted on one of the proliant servers. The board is the big deal right now...

WWJD we just went through the avatars and resized them... There is already a limit on the upload pic size and we are spanning 3 different circuits two T1's and a T3 for traffic. Ikonboard is perfect for this application.. mysql on linux with hackable code open to the public... No cost and tons of other folks are using it... I have a couple of plans to make the changes that we need. we should be able to figure something out soon.. All this on top of the fact that I am scheduled to go to the Phillipines for roughly a month soon.

CAp

 

[WWJD]

so videos/gallery are not going thru your circuits? just this board does 500 gig a month? is it possible people are running spiders or bots against it pulling data? Something seems off.

Oh the other hand, growth is good and your advertisers should be slappin out the dough for extra hits!

I'm no webhosting guru but I could hook you up with one that will work for Pizzas. If this is a profitable venture, you might invest in some consultants help. Otherwise, cut if back a lil [like you are doing] and call it good. I think most of us would adapt to where ever you need to go with this.

I bookmarked the forum directly. Maybe many others go thru the home page and that is large? How about caching and reads of the forum? I use "Backspace" key a lot instead of 'Back' in hopes that I don't RELOAD the previous page - and it speeds things up a bunch while surfing.

Do you want me to look for some offsite hosts? Again, I'm I.T. but not a hosting guru, but know people who can advise

 

[thrasherfox]

I don’t get it either Cap.

A while back you were complaining because the board was slow, now it seems like it is getting a lot of traffic. and you said the Pictures are no longer being hosted on your server, so links to the pics no matter where they are coming from should not be affecting your bandwidth if they are not on your servers.

So from what you have described of your system, all the pics are gone, you should have more storage space available, I don’t see how your bandwidth could be increasing when graphic files are not being sucked out of your system.
The removal of graphics and video from your system should have increased your available bandwidth by at least 70% I would guess (making assumptions based on general normal data transfers scenarios)


All the Avitars were made smaller..

I am trying to make sense of this so I can offer some help. This is what I do. I am an IT person, I deal with networks, bandwidth issues all the time. But I cant offer any help unless 2 and 2 = 4 and right now the equation seems to look more like 2+a=b

I am missing too many variables to come up with a solution



<!--EDIT|thrasherfox
Reason for Edit: None given...|1104275055 -->

 

[Hackabusa]

Just out of curiosity and for me to try to put this im perspective, how big is the MySQL DB?

 

[monsterspeedfreak]

It just sounds like ya might be fed up with the whole thing Cap.....true??

 

[WWJD]

thrasher, at 7pm CST, I'm looking at it and the list of threads, like under RANDOM TOPICs, kinda locks up, paints a thread subject one line at a time. MOST of it is text, with some tiny repetative icons that are already cached. I've got my browser [Netscape latest - which seems to be the fastest for me on this board home and at work, normally I'd use Firebird] settings for best performance so I can't figure why it would list so slowly like that. The threads themselves do okay. that's what I have noticed the most slowness in: thread listing. Like it's chewing thru a database looking for the next 'sorted by last reply' subject line. It used to work a lot quicker - you know, like other boards - where it just goes 'Blip' and all shows up at once.

Maybe that's it! Maybe it's the sort part.... hang on I gotta try something....

 

[PACIFICBUSA]

I haven't really come into any probs accessing it from here...and I'm on a secured (I think) server.

BTW...I didn't see Guam on the list...

 

[thrasherfox]

Doug -

I ran two trace routes from my PC

Hayabusa.org seems to have an IP address of 64.250.210.215
The trace route timed out on 64.250.192.2

Does that make sense to you? It seems it should have went all the way through to the 64.250.210.215 or at least to a router on the 64.250.210.xxx net

I also did a ping to the bradcast IP.. here are the results for that
Pinging 64.250.210.255 with 32 bytes of data:

Reply from 64.250.210.255: bytes=32 time=65ms TTL=51
Reply from 64.250.210.255: bytes=32 time=65ms TTL=51
Reply from 64.250.210.255: bytes=32 time=64ms TTL=51
Reply from 64.250.210.255: bytes=32 time=64ms TTL=51

Ping statistics for 64.250.210.255:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 64ms, Maximum = 65ms, Average = 64ms


Here are the trace route results


TRACERT WWW.HAYABUSA.ORG

Tracing route to hayabusa.org [64.250.210.215]
over a maximum of 30 hops:

1 16 ms 14 ms 15 ms adsl-67-125-96-158.dsl.frs2ca.pacbell.net [67.12
5.96.158]
2 15 ms 14 ms 14 ms dist1-vlan50.frsn02.pbi.net [66.122.172.2]
3 14 ms 14 ms 14 ms bb2.g1-3-0.frsn02.pbi.net [63.202.63.149]
4 15 ms 14 ms 16 ms bb1-p4-0.spfdmo.sbcglobal.net [151.164.40.197]
5 19 ms 19 ms 21 ms bb1-p3-1.crscca.sbcglobal.net [151.164.188.114]

6 19 ms 20 ms 20 ms ex2-p14-0.eqsjca.sbcglobal.net [151.164.242.230]

7 20 ms 19 ms 19 ms ex1-p9-0.eqsjca.sbcglobal.net [151.164.191.201]

8 19 ms 20 ms 20 ms sl-st20-sj-0-0.sprintlink.net [144.223.242.81]
9 21 ms 20 ms 20 ms sl-bb25-sj-10-0.sprintlink.net [144.232.20.62]
10 31 ms 32 ms 33 ms sl-bb23-ana-6-0.sprintlink.net [144.232.20.158]

11 56 ms 56 ms 57 ms sl-bb23-fw-9-1.sprintlink.net [144.232.18.61]
12 56 ms 56 ms 56 ms sl-bb21-fw-13-0.sprintlink.net [144.232.11.245]

13 57 ms 56 ms 56 ms sl-gw35-fw-8-0.sprintlink.net [144.232.11.254]
14 68 ms 67 ms 67 ms sl-pione3-1-0.sprintlink.net [160.81.61.130]
15 68 ms 68 ms 67 ms rback-fas6-0.kgfs.pldi.net [64.250.192.2]
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 * * * Request timed out.
26 * * * Request timed out.
27 * ^C



Tracing route to 64-250-210-215.hayabusa.org [64.250.210.215]
over a maximum of 30 hops:

1 14 ms 14 ms 14 ms adsl-67-125-96-158.dsl.frs2ca.pacbell.net [67.12
5.96.158]
2 16 ms 15 ms 14 ms dist1-vlan50.frsn02.pbi.net [66.122.172.2]
3 16 ms 14 ms 13 ms bb2.g1-3-0.frsn02.pbi.net [63.202.63.149]
4 15 ms 15 ms 16 ms bb1-p4-0.spfdmo.sbcglobal.net [151.164.40.197]
5 18 ms 19 ms 20 ms bb1-p3-1.crscca.sbcglobal.net [151.164.188.114]

6 36 ms 19 ms 20 ms ex2-p14-0.eqsjca.sbcglobal.net [151.164.242.230]

7 19 ms 20 ms 19 ms ex1-p9-0.eqsjca.sbcglobal.net [151.164.191.201]

8 21 ms 18 ms 20 ms sl-st20-sj-0-0.sprintlink.net [144.223.242.81]
9 21 ms 21 ms 21 ms sl-bb25-sj-10-0.sprintlink.net [144.232.20.62]
10 32 ms 32 ms 33 ms sl-bb23-ana-6-0.sprintlink.net [144.232.20.158]

11 56 ms 56 ms 56 ms sl-bb23-fw-9-1.sprintlink.net [144.232.18.61]
12 55 ms 56 ms 56 ms sl-bb21-fw-13-0.sprintlink.net [144.232.11.245]

13 57 ms 55 ms 55 ms sl-gw35-fw-8-0.sprintlink.net [144.232.11.254]
14 68 ms 68 ms 67 ms sl-pione3-1-0.sprintlink.net [160.81.61.130]
15 69 ms 68 ms 69 ms rback-fas6-0.kgfs.pldi.net [64.250.192.2]
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23

 

[captain]

OK so let me put everyones minds to rest...

The SQL database is only 300 meg however there are over 2.5 gig of images that have been uploaded from the members using the board. These would be the images in the post on the message board. They are a part of the message board...... So in all actuality if you took the entire message board and put it in a directory it would be roughly 3 gig.......... The gallery and the videos are on a different circuits so they do not effect the preformance of the board.... Thrasher you will not get a reply to a ping or traceroute I block all UDP and ICMP packets.... Were invisible to the world and thats the way that I like it..... I have public routable IPs and they are all working fine... The firewall distributes traffic accordingly. You are all correct in one thing VOLUME!!!! I have real time tools as part of the firewall and I can sit here and watch Japan and Korea and Germany and many many other countries hammering away at the site. WWJD thats what creates the slowness is the sheer number of folks linked to the images on the message board. I look at referrer reports and there are hundreds of links to the site from other forums every week. Shutting the site off and making folks log in will help alot. What you all can see is the 30 - 40 logged in members looking through the forum but you cannot see the 150 others logged in as guest...... This aint my first IT rodeo...... Thrasher its not about storage space, I have 20 hot swappable SCSI drives in the server we have loads of storage... Its the pipe from the servers to the rest of the world that are being bogged down by folks from all over the world linking to the site... you put 100 folks all pulling down a topic with a 300k picture and it begins to add up..

cap

 

[captain]

One more thing.... I sit here on a 100mb ethernet connection directly to the server and it dishes the site out faster than you can blink an eye so its not the server or the software.