*Server Is Under High Stress*

[Edward Campbell]

Site is moderately slow right now (nearly 0500 Hrs in Ottawa) (I'm a very early riser!); it is 'normally' quite quick in the early AM. I see this: 120 Guests, 10 Users (0 Buddies, 21 Spiders, 1 Hidden)

 

[Dissident]

Site is slow here from Richmond.

 

[kratz]

Pages are hanging for minutes to load, for the past few hours.

 

[tango22a]

Sorry Mike:

I've seen faster snail races!


tango22a

 

[Mike Bobbitt]

Well, I think we may have reached the point where we need to look at improving our lot in life.

For those of you who don't care about the technical details, skip ahead to my next reply.

For those who like to sink their teeth into the nuts and bolts, read on.

Right now, we are on Eastlink's Business Static 10 program. That gives us [theoretical] 10mbps download and 1mbps upload speeds. Normally that would work out just fine, but in our case, we do a heck of a lot more "uploading" than "downloading". (That is, every page requested is an upload, but the request itself is a download.) So while we have a 10:1 ratio of down to up available, our actual usage is probably more like 1:10, with a lot more data going out to the 'net than coming in from it.

What this means in practical terms is that our upload pipe is constantly full, and packets are often held up waiting to get "out". Sometimes they are held up so long that the user's system on the other end gives up waiting and times out.

I have contacted Eastlink and asked about redistributing the 10:1 a bit more equably for our needs. This is something they technically can do, but they won't do it. They have specific plans and that's all they'll provide... I can understand that, most businesses won't do a "one off" like that.

Unfortunately we need a business plan because the residential plans block certain ports (TCP 25/SMTP for example, which means we wouldn't be able to run a mail server). We also need a "static" plan because using dynamic IPs mean that we'd be frequently "moving" on the 'net. While Dynamic DNS solutions exist out there, I've never had much luck with them... they usually end up with a complex config and some downtime when a new IP is leased. That means we are limited to the "Business Static" options.

After looking at the problem from a few different angles, I've narrowed it down to a couple of different possibilities:

• Bump up to the next Business Static plan with Eastlink. This incurs an extra $45+ tax per month, taking our ISP bill from $118.65 to $169.50 (a diff of $50.85 monthly). This approach takes us from 10:1 to 10:1.5. Doesn't seem like much, but it adds 50% to our currently overburdened upload bandwidth. The setup for this is a single phone call, and the change would be nearly immediate.
  
• Move Army.ca et al. to a service provider. A while back someone suggested Slicehost and I've been looking into them on and off. A 2Gb slice with them would be a step down from the hardware we currently run on... Let's look at it piece by piece. The 2Gb slice would be 1/2 the RAM we currently use, 2Gb vs. 4Gb. We typically do not max out our RAM use, but we do come close. Right now we're at 96.11% memory used, though this can be a bit deceiving on Linux... RAM is only freed up when it's needed. I expect we could get by with 2Gb with some adjustments. 80Gb of disk space could be a bit of a squeeze. We currently use about 100Gb but aren't "careful" with it so we could probably trim down without too much difference. Total monthly bandwidth is 800Gb and we rarely top out 200Gb, so lots of room to grow there. This is also a shared virtual server, so while we'd be getting our "dedicated server," it would be just part of a larger, shared physical piece of hardware. That might have some performance implications.
  
  The good news is that the actual throughput is very high. For the 2Gb slice it would be 40mbps upload and effectively unlimited download. Now, can slicehost actually deliver those speeds? Doubtful, but I'm sure it would be better than the 10:1 we're getting now.
  
  Now here's the rub: setup for this option would be significant. It would require all current Army.ca pages, databases, services, etc. to be migrated to a slicehost server. Not only that, but some things would need to be reconfigured to take the reduced disk and memory into account. All in all this is a significant task and given my "personal" bandwidth it could take weeks or months to complete.
  
  In terms of price, it's $130USD/month, or roughly the same as the Eastlink option above, however this is another consideration. During the cutover period, we'd have to pay for our current connection and the Slicehost service. That would effectively double our Internet bill, which is not something we would want to do for very long!

So the summary of options is, we can get a quick 50% boost to our biggest pain point, or we can move everything to a very fast pipe on lower grade hardware. Both options are roughly equal in cost. Having spent some time considering the options, I am in favour of the first.

Bumping up our upload throughput to 1.5mbps is a quick and easy win, and if it doesn't have the desired effect, we can simply step back to our current plan or kick off something with Slicehost in parallel.

 

[Mike Bobbitt]

OK, for those of you who skipped ahead, here's where we stand. For an extra $50.85/month, we can improve our existing Internet connection by up to 50%.

Annually, that works out to $610.20, or roughly 20 new subscribers in order to sustain the service indefinitely. Many of our heavy users are already subscribers - thanks for giving us the funds to get where we are today! To put things into perspective, every post in the last 2 pages of this thread has been from a Subscriber!

I'll use this as an opportunity to urge others to consider subscribing. In addition to helping the site in a very quantifiable way, you also get a site challenge coin, shirt and sticker. Furthermore, subscribers get access to a Facebook-esque status field, all site ads are removed, and they have additional permissions and rights not available to normal users. For a full list, please see the details page on becoming a subscriber.

If we can round up 20 new subscribers, I'll bump us up to the next tier with our ISP and hopefully the log jam will break.

Thanks in advance for your support!


Cheers
Mike

 

[tango22a]

Mike:

Don't get me wrong, I'm not challenging your math, but it appears to me that to earn the required amount you would require 20 + 1 new subscribers.

Either that or you raise the tariff for new members to say $40 to $50 per year effective 1JAN10. If necessary I am willing to send another cheque to top up my dues for two years.


tango22a

 

[tango22a]

Mike:

Sorry if I started a S**tstorm by advocating an increase in dues. IMHO this site is worth a H*ll of a lot more than $30 per year. I know that you are attempting to provide max service at the minimum cost, but either we get more members or the site continues the way it is. Forgive me but I don't consider this a viable option.

If a member can't afford the increase honestly, IMHO he/she/it should pay what they can.

IF somebody has a problem with this they can PM me and we'll keep the flamefest from wasting bandwith.

Thanks,


tango22a

 

[mellian]

I will subscribe as soon as I possibly can financially (along with another website presently active on).

 

[Mike Bobbitt]

lol, thanks tango... caught me on that one. Your math is solid and the price isn't going up.

Clearly I needed to take my socks off for that last post.

mellian (and others), it's not meant to be a guilt trip. There are many valid reasons why subscribing is impractical for people. However for those who enjoy the site and have the means, it does help quite a bit. I'll never judge those who don't subscribe... like I said, it's just not always practical.

 

[Mike Bobbitt]

Sorry if I started a S**tstorm by advocating an increase in dues. IMHO this site is worth a H*ll of a lot more than $30 per year.

Not at all, discussion like this is always good. I'm trying hard to keep subscriptions accessible and attractive to users here... part of that is keeping the price down and the other part is offering a decent service to subscribers, without making regular users feel like they're second class citizens.

Frankly, I should be focusing on advertising to support the site. As it stands we have only a few advertisers. CPGear is our largest, but they sponsor the monthly contests, so the site doesn't bank anything as a result. The remaining advertisers come and go on a regular basis but don't add much to the coffers.

Before the alarm bells go off, I will not jam the site full of ads... what you see now is all there will be for the foreseeable future. However we could benefit from more advertisers using the existing space.

I've been working on a plan to entice a few larger organizations to jump in with some advertising, but as I mentioned, it's slow going so far. Ultimately, this kind of sponsorship would be the way to go as it takes the burden of improving the site off the users.

We have an extremely focused, well informed audience here that is a marketing boon for certain organizations... problem is they don't know we exist or what the options are, but I'm working to change that.

 

[Bruce Monkhouse]

Mike, money order for $200 on the way this afternoon.
Upon receiving send me 2 xxl running shirts, an xl golf shirt [2xl if you find one] and a grey xxl hoodie and put the rest towards continuing my subscription.

[I will send you a PM about this also but the fact is I'm not afraid to 'guilt' people]

 

[Mike Bobbitt]

Thanks Bruce!

 

[GAP]

Hey....hoodies?  How much are they.....(I need that 2XL by the way....)?

 

[Mike Bobbitt]

You bet I do! I have 2XL in black with the Army.ca logo and in grey with the Milnet.ca logo for $40. There are also a few other odds and sods in "the store":

http://milnet.ca/store/

 

[GAP]

You bet I do! I have 2XL in black with the Army.ca logo and in grey with the Milnet.ca logo for $40. There are also a few other odds and sods in "the store":

http://milnet.ca/store/

Just sent a confirming email with address details and also sent an email M.O. for the $$

Thanks
Gord

ps: order page is great, except there is no option to order without paypal or CC upfront....I generally pay by Email M.O. so I have to do it this way....thanks again.

 

[1feral1]

Jut after 0500 here on Sat, 07 Nov. Although I am not at home (in Brisbane at a a mates place - gunshow today ) the site has been performing very poor, either timing out, or taking a very long time to open just to make a post. Currently took about 7 minutes to get on to this topic.

Been up since 0300, I hate sleeping on a fart infested old couch and not having my own bed, but gunshows come only 2 or 3 times a year.

Regards,

OWDU

 

[George Wallace]

I'm finding site speed to be sporadic, with pages sometimes only half loading.

 

[1feral1]

From down here, again the performance has been rather disappointing. Things half load, or are super slow, or don't load at all, and keep timing out.

Frustrating to say the least, and I hope this ongoing problem can be overcome in the near future.

OWDU

 

[Antoine]

BTW, at the University of British Columbia, the site speed is slow for the last couple of months. I am wondering if there is a filter or a priority level on their web network (if possible) but I can listen to CBC on-line without problem.