• Welcome to Peterborough Wargames Club. Please login or sign up.
 
January 19, 2021, 09:34:19 pm

News:

Note for prospective members:
* Please choose a screen name that reflects your real name
* Please drop an email note to the info@ address introducing yourself.


Server Updates

Started by Mike Whitaker, February 15, 2012, 04:25:32 pm

Previous topic - Next topic

Chris Davies

The boy's toys: SAGA Anglo-Saxons;  Bolt Action Soviets 1,000 points;Mordheim Goblin/Vampire gangs; Necromunda Bax's Stabbers; Uncharted Seas Shroud Mages; Dystopian Wars Russian starter fleet; Wings of Wibble; assorted plastic spacemen, Mahdists,French Foreign Legion, Colonial Egyptians; AoM undead, Vanguard undead and goblins.

Mike Whitaker

Just as a heads up - the server's boot SSD died this lunchtime - currently running on an out of date backup drive which will be updated from the Amazon S3 backup shortly. May be a little more downtime to do that later this evening.

Your grumpy sysadmin.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Due to a slight cockup on my part, I've had to restore from a backup taken yesterday at 3pm. Any forum posts since then will have vanished.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Carl Fisher

Games Played (Armies): WAB + WaC (IER Roman, Early Saxons and variants, Normans), WFB (Empire), 40K (Spacemarines), Uncharted Seas (Empire, Shroud Mages), WAB English Civil War (English!), Operation Squad (British) SAGA (Normans, Jomsvikings), Mordheim, Wings of War/Glory WWI, Dreadball, Chain of Command (British), Frostgrave, Sails of Glory and many others!!

Carl Fisher

Games Played (Armies): WAB + WaC (IER Roman, Early Saxons and variants, Normans), WFB (Empire), 40K (Spacemarines), Uncharted Seas (Empire, Shroud Mages), WAB English Civil War (English!), Operation Squad (British) SAGA (Normans, Jomsvikings), Mordheim, Wings of War/Glory WWI, Dreadball, Chain of Command (British), Frostgrave, Sails of Glory and many others!!

Mike Whitaker

Apologies for server downtime this (Sunday) afternoon. We appeared to pick up a particularly nasty flavour of DDOS attack from some bunch of script kiddies with no brains and more time, so I've spent a little while upgrading and reconfiguring our firewall to make that harder for the next one.

The new firewall rules appear to be quite effective - it even blocked an online portscanning tool for the crime of... portscanning the network :D
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

The forum and website will be down for a bit later this evening as some nice person has just handed me 4GB of RAM to put in it :D
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Whoops. Apologies for the forum downtime just now. I installed the new RAM, forgot to restart the webserver process and headed off to church.
Terribly sorry and all that :D
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

There may have been a brief glitch around 10:30am today due to the server's home partition becoming... how shall we put this, fuller than I'd like.

Fixed short term, but I may schedule some downtime this weekend to do some partition management.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

The server will be down for a portion of this afternoon/evening, as I need to update it to patch the Shellshock vulnerability.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

The patch is a touch more complex (due to the current version of the server software being unsupported, so I shall be doing this this afternoon.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

October 01, 2014, 09:56:39 am #41 Last Edit: October 01, 2014, 12:23:29 pm by Mike Whitaker
Well. That WAS exciting. Not.

Short summary:

Server broke. Server fixed. Mike awesome.

Longer summary:

Began to notice lots and really ohfuckthisisnotgood lots of disk errors on the server around midnight last night. All three RAID partitions were marked as one disk of the pair failed, but rather alarmingly, it wasn't the same disk in all cases.

Rebooted the server, since it was in a confused state as a result and wasn't letting me fix the array.

No RAID disks visible on the SATA bus at all - BIOS times out trying to find them.

Faffed around till about 2am, gave up and slept on it. Starting to suspect the SATA card OR the motherboard was dead.

Woke up with an 'aha'. Removed the drive I thought was suspect, swapped in the spare I keep. Presto, BIOS finds all the drives, all the RAID volumes happily claim they have one good partition but could they have another please? Reformat the spare drive, add the appropriate partitions to the appropriate volumes, and leave it to rebuild. It appears that the dud drive was so dead it had been taking the entire SATA bus down with it.

Restart the web server process, and here we are.

Lessons:

Always, always, expect your shit to fail.
Leaving the email address in mdadm.conf as local when /var/mail is failing means guess what? you don't get emailed when the RAID fails :D

Note to self:

We are now running in 'no spare disks' mode. Go buy some more hard drives.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Carl Fisher

Games Played (Armies): WAB + WaC (IER Roman, Early Saxons and variants, Normans), WFB (Empire), 40K (Spacemarines), Uncharted Seas (Empire, Shroud Mages), WAB English Civil War (English!), Operation Squad (British) SAGA (Normans, Jomsvikings), Mordheim, Wings of War/Glory WWI, Dreadball, Chain of Command (British), Frostgrave, Sails of Glory and many others!!

Mike Whitaker

Server will be going down for about half an hour around about 4.30pm UK time (11.30am Eastern) today (Friday), as there are two shiny new hard drives and a smaller, more energy efficient power supply sitting on my desk waiting to be installed.

There will also be a longer period of downtime, probably around 9am UK/4am Eastern, on Saturday, to install a new boot SSD and securty updates to the operating system. Watch this space for more.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Apologies for brief site and forum downtime around 10pm Tuesday 4th - it appears the cable between my router and my BT modem has been on the way out for quite a while, and attempting to troubleshoot why we were getting rubbish download speeds finally killed it.

Fixed now.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.