• Welcome to Peterborough Wargames Club. Please login or sign up.
 
May 22, 2022, 08:26:55 pm

News:

Note for prospective members:
* Please choose a screen name that reflects your real name
* Please drop an email note to the info@ address introducing yourself.


Server Updates

Started by Mike Whitaker, February 15, 2012, 04:25:32 pm

Previous topic - Next topic

Mike Whitaker

December 09, 2016, 09:35:18 pm #45 Last Edit: December 09, 2016, 09:37:50 pm by Mike Whitaker
You'd think I'd learned from this last time.

Unfortunately, the forum wasn't terribly readable last night due to a very similar problem. On the good side, when your email starts receiving:

QuoteThis is an automatically generated mail message from mdadm
running on highport.altrion.org

A Fail event had been detected on md device /dev/md0.

It could be related to component device /dev/sde1.

Faithfully yours, etc.


... you do get a bit more warning than 'oh shit my disk is failing'

Now fixed (same problem as last time - unhappy drive making SATA bus unhappy). MORE spare drives bought.


Quote from: Mike Whitaker on October 01, 2014, 09:56:39 am
Well. That WAS exciting. Not.

Short summary:

Server broke. Server fixed. Mike awesome.

Longer summary:

Began to notice lots and really ohfuckthisisnotgood lots of disk errors on the server around midnight last night. All three RAID partitions were marked as one disk of the pair failed, but rather alarmingly, it wasn't the same disk in all cases.

Rebooted the server, since it was in a confused state as a result and wasn't letting me fix the array.

No RAID disks visible on the SATA bus at all - BIOS times out trying to find them.

Faffed around till about 2am, gave up and slept on it. Starting to suspect the SATA card OR the motherboard was dead.

Woke up with an 'aha'. Removed the drive I thought was suspect, swapped in the spare I keep. Presto, BIOS finds all the drives, all the RAID volumes happily claim they have one good partition but could they have another please? Reformat the spare drive, add the appropriate partitions to the appropriate volumes, and leave it to rebuild. It appears that the dud drive was so dead it had been taking the entire SATA bus down with it.

Restart the web server process, and here we are.

Lessons:

Always, always, expect your shit to fail.
Leaving the email address in mdadm.conf as local when /var/mail is failing means guess what? you don't get emailed when the RAID fails :D

Note to self:

We are now running in 'no spare disks' mode. Go buy some more hard drives.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Carl Fisher

But two years between failures is quite good.
Games Played (Armies): WAB + WaC (IER Roman, Early Saxons and variants, Normans), WFB (Empire), 40K (Spacemarines), Uncharted Seas (Empire, Shroud Mages), WAB English Civil War (English!), Operation Squad (British) SAGA (Normans, Jomsvikings), Mordheim, Wings of War/Glory WWI, Dreadball, Chain of Command (British), Frostgrave, Sails of Glory and many others!!

Mike Whitaker

One of the drive pair failed Friday night (obviously while I was away, because why the hell wouldn't it?)

Rebooted, swapped in the spare from the two on my bookcase, now rebuilding. Everyhing should be fine.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Advance warning - there will be some weekend downtime sometime in the next couple of weeks, as I have just bought the 5-year old server's replacement - I have a suspicion the case fan and the SATA bus may be on the way out, and it's past due for a replacement anyway.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Expect some downtime on Saturday evening (May 5th) while it all gets transferred over to the new box.

Also possibly expect some downtime Wednesday afternoon (May 2nd) as we're having a smart meter installed. Server and net connection are on the UPS, so should stay up if the power outage is brief enough.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Running a fairly large security update on the server - this will result in a bit of downtime over the next couple of hours.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

phase 1 completed - there will be more downtime, as shutting down the web server seems to be the first thing it does as part of the update :D
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Should be sorted now, will need one reboot but I want to do that with a screen on the server, not remotely,. so that will be sometime tomorrow.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Apologies for downtime from about 22.05 on 22 March - local power cut. All now restored, except that I will need to take things down again sometime tomorrow to swap the UPS back in.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Carl Fisher

Quote from: Mike Whitaker on March 22, 2019, 11:15:22 pm
Apologies for downtime from about 22.05 on 22 March - local power cut. All now restored, except that I will need to take things down again sometime tomorrow to swap the UPS back in.


Yes I reported the power cut, covered quite a large area!
Games Played (Armies): WAB + WaC (IER Roman, Early Saxons and variants, Normans), WFB (Empire), 40K (Spacemarines), Uncharted Seas (Empire, Shroud Mages), WAB English Civil War (English!), Operation Squad (British) SAGA (Normans, Jomsvikings), Mordheim, Wings of War/Glory WWI, Dreadball, Chain of Command (British), Frostgrave, Sails of Glory and many others!!

Mike Whitaker

Failed HD on server - various services are down.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

HEADS UP:

I will be shutting this server down in the next couple of hours to move the databases across. You *may* lose any new posts, replies and PMs from about 5pm onwards.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Ok. We're back, I think.
It appears the new server expects php7, so I had to do a wee bit of magic to get this to run under php5.6 :D

Any issues, please drop me a message.
Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Mike Whitaker

Blog: http://troubleatthemill.blogspot.com/
Games (currently) Played: IABSM, Dux Britanniarum, Sails of Glory, Gaslands, What A Tanker!, SW: Armada
28mm: Parthians, Early Imperial Roman, British Tribes, Saxons, Norman/Christian Spanish, English Civil War Royalists; WW2 Wehrmacht, Red Devils, Home Guard; Judge Dredd Judges, Dreadball Void Sirens. GoT Night's Watch
15mm: WW2 US, British/Canadian and Heer infantry companies + support
54mm: HYW French and British
1/1000: Sails of Glory (lots)
WIP: 28mm WW2 British Commandos; 28mm War of the Roses, Peninsular British/French/Spanish; 28mm Coalition Wars SF; 15mm Fallschirmjaeger and US Paras, lots of 15mm WW2 vehicles. 1/2400 WW2 Naval

This sig is a GW- and Flames of War-free (except for the figures I own!) zone.

Rob Farley

Ok, it turns out that the updated forum software is so cutting edge that SMF only have a single theme that's confirmed as compatible.

I've installed it, but I'll keep an eye out for other options.

Still need to reset the logo, not sure of the filepath on Mike's server.
Current games (and armies) played: Rumbleslam, Street Fighter Miniatures, Saga, Marvel Crisis Protocol, Gaslands, Lion/Dragon Rampant, Frostgrave, Stargrave, Bolt Action & Operation Squad (Desert Rats, Communist Chinese, Russians, partisans, Americans, Finns, Italian Paras), Judge Dredd Miniatures, Mobile Frame Zero, Necromunda (Pit Slaves, Escher), Wings of War (WWI and WWII), Mordheim, Warpath

Current Projects: Various halflings, Rumbleslam wrestlers, Superheroes, Sludge Samurai