City of West Memphis computer system crash causes chaos
We all rely on computers these days for day-to-day business, and the City of West Memphis is no different, so when the city’s computer systems crashed earlier this month, it threw a monkey wrench into the works in several municipal departments.
The computer servers that keeps track of utility company billings and city employee payroll went down. While the server was restored, City Treasurer Frank Martin broke the news to airport commissioners Thursday morning when they asked him for monthly financial statements during the March meeting.
“You will not have any February financial,” said Martin. “The server went down this weekend and when we brought it back up this weekend it printed all the accounts payable double,” said Martin.
“There is no way to get you your final report. We are in the midst of fixing it right now.”
The city has run for years on four small antiquated servers that have been producing more frequent and more serious glitches.
To address the issues, earlier in the month, the city utility commission voted to buy two new larger servers and to set one up on site at the utility department and have a redundant system set up at the police station. Martin outlined the process of obtaining the new equipment and asked for an emergency appropriation ordinance to get it done during the regular city council on Thursday March 21.
City Councilman Wayne Croom was at the airport meeting and asked about preventive measures to keep the servers operable.
“What is being done to make sure this won’t happen again?” asked Croom.
“I’ve prepared an emergency ordinance to order another server,” said Martin. “It literally went down when we were backing it up this weekend. So what we are recommending today is not to have a back up but instead to have repetitive servers continuously running.”
The two servers would mirror each other in real time with each updated instantly to sidestep the need from the back up to a server. The city was set to continue back to the cloud. “No more back ups, just continuously running,” said Martin. “It’s been recommended to have to matching servers, one at the police station and one at the utility. We have to order that server today and we back up to the cloud.”
On March 14 the Utilities Commission recommended the purchase of a server with a redundant unit at the police station. Commissioners recommended to the purchase to city council and asked approval by resolution. Unserviceable outdated software was the reason the utilities department wanted to update its equipment.
“Currently we are running on a windows system from 2008 and its now 2019,” said Utilities General Manager Todd Pedersen.
“The servers are starting to have glitches on them.
We’ve gotten ten years out of these. It’s about the end of life for those servers.
Our warranty is gone. As of January 2020 it won’t be serviced by Windows any longer.”
Installing the new servers was expected to take six months. The utility company will foot the $90,000 bill for the whole city. The Utility company had selected Dell equipment for its rapid response warranty and C& T Solutions will custom build and install the new servers.
“We will include an increased bandwidth between the police department and utility building,” said Pedersen. “That will be an added monthly cost.
We will have to have a two sonic walls, one in each building. Our whole city phone system and all city e-mail is out of our building. We will have to upgrade from Windows 7 to Windows 10. Potentially you could see us coming back with more budget amendments and approvals as we update certain things.”
By John Rech