Solano County, California is situated between San Francisco and Sacramento. The Solano County government provides countywide services such as elections, property records, public health, tax collection, and social services. For elections, the county’s Registrar of Voters is responsible for ensuring federal, state, and local elections are conducted in a secure, accountable, and responsible manner to maintain public confidence in the electoral process. In addition to conducting elections, the department maintains voter registration data, precinct maps, polling place locations, and publishes election results.
Cherian Alexon, Senior Manager- Software Engineering, works at Solano County at the Department of Information Technology. His team is responsible for ensuring the county website is secure and reliable for all 21 departments that utilize and update citizens on county activities. As a local government with many departments, Solano County faced a variety of challenges in keeping the site secure and reliable under limited resources.
The county conducted security audits, penetration tests and frequently works with third parties to identify vulnerabilities of their site ahead of the 2020 U.S elections. Their architecture included several legacy components undergoing upgrades, so balancing migration projects with responding to an evolving threat landscape meant little time was left to maintain their patch management program. Alexon explains, “While executing our disaster recovery plans during past crises, our website was unavailable for several hours. This was when the public relied most on us to provide up-to-date information, so being unavailable not only affected public opinion but had the potential to impact public safety.”
The amount of web traffic to Solano County increased significantly to COVID-19 shelter in place orders, as well as the wildfires that burned across Northern California in 2020. The website experienced sudden spikes in web traffic during these major events causing visitors to complain about outages and slow loading times. On March 18, 2020, the County of Solano issued a shelter-in-place order due to the public health emergency presented by COVID-19. The Department of Emergency services sent out text messages to residents with updates on the shelter-in-place order that included a link to the county website for more information.
Alexon recalled, the website was instantly flooded by more than 150,000 concurrent users, and the site struggled to keep up. “To address the large volume of traffic, we needed additional servers with the ability to balance load across them. We had to work around limitations within the legacy platform to handle the massive amount of traffic created by concerned citizens looking for information on shelter-in-place guidelines.”
The website struggled again with an influx of more than 350,000 concurrent users in August of 2020 as wildfires swept through Northern California. To provide updates on resources available to assist residents impacted by the fires, they created a separate website for the information and reached out to Cloudflare looking for help maintaining availability during unexpected spikes of traffic. “When Cloudflare offered free services to government agencies during the COVID-19 pandemic, we jumped on the opportunity to onboard the fire recovery site. Our site went down twice during emergency announcements, so we knew that we needed to be ready for future events.”
The Solano County team reached out to Cloudflare the week before the 2020 national elections looking for help protecting their main county site that provided poll locations and election results. “Security was a major concern for us as we approached election week. It was important that we stayed online during the general election as we had performance issues in the past,” explains Alexon.
During the primary election in California in March 2020, before using Cloudflare, the county had a member of the security team continually monitor and block suspicious traffic that bypassed a traditional firewall. Unfortunately, that meant at times blocking legitimate traffic, causing inconvenience to genuine users. Alexon says, “little did residents know that we had a staff manually identifying malicious traffic, monitoring resources to keep the site up and running and who had been there all night ensuring the site stayed online.”
The week before the general election, Alexon reached out to Cloudflare to see if they could onboard their main county site that was posting election results. Within a few hours, the county was onboarded to Cloudflare with best-practices tailored for election entities that use our services under the Athenian Project. Due to the past issues of the website being taken down, it was critical that the site not experience downtime due to the transition to Cloudflare. “Since we had so two major incidents in a 6-month span, we couldn’t afford to go offline for any reason, especially days before the election. With Cloudflare, we were able to easily onboard our county site and implement security tools including the web application firewall (WAF) using a ruleset managed and curated by the Cloudflare team to keep our site secure,” recalled Alexon.
During the 2020 national elections on November 3rd, Cherian noted that the team witnessed a 20-fold increase in traffic to the Solano County website that was publishing election results. Typically, the county sees around 1,500 to 2,000 concurrent users, but on election day the team says the number of concurrent users exceeded 50,000. “There was a huge difference in our security posture going into the November 3rd election,” explains Alexon, “On the security side, we were not worried about being taken down as we were confident Cloudflare would mitigate threats and help with influxes of traffic that previously took the website down.”
Before Cloudflare, Solano County was worried about public opinion.“With our site being unavailable during the COVID-19 shelter-in-place order, and then again with the spread of the wildfires, we had to stay online during the elections as trust is a fundamental part of the democratic process.”
The tools provided by Cloudflare allow Solano County to focus on its mission of bringing reliable information to its residents. Regardless of the amount of traffic heading to the county site, they are able to stay up-and-running and serve out their purpose — without the burden of expensive services.