FORDHAM IT'S THIRD SITE FOR DISASTER RECOVERY:
SANTA CLARA UNIVERSITY
by Richard Eberhardt, Multi-Platform Systems Team Lead, Computer Services
Campus of Santa Clara University
(Image from: "SCU Jumps to No. 5 in Best Places* to Work Survey."
*Best place in the Bay Area)
Did you know that Fordham University has a footprint in Santa Clara, California, the heart of Silicon Valley? We have Hurricane Sandy to thank for this development.
Let me explain.
In October 2012, Hurricane Sandy disabled both of Fordham's Internet links, leaving the entire university community without access to Gmail or the Fordham website. Since these services are hosted in the cloud, one might assume they would remain accessible even if Fordham was off the network. Unfortunately, that was not the case. We need a domaine name server (DNS) and authentication (CAS and LDAP) for the cloud services to work. As a result, Fordham was completely offline for a short time.
That situation prompted Fordham IT to be better prepared for the next “Sandy event.” In a partnership made possible through the Association of Jesuit Colleges and Universities (AJCU), we’ve joined with Santa Clara University to become each other’s disaster recovery site. One facet of that preparation involves some Rackspace we’ve traded with Santa Clara’s datacenter.
To build our infrastructure, we first had to purchase equipment, including storage, servers, firewalls and network switches. Then we configured and shipped it to California, where it was then reassembled. Computer Services, Networks, and OIS worked together to build the basic infrastructure that will keep www.fordham.edu and Gmail working if another event like Hurricane Sandy occurs.
Santa Clara University has also created a disaster recovery site here at Fordham that is similar to the one we built in California. Recently, their staff came to New York to install their equipment in our rack.
Everything at our site in California sits idle except for DNS, which takes 1/3 of active DNS requests. We configured login.fordham.edu, ldapplus.fordham.edu, www.fordham.edu, and gmail.Fordham.edu to failover automatically. Our primary failover site is Lincoln Center. If that fails, Rose Hill will take over. If both campuses are down, Santa Clara will take over.
This all sounds good, but how can you be sure it will all work when it has to? Test it, of course! On March 22, 2014, we completed a live, in-production disaster recovery test. Members from Computer Services, UISO, SSIA, and Networks gathered on a conference call early one morning. We disabled services at Lincoln Center and the Internet links at both campuses. Services failed over flawlessly. In fact, the changes propagated through the Internet faster than we expected. Now we can all feel confident that even during a localized disaster, Fordham will maintain its presence on the Internet.
This project not only strengthens the resiliency of both universities, but also gives us the opportunity to partner with another Jesuit university and work with their staff toward common goals.
Richard Eberhardt works in Computer Services as the Multi-Platform Systems Team Lead.
