Last Tech Tuesday I wrote about the importance of having backups and testing whether you can restore from them. While this sounds like something everyone should or would do it is all too easy with the many things a startup has to do to forget about them. Backup is part of the larger disaster recovery planning process. Again not something most startups spend much or any time thinking about. And in the spirit of staying lean I would suggest you don’t worry much about it until you are adding customers or endusers at a good clip.

Once you have product market fit though it is important not to forget about disaster recovery planning. We had several companies in the USV portfolio that were knocked offline by hurricane Sandy last year in some cases for several days. What went wrong? All of them were hosting in Manhattan with facilities that ran into trouble with their own backup power generation (largely by having diesel fuel stored in areas that flooded – this was also responsible for knocking out backup power at NYU medical center).

A key lesson learned here is that datacenters in dense urban centers may face a unique set of challenges in extreme conditions. And importantly: you should not require physical access to your servers in order to be able to move your service elsewhere. Put differently: everything should be backed up to a separate location, ideally with very significant geographic separation. Several of the Manhattan datacenters did not allow access to the buildings they were in as that was deemed unsafe by the police and fire departments.

Another important lesson in emergency preparedness is to have lots of ways in place for communicating between employees and with customers / endusers. That includes making sure to have cell phone numbers of key technical employees (but ideally all employees). Also you should be using subdomains such as status.company.com instead of company.com/status so that you can use changes to DNS to point status updates easily to a different service should that be necessary.

Again, don’t spend time on this before you have a business. But once you do don’t forget to come back to this or else you may no longer have one should disaster strike. If you have a great disaster recovery lesson learned – please share!