Case Study:
How to backup 350 servers and 1 million files every night
TRES internet BV
| TRES internet is a leading online marketing company in The Netherlands. With a specialization in online strategy, concepts, building, and marketing, TRES internet is successful at archiving marketing goals and gaining customer value for its clients. For more information see www.tres.nl |  |
The Challenge
TRES internet BV hosts over 350 servers, with over 1 million files. All of these files are backed up every night. As the number of files continued to increase, they faced new challenges: "We were using an in-house developed file transfer program to perform backups, but it could no longer complete the backup during the night," said Erwin Hoberg, Software Developer and a member of the team responsible for the implementation of the new backup system. "In addition, the old program wasn't handling various error conditions very well, so we needed a new software with robust error handling, so we could be 100% sure that all those million files were successfully backed up every night," he continued.
The requirements for the new backup software could be summarized like this: Fast speed, reliability, automation, and extensive reporting. The first two were needed to perform the full backup reliably every night. And the last two were needed to integrate the backup software with the existing intranet system that provides overall control, automation, and management for all of the servers.
The Solution
When the development team started to search for a solution, the first thing they needed to tackle was the speed: "We did some testing with BatchSync and it reached an improvement of over 10x the existing system. We were amazed by BatchSync's performance," Erwin Hoberg said. This improvement is due to BatchSync multithreaded nature, which uses multiple concurrent data streams to transfer data. "We also liked BatchSync's ability to handle various errors, report them, and continue the file transfer process without interruption," Hoberg continued. "With BatchSync's really fast speed and good error handling, we were able to cover the first two points of our requirements."
After having reliable data transfer in place, the development team needed to integrate BatchSync with their operational software. TRES internet BV had developed an intranet system to monitor and control all of their servers. This intranet system contains information about all of servers, including the IP addresses, login names and passwords, and current system and backup statuses. BatchSync was integrated into intranet system by using its command line interface. "We query our intranet database for all servers that need to be backed up. Then we call BatchSync with the needed command line parameters to perform the backup for each server separately. And finally, we save the backup results to the database," Hoberg explained. "This way we can see the backup status for all of the servers at one glance," he continued.
"The integration process went quite easily, but there were times when we needed technical support from Maxlevel Inc. And it was top notch. They were helpful, capable, and quick to find fixes and solutions for us. So all in all, I would definitely recommend BatchSync," Erwin Hoberg summarized.
The Conclusion
"The new backup system is now running in the production, and
it is a success. It performs the full backup in 1 hour and 20 minutes,
and it's fully automated, controlled, and monitored via our intranet
system. It also supports company growth, as we can add new servers
to the system, and they are automatically fully backed up. So we
started with our requirements for speed, reliability, automation,
and full reporting, and now we have it all," Erwin Hoberg said
in conclusion.