This week I've been volunteering at an event called DreamHack, which is the largest digital festival in the world. I've been volunteering for this event quite many years, and this time around I was editing videos and providing support to one of the teams. During the event we learned that we needed to upload some video-material, and even though we obviously could have used OneDrive, Dropbox, etc. to share the content, I decided this was a good opportunity, and a real use case, for the cloud. Simply spinning up a VM with a FTP server for the week, and then turn off the VM once we were done at the event. Worth mentioning is that we needed to transfer 2 – 3 TB of data to/from multiple machines running multiple OS platforms during the event.
A photo from one of the halls at DreamHack. Photo by Diora Kurbanova.
Starting with Microsoft Azure
Since I've been using Microsoft Azure before, I decided to start there, and I created a D2 series VM in Azure with the following configuration:
- Windows Server 2012 R2
- 2 cores
- 7 GB RAM
- 60 GB SSD (OS)
- 4 TB HDD (Data)
Note: When I added the 4 TB data disk to it. Or rather I tried, just to learn that I couldn't. Maximum size per disk for this type was 1 TB, so I created 4 of them, and then created a storage pool of 4 TB inside the VM. So I got my 4 TB volume after all. #Shiny.
Configuring disks in Azure.
Setting up the FTP Server
Now, even though Windows Server 2012 R2 does have a FTP server natively, via IIS, it's kind of limited, and difficult to work with. So I used the free FileZilla FTP Server instead. Setting up FileZilla FTP Server in a cloud VM is pretty straight forward. Here is a detailed guide: http://stackoverflow.com/questions/39095900/install-ftp-application-filezilla-server-on-your-windows-azure-virtual-machine-vhere, and in addition you find a basic checklist below:
1. In the cloud VM, install FileZilla FTP Server, and configure for passive mode, and set a port range. I used port 6100.
2. In the passive mode configuration, add your public IP address as the external interface.
3. Enable FTP over TLS to have some security (port 990).
4. Open up the ports (inbound / outbound for 21,990 and 6100).
5. In the Azure portal, open the same ports.
Note: If you know what IP addresses that will connect to this server, limit the firewall scope to them. Better than having every script-kiddie out there trying to hack your FTP Server 🙂
Uploading
Once the FTP Server was up, and the firewall ports opened, I started to transfer files from the event, and got the following upload result:
- 1 single file: 8 MB/s
- 10 files at the same time (maximum for the FileZilla client): 75-80 MB/s
Transferring only 2 TB on that speed would take about 7-8 hours, and before you make the comment, NO, it is NOT our Internet connection. The DreamHack event has a dedicated 40 gigabit Internet line, it is actually my local gigabit card limiting the upload/download. 🙂
Uploading files using the FileZilla FTP Client.
Giving Amazon AWS a run
Next vendor to test was Amazon AWS, so I signed up for Amazon AWS, and created a t2.large EC2 instance which had the following configuration:
- Windows Server 2012 R2
- 2 cores
- 8 GB RAM
- 60 GB SSD (OS)
- 4 TB HDD (Data)
Note: Unlike Azure, Amazon allowed me to create a 4 TB data disk directly so I was happy with that.
I did the same configuration of the VM as I did for my Azure VM, installed the FileZilla FTP Server etc. Worth mentioning, this was my first time using Amazon AWS, but it was pretty straight forward, and their portal (console) used to manage the machines was actually quite a bit faster compared to the Azure portal.
Starting to upload
Once the FTP Server was up on the EC2 instance, and the ports opened, I started to transfer files. This time the result was slightly better. I got the following result:
- 1 single file: 10 MB/s
- 10 files at the same time (again, maximum for the FileZilla client): 75 – 90 MB/s
Summary, Support, and Billing
Shorthand, for this scenario Amazon AWS was slightly faster for transfers then the Azure equivalent, but all in all, it sure proved that cloud could be used to solve a business problem quickly. Very shiny.
What about support you ask? For the fun of it I also reviewed the various support options available, posting a questions about upload speeds. I wasn't very interested in paying $300 for the nicer Azure support, or $100 for the nicer Amazon support. So I tried the free options. Both vendors responded within a day, pretty ok for free support.
I have yet to post the billing info, but wanted to wait until we are done with the VMs. So far the cost has been in the $100 range for each vendor. Amazon billing info is super-easy to understand, super-fast, and easy to find out exactly how much a given instance is costing. Microsoft billing info is the exact opposite. Slow, hard to read, and more or less impossible to figure out the cost for a given instance.
Written by Johan Arwidmark