About a year ago I decided to change what I am doing to manage my data beyond using a backup solution.  At the time I considered all of the options very carefully and decided to settle on Microsoft.  Primarily, because they had a lower cost cloud solution.  And because they promise quality integration into the operating system.

A year later, I’ve decided I need to rearchitect my cloud solution.  After a year of issues, slow download speeds, poor indexing, and the application randomly wiping files and redownloading everything I’ve decided that SkyDrive (sorry “OneDrive”) just isn’t ready for public consumption.

My advice is to use this beta-gone-wild as it should be used … to store a few extra GB of information you have that is not important to you on a day-to-day basis.

At this point I need to caution:  This is a bit of a tongue and cheek article with some data added for reflection.  If you came here expecting a lot of positivity for Microsoft or shocking information, this may not be the article for you.

First, Why I Need Cloud Storage

I build a lot of software – in fact I have 25GB worth of projects on my computer across ~120,000 files.  I also have 5 devices that need quick synchronization between them and frequent backups. This is important because I know several developers that have had failed hard-drives with no backups, and ended up leaving development altogether.

In the past, I relied on a home computer system running a RAID 0 & RAID 5 split which has always worked well.  But this isn’t the best solution for synchronization of data – so I decided to add a cloud solution for the synchronization layer.  This also has the side-effect of adding data replication offsite which is a big plus as we had some massive floods here last year.

When I made the jump, I did a normal amount of research (looking at data for about a week) and decided to go with SkyDrive (as it was called at the time) based on articles and information mostly led by Microsoft.  Because I made the mistake most people make, and didn’t scratch below the surface, I forgot the one cardinal rule…  “Microsoft lies” (like most gigantic companies).

My systems that require data replication today:

• My tablets (Windows Surface Pro, PlayBook, Nexus)
• An amazing laptop for development (Clevo)
• A 30TB home-built backup server (Storage System)
• Mobile devices (Primarily my BlackBerry LE)
• Several development VM’s in the cloud (including Linux, Mac and Windows)

What kind of data:

• 15 different development IDE’s for project storage
• Beta releases, Gold releases, and related marketing information
• Architectural documents for shared projects
• A rather large GitHub archive
• Development resources (purchased artwork, music, sound effects, icons, etc)
• An archive for important articles written for blogs (required for being paid for the work)

Once the data reaches the 30TB system, it automatically handles incremental backups using Areca.

Comparing Solutions

What I was looking for was a commercial level solution, that has a history of being tried and tested, with a company willing to stand by what they are selling.

DropBox:

• Limitations
  • High cost (132/year)  Edit:  DropBox has revised their prices and sizes since posting
  • Smaller company means higher risk
• Advantages
  • Solid tested performance (I was testing it at the time with 5GB +)
  • Well spoken of by larger companies
  • Better technical support (people actually talk to you)
  • Better architecture for handling files

SkyDrive:

• Limitations
  • Made by Microsoft
    • This may seem harsh, but consider:
      • Large Company with Fragmented Support
      • Bad History with New & Inspired Products
      • “Get help from the community, not us” style support model.
• Advantages
  • Integrated right into OS
  • Lower cost (50/year)
  • Solid tested performance as reported by users
  • Confirmed solid performance when I had a small SkyDrive (~50mb)

Box.Net:

• Limitations
  • Less field tested
  • More Buggy, issues with mobile applications
  • Applications prone to crashing
  • Difficult to get support
• Advantages
  • Free 50GB
  • OK performance, but not as consistent as DropBox (tested with 5GB +).

So there I was, with the evidence right in front of me.  Box.net was certainly attractive, but I didn’t want an experience that could be buggy.  Some companies like copy.com didn’t really have any big customers.  I also saw not many companies were willing to stand by Box.net, and mixed reports about performance.  This meant, no Box.net.

The high cost for Dropbox, 120/year, made me hesitate although my experience has been great.  At the time Microsoft was speaking very well about SkyDrive and advertising it as something great and to be bundled into Windows 8.1.  Reviews online seemed positive.  The online ordering process seemed easy.  And as 8.1 would eventually be used on my Tablet and PC (against my will I should caution) I decided to roll the dice and bought into SkyDrive.

As for upgrading from Windows 8 to Windows 8.1 … I held on as long as I could before I decided to make the change to finally get rid of the annoying popups after the registry solutions from Microsoft failed to work.  The upgrade also resulted in me losing all my applications and data (even though Microsoft said that wouldn’t happen).  Fortunately I didn’t trust what they told me and had backed up my applications and serial numbers.

Issues with SkyDrive

Well, I made the migration, and began the migration of my 25GB of data.

It is very important to understand, I have a fairly decent internet connection (100Mbps).  This was a result of me getting rid of Shaw, Telus and moving to an IP phone with Ooma, Internet with Nucleus and getting rid of cable altogether.

In addition, it is important to recognize there are 3 important components to cloud storage.  Upload speed.  Download speed.  And file synchronization.

Moving & Uploading Files to SkyDrive

I relocated all the files to my SkyDrive folder and allowed the application to upload.  I immediately found the upload was unbearably slow (i.e. 50kbps or less) and upon investigation realized it was caused by two issues:

• They  limited the number of consecutive uploads to 1, which causes issues for lots of small files
• They have a delay “timer” between uploads, this causes issues when you have many small files as the timer delay between files  is up-to 10 times longer than the upload time
• The timer is also compounded by resynchronization periodically between files which can add hours
• Slow ramp-up period (files upload slow and very slowly start increasing – up-to 10 minutes to reach a peak)
• In addition, there appears to be bugs that make SkyDrive “die” and stop working – but everything looks OK on the surface

To resolve some of these issues I had to dig in my %appdata% folder under Microsoft\SkyDrive and make a  bunch of modifications to the configuration files.  Specifically increasing upload limits, download limits and other settings.

After the changes were made and the system restarted, I finally saw some better upload times.

Downloading from SkyDrive to my Tablet

My initial synchronization out to my devices was unbearably slow.  But I didn’t worry about it too much as I figured I would only have to do this once.  So I let the synchronization run for 3 days or so before it was completed.

Issues after Upgrading to Windows 8.1 : Online Files Only / Files Won’t Synchronize

This part was an absolute nightmare.

The initial releases had some ridiculous bugs with SkyDrive.  The application would not synchronize files – ever.  It would simply stay “downloading” and stuck on the first few files, and would never download.  Microsoft claimed to have patched this several times on their forums … but as of the day I switched, this issue was still persistent.  In addition they would not actually provide support just direct you to forum threads – typically with no Microsoft personnel in the threads at all.  It was a very frustrating period.

The issue with the above, is in 8.1 they removed any ability for you to monitor what is going on.  It provides no level of information including basic  synchronization information.  The Metro-side application actually misleads you, and misreports what it is doing.  And it randomly marks all your files as “online only”.

This means your device never will download the files.  Most applications error on your PC when it tries to access them.  And periodically it deletes all your 25GB of files and re-downloads them.  To this day, they still have bugs where every few reboots the files are marked as Online only (again).

Further Windows 8.1 Additional Complications

For example, when upgrading, it wiped out the 25GB of SkyDrive files.  Well, it actually just renamed the folder, and started all over again.

This caused it to re-download all 25GB.  It is a painful process, as “Synchronization” for SkyDrive takes 2-3 hours to “count” what files it is missing.  And the download had absolute horrible download speeds more on that part later.

Even on a fully synched drive, the synchronization period takes up-to an hour.

SkyDrive to OneDrive

When the brilliant folks at Microsoft decided to make an irrelevant change, I ignored it like most users.

Until it created a mess with files on several of my systems.  What the change did was create a \OneDrive folder, Migrate the files over, and then forcefully create a Symbolic link at \SkyDrive that cannot be removed while the application is installed.  This is a simple change in itself (and the right way to do things I suppose) however in case you didn’t know, Windows has all kinds of ongoing issues with Symbolic links.   Even in SkyDrive itself, Symbolic links cause problems. You would think they would avoid these – or at the very least, make them removable.

This folder change caused all kinds of problems with my systems.  And at one point the OS eeven managed to create a separate \SkyDrive folder which caused files to become disjointed from OneDrive.

Why I decided to change today

Today was enough.  I rebooted this morning, which caused SkyDrive to mark all files as available “online” only again against the settings I specified – subsequently deleting my 25GB and re-downloading all 25GB of files … again.

SkyDrive / OneDrive – Summary of Ongoing Issues to Anticipate

• SkyDrive issues with Symbolic links
• SkyDrive not downloading
• SkyDrive not uploading
• Very slow downloads
• Very slow uploads
• Poor UI with no feedback on Windows 8.1
• Frequent crashing of the application
• SkyDrive using single file uploading and downloading
• Extremely slow “synchronization” component
• Support is completely non-existent

Recommendation (Go-Forward Strategy)

I’ve performed significant testing on DropBox.  I’ve confirmed downloads seem to be reaching a much higher download speed limit (near maximum).  Which can be up-to 20x the speed I’m receiving from Microsoft (yes – you read that right) and is, at the very least, consistently at least 4x faster.  In addition it calculates synchronization requirements within minutes versus SkyDrive taking hours.  It also does not force symbolic links or generate errors with symbolic links.  And it doesn’t make a whole mess of your filesystem.

OneDrive may cost 1/3 as much – but with that you inherit a lot of problems.  Consider it the beta-user discount.  With their history of accessing and deleting peoples files on SkyDrive, the difficulty with removing OneDrive from Windows 8.1, and all the issues in this article up until now – I am willing to pay extra money to make all these problems go away.

If you are architecting backup solutions in the future – make absolutely SURE to skip SkyDrive / OneDrive integration at all costs.  Especially for anything with business requirements!

Speed Tests (My spidey senses are tingling, but this is probably what you care about and what Google sent you here for)

I just completed a series of Speed Tests, which are a bit difficult with the nature of their systems the way they are.  But side-by-side here is what we have based on actual data saved by the applications and with nothing else running.

Average Download Speeds (Testing 6 times)

• A Few Minutes After Starting Transfers — Dropbox:  900kbps to 1,300kbps
• A Few Minutes After Starting Transfers — SkyDrive:  50kbps to 650kbps
• @2hrs — Dropbox:  3,300kbps to 3,650kbps
• @2hrs — SkyDrive:  350 to 700kbps
• @4hrs — Dropbox:  3,400kbps to 3,650kbps
• @4hrs — SkyDrive:  300 to 550kbps

This is a major difference after running for a few hours.  Perhaps DropBox is just better suited to having servers in central Canada.

Upload tests show both SkyDrive and Dropbox are utilizing my maximum upload link of 500KB/sec.

Synctime Average With No New Files:

• Dropox:  ~15 seconds
• SkyDrive:  ~45 minutes

Synctime Average With Many New Files:

• Dropox:  ~15 seconds
• SkyDrive:  ~120 minutes

Data Downloaded After Timers Expired:

• Dropbox:  Downloaded 414MB
• Skydrive:  Downloaded 155MB

Dropbox is the clear winner.

Please note, it is currently midnight and I am tired and annoyed at Skydrive.  So I didn’t do any upload tests.  But I’m doubtful my 5Mbps uplink will find the threshold on either anyway.

Tips and tricks:

• You can modify configuration files for SkyDrive in the %appdata%/Local/Microsoft/OneDrive to get better performance
• If you watch the file sizes under your SkyDrive/OneDrive folder properties, you can see the real size as the folders download data
• Windows 8.1 utilizes a synchronization executable for the Metro application only that provides limited information and customization, the desktop installer is not compatible with 8.1
• You can’t reinstall the stand-alone SkyDrive application by any normal means on Windows 8.1
• If you migrate to DropBox, it comes with a free Frogurt!  The frogurt is also cursed.