Support for files with the same name and more Plupload!

Yesterday, I have released a new version of media.io with the following changes:

Support for uploading the same file again

You can now upload the same file over and over again. While I want to support re-conversion of uploaded files, that may take a while. I hope this solution is good enough for a while.

Plupload for everybody

Since the original static file upload was served to few people, all file uploads on media.io are now handled by Plupload. This means that old browsers such as Internet Explorer 9 get the same user interface as up-to-date ones.

Bug Fixes

An addition to the new features, the following bugs were fixed:

  • After adding files exceeding the 750 MB upload limit, the upload limit was never cleared. Now, even if the size of the files you want to upload exceeds the limit, removing enough files will let you add new files again.

Please like media.io on Facebook, follow it on Twitter or, if you know how to make media.io even better, post a suggestion.

Linux Audio support getting better and better

2.5 M files are uploaded to media.io in a month. 90 % of them can be converted without a problem. Some cannot. Recently, I moved to a new server and a more up-to-date operating system.

Since media.io delegates all of the actual conversion work to programs such as ffmpeg, wavpack or oggenc, the newer operating system should convert more files successfully.

Statistics

The test data was 1048575 files from May 2014 and 1048575 from July 2014. Data from May 2014 was collected on the old server running Ubuntu 12.04, data from July 2014 was taken from the new server running Ubuntu 14.04.

Only input extensions that were uploaded more than 1000 times in May were counted.

ExtensionUploaded Files May 2014Success Rate May 2014Uploaded Files July 2014Success Rate July 2014Change (%)
3ga1549092.917658952.2
3gp880091.41092193.11.8
3gpp243394.9244495.60.7
aac1497793.32153892.9-0.5
aif803192.4755389.1-3.5
aiff631978.7752791.916.6
amr2734392.23293796.54.7
ape110470.612639027.6
caf1314291.53064680.3-12.2
flac6471285.65837787.72.5
gsm289797507799.22.3
m4a21175087.921517290.42.9
m4r493893.9421297.74.1
mp2260085161593.39.8
mp319228686.518002089.63.6
mp46305084.36048187.63.9
mpga131789.8717933.6
ogg6043390.34981890.30
ra22257647199.230.4
rm363193.7435082.7-11.7
wav13182787.412587989.82.7
wma19474990.119496692.83.1

Most of the formats show a slight improvement. Whether this is due to changes in the conversion software or just noise is hard to say.

The drastic improvement in support for .ape files is interesting, as is the drop in .caf files. My theory for .caf is that there may be more DRM-protected files around than a month before.

All in all, I think audio support on Linux has become very good. For the popular formats, it's hard to find one that does not achieve 90 % success rates and some even surpass 95 %.

Some new things at media.io

On 24th of May, I deployed a new version of media.io that has the following new features:

Enforcing the 750 MB limit

Back when I launched multi-file uploading, I totally forgot about the 750 MB file size limit media.io used to enforce. For a long time, people could upload as many files up to 750 MB large as they wanted in one go, bypassing the limit.

The file size limit was originally intended to

  • prevent bandwidth abuse
  • help with upload success rates
  • keep traffic manageable.

Since the biggest cost driver of media.io is the traffic, I need to make sure there are certain limits in place that prevent users from consuming extreme amounts of bandwidth.

Both the file size limit and the upload size limit affect less than 0,5 % of all visitors each.

And seriously, if you are trying to convert a 81 GB audio file on the web, you may be better off using a dedicated program.

More new things

  • media.io now prefers Arial. Not a bad idea.
  • Donation buttons are now localized. This was missing for a while.

That's it for this time. If you would like media.io do something, please post your ideas on media.io's Facebook page or use the UserVoice forum.

Web Fonts that make a difference

Way back when Google still had their Website Optimizer, I ran a split test between Arial, Tahoma and Verdana. The difference was small but Arial may have been the better performer of the three.

Nowadays, Google curates and serves a huge number of fonts, many of which even look good.

But web typography, like website design, must work for the site, not against it so using a web font should never lead to lower conversion rates, especially considering the performance hit of using web fonts in the first place.

I tested the following fonts against Bootstrap's default font stack (Helvetica Neue, Helvetica, Arial, sans-serif):

  • Arimo normal
  • Asap normal
  • Noto Sans normal
  • Open Sans normal with bold for headlines
  • Open Sans normal
  • Oxygen normal with Oxygen bold for headlines
  • Oxygen normal with Oxygen light for headlines
  • Oxygen normal
  • Roboto normal with Roboto light for headlines
  • Roboto normal with Roboto medium for headlines
  • Roboto normal

I also tested moving Arial to the first position (before Helvetica Neue and Helvetica).

Asap, Noto Sans and Open Sans didn't perform noticeably better than the default so they were taken out of the rotation. Of the remaining fonts, Oxygen (especially with Oxygen light for headlines), Roboto with Roboto medium for headlines and especially Arimo had the potential to be better than the default.

The difference was very small though with Arimo being less than 1 % better than the default (at 98.81 % confidence).

In the end, Arial as the first font also worked better than the default stack so I went with that, especially since it saves the extra request for the web font.

Chinese, Malay and support for Opus

I've just released a new version of media.io that has some really cool new features.

media.io in Chinese and Malay

media.io is now available in Chinese and Malay. If that's your native language, congratulations!

Support for Opus

Opus seems to be a very promising audio codec, especially for low bitrate applications. media.io now supports Opus files so you can convert Opus to WAV, for example.

Stability updates

I have added a highly speculative fix for the occasional Java Virtual Machine hiccup where threads seem to disappear but blocked other threads aren't woken up.

Better notification for empty files

150 people try to convert zero-byte (empty) files every day. For those, a better error message is now displayed.

That's it for this time. Please remember to like media.io on Facebook or follow media.io on Twitter for the same content but in a much more compact form.

New server with 50 % more power

media.io has been running on a server with an Intel Core i7-3770 CPU for almost a year now. When I bought the server, it could cope with the traffic easily. Of course, said traffic increased and made the CPU load on the server look like this:

media.io using pretty much all resources. This screenshot was taken in September 2013.

media.io using pretty much all resources. This screenshot was taken in September 2013.

Things didn't get less hectic so a new server was needed. Yesterday, I switched to a new server using an Intel Core i7-3930K CPU that has 12 instead of 8 cores.

The new server, looking already familiar.

The new server, looking already familiar.

One of the reasons I like Intel i7 CPUs is that they can take a beating. In addition to running flat out most of the time for 343 days without (system) downtime, the old server also moved 470 TB of data around in the old interwebs.

Coming soon: New audio formats

Since the new server is running the latest version of Ubuntu, it's possible to add support for new audio formats that weren't possible on the old server. Support for the Opus Codec will arrive in the next days.

Bug fixes and Performance improvements

Last Friday, a new version of media.io was released. This release contains the following changes:

  • Relative URIs in HTTP redirects now supported. This change will increase the robustness of media.io's URL importing.
  • Support for nginx' X-accel feature. A new servlet was added that forwards the file downloading to nginx. This will ultimately reduce CPU load and free up resources for faster processing of your files.
  • Minor performance improvement. Directories for uploaded and processed files are now only created when you start to upload or import a file. This way, the home page should load a little bit faster (in the low milliseconds range).
  • When importing a URL, the "Select Files to Upload" link sometimes didn't work. This was fixed.

In other news, the media.io online audio conventer converted two million files already in October 2013. This is of course related to the multi-file uploading feature introduced August 2013.

Results of improved URL Importing, massive Reliability upgrades and Donations

Improved URL Importing

In the last update, I mentioned improving media.io's URL import feature. Here are some statistics of how the changes affected the URL importing:

Before Change After Change
URL Imports 3906 2729
Problems while importing 156 (4 %) 78 (2.9 %)
… Timeout connecting 9 (0.2 %) 0
… Timeout reading 30 (0.8 %) 12 (0.4 %)
… Malformed URLs 47 (1.2 %) 0

As you can see, the changes improved the reliability. Unfortunately, the web is full of more-or-less broken services and it will take one or two changes until media.io works perfectly with everyone of them.

Reliability upgrades

media.io frequently had short downtimes that I just couldn't diagnose well. In the last week of December, I used the "opportunity" of a downtime to diagnose the problem and quickly found three areas that weren't bullet-proof yet:

  • Timeouts in URL imports weren't always set. Importing URLs from slowly responding servers would cause all available threads to be used up, blocking everyone else.
  • A "last resort" script wasn't working in all cases. media.io has several cron jobs running to kill zombie processes. One of these scripts wasn't working in all cases.
  • timeout was only using SIGTERM and not SIGKILL. media.io uses the timeout program to ensure that every process started terminates at some point. And, since 2010-07-30, I never noticed that timeout only sends SIGTERM to the process unless -s 9 is used.

These changes seem to work well.

No more Ads but Donations

Since ads suck, there are no more on media.io.

You can now donate to support media.io. Please do so at the conversion page.

Payment providers other than PayPal will be added sooner or later. Please comment if you would like to see a particular service.

More Changes

In addition to the changes above, the following things are also changed:

  • Upload file limit was not used for URL imports. media.io will now tell you if you try to import a URL that's larger than the allowed maximum (currently 750 MB).

 

URL Importing improved and a better design

Less "No Files Found" errors

On 25th of October, I deployed a new version of media.io that reduced the "No Files Found" error by around 18 %. The fix switched the previous cookie-based session handling to URL-based session handling which survives closing and opening the browser.

Improved URL Importing

Between the last post and the most recent release, the URL importing of media.io was improved and made much more robust. The code now handles "almost correct" URLs (for example URLs with un-encoded spaces in them) and is based on a much more robust asynchronous GET-after-HEAD design. Socket and read timeouts were also increased.

Support for .oga Files

media.io can now be used to convert OGA to MP3 and the other supported audio formats (Ogg Vorbis, WAV and WMA).

Performance Optimizations

The amount of data transferred between clients and media.io's server was reduced.

More File Extensions accepted

media.io now accepts files it previously didn't, for example files without a recognized extension but with a dot.

New Design

Between 12th and 30th of November, I ran a split test featuring slightly different designs. The new design looks more modern and increased one performance metric by almost seven percent.