Bug fixes and Performance improvements

Last Friday, a new version of media.io was released. This release contains the following changes:

  • Relative URIs in HTTP redirects now supported. This change will increase the robustness of media.io's URL importing.
  • Support for nginx' X-accel feature. A new servlet was added that forwards the file downloading to nginx. This will ultimately reduce CPU load and free up resources for faster processing of your files.
  • Minor performance improvement. Directories for uploaded and processed files are now only created when you start to upload or import a file. This way, the home page should load a little bit faster (in the low milliseconds range).
  • When importing a URL, the "Select Files to Upload" link sometimes didn't work. This was fixed.

In other news, the media.io online audio conventer converted two million files already in October 2013. This is of course related to the multi-file uploading feature introduced August 2013.

Results of improved URL Importing, massive Reliability upgrades and Donations

Improved URL Importing

In the last update, I mentioned improving media.io's URL import feature. Here are some statistics of how the changes affected the URL importing:

Before Change After Change
URL Imports 3906 2729
Problems while importing 156 (4 %) 78 (2.9 %)
… Timeout connecting 9 (0.2 %) 0
… Timeout reading 30 (0.8 %) 12 (0.4 %)
… Malformed URLs 47 (1.2 %) 0

As you can see, the changes improved the reliability. Unfortunately, the web is full of more-or-less broken services and it will take one or two changes until media.io works perfectly with everyone of them.

Reliability upgrades

media.io frequently had short downtimes that I just couldn't diagnose well. In the last week of December, I used the "opportunity" of a downtime to diagnose the problem and quickly found three areas that weren't bullet-proof yet:

  • Timeouts in URL imports weren't always set. Importing URLs from slowly responding servers would cause all available threads to be used up, blocking everyone else.
  • A "last resort" script wasn't working in all cases. media.io has several cron jobs running to kill zombie processes. One of these scripts wasn't working in all cases.
  • timeout was only using SIGTERM and not SIGKILL. media.io uses the timeout program to ensure that every process started terminates at some point. And, since 2010-07-30, I never noticed that timeout only sends SIGTERM to the process unless -s 9 is used.

These changes seem to work well.

No more Ads but Donations

Since ads suck, there are no more on media.io.

You can now donate to support media.io. Please do so at the conversion page.

Payment providers other than PayPal will be added sooner or later. Please comment if you would like to see a particular service.

More Changes

In addition to the changes above, the following things are also changed:

  • Upload file limit was not used for URL imports. media.io will now tell you if you try to import a URL that's larger than the allowed maximum (currently 750 MB).

 

URL Importing improved and a better design

Less "No Files Found" errors

On 25th of October, I deployed a new version of media.io that reduced the "No Files Found" error by around 18 %. The fix switched the previous cookie-based session handling to URL-based session handling which survives closing and opening the browser.

Improved URL Importing

Between the last post and the most recent release, the URL importing of media.io was improved and made much more robust. The code now handles "almost correct" URLs (for example URLs with un-encoded spaces in them) and is based on a much more robust asynchronous GET-after-HEAD design. Socket and read timeouts were also increased.

Support for .oga Files

media.io can now be used to convert OGA to MP3 and the other supported audio formats (Ogg Vorbis, WAV and WMA).

Performance Optimizations

The amount of data transferred between clients and media.io's server was reduced.

More File Extensions accepted

media.io now accepts files it previously didn't, for example files without a recognized extension but with a dot.

New Design

Between 12th and 30th of November, I ran a split test featuring slightly different designs. The new design looks more modern and increased one performance metric by almost seven percent.

More features and less bugs

Yesterday, I rolled out a new version of media.io with the following improvements:

More Concurrency

In September, media.io had a few downtimes caused by what turned out to be resource exhaustion bugs. I have changed some of the internals but I have found more areas to optimize so the work isn't done yet.

Multi-file upload for everybody

The multi-file upload feature introduced in early August was made available to all users.

More Reliability

With the number of people using media.io increasing every day, I brought back media.io's disk overflow code. This prevents disks from filling up.

I also switched from MPlayer to FFmpeg to identify .caf (Apple's CoreAudio) files since MPlayer ran into infinite loops on one occasion. 

Serbian translation

Translator Ivan Živković contributed a Serbian translation. Thank you!

Less Bugs

After rolling out the multi-file upload feature, I was notified of a problem which affected approximately 5 % of all visitors. It was caused by links to servlets on different ports not being URL rewritten. This bug was fixed.

Happy birthday, media.io!

Today, media.io is four years old.

It's been a very exciting (and sometimes frustrating) time for me from launching media.io in 2009 to running a web application used by 500.000 people  every month.

Still, I think there's plenty of room so I hope you all will stick with media.io.

More files than ever before

August also marked another milestone for media.io: For the first time, media.io users uploaded or imported more than 1 million files.

Your stories

I receive a lot of stories from people using media.io and the ones I like the most are stories of media.io helping people with urgent deadlines.

How has media.io helped you? Why not post in the comments?

New media.io with new and awesome features

Lots of things have changed since the last blog post.

6th of August, I rolled out the first build of the new media.io, a huge development effort aimed at adding some often-requested features to media.io.

Since then, two new versions of media.io have already been rolled out.

Right now, not all visitors will see the new version -- media.io is set to show 50 % of the visitors the new version and a version similar to the previous one to the other half. 

New Features and Bug Fixes

  • Multi-file uploading. The single most-requested feature, filed only one month after the launch of media.io. Using the excellent Plupload library, everybody can now upload multiple files. And there is immediate feedback for unsupported or overly large files.
  • URL Importing. This feature was requested a lot. While not everybody sees this right now, and no, media.io does not convert YouTube to MP3, there's a good chance this feature will soon be available to everybody.
  • More smartness. media.io now detects if connections to port 8080 are disabled and when some waiting in line is inevitable.
  • Support for files ending on _.  Yes, these files are around.
  • Progress display on the conversion page. For larger files, a progress bar is shown that displays how much of the file has been converted already. Note that for particular reasons, it doesn't work for all conversions.
  • Better upload progress. media.io now shows more robust upload progress numbers, as well as blurring any links on the page to avoid accidental clicks interrupting the upload.
  • Deeper integration with Google Analytics. This will alert me of problems and also help me evaluate the performance of media.io much better.
  • Better performance. A simple jQuery trick helped reduce the time before the conversion progress is shown by over 50 %. 
  • Handling of zero-byte files. Visitors frequently try to upload empty files and then wonder why media.io can't convert the files. media.io will now show you that it cannot convert zero-byte files.

That's it for this time. If you like media.io, why not like media.io on Facebook or suggest a feature ?

Conversion progress

Conversion progress

New server plus higher success rates for .m4b files

media.io moves servers

Due to pricing changes of its previous hosting company, media.io had to find a new host. media.io is now back at Core-Backbone  but  on a new server that is 30 % more powerful than the previous one, has  twice the disk I/O performance than the first one (which was retired late 2012) and also has a GBit/s uplink.

This means your files will upload, convert and download faster with (fingers crossed) less downtimes. 

 .m4b file handling changes

A while ago, I changed the handling of .m4b files in response to a user suggestion. These changes improved the success rates of .m4b files:

Before (March 2013): 139 of 252 files converted successfully (55 %).

After (May 2013): 587 of 759 files converted successfully (77 %).

As you can see, this change improved the .m4b to MP3 conversion quality at media.io.

8 reasons why the new media.io will be awesome

I've been working on the next big version of media.io for about three months and I'm happy to say that most of the work is now done.

Here are some reasons why I think the next version will be much better than media.io ever was:

Multi-file uploading

The single most-requested feature will finally be available: Multi-file uploading, along with drag&drop. Powered by Plupload.

Better error handling

Some edge cases were never really handled well by media.io.  For example, the new media.io handles overly large files much better.

(Hopefully) even faster

I have added some tweaks that hopefully increase performance. Some of these tweaks were rotated into some of my other sites, for example my YouTube to MP3 converter and my YouTube to MP4 converter.

Re-uploading of existing files

In the new media.io, you can re-upload files you've already uploaded. This will be a first step towards enabling re-conversion of already uploaded files.

No more "stalling"

Stalling detection is a feature that I introduced long ago to handle stalled uploads, i.e. situations in which no data is received by the server for a while.

As it just turns out, my idea of stalling may have been a bit aggressive. The new media.io does not have stalling detection.

Better diagnostics

I will have Google Analytics tracking for many more actions, including error cases. For example, I will be able to see the behaviour of visitors using the new multi-file upload form vs. the old one. Also, I'm looking to measure how long it takes between a finished upload and the first display of the files list.

Ultimately, this data will help me gauge the effectiveness of changes and move beyond my two KPIs, Net Promoter Score and download conversion rate. 

URL Importing

Another often-requested feature, the new media.io will let you import URLs. 

Saved settings

The new media.io will save your conversion settings for less clicking.  The settings are also saved for 60 days.

The new media.io is scheduled to be rolled out mid to late June. Do you have anything you would love to see in the new version of media.io?  Even if you don't, please try the new media.io, report bugs and like media.io on Facebook. Thanks!

New Audio Formats and better handling for .m4b files

I have released a new version of media.io yesterday. Between the last post and this release, the following things have changed:

New Audio Formats

The following audio formats are now supported:

  • .w64 files – Sony Wave64.
  • .act files – ACT Voice.

Bug Fixes

The following bugs have been fixed:

  • Security vulnerability through path traversal. By trying specific URLs, media.io's configuration could be obtained, which included local paths. At no point was any visitor data available. This vulnerability has been closed.
  • Better conversion for .m4b files. This user suggestion caused me to investigate the relatively low (~50 %) success rates for .m4b files. A new handling is now in place that should increase rates when converting .m4b to .mp3.

Other Things

I ran a test of the Plupload library to get an idea of the browser compatibility, the results of which are summarized here. Of course, Plupload is a much better solution to file uploading.

In addition, since Posterous will be closing down sonn, media.io's blog is now hosted by SquareSpace.