Drag ‘n drop your photo (and leave the technical details to us)

The Problem

Impressed by the likes of Facebook, Instagram and Twitter, and how they handle image upload, I decided I wanted to offer the same kind of accessible experience for an application I’m working on. Not able to find a turnkey component that met all of the requirements, I decided to built the perfect solution myself (of course, building is the part that’s most fun for a software developer!).

I think the user experience is the most important thing here. Uploading a photo shouldn’t be a hassle at all. Want to drag’n drop a picture from the file system? Shouldn’t be a problem. Rather browse and select an image on your hard drive? Shouldn’t be a problem either. Use Google’s image search and drag from there? Sure, why not!? I mean one should not be hindered by technology… Of course, creating functionality that is that easy to use usually does present a bit of a challenge, but that’s actually the fun part! I find it very rewarding to meet such a challenge and end up with something that works out as designed.

The Requirements

  • Fast
  • Minimal interaction
  • Support for drag ‘n drop
  • Crop at desired resolution

To build something that’s fast, or at least perceived as fast, I implemented background uploading. People are used to uploading images straight from their camera. This gives high resolution and extremely large files. To further speedup the upload, the image is scaled to a lower resolution and cropped on the browser side. When the image manipulation is done, the upload starts. The image sent to the server is as small as possible. During upload, the progress is visible.

Technical Implementation

Mozilla Developer Network has excellent documentation online that got me up and running in no time [1]. The sample code is a little rough, but at least I got my first prototype running real quick.

The first thing to do is handle file input. Either use drag ‘n drop or use the good old file selector [2]. Both are supported. To do image manipulations, modern browsers expose a canvas element [3]. We use the canvas to scale down our image. Basically, the possibilities with the canvas are infinite, it supports all kinds of raw pixel manipulations. Later on for example, we use the canvas to rotate our image.

When image processing is done, the upload is started. To do background uploading, browsers support the XMLHttpRequest object [4]. XMLHttpRequest makes sending HTTP requests very easy.  You simply create an instance of the object, open a URL, and send the request, in our case the image.

iPhone and iPad

From iOS version 6 and up it is possible to upload images straight from mobile Safari, the default browser on iOS. Unfortunately, this is not without a struggle. Due to an error in Safari for iOS, large photos will not display properly [5][6]. Fortunately, it is easy to detect whether the rendering of the image was successful. In case the image does not appear to have been rendered correctly, the image is re-rendered with a fix applied.

Image Orientation Flag

Exchangeable image file format (EXIF) is a standard that specifies a format for tags used by digital cameras, including smartphones [7]. Many digital cameras have a built-in orientation sensor. The output of this sensor is used to set the EXIF orientation flag in the image file’s metatdata to reflect the positioning of the camera with respect to the ground [8]. To respect the image orientation, we have to parse the meta data [9]. Unfortunately it is currently not (yet) possible to write meta data back to the image, so we have to apply the rotation to the image. Fortunately, our canvas elements supports rotation.

Internet Explorer

Internet Explorer is still a challenge. Unfortunately, older versions do not enjoy the same level of user experience. The aim is full support even for older Internet Explorer version. Currently no version other then 10 is supported. Expect some fixes in this area later on…

Key Features

Altogether summarized, the current list of features:

  • Drag ‘n drop support (drag from filesystem and webpage)
  • Awareness of rotation info saved within JPEG (EXIF)
  • Scaling and cropping is performed on the browser side
  • Upload to server in the background
  • Progress indicator shows current upload status

Live Demo

Well, enough text, see the results for yourself in this live demo!

http://sandbox.juurlink.org/html5imageuploader/

References

  1. https://hacks.mozilla.org/2011/01/how-to-develop-a-html5-image-uploader/
  2. https://developer.mozilla.org/en-US/docs/Using_files_from_web_applications
  3. https://developer.mozilla.org/en-US/docs/DOM/HTMLCanvasElement
  4. https://developer.mozilla.org/en-US/docs/DOM/XMLHttpRequest/Using_XMLHttpRequest
  5. http://stackoverflow.com/questions/11929099/html5-canvas-drawimage-ratio-bug-ios
  6. http://stackoverflow.com/questions/12554947/mobile-safari-renders-img-src-dataimage-jpegbase64-scaled-on-canvas
  7. http://en.wikipedia.org/wiki/Exchangeable_image_file_format
  8. http://www.impulseadventure.com/photo/exif-orientation.html
  9. http://stackoverflow.com/questions/7584794/accessing-jpeg-exif-rotation-data-in-javascript-on-the-client-side
  • Facebook
  • LinkedIn

17 thoughts on “Drag ‘n drop your photo (and leave the technical details to us)

  1. Thanks for an excellent library!

    I’ve noticed, however, that the iOS6 squashed-image (subsampling) bug is still apparent in portrait images. The images just get squashed horizontally instead of vertically.

    I’ve tried tweaking your code to detect for orientation and then stretch the width by the same magic number (2.041) that you use for the height, but that produces unexpected results (see code below).

    Any ideas how to fix this to work with iOS6 portrait images?

    Thanks!
    Peter

    // Try to fix IOS6s image squash bug.
    // Test for transparency. This trick only works with JPEGs.
    var iOS = navigator.userAgent.match(/(iPad|iPhone|iPod)/i);
    if (iOS && $img.data(‘fileType’) == ‘image/jpeg’) {
    var transparent = detectTransparency(ctx);
    if (transparent) {
    if(orientation == 6) {
    ctx.drawImage(img, x, y, w * 2.041, h);
    }else{
    ctx.drawImage(img, x, y, w, h * 2.041);
    }
    }
    }

  2. The number 2.041 really is magic, based on trial-and-error. :-)

    Unfortunately vertical iPhone images indeed do render wrong. If you skip the transform step (transformCoordinate method), scaling seems to work just fine. Looks like the transform step ruins the image, but I wasn’t able to figure out the exact problem yet.

    Will have a better look at it and follow up when I have a fix for the problem.

  3. Ahh, I see. Maybe I’ll just handle the rotation server-side, then. Either way, I hope you find a work-around as that will really make this a killer solution. Thanks! PK

  4. Hi Rob
    I have a WordPress project where i add a image to a post with a mobile device (iPhone, iPad) using the browser. I have this in place but i want to optimize the upload time by resizing the photo taken using canvas and then upload the resized image.
    Do you think it is possible to use your code here in a WordPress environment uploading the smaller file and inserting the url to the database?

  5. Mitchel
    Yes i know but in my project the user uploads the image in the front end. This is in place. Now i want to optimize upload time by using canvas. Ex 1.5MB photo from iPhone -> canvas resize -> 300 kb photo upload instead.

    • Torbjörn,

      This is not a requirement we have had with wordpress @ Avisi so unfortunately I can’t help you with that :(

  6. Thanks for sharing this code. I’ve noticed that on your demo site and my own local implementation that the cropped image that is displayed is always of image/png type and not a jpeg. I cannot see in the code where is this happening. I’ve reproduced it on Safari, Chrome and Firefox for Mac, and Chrome for Windows. Any ideas?

  7. I’m sorry for my lack of the basic knowledge needed here, but, I couldn’t find in the demo:

    ? where can I add my ftp server address/folder to upload the image. Some one could give me a hint, please? Thank you in advance!

    …and, thank you Rob to share this clean upload form with us!

      • Mitchel, thank you to reply!
        So, I did changed the postUrl address to my ftp address, but it still doesn’t work. I give me a “Image upload failed:” message error. May the problem is because my server is set to permission 777, and I can’t change this due some rules in this rent server. So, the way I usually do is to add a ftp+user+password address, and this has not worked. Do you have any another hint?

  8. Hey,

    I am unable to upload the picture to the server. Where can i specify the path? Do I need to implement upload.php or does your upload.php save the file on the server? Please help as soon as possible.

    • Hi

      The uploaded image is outputted via upload.php as a stream of data. You need to write this data to a new file. See below:

  9. Great sample ! Thanks
    Still we have tried it (http://sandbox.juurlink.org/html5imageuploader/) on around 50 devices and for Android – some worked and some did not work. It caused the browser to crash or displayed memory issues message. iphone all work well ..
    It might also be on the same device sometime work OK and some time not

    Any recommendations around that?

  10. if you want to avoid some frustration and google searching be sure to use a binary safe file handler in upload.php.. This really screwed me up for a minute

    //notice rb instead of r
    $handle = fopen(“php://input”, “rb”);

Leave a Reply to Rob Juurlink Cancel reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>