Image Resizing Techniques

Since time immemorial, iOS developers have been perplexed by a singular question: "How do you resize an image?". It is a question of beguiling clarity, spurred on by a mutual mistrust of developer and platform. A thousand code samples litter web search results, each claiming to be the One True Solution, and all the others false prophets.

It's embarrassing, really.

This week's article endeavors to provide a clear explanation of the various approaches to image resizing on iOS (and OS X, making the appropriate UIImage→ NSImage conversions), using empirical evidence to offer insights into the performance characteristics of each approach, rather than simply prescribing any one way for all situations.

Before reading any further, please note the following:

When setting a UIImage on a UIImageView, manual resizing is unnecessary for the vast majority of use cases. Instead, one can simply set the contentMode property to either .ScaleAspectFit to ensure that the entire image is visible within the frame of the image view, or .ScaleAspectFill to have the entire image view filled by the image, cropping as necessary from the center.

imageView.contentMode=.ScaleAspectFitimageView.image=image

Determining Scaled Size

Before doing any image resizing, one must first determine the target size to scale to.

Scaling by Factor

The simplest way to scale an image is by a constant factor. Generally, this involves dividing by a whole number to reduce the original size (rather than multiplying by a whole number to magnify).

A new CGSize can be computed by scaling the width and height components individually:

letsize=CGSizeMake(image.size.width/2.0,image.size.height/2.0)

...or by applying a CGAffineTransform:

letsize=CGSizeApplyAffineTransform(image.size,CGAffineTransformMakeScale(0.5,0.5))

Scaling by Aspect Ratio

It's often useful to scale the original size in such a way that fits within a rectangle without changing the original aspect ratio. AVMakeRectWithAspectRatioInsideRect is a useful function found in the AVFoundation framework that takes care of that calculation for you:

importAVFoundationletsize=AVMakeRectWithAspectRatioInsideRect(image.size,imageView.bounds)

Resizing Images

There are a number of different approaches to resizing an image, each with different capabilities and performance characteristics.

UIGraphicsBeginImageContextWithOptions& UIImage -drawInRect:

The highest-level APIs for image resizing can be found in the UIKit framework. Given a UIImage, a temporary graphics context can be used to render a scaled version, using UIGraphicsBeginImageContextWithOptions() and UIGraphicsGetImageFromCurrentImageContext():

letimage=UIImage(contentsOfFile:self.URL.absoluteString!)letsize=CGSizeApplyAffineTransform(image.size,CGAffineTransformMakeScale(0.5,0.5))lethasAlpha=falseletscale:CGFloat=0.0// Automatically use scale factor of main screenUIGraphicsBeginImageContextWithOptions(size,!hasAlpha,scale)image.drawInRect(CGRect(origin:CGPointZero,size:size))letscaledImage=UIGraphicsGetImageFromCurrentImageContext()

UIGraphicsBeginImageContextWithOptions() creates a temporary rendering context into which the original is drawn. The first argument, size, is the target size of the scaled image. The second argument, isOpaque is used to determine whether an alpha channel is rendered. Setting this to false for images without transparency (i.e. an alpha channel) may result in an image with a pink hue. The third argument scale is the display scale factor. When set to 0.0, the scale factor of the main screen is used, which for Retina displays is 2.0.

CGBitmapContextCreate& CGContextDrawImage

Core Graphics / Quartz 2D offers a lower-level set of APIs that allow for more advanced configuration. Given a CGImage, a temporary bitmap context is used to render the scaled image, using CGBitmapContextCreate() and CGBitmapContextCreateImage():

letimage=UIImage(contentsOfFile:self.URL.absoluteString!).CGImageletwidth=CGImageGetWidth(image)/2.0letheight=CGImageGetHeight(image)/2.0letbitsPerComponent=CGImageGetBitsPerComponent(image)letbytesPerRow=CGImageGetBytesPerRow(image)letcolorSpace=CGImageGetColorSpace(image)letbitmapInfo=CGImageGetBitmapInfo(image)letcontext=CGBitmapContextCreate(nil,width,height,bitsPerComponent,bytesPerRow,colorSpace,bitmapInfo)CGContextSetInterpolationQuality(context,kCGInterpolationHigh)CGContextDrawImage(context,CGRect(origin:CGPointZero,size:CGSize(width:CGFloat(width),height:CGFloat(height))),image)letscaledImage=UIImage(CGImage:CGBitmapContextCreateImage(context))

CGBitmapContextCreate takes several arguments to construct a context with desired dimensions and amount of memory for each channel within a given colorspace. In the example, these values are fetched from the CGImage. Next, CGContextSetInterpolationQuality allows for the context to interpolate pixels at various levels of fidelity. In this case, kCGInterpolationHigh is passed for best results. CGContextDrawImage allows for the image to be drawn at a given size and position, allowing for the image to be cropped on a particular edge or to fit a set of image features, such as faces. Finally, CGBitmapContextCreateImage creates a CGImage from the context.

CGImageSourceCreateThumbnailAtIndex

Image I/O is a powerful, yet lesser-known framework for working with images. Independent of Core Graphics, it can read and write between between many different formats, access photo metadata, and perform common image processing operations. The framework offers the fastest image encoders and decoders on the platform, with advanced caching mechanisms and even the ability to load images incrementally.

CGImageSourceCreateThumbnailAtIndex offers a concise API with different options than found in equivalent Core Graphics calls:

importImageIOifletimageSource=CGImageSourceCreateWithURL(self.URL,nil){letoptions=[kCGImageSourceThumbnailMaxPixelSize:max(size.width,size.height)/2.0,kCGImageSourceCreateThumbnailFromImageIfAbsent:true]letscaledImage=UIImage(CGImage:CGImageSourceCreateThumbnailAtIndex(imageSource,0,options))}

Given a CGImageSource and set of options, CGImageSourceCreateThumbnailAtIndex creates a thumbnail image. Resizing is accomplished by the kCGImageSourceThumbnailMaxPixelSize. Specifying the maximum dimension divided by a constant factor scales the image while maintaining the original aspect ratio. By specifying either kCGImageSourceCreateThumbnailFromImageIfAbsent or kCGImageSourceCreateThumbnailFromImageAlways, Image I/O will automatically cache the scaled result for subsequent calls.

Lanczos Resampling with Core Image

Core Image provides a built-in Lanczos resampling functionality with the CILanczosScaleTransform filter. Although arguably a higher-level API than UIKit, the pervasive use of key-value coding in Core Image makes it unwieldy.

That said, at least the pattern is consistent. The process of creating a transform filter, configuring it, and rendering an output image is just like any other Core Image workflow:

letimage=CIImage(contentsOfURL:self.URL)letfilter=CIFilter(name:"CILanczosScaleTransform")filter.setValue(image,forKey:"inputImage")filter.setValue(0.5,forKey:"inputScale")filter.setValue(1.0,forKey:"inputAspectRatio")letoutputImage=filter.valueForKey("outputImage")asCIImageletcontext=CIContext(options:nil)letscaledImage=UIImage(CGImage:self.context.createCGImage(outputImage,fromRect:outputImage.extent()))

CILanczosScaleTransform accepts an inputImage, inputScale, and inputAspectRatio, all of which are pretty self-explanatory. A CIContext is used to create a UIImage by way of a CGImageRef intermediary representation, since UIImage(CIImage:) doesn't often work as expected.

Performance Benchmarks

So how do these various approaches stack up to one another?

Here are the results of a set of performance benchmarks done on an iPod Touch (5th Generation) running iOS 8.0 GM, using XCTestCase.measureBlock():

JPEG

Scaling a large, high-resolution (12000 ⨉ 12000 px 20 MB JPEG) source image from NASA Visible Earth to 1/10^th the size:

PNG

Scaling a reasonably large (1024 ⨉ 1024 px 1MB PNG) rendering of the Postgres.app Icon to 1/10^th the size:

^{1, 5} Results were consistent across different values of CGInterpolationQuality, with negligible differences in performance benchmarks.

² The high standard deviation reflects the cost of creating the cached thumbnail, which was comparable to the performance of the equivalent Core Graphics function.

³ Creating a CIContext is an extremely expensive operation, and accounts for most of the time spent in the benchmark. Using a cached instance reduced average runtime down to metrics comparable with UIGraphicsBeginImageContextWithOptions.

^{4, 7} Setting kCIContextUseSoftwareRenderer to true on the options passed on CIContext creation yielded results an order of magnitude slower than base results.

Conclusions

• UIKit, Core Graphics, and Image I/O all perform well for scaling operations on most images.
• Core Image is outperformed for image scaling operations. In fact, it is specifically recommended in the Performance Best Practices section of the Core Image Programming Guide to use Core Graphics or Image I/O functions to crop or downsample images beforehand.
• For general image scaling without any additional functionality, UIGraphicsBeginImageContextWithOptions is probably the best option.
• If image quality is a consideration, consider using CGBitmapContextCreate in combination with CGContextSetInterpolationQuality.
• When scaling images with the intent purpose of displaying thumbnails, CGImageSourceCreateThumbnailAtIndex offers a compelling solution for both rendering and caching.

NSMutableHipster

This is a good opportunity to remind readers that NSHipster articles are published on GitHub. If you have any corrections or additional insights to offer, please open an issue or submit a pull request.