Jul 15, 2019

Improving image rendering using ImageIO

Displaying images has become standard practice for many categories of apps. But if images rendered in applications are not pre-processed properly, it can have a negativ impact on performance and therefor the user experience. So let’s look at how we can optimize images before we pass them on the the image property of a UIImageView.

For the sake of this example we will assume the application downloads the image data from a service and then uses that data to create a UIImage from it. The easiest way would be to initialize the UIImage from the data directly (UIImage(data: imageData)) and pass it on to the UIImageView. But this can cause performance problems if the image data is not optimized for the size of the UIImageView and resolution of the user’s device. Decoding an image that is 10MB in size can use up to ten times of that in memory. But with proper optimzations before decoding, it will only be around twice if its compressed size. Thankfully ImageIO exposes some APIs that will make this quite easy for us.

Let’s start creating a new class ImageView by subclassing UIImageView. This will make our code reusable and easier to manage. Next we add a new attribute to which we can pass the compressed image data to.

class ImageView: UIImageView {
  
  // image data
  var data: Data?
}

When passing new image data to the attribute data, we need to first create a CGImageSource from that data by passing it to the initializer CGImageSourceCreateWithData(data:, options:).

guard let data = data, let source = CGImageSourceCreateWithData(data as CFData, nil) else {
  image = nil
  return
}

This will enable us to create a CGImage thumbnail from that source which has the correct pixel size we need for our ImageView. For that we must pass a set of options when initializing it.

// get size of image view
let size = bounds.size
// setup option for creating CGImage from source
let options = [
  kCGImageSourceCreateThumbnailFromImageAlways: true,
  kCGImageSourceCreateThumbnailWithTransform: true,
  kCGImageSourceShouldCacheImmediately: true,
  kCGImageSourceThumbnailMaxPixelSize: max(size.width, size.height)
  ] as CFDictionary
// create CGImage from source
let cgImage = CGImageSourceCreateThumbnailAtIndex(source, 0, options)

Let’s look at what these options mean:

  1. kCGImageSourceCreateThumbnailFromImageAlways: true Always forces the creation of a thumbnail from the full image, even if there is already a thumbnail present in the source file. This is important because the present thumbnail might not be the size we want.
  2. kCGImageSourceCreateThumbnailWithTransform: true: Rotates and scales the thumbnail image to match the orientation and pixel aspect ratio of the full image.
  3. kCGImageSourceShouldCacheImmediately: true: Forces the image to be decoded and cached at creation time instead of rendering time which will increase our rendering performance.
  4. kCGImageSourceThumbnailMaxPixelSize: max(size.width, size.height): Specifies the maximum pixel size the image should have. We must set this to ensure we don’t create an image that is larger than the ImageView and is required in combination with kCGImageSourceCreateThumbnailFromImageAlways when set to true.

Now we can initialize the UIImage by passing the newly created CGImage and assign it to the image attribute in our subclass. The last optimization step left now is to move the CGImage creation to a background thread and only move back to the main thread when assigning the image itself. This will free up the main thread to do other things.

// image data
var data: Data? {
  didSet {
    // render image
    renderImage()
  }
}
  
private func renderImage() {
  // get image source from data
  guard let data = data, let source = CGImageSourceCreateWithData(data as CFData, nil) else {
    image = nil
    return
  }
  // get size of image view
  let size = bounds.size
  // move to asynchronous thread for creating image
  DispatchQueue.global(qos: .userInitiated).async { [weak self] in
    // setup option for creating CGImage from source
    let options = [
      kCGImageSourceCreateThumbnailFromImageAlways: true,
      kCGImageSourceCreateThumbnailWithTransform: true,
      kCGImageSourceShouldCacheImmediately: true,
      kCGImageSourceThumbnailMaxPixelSize: max(size.width, size.height)
      ] as CFDictionary
    // create CGImage from source
    if let cgImage = CGImageSourceCreateThumbnailAtIndex(source, 0, options) {
      // create UIImage from CGImage
      let image = UIImage(cgImage: cgImage)
      // move to main thread for actually rendering image
      DispatchQueue.main.async { [weak self] in
        guard let strongSelf = self else {
          return
        }
        // assign and animate in image by cross-dissolving
        UIView.transition(with: strongSelf, duration: 0.2, options: .transitionCrossDissolve, animations: { [weak self] in
          self?.image = image
          }, completion: nil)
      }
    }
  }
}

With these few steps we can now render images in a very performant way. Try it out in a UITableViewCell and see how smooth scrolling will be while simultaniously loading and rendering images.

You can view the complete code here.

Language: Swift 5.0 ยท Written on: iPad Pro