WebP and AVIF conversion pipelines for CMS media

Converting CMS media to WebP and AVIF synchronously during page requests inflates Time to First Byte (TTFB) and destabilizes Core Web Vitals. The reliable pattern moves transformation off the request path: a queue-driven worker encodes variants asynchronously, the CDN caches them, and an edge worker negotiates format at delivery. This eliminates format-mismatch errors and keeps Largest Contentful Paint (LCP) predictable.

Asynchronous Transformation Architecture

CMS platforms expose raw uploads over REST or GraphQL, but transforming synchronously at publish time blocks the request and produces unpredictable delivery latency. Route uploads through a webhook-triggered service instead. On upload, the CMS emits a media.published event carrying the original buffer URL, MIME type, and metadata. A message queue (Redis Streams, AWS SQS, or RabbitMQ) consumes it and dispatches to a stateless worker pool.

This decouples ingestion from delivery, so editorial workflows aren’t gated on encoding latency. The worker normalizes inputs, applies format-specific compression, and writes the resulting URLs back to the CMS asset registry — every variant precomputed, versioned, and cache-ready before the frontend asks for it. It’s the same precompute principle behind your broader Image Optimization Pipelines for CMS Assets.

The encode path (left) runs off the request path; the deliver path (right) negotiates format at the edge:

sequenceDiagram
  participant CMS as CMS
  participant Q as Message queue
  participant W as Worker pool (sharp)
  participant Reg as Asset registry
  participant Edge as Edge worker
  participant Browser as Browser
  CMS->>Q: "media.published (buffer URL, MIME)"
  Q->>W: "dispatch encode job"
  W->>W: "normalize sRGB, encode AVIF + WebP"
  W->>Reg: "write variant URLs + Content-Length"
  Note over CMS,Reg: "Async — editorial not blocked on encoding"
  Browser->>Edge: "GET asset (Accept header)"
  Edge->>Edge: "negotiate avif/webp/jpg"
  Edge->>Browser: "precomputed variant + Vary: Accept"

Encoding Configuration

Server-side conversion needs a memory-safe processing library. sharp, built on libvips, is the standard for Node.js: streaming architecture, SIMD-optimized pipelines. AVIF needs explicit color-space mapping to avoid desaturation on legacy displays; WebP needs tuned subsampling to balance fidelity against payload size.

import sharp, { Sharp, FormatEnum } from 'sharp';

export interface TransformConfig {
  buffer: Buffer;
  targetFormat: 'avif' | 'webp';
  maxWidth?: number;
}

export async function transformMedia(config: TransformConfig): Promise<Buffer> {
  const { buffer, targetFormat, maxWidth = 1920 } = config;

  const transformer: Sharp = sharp(buffer, {
    failOnError: false,
    animated: true,
    limitInputPixels: 268402689, // ~16384x16384 safety limit
  });

  // Normalize to sRGB to prevent color shift across browsers
  const normalized = transformer.resize({ width: maxWidth, withoutEnlargement: true }).toColorspace('srgb');

  if (targetFormat === 'avif') {
    return normalized.avif({
      quality: 75,
      effort: 4,
      chromaSubsampling: '4:2:0',
      lossless: false,
      alphaQuality: 80, // Preserves transparency without inflating payload
    }).toBuffer();
  }

  if (targetFormat === 'webp') {
    return normalized.webp({
      quality: 80,
      nearLossless: false,
      smartSubsample: true,
      alphaQuality: 80,
    }).toBuffer();
  }

  throw new Error(`Unsupported target format: ${targetFormat}`);
}

Configuration notes:

• effort: 4 balances compression ratio against CPU time. Above 6, gains shrink while worker latency climbs.
• alphaQuality: 80 preserves transparency when processing PNGs; AVIF renders opaque if the alpha plane isn’t explicitly kept.
• smartSubsample: true reduces gradient banding in WebP without growing the file — matters for photographic assets.

Content Negotiation and Edge Delivery

Client markup must order MIME types correctly inside <picture>: browsers parse <source> tags in sequence and stop at the first supported format. Mirror that on the server by inspecting Accept before resolving the origin.

<picture>
  <source srcset="/media/hero.avif" type="image/avif">
  <source srcset="/media/hero.webp" type="image/webp">
  <img 
    src="/media/hero.jpg" 
    alt="Hero asset" 
    loading="eager" 
    fetchpriority="high"
    decoding="async"
    width="1920"
    height="1080"
  >
</picture>

To avoid 404s on browsers without modern codec support (Safari < 16.4, older Chromium), run an edge worker that negotiates format proactively.

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    const url = new URL(request.url);
    const acceptHeader = request.headers.get('accept') || '';
    
    // Extract base path and extension
    const pathSegments = url.pathname.split('/');
    const fileName = pathSegments[pathSegments.length - 1];
    const basePath = url.pathname.replace(`/${fileName}`, '');

    let targetFormat = 'jpg';
    if (acceptHeader.includes('image/avif')) {
      targetFormat = 'avif';
    } else if (acceptHeader.includes('image/webp')) {
      targetFormat = 'webp';
    }

    // Rewrite to precomputed variant
    const rewrittenUrl = `${basePath}/${fileName.replace(/\.\w+$/, `.${targetFormat}`)}`;
    const modifiedRequest = new Request(rewrittenUrl, request);

    const response = await fetch(modifiedRequest);
    
    // Enforce long-term caching with immutable directive
    response.headers.set('Cache-Control', 'public, max-age=31536000, immutable');
    response.headers.set('Vary', 'Accept');
    
    return response;
  }
};

The CDN now serves the optimal format with no client-side JavaScript, and Vary: Accept segments the cache per format so a JPEG never leaks to an AVIF-capable client. For the header semantics, see the HTTP Accept header reference and the <picture> element documentation.

Cache Control and SEO

Precomputed variants must integrate with asset duplication and CDN sync. On completion, the pipeline updates the asset registry with deterministic URLs, cache tags, and per-format Content-Length values, which makes targeted cache purges possible when editors update content.

Consistent image delivery feeds Core Web Vitals and crawl efficiency directly. Inject structured metadata alongside transformed assets to automate alt text propagation, width/height injection, and srcset generation — folding cleanly into your broader Localization & SEO Optimization strategy so multilingual routing and format negotiation share one cache policy. Pairing Cache-Control: public, max-age=31536000, immutable with ETag validation, then exposing optimized variants through your sitemap, keeps search indexes aligned with what users actually receive.

Conclusion

A working WebP/AVIF pipeline shifts transformation from synchronous page requests to async, queue-driven workers. With libvips-backed encoders, strict MIME ordering, and edge-level negotiation, you remove format-fallback latency and hold Core Web Vitals steady — deterministic asset delivery that scales across headless deployments.