docs.rodeo

MDN Web Docs mirror

HTMLVideoElement: requestVideoFrameCallback() method

{{APIRef("HTML DOM")}} 

The requestVideoFrameCallback() method of the {{domxref("HTMLVideoElement")}}  interface registers a callback function that runs when a new video frame is sent to the compositor. This enables developers to perform efficient operations on each video frame.

Syntax

requestVideoFrameCallback(callback)

Parameters

[!NOTE] width and height may differ from {{domxref("HTMLVideoElement.videoWidth")}}  and {{domxref("HTMLVideoElement.videoHeight")}}  in certain cases (for example, an anamorphic video may have rectangular pixels).

Return value

A number representing a unique callback ID.

This can be passed to {{DOMxRef("HTMLVideoElement.cancelVideoFrameCallback()")}}  to cancel the callback registration.

Description

Typical use cases for requestVideoFrameCallback() include video processing and painting to a canvas, video analysis, and synchronization with external audio sources. Per-frame processing used to be done in a less efficient or accurate fashion by running operations on the current video display whenever the {{domxref("HTMLMediaElement.timeupdate_event", "timeupdate")}}  event fired. This technique did not provide access to the actual video frames.

requestVideoFrameCallback() is used in the same way as {{domxref("Window.requestAnimationFrame()")}} . You use it to run a callback function that performs some operation when the next video frame is sent to the compositor. The callback finishes by calling requestVideoFrameCallback() again to run the callback when the next video frame is composited, and so on. However, requestVideoFrameCallback() is tailored for video operations in several ways:

One thing to bear in mind is that requestVideoFrameCallback() does not offer any strict guarantees that the output from your callback will remain in sync with the video frame rate. It may end up being fired one vertical synchronization (v-sync) later than when the new video frame was presented. (V-sync is a graphics technology that synchronizes the frame rate of a video with the refresh rate of a monitor.)

The API runs on the main thread, while video compositing likely happens on a separate compositing thread. You’ve got to factor in the time taken for these operations to complete, as well as the time it takes for the video itself and the result of your requestVideoFrameCallback() operation to display on the screen.

You can compare the now callback parameter and the expectedDisplayTime metadata property to determine whether your callback is a v-sync late. If expectedDisplayTime is within about five to ten microseconds of now, the frame is already rendered. If the expectedDisplayTime is approximately sixteen milliseconds in the future (assuming your browser/screen is refreshing at 60Hz), then the callback is a v-sync out.

Examples

Drawing video frames on a canvas

This example shows how to use requestVideoFrameCallback() to draw the frames of a video onto a {{htmlelement("canvas")}}  element at exactly the same frame rate as the video. It also logs the frame metadata to the screen for debugging purposes.

const button = document.querySelector("button");
const video = document.querySelector("video");
const canvas = document.querySelector("canvas");
const ctx = canvas.getContext("2d");
const fpsInfo = document.querySelector("#fps-info");
const metadataInfo = document.querySelector("#metadata-info");

button.addEventListener("click", () =>
  video.paused ? video.play() : video.pause(),
);

video.addEventListener("play", () => {
  if (!("requestVideoFrameCallback" in HTMLVideoElement.prototype)) {
    console.error(
      "Your browser does not support the `Video.requestVideoFrameCallback()` API.",
    );
  }
});

let width = canvas.width;
let height = canvas.height;

let paintCount = 0;
let startTime = 0.0;

const updateCanvas = (now, metadata) => {
  if (startTime === 0.0) {
    startTime = now;
  }

  ctx.drawImage(video, 0, 0, width, height);

  const elapsed = (now - startTime) / 1000.0;
  const fps = (++paintCount / elapsed).toFixed(3);
  fpsInfo.innerText = !isFinite(fps) ? 0 : fps;
  metadataInfo.innerText = JSON.stringify(metadata, null, 2);

  video.requestVideoFrameCallback(updateCanvas);
};

video.src = "https://mdn.github.io/shared-assets/videos/flower.mp4";
video.requestVideoFrameCallback(updateCanvas);

video,
canvas {
  max-width: 49%;
}

<p>
  Start <button type="button">⏯</button> playing the video. Pause the video to
  read the metadata. Drawing video frames on the canvas is synced with the
  actual video framerate.
</p>
<video controls playsinline></video>
<canvas width="960" height="540"></canvas>
<p><span id="fps-info">0</span>fps</p>
<pre id="metadata-info"></pre>

{{embedlivesample("drawing_video_frames_on_a_canvas", , "540")}} 

Specifications

{{Specifications}} 

Browser compatibility

{{Compat}} 

See also

In this article

View on MDN