View Meta’s SAM 2 Model Identify Objects in Videos Using AI
Meta on Monday released a new artificial intelligence (AI) model that can perform complex computer vision tasks. Called Segment Anything Model 2 (SAM 2), it follows up on a predecessor that launched last year and was included in Instagram’s Backdrop and Cutouts tools. The model’s successor now comes with advanced capabilities, and the company said it can perform segment identification and tracking, even on videos. Like most of Meta’s large language models (LLMs), SAM 2 is also an open-source AI model.
In a newsroom afterMeta announced the new AI model that primarily focuses on segment analysis of videos, while improving its image segmentation capabilities. The performance of its predecessorMeta said the AI model has been used in Instagram’s Backdrop and Cutouts features, while marine scientists have used it to “segment sonar images and analyze coral reefs, analyze satellite imagery for disaster relief, and in the medical field to segment cellular images and help detect skin cancer.”
SAM 2 can segment objects in an image and video, and can track them in real-time across multiple frames of a video. The AI can also track and segment objects in scenarios where the objects are moving quickly, changing appearance, or are obscured by other objects or an entirely different scene.
The base model for prompt-based visual segmentation is built on a simple transformer architecture. It has a streaming memory that allows it to process videos in real-time. The company also claimed that the model was trained on its largest video segmentation dataset, called SA-V dataset.
Meta said the AI model could help simplify the process of video editing or AI-based video generation, as well as power new experiences in the company’s mixed-reality ecosystem. The object tracking capability in videos could also help with faster annotation of visual data to train other computer vision systems, the company added.
Since it is an open-source AI model, the company has hosted its weights on its GitHub pageInterested parties can download and test the AI model. It is specifically licensed under the Apache 2.0 license, which allows for research, academic and non-commercial use.
