Efficient-CLS

Open source Python

GitHub

Stars

Forks

Issues

Watchers

2 years

Last Commit

Browser Automation ML Frameworks Video Streaming & Transcoding

About Efficient-CLS

Efficient-CLS is an official implementation of the ICCV 2023 paper focused on label-efficient online continual object detection in streaming video. Developed by researchers from Show Lab at NUS and other institutions, this software addresses the challenge of detecting objects in continuous video streams while minimizing the need for manual data annotation. The system leverages pseudo-labeling and replay mechanisms to train continuously on unlabeled data, significantly reducing the computational and human-labeling costs associated with traditional object detection models. Built upon Facebook Research's Detectron2 framework, the codebase utilizes Faster R-CNN architectures pre-trained on PASCAL VOC datasets. It supports evaluation on benchmark datasets such as OAK and EgoObjects, allowing users to configure experiments with varying annotation budgets and pseudo-label ratios. Key features include streaming video analysis, support for incremental learning scenarios, and flexible configuration for replay sizes and

Platforms

Web Self-hosted

Languages

Python

Links

Source Code

Published by

showlab

Visit View Profile

README.md

View on GitHub

Effcient-CLS

This repository is the official implementation of the following paper:

Label-Efficient Online Continual Object Detection in Streaming Video
Jay Zhangjie Wu, David Junhao Zhang, Wynne Hsu, Mengmi Zhang, Mike Zheng Shou

Setup

Installation

Clone the repository and install the dependencies:

git clone https://github.com/showlab/Efficient-CLS.git
pip install Efficient-CLS/requirements.txt
python -m pip install -e Efficient-CLS

Datasets

We provide the processed datasets in the Google Drive (OAK, EgoObjects). Download the datasets and modify the DATA_DIR in configs/efficient_cls.yaml to the corresponding directory.

Pretrained Models

We use Faster R-CNN on PASCAL VOC object detection. Run the following commands to download the pretrained weights in Detectron2 Model Zoo.

mkdir weights && wget https://dl.fbaipublicfiles.com/detectron2/PascalVOC-Detection/faster_rcnn_R_50_C4/142202221/model_final_b1acc2.pkl -P weights/

Usage

To start training, run this:

# E.g., run experiment on OAK dataset at 4/16 annotation cost, with 12/16 unlabeled data trained with pseudo labels.
python train.py --exp=train --dataset=oak --num_oracle=4 --num_pseudo=12 --replay_size=16

Shoutouts

This code builds on detectron2. Thanks for opensourcing!
Thanks the contributors of OAK and EgoObjects for sharing the datasets!