Link to paper

The full paper is available here.

You can also find the paper on PapersWithCode here.

Abstract

  • Computer vision can automate the analysis of raw imagery from sensors.
  • Computer vision is rarely taught to ecologists.
  • This paper discusses the experience of teaching a diverse group of ecologists to prototype and evaluate computer vision systems in a summer workshop.

Paper Content

Introduction

  • Manual annotation of images and videos is time consuming
  • Computer vision algorithms can automate this process
  • Computer vision is important for ecology due to climate change
  • Ecologists need to understand and apply computer vision methods
  • Teaching machine learning, deep learning, and computer vision to computer science students
  • Teaching machine learning to cross-disciplinary audiences such as non-CS undergraduates, business students, artists, materials scientists, and biologists
  • Teaching computer vision to ecologists with background knowledge in statistics and programming
  • Workshop in which researchers build prototypes using their own research data, not a traditional classroom environment

The cv4e workshop

  • CV4E Workshop held at Caltech from August 1-19, 2022
  • Program designed to train ecologists to use computer vision
  • Application process included project proposal, personal statement, programming example, letter of reference, and CV
  • Selection process included reviews from machine learning and ecology communities
  • All participants funded for travel, room, and board
  • Participants met with instructors to finalize project plans and learn Python
  • Lectures, Invited Speakers, Reading Groups, Work Time, and Group Updates during workshop
  • All 18 participants had trained models by end of workshop
  • Slack workspace and ongoing projects formed community beyond workshop

Lessons learned

  • Enforce structured Python preparation before workshop
  • Start simple when building computer vision system
  • Collect similar projects in working groups
  • Mix experience levels in working groups
  • Make unambiguous infrastructure recommendations
  • Avoid deep learning library wrappers
  • Avoid Jupyter Notebooks
  • Make sure GPUs are available

Educational techniques

  • Guided troubleshooting to hone machine learning skills
  • Pair pseudocoding to help participants write code
  • Goal statements to help participants track progress
  • Contextualized lectures to keep topics grounded in applications

Conclusion

  • 18 participants in the inaugural 2022 CV4E Workshop
  • 11 participants from 7 different states in the US, 5 from European countries, 2 from Canada
  • Participants from diverse academic backgrounds
  • 7 main categories of projects
  • Using annotation tools to label image or audio data
  • Common Unix commands
  • Tools like nano, tmux, and screen
  • SSH command and SSH keys
  • Tools like scp or rsync for transferring files
  • Using GitHub for tracking changes made to code
  • Interacting with web interfaces of cloud computing providers
  • Creating and managing virtual environments
  • Libraries and command-line tools like OpenCV, ImageMagick, and FFmpeg
  • Classes and objects, inheritance, encapsulation, polymorphism
  • Common data structures and their methods
  • Mutable and immutable objects
  • In-place operations
  • Concept of generalization
  • Errors vs. warnings
  • Types of errors
  • Simplified dependency graph of skills required to develop a computer vision system
  • Reading documentation, finding code, understanding components of a codebase
  • Recognizing markers of quality in open source code