[summary] Dual-Camera Human Detection and Face Recognition System with Simultaneous Video Recording

Project Short Details

Title: Dual-Camera Human Detection and Face Recognition System

Description: This project aims to develop a real-time human detection and face recognition system using dual cameras—a laptop's built-in camera and an external USB camera. The system captures video from both cameras simultaneously, detects faces in the frames, and saves images of detected faces along with video recordings. The saved files are organized into separate folders based on the camera used, allowing for easy access and management. The system also employs face recognition to ensure that images of the same person are captured only if a specific time interval has elapsed, thus optimizing storage and processing.

Project Targets

1. Simultaneous Video Capture:

   • Implement a system to capture video streams from both the laptop's built-in camera and the USB camera concurrently.

2. Real-Time Face Detection:

   • Utilize Haar cascade classifiers for detecting faces in the video frames from both cameras.

3. Face Recognition:

   • Employ the face_recognition library to recognize faces, comparing them against previously detected faces to identify new individuals.

4. Efficient Image Saving:

   • Save images of detected faces only if a minimum time interval (2 minutes) has passed since the last image was captured for the same person, thereby optimizing storage.

5. Organized File Management:

   • Create a structured file storage system, with separate directories for videos and photos from each camera, named with timestamps for easy identification.

6. User Interface:

   • Provide a simple display for the video feeds from both cameras, allowing users to monitor the detection process in real time.

7. Clean Resource Management:

   • Ensure proper release of camera resources and cleanup of all processes after the application is terminated.

Target Outcomes

• Successfully build a dual-camera face detection and recognition system that operates in real time.
• Demonstrate the ability to manage and process multiple video streams efficiently.
• Showcase the capability to save and organize multimedia files effectively while minimizing redundancy in captured images.

-----------------------------------------------------------------

Here’s a detailed explanation of the final code, which involves capturing video from both the laptop's primary camera and a USB camera simultaneously, performing face detection and recognition, and saving both video and face-detected images in separate folders for each camera.

---

1. Library Imports

python
import cv2
import datetime
import os
import time
import face_recognition

• cv2 (OpenCV): Used for capturing video, detecting faces, and saving video files and images.
• datetime: Helps generate timestamps to create unique filenames for the videos and images.
• os: Used to create directories and manage file paths dynamically.
• time: Provides functions to work with time intervals (for controlling the frequency of image capture).
• face_recognition: Allows for face encoding and comparing faces for recognition.

---

2. Loading Face Detection Model

python
face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

• The Haar cascade model is pre-trained to detect faces. It is used here to quickly identify faces in frames from both cameras.

---

3. Initialize Video Capture

python
cap_primary = cv2.VideoCapture(0)  # Primary laptop camera
cap_usb = cv2.VideoCapture(1)      # USB camera

• cv2.VideoCapture(0): Initializes the primary camera (typically, the laptop's built-in camera).
• cv2.VideoCapture(1): Initializes the USB camera (external webcam).

Both cameras will capture video frames at the same time.

---

4. Creating Directories for Videos and Photos

python
now = datetime.datetime.now()
date_str = now.strftime("%Y%m%d")  # Format: YYYYMMDD
video_folder_primary = os.path.join(date_str, 'videos_primary_camera')
video_folder_usb = os.path.join(date_str, 'videos_usb_camera')
photo_folder_primary = os.path.join(date_str, 'photos_primary_camera')
photo_folder_usb = os.path.join(date_str, 'photos_usb_camera')
os.makedirs(video_folder_primary, exist_ok=True)
os.makedirs(video_folder_usb, exist_ok=True)
os.makedirs(photo_folder_primary, exist_ok=True)
os.makedirs(photo_folder_usb, exist_ok=True)

• The code creates separate directories for:

  • Videos and photos from the primary camera.
  • Videos and photos from the USB camera.

• Each directory is timestamped using the current date (YYYYMMDD), ensuring that all recordings and images are saved in a structured manner.

---

5. Setting Up Video Recording

python
fourcc = cv2.VideoWriter_fourcc(*'XVID')

# For primary camera
video_filename_primary = f'primary_video_{now.strftime("%Y%m%d_%H%M%S")}.avi'
video_path_primary = os.path.join(video_folder_primary, video_filename_primary)
out_primary = cv2.VideoWriter(video_path_primary, fourcc, 20.0, (640, 480))

# For USB camera
video_filename_usb = f'usb_video_{now.strftime("%Y%m%d_%H%M%S")}.avi'
video_path_usb = os.path.join(video_folder_usb, video_filename_usb)
out_usb = cv2.VideoWriter(video_path_usb, fourcc, 20.0, (640, 480))

• VideoWriter Objects are created for each camera:

  • out_primary: Writes the video captured from the primary camera to a .avi file.
  • out_usb: Writes the video captured from the USB camera to a different .avi file.

• Each video file is named with a timestamp (primary_video_YYYYMMDD_HHMMSS.avi and usb_video_YYYYMMDD_HHMMSS.avi) and saved in their respective directories.

---

6. Face Detection and Recognition

python
detected_faces_primary = {}
detected_faces_usb = {}
capture_interval = 120  # 2 minutes in seconds

• detected_faces_primary and detected_faces_usb: Dictionaries that store the faces detected by each camera.
• capture_interval: Ensures that images of the same face are saved only if 2 minutes have passed since the last capture.

---

7. Processing Frames for Face Detection and Saving Images

python
def process_frame(frame, detected_faces, photo_folder):
    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.1, 4)
    current_time = time.time()

    for (x, y, w, h) in faces:
        face_img = frame[y:y + h, x:x + w]
        face_enc = face_recognition.face_encodings(face_img)

        if len(face_enc) > 0:
            face_encoding = face_enc[0]

            matched_face = None
            for detected_face in detected_faces.keys():
                if face_recognition.compare_faces([detected_faces[detected_face]['encoding']], face_encoding)[0]:
                    matched_face = detected_face
                    break

            if matched_face:
                last_captured = detected_faces[matched_face]['last_captured']
                if current_time - last_captured >= capture_interval:
                    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
                    image_filename = f'image_{timestamp}.jpg'
                    image_path = os.path.join(photo_folder, image_filename)
                    cv2.imwrite(image_path, frame)
                    detected_faces[matched_face]['last_captured'] = current_time

            else:
                timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")
                image_filename = f'image_{timestamp}.jpg'
                image_path = os.path.join(photo_folder, image_filename)
                cv2.imwrite(image_path, frame)

                detected_faces[timestamp] = {
                    'encoding': face_encoding,
                    'last_captured': current_time
                }

    return detected_faces

• This function processes each frame from either the primary or USB camera:
  • It converts the frame to grayscale for face detection.
  • If a face is detected, the face encoding is computed using the face_recognition library.
  • The face encoding is compared with previously detected faces:
    • If the face has been seen before, it checks if 2 minutes have passed since the last capture. If so, a new image is saved.
    • If it’s a new face, the image is saved immediately, and the encoding is stored in the detected_faces dictionary for future comparisons.
• Face images are saved in the corresponding photo folder (photos_primary_camera or photos_usb_camera).

---

8. Main Loop to Capture and Process Video

python
while True:
    ret_primary, frame_primary = cap_primary.read()
    ret_usb, frame_usb = cap_usb.read()

    if ret_primary:
        out_primary.write(frame_primary)  # Save frame from primary camera
        detected_faces_primary = process_frame(frame_primary, detected_faces_primary, photo_folder_primary)
        cv2.imshow('Primary Camera', frame_primary)

    if ret_usb:
        out_usb.write(frame_usb)  # Save frame from USB camera
        detected_faces_usb = process_frame(frame_usb, detected_faces_usb, photo_folder_usb)
        cv2.imshow('USB Camera', frame_usb)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

• In the main loop, frames are captured simultaneously from both the primary and USB cameras.
• Each frame is written to its respective video file (out_primary for the primary camera and out_usb for the USB camera).
• Face detection and processing are performed on each frame using the process_frame function.
• The imshow function displays the video streams in separate windows for each camera.
• The loop runs continuously until the user presses the 'q' key to quit.

---

9. Cleanup

python
cap_primary.release()
cap_usb.release()
out_primary.release()
out_usb.release()
cv2.destroyAllWindows()

• Once the loop ends (on pressing 'q'), the cameras and video writers are released, and all windows are closed to free up resources.

---

Key Features of the Code:

• Dual Camera Support: Records from both the laptop's built-in camera and a USB camera at the same time.
• Face Detection and Recognition: Detects faces and saves images only if a new face is detected or if a sufficient time interval has passed since the last capture.
• Organized Storage: Videos and images are saved in separate directories for each camera, with timestamped filenames.

--------------------------------Final Code---------------------------------

import cv2

import datetime

import os

import time

import face_recognition

# Load the pre-trained Haar cascade for face detection

face_cascade = cv2.CascadeClassifier(cv2.data.haarcascades + 'haarcascade_frontalface_default.xml')

# Open the primary laptop camera (index 0) and the USB camera (index 1)

cap_primary = cv2.VideoCapture(0)

cap_usb = cv2.VideoCapture(1)

# Get current date for folder names

now = datetime.datetime.now()

date_str = now.strftime("%Y%m%d")  # Format: YYYYMMDD

# Create directories for storing videos and photos

video_folder_primary = os.path.join(date_str, 'videos_primary_camera')

video_folder_usb = os.path.join(date_str, 'videos_usb_camera')

photo_folder_primary = os.path.join(date_str, 'photos_primary_camera')

photo_folder_usb = os.path.join(date_str, 'photos_usb_camera')

os.makedirs(video_folder_primary, exist_ok=True)

os.makedirs(video_folder_usb, exist_ok=True)

os.makedirs(photo_folder_primary, exist_ok=True)

os.makedirs(photo_folder_usb, exist_ok=True)

# Define the codec and create VideoWriter objects for video saving (for both cameras)

fourcc = cv2.VideoWriter_fourcc(*'XVID')

# Video output for primary camera

video_filename_primary = f'primary_video_{now.strftime("%Y%m%d_%H%M%S")}.avi'  # Format: primary_video_YYYYMMDD_HHMMSS.avi

video_path_primary = os.path.join(video_folder_primary, video_filename_primary)

out_primary = cv2.VideoWriter(video_path_primary, fourcc, 20.0, (640, 480))

# Video output for USB camera

video_filename_usb = f'usb_video_{now.strftime("%Y%m%d_%H%M%S")}.avi'  # Format: usb_video_YYYYMMDD_HHMMSS.avi

video_path_usb = os.path.join(video_folder_usb, video_filename_usb)

out_usb = cv2.VideoWriter(video_path_usb, fourcc, 20.0, (640, 480))

# Initialize variables

detected_faces_primary = {}

detected_faces_usb = {}

capture_interval = 120  # 2 minutes in seconds

def process_frame(frame, detected_faces, photo_folder):

    # Convert the frame to grayscale for face detection

    gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

    faces = face_cascade.detectMultiScale(gray, 1.1, 4)

    # Get the current timestamp

    current_time = time.time()

    for (x, y, w, h) in faces:

        # Extract the face from the frame

        face_img = frame[y:y + h, x:x + w]

        face_enc = face_recognition.face_encodings(face_img)

        if len(face_enc) > 0:  # If face encoding is found

            face_encoding = face_enc[0]

            # Compare with existing detected faces

            matched_face = None

            for detected_face in detected_faces.keys():

                if face_recognition.compare_faces([detected_faces[detected_face]['encoding']], face_encoding)[0]:

                    matched_face = detected_face

                    break

            if matched_face:

                # Update the last captured time for the matched face

                last_captured = detected_faces[matched_face]['last_captured']

                if current_time - last_captured >= capture_interval:

                    # Save the entire frame with the detected face

                    timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")  # Format: YYYYMMDD_HHMMSS

                    image_filename = f'image_{timestamp}.jpg'

                    image_path = os.path.join(photo_folder, image_filename)

                    cv2.imwrite(image_path, frame)

                    # Update last captured time

                    detected_faces[matched_face]['last_captured'] = current_time

            else:

                # New face detected

                timestamp = datetime.datetime.now().strftime("%Y%m%d_%H%M%S")  # Format: YYYYMMDD_HHMMSS

                image_filename = f'image_{timestamp}.jpg'

                image_path = os.path.join(photo_folder, image_filename)

                cv2.imwrite(image_path, frame)

                # Store new detected face with current time

                detected_faces[timestamp] = {

                    'encoding': face_encoding,

                    'last_captured': current_time

                }

    return detected_faces

# Main loop for both cameras

while True:

    ret_primary, frame_primary = cap_primary.read()

    ret_usb, frame_usb = cap_usb.read()

    if ret_primary:

        out_primary.write(frame_primary)  # Write frame to primary camera video file

        detected_faces_primary = process_frame(frame_primary, detected_faces_primary, photo_folder_primary)

        cv2.imshow('Primary Camera', frame_primary)

    if ret_usb:

        out_usb.write(frame_usb)  # Write frame to USB camera video file

        detected_faces_usb = process_frame(frame_usb, detected_faces_usb, photo_folder_usb)

        cv2.imshow('USB Camera', frame_usb)

    # Press 'q' to quit

    if cv2.waitKey(1) & 0xFF == ord('q'):

        break

# Release everything after the job is finished

cap_primary.release()

cap_usb.release()

out_primary.release()

out_usb.release()

cv2.destroyAllWindows()

-------------------------------------------------------------------------------

Requirements to Install on Windows 10 for Face Detection and Recognition

To run the provided code for dual-camera human detection and face recognition on Windows 10, you'll need the following:

1. Python Installation:

   • Install Python (preferably 3.6 or higher). You can download it from the official Python website.

2. Install Required Libraries:

   • Open a command prompt and install the required Python libraries using pip. Run the following commands:

     bash
     pip install opencv-python
     pip install opencv-python-headless
     pip install face_recognition
     

   • Make sure to have pip installed. If it's not available, you can use the Python installer to add it.

3. Additional Dependencies:

   • Haar Cascade Classifier: This is included with OpenCV, so you don't need to install it separately. However, ensure that OpenCV is correctly set up so that it can access the haarcascade files.
   • CMake and dlib: The face_recognition library requires dlib. You may need to install CMake and Visual Studio Build Tools to compile it, which can be installed from the CMake website and Visual Studio.

4. Camera Setup:

   • Ensure that both the primary laptop camera and the USB camera are connected and properly recognized by Windows. You can check this in the Device Manager.

5. IDE or Code Editor:

   • Install an Integrated Development Environment (IDE) like Visual Studio Code or PyCharm to write and run your Python code easily.

6. System Permissions:

   • Make sure that your system permissions allow access to the camera for the Python application.

Summary of Steps to Run the Code

1. Install Python and Required Libraries:

   • Download and install Python.
   • Use pip to install opencv-python, opencv-python-headless, and face_recognition.

2. Set Up the Cameras:

   • Connect the USB camera and ensure that the laptop's built-in camera is operational.

3. Write and Save the Code:

   • Copy the provided code into a new Python file (e.g., dual_camera_face_recognition.py).

4. Run the Code:

   • Open a command prompt in the directory where your Python file is saved and run the command:

     bash
     python dual_camera_face_recognition.py
     

5. Monitor the Output:

   • The video feed from both cameras will display, and images of detected faces will be saved in the corresponding folders based on the camera used.

6. Stop the Program:

   • Press 'q' to terminate the program and release the camera resources.