GSoC 2017 Summary

2017/08/24

Overall summary of work for Jitsi-Meet, in GSoC 2017.

Overview

During the summer break of 2017, I attended Google Summer of Code (GSoC) and worked for Jitsi organization. There are mainly two projects I completed in the activity:

  1. Added face tracking mechanism in Jitsi-Meet. Jitsi-Meet supports secure, simple and scalable video conferences which could be used as a standalone app or embed in a web application. I added a face recognition feature in the tool. With my new design, when face tracking mechanism is triggered, if the user's face is not in the center, a prompt will pop out until the user's face is moved to appropriate position.

  2. Implemented a simple calendar app with Google Calendar APIs. I create a calendar app which shows forthcoming events' information according to a target calendar ID.

Additionally, during my work, I created several patches for tracking.js library, which is used in the face tracking mechanism.

Here, I list related results generated from my work:

Projects

Face Tracking in Jitsi-Meet

For the new face tracking mechanism, I created an option for it in the side panel of the current Jisti-Meet app.

Enable face tracking

No matter the Enable face tracking option is toggled or not, by default, face recognition is enabled in device settings panel. Here, the user could check whether his/her face is in the center of the screen. If not, a prompt will be displayed to ask the user to move face to the center.

Face tracking in preview video

Face tracking in the large video is controlled by Enable face tracking option. When it's toggled, face recognition will begin to work in the large video continuously until the user turns it off.

Face tracking in large video

The structure of my design of face tracking functionality is shown below.

Structure of original design

The face tracking mechanism is mainly achieved in FaceTracker component, which uses tracking.js library as the backbone. FacePrompt React component represents the text and rectangle prompts displayed when the user's face is not in the center.

Calendar Functionality

The demo of the calendar functionality is shown below. It'll be embedded in Jitsi-Meet Electron app.

Demo of calendar app

The demo fetches events of a Google Calendar through Google Calendar APIs, and lists them on the screen. The code can not only used with Google Calendar, but can also be used with other calendars, such as Outlook Calendar. To achieve that, I have an AbstractCalendar base class, which could be inherited by other calendar classes.

Structure of calendar app

Summary

  1. Collaboration is very crucial in real-world project development.

  2. For an industrial project, it may not be correct that the latest technology, the better.

  3. Pattern recognition algorithms in browser side may need optimization in practice.

Post Directory