Overall summary of work for Jitsi-Meet, in GSoC 2017.
During the summer break of 2017, I attended Google Summer of Code (GSoC) and worked for Jitsi organization. There are mainly two projects I completed in the activity:
Added face tracking mechanism in Jitsi-Meet. Jitsi-Meet supports secure, simple and scalable video conferences which could be used as a standalone app or embed in a web application. I added a face recognition feature in the tool. With my new design, when face tracking mechanism is triggered, if the user's face is not in the center, a prompt will pop out until the user's face is moved to appropriate position.
Implemented a simple calendar app with Google Calendar APIs. I create a calendar app which shows forthcoming events' information according to a target calendar ID.
Additionally, during my work, I created several patches for tracking.js library, which is used in the face tracking mechanism.
Here, I list related results generated from my work:
summary of my work in GSoC - this blog.
Face Tracking in Jitsi-Meet
For the new face tracking mechanism, I created an option for it in the side panel of the current Jisti-Meet app.
No matter the
Enable face tracking option is toggled or not, by default, face recognition is enabled in device settings panel. Here, the user could check whether his/her face is in the center of the screen. If not, a prompt will be displayed to ask the user to move face to the center.
Face tracking in the large video is controlled by
Enable face tracking option. When it's toggled, face recognition will begin to work in the large video continuously until the user turns it off.
The structure of my design of face tracking functionality is shown below.
The face tracking mechanism is mainly achieved in
FaceTracker component, which uses tracking.js library as the backbone.
FacePrompt React component represents the text and rectangle prompts displayed when the user's face is not in the center.
The demo of the calendar functionality is shown below. It'll be embedded in Jitsi-Meet Electron app.
The demo fetches events of a Google Calendar through Google Calendar APIs, and lists them on the screen. The code can not only used with Google Calendar, but can also be used with other calendars, such as Outlook Calendar. To achieve that, I have an
AbstractCalendar base class, which could be inherited by other calendar classes.
Collaboration is very crucial in real-world project development.
For an industrial project, it may not be correct that the latest technology, the better.
Pattern recognition algorithms in browser side may need optimization in practice.