Name: Forcha Pearl Fri
Location: Cameroon, Buea
Time Zone: UTC+01:00
Mentors: Jeffrey Yoo Warren, Tilda Udufo
Overview : Full-screen UI and video upload on Infragram.org
Infragram enhances environmental research capacity by providing a convenient and low cost method to monitor plant health.
During the GSOC2022 period I worked on.
- Delivering video file to the the WebGL texture for analysis.
- Added multiple resolution selection functionality on webcam streams, video files and Image files.
- Included Video controls (pause, play and sleek ) to manage and regulate video files during analysis.
- Added webcam streaming functionality on IOS browsers (Chrome and Safari ).
- Added functionality to record current analysis and download video file locally.
- Included and perform testing on the Infragram User Interface
Delivering video file to the the WebGL texture for analysis ( Process Video File Locally)
- Problem: The was no option for users to monitor plant health from video files as the previous Infragram could only handle Images and Webcam streams.
- Initialized getusemedia API for streaming video file.
- Add functionality to upload video file and append to DOM
- Differentiated between video and Image files.
- Passed Video to WebGL texture for processing.
Added multiple resolution selection functionality on webcam streams, video files and Image files
- Problem: A low resolution video file or Camera stream provides less details. Hence the vegetative analysis will fall on an inaccurate margin.
Allow selection of multiple video resolutions
- Defined standard resolution attributes of canvas height and width QVGA (300 x 200) , VGA (800 x 600), HD (1920 x 1080) and FULL HD (7680 x 4320).
- Set CSS styling of canvas element at a constant 800 x 600.
- Added function to adjust HTML width and height attributes of canvas by user selected resolution.
Included Video controls (pause, play and sleek ) to manage and regulate video files during analysis.
- Problem: Due to the addition of our new video processing functionality. There exist a need to regulate the current file under analysis.
With the help of video controls (play , pause and sleek) our problem can easily be solved.
- Created CSS styling for Sleek bar and play/Pause buttons.
- Get current video time frame and attached it to sleek bar progress.
- Attached video pause and play events to existing video file and play /pause buttons.
- Added toggle functionality between pause and play buttons to improve user experience.
Added webcam streaming functionality on IOS browsers (Chrome and Safari )
-Problem: Previous WebRTC Adapter does notsupport webcam streaming on ios versions greater than 13.0.
- Use MediaDevices.getUserMedia() API to enable webcam streaming on IOS chrome and IOS safarri.
- Use Infagram versioning to ensure compatibility between the new and old interfaces/features
Related PRs https://github.com/publiclab/infragram/pull/442
Added functionality to record current analysis and download video file locally.
-Problem: There was a need to save the current analysis for future use.
-Solution: Record the canvas texture and download.
Included and perform testing on the Infragram User Interface
- Problem: We had to ensure that the Infragram UI works exactly as indented. -Solution:
Infragram users (IOS inclusive) can upload videos while selecting the desired resolution for plant health analysis. The converted video can be downloaded and stored locally for future reference.
What is left
- Include Canvas Zooming and Panning
- Add file Drag and Drop functionality
This work would have not been possible without the support and constructive advice of public lab's research director @warren. Many thanks to outreachy intern @@stefq1111 for designing a user mobile friendly user interface. Finally , thanks to @@mathildaudufo and @cess at the public lab community and everyone else who helped out.