An Interest In:
Web News this Week
- April 20, 2024
- April 19, 2024
- April 18, 2024
- April 17, 2024
- April 16, 2024
- April 15, 2024
- April 14, 2024
September 25, 2022 05:34 pm GMT
Original Link: https://dev.to/janhavidadhania/mediapipe-implementing-custom-feature-on-android-255i
Mediapipe | Implementing custom feature on Android
project link: https://github.com/google/mediapipe
Structure of Mediapipe project repo
Compile and try out already implemented example projects on android
- build docker image using command
docker build . -t mediapipe
- run docker container over image
docker run -i -t IMAGE_ID /bin/bash
- inside the container set up Android using command
bash ./setup_android_sdk_and_ndk.sh ~/Android/Sdk ~/Android/Ndk r21
- then build android binary using bazel.
bazel build -c opt mediapipe/examples/android/src/java/com/google/mediapipe/apps/facedetectioncpu:facedetectioncpu
this command is for building facedetectioncpu project. replace project_name (facedetectioncpu here) with the name of the project you want to build. Name of the folders under path mediapipe/mediapipe/examples/android/src/java/com/google/mediapipe/apps/ is name of the inbuild projects.this command would take long for first build. for any subsequent builds, it would use the cached data and shouldn't take long. - apk will be stored at
/mediapipe/bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/project_name
. copy the apk filesudo docker cp CONTAINER_ID:mediapipe/mediapipe/bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/project_name /home/username/downloads/.
- copy this apk file to your android device and install.
- or directly install from terminal using commands
adb install project.apk
- keep the phone connected and logs could be viewed using
adb logcat
about mediapipe structure
- Image at the top has general structure of the mediapipe project.
- we call bazel build on BUILD file present at
mediapipe/examples/android/src/java/com/google/mediapipe/apps/facemeshgpu
- This build file refers to graph present at
/mediapipe/graphs/facemeshgpu/facemeshgpu.pbtxt
- This graph at
/mediapipe/graphs/facemeshgpu/facemeshgpu.pbtxt
refers to one or more calculators present at 'mediapipe/calculators/folder_name/calculator.cc` - both folders
mediapipe/calculators/folder_name/.
andmediapipe/graphs/project_name/.
has BUILD files that contains information about dependencies.
about graphs
- graphs could be visualised here https://viz.mediapipe.dev/
- copy paste content from any of your .pbtxt file e.g
https://github.com/google/mediapipe/blob/master/mediapipe/graphs/face_detection/face_detection_full_range_mobile_gpu.pbtxt
to editor on visualiser. It would show the graph on the left side. - Each box is calculator and has input and output.
- This input output relations are specified in .pbtxt graph files.
about calculators
- each box in graph visualiser is calculator. (or subgraph made of calculators)
- calculators has process function defined that performes calculations. e.g render text on frame or render assets like spectacles at specific position on frame. e.g https://github.com/google/mediapipe/blob/6cdc6443b6a7ed662744e2a2ce2d58d9c83e6d6f/mediapipe/calculators/util/landmark_visibility_calculator.cc#L66
- other than process, calculator have getcontract and open function. Open function is called once at the start and process function is called for each input again and again. see below for example.
about open vs process function in calculators
consider we need to display frame count on the camera preview. We start with 0 (for the first frame as the camera starts) and increment count for every subsequent frame. For this, we would define variable count in open function of calculator and assign value 0 to it.
We will add increment logic count = count + 1
in process function and display it on frame.
count will be incremented for every frame because process function will be called for each input, i.e frame. Open is called once at the beginning and value 0 will be assigned to count.
Modify current project
- add new calculator in project_folder inside calculator folder
- add entry in BUILD file of the project calculator folder
- create project folder inside graph folder and create .pbtxt graph for new project.
- add entry in BUILD file of project graph folder
- modify BUILD
mediapipe/examples/android/src/java/com/google/mediapipe/apps/BUILD
- compile using command
bazel build PATH TO BUILD
Original Link: https://dev.to/janhavidadhania/mediapipe-implementing-custom-feature-on-android-255i
Share this article:
Tweet
View Full Article
Dev To
An online community for sharing and discovering great ideas, having debates, and making friendsMore About this Source Visit Dev To