Integration of Image Segmentation using Huawei ML Kit in React Native

Introduction

In this article, we can learn how to implement Image Segmentation feature using ML Kit. This service segments same elements (such as human body, plant, and sky) from an image. The elements supported include human body, sky, plant, food, cat, dog, flower, water, sand, building, mountain, and others.

Image segmentation can be used in many different scenarios. It can be used in photography apps to change the background or apply some effect only on plants or human body etc. It can also be used to determine cancerous cells in a microscope image or get land usage information in a satellite image or to determine the amount of herbicides that needed to be sprayed in a field according to crop density.

Output

Create Project in Huawei Developer Console

Before you start developing an app, configure app information in AppGallery Connect.

Register as a Developer

Before you get started, you must register as a Huawei developer and complete identity verification on HUAWEI Developers. For details, refer to Registration and Verification.

Create an App

Follow the instructions to create an app Creating an AppGallery Connect Project and Adding an App to the Project.

Generating a Signing Certificate Fingerprint

Use below command for generating certificate.

keytool -genkey -keystore <application_project_dir>\android\app\<signing_certificate_fingerprint_filename>.jks -storepass <store_password> -alias <alias> -keypass <key_password> -keysize 2048 -keyalg RSA -validity 36500

Generating SHA256 key

Use below command for generating SHA256.

keytool -list -v -keystore <application_project_dir>\android\app\<signing_certificate_fingerprint_filename>.jks

Note: Add SHA256 key to project in App Gallery Connect.

React Native Project Preparation

1. Environment set up, refer below link.

https://reactnative.dev/docs/environment-setup

2. Create project using below command.

react-native init project name

3. Download the Plugin using NPM.

Open project directory path in command prompt and run this command.

npm i  @hmscore/react-native-hms-ml

4. Configure android level build.gradle.

a. Add to buildscript/repositores.

maven {url 'http://developer.huawei.com/repo/'}

b. Add to allprojects/repositories.

maven {url 'http://developer.huawei.com/repo/'}

Final Code

Add this code in App.js

Add this code in Helper.js

Add this code in Styles.js

Testing

Run the android app using the below command.

react-native run-android

Generating the Signed Apk

1. Open project directory path in command prompt.

2. Navigate to android directory and run the below command for signing the APK.

gradlew assembleRelease

Tips and Tricks

1. Set minSdkVersion to 19 or higher.

2. For project cleaning, navigate to android directory and run the below command.

gradlew clean

Conclusion

This article will help you to setup React Native from scratch and learned about integration of ML Kit Image Segmentation in react native project. We can apply image segmentation on static images and video streams if we select human body type. But we can only apply segmentation for static images in multiclass segmentation.

Thank you for reading and if you have enjoyed this article, I would suggest you to implement this and provide your experience.

Reference

ML Kit(Image Segmentation) Document, refer this URL.