A food product identified via barcode scanning
+A parent node in the food heirarchy (e.g. pasta), will only be returned if the models could not detect something more specific
+A leaf node in the food heirarchy, meaning a specific food item that has been identified by the models
+A food product identified via reading the text on the packaging label
+A leaf node in the food heirarchy, meaning a specific food item that has been identified by the models
+Generated using TypeDoc
Generated using TypeDoc
This project provides React Native bindings for the Passio SDK. It also includes the RN Quickstart application which serves as a test harness for the SDK.
+ + +Please note that the SDK will currently not run in the iOS simulator. We hope to change this in the future, but an iOS test device is required for now.
+ + +.npmrc file in the root of your project with the following lines, replacing GITHUB_ACCESS_TOKEN with the token provided to you by Passio. This grants you access to the SDK's private listing on Github Package Registry.//npm.pkg.github.com/:_authToken=GITHUB_ACCESS_TOKEN
@passiolife:registry=https://npm.pkg.github.com
+package.json and run npm install or yarn."@passiolife/react-native-passio-sdk": "1.4.13"
+For iOS, run pod install.
+cd ios; pod install
+For Android, auto-linking should handle setting up the Gradle dependency for your project.
+ + +Enter a value for NSCameraUsageDescription in your Info.plist so the camera may be utilized.
Import the SDK
+import {
PassioSDK,
DetectionCameraView,
} from '@passiolife/react-native-passio-sdk';
+// Somewhere in your component (inside of a flex container)
<DetectionCameraView style={{flex: 1, width: '100%'}} />
+
// In your component
const [isReady, setIsReady] = useState(false);
// Effect to configure the SDK and request camera permission
useEffect(() => {
Promise.all([
PassioSDK.configure({
key: 'your-developer-key',
autoUpdate: true,
}),
PassioSDK.requestCameraAuthorization(),
]).then(([sdkStatus, cameraAuthorized]) => {
console.log(
`SDK configured: ${sdkStatus.mode} Camera authorized: ${cameraAuthorized}`,
);
setIsReady(sdkStatus.mode === 'isReadyForDetection' && cameraAuthorized);
});
}, []);
// Once the SDK is ready, start food detection
useEffect(() => {
if (!isReady) {
return;
}
const config: FoodDetectionConfig = {
detectBarcodes: true,
detectOCR: true,
detectNutritionFacts: true,
}
const subscription = PassioSDK.startFoodDetection(
config,
async (detection: FoodDetectionEvent) => {
console.log('Got food detection event: ', detection);
const { candidates, nutritionFacts } = detection
if (candidates?.barcodeCandidates?.length) {
// show barcode candidates to the user
} else if (candidates?.ocrCandidates?.length) {
// show OCR candidates to the user
} else if (candidates?.detectedCandidates?.length) {
// show visually recognized candidates to the user
} else if (nutritionFacts) {
// Show scanned nutrition facts to the user
}
},
);
// stop food detection when component unmounts
return () => subscription.remove();
}, [isReady]);
+If your project does not currently contain any Swift, you might get an undefined symbol errors for the Swift standard library when adding the Passio SDK. Since the Passio SDK is a Swift framework, your app needs to link against the Swift standard library. You can accomplish this by adding a single Swift file to your project.
+Because the Passio SDK is a Swift framework and depends on React-Core, we need a modular header for this dependency. If you get an error regarding a missing module header for React-Core, update your Podfile to produce one:
pod 'React-Core', :path => '../node_modules/react-native/', :modular_headers => true
+Generated using TypeDoc
Generated using TypeDoc
Generated using TypeDoc
Generated using TypeDoc
An object defining the configuration options for the Passio SDK
+Set to true to enable download of AI models from Passio's servers.
+Occurs once per SDK version or again if the app is deleted and reinstalled.
+Does not automatically update to future model versions. When the SDK is updated,
+it will delete the old models and download the newest version.
Set to true to enable debug logging
Your Passio SDK key
+Provide local copies of Passio models that you've either bundled within your app or +downloaded from your own servers. The SDK will copy and decrypt the files and delete +the original files to free up disk space.
+You don't need to supply the localModelURLs each time you configure the SDK. +Only if you configure the SDK and it returns status "notReady" with missingFiles listed +do you need to retrieve the missing files and call configure again, passing the file URLs +in this field.
+Generated using TypeDoc
Generated using TypeDoc
A collection of food candidates detected by the models.
+Food candidate results from barcode scanning.
+Food candidate results from visual scanning. The array is sorted by confidence, with the most confident result at index 0.
+Food candidate results from the logo detection model.
+Food candidate results from OCR scanning.
+Generated using TypeDoc
A configuration object to determine which types of recognition should be performed by +the SDK. There is currently no flag to control visual food recognition models, they are +enabled by default.
+Detect barcodes on packaged food products. Results will be returned
+as BarcodeCandidates in the FoodCandidates property of FoodDetectionEvent
Detect barcodes on packaged food products. Results will be returned
+under the nutritionFacts property on FoodDetectionEvent.
Detect packaged food labels using OCR. Results will be returned
+as OCRCandidates in the FoodCandidates property of FoodDetectionEvent
Generated using TypeDoc
An object provided in the callback for food detection containing +food candidates as well as nutrition facts, if found
+A collection of food candidates detected by the models.
+Detected nutrition facts when scanning a nutrition label.
+Generated using TypeDoc
Generated using TypeDoc
Generated using TypeDoc
A candidate resulting from object detection models
+A box describing a detected object's location in the camera view
+Confidence of the classification candidate, ranging from 0.0 to 1.0
+The ID of the detected item
+Generated using TypeDoc
Generated using TypeDoc
Nutritional information for an item in the food database.
+Alcohol, in grams
+The UPC code for this food product, if available
+Calcium, in milligrams
+Calories, in kcal
+Carbohydrates, in grams
+Related items below this item in the food heirarchy (more specific)
+Cholesterol, in milligrams
+The mass of the serving size, in grams
+The entity type of the item
+Fat, in grams
+Dietary fiber, in grams
+The name of the image for this item. Provide this value to a PassioIconView
+in order to display the image.
The ingredients listed on the product packaging, if any
+Iodine, in micrograms
+Iron, in milligrams
+Magnesium, in milligrams
+Monounsaturated fat, in grams
+The name of the item
+Related items above this item in the food heirarchy (more generic)
+The ID of the item in the database
+Phosphorus, in milligrams
+Polyunsaturated fat, in grams
+Potassium, in milligrams
+Protein, in grams
+Saturated fat, in grams
+The default serving quantity
+The default serving unit
+The serving sizes available for this recipe
+The serving units available for this recipe
+Related items at the same level as this item in the food heirarchy
+Sodium, in milligrams
+Total sugars, in grams
+Added sugar, in grams
+Sugar alcohol, in grams
+Transfat, in grams
+Vitamin A, in IU
+Vitamin B12, in micrograms
+Added Vitamin B12, in micrograms
+Vitamin B6, in milligrams
+Vitamin C, in milligrams
+Vitamin D, in milligrams
+Vitamin E, in milligrams
+Added Vitamin E, in milligrams
+Generated using TypeDoc
Information associated with an item in the nutritional database.
+Check the entityType field to determine the type of the item.
Related items below this item in the food heirarchy (more specific)
+The entity type of the item
+The nutritional data for this item in the database
+The name of the image for this item. Provide this value to a PassioIconView
+in order to display the image.
The name of the item
+Related items above this item in the food heirarchy (more generic)
+The ID of the item in the database
+The recipe data for this item in the database
+Related items at the same level as this item in the food heirarchy
+Generated using TypeDoc
The food items in this recipe
+The name of the image for this recipe. Provide this value to a
+PassioIconView in order to display the image.
The name of the recipe
+The ID of the recipe in the database
+The default serving quantity
+The default serving unit
+The serving sizes available for this recipe
+The serving units available for this recipe
+Generated using TypeDoc
Configure the SDK with the given options.
+The configuration options
+A Promise resolving with a PassioStatus indicating the current state of the SDK.
Query Passio's UPC web service for nutrition attributes of a given barcode.
+The barcode value for the attributes query, typically taken from a scanned BarcodeCandidate.
A Promise resolving to a PassioIDAttributes object if the record exists in the database or null if not.
Query Passio's web service for nutrition attributes given an OCR identifier.
+The OCR identifier for the attributes query, taken from the list of OCR candidates on a FoodDetectionEvent.
A Promise resolving to a PassioIDAttributes object if the record exists in the database or null if not.
Look up the nutrition attributes for the given name of a food item. This is most often used with a string received
+from the searchForFood function.
The name of the item you'd like to query.
+A Promise resolving to a PassioIDAttributes object if the record exists in the database or null if not.
Look up the nutrition attributes for a given Passio ID.
+The Passio ID for the attributes query.
+A Promise resolving to a PassioIDAttributes object if the record exists in the database or null if not.
Prompt the user for camera authorization if not already granted.
+A Promise resolving to true if authorization has been granted or false if not.
Search the local database of foods with a given search term.
+The search term to match against food item names.
+A Promise resolving to an array of food item names.
Begin food detection using the device's camera.
+An object to determine which types of scanning should be performed.
+A callback to repeatedly receive food detection events as they occur.
+A Subscription that should be retained by the caller while food detection is running. Call remove on the subscription to terminate food detection.
Generated using TypeDoc
Generated using TypeDoc
Generated using TypeDoc
A subscription that is created when food detection begins. +The caller must retain this object in memory and call remove +when detection should be terminated so resources can be released. +Failing to remove the subscription may result in significant +memory leaks.
+Terminate the subscription and release any related resources.
+Generated using TypeDoc
Generated using TypeDoc
A UPC code captured by scanning a barcode
+A an identifier for a visually scanned packaged food label
+The ID of an item in the nutrition database.
+The possible states of the SDK after calling configure. Switch on status.mode to +access the data associated with each state.
+SDK failed to configure in an unrecoverable way. Please read the error message for more inforation.
+SDK is not ready due to missing model files. Please download the specified files
+and call configure again, passing in the localFileURLs of the downloaded files.
SDK configuration successfully. This status much be reached before calling startFoodDetection.
+It is possible that missing files may still be reported in the event that the SDK is aware of newer
+model versions than the ones currently loaded. The SDK should still function normally in this case.
A component that displays the camera feed and sends camera frames +to the food detection models.
+A component for displaying food icons from the Passio SDK.
+Generated using TypeDoc
This source code is licensed under the MIT license found in the LICENSE file in the root
+ * directory of this source tree.
+ */
+package com.example.reactnativepassiosdk;
+
+import android.content.Context;
+import com.facebook.flipper.android.AndroidFlipperClient;
+import com.facebook.flipper.android.utils.FlipperUtils;
+import com.facebook.flipper.core.FlipperClient;
+import com.facebook.flipper.plugins.crashreporter.CrashReporterPlugin;
+import com.facebook.flipper.plugins.databases.DatabasesFlipperPlugin;
+import com.facebook.flipper.plugins.fresco.FrescoFlipperPlugin;
+import com.facebook.flipper.plugins.inspector.DescriptorMapping;
+import com.facebook.flipper.plugins.inspector.InspectorFlipperPlugin;
+import com.facebook.flipper.plugins.network.FlipperOkhttpInterceptor;
+import com.facebook.flipper.plugins.network.NetworkFlipperPlugin;
+import com.facebook.flipper.plugins.react.ReactFlipperPlugin;
+import com.facebook.flipper.plugins.sharedpreferences.SharedPreferencesFlipperPlugin;
+import com.facebook.react.ReactInstanceManager;
+import com.facebook.react.bridge.ReactContext;
+import com.facebook.react.modules.network.NetworkingModule;
+import okhttp3.OkHttpClient;
+
+public class ReactNativeFlipper {
+ public static void initializeFlipper(Context context, ReactInstanceManager reactInstanceManager) {
+ if (FlipperUtils.shouldEnableFlipper(context)) {
+ final FlipperClient client = AndroidFlipperClient.getInstance(context);
+ client.addPlugin(new InspectorFlipperPlugin(context, DescriptorMapping.withDefaults()));
+ client.addPlugin(new ReactFlipperPlugin());
+ client.addPlugin(new DatabasesFlipperPlugin(context));
+ client.addPlugin(new SharedPreferencesFlipperPlugin(context));
+ client.addPlugin(CrashReporterPlugin.getInstance());
+ NetworkFlipperPlugin networkFlipperPlugin = new NetworkFlipperPlugin();
+ NetworkingModule.setCustomClientBuilder(
+ new NetworkingModule.CustomClientBuilder() {
+ @Override
+ public void apply(OkHttpClient.Builder builder) {
+ builder.addNetworkInterceptor(new FlipperOkhttpInterceptor(networkFlipperPlugin));
+ }
+ });
+ client.addPlugin(networkFlipperPlugin);
+ client.start();
+ // Fresco Plugin needs to ensure that ImagePipelineFactory is initialized
+ // Hence we run if after all native modules have been initialized
+ ReactContext reactContext = reactInstanceManager.getCurrentReactContext();
+ if (reactContext == null) {
+ reactInstanceManager.addReactInstanceEventListener(
+ new ReactInstanceManager.ReactInstanceEventListener() {
+ @Override
+ public void onReactContextInitialized(ReactContext reactContext) {
+ reactInstanceManager.removeReactInstanceEventListener(this);
+ reactContext.runOnNativeModulesQueueThread(
+ new Runnable() {
+ @Override
+ public void run() {
+ client.addPlugin(new FrescoFlipperPlugin());
+ }
+ });
+ }
+ });
+ } else {
+ client.addPlugin(new FrescoFlipperPlugin());
+ }
+ }
+ }
+}
diff --git a/example/android/app/src/main/AndroidManifest.xml b/example/android/app/src/main/AndroidManifest.xml
new file mode 100644
index 0000000..9793c8b
--- /dev/null
+++ b/example/android/app/src/main/AndroidManifest.xml
@@ -0,0 +1,32 @@
+
The type of object represented by a
+PassioIDAttributes