Skip to content

Conversation

@dccarmo
Copy link

@dccarmo dccarmo commented Aug 20, 2025

Description

Closes #210

This PR adds initial support for MLKit Text Recognition v2 for both iOS and Android.

Please let me know if I missed any collaboration steps or if there's anything else that needs to be updated.

Future work

  • Add support for all scripts besides Latin
  • Improve return record type in each platform

@changeset-bot
Copy link

changeset-bot bot commented Aug 20, 2025

🦋 Changeset detected

Latest commit: c335f27

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages
Name Type
@infinitered/react-native-mlkit-text-recognition Major
example-app Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@dccarmo dccarmo changed the title Feat/text recognition Text Recognition Aug 20, 2025
@coolsoftwaretyler
Copy link

Hey @dccarmo - sorry for the long delay here. @frankcalise and I are going to review this sometime soon and get back to you. Appreciate the work!

Copy link
Collaborator

@frankcalise frankcalise left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey thanks for the submission, couple of things to take a look at here if you don't mind.

Comment on lines +130 to +132
const result = await launchImageLibrary({
mediaType: "photo",
})
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we need to add react-native-image-picker for this over expo-image-picker? Does this one do something differently that you needed?


struct TextRecord: Record {
@Field var text: String = ""
@Field var textBlocks: [TextBlockRecord] = []
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can rename this to blocks to match the API docs more closely.

Suggested change
@Field var textBlocks: [TextBlockRecord] = []
@Field var blocks: [BlockRecord] = []

struct TextBlockRecord: Record {
@Field var text: String = ""
@Field var frame: RectRecord = RectRecord()
@Field var recognizedLanguage: String = ""
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What was the decision here to only return the first language code over retrieving the array as the data structure is documented?

Comment on lines +1 to +9
{
"permissions": {
"allow": [
"WebFetch(domain:developers.google.com)"
],
"deny": [],
"ask": []
}
} No newline at end of file

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: please remove this claude config - I don't want other people to get the same permissions you granted on your machine.

<Text preset={"heading"} text="Text Recognition" />
<Text style={$description}>Take a photo, and extract text from it.</Text>
</View>
<ImageSelector

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: did you mean to include taking pictures? Looks like this is just an image select:

ScreenRecording_11-11-2025.14-29-25_1.MP4

[],
)

useEffect(() => {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Working on iOS when selecting images:

Image


// Ensures backward compatibility
ext.getKotlinVersion = {
if (ext.has("kotlinVersion")) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: @dccarmo - did you test this out on Android? I hit this build error:

> Task :expo-modules-core:compileDebugKotlin FAILED
e: This version (1.5.14) of the Compose Compiler requires Kotlin version 1.9.24 but you appear to be using Kotlin version 1.9.25 which is not known to be compatible.  Please consult the Compose-Kotlin compatibility map located at https://developer.android.com/jetpack/androidx/releases/compose-kotlin to choose a compatible version pair (or `suppressKotlinVersionCompatibilityCheck` but don't say I didn't warn you!).

Do you have any screen recordings of it building on your end? I haven't been able to get this to build, and it does build on main.

I suspect the changes to apps/ExampleApp/package.json are part of the problem here. Can we try a smaller set of dependency changes in this PR?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@frankcalise and I hit this when upgrading to Expo 53 - might be an Expo modules core issue. Wanna try bumping that version in your module to make sure it matches what we have for other modules on Expo 52?

"@infinitered/react-native-mlkit-image-labeling": "workspace:^3.1.0",
"@infinitered/react-native-mlkit-object-detection": "workspace:^3.1.0",
"@react-native-async-storage/async-storage": "^2.0.0",
"@infinitered/react-native-mlkit-text-recognition": "workspace:^1.0.0",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue: I don't think we need all the changes here, and it might be breaking our Android builds. Can we revisit these changes to a minimal set of dependency upgrades?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add basic OCR feature

3 participants