[DMP 2024]: Automated training and inference pipeline for simple/custom models

[Gautam-Rajeev]

Goal:

Create compact TensorFlow Lite (TFLite) models that can be deployed on mobile devices for offline use, specifically for agricultural pest detection and document handling tasks. These models must be tiny, with a file size of <= 10MB, to facilitate easy integration into mobile applications.

Description:

The project involves developing two sets of TinyML models. The first set targets the agricultural sector, focusing on pest detection through image analysis. The second set aims at document detection and processing, including blur detection, alignment correction, and document type classification.

Agricultural Model:

• Pest Detection: Develop a TinyML model to analyze close-up images of crop leaves to determine their suitability for pest detection. The model should discern whether an image is a close-up of a leaf, which is essential for accurate pest detection, and not an image capturing the entire plant from a distance.

Document Detection Support Models:

1. Blurry Image Detection: A TinyML model that determines if an image of a document is blurry, ensuring only clear images proceed for further processing.
2. Alignment Correction: Design a model that corrects alignment issues in images of documents, such as PAN cards, transforming skewed images into correctly aligned rectangles.
3. Document Type Classification: Develop a TinyML model capable of classifying images into document types, e.g., PAN card, Aadhar card, etc.

Implementation Details:

• Utilize TensorFlow Lite for model development, ensuring the models are optimized for low resource consumption.
• The models should be tested extensively on diverse datasets to ensure robustness and accuracy.
• Special consideration must be given to the model architecture to maintain a balance between performance and model size, with a strict size limit of < 10MB.

Contributors are encouraged to share their progress, challenges, and insights through comments. Collaborative efforts are highly appreciated. The contribution deemed most effective and efficient will lead to further discussions and potential project assignment.

Product Name

ai-tools

Organization Name

Samagra

Domain

Agriculture

Tech Skills Needed

• TensorFlow Lite
• Python
• Machine Learning
• Image Processing
• Data Augmentation

Category

Feature Development

Mentor(s)

@ChakshuGautam

Complexity

Medium

[bruhathisp]

I'm Bruhathi, one of the participants in Code4GovTech. As I'm keen to contribute to this repository, I would greatly appreciate some clarification and guidance to ensure my efforts align with the project's goals effectively.

Specifically, I intend to focus on the implementation of blurry image detection for the document detection support models. To proceed efficiently, I kindly request the following details:

1. Project Setup: Could you please advise if it's okay to fork the project and create a new folder for the blurry image detection module?
2. Data Availability: Are there any datasets available or recommended for training the blurry image detection model?

[Azazel0203]

Hello @GautamR-Samagra,

I'm seeking clarification regarding the project, particularly regarding model development for agriculture.

The project outline specifies the use of close-up images of plant leaves. I've found a dataset containing leaf images like the examples below:

Would you recommend proceeding with this dataset, or is there another preferred option?

Thankyou

[smokeyScraper]

Hello,
are there any leads for the dataset?
while searching online I found this dataset https://www.kaggle.com/datasets/pbrant/text-image-with-motion-blur
it's having around 360 of blurred and 180 clear images similar to text documents(books in this case).

ig we can also create our own blurred images dataset using open cv (similar using "cv2.GaussianBlur(image, (n, n), 0)" or other blurring techniques) provided we have clear images dataset

[swarnim-sawane]

Clarification on Model Size Limit and Key Considerations

Hi @ChakshuGautam @GautamR-Samagra ,

Quick question regarding the model size requirement: Should each individual model (pest detection, blurry image detection, alignment correction, document type classification) be ≤ 10MB, or is this the collective size limit for all models?

Also, any specific optimization strategies (compression, quantization) or hardware considerations we should keep in mind for mobile deployment?

Thanks in advance for the insights!

Swarnim Sawane

[ashuashutosh2211]

Hi @ChakshuGautam @GautamR-Samagra , I am Ashutosh, a prefinal year student at IIT Jodhpur pursuing B.Tech. in Artificial Intelligence and Data Science. I have done projects related to deep learning and machine learning. I have worked in projects like Stock Price Prediction , Speech Emotion Recognition, Voice Controlled Music Recommendation System using Deep Learning. As part of my deep learning course lab work, I have worked on deep learning models on images also.
I am interested in this project. For this project I think we can try with multiple model architectures with a combination of CNN, ResCNN, UNets etc as well as using some pre-trained models. We can consider similar problems which are already solved and what models were used for that and reading different research papers to finalize the model architectures. Later on they can be fine tuned on multiple datasets.
Can you guide me furhter about the dataset and the project set up, so that I can contribute to this project ?

[smokeyScraper]

Hello, @ChakshuGautam @GautamR-Samagra
I tried out modelling on

Blurry Image Detection: A TinyML model that determines if an image of a document is blurry, ensuring only clear images proceed for further processing.

I simply implemented a SVM classifier having the image input gone via a pipeline for edge extraction that comprised of Sobel, Roberts and Laplacian filter, classifying the images to Blur/NotBlur by using 3filters(Sobel, Roberts, Laplacian) and 3 features(mean, max, variance) corresponding to each image, resulting in total of 9 features per image.

The results are a follows:

ig if more accurate and better model is required we can try on some more complex models/architectures on the proposed pipeline.
Thanks a lot

ps - I used the above dataset that I've mentioned,
Thanks to the article too https://medium.com/data-science-ecom-express/a-simple-approach-for-blur-image-detection-535b3c55b596

[Elisettygnanesh]

Hello, @ChakshuGautam @GautamR-Samagra
My name is Elisetty Gnanesh, I came up with problem and solution of this project. Please look into this once.

Problem:-
The project encounters various challenges including acquiring diverse and high-quality datasets for pest detection and document handling, balancing model complexity with size constraints (<10MB), managing limited computational resources on mobile devices, ensuring model robustness across different contexts, determining appropriate evaluation metrics, integrating TFLite models into mobile applications efficiently, addressing ethical and privacy concerns related to sensitive data, and maintaining compliance with regulations. Overcoming these challenges requires a multidisciplinary approach, collaboration with domain experts, and adherence to ethical guidelines throughout the project lifecycle.

Solution:-
Solutions include augmenting datasets through transfer learning and crowd-sourcing, employing model pruning and quantization to manage complexity within size constraints, optimizing models for mobile deployment via architecture optimization and hardware acceleration, applying domain adaptation techniques for robustness across contexts, selecting appropriate evaluation metrics tailored to each task, and integrating TFLite models into mobile apps efficiently through platform-specific optimizations. Additionally, implementing robust data encryption and access controls can address ethical and privacy concerns, while fostering collaboration and transparency among interdisciplinary teams ensures effective problem-solving and successful outcomes.

[AbhimanyuSamagra]

Do not ask process related questions about how to apply and who to contact in the above ticket. The only questions allowed are about technical aspects of the project itself. If you want help with the process, you can refer instructions listed on Unstop and any further queries can be taken up on our Discord channel titled DMP queries. Here's a Video Tutorial on how to submit a proposal for a project.

[07-Atharv]

Hello @ChakshuGautam @GautamR-Samagra I am student of the Computer Science and specialization with Artificial Intelligence and machine learning , I have worked on different image processing , machine learning , data analysis projects. I am interested to contribute this repository.

[Azazel0203]

Hey @ChakshuGautam @GautamR-Samagra,

I've developed a collaborative notebook focused on training a model for leave Images. However, due to the extensive dataset I'm working with (54305 images), each epoch takes about 15 minutes, making it challenging to utilize the free Google Collab for extended training sessions.

For now, I've conducted a dummy run with just one epoch, resulting in a sizable TensorFlow Lite model of approximately 14MB. With further experimentation, I believe we can reduce its size even more through trial and error.

I've included the link to the Collab notebook below. Your insights and suggestions on how to proceed would be invaluable.

Looking forward to your feedback. Thanks a bunch!

Collab Notebook Link

[Gautam-Rajeev]

Hey @ChakshuGautam @GautamR-Samagra,

I've developed a collaborative notebook focused on training a model for leave Images. However, due to the extensive dataset I'm working with (54305 images), each epoch takes about 15 minutes, making it challenging to utilize the free Google Collab for extended training sessions.

For now, I've conducted a dummy run with just one epoch, resulting in a sizable TensorFlow Lite model of approximately 14MB. With further experimentation, I believe we can reduce its size even more through trial and error.

I've included the link to the Collab notebook below. Your insights and suggestions on how to proceed would be invaluable.

Looking forward to your feedback. Thanks a bunch!

Collab Notebook Link

Hi, this looks promising.
We were internally only considering a small model to check if the image of the leaf has been clicked properly; no need of the pest/image detection itself- could use a bigger model for that. The results do look good on the tiny model for classification itself, so I think we can take a crack at creating an entire model.

[AyushSarangi]

Hi @ChakshuGautam @GautamR-Samagra

I am Ayush Sarangi, a pre-final year student at IIT, Varanasi. I have a deep interest in problems related to computer vision, which brings me here.
I trained a model on around 17k images that can detect whether plants have disease based on the image of leaves. I trained the model for three epochs on Google Collab because of resource constraints and achieved around 71% accuracy. This accuracy can be further increased without overfitting.

For your reference, I have attached the link to the notebook below. I am eagerly waiting for suggestions to improve it further.
Thank you

notebook - https://colab.research.google.com/drive/1LxrZxS7Geq2PNPpysHSm0a4AMC0xeO2C?usp=sharing
dataset - https://www.kaggle.com/datasets/arjuntejaswi/plant-village