This is how machine vision and deep learning can be used in agriculture

This is how machine vision and deep learning can be used in agriculture

There are many kinds of problems in our daily life. We are always adopting new methods for the solution of which. Technology has developed so much in recent times that every field is incomplete without it.

Today I am trying to do something different, but relevant. You must have heard the name of Computer Vision. To understand this, it is necessary to examine some experimental contexts.

Imagine a mango orchard. Now think about how to solve this space problem using machine-related resources.

In fact, computer vision is related to the visual spectrum. It has different colors that penetrate the eye and you see.

The reason why color is added here is that if a lot of light comes from an object, it looks bright. But if there is less light on an object, it looks dark.

In this sense, light is very important in machine and computer vision. Now let's talk about some of the challenges of machine vision in the external scenario.

There is no problem if you create an interior landscape with the same light in machine vision software and algorithms. But if you want to use it in agriculture, you have to go to a large area.

Where there are large fields between many trees. There the state of light may be different than usual. If you go during the day, you will see that everything is 'overexposed' there.

As the sun shines brightly, the shadow also becomes clear. Some branches, leaves, and fruits fall in the shade. Thus, not all trees have the same amount of sunlight.

How to deal with it, there is a question. For indoor rooms, you need to go to the image processing algorithm. For example, histogram equalization, where you can generalize and equalize the intensity of light.

It works to improve the part of the tree that is in the dark. But over-exposure is still a problem.

If something is over-exposed, you see white in it and do not know its true color. This affects the rest of the programming.

If you are in the field, you can use artificial light. Since it has the same level of light, it doesn't make much difference. You can use algorithms for that.

If you can solve this problem easily and cheaply, you can use LED light. Or you can take the help of a good type of straw light. Which helps you to see the object clearly.

A mobile app that removes shadows and shapes fruit

We have developed an app to take pictures of fruits. We use a backboard to prevent photo noise in the background when taking a picture of a fruit.

Then we put the size, length, width of the fruit. At the same time, we keep their weight. You can also find the weight from the available details.

When you try to segment the image by placing it on white paper, the shadow of the fruit at the bottom of the image may not be good for segmentation and its dimensions may be wrong.

So we use color space conversion to get the actual shape by removing the noise of this shadow. Generally, we have an RGB image and we can use some other software based on open source.

This color changes from one color space to another. In some cases changing the color space helps to normalize the problem to a great extent. For this, you need to change the LAB space.

When changed in this way, the blue color disappears. After that, you can see two pix holograms of its picture. If both of these are fun, you can use the dynamic threshold method.

Therefore, you do not have to specify any threshold value within the program. You can use some dynamic threshold method, which segments the fruit by looking at two pixels.

This happens behind LAB space conversion and thresholding. You can then measure everything by making a box outside the picture of the fruit, its size, and weight.

To measure the size of the fruit is a yellow square. It has a square mark with a diameter of five centimeters. Because when you measure the size of the fruit from the picture, the fruit looks big if it is close to the camera and small if it is close.

So to know the actual size of the fruit, you need to measure the distance between the camera and the fruit. For this, the yellow square acts as a scale.

Scalability

When you design a solution, you have to think about scalability experimentally in real life as well. Mobile phones have many sensors that you can use.

But that doesn't increase the scale. If you want to use this app, you can only use it on a small farm with few trees. But if you want to build a business solution for a large firm, the previous method is not possible.

You need a lot of data. One is to record a video of a farm with a camera. Another is to fly a drone or take another satellite image.

But it depends on what kind of application you are using. Satellite images are usually taken and analyzed for a visitation index to determine if plants are in good health. But if there is fruit on the side of the tree, then the satellite cannot take photos. You have to take a side image of it.

Picture from artificial light at night

We had LED lights, computers, GPS, cameras to take pictures at night. We set the camera on the car and drove off. He was recording the video.

We later analyzed the video images for fruit and flower detection. Due to the lack of variety in light at night, the fruit can be seen better and it is more accurate.

Farmers go to spray at night, then by setting the camera and spraying they can also collect data and get more information for firm management.

There is another way to do fruit size estimation using machine vision. This is also for scalability. For this, you need to take a picture of the tree. Then you take a sample of the size of the fruits on the farm.

The depth camera is used for this. It could be a stereo camera, or it could be a real-sense, time-of-flight Linux camera used in robotics.

You can see the RGB image and his map of the same picture. On the hit map, you can see different depths indicated by different colors. First of all, you detect the fruit.

But when you detect the size of the fruit, you should only detect the undisturbed fruit that is in good condition. Otherwise, you can't get accuracy.

We have enclosed the fruit using the open CV algorithm. If the ellipse is mixed with all the other fruits except the sample fruit, then you can tell that the fruit is good and not spoiled.

If you look at its hit map, you know its depth value. This means you can know the distance of that fruit. In this way, once you have the distance and the picture, you can know the size, length, and width of the fruit.

You need to have data for the machine, deep learning supervised pieces of training. Most bound box methods are used, other methods are depending on the use.

You can use pixel-level segmentation, instant segmentation, semantic segmentation. But if you can use the bounding box method, it takes less time than pixel segmentation. This can be an easy solution.

First, we talked about a kind of model. Now we come to the part where we talk about the middle ground. You can't detect flowers directly. First, you have to classify the flowers into different stages.

There are three stages of flowering. Where stage A is a very early stage. Stage B is the middle stage and stage C is the final stage.

After sorting and detection in this way, it is possible to know how many flowers and fruits my farm has reached the stage of production. Usually, our textbooks are written about the eight stages of the mango flower.

But since it will confuse you, I have only mentioned about three stages. When you easily label it. Then machine learning becomes easier and more accurate.

Is the deep learning model a black box?

Deep learning models are not black boxes. Because they are based on mathematics and can be visualized.

I have used the Deep Learning CNN model to count the number of fruits from the picture. It projects the number of fruits directly from the picture.

Flower evaluation and fruit mapping

You have already counted the number from the projection that bears fruit. The number of flowers has been counted by classifying them in different conditions.

You also have a GPS point. You can block it on Google Maps. For example, make a block in your garden and thread the number of flowers in it.

By displaying your block in any software, you can see where there are thick flowers based on the color. You can see that the fruit may come soon.

You can send people accordingly after seeing the fast ripening fruit in a block. This way you can talk in the market by projecting the size, number, and ripening of the fruit sooner or later.

In this way, you can organize market planning, storage, vehicles, and everything that works. It works as a decision support system for you.

Fruit picking time

Now is the time to pick the fruit. You can also find out the distance between the fruits. Now is the time to bring in crops with automated technology.

For this, the standard industrial arm works. Such robots have up to 10 arms. The new robot has 16 more arms. These arms pick the fruit. Different robots pick different fruits, but because of the structure of your tree, the task of picking you is a bit of a challenge for the robot.

Are they available?

There are many uses of machine vision and deep learning in agriculture. It works from quality assurance to projection of results.

We have new industrial-grade computers to use for large scale agriculture. These computers have GPU computing so that everything in the field can be processed in real-time.

High-Performance Virtual Computers have internet and cloud computing for processing and various IoT devices and sensors are available to collect data.

There are various deep learning algorithms and Tron models for this.

Where are we going

The agricultural sector has also accepted machine vision and AI as the technology of the future. Automatic vehicles without drivers have started running on a large acre of land.

Commercial growers have begun to change the size of their trees and the structure of their fields to match machine vision and robotic harvesting.

Apple trees are beginning to grow in two-spindle trees. Mango plants are also being tested.

Good visibility, accurate machine vision is easy for robotic picking arm and can also launch.

Even if you are an expert in machine learning and its software, data acquisition is always important in machine and computer vision. You can't do a good job if you don't have a good signal.

Always pay attention during data acquisition

The choice of machine vision hardware and computational resources depends on its use. It all depends on what kind of camera, what kind of machine to choose. In the beginning, you should always pay attention to the scalability of your program or productivity.