This story introduces methods for image segmentation and feature extraction. We will process an internet-downloaded image to extract quantitative information about its objects (twenty different fruit types). This information, or features, could be potentially used for classification.
Originally part of a Jupyter notebook, the code can be run locally after creating a compatible environment for the libraries and downloading two files: the FastSAM weights file fastsam-x.pt
from this link, and the analyser.py
available in my GitHub repository. I tested the code with other images and I encourage you to try yours!
The code in this story is part of work involved in two main publications:
2024 Benet, D., Costa, F., Widiwijayanti, C., Pallister, J., Pedreros, G., Allard, P., Humaida, H., Aoki, Y., F., Maeno. VolcAshDB: a Volcanic Ash DataBase of classified particle images and features. Bulletin of Volcanology. http://doi.org/10.1007/s00445-023-01695-4
2024 Benet, D., Costa, F., Widiwijayanti, C. Volcanic ash classification through machine learning. Geochemistry, Geophysics, Geosystems. https://doi.org/10.1029/2023GC011224
Please reference these publications if utilizing this code.
import os # Operating system dependent functionality
import cv2 # Image and video processing
import torch # Machine learning library
from tqdm import tqdm # Progress bars for loops
import numpy as np # Numerical computing with arrays
import tifffile # TIFF file read/write
import matplotlib.pyplot as plt # Plotting and visualization
import pandas as pd # Data manipulation and analysis
from fastsam import FastSAM, FastSAMPrompt # Image super-resolution and restoration
from sklearn.preprocessing import StandardScaler # Feature standardization
from rembg import remove # Background removal from imagesimport warnings # Handle warnings
warnings.filterwarnings('ignore') # Ignore all warnings
# Display matplotlib plots inline in the notebook
%matplotlib inline
plt.rcParams['figure.figsize'] = [20, 12] # Set default figure size for plots
# Define home directory for each user
s = os.getcwd()
if s.split('/')[1] == 'home':
HOME_DIR = ('/').join(s.split('/')[0:3])
print(HOME_DIR)
# Define directory to store
IMAGES_DIR = f'{HOME_DIR}/hands-on-21324/images'
# Define path to image
IM_PATH = f'{IMAGES_DIR}/fruits.jpg'
# Read image. Use cv2 to work with the image as a numpy array.
image = cv2.imread(IM_PATH)
# Output dimensions
print("Height of image:", image.shape[0])
print("Width of image:", image.shape[1])
print("Number of channels:", image.shape[2])
# Visualize image
plt.imshow(image[...,::-1])
plt.axis('off')
/home/usrname
Height of image: 1040
Width of image: 735
Number of channels: 3
To segment the image, we will use the FastSAM model, which is 50 times faster and as accurate as other models like SAM.
# Define FastSAM arguments
IMGSZ=1024
CONF=0.6
IOU=0.9
# Locate directory of its weights
CHECKPOINT_PATH ='/FastSAM-x.pt'
# Obtain image name from path
IMAGE_NAME = IM_PATH.split('/')[-1].split('.')[0]
# Define directory to store segmented images
SAVE_FOLDER = f'{IMAGES_DIR}/{IMAGE_NAME}_imgsz{IMGSZ}_conf{CONF}_iou{IOU}'
print(SAVE_FOLDER)
# Create the save folder if it doesn't exist
os.makedirs(SAVE_FOLDER, exist_ok=True)
# Load the FastSAM model
model = FastSAM(CHECKPOINT_PATH)
DEVICE = 'cuda' if torch.cuda.is_available() else 'cpu'
model.to(DEVICE)
# Use the FastSAM model to segment the image
everything_results = model(image, device=DEVICE, imgsz=IMGSZ, conf=CONF, iou=IOU)
prompt_process = FastSAMPrompt(image, everything_results, device=DEVICE)
annotations = prompt_process.everything_prompt()
prompt_process.plot(annotations=annotations, output_path=f'{SAVE_FOLDER}/segmented.png')
segmented = cv2.imread(f'{SAVE_FOLDER}/segmented.png')
plt.imshow(segmented)
plt.axis('off')
We will now use the contours outputted by FastSAM to crop the objects from the original image. Then, we will use the rembg
library to obtain a finely segmented image without background. Documentation of rembg
can be found in this repository. It is an effective tool to remove the background from various types of images.
# Define a margin to prevent the border from cutting the object
margin = 20 # Margin size in pixels
# Loop over all detected FastSAM contours
for idx in tqdm(range(annotations.shape[0])):
# Convert annotations to a binary mask
mask = annotations[idx].cpu().numpy().astype(np.uint8)# Upscale to match the original image size
upscaled_mask = cv2.resize(mask, (image.shape[1], image.shape[0]), interpolation=cv2.INTER_NEAREST)
# Find the extremities of the mask
rows = np.any(upscaled_mask, axis=1)
cols = np.any(upscaled_mask, axis=0)
y_min, y_max = np.where(rows)[0][[0, -1]]
x_min, x_max = np.where(cols)[0][[0, -1]]
# Apply margin, checking bounds
y_min = max(y_min - margin, 0)
y_max = min(y_max + margin, image.shape[0] - 1)
x_min = max(x_min - margin, 0)
x_max = min(x_max + margin, image.shape[1] - 1)
# Crop the image to the bounding box defined by the extremities
cropped_image = image[y_min:y_max+1, x_min:x_max+1]
# Remove background using rembg library
cropped_alpha_image = remove(cropped_image)
# Save the cropped image
particle_path = os.path.join(SAVE_FOLDER, f'fruit_{idx+1}.png')
cv2.imwrite(particle_path, cropped_alpha_image)
print(f"Segmentation completed and fruits are saved in the {SAVE_FOLDER} folder.")
Segmentation completed and fruits are saved in the /hands-on-210324/images/fruits_imgsz1024_conf0.6_iou0.9 folder.
We keep the images that best represent each fruit and discard the rest. Each image is labelled according to the labels in the original image. This step needs to be done again for any new image.
# Indices of the images with artifacts
indices_to_delete = [4, 8, 14, 16, 17, 18, 20, 21, 23, 25, 26, 27, 28, 32, 33, 34, 36] + list(range(38, 43))
# Loop through the indices and delete the corresponding files
for i in indices_to_delete:
file_path = os.path.join(SAVE_FOLDER, f'fruit_{i}.png')
try:
os.remove(file_path)
print(f"Deleted {file_path}")
except FileNotFoundError:
print(f"File {file_path} not found. Skipping.")
except Exception as e:
print(f"Error deleting {file_path}: {e}")
# Generate the expected image filenames
image_filenames = [f'fruit_{i}.png' for i in range(1, 43)]
# Filter out the existing images
existing_images = [filename for filename in image_filenames if os.path.exists(os.path.join(SAVE_FOLDER, filename))]
# Sort the existing images by their numeric value to ensure they're in ascending order
sorted_existing_images = sorted(existing_images, key=lambda x: int(x.split('_')[1].split('.')[0]))
# Plot the first 20 existing images
plt.figure(figsize=(20, 10))
for i, filename in enumerate(sorted_existing_images[:20], start=1):
img = plt.imread(os.path.join(SAVE_FOLDER, filename))
plt.subplot(5, 4, i) # Adjust subplot dimensions as needed
plt.imshow(img)
plt.title(filename)
plt.axis('off')
plt.tight_layout()
plt.show()
# Log of fruits removed: 4, 8, 14, 16, 17, 18, 20, 21, 23, 25, 26, 27, 28, 32, 33, 34, 36, 38-42
labels = ['Lime', 'Avocado', 'Papaya', 'Watermelon', 'Guava',
'Plum', 'Apple', 'Pomegranate', 'Pear', 'Grapes', 'Pineapple',
'Custard apple', 'Banana', 'Jackfruit', 'Mango', 'Peach',
'Chikoo', 'Orange', 'Cherry', 'Strawberry']
# Fetch all fruit files and sort them
fruit_files = [f for f in os.listdir(SAVE_FOLDER) if f.startswith('fruit_')]
fruit_files_sorted = sorted(fruit_files, key=lambda x: int(x.split('_')[1].split('.')[0]))
# Loop through each sorted file and rename according to the identified fruit names
for i, file in enumerate(fruit_files_sorted):
old_file_path = os.path.join(SAVE_FOLDER, file)
new_file_name = f'fruit_{i+1}_{labels[i]}.png' # Adjust the extension if needed
new_file_path = os.path.join(SAVE_FOLDER, new_file_name)
os.rename(old_file_path, new_file_path)
labeled_fruits = [i for i in os.listdir(SAVE_FOLDER) if i.startswith('fruit')]
# Plot the first 20 existing images
plt.figure(figsize=(20, 10))
for i, filename in enumerate(labeled_fruits, start=1):
img = plt.imread(os.path.join(SAVE_FOLDER, filename))
plt.subplot(5, 4, i) # Adjust subplot dimensions as needed
plt.imshow(img)
plt.title(filename.split('.')[0].split('_')[-1])
plt.axis('off')
plt.tight_layout()
plt.show()
We have obtained 20 images of the different fruit types, each of them labelled and segmented. We can now make measurements of physical properties of the fruits by extracting a series of features related to shape, texture, and color. The image Grapes
will be used to introduce the features.
import sys
# Append the directory of analyzer.py to the system path
sys.path.append('/hands-on-210324/SegmentationHandsOn/FastSAM/')
#from helper_qia import mask, shape, texture, color # Import specific functions
from skimage.color import rgb2gray
from skimage import measure
from skimage.feature import graycomatrix, graycoprops
from skimage.measure import label, regionprops
from skimage.morphology import convex_hull, convex_hull_image
from skimage.util import img_as_ubyte
from scipy import stats
from skimage.color import rgb2gray
from skimage import img_as_ubyte
from skimage.feature import graycomatrix, graycoprops
import math, skimage
The output of rembg
consists in a RGBA image. “A” for “Alpha” which is the channel that gives the transparency to png images and consists of a mask that separates foreground from background.
# Load the image
image_path = f'{SAVE_FOLDER}/fruit_10_Grapes.png'
ch4 = cv2.imread(image_path, cv2.IMREAD_UNCHANGED)
# Extract alpha channel
alpha_ch = ch4[..., 3]
# Extract RGB channels and convert from BGR to RGB
rgb_image = cv2.cvtColor(ch4[..., :3], cv2.COLOR_BGR2RGB)
# Create a binary mask using thresholding on the alpha channel
ret, thr = cv2.threshold(alpha_ch, 120, 255, cv2.THRESH_BINARY)
# Find contours in the binary mask
contours, hier = cv2.findContours(thr, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
# Select the contour with the maximum length
contour = max(contours, key=len)
# Create a binary mask for the selected contour
part_mask = np.zeros(rgb_image.shape[:2], dtype=np.uint8)
part_mask = cv2.drawContours(part_mask, [contour], -1, (255), thickness=cv2.FILLED)
# Apply the mask to the RGB image to extract the part of interest
rgb_masked = cv2.bitwise_and(rgb_image, rgb_image, mask=part_mask)
# Create a white image with the same dimensions as the alpha channel
part = 255 * np.ones(alpha_ch.shape, dtype=np.uint8)
# Draw the contour on the white image
part = cv2.drawContours(part, [contour], -1, (0), thickness=5)
# Create figure and axes
fig, axes = plt.subplots(1, 3, figsize=(15, 5))
# Display RGBA image
axes[0].imshow(cv2.cvtColor(ch4, cv2.COLOR_BGR2RGBA)) # Convert BGR to RGBA
axes[0].set_title("RGBA")
# Display RGB image
axes[1].imshow(rgb_image)
axes[1].set_title("RGB")
# Display alpha channel
axes[2].imshow(alpha_ch, cmap='gray')
axes[2].set_title("Alpha")
# Hide axes
for ax in axes:
ax.axis('off')
# Show plot
plt.show()
Shape features can be obtained from the particle contour. These are responsive to perimeter-based irregularities, particle-scale cavities, and/or overall particle form. To measure them, it is first necessary to obtain certain particle properties, such as the perimeter, the area, and the convex hull.
# Find contour and image moments
M = cv2.moments(contour) # Calculate moments of the contour
minRect = cv2.minAreaRect(contour) # Get the minimum area rectangle that encloses the contour
_, (w, l), _ = minRect # Get the width and height of the minimum area rectangle
box = cv2.boxPoints(minRect) # Transform to numpy
box = np.intp(box) # with integers
# Skimage functions to obtain morphological properties
label_image = label(alpha_ch) # Use skimage function to label regions in the binary image
regions = regionprops(label_image) # Get properties of the labeled regions
area = [ele.area for ele in regions] # Get the area of each region
largest_region_idx = np.argmax(area) # Find the index of the largest region
props = regions[largest_region_idx] # Get properties of the largest region
major_ellipse_axis = props.major_axis_length # Major axis length of the ellipse fitted to the region
minor_ellipse_axis = props.minor_axis_length # Minor axis length of the ellipse fitted to the region
# Particle descriptives
y_centroid, x_centroid = props.centroid # Get the centroid of the largest region
part_perim = props.perimeter # Get the perimeter of the largest region
area = props.area # Get the area of the largest region
# Calculate hull
hull = convex_hull_image(thr) # Calculate the convex hull of the binary image
hull_perim = measure.perimeter(hull) # Calculate the perimeter of the convex hull
# Measure properties
aspect_ratio = major_ellipse_axis / minor_ellipse_axis
eccentricity = props.eccentricity # Get the eccentricity of the largest region
solidity = props.solidity # Get the solidity of the largest region
convexity = hull_perim / part_perim # Calculate convexity
circularity_dellino = part_perim / (2 * math.sqrt(math.pi * area)) # Calculate circularity using Dellino's formula
circ_func = lambda r: (4 * math.pi * r.area) / (r.perimeter * r.perimeter) # Define a circularity function
circ = circ_func(props) # Calculate circularity using Cioni's formula
rectangularity = part_perim / (2 * l + 2 * w) # Calculate rectangularity
compactness = area / (l * w) # Calculate compactness
elongation = (props.feret_diameter_max ** 2) / area # Calculate elongation
roundness = 4 * area / (math.pi * (props.feret_diameter_max ** 2)) # Calculate roundness
# Plotting
fig, axs = plt.subplots(1, 2, figsize=(12, 6))
# Plot the first image with rectangle
black = 255 * np.ones(rgb_masked.shape, dtype=np.uint8)
black = cv2.cvtColor(black, cv2.COLOR_RGB2RGBA)
black[:, :, 3] = alpha_ch
cv2.fillPoly(black, pts=[contour], color=(200, 200, 200, 255))
hull_np = np.array(hull, dtype=np.uint8)
cnts_hull, hier = cv2.findContours(hull_np, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
contour_hull = cnts_hull[0]
cv2.fillPoly(black, pts=[contour_hull], color=(40, 180, 40, 80))
cv2.drawContours(black, contour_hull, -1, [40, 40, 180, 255], 1)
cv2.drawContours(black, contour, -1, [0, 0, 0, 255], 1)
axs[0].imshow(black)
axs[0].axis('off')
# Plot the second image with ellipse
black = 255 * np.ones(rgb_masked.shape, dtype=np.uint8)
black = cv2.cvtColor(black, cv2.COLOR_RGB2RGBA)
black[:, :, 3] = alpha_ch
# Draw rectangle
cv2.fillPoly(black, pts=[contour], color=(200, 200, 200, 255))
cv2.drawContours(black, [box], -1, [180, 40, 40, 255], 1)
# Draw ellipse
rr, cc = skimage.draw.ellipse_perimeter(int(x_centroid), int(y_centroid), int(props.minor_axis_length * 0.5), int(props.major_axis_length * 0.5), orientation=props.orientation)
angle = np.arctan2(rr - np.mean(rr), cc - np.mean(cc))
sorted_by_angle = np.argsort(angle)
rrs = rr[sorted_by_angle]
ccs = cc[sorted_by_angle]
contour_ellipse = np.stack((rrs, ccs), axis=-1)
cv2.drawContours(black, [contour_ellipse], -1, [40, 40, 180, 255], 1)
axs[1].imshow(black)
axs[1].axis('off')
plt.tight_layout()
plt.show()
plt.close()
# Create dictionary containing shape properties
shape_dict = {
'convexity': convexity,
'rectangularity': rectangularity,
'elongation': elongation,
'roundness': roundness,
'circularity': circularity_dellino,
'solidity': solidity,
'aspect_rat': aspect_ratio,
'compactness': compactness,
}
# Print the calculated shape dictionary with only two decimals
for key, value in shape_dict.items():
print(f'{key}: {value:.2f}')
convexity: 0.73
rectangularity: 1.16
elongation: 2.73
roundness: 0.47
circularity: 1.82
solidity: 0.82
aspect_rat: 1.61
compactness: 0.54
Convexity
Convexity is a measure of the contour’s complexity compared to its convex hull. It is defined by the formula:
A value of 1 indicates a perfectly convex shape, while values greater than 1 indicate deviations from convexity, reflecting the shape’s complexity.
Rectangularity
Rectangularity quantifies how much a shape resembles a rectangle, calculated as:
Values closer to 1 suggest a shape that closely approximates a rectangle.
Elongation
Elongation indicates the extent to which a shape is elongated, derived from the ellipse that has the same normalized second central moments as the shape:
Values significantly greater than 1 suggest an elongated shape.
Roundness
Roundness measures how close a shape is to being circular, calculated as:
A roundness value of 1 indicates a perfect circle.
Circularity
Circularity assesses a shape’s similarity to a circle based on its perimeter and area, defined as:
A circularity value of 1 perfectly defines a circle, with lower values indicating less circular shapes.
Solidity
Solidity is the ratio of the shape’s area to its convex hull’s area, indicating the “solidness” of the shape:
Values range from 0 to 1, with 1 indicating a completely solid shape without concavities.
Aspect Ratio
The aspect ratio is the ratio of the width to the height of the shape’s bounding rectangle:
It indicates the orientation and proportionality of the shape, with higher values indicating a landscape orientation.
Compactness
Compactness measures how efficiently a shape’s area is concentrated, related to its perimeter:
Higher compactness values suggest a more densely packed shape, which is more circular or square in nature.
Textural features can be calculated from local pixel intensity distributions on the object’s surface, providing insights into the spatial distribution of grayscale pixel intensity values. These features aim to characterize surfaces, distinguishing between high textural complexity and uniform smoothness. We apply the Gray Level Co-occurrence Matrix (GLCM), as introduced by Haralick (1973), to quantify texture.
# Measure textural features from 'image'
bins = np.array([0, 16, 32, 48, 64, 80, 96, 112, 128, 144, 160, 176, 192, 208, 224, 240, 255]) # 16-bit
gray = rgb2gray(rgb_masked) # Convert RGB image to grayscale
image = img_as_ubyte(gray) # Convert grayscale image to 8-bit unsigned integer format
inds = np.digitize(image, bins) # Digitize the image into bins
# Get dimensions of the digitized image
height, width = inds.shape
# Calculate maximum value in the digitized image
max_value = inds.max() + 1
# Define patch size and step
PATCH_SIZE = int(props.major_axis_length / 10)
STEP = int(props.major_axis_length / 10)
# Define angles every 22.5 degrees
thetas = [0, np.pi/8, np.pi/4, np.pi/2, 3*np.pi/4, np.pi, np.pi+np.pi/4, np.pi+np.pi/2, np.pi+3*np.pi/4,
np.pi/4 + np.pi/8, np.pi/2 + np.pi/8, 3*np.pi/4 + np.pi/8, np.pi + np.pi/8, np.pi+np.pi/4 + np.pi/8,
np.pi+np.pi/2 + np.pi/8, np.pi+3*np.pi/4 + np.pi/8]
# Initialize lists for extreme coordinates and locations
extreme_co = [] # Particle border at theta
locs = []
# Iterate over angles to find particle borders
for theta in thetas:
for ii in range(1000):
i = ii + 1
new_y = int(y_centroid - np.sin(theta) * STEP * i)
new_x = int(x_centroid + np.cos(theta) * STEP * i)
if new_x < width and new_x > 0 and new_y < height and new_y > 0:
co = (new_y, new_x)
if alpha_ch[co] > 0:
co_no_background = co
locs.append(co_no_background)
else:
extreme_co.append(co_no_background)
break
# Initialize lists for locations without background and patches
locs_no_background = []
patches = []
# Iterate over locations to extract patches
for idx, loc in enumerate(locs):
patch = alpha_ch[loc[0]:loc[0] + PATCH_SIZE, loc[1]:loc[1] + PATCH_SIZE]if patch.min() > 0:
glcm_patch = inds[loc[0]:loc[0] + PATCH_SIZE, loc[1]:loc[1] + PATCH_SIZE]
locs_no_background.append((loc[0], loc[1]))
patches.append(glcm_patch)
# Define distances for GLCM computation
distances = [1]
for i in range(5):
d = int((i + 1) / 5 * PATCH_SIZE)
distances.append(d)
# Initialize lists for texture features
contrast_patches = []
dissimilarity_patches = []
homogeneity_patches = []
energy_patches = []
correlation_patches = []
asm_patches = []
# Iterate over patches to compute GLCM and texture features
for idx, patch in enumerate(patches):
matrix_coocurrence = graycomatrix(patch, distances, thetas, levels=max_value, normed=False, symmetric=False)
contrast = graycoprops(matrix_coocurrence, 'contrast')
contrast_mean = contrast.mean()
contrast_patches.append(contrast_mean)
dissimilarity = graycoprops(matrix_coocurrence, 'dissimilarity')
dissimilarity_mean = dissimilarity.mean()
dissimilarity_patches.append(dissimilarity_mean)
homogeneity = graycoprops(matrix_coocurrence, 'homogeneity')
homogeneity_mean = homogeneity.mean()
homogeneity_patches.append(homogeneity_mean)
energy = graycoprops(matrix_coocurrence, 'energy')
energy_mean = energy.mean()
energy_patches.append(energy_mean)
correlation = graycoprops(matrix_coocurrence, 'correlation')
correlation_mean = correlation.mean()
correlation_patches.append(correlation_mean)
asm = graycoprops(matrix_coocurrence, 'ASM')
asm_mean = asm.mean()
asm_patches.append(asm_mean)
# Calculate averaged texture features
contrast_averaged = sum(contrast_patches) / len(contrast_patches)
dissimilarity_averaged = sum(dissimilarity_patches) / len(dissimilarity_patches)
homogeneity_averaged = sum(homogeneity_patches) / len(homogeneity_patches)
energy_averaged = sum(energy_patches) / len(energy_patches)
correlation_averaged = sum(correlation_patches) / len(correlation_patches)
asm_averaged = sum(asm_patches) / len(asm_patches)
# Create a dictionary of texture features
text_dict = {
'Contrast': contrast_averaged,
'Dissimilarity': dissimilarity_averaged,
'Homogeneity': homogeneity_averaged,
'Energy': energy_averaged,
'Correlation': correlation_averaged,
'ASM': asm_averaged
}
plt.figure(figsize=(15, 8))
# Displaying binned digitalized image
plt.imshow(inds, cmap='gray')
# Extracting x and y locations from a list of coordinates (locs)
xs_locs, ys_locs = zip(*locs)
# Extracting x and y locations from a list of coordinates excluding background (locs_no_background)
xs2, ys2 = zip(*locs_no_background)
# Adjusting x and y locations to center them based on a PATCH_SIZE
xs = [i + PATCH_SIZE/2 for i in xs2]
ys = [i + PATCH_SIZE/2 for i in ys2]
# Plotting a specific location (the 35th in the list) as a red square with full opacity
plt.plot(ys[34], xs[34], 'rs', markersize=20, alpha=0.8, c='red',
markeredgecolor=(0.3, 0.1, 0.1), markeredgewidth=0)
# Plotting a centroid location as a green square with full opacity
plt.plot(x_centroid + PATCH_SIZE/2, y_centroid + PATCH_SIZE/2, 'gs', markersize=20, alpha=0.8, c='green',
markeredgecolor=(0.1, 0.3, 0.1), markeredgewidth=0)
# Plotting locations excluding background as white circles with black edges
plt.plot(ys2, xs2, 'ro', markeredgecolor='black', markersize=8, color='white', markeredgewidth=0.2)
# Drawing lines from each extreme coordinate to the centroid, illustrating connections or distances
for x, y in extreme_co:
xs2 = [x, y_centroid]
ys2 = [y, x_centroid]
plt.plot(ys2, xs2, c='black', linestyle='-', linewidth=0.2)
plt.axis('off')
plt.show()
plt.close()
# Take our "patch" for the example
img = np.array(patches[35], dtype = np.uint16)
# Display patch
plt.imshow(img, cmap='Greys_r')
# Iterate over each pixel in the image
for i in range(img.shape[0]):
for j in range(img.shape[1]):
# Set text color based on the pixel value for better contrast
if img[i, j] <= 6:
color = "white"
else:
color = "black"
# Annotate each pixel with its value, centered in both directions
plt.text(j, i, img[i, j], ha="center", va="center", color=color, size=8)plt.xlabel('# of pixels')
plt.ylabel('# of pixels')
#plt.axis('off')
plt.show()
The GLCM is a two-dimensional matrix that represents the frequency of occurrence of pairs of pixel intensity values at a specific distance and angle in an image. The process of calculating GLCM involves the following steps:
- Choose a distance (d) and an angle (θ) to define the pixel pairs’ relative positions. Common angles are 0°, 45°, 90°, and 135°, and distances typically start from 1.
- For each pixel in the image, consider its neighboring pixel at the specified distance and angle.
- Increment the corresponding entry in the GLCM matrix by 1 for each occurrence of a particular pixel pair in the image.
GLCM calculations often use normalized values, making each entry a probability that indicates the likelihood of encountering a particular pixel pair in the image.
# Calculation of GLCM
matrix_coocurrence = graycomatrix(img, distances=[1], angles=[0], levels=max_value+1, symmetric=True, normed=False)
m = np.array(matrix_coocurrence)
# Calculate textural features from the GLCM
contrast = graycoprops(m, 'contrast')[0, 0]
dissimilarity = graycoprops(m, 'dissimilarity')[0, 0]
homogeneity = graycoprops(m, 'homogeneity')[0, 0]
energy = graycoprops(m, 'energy')[0, 0]
correlation = graycoprops(m, 'correlation')[0, 0]
ASM = graycoprops(m, 'ASM')[0, 0]
# Extract the GLCM for visualization
glcm_im = np.array(m[:,:,0])
glcm_im = glcm_im[:,:,0]
plt.imshow(glcm_im, cmap = 'viridis')
# Annotate each cell in the GLCM with its value for detailed inspection
for i in range(glcm_im.shape[0]):
for j in range(glcm_im.shape[1]):
plt.text(j, i, (glcm_im[i, j]),
ha="center", va="center", color="w", size = 8)
plt.xticks(np.arange(0, max_value, 1.0))
plt.yticks(np.arange(0, max_value, 1.0))
plt.show()
# Prepare dictionary of calculated features
text_dict = {
'Contrast': contrast,
'Dissimilarity': dissimilarity,
'Homogeneity': homogeneity,
'Energy': energy,
'Correlation': correlation,
'ASM': ASM
}
# Print each textural feature with its name and value, formatted to two decimal places
for key, value in text_dict.items():
print(f'{key}: {value:.2f}')
Contrast: 0.77
Dissimilarity: 0.61
Homogeneity: 0.71
Energy: 0.27
Correlation: 0.84
ASM: 0.07
Contrast
Measures the intensity contrast between neighboring pixels. It increases with the intensity differences between pixel pairs. Higher values indicate more diverse and contrasting textures.
The contrast (C) is calculated using the following formula:
Here, P(i, j) represents the normalized GLCM element at row i and column j, and N is the number of gray levels in the image. The contrast value is higher for images with sharp transitions and lower for more uniform textures.
Dissimilarity
Measures the average difference between pixel pairs in the image. It is similar to contrast but emphasizes the difference between pixel intensities, irrespective of their direction. The dissimilarity value is higher for textures with varying pixel intensities and lower for more uniform textures.
The dissimilarity (D) is calculated using the following formula:
Homogeneity
Also known as Inverse Difference Moment, measures the closeness of the GLCM elements to the matrix diagonal. It provides a measure of how similar neighboring pixels are in intensity. The homogeneity value is higher for textures with more similar pixel intensities, indicating smoother and more regular textures.
The homogeneity (H) is calculated using the following formula:
Correlation
Measures the linear dependency between pixel pairs in the image. It provides an indication of how much the pixel values are correlated with their neighbors.
The correlation (R) is calculated using the following formula:
Here, μ represents the mean of the GLCM, and σi and σj represent the standard deviations along the rows and columns of the GLCM, respectively. The correlation value is higher for textures with a strong linear relationship between pixel values.
Angular Second Moment (ASM)
Measures the uniformity or regularity of the image texture. It is directly related to the GLCM’s entropy and is a measure of the texture’s orderliness. The ASM value is higher for images with more homogenous and regular textures and lower for images with more complex and disordered textures.
The ASM is calculated using the following formula:
Energy
It is simply the square root of the Angular Second Moment (ASM). It provides the same information as ASM but is more intuitive because it is on the same scale as the pixel intensities. Energy ranges between 0 and 1, with 1 indicating a very uniform and smooth texture.
The energy (E) is calculated using the following formula:
Color features can be extracted from both the Red–Green–Blue (RGB) and Hue-Saturation-Value (HSV) channel distributions of the images. These features are sensitive to chromaticity, indicative of dominant color hues, intensity, and brightness.
Colors in the Hue-Saturation-Value space can be expressed as represented by this cylinder:
Implementation to visualize and compute statistics of RGB and HSV histograms
# 'rgb_masked' is the RGB image with the background already removed
image = rgb_masked
# Create a figure with 2 subplots: 1 for the RGB image and 1 for the histograms
fig, axes = plt.subplots(1, 2, figsize=(12, 6))
# Display the RGB image on the left subplot
axes[0].imshow(image)
axes[0].set_title('RGB Image')
axes[0].axis('off') # Hide axes for the image
# Loop over each color channel in the RGB image (blue, green, red) for histogram plotting on the right subplot
for i, c in enumerate(['blue', 'green', 'red']):
# Extract the current channel from the image
channel = image[..., i]
# Remove background by filtering out pixels with a value of 0
values = channel[channel > 0]
# Calculating and plotting the histogram for each color channel
hist = cv2.calcHist([values], [0], None, [255], [1, 256])
axes[1].plot(hist, color=c, label=f'{c} channel')# Calculating mean and mode for vertical lines
mean_val = np.mean(values)
mode_val = int(stats.mode(values)[0])
# Drawing vertical lines for mean and mode
axes[1].axvline(mean_val, linestyle='-', color=c, linewidth=0.6, label=f'{c} mean')
axes[1].axvline(mode_val, linestyle='--', color=c, linewidth=0.6, label=f'{c} mode')
# Setting labels and formatting for the histogram plot
axes[1].set_xlabel('Pixel value')
axes[1].set_ylabel('# of pixels')
axes[1].legend().set_visible(False)
axes[1].set_title('Histograms of RGB Channels')
# Show the complete figure with both the image and histograms
plt.tight_layout()
plt.show()
import matplotlib.ticker as mtick
# Conversion from BGR to HSV is necessary for color space analysis, especially for understanding color in terms of hue, saturation, and value
hsv_image = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)fig, ax = plt.subplots(3, 1, figsize=(8, 12))
# Define custom colors for plotting each of the HSV components for clarity and visual distinction
colors = ['darkblue', 'orange', 'purple']
# Initializing a dictionary to store calculated statistics for hue, saturation, and value
color_dict = {}
# Loop over the HSV image's channels (hue, saturation, value) to analyze and plot each
for i, c in enumerate(['hue', 'saturation', 'value']):
# Extract the current channel's data
channel = hsv_image[..., i]
# Filter out pixels with value 0, assuming these represent the background or black
# This is especially relevant for hue, where a value of 0 also represents a color (red)
values = channel[channel > 0]
# Calculate mean, standard deviation, and mode for the current channel and store them in the dictionary
color_dict[f'{c}_mean'] = values.mean()
color_dict[f'{c}_std'] = values.std()
color_dict[f'{c}_mode'] = int(stats.mode(values)[0])
# Set bins for histogram based on the channel being processed. Hue values range from 0-179 in OpenCV, while saturation and value range from 0-255
bins = 179 if i == 0 else 255
# Calculate and plot the histogram for the current channel
hist = cv2.calcHist([values], [0], None, [bins], [0, bins+1])
ax[i].plot(hist, color=colors[i], label=c, alpha=0.4)
# Draw vertical lines indicating the mode and mean of the channel's values
ax[i].axvline(int(stats.mode(values)[0]), linestyle='--', color=colors[i], linewidth=0.2)
ax[i].axvline(int(values.mean()), linestyle='-', color=colors[i], linewidth=0.2)
# Set the x-axis limit to match the range of possible values for the current channel
ax[i].set_xlim(0, bins)
# Set labels and format the y-axis to use scientific notation for readability
ax[i].set_xlabel('Range of values')
ax[i].set_ylabel('# of pixels')
ax[i].legend()
ax[i].ticklabel_format(axis='Y', style='sci', scilimits=(0, 0), useMathText=True)
plt.show()
# Print the calculated statistics for hue, saturation, and value, formatted to two decimal places
for key, value in color_dict.items():
print(f'{key}: {value:.2f}')
hue_mean: 84.14
hue_std: 3.90
hue_mode: 85.00
saturation_mean: 156.95
saturation_std: 52.21
saturation_mode: 255.00
value_mean: 177.02
value_std: 32.92
value_mode: 178.00
Mean (RGB)
The mean of a color channel (red, green, blue) represents the average intensity of that color in the image. Calculated as:
Meancolor=∑pixel valuesnumber of pixelsMeancolor=number of pixels∑pixel values
It reflects the overall brightness or dominance of a color within the image. Higher mean values indicate greater presence or intensity of the color.
Standard Deviation (RGB)
The standard deviation for each RGB channel quantifies the variation or spread of pixel intensities around the mean:
Standard Deviationcolor=∑(pixel value−Meancolor)2number of pixelsStandard Deviationcolor=number of pixels∑(pixel value−Meancolor)2
A higher standard deviation indicates a wider range of intensities and thus more variation in the presence of the color across the image.
Mode (RGB)
The mode of a color channel is the most frequently occurring pixel intensity within that channel:
Modecolor=Most frequent pixel valueModecolor=Most frequent pixel value
It represents the most dominant intensity value of the color in the image. The mode can be especially insightful in images with distinct color patterns or where a particular color intensity dominates.
Mean (HSV)
The mean in the HSV (Hue, Saturation, Value) color space for each component has similar implications as in the RGB space but reflects different aspects of color:
- Hue mean indicates the average color tone.
- Saturation mean shows the overall intensity or purity of colors.
- Value mean represents the average brightness of the image.
Standard Deviation (HSV)
The standard deviation in the HSV space indicates the variation in color tones, saturation levels, and brightness across the image:
- A higher hue standard deviation suggests a wide variety of colors.
- A higher saturation standard deviation points to areas of both intense and dull colors.
- A higher value standard deviation indicates significant differences in brightness.
Mode (HSV)
The mode for each HSV component points to the most predominant color tone, saturation level, and brightness in the image:
- Hue mode identifies the most common color.
- Saturation mode reveals the most frequent intensity/purity of colors.
- Value mode indicates the most common brightness level.
I piped the methods for extracting the shape, texture and color features into analyzer.py
. Each of the fruits is analyzed as shown below, and the results compiled into a pandas
dataframe.
from analyzer import ImageProcessor, ShapeAnalyzer, TextureAnalyzer, ColorAnalyzer
def extract_features(directory):
# Generate a list of filenames in the specified directory that start with "fruit", indicating the images to be processed.
filenames = [f for f in os.listdir(directory) if f.startswith('fruit')]# Initialize an empty dictionary to store all the quantitative image analysis (QIA) results.
qia_dict = {}
# Initialize a counter to track the total number of processed images; however, note that this counter is not incremented within the loop.
total = 0
# Iterate over each filename, using tqdm to display a progress bar for user feedback.
for filename in tqdm(filenames):
# Extract the type of fruit from the filename, assuming a naming convention where the fruit type is the third element when split by "_".
fruit = filename.split('_')[2].split('.')[0]
# Create the full path to the image file.
full_path = os.path.join(directory, filename)
# Initialize the ImageProcessor with the path to the current image.
processor = ImageProcessor(full_path)
# Perform shape analysis on the image, obtaining shape-related features.
shape_analyzer = ShapeAnalyzer(processor)
shape_features = shape_analyzer.analyze()
# Perform texture analysis, using properties determined by the shape analyzer as input.
texture_analyzer = TextureAnalyzer(processor, shape_analyzer.props)
texture_features = texture_analyzer.analyze()
# Perform color analysis to extract color-related features from the image.
color_analyzer = ColorAnalyzer(processor)
color_features = color_analyzer.analyze()
# Aggregate the extracted features into the results dictionary, keyed by filename.
qia_dict[filename] = {**shape_features, **texture_features, **color_features}
# Convert the results dictionary into a DataFrame for easier manipulation and storage.
df = pd.DataFrame.from_dict(qia_dict, orient='index')
# Save the DataFrame to a CSV file in the specified SAVE_FOLDER, appending if the file already exists. This step stores the analysis results persistently.
df.to_csv(os.path.join(SAVE_FOLDER, 'qia_fruits.csv'), mode='a', header=True)
# Return the DataFrame containing the analysis results.
return df
# Extract features from the images in the specified directory (SAVE_FOLDER needs to be defined prior to this call).
qia_df = extract_features(SAVE_FOLDER)
# Add column 'fruit_type', derived from the filenames. This facilitates easy access to the fruit type for each entry.
qia_df['fruit_type'] = qia_df.index.astype(str).map(lambda x: x.split('_')[2].split('.')[0])
Features are standardized to have a mean at 0 and a variance at 1 by applying Scikit-learn’s StandardScaler. This pre-processing step is commonly used to enable comparison amongst features.
from sklearn.preprocessing import StandardScaler
from scipy.stats import zscore
# Selecting only the numeric columns for scaling
numeric_cols = qia_df.select_dtypes(include=['float64', 'int64']).columns
# Initialize the StandardScaler
scaler = StandardScaler()
# Fit the scaler to the data and transform it
qia_df_scaled = scaler.fit_transform(qia_df[numeric_cols])
# The result is a numpy array. Let's turn it back into a DataFrame
qia_df_scaled = pd.DataFrame(qia_df_scaled, columns=numeric_cols)# Calculate z-scores for the DataFrame's numeric columns
numeric_cols = qia_df_scaled.select_dtypes(include=['float64', 'int64']).columns
qia_df_zscores = qia_df_scaled[numeric_cols].apply(zscore)
qia_df_zscores.index = qia_df['fruit_type']
# Apply a background gradient to these z-scores for visualization
styled_df = qia_df_zscores.style.background_gradient(cmap='RdYlGn', axis=None)
styled_df
Hue mean vs Elongation
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.offsetbox import OffsetImage, AnnotationBbox
def getImage(path, zoom, alpha):
"""
Load an image from a specified path and adjust its transparency and zoom.
Arguments:
path (str): The file path to the image.
zoom (float): The zoom level for the image which scales the image up or down.
A zoom level greater than 1 enlarges the image, while a zoom level less than 1 reduces the size of the image.
alpha (float): The alpha level for the image, controlling its opacity.
This value should be between 0 (completely transparent) and 1 (completely opaque).
Returns:
OffsetImage: An OffsetImage object containing the image data with applied zoom and transparency.
This object is suitable for plotting in matplotlib, allowing for detailed control over the image's appearance.
"""
# Load the image from the given path
img_data = plt.imread(path)# Check if the image has an alpha channel (transparency)
if img_data.shape[2] == 4:
# If it has an alpha channel, adjust the transparency by multiplying the alpha values by the specified alpha factor
img_data[:, :, 3] *= alpha
else:
# If the image does not have an alpha channel, create a new alpha channel filled with the specified alpha value
# This is done by first getting the height and width of the image
h, w, _ = img_data.shape
# Creating a new alpha channel matrix filled with the alpha value
new_alpha_channel = np.ones((h, w)) * alpha
# Stacking the new alpha channel with the existing image data to add transparency
img_data = np.dstack((img_data, new_alpha_channel))
# Return the image as an OffsetImage object, which can be used to display the image at a specified zoom level
return OffsetImage(img_data, zoom=zoom)
fig, ax = plt.subplots(figsize=(10, 8))
# Adjust zoom and alpha as needed
zoom_level = 0.35
alpha_value = 0.8
# Plotting images for each point
for idx, row in qia_df.iterrows():
img_path = f'{SAVE_FOLDER}/{idx}' # Construct the path to the image
ab = AnnotationBbox(getImage(img_path, zoom=zoom_level, alpha=alpha_value),
(row['HSV_hue_mean'], row['elongation']), frameon=False)
ax.add_artist(ab)
# Scatter plot for visibility of points (optional)
ax.scatter(qia_df['HSV_hue_mean'], qia_df['elongation'], color = 'black')
plt.xlabel('Hue Mean')
#plt.ylim(1.2, 2.7)
plt.ylabel('Elongation')
plt.show()
The features successfully capture the physical properties of the fruits. On the x-axis, “Hue Mean,” indicative of chromaticity, displays a marked transition from green to red, moving through shades of orange and yellow. Similarly, the y-axis “Elongation” illustrates a transition from fruits with more equal dimensions at lower values to those that are highly elongated, exemplified by the Pineapple
.
Convexity vs Dissimilarity
import numpy as np
import matplotlib.pyplot as plt
from matplotlib.offsetbox import OffsetImage, AnnotationBbox
fig, ax = plt.subplots(figsize=(10, 8))
# Adjust zoom and alpha as needed
zoom_level = 0.35
alpha_value = 0.8
# Plotting images for each point
for idx, row in qia_df.iterrows():
img_path = f'{SAVE_FOLDER}/{idx}' # Construct the path to the image
ab = AnnotationBbox(getImage(img_path, zoom=zoom_level, alpha=alpha_value),
(row['convexity'], row['dissimilarity_avg']), frameon=False)
ax.add_artist(ab)
# Scatter plot for visibility of points (optional)
ax.scatter(qia_df['convexity'], qia_df['dissimilarity_avg'], color = 'black')
plt.xlabel('Convexity')
#plt.ylim(1.2, 2.7)
plt.ylabel('Dissimilarity')
plt.show()
In PCA, principal components (PCs) are created from a linear combination of features to maximize the variance captured from the dataset. The variance each PC accounts for is known as “explained variance” and is presented as a percentage of the total variance across all features. Additionally, PCA assigns a “loading” to each feature, quantifying its contribution to each principal component.
from pca import pca
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt
import seaborn as sns# Initialize PCA
pca = PCA(n_components=3)
# Transform original values to PC space
X_pca = pca.fit_transform(qia_df_scaled.values)
y = qia_df['fruit_type']
# Convert PCA components into a DataFrame for easy plotting
pca_df = pd.DataFrame(data=X_pca, columns=['Principal Component 1', 'Principal Component 2', 'Principal Component 3'])
pca_df['Fruit Type'] = y.reset_index(drop=True) # Reset index to ensure alignment
# Example of using the updated function
fig, ax = plt.subplots(figsize=(10, 8))
for i in range(len(X_pca[:20])):
img_path = f'{SAVE_FOLDER}/{qia_df.index[i]}'
ab = AnnotationBbox(getImage(img_path, zoom=0.35, alpha=0.8), (X_pca[i, 0], X_pca[i, 1]), frameon=False)
ax.add_artist(ab)
ax.scatter(X_pca[:20, 0], X_pca[:20, 1], color = 'black')
plt.xlabel('Principal Component 1')
plt.ylabel('Principal Component 2')
plt.title('2D PCA with Images')
plt.show()
plt.close()
# pca library to call built-in model.biplpot function
model.biplot(PC=[0, 1], cmap=None, label=False, legend=False, title='', color_arrow='darkred', n_feat = 10, arrowdict={'fontsize': 20}) #
PC1 and PC2 have a combined explained variance of about 50%, indicating they capture half of the overall variability of the dataset.
The lower plot shows the magnitude and direction of the loadings. For example, the mean value of the HSV color space channel, termed HSV_value_mean
, is subparallel to PC2, indicating a significant contribution (-0.321 specifically). Observing the fruits in the upper plot reveals a gradient parallel to the loading from darker (e.g., Avocado
) to brighter (e.g., Mango
).
Imagine having a curated, extensive dataset of fruits on which we have performed similar analyses. If the feature values for each type of fruit form distinctive clusters, we could identify a set of conditions or decision rules unique to each fruit. This could be achieved for instance with a decision tree:
from sklearn.tree import DecisionTreeClassifier
from sklearn.tree import plot_tree
from sklearn.tree import _tree
# Initialize and train a decision tree classifier on the entire dataset
clf = DecisionTreeClassifier(random_state=42)
clf.fit(X_pca, y)
def tree_to_code(tree, feature_names):
"""
Outputs a decision tree model as a set of decision rules.Parameters:
- tree: The decision tree model.
- feature_names: List of feature names.
"""
tree_ = tree.tree_
feature_name = [
feature_names[i] if i != _tree.TREE_UNDEFINED else "undefined!"
for i in tree_.feature
]
def recurse(node, depth, parent_rule=""):
indent = " " * depth
if tree_.feature[node] != _tree.TREE_UNDEFINED:
name = feature_name[node]
threshold = tree_.threshold[node]
left_rule = f"{name} <= {threshold:.2f}"
right_rule = f"{name} > {threshold:.2f}"
# Left child
if parent_rule != "":
left_rule = parent_rule + " AND " + left_rule
recurse(tree_.children_left[node], depth + 1, left_rule)
# Right child
if parent_rule != "":
right_rule = parent_rule + " AND " + right_rule
recurse(tree_.children_right[node], depth + 1, right_rule)
else:
# Leaf
target = tree.classes_[tree_.value[node].argmax()]
rule = f"{indent}IF {parent_rule} THEN {target}"
print(rule)
recurse(0, 0)
# Example usage:
tree_to_code(clf, qia_df.columns[:-1])
IF solidity <= -3.82 THEN Banana
IF solidity > -3.82 AND convexity <= -2.65 THEN Strawberry
IF solidity > -3.82 AND convexity > -2.65 AND solidity <= -3.37 THEN Papaya
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity <= -2.21 THEN Orange
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity <= -2.73 THEN Mango
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity <= -1.95 THEN Chikoo
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity <= -1.98 THEN Peach
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity <= -1.32 THEN Guava
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio <= -3.48 THEN Lime
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity <= -1.43 THEN Pomegranate
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity <= -0.85 THEN Plum
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity > -0.85 AND aspect_ratio <= -2.25 THEN Apple
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity > -0.85 AND aspect_ratio > -2.25 AND convexity <= -0.52 THEN Watermelon
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity > -0.85 AND aspect_ratio > -2.25 AND convexity > -0.52 AND aspect_ratio <= -1.14 THEN Jackfruit
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity > -0.85 AND aspect_ratio > -2.25 AND convexity > -0.52 AND aspect_ratio > -1.14 AND solidity <= -0.68 THEN Grapes
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity > -0.85 AND aspect_ratio > -2.25 AND convexity > -0.52 AND aspect_ratio > -1.14 AND solidity > -0.68 AND solidity <= -0.19 THEN Pear
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity > -0.85 AND aspect_ratio > -2.25 AND convexity > -0.52 AND aspect_ratio > -1.14 AND solidity > -0.68 AND solidity > -0.19 AND convexity <= -0.16 THEN Cherry
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity > -0.85 AND aspect_ratio > -2.25 AND convexity > -0.52 AND aspect_ratio > -1.14 AND solidity > -0.68 AND solidity > -0.19 AND convexity > -0.16 AND aspect_ratio <= -0.30 THEN Custard apple
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity > -0.85 AND aspect_ratio > -2.25 AND convexity > -0.52 AND aspect_ratio > -1.14 AND solidity > -0.68 AND solidity > -0.19 AND convexity > -0.16 AND aspect_ratio > -0.30 AND convexity <= 2.44 THEN Pineapple
IF solidity > -3.82 AND convexity > -2.65 AND solidity > -3.37 AND convexity > -2.21 AND solidity > -2.73 AND solidity > -1.95 AND convexity > -1.98 AND solidity > -1.32 AND aspect_ratio > -3.48 AND convexity > -1.43 AND convexity > -0.85 AND aspect_ratio > -2.25 AND convexity > -0.52 AND aspect_ratio > -1.14 AND solidity > -0.68 AND solidity > -0.19 AND convexity > -0.16 AND aspect_ratio > -0.30 AND convexity > 2.44 THEN Avocado
Image segmentation and feature extraction are powerful tools to compile quantitative information at a single object level. Today, it is possible to compute tens of features that describe the shape, texture and color of any object. This extracted information could potentially be used for purposes like classification.
This story uses fruits as they are known to everyone (or at least I hope so!), but these methods can be applied in many fields and different types of images. I hope you enjoyed it and I am looking forward to hearing any feedback or criticism.
I’d like to end by acknowledging the open-source community for making all their work available to anyone for free. I apologise in advance if I have omitted giving credit to any used resource. Happy coding!