Set Up Medoids for Anomaly Detection (Two Approaches)

This workflow is designed as the second part of a three-part series focused on anomaly detection for a crops dataset. Its primary goal is to configure 'medoids' (cluster centers) and associated threshold scores. These medoids serve as reference points for each crop type, and the thresholds define the boundaries for anomaly detection. The workflow implements two parallel strategies:

1. Distance Matrix Approach: This method calculates a distance matrix between points within a crop cluster and identifies a medoid based on which point is most similar to all others in that cluster. It then determines a threshold score by finding the point furthest from this medoid.
2. Multimodal Embedding Model Approach: This method leverages textual descriptions of crops to find a medoid. It embeds these descriptions using a multimodal model and then queries the vector database to find the closest matching point (medoid) for each crop. A threshold is then calculated similarly to the distance matrix approach.

The workflow orchestrates calls to Qdrant for data retrieval and updates, and uses Python code for sparse matrix calculations. It also utilizes the Voyage AI API for embedding textual descriptions.