In this paper, we propose a machine-based classification technique using the scattering parameters obtained using a wearable wideband textile antenna to diagnose breast tumors. The breast phantom is formed following the dielectric properties of the human breast tissues and characterized to ensure the resemblance with a actual tissue model for the range of frequencies from 3 to 10 GHz. A biocompatible textile antenna is fabricated and embedded on an artificial breast phantom model to capture the variation of the reflection coefficient S11 and the transmission coefficient S21 for frequencies 3–10 GHz for different locations and sizes of tumors within the phantom model. Support vector machine is used to classify the healthy tissues from the malignant tumors based on the variation of the scattering parameters owing to the variation of the dielectric characteristics of the breast phantom model. The proposed method offers 84% and 89% accuracy while using S11 and S21 parameters alone for the analysis. However, the results further improve up to 93% as a combination of S11 and S21 signals is considered.