Lightweight expanded clay aggregates (LWAs) are porous materials with low density and high strength (EN-13055-1), and they are important in sustainable construction through their lightweight nature and ability to provide thermal or acoustic insulation. The objective of this work was therefore to evaluate the preparation of LWAs using a smectite clay (M1 formulation), whose application in common ceramic production is difficult. An alternative approach was proposed for the valorization of phosphate sludge and a palygorskite-rich sediment by mixing them with expanded clay (M2 formulation) for LWA production. This could result in economically cost-effective products with significant environmental benefits. Pellets were prepared and fired at various temperatures (1100°C, 1125°C and 1150°C), and relevant properties such as bloating index, density, water absorption and compressive strength were determined. Additionally, the microstructure, mineralogical transformations and phase compositions under various sintering temperatures were investigated. Increasing the temperature from 1000°C to 1150°C significantly improved the expansion properties of LWAs, and 1150°C seemed to be the optimal firing temperature at which the best expansion properties were achieved. In addition, the incorporation of the selected waste improved the properties of the final products, leading to lower density, greater strength and greater bloating with the development of the internal pore structure as compared to the LWAs without this addition. Because of their low density (0.6 g cm–3) and sufficient compressive strength (0.86 MPa), the manufactured LWAs can be used in construction (as insulating panels or in lightweight concrete) and in green roofs.