SteelAgent, which predicted steel properties, reported uncertainty, and performed inverse design with an LLM, also confirmed the limits of holdout generalization

1/ AI usage method: physics-informed machine learning with LLM orchestration
2/ Applied task: steel property prediction, uncertainty reporting, and inverse design
3/ Reported result: high performance under random splits, but performance and 90% confidence interval coverage dropped sharply under series and group holdout
4/ SteelAgent is an example in which physics-informed machine learning was used through an LLM-based interface to perform steel property prediction, uncertainty reporting, and inverse design.
5/ According to the reported results, performance was high under random splits, but prediction performance dropped significantly in evaluations that held out series or groups.
6/ In addition, 90% confidence interval coverage also worsened under holdout conditions, showing that model calibration can easily break down when the data distribution changes.
7/ From a materials perspective, this is important because random split performance alone is not sufficient when assessing extrapolation to steels with different composition and process families.
8/ However, these results were reported at the abstract level, and broader data and independent validation are needed for the actual scope of application and reproducibility. [1]