Build efficient agents with knowledge distillation on Amazon SageMaker AI

This post explains how to use sequence-level knowledge distillation (SeqKD) to fine-tune a smaller shopping agent on Amazon SageMaker AI, using a frontier teacher (GLM-5.3 on Amazon Bedrock) to generate multi-step tool-use trajectories that are filtered by ShoppingBench's programmatic scorer and used for serverless supervised fine-tuning of a Qwen3.5-4B student. The distilled student reached an average Absolute Success Rate of 0.541 on 324 held-out ShoppingBench problems, versus 0.297 for the base model and 0.540 for the greedy teacher, while IFEval general instruction-following dropped roughly 6 to 8 points. The post also covers per-step trajectory encoding, LoRA training configuration, a low-cost model-evaluation pre-check, and serving the base model and student adapter as two inference components on one real-time endpoint.