Training assistive agents without human data is an important step toward scalable human–AI collaboration, since existing approaches either rely on costly human demonstrations or remain limited to small handcrafted assistance domains. Achieving this is difficult because assistants must operate under asymmetric information: they do not observe the true goal directly, but must infer it from their partner’s behaviour while coordinating across diverse tasks and collaborator strategies. In this thesis, we address these challenges by introducing XLandAssist a two-agent assistance environment with asymmetric roles and training assistants through self-play. We establish recurrent PPO baselines and investigate several mechanisms for improving assistive behaviour, including learnability-based curriculum training, assistance-oriented training benchmarks, a staged benchmark curriculum, and architectures with explicit goal inference and graph-based rule encoders. We evaluate agents on held-out procedurally generated tasks, handcrafted assistancefocused benchmarks, and unseen partner policies. Our evaluation measures both overall task success and the assistant’s direct contribution using metrics such as steps saved, helpful episode frequency, and counterfactual performance drops. In addition, we analyse goal inference quality through online prediction accuracy. The results provide evidence that scalable assistive behaviour can emerge from purely artificial training environments without human demonstrations, while also showing that robust assistance depends critically on curriculum design, partner generalisation, and accurate online goal inference.