Autonomous robots need to be efficient and agile, and be able to handle a wide range of tasks and environmental conditions. This requires the ability to learn good representations of domains and tasks using a variety of sources such as demonstrations and simulations. Representation learning for robotic tasks needs to be generalizable and robust. I will describe some key ingredients to enable this: (1) robust self-supervised learning (2) uncertainty awareness (3) compositionality. We utilize NVIDIA Isaac for GPU-accelerated robot learning at scale on a variety of tasks and domains.