Key notions from statistical physics, such as "phase transitions" and "critical phenomena", are providing important insights in fields ranging from computer science to probability theory to epidemiology. Underlying many of the advances is the study of phase transitions on models of networks. Starting from the classic ideas of Erdos and Renyi, recent attempts to control and manipulate the nature of the phase transition in network connectivity will be discussed. Next, the influence of self-organization on phase transitions will be presented, as well as connections between the jamming transition in models of granular materials and constraint satisfaction problems in computer science. Finally, turning to network growth, I will show that local optimization can play a fundamental role leading to the mechanism of Preferential Attachment, which previously had been assumed as a basic axiom and, furthermore, resolves a long standing controversy between Herb Simon and Benoit Mandelbrot.