This paper examines the mechanisms that transmit isolation into poverty. In particular, we study the effect of isolation and transport infrastructure on welfare and agricultural productivity in the case of Madagascar. Madagascar is a good case study given the bad shape of its infrastructure and therefore the significant variation in isolation. Based on comprehensive household survey data combined with a census of communes, we discover a strong poverty-isolation relationship. Further we find the inverse relationship between agricultural productivity and isolation to be surprisingly strong. We isolate the following reasons why productivity might decline with isolation: (a) transportation-induced transaction costs, (b) the inverse relationship between plot-size and productivity, (c) increasing price variability and extensification onto less fertile land, and (d) insecurity.